
Skill Official Evaluation
- 8 installs
- 2 repo stars
- Updated July 29, 2026
- full-statck-skills/utility-skills
Evaluate an Agent Skill against the official agentskills.io specification and produce a spec-compliance assessment report.
About
Checks a skill's frontmatter, directory structure, progressive disclosure, description triggering and script safety against the official Agent Skills spec. A developer uses it to audit a skill and generate an official-style evaluation report.
- Checks frontmatter, structure, triggering and script safety
- Produces Pass/Needs-improvement conclusions tied to the official spec
Skill Official Evaluation by the numbers
- 8 all-time installs (skills.sh)
- Ranked #545 of 782 Skill Development skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/full-statck-skills/utility-skills --skill skill-official-evaluationAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 8 |
|---|---|
| repo stars | ★ 2 |
| Last updated | July 29, 2026 |
| Repository | full-statck-skills/utility-skills ↗ |
What it does
Evaluate an Agent Skill against the official agentskills.io specification and produce a spec-compliance assessment report.
Files
When to use this skill
ALWAYS use this skill when the user asks to:
- Review a skill for compliance with the official Agent Skills specification
- Check whether a skill's frontmatter, directory structure, or naming follows the rules
- Evaluate a skill's description triggering quality against official best practices
- Audit a skill's script safety (non-interactive,
--help, secrets, structured output) - Generate an official-style evaluation report with Pass/Needs-improvement conclusions
- Verify progressive disclosure is implemented correctly
- Scan for security issues (hardcoded secrets, suspicious instructions)
- "审查技能合规" (review skill compliance), "官方规范评估" (official spec evaluation)
- "Skill 规范检查" (skill spec check), "技能安全审计" (skill security audit)
- "检查 SKILL.md 格式" (check SKILL.md format), "检查技能结构" (check skill structure)
- "生成官方评估报告" (generate official evaluation report)
- "根据官方规范评估技能" (evaluate skill against official spec)
- "这个 Skill 符合规范吗" (does this skill comply with the spec)
Trigger phrases include:
- "帮我审查这个 Skill 是否符合规范" (help me review whether this skill complies with spec)
- "检查这个技能的 SKILL.md 格式对不对" (check if this skill's SKILL.md format is correct)
- "这个技能的 frontmatter 合规吗" (is this skill's frontmatter compliant)
- "审计一下这个技能的脚本安全性" (audit this skill's script safety)
- "按照 agentskills.io 规范评估这个技能" (evaluate this skill against agentskills.io spec)
- "review this skill for official spec compliance"
- "check if my skill follows the official specification"
- "generate an official evaluation report for this skill"
- "does this skill meet the agentskills.io requirements"
When NOT to use (near-miss boundaries):
- User wants a multi-dimensional quality score with radar charts → use
skill-trace-evaluationinstead (TRACE model covers T/R/A/C/E, while official evaluation focuses on spec compliance) - User wants to learn how to design a skill (know the rules, not evaluate a specific skill) → use
skill-awesomeinstead - User wants to organize skill documentation into an index → use
skill-awesomeinstead - User asks for general code review (not related to Agent Skills) → this skill is scoped to Agent Skills ecosystem only
IMPORTANT: Official Evaluation vs TRACE Evaluation — Two Different Evaluation Models:
This skill and skill-trace-evaluation evaluate skills using different frameworks:
- Official Evaluation (this skill): Based on the official Agent Skills specification from agentskills.io. Checks structural compliance, naming rules, frontmatter correctness, and script safety. Answers "Does this skill follow the rules?"
- TRACE Evaluation (different skill): Based on the SkillHub TRACE quality model. Scores across Trust, Reliability, Adaptability, Convention, and Effectiveness. Produces radar charts and per-dimension scores. Answers "How good is this skill?"
When both skills could apply:
- If the user says "evaluate this skill" or "review this skill" without specifying a framework, ask: "I can evaluate this skill using either the official specification (agentskills.io compliance) or the TRACE quality model (five-dimension scoring with radar charts). Which would you prefer?"
- If the user explicitly mentions "official spec", "agentskills.io", "compliance", "format check" → use this skill
- If the user explicitly mentions "TRACE", "quality score", "radar chart", "five dimensions" → use
skill-trace-evaluation
How to use this skill
CRITICAL: This skill evaluates a target skill against the official Agent Skills specification. The evaluation conclusion is explicitly based on the official specification and best practices published at agentskills.io. Do not invent requirements not present in the official sources.
To evaluate a skill:
Step 1: Identify the target skill
- Preferred input: Path to the target skill directory
- The target must contain a
SKILL.mdfile. If not found, report "SKILL.md not found" and stop. - Also inspect optional directories:
scripts/,references/,assets/. - If the user provides a
.skillor.ziparchive, only unpack when explicitly asked; otherwise evaluate from provided excerpts.
Step 2: Apply the official rubric
Use references/official-rubric.md as the evaluation checklist. The rubric has five inspection dimensions:
Dimension 1: Spec Compliance (MUST pass)
| Check | What to verify |
|---|---|
| SKILL.md exists | The skill root must contain a SKILL.md file |
| Frontmatter present | YAML frontmatter delimited by --- at the top of SKILL.md |
| `name` field | Must match the parent directory name. Lowercase letters, digits, and hyphens only. 1-64 characters. No leading/trailing hyphens, no consecutive --. |
| `description` field | Non-empty, max 1024 characters. Must describe both what the skill does AND when to use it. Should not be overly broad. |
| Optional fields format | If license, compatibility, metadata, or allowed-tools are present, verify their formatting is valid. |
| Directory structure | Optional directories must follow conventions: scripts/ for executable code, references/ for on-demand docs, assets/ for templates and resources. |
Dimension 2: Progressive Disclosure Quality (SHOULD meet)
| Check | What to verify |
|---|---|
| SKILL.md conciseness | Body stays concise and actionable. Ideally under 500 lines / 5000 tokens. |
| Details in references/ | Long explanations, reference tables, and supplementary content moved to references/. |
| Clear reference triggers | When a reference file is mentioned, the skill tells the agent WHEN to load it ("Read references/api-errors.md if the API returns a non-200 status code"). |
| No deep reference chains | References should be one level deep from SKILL.md. Avoid references that point to other references. |
Dimension 3: Description Triggering Quality (SHOULD meet)
| Check | What to verify |
|---|---|
| User-intent language | Description uses words users would naturally say, not implementation jargon. |
| Not implementation-only | Description goes beyond "Processes X files" — it tells the agent when the user needs X processed. |
| Trigger boundaries | Description contains both "should trigger" and "should not trigger" signals where applicable. |
Dimension 4: Script Readiness (CONDITIONAL — only if scripts/ exists)
Use references/script-safety-checklist.md to verify:
| Check | Requirement |
|---|---|
| Non-interactive | No TTY prompts. All inputs via flags, env vars, or stdin. |
| `--help` available | Prints usage, options, and examples. |
| Clear error messages | Errors say what failed, what was expected, and what to try next. |
| No secrets | No hardcoded tokens, keys, or passwords. |
| Safe defaults | Destructive operations require --force or --confirm. |
| Structured output (recommended) | --format json option. Data to stdout, diagnostics to stderr. |
| Idempotency (recommended) | Repeated runs do not corrupt state. |
Dimension 5: Security Hygiene (MUST pass)
| Check | What to verify |
|---|---|
| No secrets in files | Scan SKILL.md, scripts, and other text files for hardcoded tokens, API keys, passwords. |
| No suspicious instructions | The skill must not instruct the agent to download from untrusted sources, exfiltrate data, or execute obfuscated code. |
| Risky operations guidance | If the skill involves destructive operations, it must instruct the agent to get explicit user confirmation. |
Step 3: Run the evaluator script (recommended)
The bundled script automates data collection and formatting:
python3 scripts/official_evaluate.py --help
# Generate a Markdown evaluation report
python3 scripts/official_evaluate.py --skill-dir <path> --format md
# Generate machine-readable JSON
python3 scripts/official_evaluate.py --skill-dir <path> --format json
# Write to a file
python3 scripts/official_evaluate.py --skill-dir <path> --format md --output report.mdThe script performs automated checks for:
- SKILL.md presence and frontmatter parsing
- Name format validation (regex) and directory match
- Description length validation
- License field check
- Secret pattern scanning (AWS keys, API keys, token patterns)
- Non-interactive pattern detection in scripts
After running the script, you MUST supplement the automated results with qualitative assessment for:
- Progressive disclosure quality (is the body concise? are references well-triggered?)
- Description triggering quality (does it use user-intent language? are trigger boundaries clear?)
- Security hygiene beyond regex patterns (are there suspicious instructions?)
Step 4: Produce the evaluation report
The report MUST include these sections, in order:
Report structure
# Official Skill Evaluation Report
Target: `<path-to-skill-directory>`
## Conclusion
- Overall conclusion: **Pass** / **Needs improvement** / **Fail**
- Top issues:
1. ...
2. ...
3. ...
## Compliance Checklist
| Item | Result | Evidence | Suggestion |
|------|--------|----------|------------|
| SKILL.md frontmatter present | Pass/Fail | ... | ... |
| name matches directory | Pass/Fail | ... | ... |
| name format valid | Pass/Fail | ... | ... |
| description present & valid | Pass/Fail | ... | ... |
| license field | Pass/Needs improvement | ... | ... |
| Optional directories organized | Pass | ... | ... |
| Progressive disclosure | Pass/Needs improvement | ... | ... |
| Description trigger quality | Pass/Needs improvement | ... | ... |
| Script safety (if applicable) | Pass/Fail/N/A | ... | ... |
| Security & secrets scan | Pass/Fail | ... | ... |
## Risks & Limitations
- ...
## Improvement Suggestions (prioritized)
1. ...
2. ...
3. ...Conclusion levels
| Level | Criteria |
|---|---|
| Pass | All MUST items pass. SHOULD items are reasonably met. No security findings. |
| Needs improvement | All MUST items pass, but SHOULD items have significant gaps. No security findings. |
| Fail | One or more MUST items fail, OR security findings detected. |
Evidence rules
- Every Pass/Fail MUST include specific evidence: a file path, a field value, a line number, or a scan result.
- Do NOT use subjective language like "seems good" or "looks fine". Cite artifacts.
- If a check is N/A (e.g., no
scripts/directory), state "N/A — no scripts/ directory" as evidence.
Output format
After producing the evaluation report:
1. State the overall conclusion clearly: "Pass", "Needs improvement", or "Fail" 2. List the top 3 most important findings 3. Show the compliance checklist table with evidence 4. Provide prioritized, actionable improvement suggestions 5. Save the report if the user requests a file; otherwise display inline
Rules
1. Do not invent "official requirements" not present in the official sources above. Every finding must be traceable to the official specification or best practices. 2. Do not include secrets or reproduce sensitive content in the report. If secrets are found, note their location without reproducing the secret value. 3. Treat the rubric as the ground truth. If references/official-rubric.md says a check is "Should", do not report it as a hard failure. 4. The evaluation conclusion is explicitly based on the official specification. The report should state this clearly in the opening paragraph.
Keywords
English keywords: official-evaluation, spec-compliance, skill-review, skill-audit, frontmatter-check, naming-validation, description-quality, script-safety, security-scan, progressive-disclosure, official-rubric, agentskills-spec, skill-assessment, compliance-report, skill-inspection, format-check, structure-review
Chinese keywords (中文关键词): 审查技能合规, 官方规范评估, Skill 规范检查, 技能安全审计, 检查 SKILL.md 格式, 检查技能结构, 生成官方评估报告, 根据官方规范评估技能, 技能合规检查, 技能评估报告, frontmatter 检查, 技能命名检查, 技能描述检查, 脚本安全检查, 渐进式披露检查, 官方规范审查
能力边界
✅ 适用场景
- 当你需要使用此技能对应的技术栈时
- 当项目需要遵循最佳实践时
- 当需要快速上手或深入理解核心概念时
⚠️ 需要注意
- 复杂业务逻辑需要结合具体场景调整
- 性能优化需要根据实际数据量评估
❌ 不适用场景
- 不相关的技术栈或框架
- 需要完全自定义的特殊场景
常见陷阱 (Gotchas)
1. 版本兼容性:注意框架版本与依赖库的兼容性,不同版本 API 可能有差异 2. 配置文件格式:配置文件格式错误是最常见的问题,建议使用编辑器的语法检查 3. 环境变量:确保所有必要的环境变量已正确设置,敏感信息不要硬编码 4. 依赖冲突:多版本共存时注意依赖冲突,使用 lock 文件锁定版本 5. 性能陷阱:大数据量场景下注意性能优化,避免 N+1 查询等常见问题
Official Skill Evaluation Report
Target: /Users/wandl/workspaces/workspace-partme-ai/full-stack-skills/skills/document-skills/mermaid
Conclusion
- Overall: 需改进
- Top issues:
1. Description length
Checklist
| Item | Result | Evidence | Suggestion |
|---|---|---|---|
| SKILL.md present | Pass | SKILL.md | - |
| Name matches directory | Pass | name=mermaid | - |
| Description length | Fail | len=1106 | Reduce to 1-1024 characters |
| License basic check | Pass | Complete terms in LICENSE.txt | - |
| Optional directories | Pass | references=False, scripts=False, assets=False | - |
| Secrets scan | Pass | No high-confidence secret patterns detected | - |
| Non-interactive scripts | Pass | No obvious interactive patterns detected | - |
Risks & Limits
- Static evaluation only. It does not validate runtime behavior (e.g., network calls, sandbox behavior).
- For stronger trust conclusions, provide third-party scan reports or runtime logs.
Recommendations
- Description length: Reduce to 1-1024 characters
Skill 官方评估报告(Official Evaluating)
目标:skills/document-skills/mermaid
结论
- 总体结论:通过(有改进空间)
- 主要问题(Top 3):
1. SKILL.md 正文偏长,部分内容可迁移到 references/(渐进式披露更清晰) 2. 触发条件写得很“强”,可能产生误触发,建议配一组 should-not-trigger 例子(描述优化) 3. 若未来加入可执行脚本,建议补充 --help/结构化输出/非交互约束说明(脚本规范)
规范对照清单
| 项目 | 结果 | 证据 | 建议 |
|---|---|---|---|
| SKILL.md frontmatter 合规 | Pass | 含 name/description/license | 可补 compatibility(若有环境要求) |
| name/目录名一致 | Pass | name=mermaid,目录=mermaid | - |
| description 触发覆盖 | Pass | 覆盖“画图/可视化/mermaid”等意图 | 增加 near-miss 的不触发边界 |
| 渐进式披露(references/assets) | Needs improvement | SKILL.md 正文包含大量细节 | 迁移长清单到 references/,正文保留流程 |
| scripts 可执行性与非交互 | N/A | 无 scripts/ | - |
| 安全与敏感信息 | Pass | 未发现敏感信息 | - |
改进建议(按优先级)
1. 抽取“关键词大清单”“版本兼容细节”到 references/,正文只保留“选择图类型→参考示例→输出”工作流 2. 为 description 增加 should-trigger / should-not-trigger 测试集并迭代优化 3. 为未来可能的脚本型能力预留脚本接口规范(非交互、结构化输出)
Apache-2.0
Official evaluating rubric
This file defines the “official rubric” used by skill-official-evaluation.
1) Spec compliance (must)
1.1 SKILL.md frontmatter
- name:
- matches directory name
- lowercase letters, numbers, hyphens only
- 1–64 chars; no leading/trailing hyphen; no consecutive
-- - description:
- non-empty
- describes what + when to use
- not overly broad; includes clear triggers
- optional fields (if present):
- license / compatibility / metadata / allowed-tools formatting is valid
1.2 Directory structure
- skill root contains
SKILL.md - optional directories follow conventions:
scripts/for executable automationreferences/for on-demand docsassets/for templates/resources
2) Progressive disclosure quality (should)
- SKILL.md body stays concise and actionable
- long explanations moved to
references/ - references are linked with clear “when to read” triggers
- avoids deep chains of references
3) Trigger description quality (should)
- uses user-intent language (what users say)
- avoids implementation-only keywords
- contains both “should trigger” and “should not trigger” boundaries where needed
4) Script readiness (conditional)
If scripts/ exists, apply script-safety-checklist.md and verify:
- non-interactive CLI
--helpavailable- clear error messages
- structured output option (json)
- safe defaults and idempotency considerations
5) Security hygiene (must)
- no secrets in skill files (keys/tokens/passwords)
- no suspicious downloads/exfiltration instructions
- risky operations require explicit user confirmation guidance
Official sources (curated)
Use these sources as the “official baseline” when producing an official evaluation report.
Agent Skills spec (agentskills.io)
- Specification: https://agentskills.io/specification
- Best practices: https://agentskills.io/skill-creation/best-practices
- Optimizing descriptions: https://agentskills.io/skill-creation/optimizing-descriptions
- Using scripts: https://agentskills.io/skill-creation/using-scripts
- Docs index: https://agentskills.io/llms.txt
Claude custom skills guidance
- How to create custom Skills: https://support.claude.com/en/articles/12512198-how-to-create-custom-skills
Script safety checklist
Use this checklist when the target skill contains executable scripts or instructs the agent to run shell commands.
Required
- Non-interactive
- No prompts waiting for stdin/TTY
- All inputs via flags/env/stdin explicitly
- Clear help
--helpprints usage + examples- Clear errors
- errors say what failed + what to try next
- No secrets
- no hardcoded tokens/keys/passwords
- no logging of secrets
- Safe defaults
- destructive ops require explicit
--force/--confirm - prefer dry-run flags where applicable
Recommended
- Structured output
--format json(or--output <file>)- diagnostics to stderr, data to stdout
- Idempotency
- repeated runs do not corrupt state
- Predictable output size
- defaults to summary; supports pagination flags if needed
#!/usr/bin/env python3
"""
Evaluate an Agent Skill against “official” Agent Skills conventions and output a report.
This script is intentionally dependency-free (stdlib only) to keep it portable.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import sys
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
FRONTMATTER_BOUNDARY = re.compile(r"^---\s*$")
NAME_RE = re.compile(r"^[a-z0-9]+(?:-[a-z0-9]+)*$")
SECRET_RE = re.compile(
r"(AKIA[0-9A-Z]{16}|sk-[A-Za-z0-9]{16,}|(api[_-]?key|secret|token|password)\s*[:=]\s*['\"][^'\"]+['\"])",
re.IGNORECASE,
)
@dataclass
class CheckResult:
item: str
result: str
evidence: str
suggestion: str
def parse_args() -> argparse.Namespace:
"""Parse CLI args."""
p = argparse.ArgumentParser(
prog="official_evaluate.py",
description="Evaluate a skill directory and output an official-style report.",
)
p.add_argument("--skill-dir", required=True, help="Path to the target skill directory.")
p.add_argument("--format", choices=["md", "json"], default="md", help="Output format.")
p.add_argument("--output", default="", help="Write output to a file instead of stdout.")
return p.parse_args()
def read_text(path: Path) -> str:
"""Read UTF-8 text from file."""
return path.read_text(encoding="utf-8")
def parse_frontmatter(skill_md: str) -> Tuple[Dict[str, Any], str]:
"""Parse YAML-like frontmatter from SKILL.md without external YAML deps."""
lines = skill_md.splitlines()
if not lines or not FRONTMATTER_BOUNDARY.match(lines[0]):
return {}, skill_md
i = 1
fm_lines: List[str] = []
while i < len(lines) and not FRONTMATTER_BOUNDARY.match(lines[i]):
fm_lines.append(lines[i])
i += 1
if i >= len(lines):
return {}, skill_md
body = "\n".join(lines[i + 1 :]).lstrip("\n")
fm: Dict[str, Any] = {}
for raw in fm_lines:
if not raw.strip() or raw.strip().startswith("#"):
continue
if ":" not in raw:
continue
k, v = raw.split(":", 1)
fm[k.strip()] = v.strip()
return fm, body
def short(s: str, limit: int = 120) -> str:
"""Shorten long strings for evidence rendering."""
s = " ".join(s.split())
if len(s) <= limit:
return s
return s[: limit - 3] + "..."
def check_name(dir_name: str, fm: Dict[str, Any]) -> CheckResult:
"""Check name format and directory match."""
name = (fm.get("name") or "").strip()
if not name:
return CheckResult("Frontmatter name present", "Fail", "Missing frontmatter name", "Add name and match directory name")
if name != dir_name:
return CheckResult("Name matches directory", "Fail", f"name={name}, dir={dir_name}", "Make name match directory name")
if not NAME_RE.match(name) or len(name) > 64:
return CheckResult("Name format valid", "Fail", f"name={name}", "Use lowercase letters/numbers/hyphens; length 1-64")
if "--" in name or name.startswith("-") or name.endswith("-"):
return CheckResult("Hyphen rules", "Fail", f"name={name}", "Avoid consecutive hyphens; do not start/end with hyphen")
return CheckResult("Name matches directory", "Pass", f"name={name}", "-")
def check_description(fm: Dict[str, Any]) -> CheckResult:
"""Check description presence and basic quality."""
desc = (fm.get("description") or "").strip()
if not desc:
return CheckResult("Frontmatter description present", "Fail", "Missing frontmatter description", "Add description (what + when)")
if len(desc) > 1024:
return CheckResult("Description length", "Fail", f"len={len(desc)}", "Reduce to 1-1024 characters")
if len(desc) < 20:
return CheckResult("Description informativeness", "Needs improvement", short(desc), "Add triggers and user-intent phrasing")
return CheckResult("Description valid", "Pass", short(desc), "-")
def check_license(skill_dir: Path, fm: Dict[str, Any]) -> CheckResult:
"""Check license field and file existence when referenced."""
lic = (fm.get("license") or "").strip()
if not lic:
return CheckResult("License field", "Needs improvement", "No frontmatter license", "Add license field or reference LICENSE.txt")
if "LICENSE" in lic and not (skill_dir / "LICENSE.txt").exists():
return CheckResult("LICENSE.txt present", "Fail", f"license={lic}", "Add LICENSE.txt or fix the license reference")
return CheckResult("License basic check", "Pass", short(lic), "-")
def scan_for_secrets(skill_dir: Path) -> List[str]:
"""Scan common text files for likely secrets."""
findings: List[str] = []
for p in skill_dir.rglob("*"):
if p.is_dir():
continue
if p.suffix.lower() not in {".md", ".txt", ".json", ".yaml", ".yml", ".py", ".sh"}:
continue
try:
text = read_text(p)
except Exception:
continue
m = SECRET_RE.search(text)
if m:
findings.append(f"{p.relative_to(skill_dir)}: {short(m.group(0), 80)}")
return findings
def detect_noninteractive_issues(skill_dir: Path) -> List[str]:
"""Heuristically detect interactive patterns in scripts."""
issues: List[str] = []
scripts_dir = skill_dir / "scripts"
if not scripts_dir.exists():
return issues
for p in scripts_dir.rglob("*"):
if p.is_dir():
continue
if p.suffix.lower() not in {".py", ".sh"}:
continue
try:
text = read_text(p)
except Exception:
continue
if "input(" in text or "read -p" in text or "select " in text:
issues.append(f"{p.relative_to(skill_dir)}: may require interactive input")
return issues
def evaluate(skill_dir: Path) -> Dict[str, Any]:
"""Run checks and return a structured report object."""
skill_md_path = skill_dir / "SKILL.md"
if not skill_md_path.exists():
return {
"error": "SKILL.md not found",
"skill_dir": str(skill_dir),
}
raw = read_text(skill_md_path)
fm, _body = parse_frontmatter(raw)
dir_name = skill_dir.name
checks: List[CheckResult] = []
checks.append(CheckResult("SKILL.md present", "Pass", "SKILL.md", "-"))
checks.append(check_name(dir_name, fm))
checks.append(check_description(fm))
checks.append(check_license(skill_dir, fm))
refs = (skill_dir / "references").exists()
scripts = (skill_dir / "scripts").exists()
assets = (skill_dir / "assets").exists()
checks.append(
CheckResult(
"Optional directories",
"Pass",
f"references={refs}, scripts={scripts}, assets={assets}",
"-",
)
)
secret_findings = scan_for_secrets(skill_dir)
if secret_findings:
checks.append(
CheckResult(
"Secrets scan",
"Fail",
"; ".join(secret_findings[:3]) + (" ..." if len(secret_findings) > 3 else ""),
"Remove secrets; use environment variables or secure storage",
)
)
else:
checks.append(CheckResult("Secrets scan", "Pass", "No high-confidence secret patterns detected", "-"))
interactive_issues = detect_noninteractive_issues(skill_dir)
if interactive_issues:
checks.append(
CheckResult(
"Non-interactive scripts",
"Needs improvement",
"; ".join(interactive_issues[:3]) + (" ..." if len(interactive_issues) > 3 else ""),
"Avoid interactive input; use CLI flags instead of stdin/TTY prompts",
)
)
else:
checks.append(CheckResult("Non-interactive scripts", "Pass", "No obvious interactive patterns detected", "-"))
overall = "通过"
top: List[str] = []
for c in checks:
if c.result == "Fail":
overall = "需改进"
top.append(c.item)
elif c.result == "Needs improvement" and overall != "需改进":
overall = "通过(有改进空间)"
top.append(c.item)
return {
"skill_dir": str(skill_dir),
"skill_name": fm.get("name", ""),
"generated_at": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
"overall": overall,
"top_issues": top[:3],
"checks": [c.__dict__ for c in checks],
}
def to_markdown(report: Dict[str, Any]) -> str:
"""Render report as Markdown."""
if "error" in report:
return (
"# Official Skill Evaluation Report\n\n"
f"- Error: {report['error']}\n"
f"- Path: {report.get('skill_dir','')}\n"
)
lines: List[str] = []
lines.append("# Official Skill Evaluation Report")
lines.append("")
lines.append(f"Target: `{report.get('skill_dir','')}`")
lines.append("")
lines.append("## Conclusion")
lines.append(f"- Overall: {report.get('overall','')}")
top = report.get("top_issues") or []
if top:
lines.append("- Top issues:")
for i, t in enumerate(top, 1):
lines.append(f" {i}. {t}")
else:
lines.append("- Top issues: none")
lines.append("")
lines.append("## Checklist")
lines.append("| Item | Result | Evidence | Suggestion |")
lines.append("| --- | --- | --- | --- |")
for c in report.get("checks", []):
lines.append(f"| {c['item']} | {c['result']} | {c['evidence']} | {c['suggestion']} |")
lines.append("")
lines.append("## Risks & Limits")
lines.append("- Static evaluation only. It does not validate runtime behavior (e.g., network calls, sandbox behavior).")
lines.append("- For stronger trust conclusions, provide third-party scan reports or runtime logs.")
lines.append("")
lines.append("## Recommendations")
recs: List[str] = []
for c in report.get("checks", []):
if c["result"] in {"Fail", "Needs improvement"} and c["suggestion"] != "-":
recs.append(f"- {c['item']}: {c['suggestion']}")
if not recs:
lines.append("- None")
else:
lines.extend(recs[:10])
lines.append("")
return "\n".join(lines)
def main() -> int:
"""CLI entrypoint."""
args = parse_args()
skill_dir = Path(args.skill_dir).expanduser().resolve()
report = evaluate(skill_dir)
if args.format == "json":
out = json.dumps(report, ensure_ascii=False, indent=2)
else:
out = to_markdown(report)
if args.output:
out_path = Path(args.output).expanduser().resolve()
out_path.parent.mkdir(parents=True, exist_ok=True)
out_path.write_text(out, encoding="utf-8")
return 0
sys.stdout.write(out)
return 0
if __name__ == "__main__":
raise SystemExit(main())