Skill Standardization

{
  "skill_name": "skill-standardization",
  "evals": [
    {
      "id": 1,
      "prompt": "Check if my SKILL.md file follows the Agent Skills specification format. The file is at .agent-skills/my-skill/SKILL.md",
      "expected_output": "A validation report listing any issues with frontmatter fields, name format, description length, imperative trigger phrasing, and recommended sections.",
      "assertions": [
        "The output checks for the required 'name' field",
        "The output checks for the required 'description' field",
        "The output verifies the name follows lowercase hyphenated format",
        "The output checks description length is under 1024 characters",
        "The output checks for recommended sections such as When to use this skill, Instructions, and Examples"
      ]
    },
    {
      "id": 2,
      "prompt": "Create a new SKILL.md for a git-workflow skill that handles committing, branching, and PR creation",
      "expected_output": "A properly formatted SKILL.md with correct frontmatter, a trigger-oriented description, and the standard sections.",
      "assertions": [
        "The output includes YAML frontmatter between --- delimiters",
        "The frontmatter contains a 'name' field set to 'git-workflow'",
        "The frontmatter contains a 'description' field",
        "The description uses imperative phrasing containing 'Use when' or similar",
        "The description is under 1024 characters",
        "The output includes ## When to use this skill section",
        "The output includes ## Instructions section",
        "The output includes ## Examples section"
      ]
    },
    {
      "id": 3,
      "prompt": "My skill description is 'Helps with CSV files.' — improve it so the skill triggers more reliably",
      "expected_output": "An improved description with imperative phrasing, specific operations, trigger keywords, and route-aware wording under 1024 characters.",
      "assertions": [
        "The improved description contains 'Use when' or similar imperative phrasing",
        "The improved description lists specific operations instead of generic help wording",
        "The improved description is under 1024 characters",
        "The improved description includes trigger keywords or synonyms"
      ]
    },
    {
      "id": 4,
      "prompt": "Validate all the skills in the .agent-skills directory and give me a summary of which ones have issues",
      "expected_output": "A per-skill validation report showing errors and warnings for each skill, with a total summary.",
      "assertions": [
        "The output covers multiple skills",
        "The output distinguishes between errors and warnings",
        "The output includes a total or summary count"
      ]
    },
    {
      "id": 5,
      "prompt": "We renamed one skill, removed another, and added a new game skill. Check whether skills.json and the README/setup inventory are still in sync with the .agent-skills folders.",
      "expected_output": "A response that validates catalog/discovery surfaces, compares live folders against skills.json and README/setup counts, and identifies stale or missing entries.",
      "assertions": [
        "The output checks live skill folders against skills.json",
        "The output checks README or setup inventory counts after structural changes",
        "The output flags stale manifest-only names or missing live skills",
        "The output treats catalog sync as part of the standardization workflow"
      ]
    },
    {
      "id": 6,
      "prompt": "I rewrote a skill's SKILL.md description and route-outs, but I also ship SKILL.toon for runtime discovery. What should I validate before I call the standardization pass done?",
      "expected_output": "A response that treats SKILL.toon / SKILL.compact.md as derived discovery surfaces, checks whether compact variants need refresh after material trigger-surface changes, and keeps catalog/runtime sync in scope.",
      "assertions": [
        "The output mentions refreshing or reviewing SKILL.toon or SKILL.compact.md after material rewrites",
        "The output distinguishes compact-variant sync from only checking skills.json and README/setup counts",
        "The output keeps runtime discovery or query/list tooling in scope",
        "The output frames compact files as derived artifacts, not independent source of truth"
      ]
    },
    {
      "id": 7,
      "prompt": "I'm at the repo root of jeo-skills. What exact command should I run to validate .agent-skills/skill-standardization, and what still needs manual review after the scripts pass?",
      "expected_output": "A response that gives the truthful repo-root validator command, distinguishes repo-root from skill-local execution, and lists manual review items such as compact-surface freshness or stale discovery copy.",
      "assertions": [
        "The output uses the nested repo-root path .agent-skills/skill-standardization/scripts/validate_skill.sh instead of bare scripts/validate_skill.sh",
        "The output distinguishes repo-root execution from skill-local execution or notes the working-directory dependency",
        "The output names at least one manual review item beyond the scripts, such as SKILL.toon freshness, manifest command residue, or README/setup discovery wording"
      ]
    },
    {
      "id": 8,
      "prompt": "A valid SKILL.md uses `description: >` and says `ClawTeam's operator guide` in the folded description. Validate it without telling me to rewrite the prose just to appease shell parsing.",
      "expected_output": "A response that treats folded descriptions with apostrophes as valid input, keeps the validator bug/repair path in scope if parsing fails, and avoids author-side wording downgrades as the default fix.",
      "assertions": [
        "The output accepts ordinary apostrophes inside valid folded descriptions as something the validator should handle",
        "The output frames shell-quote failures as validator bugs or regression cases to repair, not as a reason to rewrite natural language",
        "The output keeps repo-root validator usage in scope when explaining the validation path"
      ]
    },
    {
      "id": 9,
      "prompt": "I tightened the canonical skill and refreshed skills.json plus README/setup, but the compatibility alias still has a checklist and reference packet. What else must I review before calling the standardization pass done?",
      "expected_output": "A response that treats alias-side support docs/checklists/examples as derived surfaces whenever they mirror the canonical contract, not just top-level discovery files.",
      "assertions": [
        "The output explicitly tells the maintainer to review alias-side support docs, checklists, or examples when a compatibility alias mirrors the canonical packet",
        "The output distinguishes alias-side support sync from only refreshing skills.json, README/setup, or SKILL.toon",
        "The output frames stale alias support docs as a real residue risk even if catalog validators pass"
      ]
    }
  ]
}

#!/usr/bin/env bash
set -euo pipefail

repo_root="${1:-/Users/jang_jennie/projects/jeo-skills}"
validator="$repo_root/.agent-skills/skill-standardization/scripts/validate_skill.sh"

tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT
mkdir -p "$tmpdir/problem-skill"

cat > "$tmpdir/problem-skill/SKILL.md" <<'EOF'
---
name: problem-skill
description: >
  Validate a skill when the ClawTeam's operator guide includes shell-like text and route-outs.
allowed-tools: Bash Read Write
---

# Problem Skill

## When to use this skill
- Demo.

## Instructions
- Demo.

## Examples
- Demo.

## Best practices
1. Demo.

## References
- Demo.
EOF

bash "$validator" "$tmpdir/problem-skill"

#!/usr/bin/env python3
"""Validate that shipped skill catalog surfaces stay in sync with the live filesystem."""
from __future__ import annotations

import argparse
import json
import re
import sys
from pathlib import Path


COUNT_PATTERNS = {
    'README.md': [
        (r'(\d+) local skill folders', 'local skill folder count'),
        (r'← (\d+) skill folders', 'tree snippet skill folder count'),
    ],
    'README.ko.md': [
        (r'(\d+)개 로컬 스킬 폴더', 'local skill folder count'),
        (r'← (\d+) skill folders', 'tree snippet skill folder count'),
    ],
    'setup-all-skills-prompt.md': [
        (r'Install ALL (\d+) skills', 'install-all count'),
        (r'Skill Inventory \((\d+) skills\)', 'inventory count'),
    ],
}


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description='Validate skill catalog / docs sync against live skill folders.')
    parser.add_argument('--repo-root', default=Path(__file__).resolve().parents[3], type=Path)
    parser.add_argument('--json', action='store_true', help='Emit machine-readable JSON')
    return parser.parse_args()


def live_skills(skills_root: Path) -> list[str]:
    return sorted(p.parent.name for p in skills_root.glob('*/SKILL.md'))


def load_manifest(path: Path) -> dict:
    if not path.exists():
        return {}
    return json.loads(path.read_text(encoding='utf-8'))


def manifest_names(data: dict) -> list[str]:
    return sorted(entry.get('name') for entry in data.get('skills', []) if isinstance(entry, dict) and entry.get('name'))


def check_counts(repo_root: Path, live_count: int) -> list[dict]:
    findings = []
    for rel, patterns in COUNT_PATTERNS.items():
        path = repo_root / rel
        if not path.exists():
            findings.append({'level': 'warn', 'surface': rel, 'message': 'file missing'})
            continue
        text = path.read_text(encoding='utf-8')
        for pattern, label in patterns:
            for match in re.finditer(pattern, text):
                value = int(match.group(1))
                if value != live_count:
                    findings.append({
                        'level': 'error',
                        'surface': rel,
                        'message': f'{label} = {value}, expected {live_count}',
                    })
    return findings


def main() -> int:
    args = parse_args()
    repo_root = args.repo_root.expanduser().resolve()
    skills_root = repo_root / '.agent-skills'
    manifest_path = skills_root / 'skills.json'

    live = live_skills(skills_root)
    live_set = set(live)
    data = load_manifest(manifest_path)
    manifest = manifest_names(data)
    manifest_set = set(manifest)

    findings = []
    missing = sorted(live_set - manifest_set)
    extras = sorted(manifest_set - live_set)
    if missing:
        findings.append({'level': 'error', 'surface': 'skills.json', 'message': f'missing live skills: {missing}'})
    if extras:
        findings.append({'level': 'error', 'surface': 'skills.json', 'message': f'stale manifest-only skills: {extras}'})

    declared_count = data.get('skill_count')
    if declared_count is not None and declared_count != len(live):
        findings.append({'level': 'error', 'surface': 'skills.json', 'message': f'skill_count={declared_count}, expected {len(live)}'})

    for entry in data.get('skills', []):
        if not isinstance(entry, dict) or not entry.get('name'):
            continue
        name = entry['name']
        path_value = entry.get('path')
        if path_value:
            target = skills_root / path_value
            if name in live_set and not target.exists():
                findings.append({'level': 'warn', 'surface': 'skills.json', 'message': f'path mismatch for {name}: {path_value}'})

    findings.extend(check_counts(repo_root, len(live)))

    summary = {
        'live_skill_count': len(live),
        'manifest_skill_count': len(manifest),
        'missing_from_manifest': missing,
        'manifest_only': extras,
        'findings': findings,
    }

    if args.json:
        print(json.dumps(summary, indent=2, ensure_ascii=False))
    else:
        print(f'Live skills: {len(live)}')
        print(f'Manifest skills: {len(manifest)}')
        if missing:
            print(f'Missing from manifest: {", ".join(missing)}')
        if extras:
            print(f'Manifest-only entries: {", ".join(extras)}')
        if findings:
            print('\nFindings:')
            for item in findings:
                print(f"- [{item['level']}] {item['surface']}: {item['message']}")
        else:
            print('\nNo drift detected.')

    return 1 if any(item['level'] == 'error' for item in findings) else 0


if __name__ == '__main__':
    raise SystemExit(main())

#!/usr/bin/env bash
# validate_skill.sh — Validates a SKILL.md against the Agent Skills specification
# Usage:
#   validate_skill.sh <skill-directory>       # validate one skill
#   validate_skill.sh --all <skills-dir>      # validate all skills in directory
#   validate_skill.sh --help                  # show usage

set -euo pipefail

ERRORS=0
WARNINGS=0
TOTAL_ERRORS=0
TOTAL_WARNINGS=0

usage() {
  cat <<EOF
Usage: validate_skill.sh [--all] <path>

  validate_skill.sh my-skill/            Validate a single skill directory
  validate_skill.sh --all .agent-skills/ Validate all skill directories

Checks performed:
  - Required frontmatter fields: name, description
  - name format: lowercase, no consecutive hyphens, matches directory name
  - description length: 1-1024 characters
  - description phrasing: warns if missing imperative trigger language
  - allowed-tools format: space-delimited (not YAML list)
  - Recommended sections: When to use, Instructions, Examples, Best practices, References
  - File length: warns if over 500 lines

Exit codes:
  0 = no errors (warnings may exist)
  1 = one or more errors found
EOF
}

check() {
  local status="$1" msg="$2"
  if [[ "$status" == "ok" ]]; then
    echo "  ✓ $msg"
  elif [[ "$status" == "warn" ]]; then
    echo "  ⚠ $msg"
    ((WARNINGS++)) || true
  else
    echo "  ✗ $msg"
    ((ERRORS++)) || true
  fi
}

extract_fm_field() {
  local content="$1" field="$2"
  # Handle both single-line and block scalar (>) values
  # Use || true to prevent set -e from triggering when grep finds no match
  echo "$content" | grep -E "^${field}:" | head -1 | sed "s/^${field}: *//" | tr -d '"' | sed 's/^>//' || true
}

validate_skill() {
  local skill_dir="$1"
  local skill_md="${skill_dir%/}/SKILL.md"
  ERRORS=0
  WARNINGS=0

  if [[ ! -f "$skill_md" ]]; then
    echo "✗ No SKILL.md found in $skill_dir"
    TOTAL_ERRORS=$((TOTAL_ERRORS + 1))
    return
  fi

  echo "Validating: $skill_md"

  # Extract frontmatter block
  local in_fm=0 fm_content="" body_started=0 line_count=0
  while IFS= read -r line; do
    ((line_count++)) || true
    if [[ "$line" == "---" && $in_fm -eq 0 && $body_started -eq 0 ]]; then
      in_fm=1
    elif [[ "$line" == "---" && $in_fm -eq 1 ]]; then
      in_fm=0
      body_started=1
    elif [[ $in_fm -eq 1 ]]; then
      fm_content+="$line"$'\n'
    fi
  done < "$skill_md"

  if [[ -z "$fm_content" ]]; then
    check "error" "No YAML frontmatter found (expected --- ... --- block)"
    echo ""
    echo "  Issues: ${ERRORS} errors, ${WARNINGS} warnings"
    TOTAL_ERRORS=$((TOTAL_ERRORS + ERRORS))
    TOTAL_WARNINGS=$((TOTAL_WARNINGS + WARNINGS))
    return
  fi

  # --- Check: name field ---
  local name
  name=$(extract_fm_field "$fm_content" "name") || true
  if [[ -n "$name" ]]; then
    check "ok" "Required field: name = '$name'"
    # Format check: lowercase, alphanumeric + hyphens
    if echo "$name" | grep -qE '^[a-z0-9]([a-z0-9-]*[a-z0-9])?$'; then
      if echo "$name" | grep -q -- '--'; then
        check "error" "Name contains consecutive hyphens: '$name'"
      else
        check "ok" "Name format: valid (lowercase alphanumeric + hyphens)"
      fi
    else
      check "error" "Name format: invalid characters or leading/trailing hyphen: '$name'"
    fi
    # Length check
    if [[ ${#name} -gt 64 ]]; then
      check "error" "Name length: ${#name} chars (max 64)"
    fi
    # Directory name match
    local dir_name
    dir_name=$(basename "${skill_dir%/}")
    if [[ "$name" == "$dir_name" ]]; then
      check "ok" "Name matches directory: '$name'"
    else
      check "warn" "Name/directory mismatch: name='$name' vs dir='$dir_name'"
    fi
  else
    check "error" "Required field 'name' is missing"
  fi

  # --- Check: description field ---
  local desc
  desc=$(extract_fm_field "$fm_content" "description") || true
  # For block scalars, also collect continuation lines.
  # Important: do not use xargs for trimming here. xargs performs shell-like
  # quote parsing and can abort on ordinary apostrophes in valid prose such as
  # "ClawTeam's" with `xargs: unterminated quote`.
  if [[ -z "$desc" ]] || [[ "$desc" == ">" ]] || [[ "$desc" == "|" ]]; then
    desc=$(python3 - "$skill_md" <<'PY'
import re
import sys
from pathlib import Path

text = Path(sys.argv[1]).read_text(encoding='utf-8')
match = re.search(r'^description:\s*[>|]\s*\n((?:  .*\n?)*)', text, re.M)
if not match:
    print('')
    raise SystemExit(0)

parts = []
for raw_line in match.group(1).splitlines():
    line = raw_line[2:] if raw_line.startswith('  ') else raw_line.lstrip()
    stripped = line.strip()
    if stripped:
        parts.append(stripped)

print(' '.join(parts))
PY
)
  fi

  if [[ -n "$desc" ]]; then
    check "ok" "Required field: description present"
    local desc_len=${#desc}
    if [[ $desc_len -gt 1024 ]]; then
      check "error" "Description length: ${desc_len} chars (max 1024)"
    elif [[ $desc_len -lt 20 ]]; then
      check "warn" "Description seems too short (${desc_len} chars): '$desc'"
    else
      check "ok" "Description length: ${desc_len} chars (OK)"
    fi
    # Phrasing check
    if echo "$desc" | grep -qiE "(use when|use this skill when|triggers on|use for)"; then
      check "ok" "Description has imperative trigger phrasing"
    else
      check "warn" "Description may lack imperative phrasing — consider adding 'Use when...' or 'Triggers on:'"
    fi
  else
    check "error" "Required field 'description' is missing or empty"
  fi

  # --- Check: allowed-tools format ---
  local tools_line
  tools_line=$(echo "$fm_content" | grep -E "^allowed-tools:" | head -1 || true)
  if [[ -n "$tools_line" ]]; then
    local tools_val
    tools_val=$(echo "$tools_line" | sed 's/^allowed-tools: *//')
    if echo "$tools_val" | grep -qE '^\['; then
      check "warn" "allowed-tools uses YAML list syntax — spec requires space-delimited string"
    else
      check "ok" "allowed-tools format: space-delimited (OK)"
    fi
  fi

  # --- Check: compatibility field length ---
  local compat
  compat=$(extract_fm_field "$fm_content" "compatibility") || true
  if [[ -n "$compat" && ${#compat} -gt 500 ]]; then
    check "warn" "compatibility field: ${#compat} chars (max 500)"
  fi

  # --- Check: recommended sections ---
  for section in "When to use this skill" "Instructions" "Examples" "Best practices" "References"; do
    if grep -q "^## ${section}" "$skill_md"; then
      check "ok" "Recommended section: $section"
    else
      check "warn" "Missing recommended section: ## ${section}"
    fi
  done

  # --- Check: file length ---
  if [[ $line_count -gt 500 ]]; then
    check "warn" "File length: ${line_count} lines (recommended max: 500 — move details to references/)"
  else
    check "ok" "File length: ${line_count} lines (OK)"
  fi

  echo ""
  echo "  Issues: ${ERRORS} errors, ${WARNINGS} warnings"
  echo ""

  TOTAL_ERRORS=$((TOTAL_ERRORS + ERRORS))
  TOTAL_WARNINGS=$((TOTAL_WARNINGS + WARNINGS))
}

main() {
  if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
    usage
    exit 0
  fi

  if [[ "${1:-}" == "--all" ]]; then
    local skills_dir="${2:-.}"
    local found=0
    for dir in "${skills_dir%/}"/*/; do
      if [[ -f "${dir}SKILL.md" ]]; then
        found=1
        validate_skill "$dir"
      fi
    done
    if [[ $found -eq 0 ]]; then
      echo "No skill directories (containing SKILL.md) found in: $skills_dir"
      exit 1
    fi
    echo "========================================"
    echo "Total: ${TOTAL_ERRORS} errors, ${TOTAL_WARNINGS} warnings"
    [[ $TOTAL_ERRORS -eq 0 ]] && exit 0 || exit 1
  else
    local skill_dir="${1:-.}"
    validate_skill "$skill_dir"
    [[ $TOTAL_ERRORS -eq 0 ]] && exit 0 || exit 1
  fi
}

main "$@"

N:skill-standardization
D:Standardize and validate SKILL.md files against the Agent Skills specification. Use when creating or rewriting a skill, auditing spec compliance, sharpening trigger descriptions, canonicalizing overlapping skills into a canonical skill plus compatibility alias, or checking whether derived discovery surfaces (`skills.json`, README/setup inventories, `SKILL.toon`, `SKILL.compact.md`) still match the live skill folders. In repo-root maintenance loops, prefer truthful validator commands instead of bare `scripts/...` examples.
G:skill-management standardization validation agentskills-spec automation deduplication catalog-sync compact-drift canonical-skill
U[9]:
  Create a new SKILL.md from scratch
  Audit an existing skill for spec compliance
  Repair weak trigger descriptions or stale route-outs
  Canonicalize duplicates into a default skill plus compatibility alias
  Validate repo-level discovery/catalog drift after add/rename/remove/alias work
  Review SKILL.toon / SKILL.compact.md drift after material rewrites
  Review alias-side support docs/checklists/examples when a compatibility alias mirrors the canonical contract
  Use repo-root validator commands truthfully in maintenance loops
  Do residue scans for stale command/path/discovery copy
S[5]{n,action}:
  1,Pick the smallest working mode from references/working-modes.md
  2,Run the truthful validator command for the current working directory
  3,Fix or write the primary SKILL.md and its description/evals
  4,Run catalog sync after structural changes and refresh derived surfaces, including alias-side support packets, when the rewrite was material
  5,Do a residue scan before calling the pass done