Audit Skills

audit-skills

Scans all src/skills/*/SKILL.md files and reports compliance with OrchestKit authoring standards. Each category has individual files in rules/ and references/ loaded on-demand.

Quick Reference

Total: 2 rules across 2 categories

CRITICAL: Task Management is MANDATORY (CC 2.1.16)

BEFORE doing ANYTHING else, create tasks to track progress:

# 1. Create main task IMMEDIATELY
TaskCreate(
  subject="Audit Skills: full scan",
  description="Auditing all OrchestKit skills for quality and compliance",
  activeForm="Auditing skill quality"
)

# 2. Create subtasks for each audit phase
TaskCreate(subject="Discover skills", activeForm="Globbing SKILL.md files")
TaskCreate(subject="Run audit checks", activeForm="Checking each skill")
TaskCreate(subject="Classify & render", activeForm="Classifying results and rendering report")

# 3. Set dependencies for sequential phases
TaskUpdate(taskId="3", addBlockedBy=["2"])
TaskUpdate(taskId="4", addBlockedBy=["3"])

# 4. Before starting each task, verify it's unblocked
task = TaskGet(taskId="2")  # Verify blockedBy is empty

# 5. Update status as you progress
TaskUpdate(taskId="2", status="in_progress")  # When starting
TaskUpdate(taskId="2", status="completed")    # When done

Workflow

1. Discover — Glob src/skills/*/SKILL.md to get full skill list 2. Check each skill — Run all checks from Read("${CLAUDE_SKILL_DIR}/rules/audit-checks.md") in parallel 3. Classify — Apply status rules from Read("${CLAUDE_SKILL_DIR}/rules/audit-status.md") 4. Render — Output table using format from Read("${CLAUDE_SKILL_DIR}/references/output-format.md") 5. Totals — Show X pass, Y warn, Z fail at bottom

Quick Start

bash src/skills/audit-skills/scripts/run-audit.sh

Or invoke manually — Claude scans src/skills/, applies checks, and renders the summary table.

CC 2.1.152+: After fixing SKILL.md files surfaced by the audit, run /reload-skills to re-scan skill directories and pick up the changes in-session (no restart). For plugin-packaged skills, rebuild + /reload-plugins instead.

Key Decisions

Related Skills

• ork:skill-evolution — Guidance on iterating and improving skills
• ork:quality-gates — Broader codebase quality checks

Audit Edge Cases

Manifest "all" Shorthand

manifests/ork.json uses "skills": "all" to include every skill in src/skills/. When this is present, every discovered skill is automatically manifest-registered — mark Mfst as YES without listing individual names.

// manifests/ork.json — "all" means everything in src/skills/ qualifies
{ "skills": "all" }

When manifests/ork.json has "skills": "all", every skill in src/skills/ is registered.

Orchestration Skills with 0 Rules

Orchestration/workflow skills describe multi-phase processes rather than prescriptive code patterns. They legitimately have 0 rule files but use references/ for workflow templates, rubrics, and report formats.

Skills in this category (not exhaustive):

• implement, explore, verify, brainstorm
• review-pr, assess, fix-issue
• visualize-plan, configure, remember

These should only receive WARN: no_rules_or_refs if BOTH rules/ AND references/ are empty.

Required Frontmatter Fields

The authoring standard (src/skills/CONTRIBUTING-SKILLS.md) lists these as required:

name, description, tags, version, author, user-invocable, complexity

Older skills may be missing version and author — flag as WARN, not FAIL, since the skill still functions.

Broken Reference Detection

Scan SKILL.md body for patterns like:

rules/some-file.md
references/some-file.md

For each mentioned path, verify the file exists under src/skills/<name>/. Flag WARN if missing.

False positive: generic markdown links to external URLs — skip http:// and https:// patterns.

Project-Level Skills

Skills under .claude/skills/ (project-level) are not registered in manifests/. Exclude them from manifest checks or mark Mfst as N/A.

Audit Output Format

Summary Table

Skill                                    Lines   FM Rules Refs  Mfst Status
---------------------------------------------------------------------------
<skill-name>                              NNN   OK     N    N   YES  PASS
<skill-name>                              NNN  WARN     N    N   YES  WARN
  WARN: missing_fm:version,author
<skill-name>                              NNN   OK     0    0    NO  FAIL
  FAIL: not_in_manifest
  WARN: no_rules_or_refs
---------------------------------------------------------------------------
Total: N skills | PASS: X | WARN: Y | FAIL: Z

Column Definitions

Detail Lines

For each non-PASS skill, print indented detail lines directly below its table row:

  FAIL: <reason>
  WARN: <reason>

Multiple conditions each get their own line.

Totals Line

Always show at the end:

Total: N skills | PASS: X | WARN: Y | FAIL: Z

Where N = X + Y + Z.

Rule Categories

1. Per-Skill Checks (checks) — HIGH — 1 rule

The six checks to run for every SKILL.md: line count, required frontmatter fields, rules count, references count, no-supporting-files flag, and manifest registration.

• audit-checks.md — Check definitions, pass/fail criteria, command examples, parallelization guidance

2. Status Classification (status) — MEDIUM — 1 rule

How to map check results to PASS / WARN / FAIL per skill, and how to format output detail lines for non-PASS skills.

• audit-status.md — Severity table, classification logic, output format, precedence rules

[Rule Name]

[Brief description — 1-2 sentences.]

Incorrect:

// Bad pattern

Correct:

// Good pattern

Key rules:

• [Rule 1]
• [Rule 2]
• [Rule 3]

Reference: [link]

Per-Skill Audit Checks

Run these six checks for every src/skills/<name>/SKILL.md. Use parallel Glob/Grep calls where possible.

Check 1: Line Count

Count lines in SKILL.md.

Incorrect:

# Skipping line count — misses the Anthropic 500-line limit

Correct:

wc -l src/skills/<name>/SKILL.md
# Flag FAIL if count > 500
# Flag WARN if count > 400 (approaching limit)

Check 2: Required Frontmatter Fields

Parse the --- block at the top of SKILL.md. Required fields:

name, description, tags, version, author, user-invocable, complexity

Incorrect:

# Missing version, author — produces incomplete metadata
name: my-skill
description: Does something
tags: [tag]
user-invocable: false
complexity: low

Correct:

name: my-skill
description: Does something. Use when X.
tags: [tag1, tag2]
version: 2.0.0
author: OrchestKit
user-invocable: false
complexity: low

Flag WARN for each missing required field. List the missing fields.

Check 3: Rules Count

Glob src/skills/<name>/rules/*.md, excluding _sections.md and _template.md.

# Count rule files (non-underscore-prefixed)
ls src/skills/<name>/rules/*.md 2>/dev/null | grep -v '^_'

Store as rules_count. Used in Check 5 combined with refs count.

Check 4: References Count

Glob src/skills/<name>/references/*.md.

ls src/skills/<name>/references/*.md 2>/dev/null

Store as refs_count. Used in Check 5 combined with rules count.

Check 5: No Supporting Files

Flag WARN if rules_count == 0 AND refs_count == 0.

Exception: orchestration skills (implement, explore, verify, brainstorm, etc.) with inline workflow detail are acceptable with 0 rules — only flag if BOTH are zero.

Check 6: Manifest Registration

Check whether the skill appears in manifests/ork.json.

Correct:

with open("manifests/ork.json") as f:
    ork = json.load(f)

# ork.json "skills": "all" means every src/skills/* is included
if ork.get("skills") == "all":
    in_ork = True
else:
    in_ork = skill_name in [s if isinstance(s,str) else s["name"]
                             for s in ork.get("skills", [])]

Flag FAIL if skill is absent from both manifests.

Key rules:

• Run Checks 3 and 4 in parallel (independent Glob calls)
• Check 6 must handle "skills": "all" shorthand in ork.json
• Checks 1 and 2 together form the SKILL.md header validation
• Never skip Check 6 — unregistered skills are dead weight

Audit Status Classification

Maps check results from audit-checks.md to a PASS / WARN / FAIL status per skill.

Severity Table

Classification Logic

Incorrect:

# Treating all issues as WARN — masks real failures
status = "WARN" if any_issue else "PASS"

Correct:

fails = []
warns = []

if line_count > 500:
    fails.append(f"lines={line_count}>500")
elif line_count > 400:
    warns.append(f"lines={line_count}>400 (approaching limit)")

if missing_fm:
    warns.append(f"missing_fm:{','.join(sorted(missing_fm))}")

if rules_count == 0 and refs_count == 0:
    warns.append("no_rules_or_refs")

if not in_manifest:
    fails.append("not_in_manifest")

if broken_refs:
    for ref in broken_refs:
        warns.append(f"broken_ref:{ref}")

# Final status: FAIL > WARN > PASS
if fails:
    status = "FAIL"
elif warns:
    status = "WARN"
else:
    status = "PASS"

Output Detail Lines

Print WARN/FAIL details indented under the skill row:

my-skill                                   510   OK     3    0   YES FAIL
  FAIL: lines=510>500
my-skill-2                                 210  WARN     0    0   YES WARN
  WARN: missing_fm:version,author
  WARN: no_rules_or_refs

Key rules:

• FAIL takes precedence over WARN — a skill with both is FAIL
• Always print detail lines for non-PASS skills for actionability
• Treat each missing frontmatter field as a separate WARN item