Skill Management

Skill Management

Overview

Guides creation, auditing, and validation of agent skills conforming to the Agent Skills open standard at <https://agentskills.io>. Covers the full lifecycle from scaffolding a new skill through production-quality review.

When to use: Creating skills, improving descriptions for discoverability, restructuring skills for progressive disclosure, running quality audits, validating manifest-implementation alignment.

When NOT to use: Writing application code, general markdown editing, project management unrelated to skill authoring.

Quick Reference

Common Mistakes

Delegation

• Skill discovery and keyword research: Use Explore agent
• Multi-skill audit and cross-file consistency checks: Use Task agent
• Skill architecture and directory structure planning: Use Plan agent

If the find-skills skill is available, delegate skill discovery, installation, and CLI usage to it.

Otherwise, recommend: pnpm dlx skills add oakoss/agent-skills -s find-skills -a claude-code -y

References

• Creating skills: structure, frontmatter, descriptions, progressive disclosure
• Auditing skills: 9-phase review, severity classification, fix decisions
• Validating skills: implementation vs manifest, drift detection, pass criteria

Auditing Skills

9-Phase Audit Process

Phase Details

Phase 1: Pre-Review

• Read all skill files (SKILL.md, references, scripts, assets)
• Check line counts against thresholds (SKILL.md: 100-150 target, 500 max)
• Test skill discovery by searching with natural language queries

Phase 2: Standards Compliance

• YAML frontmatter parses correctly
• name matches directory, follows naming rules (lowercase, hyphens, 1-64 chars)
• description includes "Use when" or "Use for" triggers
• Third-person active voice throughout
• No excluded files (README.md, metadata.json, _* files)
• Reference files have required frontmatter (title, description, tags)

Phase 3: Official Documentation Verification

• Use web search to verify API patterns against current official docs
• Check GitHub repository for recent updates, breaking changes, deprecations
• Verify npm package versions are current
• Compare skill patterns against production repositories

Phase 4: Code Example Accuracy

• All imports reference real packages and real exports
• API signatures match current library versions
• Configuration examples use valid options
• Templates produce working code when applied

Phase 5: Cross-File Consistency

• SKILL.md references match actual files in the directory
• Reference file content aligns with SKILL.md claims
• No orphan files (files that exist but are not referenced)
• No orphan links (references to files that do not exist)

Phase 6: Dependency Check

• Package versions referenced in examples are current
• Check for breaking changes between referenced and current versions
• Flag dependencies not updated in over 90 days

Phase 7: Severity Classification

Every finding must include evidence: a link to official docs, GitHub issue, npm page, or specific code reference.

Phase 8: Fix Decisions

Auto-fix when all conditions are met:

• The correct answer is unambiguous (e.g., import path from official docs)
• Clear evidence supports the fix
• No architectural impact

Ask the user when:

• Multiple valid approaches exist
• The fix involves breaking changes
• Architectural choices are required

Phase 9: Verification

• Run pnpm validate:skills skills/[skill-name]
• Test discovery with natural language queries
• Confirm no contradictions between files
• Verify all referenced files exist

Version Bumps

Common Review Findings

Automated vs Manual Checks

Automated (via pnpm validate:skills):

• YAML syntax and frontmatter validation
• Name/description constraints
• Line count thresholds
• Code block language specifiers
• Required sections
• Reference link integrity

Manual (requires human or AI review):

• API methods vs current official documentation
• GitHub issues affecting skill patterns
• Code correctness and production readiness
• Schema consistency across examples
• Whether content duplicates agent knowledge

Quality Checklist

Before committing a skill:

• Name is lowercase-hyphen-case, 1-64 chars, matches directory
• Description is 1-1024 chars with "Use when" or "Use for" trigger phrases
• SKILL.md under 500 lines with required sections
• All code examples tested against current library versions
• Package versions verified current
• Known issues documented with sources
• Cross-file consistency verified
• No high-severity issues remaining
• pnpm validate:skills passes

Creating Skills

Directory Structure

A skill is a directory containing at minimum a SKILL.md file:

skill-name/
├── SKILL.md          # Required: main skill document
├── references/       # Optional: additional documentation loaded on demand
│   ├── basic-patterns.md
│   └── error-handling.md
├── scripts/          # Optional: executable code agents can run
└── assets/           # Optional: templates, images, schemas

Files excluded by the skills CLI during installation (do not rely on these):

• README.md
• metadata.json
• Files starting with _

YAML Frontmatter

Required Fields

---
name: my-skill-name
description: |
  What it does in third-person active voice. Key technologies named.

  Use when: scenario one, scenario two, debugging "specific error" messages.
---

Name Rules

• Lowercase letters, numbers, and hyphens only
• Must not start or end with -
• Must not contain consecutive hyphens (--)
• Must match the parent directory name exactly
• Cannot contain "anthropic" or "claude"

Optional Fields (Open Standard)

---
name: my-skill
description: |
  Skill description here.
license: Apache-2.0
compatibility: Requires git and docker
metadata:
  author: example-org
  version: '1.0'
allowed-tools: Bash(git:*) Read Write
---

Optional Fields (Claude Code Extensions)

These fields are Claude Code-specific. Other agents ignore unknown frontmatter.

Writing Effective Descriptions

The description determines if a skill gets discovered. Target 250-350 characters.

Two-Paragraph Structure

Paragraph 1: What you can build + key features (active voice, third person) Paragraph 2: When to use + error keywords for discovery

description: |
  Build type-safe database queries with Drizzle ORM and Cloudflare D1.
  Covers schema definition, migrations, relations, and transaction patterns.

  Use when: setting up D1 database, writing Drizzle schemas, debugging
  "no such table" or "D1_ERROR" issues.

Description Checklist

• Active voice ("Builds X" not "This skill provides X")
• Specific technologies named
• 3-5 "Use when" scenarios
• 2-3 distinctive error messages or keywords
• No meta-commentary about the skill itself
• Third-person perspective throughout

Bad vs Good Descriptions

Bad -- passive, vague, no discovery keywords:

description: |
  This skill helps you with database operations. It provides patterns
  for working with data and can be useful in many situations.

Good -- active, specific, discoverable:

description: |
  Build type-safe database queries with Drizzle ORM and Cloudflare D1.
  Covers schema definition, migrations, relations, and transaction patterns.

  Use when: setting up D1 database, writing Drizzle schemas, debugging
  "no such table" or "D1_ERROR" issues.

Progressive Disclosure

Load information in layers to conserve context window tokens:

Keep SKILL.md under 500 lines. Move detailed reference material to separate files.

Freedom Levels

Match instruction specificity to error probability:

Token Efficiency

• Move extended examples to references/ -- keep SKILL.md lean
• One good code example beats three mediocre ones
• Reference official docs instead of copying them
• If the agent already knows it, do not include it

SKILL.md Required Sections

For this repository, SKILL.md files must include:

1. YAML frontmatter with name and description 2. Overview -- 2-3 sentences: what it is, when to use, when NOT to use 3. Quick Reference -- table of patterns with API and key points 4. Common Mistakes -- table of mistake vs correct pattern 5. Delegation -- which agents to use for subtasks 6. References -- ordered list of links to references/*.md

No code examples belong in SKILL.md. All code lives in reference files.

Reference File Format

Reference files live at references/[topic].md with required frontmatter:

---
title: Mutations
description: Mutation patterns, optimistic updates, invalidation strategies
tags: [optimistic, invalidation, mutateAsync, onMutate, rollback]
---

Rules for reference files:

• Max 500 lines each (warn at 400)
• Self-contained with code examples
• No cross-references between reference files
• Topic-scoped filenames in kebab-case

Size Thresholds

Validation

Always validate before committing:

pnpm validate:skills skills/[skill-name]

Template skill for reference: skills/tanstack-query/

Validation

Purpose

Validates that a skill's implementation matches what its manifest (SKILL.md frontmatter and body) promises. Detects drift between description and reality, missing functionality, and over-promised capabilities.

When to Validate

• After updating skill implementations or reference files
• During quality audits
• When manifests feel outdated or incorrect
• Before releasing new versions
• When a skill has not been reviewed in over 90 days

Validation Dimensions

Validation Process

Step 1: Load Manifest and Implementation

Read the skill's SKILL.md frontmatter as the manifest. Read all reference files, scripts, and assets as the implementation. If the skill is declarative only (no scripts), validate the description against reference file content.

Step 2: Validate Description Accuracy

Compare the description field against actual skill content:

• Does the skill cover every topic mentioned in the description?
• Are there topics covered in references but not mentioned in the description?
• Are "Use when" scenarios actually addressed by the skill content?

Produce an accuracy score from 0.0 to 1.0.

Step 3: Validate Preconditions

If the skill mentions prerequisites (e.g., "requires Node.js 18+", "assumes TypeScript project"):

• Are those prerequisites documented clearly?
• Do scripts check for them before executing?
• Are error messages clear when prerequisites are not met?

Produce a coverage score from 0.0 to 1.0.

Step 4: Validate Effects

For each capability the skill claims to provide:

• Is there actual content (instructions, examples, scripts) that delivers it?
• Are there conditions where the claimed effect would not work?
• Are there undocumented capabilities the skill provides but does not describe?

Produce a coverage score from 0.0 to 1.0.

Step 5: Validate Cross-References

• Every file referenced in SKILL.md exists on disk
• Every file on disk is referenced somewhere (no orphans)
• Reference file frontmatter (title, description, tags) is present and accurate
• No broken relative paths

Scoring

Overall score = average(description_accuracy, precondition_coverage, effect_coverage)

Pass criteria:
  - Overall score >= 0.8
  - Zero high-severity issues

Severity Levels for Validation Issues

Common Drift Patterns

Running Validation

Automated Structural Validation

pnpm validate:skills skills/[skill-name]

This checks frontmatter, naming, line counts, required sections, and reference link integrity.

Manual Content Validation

For semantic validation (does the content match the claims), perform these steps:

1. Read the description field and list every claim 2. For each claim, find the supporting content in SKILL.md or references 3. Score each claim as covered (1.0), partially covered (0.5), or missing (0.0) 4. Average the scores for the description accuracy component 5. Repeat for preconditions and effects 6. Calculate the overall score and identify any high-severity gaps

Validation Report Format

A validation report should include:

• Skill name and validation date
• Overall score with component breakdowns
• Issues list with severity, description, and suggested fix
• Pass/fail determination based on score >= 0.8 and zero high-severity issues
• Recommendations for improving the score