Skill Expert

Skill Expert

Create skills that are small enough to load, specific enough to trigger correctly, and complete enough for another agent to execute without guessing.

Operating Principles

• Treat the context window as a shared budget: the skill loads alongside the host agent's own task, so every line you add spends budget another agent needs. Add only instructions, examples, scripts, references, and assets that change what the agent does.
• Assume the host agent is capable. Preserve space for non-obvious domain knowledge, fragile command sequences, exact validation steps, and reusable artifacts.
• Keep one skill focused on one durable capability. Split unrelated domains into separate skills.
• Prefer runtime-neutral language such as "host agent" unless the target repository explicitly requires Claude Code, Codex, or another runtime.
• Preserve existing repository conventions for skill locations, metadata, validators, and invocation style.
• Design for composability. A skill may load alongside other skills, so avoid global claims, broad ownership of unrelated work, or instructions that conflict with normal agent behavior.
• Keep the skill unsurprising. Its files, scripts, and instructions should match the user's stated intent; never create misleading skills, hidden exfiltration, unauthorized access workflows, or disguised harmful behavior.
• Inspect external or third-party skills before use, especially when network access, shell tools, secrets, or hosted execution are involved.
• Do not create extra human-facing files such as README.md, CHANGELOG.md, INSTALLATION.md, or quick-reference documents unless the repo already requires them, and leave out unused placeholder directories and files.

Workflow

1. Determine the scope.

• Identify the task or domain the skill should cover.
• Start from 2-3 concrete use cases, including trigger phrases, expected steps, inputs, outputs, and edge cases.
• Ground the skill in real project artifacts where possible: runbooks, docs, schemas, previous fixes, execution traces, review comments, or repeated user corrections.
• Match the explanation level to the user's context. Briefly define terms like eval, benchmark, JSON, or assertion when the user has not shown they use that vocabulary comfortably.
• Ask at most the blocking questions. If the user asked for autonomous work or provided enough context, make reasonable assumptions and continue.
• In non-interactive or headless mode, proceed with explicit assumptions and report them at the end.

2. Choose the skill location.

• Use the current repo's skill directory when editing project skills.
• Use .agents/skills/<name> for Codex repo skills and .claude/skills/<name> for Claude Code project skills unless the repo already uses another convention.
• Use a personal skill directory only when the user asks for personal/global installation.
• Match the existing naming and metadata conventions before inventing new ones.
• For distributable Codex workflows, consider whether the skill should later be bundled in a plugin rather than left as a local folder.

3. Design the reusable contents.

• Use SKILL.md for the essential workflow and navigation.
• Add scripts/ when operations are deterministic, fragile, repetitive, parse-heavy, destructive enough to need dry-run behavior, or need explicit error handling.
• Add references/ for detailed docs, schemas, policies, variants, examples, or rare advanced paths that should be loaded only when needed.
• Add assets/ for templates, fixtures, boilerplate, fonts, images, or files copied into outputs rather than read as instructions.
• Add evals/ only when the skill needs repeatable quality or trigger testing; keep initial evals small and realistic.

4. Initialize or update the skill.

• If the repo provides an initializer, prefer it. Example from a repo root:

     .agents/skills/skill-creator/scripts/init_skill.py <skill-name> --path .agents/skills

• If no initializer exists, create <skills-dir>/<skill-name>/SKILL.md manually.
• When updating an existing skill, preserve useful resources and change only what supports the requested behavior.
• Preserve the original directory name and frontmatter name when updating an installed skill. If the installed copy is read-only, copy it to a writable temp path, edit there, then package from the copy.

5. Write the metadata.

• name: lower-case letters, numbers, and hyphens only; max 64 characters; do not start/end with hyphen; avoid consecutive hyphens; match the directory name.
• description: max 1024 characters; include what the skill does and when to use it.
• Front-load the core use case and trigger words so the skill still works if clients shorten descriptions in crowded skill lists.
• Prefer this formula: <capability>. Use when <specific triggers, file types, tasks, or user phrases>.
• Focus on user intent, not internal implementation details.
• Make descriptions assertive enough to avoid under-triggering, but not so broad that they hijack adjacent tasks.
• Add negative triggers only when they prevent realistic conflicts with nearby skills.
• Avoid optional frontmatter unless the current runtime or repo already uses it. If used, treat runtime-specific fields as portability tradeoffs.
• Examples of runtime-specific extras: Claude Code may use invocation controls, arguments, paths, hooks, or subagent context; Codex may use agents/openai.yaml for UI metadata, implicit invocation policy, and dependencies; OpenAI API skills are versioned bundles mounted in hosted or local shell environments.

6. Write SKILL.md.

• Use imperative instructions for the host agent.
• Do not put discovery-only "when to use" guidance in the body; the body loads only after the skill is selected.
• Start with the smallest useful mental model, then the execution workflow.
• Anchor each load-bearing concept with a leitwort — a named, evocative term of art (connascence, behavior-preserving, cold-start test) the host agent will repeat back while acting, rather than a generic instruction it reads once. Name it where the agent narrates its decisions, not only in a heading; model the sentence you want echoed. When the repo defines a leitwörter convention (e.g. LEITWORTER.md and leitworter.json), follow it and reuse the canonical word over a synonym. See References.
• Include concrete examples only when they clarify behavior or prevent common mistakes.
• Add a short gotchas section when the agent is likely to make a non-obvious mistake.
• Provide defaults, not menus. Pick the recommended path and mention alternatives only as escape hatches.
• Define fallback paths for missing scripts, optional tools, unavailable services, or non-interactive execution.
• Define the output contract for any generated artifacts: path, file name pattern, required sections, and validation expectations.

7. Apply progressive disclosure to defend the shared budget.

• Keep SKILL.md lean; split content as it approaches 100-500 lines or mixes distinct domains. Reference files cost nothing until read, so move rarely-needed detail there.
• Keep references one level deep from SKILL.md, such as references/schema.md.
• Tell the agent exactly when to read each reference.
• Add a table of contents to any reference longer than 100 lines.
• Avoid duplicating the same information in SKILL.md and a reference file.

8. Validate and package.

• Run repo-local validation when available. Examples:

     python3 .agents/skills/skill-creator/scripts/quick_validate.py .agents/skills/<skill-name>
     python3 .agents/skills/skill-creator/scripts/package_skill.py .agents/skills/<skill-name> /tmp

• If those helpers are unavailable, manually verify frontmatter, name matching, description quality, paths, resource references, and file organization.
• When the repo guards leitwörter (e.g. scripts/check_leitworter.py), run it so an edit does not silently delete a named anchor or drift a canonical word back to a generic phrasing.
• Run any added scripts at least once with representative inputs.
• Grep for stale project names, old stack assumptions, or copied-template leftovers before finalizing.

9. Evaluate real behavior when risk justifies it.

• Test trigger accuracy and output quality against a baseline before trusting the skill.
• See References below for when to load the detailed eval and description-tuning guides.

Quality Bar

• The skill has one clear purpose and a discoverable description.
• The metadata alone is enough for a host agent to choose the skill correctly.
• The body explains how to execute, not why the skill exists.
• Load-bearing concepts are named with leitwörter the agent will repeat while acting; generic filler verbs ("carefully", "thoroughly", "make sure") are replaced by named anchors, and shared concepts use the canonical word, not a synonym.
• When the skill completes a task or produces an artifact, its Output Contract states an acceptance contract — the explicit, checkable criteria (real commands, observable behavior) that define done — so the host agent verifies completion instead of declaring done by assumption.
• The skill avoids stale, time-sensitive, or repo-foreign assumptions.
• All referenced files exist and are linked from SKILL.md.
• Scripts communicate success and failure through clear stdout, stderr, and exit codes.
• The final directory contains only files that directly support the skill.

Troubleshooting

• If the skill does not activate, make the description more specific with trigger words, file types, operations, and user phrases.
• If multiple skills conflict, narrow each description and add negative triggers where appropriate.
• If the skill activates too often, add near-miss exclusions and move broad background material into a narrower skill.
• If validation fails, fix the first structural problem before polishing prose.
• If the body is long, extract rarely used details into references/.
• If the agent would need to guess a path, command, schema, or output shape, add that detail or provide a fallback.
• If instructions are ignored, cut verbosity, move detailed material to references, surface gotchas earlier, and add a validation loop.
• If the body feels generic and unmemorable, find its load-bearing concept and give it a leitwort — a precise, narratable term of art — then make the agent state it as it works.

Delivery Contract

When creating or updating a skill, finish with:

• created_files: new files and important changed files.
• validation: commands run and results, or why validation was not available.
• sources_used: external specs, docs, project artifacts, or execution traces used to shape the skill.
• assumptions: decisions made without asking the user.
• optional_refinements: concise ideas that would improve the skill later without blocking current use.

References

• Read references/script-standards.md when a skill bundles scripts/, or when deciding how tightly to constrain the host agent (text vs. pseudocode vs. script — the degrees of freedom).
• Read references/evaluation.md when the user asks to test, benchmark, compare, or prove a skill is better, when the skill has objectively verifiable outputs, or when you need the detailed eval workspace, grading, benchmark, and human-review loop.
• Read references/description-optimization.md when the user asks why a skill does or does not trigger, when improving frontmatter description text, or when optimizing a skill description for trigger accuracy.
• Read the repo's LEITWORTER.md (with the leitworter.json registry) when authoring or improving a skill's body, to apply the leitwörter convention and the canonical cross-skill vocabulary.

Description Optimization

Use this reference when a skill under-triggers, over-triggers, competes with nearby skills, or needs a measurable frontmatter description improvement.

Build Trigger Evals

Create about 20 queries:

• 8-10 should-trigger queries.
• 8-10 should-not-trigger queries.
• Mix terse, detailed, formal, casual, typo-prone, and context-heavy prompts.
• Include file paths, URLs, column names, project names, backstory, and adjacent tasks where realistic.
• Include cases where the user does not name the skill or file type but clearly needs it.
• Make should-not-trigger queries near misses that share keywords but need another skill, not obviously irrelevant prompts.

Poor trigger evals are too abstract:

Format this data.
Extract text from a PDF.
Create a chart.

Better trigger evals are concrete and a little messy:

my manager sent ~/Downloads/Q4 sales final FINAL v2.xlsx and wants margin % added from revenue col C and cost col D. can you fix the sheet and send it back?

Avoid simple one-step tasks when measuring automatic triggering. Some agents skip skills for tasks they can solve directly, even when the description matches.

Measure

Run each query against the same client/model that will use the skill. If possible, run each query 3 times and compute trigger rate.

Passing criteria:

• Should-trigger query: skill triggers above the chosen threshold.
• Should-not-trigger query: skill stays below the threshold.

If no trigger-observability tooling exists, still build the query set and review the description manually, but do not claim measured accuracy.

Improve

When rewriting the description:

• Generalize from failures instead of listing every missed query.
• Stay comfortably below 1024 characters; 100-200 words is usually enough.
• Front-load the distinctive user intent and key trigger words.
• Mention adjacent exclusions only when they reduce real conflicts.
• Change wording structure when repeated attempts fail; do not just append more phrases.
• Prefer "Use when..." language that names outcomes and user intent.
• Avoid implementation details unless they are important triggers.

Keep a held-out portion of the eval set. Choose the description that performs best on held-out prompts, not only the prompts used to improve it.

Report

When applying a new description, show:

• Previous description.
• New description.
• Trigger-eval score or qualitative rationale.
• Remaining known false negatives or false positives.

Skill Evaluation

Use this reference when a skill needs proof that it works, especially for file transforms, data extraction, code generation, fixed workflow steps, or other objectively verifiable outputs. For subjective work, use the same loop but rely more on human review than numeric assertions.

Eval Set

Start small:

• Create 2-3 realistic prompts that resemble what a user would actually type; expand only after the first results show useful signal.
• Include expected outputs and input files when relevant.
• Do not write detailed assertions until the task shape is clear.
• Ask the user to review the prompts when interactive; in headless mode, continue with explicit assumptions.

Suggested evals/evals.json shape:

{
  "skill_name": "example-skill",
  "evals": [
    {
      "id": 1,
      "prompt": "User task prompt",
      "expected_output": "Human-readable success description",
      "files": [],
      "expectations": []
    }
  ]
}

Workspace

Write results outside the skill directory so the skill stays clean:

<skill-name>-workspace/
  skill-snapshot/                # optional old copy for update baselines
  iteration-1/
    eval-<descriptive-name>/
      eval_metadata.json
      with_skill/
        outputs/
      without_skill/             # for brand-new skills
        outputs/
      old_skill/                 # for improving an existing skill
        outputs/

Use descriptive eval directory names instead of only numeric names.

For existing skills, snapshot the original before editing and use that snapshot as the baseline. For new skills, compare against no skill when possible.

Run Loop

1. Run the candidate and baseline on the same prompts. 2. If parallel workers are available, launch candidate and baseline runs together so they finish under comparable conditions. If not, run sequentially and record that limitation. 3. Save outputs, transcripts, and any available timing or token data immediately. 4. While runs execute, draft objective expectations. Do not force numeric assertions onto subjective outputs. 5. Inspect actual output files, not just the transcript summary. 6. Review execution traces, not just final answers, to spot wasted work, vague instructions, over-triggering, or missing defaults. 7. Grade each expectation with text, passed, and evidence. 8. Aggregate pass rate, time, token/tool cost, errors, and notes. 9. Show outputs and benchmark data to the user before revising when human judgment matters. 10. Iterate until the user is satisfied, feedback is empty, or changes stop improving results, and until the skill improves reliability enough to justify its added context and execution cost.

Grading

Use this structure for each run's grading.json:

{
  "expectations": [
    {
      "text": "The output contains the required section",
      "passed": true,
      "evidence": "Found heading 'Risk Summary' in report.md"
    }
  ],
  "summary": {
    "passed": 1,
    "failed": 0,
    "total": 1,
    "pass_rate": 1.0
  }
}

Grade conservatively:

• Pass only when the evidence shows real task completion, not surface compliance.
• Fail when the evidence is missing, contradictory, unverifiable, or coincidental.
• Quote or describe the evidence tightly.
• Critique weak expectations that would pass for bad outputs.
• Extract important output claims and verify them when feasible.

Benchmark Analysis

Look past aggregate pass rates:

• Expectations that pass in both candidate and baseline may not measure skill value.
• Expectations that fail everywhere may be broken, too hard, or checking the wrong thing.
• High variance suggests flaky prompts, ambiguous instructions, or nondeterministic execution.
• Large time, token, or tool-call increases must buy meaningful quality improvements.
• Repeated helper-code invention across runs is a signal to bundle a script.

Human Review

Prefer a reviewer UI or static HTML when available. If no browser/display is available, present each prompt, output path, grade summary, and key diff inline.

Empty feedback usually means the user accepted that case. Focus revisions on specific complaints and clear benchmark failures.

Script and Degrees-of-Freedom Standards

Read this when a skill bundles scripts/, or when deciding how much to constrain the host agent (text vs. pseudocode vs. script). The rest of authoring lives in SKILL.md.

Degrees of Freedom

• Use text instructions when multiple approaches are valid and context should guide execution.
• Use pseudocode, examples, or parameterized commands when there is a preferred pattern with valid variation.
• Use scripts with narrow inputs when the workflow is fragile, error-prone, security-sensitive, or likely to be repeated.

Script Standards

• Prefer a one-off pinned command when an existing tool with a few flags is enough.
• Move complex or repeatedly generated commands into scripts/.
• Reference bundled files with paths relative to the skill root.
• Make scripts non-interactive. Accept inputs through flags, environment variables, files, or stdin.
• Provide --help output with usage, flags, and examples.
• Send machine-readable results to stdout and diagnostics to stderr.
• Use clear error messages that say what failed, what was expected, and what to try next.
• Prefer structured output such as JSON, CSV, or TSV for data the agent will consume.
• Add idempotency and --dry-run support for stateful or destructive operations.