Improve Skill

Skill Improver

Increase the effectiveness of an existing skill by modeling user intent, testing the skill against that intent through mental simulation and live doc validation, and proposing ranked improvements — new features, UX gains, accuracy fixes, efficiency wins.

This skill is about whether the skill accomplishes what users need. Structural correctness (malformed frontmatter, missing fields, violated conventions) is repair-skill's domain — if obvious structural issues are present, note them briefly, recommend repair-skill, and continue with effectiveness analysis.

Phase 0: Load the Skill

Read $ARGUMENTS as the path to a skill directory or SKILL.md file.

• If a directory: read SKILL.md, then note which of references/, scripts/, examples/,

assets/ exist

• If a file: read it directly, then discover sibling resource directories

If the path is missing or ambiguous, use AskUserQuestion to resolve before proceeding.

Note any obvious structural issues in one sentence ("this description is first-person — recommend repair-skill for structural fixes") and move on. Do not run a full structural audit.

Phase 0 is complete when SKILL.md is loaded and the sibling directory inventory is noted.

Phase 1: Understand User Intent

Before analyzing, establish what the user wants. Use AskUserQuestion:

• "What specifically does this skill not do well?"
• Offer options: a specific gap they've noticed, "I'm not sure — run a full effectiveness

audit", "it works but I want new capabilities", "the workflow feels clunky"

Regardless of the answer, also infer the skill's purpose from its description and body. State your understanding of what problem it solves and for whom — one sentence — before proceeding. This grounds the entire analysis in the correct frame.

If the user named a specific complaint: orient the analysis toward that area and scan for related issues in the same workflow region. If the user is unsure: run the full Phase 2 audit across all five sub-analyses.

Phase 1 is complete when user intent is established and the skill's purpose is stated.

Phase 2: Effectiveness Analysis

Load ${CLAUDE_PLUGIN_ROOT}/skills/improve-skill/references/effectiveness-rubric.md before starting. It contains the severity framework, improvement type definitions, and effort/impact calibration criteria used in Phase 3.

Run all five sub-analyses. For each finding record: what the issue is, why it reduces effectiveness for the user, and the concrete improvement.

---

2a — Mental Simulation

Walk through the skill as Claude executing it with a concrete representative user request. Choose an input that exercises the main workflow path — not an edge case, but the typical use.

For each phase of the skill, evaluate:

• Missing info: What does this step need that hasn't been provided or gathered yet? If the

user hasn't specified something and the skill doesn't ask, what does Claude have to guess?

• Divergence points: Where would two different Claude instances execute this instruction and

arrive at meaningfully different outputs? These are underspecified steps.

• Dead ends: Where does the skill's workflow stop but the user's actual goal isn't yet

accomplished? What does the user have to do manually after the skill finishes?

• Friction: Where does the skill pause the user at a low-value decision point (Claude could

make a good default), or skip user input at a high-value moment (user has a strong preference)?

Document findings by type: stuck points, divergence points, dead ends, friction points.

---

2b — Live Doc Validation

Identify all factual claims in the skill that reference external standards: frontmatter field names, Claude tool names and behavior, API parameters, CLI flags, third-party service interfaces.

For each claim, verify against current documentation:

For Claude-specific claims (frontmatter options, tool names, model IDs):
  Use Task tool with subagent_type=claude-code-guide — faster and more accurate than WebSearch.

For third-party claims (npm packages, APIs, external CLIs):
  Use WebSearch or WebFetch against official documentation.

Flag drift between what the skill states and current reality. Severity: high if the claim produces broken output; medium if it produces outdated guidance; low if it's a naming change with no behavioral difference.

---

2c — Feature Adjacency Scan

Given the skill's purpose, identify capabilities that are absent but would be high-value:

• Adjacent: Naturally extends what the skill already does — same domain, one step further.

The user who just ran this skill would almost certainly want this next.

• Complementary: Commonly needed right before or after this skill. The user does this

manually today.

• End-to-end gap: The skill starts a job the user finishes by hand — workflow stops at

"here's a plan" when the user wanted "and apply it."

For each candidate: estimate implementation effort (one instruction change, new phase, or new script) and user value (rare edge case, common scenario, or blocks the skill in a key scenario).

---

2d — UX Flow Review

Evaluate the skill's interaction design:

• Does the skill surface all necessary questions at the start (before heavy work), or does it

interrupt mid-workflow with requests the user didn't anticipate?

• Is the "I don't know" path explicit? If a user triggers the skill without a specific

complaint, does the skill handle that gracefully, or does it assume the user knows?

• Does the output format match how users consume it? A report users read once can be dense

prose; one they apply iteratively needs more structure.

• Are there steps where the skill makes a consequential decision without user input?

---

2e — Edge Case Stress Test

After walking the main path (2a), deliberately try to break the skill. Identify 3–5 adversarial inputs that test failure modes:

• Missing or malformed input: What happens when expected files don't exist, paths are

wrong, or arguments are empty/garbled?

• Contradictory requirements: What if the user's stated goal conflicts with their input

(e.g., "improve this skill" on a file that isn't a skill)?

• Unsupported configurations: What if the skill encounters a structure it wasn't designed

for — a different framework version, an edge-case project layout, an unexpected file format?

• Boundary conditions: What if the input is very large (500-line SKILL.md), very small

(empty file), or has unusual characters in paths/names?

For each adversarial input, evaluate: does the skill detect the problem and surface a useful error, silently produce wrong output, or crash the workflow? Map findings to improvement types: missing error handling → NEW FEATURE, poor failure message → UX IMPROVEMENT, undetected bad state → ACCURACY FIX.

Phase 2 is complete when all five sub-analyses are finished and findings are recorded.

Phase 3: Improvement Proposal

Load ${CLAUDE_PLUGIN_ROOT}/skills/improve-skill/references/effectiveness-report-template.md for the output format before constructing the report. Reference ${CLAUDE_PLUGIN_ROOT}/skills/improve-skill/examples/sample-analysis.md to calibrate depth and specificity if needed.

Present findings grouped by improvement type — users think in terms of outcomes, not audit dimensions. Each entry must include: the sub-analysis code in brackets, what the gap is, why it matters to the user, and the specific fix. Calibrate severity using the criteria in references/effectiveness-rubric.md.

Ask: "Apply all improvements? Or select specific ones?"

Phase 3 is complete when the report is delivered and user selection is confirmed.

Phase 4: Apply Improvements

Apply confirmed items in order: new features → accuracy fixes → UX improvements → efficiency gains.

For each item:

• State what is being changed and why — reference the effectiveness principle, not "you asked"
• Make the edit or create the file
• Confirm the change integrates cleanly with surrounding content

After applying, briefly explain:

• What was changed and why
• What was added and why
• What was left out and why (effort outweighs benefit, or requires domain knowledge the

user must supply)

• What was not selected — note it remains available to apply later

Validation: After delivering the explanation, re-read the modified SKILL.md in full and confirm: all selected improvements are present, no surrounding content was inadvertently altered, phases are still numbered with exit conditions, and all references point to files that exist.

Phase 4 is complete when all confirmed items are applied, the explanation is delivered, and the validation pass finds no integration failures.

Phase 5: Structural Lint

After applying effectiveness improvements, invoke the skill-lint agent for a structural quality pass:

Use Task tool with subagent_type=claude-skills:skill-lint:
"Lint the skill at <path-to-skill-directory>. Auto-apply critical and major fixes, report
minor findings for user decision."

Wait for the agent to complete. If it auto-applied structural fixes, note them alongside the effectiveness changes from Phase 4. If it reports minor findings, present them to the user.

Phase 5 is complete when the lint agent returns and any user-selected minor fixes are applied.

Sample Analysis: create-skill

This is a complete example of an improve-skill effectiveness analysis run against create-skill. Use it to calibrate the expected depth and format.

---

Skill Purpose (stated in Phase 1)

create-skill generates new Claude Code skills or commands from scratch. A user provides requirements and receives a complete SKILL.md, ready to deliver.

---

Phase 2 Findings

2a — Mental Simulation

Representative request: "Create a skill that reviews pull requests for security issues."

Stuck point — Phase 0 (fetch docs): The phase launches a Task with subagent_type=claude-code-guide to check for new frontmatter options. If the subagent fails or returns nothing new, the phase continues. But there is no instruction for what to do if the subagent returns conflicting information (e.g., a field has changed its valid values). Claude has to guess.

Divergence point — Phase 4 (self-evaluation): The skill instructs Claude to score the generated skill on five dimensions against a "9.0/10.0" target. Two different Claudes will score the same skill differently — the criteria (Clarity, Precision, Efficiency) are named but not defined. One Claude might score Efficiency as 8/10, another as 7/10, and arrive at different decisions about whether to refine.

Dead end — post-delivery: After the skill is delivered, the user has a SKILL.md but no path to quality assurance. repair-skill exists for this, but create-skill never mentions it. The user who doesn't know about repair-skill misses a natural next step.

2b — Live Doc Validation

All frontmatter field names verified against current docs: ✓

No drift found in this example.

2c — Feature Adjacency Scan

Adjacent (high value): After generating a skill, suggest running repair-skill for a structural audit. Users who generate skills rarely know to do this separately, and repair often catches things the generation phase missed. This is a one-sentence addition at the end of Phase 3. Implementation effort: minimal.

Complementary (medium value): Before generating, help the user decide whether they need a skill vs a command vs a custom agent. The skill currently assumes the user knows the right artifact type. A one-question disambiguation in Phase 1 ("Is this something you'd trigger automatically, or invoke manually with /?") would prevent the wrong artifact being generated. Implementation effort: one AskUserQuestion in Phase 1.

2d — UX Flow Review

Friction: The Phase 1 interview asks five open-ended questions (Primary objective, Trigger scenarios, Inputs/outputs, Complexity, Execution needs). A user who doesn't know skill development best practices doesn't know what a "good" answer looks like. The interview gathers information but doesn't guide it — a user who says "complexity: high" because their problem feels hard might not realize how this maps to frontmatter choices.

Consequential silent decision: In Phase 2 Step 5 (delegation check), the skill instructs Claude to scan for existing resources before finalizing. But if an existing skill partially covers the requirement, Claude decides silently whether to extend it or create a new one. This is a significant decision the user would want to weigh in on.

---

Phase 3 Report

SKILL EFFECTIVENESS REPORT: create-skill

NEW FEATURES (capabilities the skill should have but doesn't)
──────────────────────────────────────────────────────────────
HIGH VALUE
  [2c] No post-delivery handoff to repair-skill — users who don't know it exists miss a
       natural quality-assurance step. The skill creates but never validates its output.
       Fix: Add one sentence at the end of Phase 3: "After delivery, suggest running
       repair-skill on the generated skill for a structural quality check."

MEDIUM VALUE
  [2c] No artifact-type disambiguation — skill assumes the user wants a skill, not a
       command or custom agent. Wrong artifact type requires starting over.
       Fix: Add AskUserQuestion in Phase 1: "Is this triggered automatically (skill) or
       invoked by the user via /? (command)" to route before generation begins.

UX IMPROVEMENTS (friction in the current workflow)
────────────────────────────────────────────────────
  [2d] Phase 1 interview asks open-ended questions but doesn't guide what good answers
       look like. Users unfamiliar with skill development give answers that don't map
       to the right frontmatter choices (e.g., complexity ≠ allowed-tools scope).
       Fix: Add one example per interview question showing how the answer shapes the
       generated skill.

EFFICIENCY GAINS (same outcome with less token cost)
──────────────────────────────────────────────────────
  [2a] Phase 4 self-evaluation scores 5 dimensions with undefined criteria — different
       Claudes score differently, making the 9.0 target arbitrary.
       Fix: Define each dimension's scoring criteria in one sentence, or move the scoring
       rubric to references/ where it can be elaborated without loading every invocation.

Effectiveness Report Template

Load at Phase 3. Use this exact structure. Omit any section with no findings — do not include empty headers.

---

Report Format

SKILL EFFECTIVENESS REPORT: <skill-name>

NEW FEATURES (capabilities the skill should have but doesn't)
──────────────────────────────────────────────────────────────
HIGH VALUE
  [2x] <what the gap is> — <why it matters to the user>.
       Fix: <specific change or addition>

MEDIUM VALUE
  [2x] <what the gap is> — <why it matters to the user>.
       Fix: <specific change or addition>

UX IMPROVEMENTS (friction in the current workflow)
────────────────────────────────────────────────────
  [2x] <what the friction is> — <effect on user>.
       Fix: <specific change>

ACCURACY FIXES (factual drift from current docs)
──────────────────────────────────────────────────
  [2x] <what claim is outdated> — <what docs say vs what skill says>.
       Fix: <specific correction>

EFFICIENCY GAINS (same outcome with less token cost)
──────────────────────────────────────────────────────
  [2x] <what the inefficiency is> — <cost to user>.
       Fix: <specific change>

Ordering Rules

• Within each section: HIGH → MEDIUM → LOW value
• Across sections: no required order — lead with the most important findings
• Each entry: sub-analysis code [2a/2b/2c/2d], the gap, why it matters, the fix
• Calibrate HIGH/MEDIUM/LOW using references/effectiveness-rubric.md

Closing

After presenting the report, ask:

"Apply all improvements? Or select specific ones?"

See examples/sample-analysis.md for a complete example of a real analysis output.

Effectiveness Rubric

Loaded during Phase 2. Defines improvement types, severity calibration, and effort/impact criteria used to rank findings in Phase 3.

---

Improvement Types

NEW FEATURES

Capabilities the skill doesn't have but should. The user needs to do something manually today, or the skill stops before the user's actual goal is accomplished.

Examples:

• Skill generates a plan but doesn't apply it — user applies manually
• Skill processes one file at a time but the user almost always has multiple
• Skill never verifies its own factual claims against current docs

UX IMPROVEMENTS

Things that work correctly but create friction. The outcome is right; the path to it is rough.

Examples:

• Necessary question asked in Phase 3 after heavy work is complete (should be Phase 1)
• Skill produces dense prose where the user will need to copy individual sections
• No explicit path for "I don't know what I want" — skill assumes a specific user goal
• Consequential decision made silently where user would want to choose

ACCURACY FIXES

Factual claims in the skill that have drifted from current reality. The skill is confidently wrong about something external — a tool name, an API parameter, a frontmatter field.

Examples:

• Frontmatter field listed that no longer exists or has been renamed
• Model ID that has been deprecated
• Third-party CLI flag that changed between versions

EFFICIENCY GAINS

The same outcome is achievable with less token cost, fewer tool calls, or fewer user interactions — without changing the output quality.

Examples:

• Three sequential Read calls that could be one
• A tool call where inline @path injection would work
• An agentic reasoning step that always produces the same structure (script candidate)

---

Severity Calibration

Severity is about impact on the user when the skill is triggered — not how hard the fix is.

HIGH VALUE

Apply this label when the absence of the improvement means the skill fails the user in a common scenario. The user either can't complete their goal, gets wrong output they'll trust, or has to do significant manual work that the skill should have done.

Signs: blocks the main use case, produces wrong output silently, creates a dead end after a long workflow, applies to every invocation not just edge cases.

MEDIUM VALUE

Apply when the improvement makes the skill meaningfully better for a common scenario, but the current behavior produces a usable (if degraded) outcome. The user can work around it.

Signs: adds a capability users would reach for regularly, removes friction from the main path, fixes docs that are outdated but the user could discover the right value independently.

LOW VALUE

Apply for polish — the skill works well, this makes it slightly better. Affects edge cases or infrequent scenarios.

Signs: applies only to a specific edge case, or the fix is cosmetic with no behavioral change.

---

Effort Estimation

Use effort estimates to help the user decide what to apply first. Three tiers:

One instruction change — Edit one sentence or paragraph in SKILL.md. No new phases, no new files. Examples: adding an explicit full-audit path, clarifying an underspecified step, updating a stale field name.

New phase or section — Add a meaningful new workflow phase with its own entry condition and exit condition. Requires ~10–30 lines of SKILL.md. Examples: adding live doc validation as a distinct phase, adding a post-apply explanation phase.

New script — Requires creating a file in scripts/, wiring it into SKILL.md with trigger condition, exact invocation, and output handling. Higher effort but produces deterministic, reliable behavior for consistency-critical steps.

---

The Stuck / Diverge / Dead-End / Friction Framework (2a)

These four categories from mental simulation map to improvement types:

---

Quick Reference: High-Signal Improvements

These patterns appear often and are almost always high-value:

1. No "I don't know" path — Any skill that starts by asking for user intent but offers no fallback for an uncertain user has a gap. Add an explicit full-audit mode.

2. Factual claims without verification — Any skill that references versioned external standards (frontmatter fields, API parameters, tool names) without checking them will eventually produce wrong output. Live doc validation is almost always worth adding.

3. Workflow stops at proposal — If the skill's final output is a plan, checklist, or report but the user obviously wants those things applied, that's a dead end gap.

4. Mid-workflow questions — Any question asked after Phase 1 that could have been asked in Phase 1 is friction. If the answer is needed to do the work, it belongs at the start.

5. Silent consequential decisions — If the skill makes a significant choice (what model to use, what to include in output, what to skip) without surfacing it, that's a divergence point. Either make it explicit or give the user control.