Utility Pm Skill Validate

PM Skill Validate

This skill audits an existing pm-skills skill against the repo's structural conventions and quality criteria. It produces a validation report that a human can scan and that utility-pm-skill-iterate can consume as input.

The validator checks two tiers:

• Tier 1 (Structural) - deterministic checks that mirror CI: frontmatter,

naming, file presence, description word count.

• Tier 2 (Quality) - LLM-assessed coherence checks: does the output

contract reference the template? Is the example complete? Are checklist items testable?

When to Use

• After creating a skill with utility-pm-skill-builder, before shipping
• After manually editing a skill, to confirm it still passes conventions
• Before running utility-pm-skill-iterate, to identify what needs improvement
• When a convention changes, to audit which skills need updating (batch mode)
• When reviewing a contributed skill for quality and completeness

When NOT to Use

• To create a new skill from scratch -> use utility-pm-skill-builder
• To fix or improve a skill -> use utility-pm-skill-iterate (feed it this report)
• To run CI checks in a pipeline -> use scripts/lint-skills-frontmatter.sh

(this skill is for interactive, deeper-than-CI validation)

Instructions

When asked to validate a skill, follow these steps:

Step 1: Identify the Target

Accept the skill name in any form:

• Directory name: deliver-prd
• Full path: skills/deliver-prd/SKILL.md
• Slash command: deliver-prd

Resolve to the canonical directory path: skills/{name}/.

If the skill directory does not exist, report immediately:

# Validation Report: {input}
Result: FAIL
Skill directory `skills/{input}/` does not exist.

Batch mode: If the input is --all, run Tier 1 structural checks across all skills and produce a summary table (see Step 5). Do not run Tier 2 in batch mode.

Step 2: Read Skill Files

Read all files in the skill directory:

Also read:

• The corresponding command file: commands/{command-name}.md
• The AGENTS.md entry for this skill

If reading files is not possible (MCP/embedded environment), ask the user to paste the content of each file before proceeding (see Degraded Mode).

Step 3: Run Tier 1 - Structural Checks

Run these deterministic checks. Each produces a PASS or FAIL line.

Step 4: Run Tier 2 - Quality Checks

Run these LLM-assessed checks. Each produces a PASS, WARN, or INFO line. Tier 2 findings are capped at WARN unless objectively grounded (placeholder leakage is the exception - it can FAIL).

Quality standard framing: These checks validate against current library conventions - what the shipped library actually does today. Findings graded WARN or INFO represent the v2.8 quality standard that newer skills (built with utility-pm-skill-builder) meet. Older skills may legitimately receive these findings until iterated through the lifecycle.

Step 5: Produce the Validation Report

Assemble the report using this exact structure. F-11 (utility-pm-skill-iterate) parses this report by section headings and pipe-delimited fields.

# Validation Report: {skill-name}
Date: {YYYY-MM-DD}
Skill version: {version from frontmatter}
Validator version: 1.0.0
Report schema: v1
Result: {PASS | WARN | FAIL}

## Summary
{1-2 sentence overall assessment.}
Errors: {n} | Warnings: {n} | Info: {n}

> Tier 2 findings are heuristic quality assessments and may require human review.

## Structural Checks
- {STATUS} | structural | {check-id} | {message}
- {STATUS} | structural | {check-id} | {message}
...

## Quality Checks
- {STATUS} | quality | {check-id} | {message}
- {STATUS} | quality | {check-id} | {message}
...

## Recommendations
1. {STATUS} | {check-id} | Target: {file-path}
   Action: {what to do}
2. {STATUS} | {check-id} | Target: {file-path}
   Action: {what to do}
...

Report rules:

• Result = worst severity found: any FAIL → FAIL, else any WARN → WARN, else PASS.
• Structural Checks: one line per Tier 1 check. STATUS is PASS or FAIL.
• Quality Checks: one line per Tier 2 check. STATUS is PASS, WARN, or INFO.
• Recommendations: only include checks that did NOT pass. Each recommendation

includes the check ID, the target file path, and a specific action.

• If all checks pass, the Recommendations section should say: "No issues found."
• Omit passing checks from Recommendations - only list findings that need action.

Batch mode output (when input is --all):

Run Tier 1 structural checks only across all skills. Produce a summary table:

# Batch Validation Summary
Date: {YYYY-MM-DD}
Validator version: 1.0.0
Report schema: v1
Skills checked: {n}

| Skill | Result | Errors | Warnings |
|-------|--------|--------|----------|
| deliver-prd | PASS | 0 | 0 |
| define-hypothesis | WARN | 0 | 1 |
| foundation-persona | FAIL | 1 | 0 |
...

Skills passing: {n}/{total}
Run `utility-pm-skill-validate {skill}` for a detailed report.

Degraded Mode

If you cannot read skill files directly (e.g., running via MCP or in an embedded environment without file system access):

1. Ask the user to provide the content of each required file:

• skills/{name}/SKILL.md
• skills/{name}/references/TEMPLATE.md
• skills/{name}/references/EXAMPLE.md

2. Run all checks against the provided content. 3. Note in the report: "Validated from user-provided content (file system not available)." 4. Batch mode is not available in degraded mode - single skill only.

Output Contract

The validator MUST produce a validation report following the format in Step 5.

The report:

• Uses the exact section headings: ## Summary, ## Structural Checks,

## Quality Checks, ## Recommendations

• Uses pipe-delimited check lines: STATUS | TIER | CHECK-ID | message
• Uses pipe-delimited recommendations: STATUS | CHECK-ID | Target: path

followed by Action: description on the next line

• Includes Report schema: v1 in the header for F-11 compatibility
• Includes the Tier 2 caveat line in the Summary section

Quality Checklist

Before delivering the report, verify:

• [ ] All Tier 1 structural checks were run (not skipped), including the eval-asset checks (eval-trigger-fixtures, eval-output-scenarios, reciprocal-boundary-pointers); absent eval assets are INFO, not FAIL, except a one-directional declared collision pair which FAILs
• [ ] All Tier 2 quality checks were run (not skipped) - single skill mode only
• [ ] Report follows the exact section and line format from Step 5
• [ ] Every non-passing check appears in Recommendations with a target file path
• [ ] Result field reflects the worst severity found
• [ ] Tier 2 findings are capped at WARN (except placeholder-leakage which can FAIL)
• [ ] No Tier 2 check was marked FAIL unless objectively grounded

Examples

See references/EXAMPLE.md for a completed validation report demonstrating both Tier 1 and Tier 2 checks against a real shipped skill.

utility-pm-skill-validate - Version History

1.1.0 (2026-06-15)

Eval-coverage integration (v2.27.0 eval program, plan item C-5 second half): the validator now reports on a skill's eval readiness alongside its structural and quality conventions.

• Three new Tier 1 (deterministic) checks: eval-trigger-fixtures (the B-4 routing-fixture contract: schema, >=16 queries, >=8/class, >=2 near-misses - INFO when absent, since not every skill is in the roster), eval-output-scenarios (the B-7 output-scenario contract: scenario/skill/family frontmatter mapping to an existing rubric - INFO when absent), and reciprocal-boundary-pointers (C-5: a declared COLLISION_PAIRS entry must cross-point in both skills' "When NOT to Use" - FAIL if one-directional).
• `when-not-to-use` upgraded INFO -> WARN: the section became a convention in the v2.26.0 rewrites and now should name the skill's neighbors; the stale "present in 1/27 shipped skills" rationale is removed.
• Each new check names its authoritative deterministic CI gate (check-trigger-fixtures.mjs, check-output-eval-assets.mjs, check-reciprocal-boundary-pointers.mjs) so the interactive report and CI stay consistent.

Report schema is unchanged (the new checks live under ## Structural Checks), so the F-11 parser in utility-pm-skill-iterate is unaffected.

1.0.0 (2026-04-03)

Baseline row for the prior published version; see git history for its changes.

Validation Report: deliver-prd

Date: 2026-04-03 Skill version: 2.0.0 Validator version: 1.0.0 Report schema: v1 Result: WARN

Summary

deliver-prd passes all structural checks. Two quality findings: the output contract uses a generic template reference (acceptable but below v2.8 standard), and the skill lacks a "When NOT to Use" section (informational).

Errors: 0 | Warnings: 1 | Info: 1

Tier 2 findings are heuristic quality assessments and may require human review.

Structural Checks

• PASS | structural | name-match | frontmatter name matches directory
• PASS | structural | description-present | description exists
• PASS | structural | description-length | 31 words (20-100 range)
• PASS | structural | version-present | version "2.0.0"
• PASS | structural | updated-present | updated 2026-01-26
• PASS | structural | license-present | Apache-2.0
• PASS | structural | phase-classification | phase: deliver (domain skill, no classification field)
• PASS | structural | template-exists | references/TEMPLATE.md present
• PASS | structural | template-sections | 10 level-2 headers (≥3 required)
• PASS | structural | example-exists | references/EXAMPLE.md present
• PASS | structural | command-exists | commands/prd.md exists and references correct path
• PASS | structural | agents-entry | AGENTS.md entry found with matching path

Quality Checks

• WARN | quality | output-contract-coverage | Output Format section says "Use the template in references/TEMPLATE.md to structure the output" but does not enumerate template sections explicitly. Accepted as valid - this is the current library convention.
• PASS | quality | checklist-verifiability | all 7 checklist items are specific and testable (e.g., "Success metrics are specific and measurable", "Document is readable in under 15 minutes")
• PASS | quality | example-completeness | EXAMPLE.md fills all 10 template sections with concrete content. No placeholder scaffolding detected. Line count: 212 (informational).
• PASS | quality | template-example-alignment | EXAMPLE.md section headers match TEMPLATE.md in order and naming
• PASS | quality | description-actionability | description includes trigger phrase: "Use when specifying features, epics, or product initiatives for engineering handoff"
• PASS | quality | instruction-clarity | 8 numbered steps with imperative verbs (Summarize, Define, Outline, Detail, Define, Address, Identify, Propose)
• PASS | quality | placeholder-leakage | no unresolved placeholders, HTML comments (except license header), or authoring notes found
• INFO | quality | when-not-to-use | no "When NOT to Use" section present (optional - present in 1/27 shipped skills)

Recommendations

1. WARN | output-contract-coverage | Target: skills/deliver-prd/SKILL.md Action: Consider enumerating the template sections explicitly in the Output Format section (e.g., "The output MUST include: Problem Summary, Goals, Solution Overview..."). Current generic reference is valid but below v2.8 standard. 2. INFO | when-not-to-use | Target: skills/deliver-prd/SKILL.md Action: Consider adding a "When NOT to Use" section to help users pick the right skill. Example: "Not for lightweight feature briefs -> use solution-brief instead."

---

Example Notes

This report demonstrates:

• All 12 Tier 1 structural checks passing (common for shipped skills)
• Tier 2 quality checks surfacing one WARN and one INFO against a well-established skill
• The WARN for output-contract-coverage illustrates the "current conventions vs. v2.8 standard" framing - the skill is valid by current standards, but the validator surfaces the higher standard as a suggestion
• The INFO for when-not-to-use illustrates a finding that is purely optional guidance
• Recommendations include target file paths and specific actions for each finding
• The overall Result is WARN (worst severity found), not FAIL

Validation Report: {skill-name}

Date: {YYYY-MM-DD} Skill version: {version from frontmatter} Validator version: 1.0.0 Report schema: v1 Result: {PASS | WARN | FAIL}

Summary

{1-2 sentence overall assessment of the skill's compliance with structural conventions and quality criteria.}

Errors: {n} | Warnings: {n} | Info: {n}

Tier 2 findings are heuristic quality assessments and may require human review.

Structural Checks

{One line per Tier 1 check. Use pipe-delimited format:}

• {STATUS} | structural | {check-id} | {message}

{Include all checks from the Tier 1 table in SKILL.md Step 3. STATUS is PASS or FAIL.}

Quality Checks

{One line per Tier 2 check. Use pipe-delimited format:}

• {STATUS} | quality | {check-id} | {message}

{Include all checks from the Tier 2 table in SKILL.md Step 4. STATUS is PASS, WARN, or INFO. Tier 2 findings capped at WARN unless objectively grounded (placeholder-leakage can FAIL).}

Recommendations

{Only include checks that did NOT pass. Each recommendation includes the check ID, target file path, and a specific action.}

1. {STATUS} | {check-id} | Target: {file-path} Action: {what to do to resolve this finding}

{If all checks pass, write: "No issues found."}