Docs Guard

Docs Guard

You are reviewing generated or changed documentation before it ships. Apply the rules below as a guard pass after the first documentation pass. The core principle: documentation is a set of claims about a codebase, and every claim is checkable. Your job is to check them.

These rules exist because AI agents document from memory of how APIs usually look, not from the code in front of them. Published research: half of AI answers to programming questions contain incorrect information, and models produce valid invocations for infrequent APIs barely a third of the time — yet the prose sounds authoritative either way. Readers cannot tell verified docs from hallucinated docs. You can, because you have the source.

How to use this skill

Guard-pass mode (recommended): after documentation or docstrings have been generated or edited, verify every claim against the source and run the self-check before delivery.

Live mode (explicit): when the user invokes this skill before writing docs, verify before you write — read the actual implementation, then document what it does. Run the self-check before delivery.

Review mode (the user asks you to review, audit, or fact-check docs): walk references/review-checklist.md against the target docs and produce a findings report with file:line evidence. Do not rewrite in review mode unless asked.

Adapt to the project first

1. Read the project's agent instructions (CLAUDE.md, AGENTS.md) and any docs style guide. Project conventions win on conflict. 2. Identify the docs surfaces that must move together: README, reference docs, docstrings, changelog, examples, config samples. A change to one usually owes a change to others (Rule 8). 3. Note the documented version policy: which versions does the project support, and where are features version-tagged?

The Rules

Accuracy — must fix

1. Every referenced symbol must exist. Every function, method, class, hook, CLI command, flag, endpoint, config key, env var, and file path mentioned in the docs gets verified against the actual source, CLI help output, route table, or schema — by reading it, not recalling it. The verification procedure is in references/verification.md. An unverifiable reference does not ship.

2. Every code sample must work. Imports resolve, APIs exist with the documented signatures (names, argument order, defaults, return shape), and the sample runs outside the author's machine — no hardcoded local paths, no real credentials, no implicit prior state. Sample rules: references/code-samples.md.

3. Document the code's actual behavior, not its intended behavior. Read the implementation before describing it. Where code and comments/specs disagree, the code is the truth — and flag the disagreement to the user instead of silently picking a side.

4. No unverifiable claims. Performance numbers, compatibility matrices, scale limits, and "production-ready" assertions require a source in the repository (benchmark script, CI matrix, changelog entry) or they come out. "Fast" is marketing; "O(n log n), benchmarked in bench/sort.md" is documentation.

Versioning and drift

5. Versions are explicit. Features, flags, and behaviors state the version that introduced them when the project tracks versions. Prerequisites are pinned or ranged, never "latest". Deprecated items say so, with the replacement.

6. A code change owes a docs change. When editing code whose behavior is documented — rename, signature change, new default, removed flag — update every doc surface that mentions it in the same change. Grep the docs for the old symbol before finishing.

Substance — should fix

7. No filler, no slop. Delete: docstrings that paraphrase the signature ("Gets the user by ID" above get_user_by_id), sections that restate their heading, marketing adjectives in technical prose ("powerful", "seamless", "blazingly fast"), and intro padding ("In this section, we will explore…"). A docstring earns its place by adding contracts the signature cannot express: units, ranges, error conditions, side effects, threading/ordering guarantees.

8. Don't paraphrase upstream docs. Link to external documentation instead of restating it — paraphrased upstream docs drift the moment upstream changes. Document only your project's relationship to the external thing (which subset you use, what you configure differently).

9. Examples cover the failure path too. A tutorial that only shows the happy path documents half the API. Show what the error looks like and what the caller should do — using the error types the code actually raises (verify per Rule 1).

Structure — worth noting

10. Navigation tells the truth. Headings describe their sections, the table of contents matches the actual headings, internal links and anchors resolve, and there are no TODO stubs or "coming soon" sections in published docs — unwritten sections are removed, not promised.

Self-check before delivery

1. List every symbol, flag, endpoint, config key, and path your docs mention. Did you verify each one against the source in this session — not from memory? 2. Would every code sample run on a clean machine? Did you check each import and signature? 3. Any number, compatibility claim, or superlative without a repo-verifiable source? 4. If this change touched code: did you grep all docs surfaces for the old names? 5. Any docstring that just restates the signature? Any section that restates its heading? 6. Do all internal links and anchors resolve?

If any answer is wrong, fix it before showing the user.

Reporting format (review mode)

**Rule N violation** in `docs/path.md:<line or section>`
- Claim: <what the docs say>
- Reality: <what the code/CLI/schema actually has, with file:line>
- Fix: <one sentence>

Lead with Rule 1–4 findings (false claims), then drift, then substance. If a doc is clean, say so in one line — accuracy deserves credit.

Severity guide

• Must fix: Rules 1–4 — false documentation is worse than no documentation; readers act on it
• Should fix: Rules 5–9 — drift debt and noise that buries the signal
• Worth noting: Rule 10 — navigation and polish

References

• references/verification.md — the mechanical procedure: extracting claims, verifying symbols, signatures, CLI flags, endpoints, config keys, links
• references/code-samples.md — what makes a sample shippable: runnability, realistic data, secrets hygiene, error paths
• references/docstrings.md — docstring/PHPDoc/JSDoc-specific rules: when one is justified, what it must contain, paraphrase detection
• references/review-checklist.md — structured walk-through for review mode
• references/sources.md — research and style-guide URLs; read only when citing a source

What this skill does not do

• Review the code itself — clean-code-guard's jurisdiction. This skill reviews what the docs claim about the code.
• Generate documentation strategy or information architecture from scratch — it guards accuracy and substance, not scope decisions.
• Enforce a prose style guide — tone belongs to the project; truth belongs to this skill.

Docs Guard — Code Sample Rules

A code sample is the most-trusted part of any documentation: readers paste it. That trust makes a broken sample worse than no sample.

Contents

• Shippable sample checklist
• Realistic data
• Secrets and credentials
• Error-path examples
• Language and environment tags
• Sample drift

Shippable sample checklist

Every sample must pass all of these:

1. Imports resolve — every import/require/use names a real module at the documented version. 2. APIs are real — every call verified per verification.md: name, argument order, defaults, return shape. 3. Self-contained or explicit — runs on a clean machine, or states its prerequisites immediately above the block ("requires the client from the previous step"). 4. No local residue — no /Users/yourname/, no C:\Dev\, no machine-specific ports or hostnames; use placeholder conventions the project already uses. 5. Syntactically valid — parse it mentally line by line; when the runtime allows, actually run or lint it. 6. Output shown is output produced — if the sample shows a result, that result must be what the code yields, not an idealized version.

Realistic data

Use data that exposes the API's shape honestly: realistic field names, plausible values, at least one non-ASCII string where text handling matters. foo/bar hides bugs that "Café Münster" reveals — especially for the i18n-aware audience.

Secrets and credentials

• Placeholders that cannot be mistaken for real values: YOUR_API_KEY, example.com, RFC 5737 IPs (192.0.2.x), sk_test_… style markers only when the provider defines them.
• Never paste real-looking tokens — even invented ones train readers to paste theirs, and secret scanners will flag the repo.
• Auth setup goes in one place and is linked, not repeated with variations in every sample (Rule 8: repetition drifts).

Error-path examples

For any API that can fail in normal operation, show one failure: what the error object/exception/status looks like and the minimal correct response to it — using the error types the code actually raises (verify the raise site). Happy-path-only documentation produces catch-all error swallowing downstream, the exact failure clean-code-guard exists to stop.

Language and environment tags

• Fenced blocks carry the correct language tag — broken highlighting is a trust signal readers notice.
• State the environment when it changes behavior: shell prompts ($ vs #), OS-specific paths, version-gated syntax.

Sample drift

Samples drift faster than prose because nobody re-runs them. On any code change touching a documented API, grep the docs for the old symbol (SKILL.md Rule 6) — samples are where the stale hits hide. Projects with executable-docs tooling (doctest and kin): prefer it; this skill's manual verification is the floor, not the ceiling.

Docs Guard — Docstring, PHPDoc, and JSDoc Rules

In-code documentation has one extra constraint the other surfaces lack: it sits next to the truth. There is no excuse for a docstring that disagrees with the signature three lines below it.

Contents

• When a docstring is justified
• The paraphrase test
• What a good docstring contains
• Tag accuracy (PHPDoc/JSDoc)
• Generated-docs hygiene

When a docstring is justified

• Public API surface: always — it feeds IDEs, generated references, and agents.
• Internal helpers: only when the contract is not expressible in the signature (units, invariants, side effects, "why"). An internal one-liner with an intention-revealing name usually needs nothing — and clean-code-guard's comment rules apply.

The paraphrase test

Delete any docstring whose entire information content is recoverable from the signature:

// Fails the test — restates the obvious, documents nothing.
/**
 * Gets the user by ID.
 *
 * @param int $user_id The user ID.
 * @return WP_User The user.
 */
function get_user_by_id( $user_id ) { /* … */ }

AI generators emit these by the thousand; they are comment pollution wearing a suit. Either say something the signature cannot, or say nothing.

What a good docstring contains

The contract the types cannot express:

• Units and ranges ($timeout in seconds? milliseconds? what happens at 0?)
• Error behavior: which exceptions/returns on failure, and when (verify the raise/return sites)
• Side effects: writes, cache invalidation, events fired, global state touched
• Null/empty semantics: what null means here, what an empty array does
• Ordering, idempotency, concurrency guarantees when callers depend on them
• The "why" for surprising design ("returns 1.0 on API failure so prices never disappear")

Tag accuracy (PHPDoc/JSDoc)

• @param names and order match the signature exactly — drift here actively lies to IDEs.
• @param and @return types match the real types, including nullability (int|WP_Error, ?string) and generics where the project uses them.
• @throws lists what the body actually throws — verify each raise site; remove what no longer throws.
• @since matches the changelog/tag where the project versions its API.
• @deprecated always names the replacement.

Generated-docs hygiene

When docstrings feed a generated reference (phpDocumentor, JSDoc, Sphinx, TypeDoc):

• A wrong docstring becomes a published wrong reference page — Rule 1 severity applies as if it were the README.
• Check that examples inside docstrings obey code-samples.md — they are the least-reviewed samples in any codebase.
• Markup must be valid for the generator in use; broken tags silently truncate published pages.

Docs Guard — Review Checklist

Structured walk for review mode. Findings first, file:line evidence always. Priorities: false claims → drift → substance → navigation.

Contents

• Pass 1: Claim verification
• Pass 2: Code samples
• Pass 3: Drift scan
• Pass 4: Substance
• Pass 5: Navigation
• Reporting

Pass 1: Claim verification (must fix)

Run the full procedure in verification.md:

• Extract every symbol, flag, endpoint, config key, path, version, and behavioral claim.
• Verify each against its source of truth (definition site, parser, route table, changelog).
• Every unverified or contradicted claim is a Rule 1/3 finding with the contradicting file:line.
• Numbers and superlatives without a repo source are Rule 4 findings.

Pass 2: Code samples (must fix)

For every fenced block, run the code-samples.md checklist: imports, signatures, self-containment, local residue, secrets, shown output. Samples in docstrings count.

Pass 3: Drift scan (should fix)

• Take the public API surface (or the diff, when reviewing a change) and grep the docs for renamed/removed symbols, old defaults, and dead flags.
• Check the changelog mentions what the docs claim is new, and versions agree (Rule 5).
• Cross-surface consistency: README vs reference vs docstrings vs config samples — one claim per fact, surfaces agreeing (Rule 6).

Pass 4: Substance (should fix)

• Paraphrase docstrings, heading-restating sections, marketing adjectives, intro padding (Rule 7).
• Paraphrased upstream documentation that should be a link (Rule 8).
• Happy-path-only tutorials and API examples (Rule 9).

Pass 5: Navigation (worth noting)

• TOC vs actual headings; internal links and anchors resolve; no TODO stubs or "coming soon" in published docs (Rule 10).

Reporting

Use the SKILL.md reporting format (Claim / Reality / Fix). Lead with the count: "N claims checked, M false, K unverifiable." End with a verdict — publish / fix first / do not publish — and at most three things the docs do well. A review that verifies 40 claims and finds 2 false is a good result; say so.

Docs Guard — Sources

Central bibliography. Operational guidance lives in the other references; read this file only when a source URL is needed.

Contents

• Research on AI-generated documentation defects
• Documentation craft references

Research on AI-generated documentation defects

• Kabir et al., "Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions", CHI 2024 — 52% of answers contained incorrect information; evaluators missed AI errors 39% of the time: https://dl.acm.org/doi/10.1145/3613904.3642596
• CloudAPIBench, "On Mitigating Code LLM Hallucinations with API Documentation" (AWS, 2024) — 38.58% valid invocations for low-frequency APIs: https://arxiv.org/abs/2407.09726
• De-Hallucinator (2024) — LLMs routinely invent non-existent APIs; project-API grounding fixed 63.2% of hallucination-caused failures: https://arxiv.org/abs/2401.01701
• Spracklen et al., "We Have a Package for You!" (USENIX Security 2025) — package hallucination averages 19.6% across 16 models: https://www.usenix.org/conference/usenixsecurity25/presentation/spracklen

Documentation craft references

• Diátaxis — the four documentation modes (tutorial, how-to, reference, explanation): https://diataxis.fr/
• Write the Docs — style guide collection: https://www.writethedocs.org/guide/writing/style-guides/
• RFC 5737 — IPv4 address blocks reserved for documentation: https://www.rfc-editor.org/rfc/rfc5737
• Google developer documentation style guide: https://developers.google.com/style

Docs Guard — Verification Procedure

The mechanical heart of this skill: turn a document into a list of claims, then check each claim against the source of truth.

Contents

• Step 1: Extract the claims
• Step 2: Verify each claim type
• Step 3: Record what you verified
• When you cannot verify

Step 1: Extract the claims

Scan the doc and list every:

• Function, method, class, constant, hook, event name
• CLI command, subcommand, flag, default value
• HTTP endpoint, method, status code, request/response field
• Config key, env var, file path, directory layout claim
• Version number, compatibility statement, dependency requirement
• Behavioral claim ("retries three times", "case-insensitive", "idempotent")

Inline code spans and code blocks are claim-dense; prose hides claims in verbs ("automatically reconnects" is a claim).

Step 2: Verify each claim type

Step 3: Record what you verified

In write-time mode, keep a short verification trail in your working notes (not in the doc): claim → file:line where confirmed. In review mode, this trail becomes your evidence — every finding cites the definition site that contradicts the doc.

When the runtime allows execution, prefer executable checks: run --help, run the sample, run a link checker. When it does not, source-reading is the standard — never skip to "it looks right."

When you cannot verify

If the source of truth is unavailable (private dependency, external service, missing schema):

1. Say so explicitly rather than guessing. 2. Downgrade the claim to what you can verify ("the client calls the /v2/orders endpoint" → verified in client code, even if the server is unreachable). 3. Never decorate an unverified claim with confident language. "Should", "appears to", or a direct question to the user beats a fluent hallucination.