Ia Document Review

Document Review

Improve brainstorm or plan documents through structured review.

Step 1: Get the Document

If a document path is provided: Read it, then proceed to Step 2.

If no document is specified: Ask which document to review, or look for the most recent brainstorm/plan in docs/brainstorms/ or docs/plans/.

Step 2: Assess

Read through the document and ask:

• What is unclear?
• What is unnecessary?
• What decision is being avoided?
• What assumptions are unstated?
• Where could scope accidentally expand?
• Is this technically feasible with the current architecture?
• Are there security implications in what's proposed?

These questions surface issues. Don't fix yet--just note what you find.

Step 3: Activate Review Lenses

Based on the document's content, activate specialized review perspectives. Scan for signals and apply matching lenses:

Activate a lens when ANY of its signals match. Most documents trigger 1-2 lenses; brainstorm notes may trigger none. When a lens is active, weave its checks into the assessment and evaluation steps rather than running it as a separate pass.

Step 4: Evaluate

Score the document against these criteria:

If invoked within a workflow (after /ia-brainstorm or /ia-plan), also check:

• User intent fidelity -- Document reflects what was discussed, assumptions validated

Step 5: Identify the Critical Improvement

Among everything found in Steps 2-4, does one issue stand out? If something would significantly improve the document's quality, this is the "must address" item. Highlight it prominently.

Step 6: Make Changes

Present your findings, then:

1. Auto-fix minor issues (vague language, formatting) without asking 2. Ask approval before substantive changes (restructuring, removing sections, changing meaning) 3. Update the document inline--no separate files, no metadata sections

Simplification Guidance

Simplification is purposeful removal of unnecessary complexity, not shortening for its own sake.

Simplify when:

• Content serves hypothetical future needs, not current ones
• Sections repeat information already covered elsewhere
• Detail exceeds what's needed to take the next step
• Abstractions or structure add overhead without clarity

Don't simplify:

• Constraints or edge cases that affect implementation
• Rationale that explains why alternatives were rejected
• Open questions that need resolution

Step 7: Reader Test (Optional)

For standalone documents that must be self-contained (onboarding guides, ADRs, external-facing docs), dispatch a zero-context sub-agent to simulate a first-time reader. The sub-agent has no conversation history — it sees only what a future reader would see.

How to run the test:

1. Predict 5-10 reader questions from the document's stated goals — one per major section or decision. Mix three kinds:

• Concrete retrieval: "What command sets up the dev environment?"
• Decision rationale: "Why did we pick X over Y?"
• Ambiguity probe: "Could a reader interpret <specific phrase> in more than one way?"

2. Dispatch a fresh sub-agent with the document attached and the questions. No prior context, no session history. 3. Compare the sub-agent's answers against author intent. Also ask the sub-agent directly: "What feels ambiguous? What prior knowledge does this assume? Are there internal contradictions?"

Interpret results:

• Correct, confident answers → document is self-contained for that question.
• Wrong answer with high confidence → document actively misleads. Highest-priority fix.
• Hedged or "insufficient information" → the document has a gap the author didn't notice. Fill it.
• Sub-agent flags ambiguity the author didn't intend → reword for precision.

Skip for context-dependent docs (brainstorm notes, plan files, internal working docs) where the reader will always have prior context. The sub-agent test only adds value when the real reader has no other channel.

Step 8: Offer Next Action

After changes are complete, ask:

1. Refine again - Another review pass 2. Review complete - Document is ready

Iteration Guidance

After 2 refinement passes, recommend completion--diminishing returns are likely. If the user wants to continue, allow up to 4 passes total. After 4, stop and report "review converged -- further changes require new direction." Do not continue past 4 even on user request without a fresh framing.

Return control to the caller (workflow or user) after selection.

Constraints

• Fix targeted sections, don't rewrite the whole document. If the structure is fundamentally broken, surface the structural problem and ask for permission to restructure.
• Flag missing sections in your review, but don't add them. The user decides what to include.
• Keep changes minimal. If a paragraph needs tightening, tighten it. Don't expand scope.
• Review inline. No separate review files or metadata sections.

Success Criteria

• Document read and scored on all four quality criteria
• Relevant review lenses activated and checks applied
• Critical improvements identified with specific suggestions
• User presented with clear next-action choice (refine or complete)
• Revised document saved if changes were approved

ia-document-review Specification

Intent

ia-document-review is a workflow-class skill (a multi-step process producing concrete artifacts). Structural review of documents for gaps, clarity, completeness, and organization. Use when a brainstorm, plan, spec, ADR, or any doc needs polish before the next workflow step. For exploring new ideas from scratch, use brainstorming instead.

Scope

In scope:

• Behaviors described in SKILL.md and routed via the should_trigger phrasings in distillery/tests/fixtures/triggers/ia-document-review.jsonl.
• Updates to runtime behavior, structure, trigger precision, references, and validation.

Out of scope:

• Acting as the runtime instructions themselves (those live in SKILL.md).
• Trigger phrasings already covered by adjacent ia-* skills (validate-plugin flags >70% description overlap as DUPLICATE_TRIGGER).
• <!-- to fill in: domain-specific exclusions when the skill drifts -->

Trigger Context

• Class: workflow
• Hook regex: plugins/whetstone/hooks/skill-patterns.sh -> SKILL_PATTERNS[ia-document-review]
• Common requests (from fixture should_trigger):
• "review the brainstorm document for completeness"
• "refine the plan before we start implementation"
• "review my plan doc for gaps before I implement"
• Should not trigger for (from fixture should_not_trigger):
• "add error handling to the file upload endpoint"
• "set up monitoring alerts for the production database"
• "review this pull request for code quality"

Source And Evidence Model

Authoritative sources:

• SKILL.md -- runtime instructions and reference routing.
• references/*.md -- bundled supplementary content (0 file(s)).
• distillery/tests/fixtures/triggers/ia-document-review.jsonl -- positive and negative trigger phrasings under regression test.
• plugins/whetstone/hooks/skill-patterns.sh -- regex pattern that fires this skill.
• distillery/.eval-data/ia-document-review/ -- harvested session examples (when present).

Data that must not be stored in this skill or its references:

• Secrets, credentials, tokens.
• Machine-specific filesystem paths (/home/..., /Users/..., ~/ai/...). The validator (MACHINE_PATH_LEAK) flags these as HIGH.
• Private URLs, customer data, or unredacted personal information.

Coverage matrix

Evaluation

Lightweight (run on every change):

python3 distillery/scripts/distiller.py validate-plugin --component ia-document-review
python3 distillery/scripts/distiller.py test-triggers --skill ia-document-review

Deeper (when behavior risk warrants):

python3 distillery/scripts/distiller.py dspy-eval ia-document-review
python3 distillery/scripts/distiller.py diagnose-negatives ia-document-review

Acceptance gates:

• validate-plugin --component ia-document-review returns 0 HIGH findings.
• test-triggers --skill ia-document-review returns F1 = 1.0 with floors of 5 should_trigger and 5 should_not_trigger.
• For dspy-eval, the composite score does not regress against the most recent saved baseline (see distillery/.eval-data/ia-document-review/history.json).

Known Limitations

<!-- to fill in over time as drift surfaces. Default rule: any time diagnose-negatives surfaces a recurring failure pattern, document it here so future maintainers understand the trade-off the current implementation accepts. -->

Maintenance Notes

• Update SKILL.md when the runtime workflow, branch conditions, or output contract changes.
• Update this SPEC.md when intent, scope, evidence model, evaluation gates, or maintenance expectations change.
• Update the trigger fixture when adding new positive phrasings, removing stale ones, or expanding scope (the 5/5 floor is a hard validator gate).
• Update the hook regex in skill-patterns.sh whenever fixture positives expose a missed phrasing; verify F1 = 1.0 with eval-triggers before committing.
• Run the full release pipeline via /release -- never bump versions or update CHANGELOG.md from a per-skill edit.