Verification Before Completion

Verification Before Completion

The Rule

No completion claims without fresh verification evidence. If the verification command has not been run immediately before the claim, the claim cannot be made. Violating the letter of this rule is violating the spirit -- rephrasing a claim to technically avoid the wording does not exempt it from verification.

"Should pass", "probably works", and "looks correct" are not verification. Only command output confirming the claim counts (typically exit code 0). If pre-existing failures cause non-zero exits unrelated to your changes, see "When Verification Fails" below.

Pre-Verification Check

Before running verification, check the working tree state: git status --porcelain. If there are uncommitted changes unrelated to the current task, handle them first (commit, stash, or acknowledge). Adding verification commits on top of a dirty tree creates tangled history. When the change is high-stakes (touches shared modules) or the tree carries unrelated dirty/untracked files that cannot be cleared first, a passing local run is not proof -- unrelated WIP can supply a missing symbol or mask a break. Reproduce the pass against a known-good commit with only the owned diff applied before claiming done; see isolated-verification.md.

When verifying work delegated to a subagent, do not trust the implementer's own report. Read the actual code or test output independently. Spec compliance and quality are separate concerns -- verify both.

Scope Confirmation (Pre-Edit Gate)

When a request uses ambiguous spatial scope -- "migrate my project", "refactor the codebase", "update everywhere", "fix this across the app", "my code/repo/project" -- confirm the concrete scope before any Write or Edit. Imperative phrasing is not the same as defined scope.

Run a breakdown command to surface the real blast radius, then present it for confirmation:

# How many files actually match?
rg -l 'pattern' | cut -d/ -f1 | sort | uniq -c | sort -rn

# Which directories are affected?
rg -l 'pattern' | xargs dirname | sort -u

Present the result and ask: "This touches N files across M subsystems. Scope options: (a) everything, (b) just <subset>, (c) let me pick specific files." Do not start editing until the user commits to one.

This is cheap insurance: the alternative is shipping a sprawling diff that the user then has to unwind. "Migrate my project" is the #1 request shape that produces accidental cross-cutting damage.

When this applies: any request whose scope could plausibly span more than one directory AND where the user has not enumerated files. For a request with explicit file paths, skip this gate.

Gate Function

Before any success claim, run through these five steps:

Verification Strategies by Change Type

Type-check and unit tests are the universal baseline — they are not sufficient proof on their own. Match the strategy to the change:

Reading code is not a strategy. If the table above doesn't have a row for your change, state explicitly: "No runtime verification available — verified by reading the diff."

Adversarial Probes

For any change that touches production logic, include at least one adversarial probe in the verification — not a happy-path confirmation dressed up as verification. Pick the most relevant from:

• Boundary value: 0, -1, empty string, empty array, null, undefined, MAX_INT, 1-char unicode combining mark
• Concurrency: two parallel requests with the same identifier (for state changes, races, double-spend classes)
• Idempotency: run the same mutation twice; the second should either no-op or error cleanly, not corrupt state
• Orphan op: delete/update/get a nonexistent ID — does it 404/return-null as expected, or throw an internal error?

Docs changes, trivial typo fixes, and pure rename refactors are exempt. Everything else: one probe minimum. A report with zero adversarial probes is a happy-path confirmation, not verification.

Review Staleness

Before shipping, check whether prior reviews (agent or human) are still valid. If commits landed after the last review, verify the new changes don't invalidate review conclusions -- check that previously flagged issues are still fixed and no new code contradicts the review's approval. git log --oneline <review-commit>..HEAD shows what changed since the review.

When This Applies

• About to say "tests pass" or "build succeeds"
• About to commit, push, or create a PR
• About to claim a bug is fixed
• About to mark a task as complete
• Moving to the next task in a plan
• After completing each function or component during long sessions
• At periodic checkpoints (~every 15 minutes of active coding)
• Reporting results to the user
• Agent reports success on delegated work
• ANY expression of satisfaction about work state ("looking good", "that should do it")
• ANY positive statement about completion, including paraphrases and synonyms

Red Flags

Fantasy assessment auto-fail. A claim of "zero issues found" on a first implementation pass is a red flag, not a green light. First implementations typically need 2-3 revision cycles. "Perfect on the first try" more likely means incomplete verification than flawless code. Re-verify with a broader scope.

Negative confirmation at signoff. When reporting verification results, include a brief statement of what defect classes were checked and NOT found, not just what passed. "Tests pass, no type errors, no lint warnings, no security flags in the changed files" is stronger than "tests pass" because it proves the scope of verification.

Agent Delegation

When a subagent reports success:

1. Check the VCS diff -- did the agent actually make changes? 2. Run the verification command yourself 3. Report the actual state, not the agent's claim

Never forward an agent's "all tests pass" without running the tests yourself.

Requirements vs Tests

"Tests pass" and "requirements met" are different claims:

1. Re-read the plan or requirements 2. Create a line-by-line checklist 3. Verify each item against the implementation 4. Report gaps or confirm completion

Passing tests prove the code works. They don't prove the right code was written.

Common Claims and Their Proof

When No Verification Command Exists

Some changes have no obvious test command -- documentation, configuration, infrastructure-as-code, skill files. In these cases:

• Documentation/prose -- verify by reading the rendered output. Confirm links work, formatting is correct, content matches intent.
• Configuration/infra -- verify syntax (jq . for JSON, yamllint for YAML, terraform validate, docker build). If no validator exists, read the file and confirm it matches the intended change.
• Non-runnable changes -- verify by diffing (git diff) and confirming the diff matches what was intended. State explicitly: "No automated verification available. Verified by reading the diff."

The principle holds: state what you checked and how, even when a test suite doesn't apply.

When Verification Fails

If the output does not confirm the claim:

1. Do not claim completion. Report the actual failure output to the user. 2. Do not retry the same verification hoping for a different result. If it failed, something is wrong. 3. Return to implementation. Fix the issue, then re-run verification from Step 1 of the Gate Function. 4. If the failure is unrelated to your changes (pre-existing flaky test, environment issue), state this explicitly with evidence -- show that the failure also occurs on the base branch or is a known issue.

Pre-Commit Hook Failures

A failing pre-commit hook is a verification checkpoint, not an obstacle to route around. git commit --no-verify is permitted only when the hook fails on pre-existing or unrelated changes -- a stricter lint landed upstream, a sibling file already violated a rule -- and even then, surface the failure to the user first under the same evidence bar as "When Verification Fails" above (show the failure exists on the base branch). Never pass --no-verify for a failure caused by the current session's own changes: fix the root cause and let the hook pass. A --no-verify the user never saw is a defeated check -- the same failure mode as claiming completion without evidence.

Rationalization Prevention

If you're reasoning about the outcome instead of running the command, the Gate is not satisfied. "Should work", "trivial change", "just a refactor", "new tests pass" (not "all tests pass"), "CI will catch it" -- these are all the same failure mode: substituting confidence for evidence. Any satisfaction expression ("looks good", "seems correct") triggers the Gate, spirit over letter.

Completion Report Format

After verification passes, produce a structured report rather than an open-ended summary. This surfaces scope discipline explicitly and makes the agent's restraint visible to reviewers.

## Completion report

**Status**: DONE / DONE_WITH_CONCERNS / BLOCKED / NEEDS_CONTEXT

**Changes made**
- path/to/file.ts: [one-line description of what changed and why]
- path/to/other.ts: [one-line description]

**Things I didn't touch (intentionally)**
- [thing noticed but out of scope, with one-line reason]
- [adjacent issue deferred, with one-line reason]

**Potential concerns**
- [any risk, uncertainty, or open question the reviewer should know about]
- [or "none"]

**Verification evidence**
- [command]: [exit code / result summary]

Status meanings:

• DONE — task complete, all tests pass, no concerns
• DONE_WITH_CONCERNS — complete but flagging risks; the Potential concerns section is mandatory
• BLOCKED — cannot proceed. Name the blocker and what would unblock it
• NEEDS_CONTEXT — missing information to start or continue. Name what's missing

The Things I didn't touch section is not optional — if nothing was noticed, write "nothing noticed." The goal is to prove the agent considered scope, not to pad the report.

References

• System-Wide Test Check -- blast-radius verification for task completion (callbacks, integration, orphaned state)

Integration

This skill is referenced by:

• /ia-work -- before marking tasks complete, before shipping, and before merge/PR creation (Phase 4)
• ia-receiving-code-review -- verify each fix before marking resolved
• ia-debugging -- before claiming a bug is fixed
• ia-writing-tests -- tests as primary verification evidence
• ia-design-iterator agent -- verify design changes render correctly
• ia-figma-design-sync agent -- verify implementation matches Figma
• /ia-verify command -- runs the full pre-PR verification pipeline

Isolated Verification

A green build or test run in the working tree is not proof the change is sound. Unrelated work-in-progress already present -- uncommitted edits, untracked files, a sibling branch's leftovers -- can supply a missing symbol, satisfy an import, or mask a break that the change alone would expose. The contaminated local pass is not the evidence; a clean pass in isolation is.

When the change is high-stakes (touches shared modules consumed elsewhere) or the tree cannot be made clean first, reproduce the pass against a known-good commit with only the owned diff applied:

# 1. Stage a detached worktree at a known-good base (last green commit)
git worktree add --detach .worktrees/verify <known-good-commit>

# 2. Apply ONLY the diff of the files owned by this change
git diff -- path/to/owned-file path/to/other-owned-file | git apply --directory=.worktrees/verify -

# 3. Build and test there, in isolation
( cd .worktrees/verify && <build-command> && <test-command> )

# 4. Tear down
git worktree remove .worktrees/verify

A clean pass in the isolated tree is the proof. A failure there -- while the local tree stays green -- means surrounding WIP was masking the break: return to implementation, do not claim done. State which base commit and which files were isolated in the verification evidence.

System-Wide Test Check

Before marking a task done, pause and trace the blast radius of the change. This catches integration failures that pass-in-isolation testing misses.

When to skip: Leaf-node changes with no callbacks, no state persistence, no parallel interfaces. Purely additive changes (new helper, new view partial) take 10 seconds to verify "nothing fires."

When this matters most: Changes touching models with callbacks, error handling with fallback/retry, or functionality exposed through multiple interfaces.

ia-verification-before-completion Specification

Intent

ia-verification-before-completion is a discipline-class skill (an engineering practice not tied to one stack). Enforces fresh verification evidence before any completion claim. Use when about to claim "tests pass", "bug fixed", "done", "ready to merge", or handing off work.

Scope

In scope:

• Behaviors described in SKILL.md and routed via the should_trigger phrasings in distillery/tests/fixtures/triggers/ia-verification-before-completion.jsonl.
• Updates to runtime behavior, structure, trigger precision, references, and validation.

Out of scope:

• Acting as the runtime instructions themselves (those live in SKILL.md).
• Trigger phrasings already covered by adjacent ia-* skills (validate-plugin flags >70% description overlap as DUPLICATE_TRIGGER).
• <!-- to fill in: domain-specific exclusions when the skill drifts -->

Trigger Context

• Class: discipline
• Hook regex: plugins/whetstone/hooks/skill-patterns.sh -> SKILL_PATTERNS[ia-verification-before-completion]
• Common requests (from fixture should_trigger):
• "verify before pushing the release branch"
• "verify that tests pass before committing"
• "about to mark this done -- verify first"
• Should not trigger for (from fixture should_not_trigger):
• "write a new middleware for rate limiting"
• "refactor the payment gateway integration"
• "I'm confident the refactor didn't break anything"

Source And Evidence Model

Authoritative sources:

• SKILL.md -- runtime instructions and reference routing.
• references/*.md -- bundled supplementary content (1 file(s)).
• distillery/tests/fixtures/triggers/ia-verification-before-completion.jsonl -- positive and negative trigger phrasings under regression test.
• plugins/whetstone/hooks/skill-patterns.sh -- regex pattern that fires this skill.
• distillery/.eval-data/ia-verification-before-completion/ -- harvested session examples (when present).

Data that must not be stored in this skill or its references:

• Secrets, credentials, tokens.
• Machine-specific filesystem paths (/home/..., /Users/..., ~/ai/...). The validator (MACHINE_PATH_LEAK) flags these as HIGH.
• Private URLs, customer data, or unredacted personal information.

Coverage matrix

Evaluation

Lightweight (run on every change):

python3 distillery/scripts/distiller.py validate-plugin --component ia-verification-before-completion
python3 distillery/scripts/distiller.py test-triggers --skill ia-verification-before-completion

Deeper (when behavior risk warrants):

python3 distillery/scripts/distiller.py dspy-eval ia-verification-before-completion
python3 distillery/scripts/distiller.py diagnose-negatives ia-verification-before-completion

Acceptance gates:

• validate-plugin --component ia-verification-before-completion returns 0 HIGH findings.
• test-triggers --skill ia-verification-before-completion returns F1 = 1.0 with floors of 5 should_trigger and 5 should_not_trigger.
• For dspy-eval, the composite score does not regress against the most recent saved baseline (see distillery/.eval-data/ia-verification-before-completion/history.json).

Known Limitations

<!-- to fill in over time as drift surfaces. Default rule: any time diagnose-negatives surfaces a recurring failure pattern, document it here so future maintainers understand the trade-off the current implementation accepts. -->

Maintenance Notes

• Update SKILL.md when the runtime workflow, branch conditions, or output contract changes.
• Update this SPEC.md when intent, scope, evidence model, evaluation gates, or maintenance expectations change.
• Update the trigger fixture when adding new positive phrasings, removing stale ones, or expanding scope (the 5/5 floor is a hard validator gate).
• Update the hook regex in skill-patterns.sh whenever fixture positives expose a missed phrasing; verify F1 = 1.0 with eval-triggers before committing.
• Run the full release pipeline via /release -- never bump versions or update CHANGELOG.md from a per-skill edit.