Rule Validate

Rule Validate

Use this as stage 3 of the React Doctor rule pipeline.

Pipeline:

1. rule-research defines the rule contract. 2. rule-writing turns the contract into tests and implementation. 3. rule-validate verifies noise, correctness, changesets, PR copy, and review feedback.

Validation is not just running tests. It checks whether the rule still matches the contract on real code.

Interactive Coaching

Before broad or expensive validation, tell the user what will run and what evidence it will produce.

Pause for the user only when:

• RDE or OSS validation is expensive or needs paths/manifests.
• A review comment is ambiguous and could broaden v1 scope.
• A false-positive fix would change the rule contract.
• A check fails for unrelated repo state and the next step is not obvious.

Otherwise, fix real findings and add regression tests.

Local Validation

Run the tightest useful checks first:

• Focused rule tests.
• Typecheck for the touched package.
• Lint or format checks required by the repo.
• Full test/lint/typecheck only when risk or user request justifies it.

Record every command as passed, failed, or not run. If a broad command fails because of unrelated repo state, record the failure location and the focused command that passed.

Implementation Review

Review the diff like a rule reviewer. Lead with bugs:

• False positives.
• False negatives for claimed behavior.
• Scope or binding mistakes.
• Control-flow path merges that create impossible behavior.
• Nested functions analyzed as immediate execution.
• Dynamic computed properties treated as static names.
• Transparent wrappers missed.
• Imported or unknown code reported without support.
• Diagnostic wording that overclaims.
• Missing valid or invalid regression tests.

Before approving a new helper or utility in the diff, confirm it does not duplicate an existing symbol with truffler (the find-similar-functions skill): bunx @rayhanadev/truffler "<helper name>" packages --kind function,method,interface,type,constant --limit 20.

Fix every real implementation bug with a targeted regression test.

RDE Rule Validation

Use RDE after implementation when the rule is broad, heuristic, scope-aware, path-aware, or touches common React idioms.

Run it via the rde-eval skill — a fast local loop (--runner local, uses your working tree) or a cloud fan-out across the corpus (push a branch, diff git:…@main vs git:…@<branch> with --pool vercel). path: is local-only; it never reaches the Vercel pool.

Required handling:

• Scan distinct repos, not just manifest entries.
• Record rootDir scan count separately from repo count.
• Filter output to the target rule before judging results.
• Inspect every hit manually when counts are low.
• Sample hits manually when counts are high.
• Add regression tests for false positives found by evals.

Record:

React Doctor checkout:
RDE eval harness:
Repo manifest:
Distinct repos scanned:
RootDir scans:
Target rule:
Filtered output:
Target diagnostics:
Manually inspected hits:
False positives found:

PR Description

Write PR copy after validation, not before. Use this structure:

````md

Why

Catches <specific issue>.

<Runtime reason in 1-3 sentences.>

Before:

<bad example>

After:

<good example>

What changed

• Added <rule-name>.
• Detects <main detection surface>.
• Reports <exact condition>.
• Allows <important valid patterns>.
• Adds tests for <edge cases>.

Eval results

Test plan

• <focused test command>
• <typecheck command>
• <lint/format command or Not run>

````

Do not include the eval table if RDE was not run; state why it was skipped when useful.

Changeset

Generate a changeset after validation and before PR copy for user-facing changes to published packages.

Required handling:

• Use nr changeset unless the user explicitly asks for a manual file.
• Select the affected published package or packages.
• Use patch for new rules, bug fixes, false-positive fixes, diagnostic wording changes, and test-backed behavior refinements unless the release impact clearly requires otherwise.
• Summarize the user-visible behavior, including the rule name and the runtime problem it catches or avoids.
• Skip the changeset only for private-only, docs-only, test-only, or tooling-only changes, and record why it was skipped.

Review Comment Triage

Classify each bot or human review comment:

• Fix now: real false positive, false negative for claimed behavior, AST mistake, scope/binding bug, or reasonable control-flow bug.
• Usually fix: duplicated helper, misleading name, unnecessary abstraction, or confusing comment.
• Document or defer: false-negative coverage outside v1, path explosion, complex unsupported control flow, or imported file analysis.
• Reject: broadens the rule beyond its message, increases false positives, or conflicts with repo conventions.

Resolve review threads only after the fix or explanation has landed.

Validation Output

Return:

Validation summary:

- <commands and results>
- <implementation review findings>
- <RDE summary or skip reason>
- <false positives found and fixed>
- <regression tests added>
- <changeset path or skip reason>

PR-ready notes:

- <Why/What/Test plan highlights>

Residual risk:

- <known v1 non-goals or unchecked areas>