Autoresearch

Autoresearch — Autonomous Goal-directed Iteration

Safety Invariants (all subcommands)

• Never push, publish, or deploy without explicit user approval.
• Bounded by default. Override with Iterations: unlimited.
• All results logged to autoresearch/{subcommand}-{YYMMDD}-{HHMM}/ directory.
• Chain handoff via handoff.json. Evals reads *-results.tsv.

Dispatch (bare $autoresearch)

Parse the invocation in this order:

Print a banner on every invocation: [autoresearch] mode: classic | orchestrator | wizard.

Subcommands

Universal Flags

Orchestrator

Activated when a plain-language goal is given without Metric:/Verify:. Classifies the goal into a Goal archetype — see references/orchestrator-routing.md for the archetype table and router decision table.

Two modes based on archetype:

• Orchestration loop — predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore). Goal has a mechanical Success predicate; the loop runs until that predicate is met.
• Single-pass dispatch — subjective/terminal archetypes (document, what-to-build, decide-design). Routes once to the fitting subcommand (learn / improve / reason), lets it self-terminate, then reports. No loop, no Plateau, no ship gate.

Orchestration Loop Steps

Backed by scripts/orchestrate.sh (deterministic seam — all routing logic lives there). Subcommands exposed: classify, next-hop, units, plateau, screen-cmd, verdict, validate-state, screen-state-predicate.

1. Classify — scripts/orchestrate.sh classify "<goal>" → archetype label + mode. 2. Derive predicate — reuse plan logic to produce a concrete Success predicate: exact shell command + expected output. For optimize-metric, run the full plan/wizard derivation internally. 3. Confirm — ONE request_user_input showing: archetype, mode, concrete predicate (command + expected output), terminal choice (stop-at-verified vs proceed-to-ship). Misclassifications are caught here, not mid-run. 4. Round-0 dry-run — prove the predicate command runs and returns a value; safety-screen every derived command via screen-cmd; print projected cycle budget. Stop here if --dry-run. 5. Loop until predicate satisfied: a. Assess state via cheap signals (last handoff.json, regression verdict, error count) + affected-test verify. b. scripts/orchestrate.sh next-hop orchestrator-state.json → next subcommand. c. Run subcommand (its own bounded inner loop). d. Record per-hop outcome ∈ {progressed, no-op, failed, blocked}. e. Fold hop's handoff.json into orchestrator-state.json. f. scripts/orchestrate.sh units → recompute Units remaining. 6. Stop conditions (checked after each hop):

• Predicate met → ship gate (only if ship is in the pipeline) else CONVERGED.
• scripts/orchestrate.sh plateau orchestrator-state.json → true → stop + report PLATEAU.
• Cycles > ceiling (default 50, override --max-cycles N) → stop + report CEILING.
• Hop outcome blocked/failed with no alternative route → checkpoint + stop + report BLOCKED.

Orchestrator State

orchestrator-state.json — orchestrator-owned, additive. Tracks: goal, archetype, predicate, terminal-choice, units_remaining history, cycle count, per-hop pipeline log with outcomes, current incumbent. Each hop's handoff.json is unchanged (single-hop bridge); the orchestrator reads it and folds it in. Two clearly-owned state objects, no overlap.

Orchestrator Safety Invariants

• Never auto-approve ship/deploy/push. The orchestrator never passes --auto to ship; deploy always requires explicit user approval.
• Data-migration behind anchored DB-URL allowlist. Reuses regression's allowlist — host must be localhost/127.0.0.1/container hostname, or database name carries _test/_ci suffix. Bare substring match does not qualify. Anything else refused.
• screen-cmd on every derived command — run before the loop starts AND on every command read from a persisted state file on resume. Persisted commands are never trusted; resume re-screens the pinned predicate via screen-state-predicate and refuses on refuse.
• No un-screened commands mid-loop. The autonomous loop cannot introduce new shell commands that bypass screen-cmd.
• Predicate pinned, not re-derived. Round-0 writes the derived Success predicate verbatim into orchestrator-state.json; every cycle and every resume reuses that exact string so "done" is reproducible across runs.
• Validate the ledger before routing. validate-state gates orchestrator-state.json (required fields + coarse types); a malformed ledger is not trusted to route from.
• Independent verify before convergence. High-impact changes accepted on the working signal set pending_verify; next-hop routes to a verify hop (held-out / adversarial check) before DONE or ship. The verify hop never auto-approves ship.
• Unknown-units cycles excluded from Plateau counter. A cycle where units returns unknown (e.g. runner crash) is not counted as zero-progress; repeated unknown routes to BLOCKED.

EXECUTE IMMEDIATELY — do not deliberate before reading this protocol.

Parse Arguments

Extract from $ARGUMENTS:

• Goal: — what to improve
• Scope: or --scope — file globs
• Metric: — what to measure
• Direction: — higher_is_better (default) or lower_is_better
• Verify: — shell command that outputs a number
• Guard: — optional safety command (must always pass)
• Iterations: or --iterations — integer N for bounded mode (default: 25). "unlimited" for unbounded.
• --evals — enable mid-loop checkpoints
• --evals-interval N — checkpoint frequency override
• --chain <targets> — comma-separated downstream commands

Setup (if required context missing)

If Goal, Scope, Metric, or Verify missing → use request_user_input (single batched call): Q1 (Goal): "What do you want to improve?" Q2 (Scope): "Which files?" — suggest globs from project Q3 (Metric+Verify): "How to measure? Provide a shell command that outputs a number" Q4 (Guard): "Safety command that must always pass?" — options: test cmd, build cmd, skip If ALL provided inline → skip setup, proceed directly.

Precondition Checks

1. Verify git repo exists (git rev-parse --git-dir) 2. Check clean working tree (git status --porcelain) — warn if dirty 3. Check for stale lock files, detached HEAD 4. If Guard set → run Guard to establish guard baseline 5. Fail fast on any critical issue. Warn on non-critical.

Verify Safety Screen

Before first dry-run, screen Verify command for: rm -rf, fork bombs, curl|sh, embedded credentials, outbound writes. Block dangerous commands.

Establish Baseline (Iteration 0)

1. Run Verify command → extract numeric metric 2. Record as iteration 0 in TSV: 0\t{timestamp}\t{commit}\t{metric}\t0.0\t{guard}\t-\tbaseline\tinitial state 3. Create output directory: autoresearch/loop-{YYMMDD}-{HHMM}/ 4. Write TSV header: # metric_direction: {direction}\niteration\ttimestamp\tcommit\tmetric\tdelta\tguard\tguard-metric\tstatus\tdescription

Iteration Loop

For each iteration (1 to max_iterations, or unbounded):

Phase 1: Review (read git history as memory)

• Read last 10-20 lines of results TSV
• Run git log --oneline -20 — see what worked/failed
• If last iteration was "keep" → run git diff HEAD~1 to see what improved metric
• Identify: what worked, what failed, what's untried

Phase 2: Modify

• Based on review, make ONE focused change to improve the metric
• Change must be atomic — one logical unit of work

Phase 3: Commit

• Stage and commit with experiment: {description} prefix
• Record commit SHA

Phase 4: Verify

• Run Verify command → extract new metric value
• Calculate delta from previous iteration
• Metric improved (correct direction) → candidate for keep

Phase 5: Guard (if configured)

• Run Guard command. If fails → revert regardless of metric improvement

Phase 6: Decide

• keep — metric improved, guard passed → commit stays
• discard — metric worsened → git revert HEAD --no-edit
• crash — verify/guard command failed → git revert HEAD --no-edit
• no-op — no change made this iteration
• hook-blocked — git hook blocked the commit
• metric-error — verify output not a valid number → git revert HEAD --no-edit

Phase 7: Log

Append row to TSV: iteration, timestamp, commit/-, metric, delta, guard status, guard-metric, status, description

Eval Checkpoint

If --evals: check if current_iteration % interval == 0 → run checkpoint analysis.

Bounded Check

If bounded: current_iteration >= max_iterations → exit loop, print summary.

Summary (after loop ends)

Print: total iterations, kept/discarded counts, starting metric → final metric, improvement %, top 3 most effective changes.

Eval Checkpoint (--evals flag)

If --evals present:

• Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded. Override: --evals-interval N.
• Every {interval} iterations, pause and analyze current results TSV.
• Print: --- Eval Checkpoint (iterations {X}-{Y}) ---\nMetric: {start} → {end} ({delta}) | Kept: {n}/{total} | Trend: {up/flat/down}\n{one-line recommendation}\n---
• If plateau 3+ checkpoints → recommend early stop.
• At loop end → full evals summary to evals-summary.md in output directory.

Chain Handoff

After completion, write handoff.json to output directory: version "2.1.0", source "loop", timestamp, status (COMPLETE|USER_INTERRUPT|BOUNDED|ERROR), results_tsv path, findings[], config{goal, scope, metric, direction, verify}. Invoke next target in --chain order. Propagate --evals flag.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Scope: or --scope — file globs to investigate
• Symptom: or --symptom — error message or behavior description
• Iterations: or --iterations — default 15. "unlimited" for unbounded.
• --fix — shorthand for --chain fix
• --severity — filter: critical, high, medium, low
• --technique — force specific technique
• --evals, --evals-interval N, --chain

Setup (if required context missing)

If Scope and Symptom both missing: 1. Auto-scan: run tests, lint, typecheck to detect existing failures 2. request_user_input (single batch): Q1 (Issue): "What's the problem?" — hunt all bugs, specific error, failing tests, CI failure, performance Q2 (Scope): "Which files?" — suggested globs + entire codebase Q3 (Depth): "How deep?" — quick (5), standard (15), deep (30+), unlimited Q4 (After): "When bugs found?" — report only, find and fix (--chain fix), chain to other, ask each time If all provided → skip.

Investigation Techniques

Establish Baseline (before loop)

1. Auto-scan for failures if no symptom provided 2. Create output directory: autoresearch/debug-{YYMMDD}-{HHMM}/ 3. TSV header: # metric_direction: higher_is_better\niteration\ttimestamp\thypothesis\tstatus\ttechnique\tevidence\tfile_line 4. Metric = cumulative confirmed findings count

Iteration Loop

Phase 1: Review Context

• Read results TSV (past findings)
• Assess: what's been tested, what vectors remain
• If no hypotheses left → early stop

Phase 2: Hypothesize

• Form ONE specific, falsifiable hypothesis
• Format: "I hypothesize that {X} because {evidence}. Test by {Y}."
• Hypothesis must be testable and different from all previous

Phase 3: Investigate

• Apply appropriate technique for this hypothesis
• Read relevant code, run targeted tests, check logs
• Collect evidence (file:line references required)

Phase 4: Classify

• confirmed — hypothesis correct, bug found with evidence
• disproven — hypothesis wrong, evidence against it
• inconclusive — can't prove or disprove, needs different approach

Phase 5: Log

Append to TSV: iteration, timestamp, hypothesis, status, technique, evidence, file_line

Eval Checkpoint

If --evals: check if current_iteration % interval == 0 → run checkpoint analysis.

Bounded Check

If bounded: current_iteration >= max_iterations → exit loop, print summary.

Summary

Print: total hypotheses tested, confirmed/disproven/inconclusive counts, all confirmed bugs with severity and file:line.

Eval Checkpoint (--evals flag)

If --evals present:

• Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded. Override: --evals-interval N.
• Every {interval} iterations, pause and analyze current results TSV.
• Print: --- Eval Checkpoint (iterations {X}-{Y}) ---\nFindings: {confirmed} confirmed | Trend: {up/flat/down}\n{one-line recommendation}\n---
• If plateau 3+ checkpoints (no new confirmed) → recommend early stop.
• At loop end → full evals summary to evals-summary.md in output directory.

Chain Handoff

After completion, write handoff.json to output directory: version "2.1.0", source "debug", timestamp, status (COMPLETE|USER_INTERRUPT|BOUNDED|ERROR), results_tsv path, findings = confirmed bugs with severity + file:line, config{scope, symptom}. If --fix flag → chain to fix automatically. Invoke next target in --chain order. Propagate --evals flag.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Positional path to a specific TSV file
• --format — output format: text (default console), json, md (markdown file)
• --compare <path> — (v2.2.0 placeholder, not yet implemented)

Input Discovery

1. If path provided → use that TSV directly 2. If no path → scan current directory + autoresearch/*/ for *-results.tsv files 3. If multiple found → request_user_input: "Which results to analyze?" — list found files 4. If none found → request_user_input: "Provide path to results TSV" 5. Also scan project root for v2.0.03 legacy TSV files (backward compat)

Parse TSV

1. Read line 1: extract # metric_direction: higher_is_better|lower_is_better comment

• If missing → infer from column names (metric/error_count → guess, or ask user)

2. Read line 2: header row → detect available columns 3. Read remaining lines: data rows 4. Handle missing timestamp column gracefully (v2.0.03 compat)

Column Detection & Analysis

Activate analysis based on columns present in header:

Unknown columns: report presence but skip analysis. Forward-compatible with future subcommands.

Report Structure

## Evals Summary — {subcommand} ({N} iterations)

### Key Metrics
- Total iterations: N | Kept: X | Reverted: Y | Revert rate: Z%
- Starting metric: A | Final metric: B | Improvement: C%

### Trend Analysis
- Metric progression: [description of trajectory]
- Plateau detected at iteration N (metric stable for M iterations)
- Biggest win: iteration X (+delta, description)
- Biggest loss: iteration Y (-delta, description)
- Diminishing returns: [after iteration N, average delta dropped below threshold]

### Patterns
- What types of changes succeeded: [extracted from descriptions of kept iterations]
- What types of changes failed: [extracted from descriptions of discarded iterations]
- File hotspots: [files changed most in kept iterations, if commit data available]
- Technique effectiveness: [ranked by confirmation rate, if technique column present]

### Recommendation
- [continue / stop / change strategy — based on trend, plateau, revert rate]
- [specific actionable suggestion based on pattern analysis]

Output

• Console: structured report (30-50 lines)
• If --format md → write evals-summary.md in same directory as input TSV
• If --format json → write evals-summary.json with structured data

Mid-Loop Checkpoint Protocol (for --evals flag in other commands)

This section documents the checkpoint protocol that looping commands embed:

• Adaptive interval: floor(max_iterations / 3), minimum 1. Fixed 10 for unbounded. Override: --evals-interval N.
• Checkpoint format (5 lines max):

  --- Eval Checkpoint (iterations {X}-{Y}) ---
  Metric: {start} → {end} ({delta}) | Kept: {n}/{total} | Trend: {up/flat/down}
  {one-line recommendation}
  ---

• Early stop recommendation: if plateau detected for 3+ consecutive checkpoints
• Final summary: at loop end, produce full evals report to console + evals-summary.md

Adaptive Interval Examples

Backward Compatibility

• v2.0.03 TSV files: column names preserved, timestamp absence handled gracefully
• Fuzzy column matching: metric_value → metric, error_count → metric
• Files in project root (not autoresearch/ subdirectory) → discovered during scan
• v2.0.03 status values all supported: baseline, keep, keep (reworked), discard, crash, no-op, hook-blocked, metric-error

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Target: or --target — command that shows errors (e.g., npm test, tsc --noEmit)
• Scope: or --scope — file globs to modify
• Guard: or --guard — safety command (must always pass)
• Iterations: or --iterations — default 20. "unlimited" for unbounded.
• --from-debug — read handoff.json from previous debug run
• --category — filter: test, type, lint, build
• --evals, --evals-interval N, --chain

Setup (if required context missing)

If Target and Scope both missing: 1. Auto-detect failures: run test suite, type checker, linter, build 2. Present results via request_user_input (single batched call): Q1 (Fix What): "Found [N] test failures, [M] type errors, [K] lint errors. Fix what?" — everything, only tests, only types, only lint Q2 (Guard): "Safety command that must always pass?" — npm test, tsc, npm run build, skip Q3 (Scope): "Which files can I modify?" — suggested globs from error locations + all Q4 (Launch): "Ready?" — fix until zero, fix with limit, cancel If all provided → skip setup. If --from-debug → read handoff.json for scope and findings.

Precondition Checks

Verify: git repo exists, clean working tree, no lock files, no detached HEAD. Fail fast on critical issues.

Establish Baseline (Iteration 0)

1. Run Target command → count errors (metric = error count, direction = lower_is_better) 2. Record baseline in TSV 3. Create output directory: autoresearch/fix-{YYMMDD}-{HHMM}/ 4. TSV header: # metric_direction: lower_is_better\niteration\ttimestamp\terror_type\terror_fixed\tcommit\tmetric\tdelta\tguard\tstatus\tdescription

Iteration Loop (until zero errors or max_iterations)

Phase 1: Review

• Read results TSV + git log
• Run Target to get current error list
• If error count == 0 → exit loop (SUCCESS)

Phase 2: Prioritize

Order: crash/fatal → test failures → type errors → lint → warnings. Within category: easiest first (single-file fixes before cross-file).

Phase 3: Fix ONE Thing

• Pick the highest-priority error
• Make ONE focused fix (atomic — addresses exactly one error)
• Record error type and which error was fixed

Phase 4: Commit

• Stage and commit: experiment: fix {error_type} — {description}

Phase 5: Verify

• Run Target → count errors → compute delta
• Expected: error count decreased by 1 or more

Phase 6: Guard

• If Guard set → run Guard. If fails → revert.

Phase 7: Decide

• keep — error count decreased AND guard passes
• keep (reworked) — fix needed adjustment, second attempt worked
• discard — error count same/increased → git revert HEAD --no-edit
• crash — target/guard command failed → revert
• hook-blocked — git hook blocked the commit
• metric-error — target output not parseable → revert

Phase 8: Log

Append row: iteration, timestamp, error_type, error_fixed, commit/-, metric (error count), delta, guard, status, description

Eval Checkpoint

If --evals: check if current_iteration % interval == 0 → run checkpoint analysis.

Bounded Check

If bounded: current_iteration >= max_iterations → exit loop, print summary.

Summary

Print: total errors fixed, remaining errors, error types distribution, fix success rate.

Eval Checkpoint (--evals flag)

If --evals present:

• Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded. Override: --evals-interval N.
• Every {interval} iterations, pause and analyze current results TSV.
• Print: --- Eval Checkpoint (iterations {X}-{Y}) ---\nErrors: {start} → {end} ({delta}) | Kept: {n}/{total} | Trend: {up/flat/down}\n{one-line recommendation}\n---
• If plateau 3+ checkpoints → recommend early stop.
• At loop end → full evals summary to evals-summary.md in output directory.

Chain Handoff

After completion, write handoff.json to output directory: version "2.1.0", source "fix", timestamp, status (COMPLETE|USER_INTERRUPT|BOUNDED|ERROR), results_tsv path, findings = unfixed errors, config{target, scope, guard}. Invoke next target in --chain order. Propagate --evals flag.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Goal: — product area to improve (or full $ARGUMENTS if no keyword)
• --icp or ICP: — ideal customer profile description
• --discover — force inline codebase scan even when context exists
• --no-discover — skip auto-discover, warn instead
• --seeds <categories> — override default research category seeds
• --depth — shallow (5 iterations), standard (15), deep (30)
• --features — comma-separated feature names to pre-select for PRD generation
• Iterations: or --iterations — default 15. "unlimited" for unbounded.
• --evals, --evals-interval N

If upstream handoff.json exists in CWD → read it. Map source findings to default seed categories:

• probe → ICP challenges, UX & experience
• predict → Competitor gaps, Revenue & growth
• debug/security → Competitor gaps, ICP challenges
• Override with --seeds.

Setup (if Goal or ICP missing)

request_user_input (single batch): Q1 (Goal): "What product area to improve?" — open text Q2 (ICP): "Who is your ideal customer?" — open text describing target buyer/user Q3 (Pain points): "Top 3 pain points your customers face?" — open text Q4 (Competitors): "Key competitors?" — open text, or "skip" Q5 (Depth): "How deep?" — shallow (5 iterations, quick scan), standard (15, recommended), deep (30+, exhaustive) If all provided inline → skip.

Phase 1: Product Context

Resolve product context (priority chain): 1. Learn summary (autoresearch/learn-*/summary.md, most recent) → read it 2. README.md (≥500 chars, non-boilerplate) → extract product description 3. package.json / pyproject.toml / Cargo.toml description (≥10 chars) → use it 4. If ALL above absent AND NOT --no-discover → auto-discover: scan 10 key files (manifest, routes, models, config), cap 1500 tokens 5. If --discover → force scan regardless of above 6. If nothing found → warn: "No product context. Run $autoresearch learn --mode summarize for better results."

Phase 2: Research Loop

Create output directory: autoresearch/improve-{YYMMDD}-{HHMM}/ TSV header: # metric_direction: higher_is_better Columns: iteration|timestamp|category|research_question|status|source|insight_problem|insight_mechanism|confidence|classification

5 research categories: 1. ICP challenges — pain points, jobs-to-be-done, unmet needs 2. Competitor gaps — weaknesses, missing features, technical differentiators 3. Market trends — timing signals, emerging patterns, regulatory shifts 4. UX & experience — interaction models, onboarding, retention mechanics 5. Revenue & growth — pricing, acquisition, monetization, upsell/expansion

Iteration protocol:

• Reserve first 5 iterations: one per category (forced breadth)
• Remaining iterations: target categories with richest signal
• Per iteration: form research question → WebSearch → synthesize → normalize to canonical insight schema → classify (new/extension/duplicate) → tag confidence (HIGH: 3+ sources, MEDIUM: 2, LOW: 1) → cross-check against codebase → log
• Saturation: net-new insights < 2 for 3 consecutive non-reserved iterations → SATURATED, exit loop
• Hard ceiling (Iterations flag) as infinite-loop guard

Insight schema: {problem: 10-word canonical form, affected_persona: ICP segment, proposed_mechanism: how to address, expected_outcome: what success looks like} Classification: New = novel {problem, persona} pair. Extension = same pair, different mechanism. Duplicate = same pair + mechanism → skip.

Eval Checkpoint

If --evals: check if current_iteration % interval == 0 → run checkpoint. Print: --- Eval Checkpoint (iterations {X}-{Y}) ---\nInsights: {total} (+{new}) | Categories: {covered}/5 | Saturation: {window}/3\n{recommendation}\n---

Phase 3: Feature Ranking + Selection

1. ICP binary gate — filter insights not serving the stated ICP 2. 3-tier bucketing — Must-have / Nice-to-have / Moonshot 3. Pairwise ranking within Must-have tier only (cap 7-10 items) 4. 2-sentence rationale per item citing research evidence 5. Confidence indicator per item (HIGH / MEDIUM / LOW)

Write improvement-plan.md with full tiered ranking.

request_user_input (multi-select): present tiered list, user selects which features become PRDs. If --features provided → pre-select matching items, still show for confirmation.

Phase 4: PRD Generation

Per selected feature, write prd-{feature-slug}.md:

• Top disclaimer: "Auto-generated from research findings. DECISION NEEDED items and LOW-confidence sections require your judgment."
• Problem statement (from research evidence chain)
• User stories (from ICP + persona data)
• Requirements (functional + non-functional, MoSCoW from tier)
• Acceptance criteria
• Technical approach (from codebase context, framed as "suggested starting points")
• Risks + confidence (evidence tiers: primary = codebase, secondary = web research)
• Success metrics
• DECISION NEEDED markers for unresolvable tradeoffs
• Open Questions section

Write research-findings.md — all insights with citations + confidence. Write summary.md — overview, research stats, category coverage, saturation status.

Summary

Print: total iterations, insights discovered (new/extension), categories covered, saturation status, PRDs generated, output directory path.

Eval Summary (--evals flag)

If --evals: write evals-summary.md to output directory with full analysis.

Handoff

Write handoff.json: version "2.1.0", source "improve", timestamp, status (COMPLETE|SATURATED|USER_INTERRUPT|BOUNDED|ERROR), results_tsv path, findings = improvements with tier + confidence + prd_path, config{goal, icp, depth, categories_explored, insights_total, prds_generated}. Improve is a terminal emitter — no downstream chain invocation.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Mode: or --mode — init (create from scratch), update (refresh existing), check (validate), summarize (brief overview), wiki (navigable knowledge base)
• Scope: or --scope — file globs to document
• Depth: or --depth — overview, standard, comprehensive
• --file <path> — specific file to document
• --scan — force fresh codebase scout
• --topics — comma-separated focus topics
• --modules <list> — wiki mode: comma-separated module names/paths overriding auto-detection
• --force — wiki mode: regenerate all pages from scratch, ignore existing manifest
• --no-fix — validate only, don't auto-fix issues
• --format — markdown (default), json, rst
• Iterations: or --iterations — default 10. "unlimited" for unbounded.
• --evals, --evals-interval N, --chain, --<subcommand>

Setup (if Mode or Scope missing)

request_user_input (single batch): Q1 (Mode): "What to do?" — init (generate docs), update (refresh), check (validate), summarize (overview), wiki (knowledge base) Q2 (Scope): "Which files?" — suggested globs + entire codebase Q3 (Depth): "How detailed?" — overview only, standard, comprehensive Q4 (Topics): "Focus on?" — architecture, API, database, testing, all If all provided → skip.

Establish Baseline

1. Scout codebase: file tree, imports/exports, existing docs 2. Identify documentation gaps (undocumented files, outdated docs, missing READMEs) 3. Create output directory: autoresearch/learn-{YYMMDD}-{HHMM}/ 4. TSV header: # metric_direction: higher_is_better\niteration\ttimestamp\tfile_documented\tvalidation_status\tissues_found\tissues_fixed\tdescription 5. Metric = files with valid documentation (higher is better)

Summarize Mode (no loop)

If mode == summarize:

• One-shot: scan codebase → produce structured summary
• Write summary.md to output directory
• Skip iteration loop entirely

Wiki Mode (no per-file loop)

If mode == wiki: reuse Scout (Phase 1) + Analyze output, then generate a navigable wiki/ knowledge base. Skip the init/update/check loop. Metric = pages_generated / pages_planned × 100 (from manifest); size target 300 lines/page.

Module Discovery (priority order)

1. --modules flag (explicit override, always wins; every path must resolve inside project root — reject escapes) 2. Monorepo workspaces (workspaces in package.json, Cargo workspace members, pnpm-workspace.yaml) 3. Per-directory project files (pyproject.toml, Cargo.toml, go.mod, *.csproj) 4. Heuristic: dirs with 3+ source files (code extensions only — .ts/.py/.go/.rs/.java/.rb/.swift/.kt/.c/.cpp/.cs; tests count, config/markdown don't); nested dirs roll up to nearest module ancestor

Cap 10 modules. If >10, group by top-level dir; if a group has >5 sub-modules, expand and take 10 largest by file count.

Plan (write-ahead)

1. mkdir -p wiki/modules/; append wiki-manifest.json to .gitignore if absent 2. Write wiki-manifest.json BEFORE generating — {version:"1", generated_at, generation_status:"in_progress", modules_detected:[…], pages_planned:N, pages:{ "wiki/architecture.md":{status:"pending",type:"architecture"}, "wiki/modules/<name>.md":{…"module"}, "wiki/glossary.md":{…}, "wiki/onboarding.md":{…}, "wiki/index.md":{…} }} 3. Write stub wiki/index.md listing every planned page as [pending] (navigation survives interruption) 4. Resume: if valid manifest exists → --force deletes it and regenerates all, else skip "generated" pages and only do "pending". Corrupted manifest (invalid JSON, or missing version/pages) without --force → error directing user to --force.

Generate (priority-first, one agent call per page, bounded context)

1. architecture.md — system overview from scout context. Up to 5 Mermaid diagrams, 3 types only (graph TD, sequenceDiagram, classDiagram); include one canonical example of each in the prompt; pick by signal. 2. Module pages (alphabetical) — per-module agent gets: (a) file listing, (b) first 50 lines of ≤10 key files (entry points → largest → alphabetical), (c) Phase 2 overview. Required sections: Overview, Key Files; optional: Patterns/API/Dependencies/Getting Started. 3. glossary.md — domain terms from class names, exports, types, comments; filter language keywords + stdlib; soft cap ~60-80, prioritize terms in 3+ files. 4. onboarding.md — reading order, env setup, first-contribution workflow, gotchas. Sources: dir structure, README/docs, manifests, entry-point sampling, git log --since='6 months ago' directory frequency (skip with note if not a git repo or >10s). 5. index.md — final pass: replace stub with real page descriptions + reading order.

Per-page contract

• generated_by: autoresearch in YAML frontmatter
• ~300 lines/page (soft); Mermaid ≤15 nodes/diagram; forward-only cross-links, ≤10 per page

Safety

• Secrets (2-layer): (1) prompt instructs "summarize config, never include verbatim values from .env/credentials or strings matching key/secret/token/password; extract env var names not values"; (2) post-gen, grep -rlE '(AKIA[0-9A-Z]{16}|sk-[a-zA-Z0-9]{20,}|ghp_[a-zA-Z0-9]{36}|password\s*[:=]\s*\S+|mongodb(\+srv)?://\S+|postgres(ql)?://\S+)' wiki/ and warn (non-blocking) in the report.
• Name collision: before overwriting a page, check for generated_by: autoresearch frontmatter; if absent (user-created) skip with warning. --force overrides.

Finish

After each page is written, flip its manifest entry pending→generated (interrupt-safe). When all done, set generation_status:"complete". Then run Phase 3 (Validate) with wiki path swap: replace docs/ with wiki/ (ls wiki/*.md wiki/modules/*.md 2>/dev/null), use maxLoc 300. Output: ✓ Wiki: [N] modules, [M] pages generated.

Iteration Loop (init/update/check modes)

Phase 1: Scout

• Scan for documentation gaps
• Prioritize: no docs → outdated docs → incomplete docs
• If no gaps remain → early stop (SUCCESS)

Phase 2: Generate/Update

• Pick highest-priority gap
• Write or update documentation for ONE file/module
• Follow project conventions for doc format and location

Phase 3: Validate

• Check generated docs against code: descriptions accurate? Examples valid? Links work?
• Run doc linters if available
• Record: validation_status (pass/fail), issues found

Phase 4: Fix (unless --no-fix)

• If validation finds issues → fix the doc
• Commit clean doc: docs: document {file/module}

Phase 5: Log

Append to TSV: iteration, timestamp, file_documented, validation_status, issues_found, issues_fixed, description

Eval Checkpoint

If --evals: check if current_iteration % interval == 0 → run checkpoint.

Bounded Check

If bounded: current_iteration >= max_iterations → exit loop.

Output

• learn-results.tsv
• summary.md — documentation overview
• validation-report.md — issues found/fixed

Summary

Print: files documented, validation pass rate, issues found/fixed, remaining gaps.

Eval Checkpoint (--evals flag)

If --evals present:

• Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded.
• Print: --- Eval Checkpoint (iterations {X}-{Y}) ---\nDocs written: {n} | Validation: {pass}/{total} | Gaps remaining: {m}\n{recommendation}\n---
• If 3+ checkpoints with no new docs → recommend early stop.
• At loop end → full evals summary to evals-summary.md.

Chain Handoff

After completion, write handoff.json: version "2.1.0", source "learn", timestamp, status, results_tsv path, findings = documentation gaps remaining, config{mode, scope, depth}. Invoke next target in --chain order. Propagate --evals flag.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Goal: — text after keyword, or full $ARGUMENTS if no keyword
• --chain <targets> — comma-separated downstream commands
• --<subcommand> — chain shorthand

Remaining text = goal description.

Setup (if Goal missing)

request_user_input (single batch): Q1 (Goal): "What do you want to achieve?" — open text Q2 (Type): "What kind of goal?" — improve a metric, fix errors, audit security, explore edge cases, document code, ship something If Goal provided → skip.

Phase 1: Analyze Goal

Parse the goal to determine:

• Is it measurable? (metric-driven vs subjective)
• What's the natural scope? (files, modules, entire codebase)
• What subcommand fits best? (core loop, fix, debug, security, etc.)

Phase 2: Derive Scope

1. Scan project structure 2. Identify files relevant to the goal 3. Propose file globs 4. If ambiguous → ask user to confirm

Phase 3: Derive Metric + Direction

For metric-driven goals:

• Identify what to measure (test coverage, error count, bundle size, latency, etc.)
• Determine direction: higher_is_better or lower_is_better
• Propose metric name and description

For subjective goals:

• Suggest proxy metrics where possible
• Or recommend $autoresearch reason for non-measurable goals

Phase 4: Derive Verify Command

1. Identify how to extract the metric as a number from a shell command 2. Propose Verify command (e.g., npm test -- --coverage | grep "All files" | awk '{print $10}') 3. Safety screen: check proposed command for rm -rf, fork bombs, curl|sh, credentials 4. Dry-run the Verify command → confirm it outputs a valid number 5. If dry-run fails → adjust command and retry

Phase 5: Derive Guard (optional)

Propose a Guard command if applicable:

• Test suite: npm test / pytest / go test ./...
• Type check: tsc --noEmit / mypy
• Build: npm run build
• None if not applicable

Phase 6: Suggest Iterations

Based on goal complexity:

• Simple metric improvement → 10-15
• Moderate refactoring → 20-25
• Complex multi-file changes → 30+
• Recommend bounded default, mention Iterations: unlimited option

Phase 7: Present Config

Output a ready-to-run autoresearch config block:

$autoresearch
Goal: {derived goal}
Scope: {derived globs}
Metric: {derived metric}
Direction: {higher_is_better|lower_is_better}
Verify: {derived command}
Guard: {derived guard or omit}
Iterations: {suggested count}

Ask user: "Run this config now, or adjust?"

Chain Handoff

If --chain set:

• Write handoff.json: version "2.1.0", source "plan", timestamp, status COMPLETE, config = derived config block
• Invoke next target with the derived config

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Scope: or --scope — file globs to analyze
• Goal: or --goal — focus area for analysis
• Depth: or --depth — shallow (3 personas, 1 round), standard (5, 2), deep (8, 3)
• --personas N — override persona count (3-8)
• --rounds N — override debate rounds (1-3)
• --adversarial — use hostile reviewer personas instead of default
• --budget N — max findings across all personas (default 40)
• --fail-on <severity> — CI gate: exit non-zero if findings at/above threshold
• --incremental — reuse existing knowledge files, update only changed files
• --chain, --<subcommand>

Remaining text not matching flags = goal description.

Setup (if Scope or Goal missing)

request_user_input (single batch): Q1 (Scope): "Which files to analyze?" — suggested globs + entire codebase Q2 (Goal): "What should personas focus on?" — code quality, security, performance, architecture, all Q3 (Depth): "How deep?" — shallow (3 personas, 1 round), standard (5, 2 — recommended), deep (8, 3) Q4 (Chain): "After analysis, chain to?" — debug, security, fix, ship, scenario, no chain If all provided → skip.

Phase 1: Reconnaissance

Scan all in-scope files. Build structured knowledge:

• File inventory with purpose annotations
• Dependency graph (imports/exports)
• API surface (routes, handlers, types)
• Data flow (inputs → processing → outputs → storage)
• Existing test coverage map

Phase 2: Persona Generation

Load references/predict-personas.md for persona definitions.

Default set (5): Architect, Security Analyst, Performance Engineer, Reliability Engineer, Devil's Advocate. Adversarial set (--adversarial): Breaker, Cheater, Scaler, Newbie, Malicious Insider.

Each persona receives: task description + codebase knowledge + their specific evaluation criteria. Personas are isolated — no shared context between them.

Phase 3: Independent Analysis

Each persona analyzes the codebase independently:

• Read relevant code through their lens
• Produce findings with: title, severity, confidence (0-100%), file:line, recommendation
• Max findings per persona: budget / persona_count

Phase 4: Debate (per round)

For each debate round: 1. Present all personas' findings to each other 2. Each persona can: challenge findings, raise new issues, change confidence 3. Cross-examination: personas must respond to challenges with evidence 4. No persona can dismiss without counter-evidence

Phase 5: Consensus

Synthesizer aggregates all findings: 1. Deduplicate (same file:line + same issue = merge, keep highest severity) 2. Resolve conflicts (if personas disagree, note dissent) 3. Anti-herd check: if all personas agree on everything, synthesizer MUST find at least 1 counter-argument 4. Rank by: severity × average confidence × persona agreement count

Phase 6: Report

Create output directory: autoresearch/predict-{YYMMDD}-{HHMM}/

Write:

• summary.md — top findings, consensus view, risk assessment
• debate.md — full persona analysis + debate transcript
• Per-persona sections with individual findings

Print to console: top 10 findings ranked by severity × confidence.

Phase 7: CI Gate

If --fail-on set: check findings against threshold. Exit non-zero if exceeded.

Chain Handoff

Write handoff.json: version "2.1.0", source "predict", timestamp, status (COMPLETE|ERROR), findings = consensus findings with severity + confidence + file:line, config{scope, goal, depth}. Invoke next target in --chain order.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Topic: — strip keyword, remaining text is topic (or full $ARGUMENTS if no keyword)
• Scope: or --scope — file globs for codebase grounding
• Depth: or --depth — shallow (5 rounds), standard (15), deep (30)
• --personas N or Personas: — active persona count (3-8, default 6)
• --saturation-threshold N — net-new constraints/round below which counts toward saturation (default 2)
• --mode or Mode: — interactive (default, uses request_user_input) or autonomous (self-answers from codebase)
• --adversarial — rotate hostile personas to front
• Iterations: or --iterations — default 15 rounds. "unlimited" for unbounded.
• --evals, --evals-interval N, --chain, --<subcommand>

Setup (if Topic missing)

request_user_input (single batch): Q1 (Topic): "What to probe?" — open text describing feature, requirement, or design Q2 (Scope): "Which files for context?" — suggested globs + entire codebase Q3 (Depth): "How deep?" — shallow (5 rounds), standard (15), deep (30), unlimited Q4 (Mode): "How to answer persona questions?" — interactive (you answer), autonomous (agent infers from code) If all provided → skip.

8 Personas

If --adversarial: rotate Skeptic + Contradiction Finder + Edge-Case Hunter to front.

Phase 1: Seed

• Parse topic into initial constraint set
• Read codebase context (if --scope provided)
• Initialize constraint registry (empty)

Round Loop

Phase 2: Persona Activation

• Select 2-3 personas for this round (rotate through all 8)
• Each persona generates 3-5 probing questions from their perspective

Phase 3: Codebase Grounding

• Check questions against existing code for evidence
• Annotate questions with: relevant file:line, existing behavior, gaps

Phase 4: Answer Capture

• Interactive mode: present questions via request_user_input, collect answers
• Autonomous mode: infer answers from codebase context, label confidence (high/medium/low)

Phase 5: Constraint Extraction

• Parse answers into atomic constraints
• Each constraint: id, source persona, description, confidence, evidence
• Deduplicate against existing registry

Phase 6: Cross-Check

• Check new constraints against existing for conflicts
• Flag contradictions for resolution (interactive → ask user, autonomous → note uncertainty)

Phase 7: Saturation Check

• Count net-new constraints this round
• If net-new < saturation_threshold for 3 consecutive rounds → SATURATED, exit loop
• Track: total constraints, new this round, saturation window

Phase 8: Log

Append to output: round number, personas active, questions asked, constraints extracted, net-new count

Eval Checkpoint

If --evals: check if current_round % interval == 0 → run checkpoint.

Bounded Check

If bounded: current_round >= max_iterations → exit loop.

Phase 9: Synthesize & Output

Create output directory: autoresearch/probe-{YYMMDD}-{HHMM}/

1. Write constraints.md — full constraint registry organized by category 2. Write conflicts.md — unresolved contradictions 3. Generate ready-to-run autoresearch config:

• Derived Goal, Scope, Metric, Verify from constraints
• Include as code block in summary.md

Print: total rounds, constraints found, saturation status, unresolved conflicts.

Summary

Print: total rounds, total constraints, net-new trend, saturation status, top 5 most impactful constraints.

Eval Checkpoint (--evals flag)

If --evals present:

• Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded.
• Print: --- Eval Checkpoint (rounds {X}-{Y}) ---\nConstraints: {total} (+{new}) | Saturation: {window_count}/3\n{recommendation}\n---
• If saturated 3+ checkpoints → recommend early stop.
• At loop end → full evals summary to evals-summary.md.

Chain Handoff

Write handoff.json: version "2.1.0", source "probe", timestamp, status (COMPLETE|SATURATED|USER_INTERRUPT|BOUNDED|ERROR), findings = constraints, config = derived autoresearch config. Invoke next target in --chain order. Propagate --evals flag.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Task: — question, proposal, design, argument, or claim to refine
• Domain: or --domain — software, product, business, security, research, content
• Mode: or --mode — convergent (default), creative, debate
• --judges N or Judges: — blind judge count (3 default, 5 thorough, 7 deep)
• --convergence N or Convergence: — stop when incumbent wins N consecutive rounds (default 3)
• Iterations: or --iterations — default 8. "unlimited" for unbounded.
• --judge-personas — custom judge persona overrides
• --no-synthesis — skip synthesis, pure debate only
• --temperature — generation temperature hint
• --evals, --evals-interval N, --chain, --<subcommand>

Remaining text not matching flags = task description.

Setup (if Task or Domain missing)

request_user_input (single batch): Q1 (Task): "What should be reasoned about?" — open text Q2 (Domain): "What domain?" — software architecture, product strategy, business decision, security, research, content Q3 (Mode): "Refinement mode?" — convergent (stop when winner repeats), creative (never auto-stop), debate (no synthesis) Q4 (Judges): "How many blind judges?" — 3 (default), 5 (thorough), 7 (deep) If all provided → skip.

Setup Phase

1. Load references/reason-judge-protocol.md for judge and convergence specs 2. Parse domain → select domain-specific judge criteria 3. Create output directory: autoresearch/reason-{YYMMDD}-{HHMM}/ 4. TSV header: round\ttimestamp\tcandidate_label\tjudge_verdict\tconvergence_count\tdescription 5. Initialize: incumbent = null, convergence_count = 0

Round Loop

Phase 1: Generate-A

• If round 1: Author-A generates first candidate from task description
• If round N>1: incumbent is Author-A's candidate
• Cold-start: Author-A sees ONLY task description + domain context

Phase 2: Critic

• Critic receives candidate-A (cold-start, no shared session)
• MUST find at least 3 specific weaknesses
• MUST suggest what a superior candidate would do differently
• Role is purely adversarial — never compliment

Phase 3: Generate-B

• Author-B receives: task + candidate-A + critique (cold-start)
• Produces candidate-B addressing critique while preserving A's strengths

Phase 4: Synthesize (unless --no-synthesis or debate mode)

• Synthesizer receives: task + A + B (cold-start)
• Produces hybrid candidate-AB merging best of both

Phase 5: Blind Judge Panel

• Each judge receives 3 candidates with RANDOMIZED labels (Label-X, Label-Y, Label-Z)
• Judges evaluate independently on domain-specific criteria
• Each produces ranking + one-paragraph justification
• Verdict: majority vote. Tie → synthesized candidate wins.

Phase 6: Convergence Check

• If winner == incumbent → convergence_count++
• If winner != incumbent → convergence_count = 1, winner becomes incumbent
• Convergent mode: convergence_count >= N → CONVERGED, stop
• Creative mode: never auto-stop
• Debate mode: same as convergent, no synthesis

Phase 7: Oscillation Guard

If incumbent changed 5+ times in last 8 rounds → recommend early stop (not converging).

Phase 8: Log

Append to TSV: round, timestamp, winning candidate label, judge verdict, convergence_count, description

Eval Checkpoint

If --evals: check if current_round % interval == 0 → run checkpoint.

Bounded Check

If bounded: current_round >= max_iterations → exit loop.

Output

• reason-results.tsv — per-round results
• lineage.md — full history of candidates + critiques + judge reasoning
• summary.md — final winner, convergence trajectory, key insights

Summary

Print: total rounds, convergence status, final winner summary, judge agreement rate.

Eval Checkpoint (--evals flag)

If --evals present:

• Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded.
• Print: --- Eval Checkpoint (rounds {X}-{Y}) ---\nIncumbent: {label} | Convergence: {count}/{target} | Oscillations: {n}\n{recommendation}\n---
• If oscillation detected 3+ checkpoints → recommend early stop.
• At loop end → full evals summary to evals-summary.md.

Chain Handoff

After completion, write handoff.json: version "2.1.0", source "reason", timestamp, status (COMPLETE|CONVERGED|USER_INTERRUPT|BOUNDED|ERROR), results_tsv path, findings = [{id, type: "recommendation", summary: winner description}], config{task, domain, mode}. Invoke next target in --chain order. Propagate --evals flag.

Orchestrator Routing

Goal Archetypes

Keyword matching is fuzzy — partial matches and synonyms qualify. When a goal matches multiple archetypes, prefer the more specific one (fix-broken over explore; ship-ready over fix-broken if "ship" is explicit). When ambiguous, show the top two candidates in the upfront confirm and let the user choose.

Router Decision Table

The next-hop subcommand of scripts/orchestrate.sh reads orchestrator-state.json and applies these rules in order. First match wins.

State signals are cheap reads — last handoff.json plus the regression verdict field and error count. No re-run of the full suite just to route.

Independent Verify & Overfit Guard

The orchestrator must not optimize and accept against the same signal — that lets a change game its own metric. For optimize-metric and build-feature, the acceptance check runs on a held-out set (a fresh scenario set or holdout assertions), separate from the units signal used to choose the change. When a high-impact change is accepted on the working signal, the orchestrator sets pending_verify in orchestrator-state.json; next-hop then routes to a verify hop (dispatched to reason or predict as an independent adversarial check) before declaring DONE or shipping. The verify hop is advisory input to convergence — it never auto-approves ship, which stays human-gated.

Two-Mode Split

Orchestration loop — used when the goal has an external, mechanical Success predicate: a shell command that returns a value the orchestrator can compare across cycles. Progress is objective (Units remaining falls), plateau is well-defined, and the loop terminates on convergence or a safety backstop. Archetypes: ship-ready, optimize-metric, fix-broken, harden, build-feature, explore.

Single-pass dispatch — used when no mechanical predicate exists. The goal is subjective or the subcommand is internally-converging (reason runs its own adversarial loop) or a one-shot terminal emitter (learn, improve produce a document and stop). The orchestrator routes once, the subcommand self-terminates, and the orchestrator reports the result. No Units remaining, no Plateau counter, no ship gate. Archetypes: document, what-to-build, decide-design.

The criterion is: "Can the orchestrator independently verify done without re-running the subcommand?" If yes → loop. If no → dispatch.

Build-Feature: TDD Ladder

The build-feature archetype has no pre-existing metric, so progress is reframed as green-assertion-count (monotone integer, higher-is-better). A change that turns a red sub-test green is kept; a change that regresses a green sub-test is reverted. A floor-guard prevents reverting scaffolding commits that compile and add no new failures but pass zero new tests. Large net-new scope (greenfield with no existing test suite) is detected and the orchestrator advises handing off to a dedicated build command rather than grinding cycles.

Preset Pipelines (Reference)

Presets are starting pipelines. The router adapts per cycle from observed state — it may skip, repeat, or reorder steps based on the decision table above. The preset is a prior, not a fixed schedule.

Glossary

Terms used consistently across this file, SKILL.md, and orchestrator-state.json. Definitions live in CONTEXT.md.

Predict Personas

Default Persona Set (5 personas)

1. Software Architect

• Focus: System design, component boundaries, data flow, scalability
• Questions: Does this design scale? Are boundaries clean? Is coupling minimized? Will this survive 10x growth?
• Evidence required: file:line citations, dependency graphs, coupling metrics
• Red flags: God classes, circular dependencies, leaky abstractions, shared mutable state

2. Security Analyst

• Focus: Attack surfaces, auth/authz, data protection, injection vectors
• Questions: Can this be exploited? Are trust boundaries enforced? Is data sanitized? Are secrets protected?
• Evidence required: file:line citations, attack scenarios, data flow through trust boundaries
• Red flags: Raw SQL, missing authz, hardcoded secrets, unsanitized user input

3. Performance Engineer

• Focus: Latency, throughput, resource usage, algorithmic complexity
• Questions: Will this be fast enough? What's the worst case? Where are the bottlenecks? Is caching effective?
• Evidence required: file:line citations, complexity analysis, resource estimates
• Red flags: N+1 queries, unbounded loops, missing indexes, synchronous I/O in hot paths

4. Reliability Engineer

• Focus: Error handling, failure modes, observability, recovery
• Questions: What happens when this fails? Can we detect it? Can we recover? Is it observable?
• Evidence required: file:line citations, failure scenarios, recovery paths
• Red flags: Swallowed errors, missing retries, no circuit breakers, silent failures

5. Devil's Advocate

• Focus: Assumptions, edge cases, hidden complexity, maintainability
• Questions: What assumptions are wrong? What's the simplest thing that breaks this? Is this over-engineered?
• Evidence required: Concrete counter-examples, edge case scenarios
• Red flags: Happy-path-only design, untested assumptions, complexity without justification

Adversarial Persona Set (activated with --adversarial)

Replace default personas with hostile reviewers: 1. The Breaker — tries to crash/corrupt the system 2. The Cheater — finds ways to bypass rules and abuse features 3. The Scaler — imagines 1000x load and finds what breaks 4. The Newbie — misuses every API and expects it to work 5. The Malicious Insider — has credentials, wants to exfiltrate

Debate Protocol

1. Each persona analyzes independently (no shared context between personas) 2. Findings reported with confidence score (0-100%) 3. Cross-examination: personas challenge each other's findings 4. Synthesizer aggregates, removes duplicates, resolves conflicts 5. Anti-herd check: if all personas agree, synthesizer must find at least 1 counter-argument 6. Final consensus: ranked findings with persona attribution

Output Format

Each persona produces:

### [Persona Name] — [N findings]
| # | Finding | Severity | Confidence | File:Line | Recommendation |

Synthesizer produces:

### Consensus — [N findings after dedup]
| # | Finding | Severity | Agreement | Source Personas | Action |

Reason Judge Protocol

Adversarial Refinement Loop

Round N:
  1. Author-A generates candidate (or incumbent from previous round)
  2. Critic attacks candidate — MUST find weaknesses (forced adversarial)
  3. Author-B reads task + candidate-A + critique → produces candidate-B
  4. Synthesizer reads A + B → produces hybrid candidate-AB
  5. Judge panel receives 3 candidates with randomized labels → picks winner
  6. Winner becomes incumbent for round N+1

Agent Isolation Rules

• Each agent (Author-A, Critic, Author-B, Synthesizer, Judges) runs COLD START
• No shared session state between agents — prevents sycophancy
• Agents receive ONLY: task description + relevant candidate(s) + critique
• Judges receive candidates with randomized labels (Label-X, Label-Y, Label-Z)
• Judges MUST compare and rank — "all are good" is not a valid verdict

Critic Protocol

The critic MUST: 1. Identify at least 3 specific weaknesses in the candidate 2. Provide concrete evidence for each weakness 3. Suggest what a superior candidate would do differently 4. Rate candidate on domain-specific criteria (1-10 scale) 5. Never compliment the candidate — role is purely adversarial

Judge Protocol

Each judge receives:

• Task description (identical for all judges)
• 3 candidates with randomized labels (Label-X, Label-Y, Label-Z)
• Evaluation criteria relevant to the domain

Each judge MUST: 1. Evaluate each candidate independently on all criteria 2. Produce a ranking (1st, 2nd, 3rd) with reasoning 3. Select a winner with one-paragraph justification 4. Label randomization prevents position bias

Verdict: majority vote. Tie → synthesized candidate (Label-Z) wins.

Convergence Detection

Oscillation Guard

If the incumbent changes more than 5 times in the last 8 rounds → recommend early stop. The candidates are not converging — further rounds waste context.

Domain-Specific Judge Criteria

Output Files

TSV Schema

round	timestamp	candidate_label	judge_verdict	convergence_count	description
1	2026-05-19T00:00:00Z	Candidate-A	winner	1	Event sourcing with CQRS
2	2026-05-19T00:05:00Z	Candidate-AB	winner	1	Hybrid: event sourcing for writes, read projections
3	2026-05-19T00:10:00Z	Candidate-AB	winner	2	Refined hybrid with materialized views
4	2026-05-19T00:15:00Z	Candidate-AB	winner	3	CONVERGED — same approach refined

Security Audit Checklist

STRIDE Threat Categories

OWASP Top 10 (2021) Checklist

Red-Team Personas

Severity Classification

Composite Metric Formula

score = (owasp_categories_tested / 10) * 50
      + (stride_categories_tested / 6) * 30
      + min(unique_findings, 20)

Higher is better. Perfect score = 100 (all OWASP tested + all STRIDE tested + 20 findings).

Coverage Tracking

Print coverage summary every 5 iterations:

OWASP: [A01✓ A02✓ A03✗ A04✗ A05✓ A06✗ A07✓ A08✗ A09✗ A10✗] 4/10
STRIDE: [S✓ T✓ R✗ I✓ D✗ E✗] 3/6
Score: 48.3 | Findings: 7

Finding Format

Every finding requires: 1. Title — one-line summary 2. Severity — Critical/High/Medium/Low/Info 3. OWASP — A01-A10 category 4. STRIDE — S/T/R/I/D/E category 5. Evidence — file:line + attack scenario (no theoretical fluff) 6. Reproduction — steps to trigger 7. Mitigation — concrete fix recommendation

EXECUTE IMMEDIATELY.

A regression is a green→red transition ONLY. The gate orchestrates the project's OWN test/bench/snapshot/migrate commands (it is a protocol, not a bundled framework), captures baseline behavior in an isolated git worktree, re-runs the candidate, and reports a tiered ship/no-ship verdict.

Parse Arguments

Extract from $ARGUMENTS:

• Base: or --base — base ref to diff against. Default: git merge-base HEAD main (else main/master).
• Scope: or --scope — file globs limiting the change surface.
• --select auto|full|affected — test selection (default auto). auto = use the detected affected-test mapper if available, else FULL suite. Never a silent subset.
• --samples N — SCORE samples/side (default 7). --noise-band % — perf tolerance (default 5%).
• --matrix — opt-in matrix axis (OFF by default). --max-runs N — ceiling (default 200).
• --baseline-cache (default on) — reuse baseline/<full-sha>/ by SHA. Baseline: <prebuilt-ref> — bypass capture.
• --probe (default) / --probe deep / --no-probe.
• --predict --reason --debug --fix --fix-cycles N --evals --evals-interval N --chain <targets> and --<sub> shorthand.
• Iterations: — repeat-axis count for --select/repeat sweeps.

Setup / Probe-on-launch

1. Auto-detect per-dimension verify commands: package.json scripts, Makefile, nx, migrate config, bench/snapshot/size scripts. 2. AskUserQuestion (single batch) to confirm detected commands + base ref + which dimensions to run. 3. Auto-skip probe when CI / no-TTY / --mode autonomous / complete-config / chained-handoff — log the inferred config instead of asking.

Classification Phase (first-class, before any differential)

Establish the baseline green-set per dimension, then tag each unit. Match by test-id first, then path.

Core invariant: red→red, absent→red, and flake→red are NOT regressions. Run flakiness N× on both baseline and candidate; a candidate failure inside the baseline flake-envelope routes to flakiness SCORE, never to a regression. 5/5 green ≠ non-flaky — detection probability is 1−(1−p)^n (≈23% at p=5%, n=5); print it.

Baseline Capture

git worktree add --detach <full-sha> (detached SHA — avoids "branch already checked out" when Base==HEAD) → baseline/<full-sha>/; --baseline-cache reuses by SHA. Then per worktree: git submodule update --init + dependency install (lockfile is SHA-pinned so the cache stays sound). Per-dimension setup tiers: api-contract = file-diff, no build; functional / integration-e2e / data-migration = full env. On completion or crash: git worktree remove + git worktree prune. Warn on concurrent index-lock contention. Baseline: <prebuilt-ref> bypasses capture.

Dimension Registry (8)

--select auto mapper (jest --findRelatedTests fed the changed-file list; nx affected project-graph) is best-effort static-import — blind to dynamic/runtime/global-setup couplings. The report names the mapper + its blind-spot caveat; a HARD STABLE earned on an affected subset prints "run --select full for high-stakes". FULL suite is the correctness default.

• performance independence: each sample = an independent process launch (warmups discarded), never an in-process iteration — autocorrelation/GC/thermal otherwise violate Mann-Whitney's independence assumption. At n=7 the test detects only ≳1σ regressions; raise --samples for tight gates.
• data-migration guard: opt-in. Before any migration the DB URL MUST pass an anchored allowlist — host is exactly localhost / 127.0.0.1 / a container or service hostname, OR the database name carries a _test / _ci suffix. A bare substring (e.g. test inside latest, ci inside precision) does not qualify. Anything else is refused — ephemeral only, never dev/prod — and even an allowlisted URL requires explicit user confirm before applying. Missing/absent down-migration = forward-only advisory, never a finding.

Differential Loop (per dim × axis × run)

Run candidate verify vs baseline metric → compute regressed bool + 0-100 subscore. Axes: diff (default), repeat N×, full, matrix (opt-in). Log one TSV row per cell.

--max-runs ceiling: projected = dims × axes × samples × matrix-cells; if > --max-runs (default 200) → warn + require confirm (CI default = abort with message).

Verdict

• Any HARD regressed=true with classification=eligible → UNSTABLE (green→red hard-blocks).
• Else stability_score = Σ(weight × dim_subscore) over SCORE dims that ran (flakiness .30 / performance .30 / resource .20 / visual .20, renormalized over present dims). STABLE iff ≥ 95 (REG_THRESHOLD/weights overridable).
• Print the score math (per-dim contribution table) + declare dims-ran vs UNAVAILABLE — an UNAVAILABLE dimension is always listed, never silently passed.

Backed by scripts/score-regression.sh verdict <results.tsv> (exit 0 STABLE / 1 UNSTABLE).

Hunter (root cause)

On a confirmed HARD regression, auto-engage. Bisect (reuse debug) ONLY when the failing case passes a 3/3 reproducibility gate. SCORE / non-deterministic regressions → differential root-cause + optional --reason / --predict, no bisect. Non-reproducible → "manual triage" finding.

--fix Re-gate

--fix repairs blocking regressions, max 3 cycles (--fix-cycles N). Each cycle MUST strictly shrink the blocking-set else STOP "fix not converging". Intermediate re-gate scopes to failing+touched dims; the final cycle runs the full battery. No HARD-gate bypass.

Output

autoresearch/regression-{YYMMDD}-{HHMM}/ → regression-results.tsv, stability-report.md, dimensions/<dim>.md, baseline/, evals-summary.md (if --evals), handoff.json.

TSV header: # metric_direction: higher_is_better then iteration\ttimestamp\tdimension\taxis\ttier\tclassification\tbaseline\tcandidate\tdelta\tregressed\tsubscore\tseverity\tstatus\tfile_line\tdescription.

Eval Checkpoint (--evals flag)

Interval = floor(max_runs / 3), min 1 (fixed 10 if unbounded); override --evals-interval N. Every interval, analyze the results TSV; print trend (up/flat/down) + one-line recommendation. Plateau 3+ checkpoints → recommend early stop. At end → full summary to evals-summary.md.

Chain Handoff

Write handoff.json to the output directory: version "2.1.0", source "regression", timestamp, status ∈ family enum {COMPLETE, CONVERGED, SATURATED, BOUNDED, USER_INTERRUPT, ERROR} (backward-compat with evals/ship consumers), verdict ∈ {STABLE, UNSTABLE, BASELINE_UNAVAILABLE} + regression_state ∈ {REGRESSION_FOUND, REGRESSION_FIXED, none} — ship reads verdict for the deploy-gate, results_tsv path, findings = blocking regressions (dim, severity, file_line, classification), config{base, scope, dims, axes, verdict-math}.

If --fix → chain to fix automatically. Invoke next --chain target in order; propagate --evals. Canonical combo: --predict --evals --fix --ship = predict → gate → (hunter on HARD) → fix(≤3) → re-gate → ship iff STABLE (deploy still needs explicit approval).

Safety

Verify-command screen (no rm -rf / curl|sh); worktree cleanup + prune on crash; data-migration refuses any non-allowlisted DB URL; probe auto-skips non-interactively; chained ship never auto-deploys.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Scenario: — seed scenario description (or full $ARGUMENTS text if no keyword)
• Domain: or --domain — web, mobile, API, CLI, data pipeline, infrastructure
• Scope: or --scope — file globs for codebase context
• Focus: or --focus — specific dimension to prioritize
• --depth — shallow (10), standard (20), deep (40+)
• --format — markdown (default), json, gherkin
• Iterations: or --iterations — default 20. "unlimited" for unbounded.
• --evals, --evals-interval N, --chain, --<subcommand>

Setup (if Scenario or Domain missing)

request_user_input (single batch): Q1 (Scenario): "Describe the feature/flow to explore" Q2 (Domain): "What domain?" — web app, mobile app, API, CLI, data pipeline, infrastructure Q3 (Scope): "Which files for context?" — suggested globs + entire codebase Q4 (Depth): "How deep?" — quick (10), standard (20), deep (40+), unlimited If all provided → skip.

12 Dimensions

Establish Baseline

1. Read seed scenario + codebase context 2. Create output directory: autoresearch/scenario-{YYMMDD}-{HHMM}/ 3. TSV header: iteration\ttimestamp\tscenario\tdimension\tclassification\tseverity\tdescription 4. No metric_direction comment (exploration, not optimization)

Iteration Loop

Phase 1: Review

• Read results TSV, check dimension coverage
• Identify underexplored dimensions
• If --focus → prioritize that dimension

Phase 2: Generate

• Pick next dimension (round-robin, or priority if --focus)
• Generate 3-5 specific scenarios for this dimension
• Each: title, dimension, classification, severity, description

Phase 3: Classify

• new — genuinely novel edge case
• extension — builds on previously found scenario
• duplicate — already covered (skip, don't log)

Phase 4: Log

Append new/extension scenarios to TSV. Skip duplicates. Severity: critical/high/medium/low.

Phase 5: Saturation Check

If 3 consecutive iterations produce only duplicates → dimension saturated, move to next. If ALL dimensions saturated → early stop.

Eval Checkpoint

If --evals: check if current_iteration % interval == 0 → run checkpoint.

Bounded Check

If bounded: current_iteration >= max_iterations → exit loop.

Output

• Write scenarios.md (organized by dimension, severity-ranked within each)
• Write edge-cases.md (flat severity-ranked list)
• scenario-results.tsv

Summary

Print: total scenarios (new/extension/duplicate), dimension coverage (X/12 explored), severity distribution.

Eval Checkpoint (--evals flag)

If --evals present:

• Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded.
• Print: --- Eval Checkpoint (iterations {X}-{Y}) ---\nNew scenarios: {n} | Dimensions covered: {x}/12 | Saturation: {status}\n{recommendation}\n---
• If 3+ checkpoints with mostly duplicates → recommend early stop.
• At loop end → full evals summary to evals-summary.md.

Chain Handoff

After completion, write handoff.json: version "2.1.0", source "scenario", timestamp, status, results_tsv path, findings = scenarios by severity, config{scenario, domain, scope}. Invoke next target in --chain order. Propagate --evals flag.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Scope: or --scope — file globs to audit
• Focus: — specific area (auth, API, data handling, etc.)
• Depth: or --depth — quick (5 iterations), standard (15), deep (30+)
• Iterations: or --iterations — default 15. "unlimited" for unbounded.
• --diff — delta mode: only audit files changed since last audit
• --fix — after audit, auto-fix Critical/High findings (chains to fix)
• --fail-on <severity> — exit non-zero if findings at/above threshold (CI gate)
• --evals, --evals-interval N, --chain, --<subcommand>

Setup (if required context missing)

If Scope missing and no --diff: 1. Scan codebase for tech stack, frameworks, API routes 2. request_user_input (single batch): Q1 (Scope): "What to audit?" — entire codebase, API + middleware, auth, external-facing Q2 (Depth): "How thorough?" — quick (5), standard (15), deep (30+), unlimited Q3 (Action): "What to do with findings?" — report only, report + auto-fix, report + CI gate If all provided → skip.

Setup Phase (once, before loop)

1. Reconnaissance — scan: package.json/requirements.txt (deps), .env.example (secrets), Dockerfile (infra), API route files (attack surface), auth/middleware (trust boundaries), DB schemas (data assets), CI/CD configs (supply chain) 2. Asset Identification — catalog data stores, auth systems, external services, user inputs 3. Trust Boundary Mapping — browser↔server, public↔authenticated, user↔admin, CI↔prod 4. STRIDE Threat Model — generate threats per category. Load references/security-checklist.md for checklist. 5. Attack Surface Map — entry points, data flows, abuse paths 6. Baseline — count known issues, initialize coverage tracking

Create output directory: autoresearch/security-{YYMMDD}-{HHMM}/ Write: overview.md, threat-model.md, attack-surface-map.md TSV header: # metric_direction: higher_is_better\niteration\ttimestamp\tfinding\tseverity\towasp\tstride\tevidence\tfile_line

Iteration Loop

Phase 1: Review

• Read results TSV + coverage tracking
• Identify untested attack vectors from threat model
• Prioritize: untested OWASP categories → untested STRIDE → depth on existing

Phase 2: Attack

• Adopt red-team persona for this vector (rotate: Security Adversary, Supply Chain, Insider Threat, Infra Attacker)
• Deep-dive into relevant code with adversarial mindset
• Look for: code paths, input handling, auth checks, data flows

Phase 3: Validate

• Construct proof: file:line + specific attack scenario
• Every finding MUST have code evidence — no theoretical fluff
• Classify severity: Critical/High/Medium/Low/Info
• Map to OWASP (A01-A10) and STRIDE (S/T/R/I/D/E)

Phase 4: Log

• Append finding to TSV
• Update coverage tracking
• Print coverage every 5 iterations:

OWASP: [A01✓ A02✓ A03✗ ...] X/10 | STRIDE: [S✓ T✓ R✗ ...] Y/6 | Score: Z

Composite Metric

score = (owasp_tested/10)*50 + (stride_tested/6)*30 + min(findings, 20)

Eval Checkpoint

If --evals: check if current_iteration % interval == 0 → run checkpoint.

Bounded Check

If bounded: current_iteration >= max_iterations → exit loop.

After Loop

1. Write findings.md (severity-ranked) 2. Write owasp-coverage.md 3. Write recommendations.md 4. If --fix → chain to fix with Critical/High findings 5. If --fail-on → check findings against threshold, exit non-zero if exceeded

Summary

Print: total findings by severity, OWASP coverage X/10, STRIDE coverage Y/6, composite score.

Eval Checkpoint (--evals flag)

If --evals present:

• Compute interval: floor(max_iterations / 3), min 1. Fixed 10 if unbounded. Override: --evals-interval N.
• Every {interval} iterations, analyze results TSV.
• Print: --- Eval Checkpoint (iterations {X}-{Y}) ---\nScore: {start} → {end} | New findings: {n} | Coverage: OWASP {x}/10, STRIDE {y}/6\n{recommendation}\n---
• If no new findings 3+ checkpoints → recommend early stop.
• At loop end → full evals summary to evals-summary.md.

Chain Handoff

After completion, write handoff.json to output directory: version "2.1.0", source "security", timestamp, status (COMPLETE|USER_INTERRUPT|BOUNDED|ERROR), results_tsv path, findings = all findings with severity + OWASP + STRIDE + file:line, config{scope, focus, depth}. Invoke next target in --chain order. Propagate --evals flag.

EXECUTE IMMEDIATELY.

Parse Arguments

Extract from $ARGUMENTS:

• Target: or --target — what to ship (path, PR, artifact, deployment)
• --type <type> — override auto-detection: code-pr, code-release, deployment, content, docs, package, config
• --dry-run — validate everything but don't ship
• --auto — auto-approve if no errors found
• --force — skip non-critical items (blockers still enforced)
• --rollback — undo last ship action
• --monitor N — post-ship monitoring for N minutes
• --checklist-only — only generate checklist, don't execute
• --chain, --<subcommand>

Remaining text = description of what to ship.

Setup (if Target or Type unclear)

1. Auto-detect ship type from context:

• Has uncommitted changes or PR → code-pr
• Has version bump / changelog → code-release
• Has Dockerfile / deploy config → deployment
• Has markdown / content files → content
• Has package.json version change → package

2. If still unclear → request_user_input (single batch): Q1 (What): "What are you shipping?" — code PR, release, deployment, content, docs, package Q2 (Target): "Specific target?" — current branch, specific PR, specific path Q3 (Mode): "How to ship?" — full workflow, dry-run only, checklist only If all clear → skip.

Phase 1: Identify

• Determine ship type (auto-detected or --type override)
• Identify target artifact(s)
• Map to domain-specific checklist

Phase 2: Inventory

Gather everything that will be shipped:

• Files changed (git diff)
• Dependencies affected
• Config changes
• Migration files
• Breaking changes

Phase 3: Checklist

Generate domain-specific checklist:

Code PR: tests pass, types check, lint clean, no secrets, PR description, reviewers assigned Release: version bumped, changelog updated, migration tested, rollback plan Deployment: env vars set, health checks configured, rollback ready, monitoring active Content: links valid, images optimized, SEO metadata, spell check Package: version bumped, README updated, breaking changes documented, CI green

If --checklist-only → output checklist and stop.

Phase 4: Prepare

Execute pre-ship tasks:

• Run test suite
• Run type checker
• Run linter
• Check for secrets in diff
• Validate configs
• Flag blockers (must-fix) vs warnings (can-ship-with)

If blockers found → STOP, report blockers, ask user to fix.

Phase 5: Dry-Run

If --dry-run or always before actual ship:

• Simulate the ship action without executing
• Report what WOULD happen
• If --dry-run → stop here

Phase 6: Ship

REQUIRES EXPLICIT USER APPROVAL (unless --auto with zero errors).

Execute the ship action:

• Code PR: create/update PR, request reviewers
• Release: tag, build, publish
• Deployment: deploy to target environment
• Content: publish to CMS/platform

Phase 7: Verify

Post-ship verification:

• Confirm artifact is live/accessible
• Run smoke tests if available
• Check monitoring for errors
• If --monitor N → watch for N minutes

Phase 8: Log

Create output directory: autoresearch/ship-{YYMMDD}-{HHMM}/ Write:

• checklist.md — completed checklist with pass/fail per item
• summary.md — what was shipped, verification results
• ship-log.tsv — phase-by-phase log

Rollback

If --rollback:

• Identify last ship action from most recent ship log
• Reverse it (revert PR, unpublish, rollback deployment)
• Verify rollback succeeded

Chain Handoff

Write handoff.json: version "2.1.0", source "ship", timestamp, status (COMPLETE|DRY_RUN|ROLLBACK|ERROR), findings = blockers/warnings found during prep. Invoke next target in --chain order.