Tiered Audit

Tiered Audit

Table of Contents

• When to Use
• When NOT to Use
• Tier 1: Git History Audit
• Tier 2: Targeted Area Audit
• Tier 3: Full Codebase Audit
• Output Contract

When To Use

• Auditing codebase quality, patterns, or problems
• Reviewing what changed on a branch before merge
• Investigating areas of instability or churn
• Pre-PR quality assessment

When NOT to Use

• Reviewing a specific file (use pensive:code-reviewer)
• Architecture-only review (use pensive:architecture-review)
• Single-commit review (use imbue:diff-analysis)

Tier 1: Git History Audit

Always runs first. Analyzes git log, diff stats, and blame to identify areas of concern without reading any source files.

What Tier 1 Analyzes

Run these git commands for the target commit range (default: current branch vs main):

# 1. Churn hotspots: files changed most often
git log --format="" --name-only {base}..HEAD \
  | sort | uniq -c | sort -rn | head -20

# 2. Diff stats: size of changes per file
git diff --stat {base}..HEAD

# 3. Fix-on-fix patterns: commits fixing previous commits
git log --oneline {base}..HEAD \
  | grep -iE "(fix|revert|patch|hotfix)"

# 4. New file clusters: modules with many new files
git diff --name-status {base}..HEAD \
  | grep "^A" | cut -f2 \
  | sed 's|/[^/]*$||' | sort | uniq -c | sort -rn

# 5. Large commits: single commits with big diffs
git log --format="%h %s" --shortstat {base}..HEAD

Verification: Confirm each command produces output. If a command returns empty, the commit range may be wrong; verify {base} resolves correctly with git merge-base.

Tier 1 Output Format

Write findings to .coordination/agents/tier1-audit.findings.md:

---
agent: tier1-audit
tier: 1
evidence_count: {N}
---

## Summary

{1-2 sentence overview of what the git history reveals}

## Churn Hotspots

{top 10 most-changed files with change counts}

For each flagged file, include:
- Location: path/to/file.py:line (most-changed function or block)
- Anchor: `verbatim source text at that line`

[E1] Command: git log --format="" --name-only ...
     Output: {relevant output}

## Fix-on-Fix Patterns

{commits that fix previous commits in the same area}

[E2] Command: git log --oneline ... | grep -iE ...
     Output: {relevant output}

## New File Clusters

{modules with 5+ new files}

## Large Diffs

{commits with 200+ line changes}

## Escalation Recommendation

{list of areas flagged for Tier 2, or "no escalation needed"}

Escalation Decision

After Tier 1 completes, check findings against the escalation criteria in modules/escalation-criteria.md.

If NO criteria are met: audit is complete. Report findings.

If criteria ARE met: list flagged areas and proceed to Tier 2 for each area sequentially.

Tier 2: Targeted Area Audit

Runs only for areas flagged by Tier 1. Each flagged area is audited one at a time, not in parallel.

What Tier 2 Analyzes

For each flagged area:

1. Read the source files in the area 2. Check for patterns, anti-patterns, bugs 3. Verify test coverage exists 4. Check documentation currency 5. Assess architectural fit

Tier 2 Output Format

One findings file per area: .coordination/agents/tier2-{area-name}.findings.md

Each file follows the output contract for audits (see imbue:proof-of-work/modules/output-contracts).

Tier 3: Full Codebase Audit

Requires explicit user approval. See modules/escalation-criteria.md for the gate protocol.

Tier 3 should use dedicated sessions (one per area) with file-based coordination, NOT parallel subagents.

Output Contract

All tiers use this contract:

output_contract:
  required_sections:
    - summary
    - evidence
  min_evidence_count: 3    # Tier 1
  # min_evidence_count: 8  # Tier 2
  expected_artifacts: []
  retry_budget: 1
  strictness: normal

Tier 2 raises the minimum evidence count to 8 because it reads source files and should produce deeper analysis.

Verification: After each tier completes, verify the findings file exists and contains at least the minimum evidence count ([E1], [E2], etc.) before proceeding to the next tier or reporting results.

Verify Findings Are Grounded (tiered-audit:findings-verified)

Every finding must cite a real location and a verbatim anchor. Write findings to .review/findings.json and confirm each citation resolves:

python plugins/imbue/scripts/citation_verifier.py \
  --findings .review/findings.json --repo-root .

Drop or label UNVERIFIED any finding the verifier fails (exit 1); only verified findings enter the report. See Skill(imbue:review-core) Step 5 and Skill(imbue:structured-output) for the schema.

Exit Criteria

• [ ] Tier 1 findings file exists at

.coordination/agents/tier1-audit.findings.md and contains at least 3 evidence entries ([E1]–[E3]).

• [ ] Tier 2 is only started for areas explicitly flagged by Tier 1

escalation criteria.

• [ ] Every reported finding carries a Location + verbatim Anchor

confirmed by citation_verifier.py (exit 0), or unverified findings were dropped or labeled UNVERIFIED.

Escalation Criteria

Audit tiers escalate based on evidence from the previous tier, not by default. Each escalation requires documented justification.

Tier 1 -> Tier 2 Escalation

Tier 1 (git-history analysis) flags areas for Tier 2 when ANY of these criteria are met:

Churn Hotspots

• 3+ files in the same module changed in the

analyzed commit range

• AND at least one file changed more than twice
• Indicates active development area worth deeper review

Fix-on-Fix Patterns

• A commit that fixes a previous fix within the same

module (commit messages containing "fix", "revert", "patch", "hotfix" targeting the same files)

• Indicates instability or insufficient testing

Large Diffs

• Any single commit touching 200+ lines in one

module

• Large changes are statistically more likely to

contain defects

Suspicious Patterns

• Reverted commits (indicates something went wrong)
• Commits with no tests added alongside implementation

changes

• Force-pushed branches affecting the module

New File Clusters

• 5+ new files added to a single module in the

analyzed range

• Indicates new feature work that may lack review

coverage

Tier 2 -> Tier 3 Escalation

Tier 2 (targeted area audit) recommends Tier 3 when ANY of these criteria are met:

Cross-Cutting Concerns

• Findings in one area reveal issues that likely

affect other areas (e.g., a shared utility function with a bug, a pattern used across modules)

Architectural Issues

• Tier 2 findings indicate structural problems

(circular dependencies, layering violations, inconsistent patterns across modules)

Coverage Gaps

• Tier 2 reveals that the flagged area is

representative of a broader pattern (e.g., all plugins share the same anti-pattern)

Severity Threshold

• Tier 2 finds 3+ critical-severity issues in

a single area, suggesting systemic quality problems

Tier 3 Gate

Tier 3 (full codebase audit) requires:

1. Documented justification from Tier 2 findings 2. Explicit user approval before proceeding 3. Recommended execution mode: dedicated sessions (not subagents), one area at a time, sequential

The system MUST present the justification and wait for confirmation. It MUST NOT auto-escalate to Tier 3.

Escalation Log Format

Every escalation records:

## Escalation: Tier {N} -> Tier {N+1}

**Date**: {timestamp}
**From tier**: {N}
**To tier**: {N+1}
**Target areas**: {list of modules/directories}

### Triggering Evidence

{specific findings from the previous tier that
triggered this escalation, with evidence tags}

### Justification

{why this escalation is warranted, referencing
the criteria above}

No-Escalation Path

When Tier 1 finds NO flags:

• Audit completes at Tier 1
• Summary reports "no areas flagged for deeper review"
• No Tier 2 is triggered
• This is the expected happy path for stable codebases

Tier 2: Targeted Area Audit

Runs ONLY for areas flagged by Tier 1 escalation. Each area is audited sequentially, never in parallel.

Execution Protocol

For each flagged area in the escalation list:

1. Load the area context from plugin CLAUDE.md and skill descriptions 2. Read source files in the area 3. Analyze for:

• Code quality patterns and anti-patterns
• Test coverage (do tests exist for this code?)
• Documentation currency (do docs match the code?)
• Architectural fit (does this follow project

conventions?) 4. Write findings to .coordination/agents/tier2-{area-slug}.findings.md 5. Validate findings against the Tier 2 output contract 6. Move to next area

Output Contract (Tier 2)

output_contract:
  required_sections:
    - summary
    - scope_analyzed
    - findings_by_severity
    - recommendations
    - evidence
  min_evidence_count: 8
  expected_artifacts: []
  retry_budget: 1
  strictness: strict

Tier 2 uses strict mode because it reads source files and should produce thorough, evidence-backed analysis.

Sequential Execution

Areas are processed one at a time because:

• Each area analysis fills a significant portion of

the agent's context

• Sequential processing prevents context cross-

contamination between areas

• The coordinator can review each area's findings

before proceeding to the next

• If early areas reveal the issue is resolved, later

areas can be skipped

Escalation to Tier 3

After all Tier 2 areas are audited, check whether Tier 3 is warranted per escalation-criteria.md. If so, present justification to the user and wait for explicit approval.

Tier 3: Full Codebase Audit Gate

Tier 3 is the most expensive audit tier. It MUST NOT run without explicit user approval.

Why this stays opt-in. Per

[docs/inclusive-defaults.md][inc] (TRUE-exception

category 7), Tier 1 (git history) is the inclusive

default. Full-codebase scans burn compute and tokens

at a rate that requires explicit user authorization.

[inc]: ../../../../../docs/inclusive-defaults.md

Gate Protocol

When Tier 2 findings indicate Tier 3 is warranted:

1. Present the justification to the user:

## Tier 3 Escalation Recommended

Tier 2 findings suggest a full codebase audit is
warranted.

### Justification

{specific Tier 2 findings that triggered this}

### Areas Already Reviewed (Tier 2)

{list of areas already audited}

### Estimated Scope

{number of remaining areas / files}

### Recommended Approach

- Use dedicated sessions (one per area)
- Process areas sequentially
- Coordinate via .coordination/ files
- Do NOT use parallel subagents

Proceed with Tier 3? [requires explicit yes]

2. Wait for user confirmation 3. If approved, execute with dedicated sessions 4. If declined, finalize with Tier 2 findings

Execution Mode

Tier 3 MUST use dedicated sessions because:

• Full codebase analysis fills context windows quickly
• Parallel subagents would degrade quality

(the exact problem this system solves)

• Dedicated sessions get full context windows with

no completion pressure

• File-based coordination preserves all findings

Output

Each area produces findings in the standard format: .coordination/agents/tier3-{area-slug}.findings.md

Final synthesis reads all findings files and produces a thorough report.