
Quality Auditor
- 73 installs
- 14 repo stars
- Updated March 2, 2026
- oakoss/agent-skills
Helps with ai & agent building tasks during AI-assisted development.
About
quality-auditor is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted coding.
- quality-auditor
- AI & Agent Building
- AI-coding skill
Quality Auditor by the numbers
- 73 all-time installs (skills.sh)
- +1 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #5,555 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/oakoss/agent-skills --skill quality-auditorAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 73 |
|---|---|
| repo stars | ★ 14 |
| Last updated | March 2, 2026 |
| Repository | oakoss/agent-skills ↗ |
What it does
Helps with ai & agent building tasks during AI-assisted development.
Files
Quality Auditor
Overview
Evaluates tools, frameworks, systems, and codebases against the highest industry standards across 12 weighted dimensions. Produces evidence-based scores, identifies anti-patterns, and generates prioritized improvement roadmaps. Applies extra scrutiny to AI-generated code through the verification gap protocol, ensuring velocity does not compromise integrity.
When to use: Auditing code quality, reviewing AI-generated code, scoring codebases against industry benchmarks, enforcing pre-commit quality gates, comparing tools or frameworks, assessing technical debt.
When NOT to use: Quick code reviews without scoring, style-only linting (use a linter), feature implementation, routine PR reviews that do not require a full audit.
Quick Reference
| Dimension | Weight | What to Evaluate |
|---|---|---|
| Code Quality | 10% | Structure, patterns, SOLID, duplication, complexity, error handling |
| Architecture | 10% | Design, modularity, scalability, coupling/cohesion, API design |
| Documentation | 10% | Completeness, clarity, accuracy, examples, troubleshooting |
| Usability | 10% | Learning curve, installation ease, error messages, ergonomics |
| Performance | 8% | Speed, resource usage, caching, bundle size, Core Web Vitals |
| Security | 10% | OWASP Top 10, input validation, auth, secrets, dependencies |
| Testing | 8% | Coverage (unit/integration/e2e), quality, automation, organization |
| Maintainability | 8% | Technical debt, readability, refactorability, versioning |
| Developer Experience | 10% | Setup ease, debugging, tooling, hot reload, IDE integration |
| Accessibility | 8% | WCAG compliance, keyboard nav, screen readers, cognitive load |
| CI/CD | 5% | Automation, pipelines, deployment, rollback, monitoring |
| Innovation | 3% | Novel approaches, forward-thinking design, unique value |
Audit Phases
| Phase | Name | Purpose |
|---|---|---|
| 0 | Resource Completeness | Verify registry/filesystem parity; audit fails if this fails |
| 1 | Discovery | Read docs, examine code, test system, review supporting materials |
| 2 | Evaluation | Score each dimension with evidence, strengths, and weaknesses |
| 3 | Synthesis | Executive summary, detailed scores, recommendations, risk matrix |
Scoring Scale
| Score | Rating | Meaning |
|---|---|---|
| 10 | Exceptional | Industry-leading, sets new standards |
| 8-9 | Excellent | Exceeds expectations significantly |
| 6-7 | Good | Meets expectations with improvements needed |
| 5 | Acceptable | Below average, significant improvements |
| 3-4 | Poor | Major gaps and fundamental problems |
| 1-2 | Critical | Barely functional or non-functional |
Common Mistakes
| Mistake | Correct Pattern |
|---|---|
| Giving inflated scores without evidence | Every score must cite specific files, metrics, or code examples as evidence |
| Skipping Phase 0 resource completeness check | Always verify registry completeness first; missing resources cap the overall score at 6/10 |
| Evaluating only code quality, ignoring dimensions | Score all 12 dimensions with appropriate weights; architecture, security, and DX matter equally |
| Accepting superficial "LGTM" reviews | Perform deep semantic audits checking contract integrity, security sanitization, and performance hygiene |
| Trusting AI-generated code without verification | Apply the verification gap protocol: critic agents, verifiable goals, human oversight for critical paths |
| Proceeding after audit failure without re-audit | Stop, analyze the deviation, remediate, then restart the checklist from step 1 |
| Using 10/10 scores without exceptional evidence | Reserve 10/10 for truly industry-leading work; most quality tools score 6-7 |
| Surface-level static analysis only | Combine linting with architectural fit checks, risk-based PR categorization, and context-aware validation |
Delegation
- Discover codebase structure and gather audit evidence: Use
Exploreagent to survey file organization, dependencies, test coverage, and documentation - Execute targeted quality checks across dimensions: Use
Taskagent to run linters, security scanners, performance profilers, and accessibility audits - Design quality improvement roadmap: Use
Planagent to prioritize quick wins, short-term, and long-term recommendations from audit findings
For stylistic cleanup of AI-generated prose and code (emdash overuse, slop vocabulary, over-commenting, verbose naming), use the de-slopify skill.>
If the usability-tester skill is available, delegate usability dimension evaluation and user flow validation to it.Otherwise, recommend: pnpm dlx skills add oakoss/agent-skills -s usability-tester -a claude-code -yReferences
- Audit Rubric -- pass/warn/fail thresholds, weighted scoring methodology, automated vs manual checklists, score caps, report format
- Dimension Rubrics -- detailed scoring criteria, evidence requirements, and rubric tables for all 12 dimensions
- Audit Report Template -- structured report format, executive summary, recommendations, risk assessment
- Anti-Patterns Guide -- code, architecture, security, testing, and process anti-patterns to identify during audits
- Verification Gap Protocol -- AI code verification methodology, critic agents, rejection protocol, risk-based review strategies
Anti-Patterns Guide
A catalog of anti-patterns to identify during quality audits, organized by category. Each anti-pattern includes detection strategies and severity levels.
Code Anti-Patterns
| Anti-Pattern | Signal | Severity |
|---|---|---|
| God objects | Single class/module doing everything | High |
| Spaghetti code | No clear flow, tangled dependencies | High |
| Copy-paste programming | Duplicated blocks with minor variations | Medium |
| Magic numbers | Unexplained literals in logic | Medium |
| Global state abuse | Mutable shared state without controls | High |
| Deep nesting | 4+ levels of indentation | Medium |
| Long methods | Functions exceeding 50 lines | Medium |
| Primitive obsession | Using primitives instead of value types | Low |
| Feature envy | Method uses more of another class's data | Medium |
| Dead code | Unreachable or unused code paths | Low |
| Type bypassing | any and @ts-ignore causing drift | High |
| Debug artifacts | console.log and test scaffolding left | Medium |
Detection Strategies
# Find functions over 50 lines (approximate)
rg -c "^}" -g "*.ts" | sort -t: -k2 -rn | head -20
# Find potential magic numbers
rg "\b(?:100|200|404|500|1000|3600|86400)\b" -g "*.ts" --stats
# Find duplicated string literals
rg -o '"[^"]{10,}"' -g "*.ts" --no-filename | sort | uniq -c | sort -rn | head -10
# Find deeply nested code
rg "^\s{16,}" -g "*.ts" -c | sort -t: -k2 -rn
# Find type bypassing
rg "(: any|as any|@ts-ignore|@ts-expect-error)" -g "*.ts" --stats
# Find debug artifacts
rg "console\.(log|debug|info|warn)" -g "*.ts" -g "!*.test.*" -g "!*.spec.*"Architecture Anti-Patterns
| Anti-Pattern | Signal | Severity |
|---|---|---|
| Tight coupling | Changes cascade across modules | High |
| Circular dependencies | A depends on B depends on A | High |
| Missing abstractions | Implementation details leak across boundaries | Medium |
| Over-engineering | Unnecessary complexity for simple problems | Medium |
| Anemic domain model | Data structures with no behavior | Medium |
| Big ball of mud | No discernible architecture pattern | Critical |
| Leaky abstraction | Internal details exposed through interface | Medium |
| Vendor lock-in | Core logic depends on specific vendor APIs | High |
| Paradigm mixing | Inconsistent patterns confuse humans and AI | Medium |
Detection Strategies
# Find files importing from many different modules (high fan-out)
rg -c "^import" -g "*.ts" | sort -t: -k2 -rn | head -20
# Find circular imports (approximate -- look for mutual references)
rg "^import.*from" -g "*.ts" --no-heading | sort > /tmp/imports.txt
# Find barrel files that re-export everything (coupling amplifier)
rg "^export \* from" -g "index.ts" -g "index.tsx"Security Anti-Patterns
| Anti-Pattern | Signal | Severity |
|---|---|---|
| Hardcoded secrets | API keys, passwords in source | Critical |
| SQL injection | Unsanitized user input in queries | Critical |
| XSS vulnerabilities | Unescaped output in templates | High |
| Missing authentication | Endpoints without auth checks | Critical |
| Insecure defaults | Debug mode in production | High |
| Excessive permissions | Over-scoped API keys or roles | Medium |
| Missing rate limiting | Endpoints without throttling | Medium |
| Logging sensitive data | PII or secrets in log output | High |
| Missing input validation | No schema validation on user data | High |
Detection Strategies
# Find potential hardcoded secrets
rg -i "(api.?key|secret|password|token)\s*[:=]" -g "*.ts" -g "!*.test.*"
# Find potential SQL injection
rg "query\(.*\$\{" -g "*.ts"
rg "execute\(.*\+" -g "*.ts"
# Find missing auth checks in API routes
rg "export (async )?function (GET|POST|PUT|DELETE)" -g "*/api/**/*.ts" -l
# Find potential XSS in React (dangerouslySetInnerHTML)
rg "dangerouslySetInnerHTML" -g "*.tsx"
# Check for sensitive data in logs
rg "console\.\w+\(.*(?:password|secret|token|key)" -g "*.ts" -iTesting Anti-Patterns
| Anti-Pattern | Signal | Severity |
|---|---|---|
| No tests | Zero coverage | Critical |
| Flaky tests | Non-deterministic pass/fail | High |
| Test duplication | Same scenario tested multiple ways | Medium |
| Testing implementation | Tests break on refactor without behavior change | High |
| Missing edge cases | Only happy path tested | Medium |
| Slow tests | Test suite takes >10 minutes | Medium |
| Test pollution | Tests depend on execution order | High |
| Over-mocking | Tests verify mocks, not behavior | Medium |
| Snapshot abuse | Snapshots used as primary assertion strategy | Medium |
| Missing integration tests | Only unit tests, no integration coverage | High |
Detection Strategies
# Check test coverage
npx vitest --coverage --reporter=json
# Find tests without assertions
rg "it\(|test\(" -A 20 -g "*.test.*" | rg -v "expect\(|assert"
# Find snapshot-heavy test files
rg "toMatchSnapshot|toMatchInlineSnapshot" -c -g "*.test.*" | sort -t: -k2 -rn
# Find tests with timeouts (flaky test signal)
rg "setTimeout|waitFor|sleep" -g "*.test.*" -c | sort -t: -k2 -rnProcess Anti-Patterns
| Anti-Pattern | Why It Fails | Correct Alternative |
|---|---|---|
| "LGTM" mentality | Superficial review hides deep bugs | Deep semantic audit |
| Bypassing types | any and @ts-ignore cause drift | Total type integrity |
| Mixing paradigms | Confuses AI context and humans | Strict pattern consistency |
| Silent delivery | User does not know what was validated | Verification reporting |
| Debt for speed | "We'll fix it later" = never | Zero-debt policy |
| No code ownership | Nobody responsible for quality | Clear ownership model |
| Review without context | Reviewer lacks domain knowledge | Domain-expert review |
| AI trust without check | AI-generated code accepted blindly | Verification gap protocol |
| Score inflation | Giving high scores without evidence | Evidence-based scoring only |
Pre-Commit Audit Checklist
Before declaring any task finished, verify:
1. Contract integrity -- code matches defined interfaces (Zod schemas, TypeScript types) 2. Architectural alignment -- idiomatic patterns for the stack in use 3. Security sanitization -- all inputs validated, no secrets in logs or code 4. Performance hygiene -- no N+1 queries, images optimized, bundle size checked 5. Cleanliness audit -- no console.log, debug artifacts, or commented-out code 6. Traceability -- the "why" is documented for non-obvious decisions (comments or ADRs)
Risk-Based PR Categorization
Categorize changes by risk level to allocate review effort appropriately:
| Risk Level | Examples | Review Strategy |
|---|---|---|
| Critical | Auth logic, payment flows, schema migrations | Human expert review |
| High | API contracts, cross-service changes | Senior engineer review |
| Medium | Feature code, UI components | Standard code review |
| Low | Config changes, dependency bumps, typo fixes | Automated review + spot check |
Route high-risk changes to domain experts. Auto-approve low-risk, well-scoped changes that pass automated gates. Flag unrelated changes bundled in the same PR for separation.
Audit Report Template
Use this template structure when producing quality audit reports. Every section requires evidence -- no scores without justification.
Phase 0: Resource Completeness Check
This check is mandatory before scoring. The audit must fail if resources are incomplete.
Verification steps:
1. Count resources in directories vs registry entries (must match) 2. Verify all resources in filesystem are discoverable through the registry 3. Check cross-references point to existing entries 4. Verify CLI reads from registry (not hardcoded data)
Critical failure conditions (cap overall score at 6/10):
- Registry missing >10% of resources from directories
- README documents resources not in registry
- CLI uses mock/hardcoded data instead of registry
- Cross-references point to non-existent resources
Report Template
# Quality Audit Report: [Tool/System Name]
**Date:** [Date]
**Version Audited:** [Version]
**Auditor:** [Agent/Skill name]
---
## Executive Summary
**Overall Score:** [X.X]/10 - [Rating]
**Rating Scale:**
- 9.0-10.0: Exceptional
- 8.0-8.9: Excellent
- 7.0-7.9: Very Good
- 6.0-6.9: Good
- 5.0-5.9: Acceptable
- Below 5.0: Needs Improvement
**Key Strengths:**
1. [Strength 1 with evidence -- file path, metric, or example]
2. [Strength 2 with evidence]
3. [Strength 3 with evidence]
**Critical Areas for Improvement:**
1. [Weakness 1 with specific file references]
2. [Weakness 2 with specific file references]
3. [Weakness 3 with specific file references]
**Recommendation:** [Exceptional / Excellent / Very Good / Good / Acceptable / Needs Improvement]
---
## Detailed Scores
| Dimension | Weight | Score | Rating | Priority |
| -------------------- | ------ | ----- | -------- | ----------------- |
| Code Quality | 10% | X/10 | [Rating] | [High/Medium/Low] |
| Architecture | 10% | X/10 | [Rating] | [High/Medium/Low] |
| Documentation | 10% | X/10 | [Rating] | [High/Medium/Low] |
| Usability | 10% | X/10 | [Rating] | [High/Medium/Low] |
| Performance | 8% | X/10 | [Rating] | [High/Medium/Low] |
| Security | 10% | X/10 | [Rating] | [High/Medium/Low] |
| Testing | 8% | X/10 | [Rating] | [High/Medium/Low] |
| Maintainability | 8% | X/10 | [Rating] | [High/Medium/Low] |
| Developer Experience | 10% | X/10 | [Rating] | [High/Medium/Low] |
| Accessibility | 8% | X/10 | [Rating] | [High/Medium/Low] |
| CI/CD | 5% | X/10 | [Rating] | [High/Medium/Low] |
| Innovation | 3% | X/10 | [Rating] | [High/Medium/Low] |
**Overall Score:** [Weighted Average]/10
---
## Dimension Analysis
### [N]. [Dimension Name]: [Score]/10
**Rating:** [Excellent/Good/Acceptable/Poor]
**Strengths:**
- [Specific strength with file reference]
**Weaknesses:**
- [Specific weakness with file reference]
**Evidence:**
- [Code examples, metrics, tool output]
**Improvements:**
1. [Specific actionable improvement]
[Repeat for all 12 dimensions]
---
## Recommendations
### Immediate Actions (Quick Wins) -- 1-2 weeks
1. **[Action]**
- Impact: High
- Effort: Low
- Timeline: [X] days
### Short-term Improvements -- 1-3 months
1. **[Improvement]**
- Impact: Medium-High
- Effort: Medium
- Timeline: [X] weeks
### Long-term Strategic -- 3-12 months
1. **[Strategic improvement]**
- Impact: High
- Effort: High
- Timeline: [X] months
---
## Risk Assessment
### High-Risk Issues
**[Issue]:**
- Risk Level: Critical/High
- Impact: [Description]
- Mitigation: [Specific steps]
### Medium-Risk Issues
[List with same format]
### Low-Risk Issues
[List with same format]
---
## Comparative Analysis
| Feature/Aspect | [This Tool] | [Leader 1] | [Leader 2] |
| -------------- | ----------- | ---------- | ---------- |
| [Aspect] | [Score] | [Score] | [Score] |
---
## Quality Metrics
| Metric | Result | Target | Status |
| ---------------- | ------ | -------- | ----------- |
| Code Coverage | [X]% | 80%+ | [Pass/Fail] |
| Complexity Avg | [X] | <15 | [Pass/Fail] |
| Dependency Vulns | [X] | 0 high | [Pass/Fail] |
| Bundle Size | [X] KB | [Budget] | [Pass/Fail] |
---
## Conclusion
[Summary of findings, overall assessment, final recommendation]
**Final Verdict:** [Detailed recommendation with next steps]Weighted Score Calculation
Calculate the overall score using dimension weights:
Overall = (CodeQuality * 0.10) + (Architecture * 0.10) + (Documentation * 0.10)
+ (Usability * 0.10) + (Performance * 0.08) + (Security * 0.10)
+ (Testing * 0.08) + (Maintainability * 0.08) + (DX * 0.10)
+ (Accessibility * 0.08) + (CICD * 0.05) + (Innovation * 0.03)Round to one decimal place. Apply any caps (Phase 0 failure caps at 6.0).
Evidence Standards
Every score must include at least one of:
- File reference -- specific path and line number
- Metric -- quantitative measurement from a tool
- Code example -- concrete snippet demonstrating the issue
- Tool output -- scan results, coverage reports, benchmark data
Scores without evidence are invalid. If a dimension cannot be evaluated (e.g., no UI means accessibility is not applicable), note it as "N/A" and redistribute the weight proportionally.
The Rejection Protocol
If the audit fails any critical check:
1. Stop -- do not proceed with the commit or final report 2. Analyze -- identify the specific deviation from standards 3. Remediate -- apply the fix immediately 4. Re-audit -- restart the checklist from Phase 0
Audit Rubric
A structured rubric for executing quality audits with clear pass/warn/fail thresholds, weighted scoring, and a distinction between automated and manual audit items.
Pass / Warn / Fail Thresholds
Each dimension receives a 1-10 score, then maps to a threshold:
| Threshold | Score Range | Meaning | Action Required |
|---|---|---|---|
| Pass | 7-10 | Meets or exceeds standards | No action |
| Warn | 5-6 | Below standard but functional | Improvement plan required |
| Fail | 1-4 | Critical gaps, blocks release or further work | Immediate remediation |
| N/A | -- | Dimension does not apply (e.g., no UI for a11y) | Redistribute weight |
Hard fail conditions (override any score):
- Phase 0 resource completeness fails -- cap overall at 6.0
- Any security dimension score of 3 or below -- flag as release blocker
- Any critical anti-pattern found without a mitigation plan -- fail the audit
Dimension Rubric Summary
Each dimension uses consistent criteria across three levels. See dimension-rubrics.md for full per-dimension detail.
Code Quality (10%)
| Level | Criteria |
|---|---|
| Pass | Consistent patterns, low duplication, complexity under limits |
| Warn | Some code smells, moderate duplication, complexity hotspots |
| Fail | No patterns, widespread duplication, god objects |
Architecture (10%)
| Level | Criteria |
|---|---|
| Pass | Clear module boundaries, low coupling, scalable design |
| Warn | Some coupling, unclear boundaries in non-critical areas |
| Fail | Circular dependencies, tight coupling, no separation of concern |
Documentation (10%)
| Level | Criteria |
|---|---|
| Pass | All features documented, examples work, accurate to code |
| Warn | Major features documented, some gaps, some stale examples |
| Fail | Missing docs for critical paths, misleading or no examples |
Usability (10%)
| Level | Criteria |
|---|---|
| Pass | Under 10 minutes to first success, clear error messages |
| Warn | 10-30 minutes to first success, some confusing error messages |
| Fail | Over 30 minutes to first success, cryptic errors, no defaults |
Performance (8%)
| Level | Criteria |
|---|---|
| Pass | Within performance budget, no N+1 queries, acceptable Core Web Vitals |
| Warn | Occasional slow paths, minor budget overruns, no profiling |
| Fail | Unusable latency, resource exhaustion, no optimization |
Security (10%)
| Level | Criteria |
|---|---|
| Pass | OWASP Top 10 addressed, inputs validated, no hardcoded secrets |
| Warn | Most inputs validated, minor dependency vulnerabilities |
| Fail | Hardcoded secrets, SQL injection, missing auth on endpoints |
Testing (8%)
| Level | Criteria |
|---|---|
| Pass | 80%+ coverage, unit/integration/e2e present, tests verify behavior |
| Warn | 60-79% coverage, mostly unit tests, some flaky tests |
| Fail | Under 40% coverage, no integration tests, tests verify mocks |
Maintainability (8%)
| Level | Criteria |
|---|---|
| Pass | Low tech debt, clear contribution path, semver compliance |
| Warn | Moderate debt, some outdated deps, unclear contribution path |
| Fail | High debt, abandoned dependencies, no versioning strategy |
Developer Experience (10%)
| Level | Criteria |
|---|---|
| Pass | Under 5 minutes setup, fast feedback loop, good error DX |
| Warn | 5-15 minutes setup, some friction, basic error messages |
| Fail | Over 15 minutes setup, broken tooling, no debugging support |
Accessibility (8%)
| Level | Criteria |
|---|---|
| Pass | WCAG 2.1 AA compliant, keyboard navigable, screen reader OK |
| Warn | Partial WCAG compliance, keyboard works for main flows |
| Fail | No WCAG compliance, keyboard traps, inaccessible content |
CI/CD (5%)
| Level | Criteria |
|---|---|
| Pass | Automated build, test, deploy pipeline with rollback |
| Warn | Automated build and test, manual deploy |
| Fail | No automation, manual build and deploy |
Innovation (3%)
| Level | Criteria |
|---|---|
| Pass | Novel approach or unique value proposition |
| Warn | Conventional but competent implementation |
| Fail | Derivative with no differentiation (rarely blocks) |
Weighted Scoring Methodology
Calculating the Overall Score
Multiply each dimension score by its weight, then sum:
Overall = (CodeQuality * 0.10) + (Architecture * 0.10) + (Documentation * 0.10)
+ (Usability * 0.10) + (Performance * 0.08) + (Security * 0.10)
+ (Testing * 0.08) + (Maintainability * 0.08) + (DX * 0.10)
+ (Accessibility * 0.08) + (CICD * 0.05) + (Innovation * 0.03)Round to one decimal place.
Handling N/A Dimensions
When a dimension does not apply, redistribute its weight proportionally:
Example: CLI tool with no UI -- Accessibility (8%) is N/A
Remaining weight = 100% - 8% = 92%
Adjusted weights = each remaining dimension weight / 0.92
Code Quality: 10% / 0.92 = 10.9%
Security: 10% / 0.92 = 10.9%
...Weight Adjustments by Project Type
Customize weights based on what matters most:
| Project Type | Increase Weight | Decrease Weight |
|---|---|---|
| Public API | Security, Docs, DX | Innovation, A11y |
| Consumer Web | A11y, Perf, UX | CI/CD, Innovation |
| Internal Tool | DX, Maintainability | A11y, Innovation |
| Library/Package | Architecture, Tests | Usability, CI/CD |
| CLI Tool | DX, Usability, Docs | A11y, Perf |
Document any weight adjustments in the audit report with justification.
Score Caps
| Condition | Cap |
|---|---|
| Phase 0 resource completeness fails | 6.0 |
| Any dimension scores 1-2 | 7.0 |
| Security scores 3 or below | 5.0 |
| No tests exist | 6.0 |
Apply the lowest applicable cap.
Automated vs Manual Audit Items
Automated Checks (Run First)
These checks produce objective, repeatable results. Run them before manual review.
| Check | Tool / Command | Pass Criteria |
|---|---|---|
| Type safety | npx tsc --noEmit --strict | Zero errors |
| Lint | npx eslint . --max-warnings 0 | Zero errors |
| Format | npx prettier --check . | No formatting diffs |
| Tests | npx vitest run --coverage | All pass, coverage target |
| Security deps | npm audit --audit-level=high | No high/critical vulns |
| Bundle size | Build output size check | Within budget |
| Hardcoded secrets | `rg -i "(api.?key\ | secret\ |
| Dead code | `rg "console\\.(log\ | debug)" -g ".ts" -g "!.test.*"` |
| Complexity | SonarQube or custom threshold | Under configured limit |
Manual Review Items
These require human or AI judgment. Perform after automated checks pass.
| Check | What to Evaluate |
|---|---|
| Architectural fit | Does the code follow the system's design direction? |
| Requirement alignment | Does implementation match what was requested? |
| Convention consistency | Does code follow established team patterns? |
| API design quality | Are endpoints/interfaces intuitive, consistent, versioned? |
| Error handling adequacy | Are error paths thoughtful, not just catch-and-log? |
| Edge case coverage | Are boundary conditions and failure modes handled? |
| Documentation accuracy | Does documentation match the actual implementation? |
| Future maintainability | Will this code be easy to modify in six months? |
| Business logic correctness | Do calculations and rules match requirements? |
| Integration safety | Does the change interact correctly with existing code? |
Audit Execution Order
1. Run all automated checks and collect results 2. If any automated check fails, stop and remediate before manual review 3. Perform manual review items, scoring each dimension 4. Calculate weighted overall score 5. Apply score caps if any hard-fail conditions triggered 6. Generate the audit report
Audit Report Format
Structure every audit report with these sections in order:
# Quality Audit Report: [Subject]
**Date:** [Date]
**Version:** [Version]
**Auditor:** [Agent or person]
## Executive Summary
**Overall Score:** [X.X]/10 - [Pass/Warn/Fail]
**Top 3 Strengths:** [With evidence]
**Top 3 Weaknesses:** [With evidence]
**Recommendation:** [Exceptional / Excellent / Good / Acceptable / Needs Improvement]
## Automated Gate Results
| Gate | Result | Details |
| ---- | ------ | ------- |
## Dimension Scores
| Dimension | Weight | Score | Level | Evidence Summary |
| --------- | ------ | ----- | ----- | ---------------- |
**Weighted Overall:** [X.X]/10
**Caps Applied:** [None / list]
## Findings by Priority
### Critical (Fix Before Release)
### High (Fix This Sprint)
### Medium (Plan for Next Sprint)
### Low (Backlog)
## Recommendations
### Quick Wins (High Impact / Low Effort)
### Short-Term (1-3 Months)
### Long-Term (3-12 Months)
## Risk Matrix
| Risk | Likelihood | Impact | Mitigation |
| ---- | ---------- | ------ | ---------- |See audit-report-template.md for the full template with all subsections.
Calibration Guidelines
To maintain consistent scoring across audits:
- 10/10 is rare -- reserved for industry-leading implementations (Linux kernel code quality, Stripe documentation)
- Most quality software scores 6-7 -- this is the realistic baseline
- A score of 8+ means genuinely above average -- requires strong evidence
- Compare against industry leaders, not team averages or personal standards
- Every score needs evidence -- file paths, metrics, tool output, or concrete examples
- Scores without evidence are invalid -- reject them during review
Dimension Rubrics
Each of the 12 dimensions requires specific evaluation criteria, evidence, and a scoring rubric. Weights total 100% and can be adjusted based on the project type and priorities.
Scoring Principles
- Compare against industry leaders, not average tools
- Reference established standards (OWASP, WCAG, IEEE, ISO)
- Consider real-world usage and edge cases
- Provide specific examples for each score -- cite file paths, line numbers, metrics
- 10/10 is rare -- reserved for truly exceptional, industry-leading work
- Most quality tools score 6-7 -- this is the realistic baseline
1. Code Quality (10%)
Evaluate: Structure, naming conventions, duplication, cyclomatic/cognitive complexity, error handling, code smells, design patterns, SOLID principles.
| Score | Criteria |
|---|---|
| 10 | Perfect structure, zero duplication, excellent patterns |
| 8 | Well-structured, minimal issues, good patterns |
| 6 | Acceptable structure, some code smells |
| 4 | Poor structure, significant technical debt |
| 2 | Chaotic, unmaintainable code |
Evidence required:
- Specific file examples with line references
- Complexity metrics (cyclomatic, cognitive)
- Pattern identification (design patterns used or missing)
- Duplication analysis results
Key checks:
# Find functions over 50 lines (approximate complexity signal)
rg -c "^}" -g "*.ts" | sort -t: -k2 -rn | head -20
# Find potential magic numbers
rg "\b(?:100|200|404|500|1000|3600|86400)\b" -g "*.ts" --stats
# Find duplicated string literals
rg -o '"[^"]{10,}"' -g "*.ts" --no-filename | sort | uniq -c | sort -rn | head -102. Architecture (10%)
Evaluate: System design, modularity, separation of concerns, scalability, dependency management, API design, data flow, coupling/cohesion, architectural patterns.
| Score | Criteria |
|---|---|
| 10 | Exemplary, highly scalable, perfect modularity |
| 8 | Solid separation, scalable design |
| 6 | Adequate, some coupling issues |
| 4 | High coupling, not scalable |
| 2 | Fundamentally flawed architecture |
Evidence required:
- Component analysis and dependency graph
- Coupling/cohesion assessment
- Architectural fitness for stated requirements
- API design consistency
Key checks:
# Find files importing from many different modules (high fan-out)
rg -c "^import" -g "*.ts" | sort -t: -k2 -rn | head -20
# Find deeply nested code (architectural complexity signal)
rg "^\s{16,}" -g "*.ts" -c | sort -t: -k2 -rn3. Documentation (10%)
Evaluate: Completeness, clarity, accuracy, organization, practical examples, API docs, troubleshooting guides, architecture docs.
| Score | Criteria |
|---|---|
| 10 | Thorough, crystal clear, excellent examples |
| 8 | Very good coverage, clear, good examples |
| 6 | Adequate coverage, some gaps |
| 4 | Poor coverage, confusing, lacks examples |
| 2 | Minimal or misleading documentation |
Evidence required:
- Documentation inventory (what exists vs what should exist)
- Missing sections identified
- Quality assessment of examples (do they work?)
- Accuracy check (does documentation match implementation?)
4. Usability (10%)
Evaluate: Learning curve, installation ease, configuration complexity, workflow efficiency, error message quality, defaults, command/API ergonomics.
| Score | Criteria |
|---|---|
| 10 | Zero friction, delightful UX |
| 8 | Minimal learning curve |
| 6 | Usable but requires learning |
| 4 | Steep learning curve, difficult |
| 2 | Nearly unusable |
Evidence required:
- Time-to-first-success measurement
- Pain points identified during testing
- Error message quality assessment
- Default configuration adequacy
5. Performance (8%)
Evaluate: Execution speed, resource usage (CPU, memory), startup time, scalability under load, optimization techniques, caching, database queries, bundle size.
| Score | Criteria |
|---|---|
| 10 | Blazingly fast, minimal resources, highly optimized |
| 8 | Very fast, efficient resource usage |
| 6 | Acceptable performance |
| 4 | Slow, resource-heavy |
| 2 | Unusably slow, resource exhaustion |
Evidence required:
- Benchmarks (startup time, operation latency)
- Resource measurements (memory, CPU)
- Bottleneck identification
- N+1 query detection (if applicable)
- Bundle size analysis (if applicable)
6. Security (10%)
Evaluate: Vulnerability assessment, input validation, authentication/authorization, data encryption, dependency vulnerabilities, secret management, OWASP Top 10.
| Score | Criteria |
|---|---|
| 10 | Zero vulnerabilities, exemplary practices |
| 8 | Very secure, minor concerns |
| 6 | Adequate, some issues |
| 4 | Significant vulnerabilities |
| 2 | Critical security flaws |
Evidence required:
- Vulnerability scan results (npm audit, Snyk, Semgrep)
- OWASP Top 10 checklist completion
- Input validation coverage
- Secret management audit
Key checks:
# Find potential hardcoded secrets
rg -i "(api.?key|secret|password|token)\s*[:=]" -g "*.ts" -g "!*.test.*"
# Find potential SQL injection
rg "query\(.*\$\{" -g "*.ts"
# Check dependency vulnerabilities
npm audit --json | jq '.vulnerabilities | length'7. Testing (8%)
Evaluate: Coverage (unit/integration/e2e), test quality, automation, CI integration, test organization, mocking strategies, performance tests, security tests.
| Score | Criteria |
|---|---|
| 10 | Thorough, automated, >90% coverage |
| 8 | Very good, automated, >80% coverage |
| 6 | Adequate, >60% coverage |
| 4 | Poor, <40% coverage |
| 2 | Minimal or no tests |
Evidence required:
- Coverage reports with branch/line/function breakdown
- Test inventory (unit vs integration vs e2e ratio)
- Test quality assessment (do tests verify behavior or implementation?)
- CI integration status
8. Maintainability (8%)
Evaluate: Technical debt, code readability, refactorability, modularity, developer documentation, contribution guidelines, code review process, versioning strategy.
| Score | Criteria |
|---|---|
| 10 | Zero debt, highly maintainable, excellent guidelines |
| 8 | Low debt, easy to maintain |
| 6 | Moderate debt, maintainable |
| 4 | High debt, difficult to maintain |
| 2 | Unmaintainable |
Evidence required:
- Technical debt analysis (TODO/FIXME count, deprecated API usage)
- Contribution difficulty assessment
- Versioning strategy evaluation (semver compliance)
- Dependency freshness (outdated packages)
9. Developer Experience (10%)
Evaluate: Setup ease, debugging experience, error messages, tooling support, hot reload/fast feedback, CLI ergonomics, IDE integration.
| Score | Criteria |
|---|---|
| 10 | Delightful to work with |
| 8 | Very productive |
| 6 | Some friction |
| 4 | Frustrating |
| 2 | Actively hostile |
Evidence required:
- Setup time measurement (clone to running)
- Developer pain points during evaluation
- Tooling assessment (TypeScript, linting, formatting)
- Feedback loop speed (save to result)
10. Accessibility (8%)
Evaluate: WCAG compliance, keyboard navigation, screen reader support, color contrast, cognitive load, inclusive design.
| Score | Criteria |
|---|---|
| 10 | Universally accessible |
| 8 | Highly accessible, inclusive |
| 6 | Meets accessibility standards |
| 4 | Poor accessibility |
| 2 | Inaccessible to many users |
Evidence required:
- WCAG 2.2 AA/AAA audit results
- Keyboard navigation testing
- Screen reader compatibility
- Color contrast ratios
11. CI/CD (5%)
Evaluate: Automation level, build pipeline, testing automation, deployment automation, release process, monitoring/alerts, rollback capabilities.
| Score | Criteria |
|---|---|
| 10 | Fully automated, zero-touch deployments |
| 8 | Highly automated, minimal manual steps |
| 6 | Partially automated |
| 4 | Mostly manual |
| 2 | No automation |
Evidence required:
- Pipeline configuration review
- DORA metrics assessment (deployment frequency, lead time for changes, change failure rate, time to restore service, reliability)
- Rollback capability verification
12. Innovation (3%)
Evaluate: Novel approaches, creative solutions, forward-thinking design, industry leadership, unique value proposition.
| Score | Criteria |
|---|---|
| 10 | Groundbreaking, sets new standards |
| 8 | Pushes boundaries |
| 6 | Some innovation |
| 4 | Mostly conventional |
| 2 | No innovation |
Evidence required:
- Novel features or approaches identified
- Comparison with alternatives in the ecosystem
- Industry impact or influence assessment
Special Evaluation Criteria
Developer Tools
Additional factors: setup time (<5 min = 10/10), error message quality, debugging experience, community support, IDE integration depth.
Frameworks and Libraries
Additional factors: bundle size, tree-shaking support, TypeScript support (strict mode), browser compatibility, migration path from previous versions.
CLI Tools
Additional factors: one-command simplicity, automatic defaults, clear visual feedback (progress indicators, colors), minimal required decisions, forgiving design (easy undo, backups).
Industry Benchmarks
| Dimension | Industry Leader |
|---|---|
| Code Quality | Linux kernel, SQLite |
| Documentation | Stripe, Tailwind CSS |
| Usability | Vercel, Netlify |
| Developer Experience | Vite, Next.js |
| Testing | Playwright, Vitest |
| Security | 1Password, Signal |
| Architecture | PostgreSQL, Redis |
Standards Referenced
- Code Quality: Clean Code (Martin), Code Complete (McConnell), SonarQube quality gates
- Architecture: Clean Architecture (Martin), Domain-Driven Design (Evans)
- Security: OWASP Top 10, SANS Top 25, CWE/SANS
- Accessibility: WCAG 2.2 (AA/AAA), inclusive design guidelines
- Testing: Test Pyramid (Cohn), 80% minimum coverage target
- Performance: Core Web Vitals, RAIL model (Google), performance budgets
Verification Gap Protocol
AI-assisted development accounts for a significant and growing share of committed code. The bottleneck has shifted from writing code to reviewing, validating, and governing it. This protocol ensures velocity does not compromise integrity.
The Verification Gap
The verification gap is the distance between what AI-generated code appears to do and what it actually does in production. AI output can produce functions that compile and pass basic tests yet introduce redundancy, fragile logic, security vulnerabilities, or compliance risks.
Key risks of unverified AI code:
- Compiles and runs but contains subtle logic errors
- Passes unit tests but fails under real-world edge cases
- Introduces architectural drift by not following project conventions
- Creates security vulnerabilities through incomplete input validation
- Adds unnecessary complexity or dead code paths
- Generates plausible but incorrect implementations of business rules
Closing the Gap
1. Critic Agent Pattern
Use high-reasoning models to audit the output of faster code-generation models. The critic agent reviews with a different perspective than the generator.
Critic agent responsibilities:
- Verify contract integrity (does output match interfaces, schemas, types?)
- Check architectural alignment (does it follow project patterns?)
- Validate security sanitization (are inputs validated, secrets protected?)
- Assess performance hygiene (N+1 queries, bundle size, resource usage)
- Confirm cleanliness (no debug artifacts, commented-out code, TODOs)
How to implement:
// Example: structured audit output from a critic agent
type AuditResult = {
dimension: string;
score: number;
evidence: string[];
issues: Array<{
severity: 'critical' | 'high' | 'medium' | 'low';
file: string;
line: number;
description: string;
suggestion: string;
}>;
};2. Verifiable Goals
Every PR must produce a measurable signal of success. No code merges without at least one verifiable gate passing.
Required signals:
| Gate | Tool | Pass Criteria |
|---|---|---|
| Type safety | TypeScript compiler | Zero errors in strict mode |
| Lint | ESLint / Biome | Zero errors |
| Tests | Vitest / Playwright | All pass, coverage met |
| Build | Bundler / compiler | Successful build |
| Security scan | npm audit / Snyk | No high/critical vulns |
| Format | Prettier / Biome | No formatting diffs |
Verification script example:
#!/usr/bin/env bash
set -euo pipefail
echo "Running verification gates..."
echo "1. Type check"
npx tsc --noEmit --strict
echo "2. Lint"
npx eslint . --max-warnings 0
echo "3. Tests"
npx vitest run --coverage
echo "4. Build"
npm run build
echo "5. Security audit"
npm audit --audit-level=high
echo "All gates passed."3. Human Oversight
Mandatory human sign-off for critical paths. Not all code requires the same level of review.
Critical paths requiring human review:
- Authentication and authorization logic
- Payment processing and financial calculations
- Data migration and schema changes
- Cryptographic operations
- Privacy-sensitive data handling (PII, HIPAA, GDPR)
- Infrastructure and deployment configuration
Standard paths for automated review:
- UI component styling
- Test additions for existing features
- Documentation updates
- Dependency version bumps (with passing gates)
- Configuration file changes
AI Code Standards
Excellence Over Mimicry
Do not repeat bad local patterns just because they exist in the codebase. AI-generated code should follow idiomatic standards for the stack, even if existing code does not.
// BAD: Mimicking a poor existing pattern found in the codebase
function getUser(id: any) {
const res: any = db.query(`SELECT * FROM users WHERE id = ${id}`);
return res;
}
// GOOD: Idiomatic, type-safe, parameterized
async function getUser(id: string): Promise<User | null> {
return db.user.findUnique({ where: { id } });
}No Black Boxes
Every complex function must explain its reasoning. AI-generated code is particularly prone to producing "it works" implementations without documenting non-obvious decisions.
// BAD: No explanation for the magic
function calculateDiscount(total: number): number {
return total > 100 ? total * 0.85 : total > 50 ? total * 0.9 : total;
}
// GOOD: Business rules documented
function calculateDiscount(total: number): number {
// Tiered discount per 2024 pricing agreement:
// - Orders over $100: 15% discount
// - Orders over $50: 10% discount
// - Below $50: no discount
if (total > 100) return total * 0.85;
if (total > 50) return total * 0.9;
return total;
}Metadata Tagging
Tag AI-generated files for future auditing. This helps track which code was generated vs hand-written, enabling targeted reviews during audits.
// @generated - AI-assisted implementation
// @audit-date - 2026-01-15
// @generator - claude-opus-4-5The Rejection Protocol
If the audit fails any check in the supreme checklist:
Step 1: Stop
Do not proceed with the commit, push, or report. A failed audit means the code is not ready.
Step 2: Analyze
Identify the specific deviation:
- Which gate failed?
- What is the root cause?
- Is this a pattern issue or an isolated problem?
Step 3: Remediate
Apply the fix immediately. Do not defer:
- Fix the type error, not suppress it
- Add the missing test, not skip the check
- Resolve the security issue, not add a TODO
- Correct the pattern, not document the workaround
Step 4: Re-Audit
Restart the checklist from step 1. A partial re-run is not sufficient because fixes can introduce new issues.
The Supreme Audit Checklist
Before any AI-generated code is accepted:
| Check | What to Verify |
|---|---|
| Contract integrity | Code matches defined interfaces (Zod, TypeScript types) |
| Architectural alignment | Idiomatic patterns for the stack (not cargo-culted) |
| Security sanitization | All inputs validated, no secrets in logs or code |
| Performance hygiene | No N+1 queries, images optimized, bundle size within budget |
| Cleanliness audit | No console.log, debug artifacts, commented-out code |
| Traceability | Non-obvious decisions documented with "why" comments |
| Test coverage | New code has tests, existing tests still pass |
| Dependency audit | No new high/critical vulnerabilities introduced |
Context-Aware Review
Surface-level static analysis catches formatting and basic lint issues but misses deeper problems. Effective AI code review must also assess:
- Architectural fit -- does the change align with the system's design direction?
- Requirement alignment -- does the implementation match what was actually requested?
- Convention consistency -- does the code follow the team's established patterns?
- Future maintainability -- will this code be easy to modify in six months?
- Integration safety -- does the change interact correctly with existing code?
Quality Metrics for AI Code
Track these metrics to measure the effectiveness of AI code review:
| Metric | Target | Why It Matters |
|---|---|---|
| Post-merge defect rate | <2% of AI-gen PRs | Catches issues the review missed |
| Review iteration count | <3 rounds per PR | Measures AI output quality |
| Time from PR to merge | <24 hours | Measures review bottleneck |
| Gate pass rate | >90% on first run | Measures code generation quality |
| Revert rate | <1% of merged PRs | Measures overall process reliability |