
Doc Completeness Audit
- 18 installs
- 28 repo stars
- Updated June 29, 2026
- nickcrew/claude-ctx-plugin
Helps with security tasks.
About
doc-completeness-audit is a Claude Code skill for security. It helps solo builders move faster with AI-assisted development.
- doc-completeness-audit
- Security
- AI-coding skill
Doc Completeness Audit by the numbers
- 18 all-time installs (skills.sh)
- Ranked #1,593 of 2,202 Security skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/nickcrew/claude-ctx-plugin --skill doc-completeness-auditAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 18 |
|---|---|
| repo stars | ★ 28 |
| Last updated | June 29, 2026 |
| Repository | nickcrew/claude-ctx-plugin ↗ |
What it does
Helps with security tasks.
Files
Documentation Completeness Audit
Determine whether a documentation set covers everything it should by building an inventory of what needs documenting and comparing it to what exists. The output is a prioritized gap report — not new documentation.
When to Use
- After shipping a feature — verify docs cover the new surface area
- Before a release — ensure no undocumented public APIs, CLI flags, or config options
- When users or new hires report "I couldn't find docs for X"
- Periodic health check on doc coverage
- After running
doc-maintenance(structural) anddoc-claim-validator(accuracy) to go wider
Quick Reference
| Resource | Purpose | Load when |
|---|---|---|
references/coverage-model.md | Defines what "complete" means per doc type | Always (Phase 1) |
---
Workflow Overview
Phase 1: Inventory → Build the "should exist" list from code and config
Phase 2: Map → Match inventory items to existing documentation
Phase 3: Classify → Score each gap by audience impact
Phase 4: Report → Produce the prioritized gap report---
Phase 1: Build the Inventory
Construct a list of everything that should be documented. Use four sources, checking all of them:
Source 1: Public Code Surface
Run the bundled inventory script to extract documentable surface area deterministically:
python3 skills/doc-completeness-audit/scripts/inventory.py --root . --json > inventory.json
# Or human-readable:
python3 skills/doc-completeness-audit/scripts/inventory.py --root .
# Run specific detectors only:
python3 skills/doc-completeness-audit/scripts/inventory.py --root . --detectors env_vars,cli_commandsThe script scans source files across Python, JavaScript/TypeScript, Rust, Go, Ruby, Java, and shell, extracting six categories:
| Detector | What it extracts |
|---|---|
env_vars | Environment variable references (os.environ, process.env, env::var, etc.) |
cli_commands | CLI commands and flags (argparse, click, clap, cobra, commander) |
config_keys | Configuration key access in config-related files |
http_endpoints | HTTP route definitions (Flask, FastAPI, Express, Actix, Axum, net/http) |
public_exports | Public module exports (__init__.py, export, pub fn, Go capitalized funcs) |
error_types | Custom error/exception class definitions |
| Event types, webhooks, callbacks | Every event name and payload shape |
Dispatch an Explore agent to scan for these signals. Provide it with the project's primary language and entry points.
Source 2: User-Facing Features
Identify features a user interacts with:
- TUI screens, views, keybindings
- CLI workflows (multi-step operations)
- Integration points (hooks, plugins, extensions)
- Authentication/authorization flows
- Error messages that imply user action
Source 3: Operational Surface
Identify what operators and maintainers need:
- Installation and setup procedures
- Upgrade and migration paths
- Backup and restore procedures
- Troubleshooting common errors
- Environment requirements and dependencies
- CI/CD integration points
Source 4: Existing Docs Cross-References
Check existing docs for promises of documentation that doesn't exist:
- "See [link]" references to pages that don't exist
- "Coming soon" or "TODO" markers
- Table of contents entries without corresponding pages
- Navigation entries without targets
Source 5: Architectural / Operational / Migration Topic Discovery (sonnet)
The first four sources catch code-detectable surface (env vars, CLI flags, endpoints, exported APIs, broken cross-references). They miss topics that exist as architectural patterns, user flows, ops procedures, or migration paths but don't surface as a single greppable symbol. Examples:
- Architectural patterns the system implements (CQRS, event sourcing,
saga) — should be documented but won't show up in inventory.py
- User flows implicit across UI surfaces — "how to share a project" may
span multiple components and isn't a single CLI command
- Migration paths between versions — typically tribal knowledge until
someone needs them
- Operational runbooks (incidents, rollbacks, capacity events)
- Recovery procedures and disaster scenarios
Dispatch one general-purpose + sonnet agent for topic discovery:
subagent_type: "general-purpose"
model: "sonnet"
description: "Architectural/operational topic discovery"Prompt: read README, top-level docs, and a sample of code (architecture files, integration boundaries, deployment configs, major feature directories). Identify topics that should be documented but aren't captured by the code-surface inventory. For each topic, name:
topic— what needs documenting (one phrase)evidence— what in the codebase implies this topic exists (path:line citations)audience— who would read this (operators, contributors, advanced users)type— reference, tutorial, guide, explanation, runbookconfidence— high (clear evidence), medium (inferred), low (speculative)
Append the agent's output to the inventory list before Phase 2.
Output: A structured inventory list. Each item has:
topic— what needs documentingsource— where the requirement was discovered (code path, config key, user flow, sonnet inference)audience— who needs this (end user, developer, operator)type— what kind of doc it needs (reference, tutorial, guide, explanation, runbook)confidence— high (deterministic) | medium | low (sonnet-inferred speculative)
---
Phase 2: Map to Existing Documentation (per-docfile sonnet dispatch)
For each inventory item, determine whether it's documented and how well. "Adequate coverage" requires reading surrounding context — a grep hit doesn't tell you whether the topic is truly explained vs. just mentioned in passing. Orchestrator-side execution would require reading every doc N times (once per inventory item), which strains the context window.
Dispatch strategy
Two-phase mapping:
1. Bulk grep pass (orchestrator) — for each inventory item, grep docs for the topic name. Build a candidate match map: which docs mention each topic. 2. Per-docfile sonnet pass — for each docfile that surfaced as a candidate match for any inventory item, dispatch one general-purpose + sonnet agent. The agent receives the doc + the list of inventory items that grep'd to this doc, and judges each as Documented / Shallow / Misplaced.
This keeps total agent calls ≈ N candidate docfiles (not N inventory items × M docs). For a typical project with 100 inventory items and 50 docs, the candidate map usually has 30–50 docs needing review.
Per-docfile prompt template
subagent_type: "general-purpose"
model: "sonnet"
description: "Coverage mapping for <docfile>"Prompt:
Read the doc at <DOCFILE_PATH>. The following inventory items grep-matched
this doc — judge each:
<INVENTORY_ITEMS_FOR_THIS_DOC>
For each item, classify as one of:
- Documented: dedicated section or page provides adequate coverage
- Shallow: mentioned but insufficient (missing examples, edge cases,
parameter listings; flag-in-table without explanation)
- Misplaced: covered, but in the wrong doc type for the audience (API
reference embedded in a tutorial; user-facing topic in dev-only docs)
- No real match: grep matched but the doc doesn't actually cover the topic
(incidental mention, different concept with the same word)
Output as YAML:
doc_path: <path>
items_reviewed: N
classifications:
- item: <topic>
classification: Documented | Shallow | Misplaced | No real match
section: <heading or line range where the topic is covered>
evidence: <quote or paraphrase of the relevant content>
gap: <if Shallow, what's missing; if Misplaced, where it should live>Items with no candidate match
Inventory items that grep'd 0 docs go directly to the "Missing" bucket without a sonnet review. The orchestrator handles these in Phase 3.
Why per-docfile rather than per-item
Per-item dispatch (one sonnet call per inventory item, reading every candidate doc fresh) blows up at any meaningful scale (100 items × 5 candidates = 500 calls). Per-docfile lets the agent see all related items in one pass and cross-reference within the doc — also higher precision than fragmented per-item judgments.
---
Phase 3: Classify Gaps by Impact
Not all gaps are equal. Score each gap using audience impact:
Priority Framework
| Priority | Criteria | Example |
|---|---|---|
| P0 | User cannot accomplish a core task without this | No installation guide, undocumented required config |
| P1 | User can work around it but wastes significant time | CLI flag exists but undocumented, error message without troubleshooting |
| P2 | Missing docs for secondary features or advanced use cases | Plugin API undocumented, advanced config options missing |
| P3 | Missing docs for edge cases or rarely used features | Obscure env var, deprecated feature migration path |
| P4 | Nice to have — explanatory content, design rationale | Architecture decision records, "why" behind defaults |
Audience Weighting
Apply a multiplier based on audience:
| Audience | Weight | Rationale |
|---|---|---|
| New users / onboarding | 1.5x | First impressions; high abandonment risk |
| Daily users | 1.0x | Core audience |
| Advanced users / contributors | 0.8x | Can read source when docs fail |
| Internal operators | 0.7x | Can ask the team |
A P2 gap for new users (P2 × 1.5 = 3.0) outranks a P1 gap for internal operators (P1 × 0.7 = 2.1).
---
Phase 4: Produce the Gap Report
Report Format
# Documentation Completeness Audit
**Audit date:** YYYY-MM-DD
**Scope:** [directories or doc sets audited]
**Inventory items:** N total
**Coverage:** N documented / N shallow / N missing / N misplaced
---
## Summary
[2-3 sentences: overall completeness assessment]
Coverage by audience:
| Audience | Documented | Shallow | Missing | Coverage % |
|----------|-----------|---------|---------|------------|
| New users | N | N | N | N% |
| Daily users | N | N | N | N% |
| Contributors | N | N | N | N% |
| Operators | N | N | N | N% |
---
## P0 Gaps — Blocking
| # | Topic | Audience | Source | Current State | What's Needed |
|---|-------|----------|--------|---------------|---------------|
| 1 | [topic] | [who] | [code path] | Missing | [what to write] |
## P1 Gaps — High Impact
| # | Topic | Audience | Source | Current State | What's Needed |
|---|-------|----------|--------|---------------|---------------|
## P2 Gaps — Moderate Impact
| # | Topic | Audience | Source | Current State | What's Needed |
|---|-------|----------|--------|---------------|---------------|
## P3-P4 Gaps — Low Priority
| # | Topic | Audience | Priority | Current State |
|---|-------|----------|----------|---------------|
---
## Shallow Coverage Details
For each Shallow item, explain what's insufficient:
### [Topic]
**Current doc:** [path and section]
**Problem:** [what's missing — examples, edge cases, complete reference, etc.]
**Recommended action:** [specific improvement]
---
## Misplaced Documentation
| Topic | Current Location | Recommended Location | Why |
|-------|-----------------|---------------------|-----|
---
## Well-Documented (No Action Needed)
[List topics with adequate coverage, grouped by audience, so the report
shows the full picture and not just the gaps]---
Integration with Other Doc Skills
This skill fits into the documentation health pipeline:
doc-maintenance → Structural health (links, orphans, folders)
doc-claim-validator → Semantic accuracy (do claims match code?)
doc-completeness-audit → Topic coverage (is everything documented?)
doc-quality-review → Prose quality (is it well-written?)
doc-architecture-review → Information architecture (is it findable?)Route gap remediation to the appropriate producer:
- Reference gaps →
reference-documentation - Tutorial gaps →
tutorial-design - Explanation gaps →
documentation-production
---
Anti-Patterns
- Do not count files as coverage — a file can exist and say nothing useful
- Do not manufacture gaps to look thorough — if coverage is good, say so
- Do not audit archived docs (
docs/archive/) — they are historical - Do not require documentation for internal implementation details — only public surface
- Do not treat every function as needing its own doc page — aggregate by topic
- Do not conflate "not documented" with "needs documenting" — some things are correctly undocumented (internal helpers, deprecated code scheduled for removal)
---
Bundled Resources
Scripts
scripts/inventory.py— Extract documentable surface area from any codebase (env vars, CLI commands, config keys, HTTP endpoints, public exports, error types)
References
references/coverage-model.md— Defines coverage expectations per doc type and audience
Coverage Model
Defines what "complete documentation" means for different doc types and audiences. Use this reference to calibrate expectations during Phase 1 inventory building.
---
Coverage Expectations by Doc Type
Reference Documentation
A reference page is complete when it covers:
- [ ] Every public parameter, flag, option, or config key
- [ ] Default values for all optional parameters
- [ ] Type/format constraints (string, integer, enum values)
- [ ] At least one example per parameter showing its effect
- [ ] Error conditions and what triggers them
- [ ] Related/see-also links to guides and tutorials that use this feature
- [ ] Version or changelog note if behavior changed recently
Minimum bar: Every public API surface has a reference entry. A missing entry is a P0-P1 gap.
Tutorials
A tutorial is complete when:
- [ ] Prerequisites are stated explicitly (tools, versions, prior knowledge)
- [ ] Every step is numbered and actionable
- [ ] Expected output is shown for key steps
- [ ] The reader achieves a tangible result by the end
- [ ] Common errors at each step have troubleshooting guidance
- [ ] Estimated completion time is stated
- [ ] Next steps or further reading are linked
Minimum bar: A new user can follow the tutorial from start to finish without external help. A tutorial that can't be completed is a P0 gap.
Guides (How-To)
A guide is complete when:
- [ ] The problem or goal is stated clearly at the top
- [ ] Prerequisites are listed
- [ ] The solution is presented with enough context to adapt
- [ ] Edge cases or variations are noted
- [ ] The guide links to reference docs for parameters it uses
- [ ] Expected result is described
Minimum bar: A user with the stated prerequisites can accomplish the goal.
Explanations
An explanation is complete when:
- [ ] The concept or system being explained is named and scoped
- [ ] "Why" is addressed, not just "what"
- [ ] Diagrams or visualizations support complex relationships
- [ ] Trade-offs and alternatives are acknowledged
- [ ] Links to reference and guides provide the "how"
Minimum bar: A reader understands the design rationale, not just the mechanics.
README
A README is complete when:
- [ ] One-line description of what the project does
- [ ] Installation instructions (or link to them)
- [ ] Basic usage example
- [ ] Link to full documentation
- [ ] License information
- [ ] How to contribute (or link to CONTRIBUTING.md)
---
Coverage Expectations by Audience
New Users
Must have:
- Installation guide (P0 if missing)
- Quick start / first-use tutorial (P0 if missing)
- Core concepts explanation (P1 if missing)
- Troubleshooting for common first-run errors (P1 if missing)
Daily Users
Must have:
- Complete reference for all features they interact with (P1 if missing)
- Guides for common workflows (P2 if missing)
- Changelog or what's new for upgrades (P2 if missing)
Contributors / Developers
Must have:
- Development setup guide (P1 if missing)
- Architecture overview (P2 if missing)
- Testing guide (P2 if missing)
- Contribution guidelines (P2 if missing)
Operators
Must have:
- Deployment guide (P1 if missing)
- Configuration reference with all env vars and defaults (P1 if missing)
- Monitoring and observability guide (P2 if missing)
- Backup and recovery procedures (P2 if missing)
- Upgrade and migration guides (P2 if missing)
---
Determining "Should Document" vs. "Correctly Undocumented"
Not everything needs docs. Skip inventory items that are:
- Internal implementation details — private functions, internal data structures
- Deprecated and scheduled for removal — unless the migration path is undocumented
- Generated from source — if auto-generated docs are complete and accurate
- Third-party documentation — link to upstream docs, don't duplicate
- Self-documenting — trivially obvious from names and types (use judgment)
When in doubt, check: "Would a real user ever search for this?" If yes, it should be in the inventory. If no, skip it.
#!/usr/bin/env python3
"""Extract documentable surface area from a codebase.
Scans source files for signals that indicate something a user or developer
might need documentation for: environment variables, CLI commands, config
keys, public exports, HTTP endpoints, and error types.
Output is a JSON inventory that can be diffed against existing documentation
to find coverage gaps.
Usage:
python3 inventory.py [--root PATH] [--json] [--detectors LIST]
Detectors:
env_vars Environment variable references
cli_commands CLI framework commands and flags
config_keys Configuration file key access
http_endpoints HTTP route definitions
public_exports Public module/package exports
error_types Custom error/exception definitions
Default: all detectors enabled.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import sys
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Dict, List, Optional, Set
# ---------------------------------------------------------------------------
# Data model
# ---------------------------------------------------------------------------
@dataclass
class InventoryItem:
"""A single documentable item found in the codebase."""
topic: str
category: str # env_var, cli_command, cli_flag, config_key, endpoint, export, error_type
source_file: str
source_line: int
detail: str = "" # extra context (default value, type, HTTP method, etc.)
audience: str = "user" # user, developer, operator
@dataclass
class Inventory:
"""Complete inventory of documentable surface area."""
root: str
items: List[InventoryItem] = field(default_factory=list)
detectors_run: List[str] = field(default_factory=list)
files_scanned: int = 0
def add(self, item: InventoryItem) -> None:
self.items.append(item)
def summary(self) -> Dict[str, int]:
counts: Dict[str, int] = {}
for item in self.items:
counts[item.category] = counts.get(item.category, 0) + 1
return counts
# ---------------------------------------------------------------------------
# File discovery
# ---------------------------------------------------------------------------
# Extensions to scan per language family
SOURCE_EXTENSIONS: Dict[str, List[str]] = {
"python": [".py"],
"javascript": [".js", ".mjs", ".cjs"],
"typescript": [".ts", ".tsx"],
"rust": [".rs"],
"go": [".go"],
"ruby": [".rb"],
"java": [".java", ".kt"],
"shell": [".sh", ".bash", ".zsh"],
}
# Directories to skip
SKIP_DIRS: Set[str] = {
"node_modules", ".git", "__pycache__", ".tox", ".venv", "venv",
"dist", "build", ".eggs", "*.egg-info", "target", "vendor",
".mypy_cache", ".pytest_cache", ".ruff_cache", "htmlcov",
"_site", ".jekyll-cache", ".next", ".nuxt",
}
ALL_EXTENSIONS: Set[str] = set()
for exts in SOURCE_EXTENSIONS.values():
ALL_EXTENSIONS.update(exts)
def discover_files(root: Path) -> List[Path]:
"""Find source files, skipping common non-source directories."""
files: List[Path] = []
for dirpath, dirnames, filenames in os.walk(root):
# Prune skipped directories in-place
dirnames[:] = [
d for d in dirnames
if d not in SKIP_DIRS and not d.endswith(".egg-info")
]
for fname in filenames:
if any(fname.endswith(ext) for ext in ALL_EXTENSIONS):
files.append(Path(dirpath) / fname)
return sorted(files)
def read_file_lines(path: Path) -> List[str]:
"""Read file lines, returning empty list on decode errors."""
try:
return path.read_text(encoding="utf-8", errors="replace").splitlines()
except OSError:
return []
# ---------------------------------------------------------------------------
# Detectors
# ---------------------------------------------------------------------------
# -- Environment variables --------------------------------------------------
ENV_PATTERNS = [
# Python
(re.compile(r'''os\.environ\.get\(\s*["']([A-Z_][A-Z0-9_]+)["']'''), None),
(re.compile(r'''os\.environ\[["']([A-Z_][A-Z0-9_]+)["']\]'''), None),
(re.compile(r'''os\.getenv\(\s*["']([A-Z_][A-Z0-9_]+)["']'''), None),
# JavaScript / TypeScript
(re.compile(r'''process\.env\.([A-Z_][A-Z0-9_]+)'''), None),
(re.compile(r'''process\.env\[["']([A-Z_][A-Z0-9_]+)["']\]'''), None),
# Rust
(re.compile(r'''env::var\(\s*["']([A-Z_][A-Z0-9_]+)["']'''), None),
(re.compile(r'''env::var_os\(\s*["']([A-Z_][A-Z0-9_]+)["']'''), None),
# Go
(re.compile(r'''os\.Getenv\(\s*["']([A-Z_][A-Z0-9_]+)["']'''), None),
(re.compile(r'''os\.LookupEnv\(\s*["']([A-Z_][A-Z0-9_]+)["']'''), None),
# Ruby
(re.compile(r'''ENV\[["']([A-Z_][A-Z0-9_]+)["']\]'''), None),
# Shell
(re.compile(r'''\$\{([A-Z_][A-Z0-9_]+)(?::[-=+?])'''), None),
(re.compile(r'''\$([A-Z_][A-Z0-9_]+)'''), None),
]
# Common env vars that don't need documentation
IGNORE_ENV_VARS: Set[str] = {
"HOME", "PATH", "USER", "SHELL", "TERM", "LANG", "LC_ALL",
"PWD", "OLDPWD", "TMPDIR", "TMP", "TEMP", "EDITOR", "VISUAL",
"HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "CI", "DEBUG",
"NODE_ENV", "PYTHONPATH", "PYTHONDONTWRITEBYTECODE",
"VIRTUAL_ENV", "CONDA_DEFAULT_ENV",
}
def detect_env_vars(files: List[Path], root: Path, inv: Inventory) -> None:
"""Detect environment variable references."""
seen: Dict[str, InventoryItem] = {}
for fpath in files:
lines = read_file_lines(fpath)
for lineno, line in enumerate(lines, 1):
for pattern, _ in ENV_PATTERNS:
for match in pattern.finditer(line):
name = match.group(1)
if name in IGNORE_ENV_VARS:
continue
if name not in seen:
# Try to extract default value
default = _extract_default(line, name)
item = InventoryItem(
topic=name,
category="env_var",
source_file=str(fpath.relative_to(root)),
source_line=lineno,
detail=f"default: {default}" if default else "",
audience="operator",
)
seen[name] = item
inv.add(item)
def _extract_default(line: str, var_name: str) -> Optional[str]:
"""Try to extract a default value from env var access."""
# Python: os.environ.get("X", "default") or os.getenv("X", "default")
m = re.search(
rf'''(?:os\.environ\.get|os\.getenv)\(\s*["']{var_name}["']\s*,\s*(.+?)\)''',
line,
)
if m:
return m.group(1).strip().strip("\"'")
# Shell: ${VAR:-default}
m = re.search(rf'''\$\{{{var_name}:-([^}}]*)\}}''', line)
if m:
return m.group(1)
return None
# -- CLI commands -----------------------------------------------------------
CLI_FRAMEWORK_PATTERNS = {
# Python argparse
"argparse": [
(re.compile(r'''add_parser\(\s*["']([a-z][\w-]*)["']'''), "subcommand"),
(re.compile(r'''add_argument\(\s*["'](--[\w-]+)["']'''), "flag"),
(re.compile(r'''add_argument\(\s*["'](-\w)["']'''), "short_flag"),
],
# Python click
"click": [
(re.compile(r'''@(?:click\.)?command\(\s*(?:["'](\w[\w-]*)["'])?'''), "command"),
(re.compile(r'''@(?:click\.)?group\(\s*(?:["'](\w[\w-]*)["'])?'''), "group"),
(re.compile(r'''@(?:click\.)?option\(\s*["'](--[\w-]+)["']'''), "flag"),
(re.compile(r'''@(?:click\.)?argument\(\s*["'](\w+)["']'''), "argument"),
],
# Rust clap
"clap": [
(re.compile(r'''\.subcommand\(\s*Command::new\(\s*["'](\w[\w-]*)["']'''), "subcommand"),
(re.compile(r'''Arg::new\(\s*["'](\w[\w-]*)["']'''), "argument"),
(re.compile(r'''\.long\(\s*["']([\w-]+)["']'''), "flag"),
],
# Go cobra
"cobra": [
(re.compile(r'''&cobra\.Command\{[^}]*Use:\s*["'](\w[\w-]*)''', re.DOTALL), "command"),
],
# Node commander
"commander": [
(re.compile(r'''\.command\(\s*["'](\w[\w-]*)'''), "command"),
(re.compile(r'''\.option\(\s*["'](--[\w-]+)'''), "flag"),
],
}
def detect_cli_commands(files: List[Path], root: Path, inv: Inventory) -> None:
"""Detect CLI framework commands and flags."""
seen_commands: Set[str] = set()
seen_flags: Set[str] = set()
for fpath in files:
lines = read_file_lines(fpath)
content = "\n".join(lines)
relpath = str(fpath.relative_to(root))
for framework, patterns in CLI_FRAMEWORK_PATTERNS.items():
for pattern, kind in patterns:
for match in pattern.finditer(content):
name = match.group(1)
if not name:
continue
# Find the line number
pos = match.start()
lineno = content[:pos].count("\n") + 1
if kind in ("command", "subcommand", "group"):
if name not in seen_commands:
seen_commands.add(name)
# Try to extract help text
help_text = _extract_help(content, pos)
inv.add(InventoryItem(
topic=name,
category="cli_command",
source_file=relpath,
source_line=lineno,
detail=help_text or f"framework: {framework}",
audience="user",
))
elif kind in ("flag", "short_flag"):
if name not in seen_flags:
seen_flags.add(name)
help_text = _extract_help(content, pos)
inv.add(InventoryItem(
topic=name,
category="cli_flag",
source_file=relpath,
source_line=lineno,
detail=help_text or f"framework: {framework}",
audience="user",
))
elif kind == "argument":
inv.add(InventoryItem(
topic=name,
category="cli_argument",
source_file=relpath,
source_line=lineno,
detail=f"framework: {framework}",
audience="user",
))
def _extract_help(content: str, match_pos: int) -> Optional[str]:
"""Try to extract a help string near a CLI definition."""
# Look for help="..." within 200 chars after match
window = content[match_pos:match_pos + 300]
m = re.search(r'''help\s*=\s*["']([^"']+)["']''', window)
if m:
return m.group(1)
# Rust/Go: doc comment before
before = content[max(0, match_pos - 200):match_pos]
m = re.search(r'''///\s*(.+)$''', before, re.MULTILINE)
if m:
return m.group(1).strip()
return None
# -- Config keys ------------------------------------------------------------
CONFIG_PATTERNS = [
# Python dict .get() with string key
(re.compile(r'''\.get\(\s*["']([a-z][\w._-]*)["']'''), "dict_get"),
# Python dict["key"] access
(re.compile(r'''\[["']([a-z][\w._-]*)["']\]'''), "dict_access"),
# TOML/INI section headers
(re.compile(r'''^\[([a-z][\w._-]*)\]\s*$''', re.MULTILINE), "section"),
# YAML top-level keys (simplified)
(re.compile(r'''^([a-z][\w_-]*):\s''', re.MULTILINE), "yaml_key"),
# JS/TS config access: config.key or config["key"]
(re.compile(r'''config\.([a-z][\w_]*)'''), "js_config"),
]
# Strings too generic to be config keys
IGNORE_CONFIG_KEYS: Set[str] = {
"name", "type", "value", "key", "data", "id", "get", "set",
"items", "keys", "values", "pop", "update", "default", "format",
"path", "file", "line", "text", "string", "result", "output",
"input", "error", "message", "status", "code", "index", "count",
"size", "length", "start", "end", "true", "false", "none", "null",
}
# Only scan files that look like config loaders
CONFIG_FILE_PATTERNS = [
re.compile(r"config", re.IGNORECASE),
re.compile(r"settings", re.IGNORECASE),
re.compile(r"options", re.IGNORECASE),
re.compile(r"preferences", re.IGNORECASE),
]
def detect_config_keys(files: List[Path], root: Path, inv: Inventory) -> None:
"""Detect configuration key access patterns in config-related files."""
seen: Set[str] = set()
# Only scan files whose name suggests config handling
config_files = [
f for f in files
if any(p.search(f.name) for p in CONFIG_FILE_PATTERNS)
]
for fpath in config_files:
lines = read_file_lines(fpath)
relpath = str(fpath.relative_to(root))
for lineno, line in enumerate(lines, 1):
for pattern, kind in CONFIG_PATTERNS:
for match in pattern.finditer(line):
key = match.group(1)
if key in IGNORE_CONFIG_KEYS or len(key) < 3:
continue
if key not in seen:
seen.add(key)
inv.add(InventoryItem(
topic=key,
category="config_key",
source_file=relpath,
source_line=lineno,
detail=f"pattern: {kind}",
audience="operator",
))
# -- HTTP endpoints ---------------------------------------------------------
ENDPOINT_PATTERNS = [
# Python Flask/FastAPI decorators
(re.compile(r'''@\w+\.(get|post|put|patch|delete|route)\(\s*["']([^"']+)["']''', re.IGNORECASE), "decorator"),
# Express.js
(re.compile(r'''(?:app|router)\.(get|post|put|patch|delete)\(\s*["']([^"']+)["']'''), "express"),
# Rust actix/axum
(re.compile(r'''#\[(get|post|put|patch|delete)\(\s*"([^"]+)"'''), "rust_macro"),
(re.compile(r'''\.route\(\s*["']([^"']+)["']\s*,\s*(?:get|post|put|patch|delete)'''), "axum"),
# Go net/http
(re.compile(r'''(?:Handle|HandleFunc)\(\s*["']([^"']+)["']'''), "go_http"),
]
def detect_http_endpoints(files: List[Path], root: Path, inv: Inventory) -> None:
"""Detect HTTP route/endpoint definitions."""
seen: Set[str] = set()
for fpath in files:
lines = read_file_lines(fpath)
relpath = str(fpath.relative_to(root))
for lineno, line in enumerate(lines, 1):
for pattern, kind in ENDPOINT_PATTERNS:
for match in pattern.finditer(line):
groups = match.groups()
if kind == "decorator" or kind == "express" or kind == "rust_macro":
method, path = groups[0].upper(), groups[1]
elif kind == "go_http":
method, path = "ANY", groups[0]
elif kind == "axum":
path, method = groups[0], "ANY"
else:
continue
endpoint = f"{method} {path}"
if endpoint not in seen:
seen.add(endpoint)
inv.add(InventoryItem(
topic=endpoint,
category="http_endpoint",
source_file=relpath,
source_line=lineno,
detail=f"framework: {kind}",
audience="developer",
))
# -- Public exports ---------------------------------------------------------
EXPORT_PATTERNS = [
# Python __init__.py imports
(re.compile(r'''from\s+\.\w+\s+import\s+(\w+)'''), "python_reexport"),
# Python __all__
(re.compile(r'''__all__\s*=\s*\[([^\]]+)\]'''), "python_all"),
# JS/TS named exports
(re.compile(r'''export\s+(?:const|let|var|function|class|interface|type|enum)\s+(\w+)'''), "js_export"),
# JS/TS re-exports
(re.compile(r'''export\s+\{([^}]+)\}'''), "js_reexport"),
# Rust pub items
(re.compile(r'''pub\s+(?:fn|struct|enum|trait|type|const|static|mod)\s+(\w+)'''), "rust_pub"),
# Go exported (capitalized) functions
(re.compile(r'''func\s+(?:\(\w+\s+\*?\w+\)\s+)?([A-Z]\w+)\('''), "go_export"),
]
def detect_public_exports(files: List[Path], root: Path, inv: Inventory) -> None:
"""Detect public module exports and API surface."""
seen: Set[str] = set()
# For Python, focus on __init__.py and public modules
# For JS/TS, focus on index files and files with 'export'
# For Rust, focus on lib.rs and mod.rs
export_files = [
f for f in files
if f.name in ("__init__.py", "index.ts", "index.js", "lib.rs", "mod.rs")
or "export" in (read_file_lines(f)[:5] and " ".join(read_file_lines(f)[:5]) or "")
]
# Also include all Rust/Go files for pub/exported detection
for f in files:
if f.suffix in (".rs", ".go") and f not in export_files:
export_files.append(f)
for fpath in export_files:
lines = read_file_lines(fpath)
relpath = str(fpath.relative_to(root))
content = "\n".join(lines)
for pattern, kind in EXPORT_PATTERNS:
for match in pattern.finditer(content):
raw = match.group(1)
# Handle multi-name exports (__all__, export { a, b })
if kind in ("python_all", "js_reexport"):
names = [
n.strip().strip("\"'")
for n in raw.split(",")
if n.strip().strip("\"'")
]
else:
names = [raw]
pos = match.start()
lineno = content[:pos].count("\n") + 1
for name in names:
if name.startswith("_") or name in seen:
continue
seen.add(name)
inv.add(InventoryItem(
topic=name,
category="public_export",
source_file=relpath,
source_line=lineno,
detail=f"kind: {kind}",
audience="developer",
))
# -- Error types ------------------------------------------------------------
ERROR_PATTERNS = [
# Python exception classes
(re.compile(r'''class\s+(\w*(?:Error|Exception|Failure|Fault)\w*)\s*\('''), "python"),
# JS/TS error classes
(re.compile(r'''class\s+(\w*(?:Error|Exception)\w*)\s+extends'''), "js"),
# Rust error enums/structs
(re.compile(r'''(?:pub\s+)?(?:enum|struct)\s+(\w*(?:Error|Err)\w*)'''), "rust"),
# Go error types
(re.compile(r'''type\s+(\w*(?:Error|Err)\w*)\s+struct'''), "go"),
# HTTP error status codes in route handlers
(re.compile(r'''(?:status|code)\s*[=:]\s*(4\d{2}|5\d{2})'''), "http_status"),
]
def detect_error_types(files: List[Path], root: Path, inv: Inventory) -> None:
"""Detect custom error/exception type definitions."""
seen: Set[str] = set()
for fpath in files:
lines = read_file_lines(fpath)
relpath = str(fpath.relative_to(root))
for lineno, line in enumerate(lines, 1):
for pattern, kind in ERROR_PATTERNS:
for match in pattern.finditer(line):
name = match.group(1)
if name not in seen:
seen.add(name)
category = "error_type"
if kind == "http_status":
category = "http_error_status"
inv.add(InventoryItem(
topic=name,
category=category,
source_file=relpath,
source_line=lineno,
detail=f"language: {kind}",
audience="developer",
))
# ---------------------------------------------------------------------------
# Detector registry
# ---------------------------------------------------------------------------
DETECTORS = {
"env_vars": detect_env_vars,
"cli_commands": detect_cli_commands,
"config_keys": detect_config_keys,
"http_endpoints": detect_http_endpoints,
"public_exports": detect_public_exports,
"error_types": detect_error_types,
}
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def run_inventory(
root: Path,
src_dirs: Optional[List[Path]] = None,
detectors: Optional[List[str]] = None,
) -> Inventory:
"""Run the inventory extraction pipeline.
Args:
root: Project root directory (used for relative path display).
src_dirs: Specific source directories to scan. If None, scans root.
detectors: Specific detectors to run. If None, runs all.
"""
inv = Inventory(root=str(root))
# Discover source files from specified dirs or root
files: List[Path] = []
scan_roots = src_dirs if src_dirs else [root]
for scan_root in scan_roots:
if scan_root.is_dir():
files.extend(discover_files(scan_root))
files = sorted(set(files))
inv.files_scanned = len(files)
# Run selected detectors
detector_names = detectors or list(DETECTORS.keys())
for name in detector_names:
if name in DETECTORS:
DETECTORS[name](files, root, inv)
inv.detectors_run.append(name)
return inv
# ---------------------------------------------------------------------------
# Coverage checking
# ---------------------------------------------------------------------------
def discover_doc_files(docs_dir: Path) -> List[Path]:
"""Find markdown files in a docs directory."""
files: List[Path] = []
for dirpath, dirnames, filenames in os.walk(docs_dir):
dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]
for fname in filenames:
if fname.endswith((".md", ".mdx", ".rst", ".adoc")):
files.append(Path(dirpath) / fname)
return sorted(files)
def check_coverage(inv: Inventory, docs_dir: Path) -> Dict[str, List[InventoryItem]]:
"""Check which inventory items appear in documentation.
Reads all markdown files in docs_dir and searches for each inventory
item's topic string. Returns a dict with 'documented' and 'missing' lists.
"""
# Build a single searchable blob from all doc content
doc_files = discover_doc_files(docs_dir)
doc_content = ""
for fpath in doc_files:
try:
doc_content += fpath.read_text(encoding="utf-8", errors="replace") + "\n"
except OSError:
continue
doc_content_lower = doc_content.lower()
documented: List[InventoryItem] = []
missing: List[InventoryItem] = []
for item in inv.items:
topic = item.topic
# Search strategies by category
found = False
if item.category == "env_var":
# Env vars: exact match (case-sensitive, they're uppercase)
found = topic in doc_content
elif item.category in ("cli_command", "cli_flag", "cli_argument"):
# CLI: search for the command/flag name
found = topic in doc_content or topic.lstrip("-") in doc_content_lower
elif item.category == "config_key":
# Config keys: exact or dotted notation
found = topic in doc_content_lower
elif item.category == "http_endpoint":
# Endpoints: search for the path portion
parts = topic.split(" ", 1)
path = parts[1] if len(parts) > 1 else parts[0]
found = path in doc_content
elif item.category in ("public_export", "error_type"):
# Symbols: search for the name
found = topic in doc_content
else:
found = topic.lower() in doc_content_lower
if found:
documented.append(item)
else:
missing.append(item)
return {"documented": documented, "missing": missing}
def main() -> None:
parser = argparse.ArgumentParser(
description="Extract documentable surface area from a codebase.",
)
parser.add_argument(
"--root",
default=".",
help="Project root directory (default: current directory)",
)
parser.add_argument(
"--src",
action="append",
default=None,
help="Source directory to scan (can be repeated; default: scan --root)",
)
parser.add_argument(
"--docs",
default=None,
help="Documentation directory to check coverage against",
)
parser.add_argument(
"--json",
action="store_true",
help="Output as JSON (default: human-readable table)",
)
parser.add_argument(
"--detectors",
default=None,
help="Comma-separated list of detectors to run (default: all)",
)
args = parser.parse_args()
root = Path(args.root).resolve()
if not root.is_dir():
print(f"Error: {root} is not a directory", file=sys.stderr)
sys.exit(1)
src_dirs = [Path(s).resolve() for s in args.src] if args.src else None
detector_list = args.detectors.split(",") if args.detectors else None
inv = run_inventory(root, src_dirs, detector_list)
# If --docs provided, check coverage
coverage = None
if args.docs:
docs_dir = Path(args.docs).resolve()
if not docs_dir.is_dir():
print(f"Error: {docs_dir} is not a directory", file=sys.stderr)
sys.exit(1)
coverage = check_coverage(inv, docs_dir)
if args.json:
output: Dict[str, object] = {
"root": inv.root,
"files_scanned": inv.files_scanned,
"detectors_run": inv.detectors_run,
"summary": inv.summary(),
"items": [asdict(item) for item in inv.items],
}
if coverage is not None:
output["coverage"] = {
"docs_dir": args.docs,
"documented": len(coverage["documented"]),
"missing": len(coverage["missing"]),
"coverage_pct": round(
len(coverage["documented"]) / max(len(inv.items), 1) * 100, 1
),
"missing_items": [asdict(item) for item in coverage["missing"]],
}
json.dump(output, sys.stdout, indent=2)
print()
else:
print(f"Inventory: {root}")
if src_dirs:
print(f"Source dirs: {', '.join(str(s) for s in src_dirs)}")
print(f"Files scanned: {inv.files_scanned}")
print(f"Detectors: {', '.join(inv.detectors_run)}")
print(f"Items found: {len(inv.items)}")
print()
summary = inv.summary()
for category, count in sorted(summary.items()):
print(f" {category}: {count}")
print()
if coverage is not None:
total = len(inv.items)
doc_count = len(coverage["documented"])
miss_count = len(coverage["missing"])
pct = round(doc_count / max(total, 1) * 100, 1)
print(f"Coverage against {args.docs}:")
print(f" Documented: {doc_count}/{total} ({pct}%)")
print(f" Missing: {miss_count}/{total}")
print()
if coverage["missing"]:
# Group missing by category
by_cat: Dict[str, List[InventoryItem]] = {}
for item in coverage["missing"]:
by_cat.setdefault(item.category, []).append(item)
print("## Missing from docs")
print()
for category in sorted(by_cat.keys()):
items = by_cat[category]
print(f"### {category} ({len(items)})")
print()
for item in sorted(items, key=lambda i: i.topic):
loc = f"{item.source_file}:{item.source_line}"
detail = f" ({item.detail})" if item.detail else ""
print(f" {item.topic}{detail}")
print(f" {loc}")
print()
else:
# No coverage check — just show full inventory
by_category: Dict[str, List[InventoryItem]] = {}
for item in inv.items:
by_category.setdefault(item.category, []).append(item)
for category in sorted(by_category.keys()):
items = by_category[category]
print(f"## {category} ({len(items)})")
print()
for item in sorted(items, key=lambda i: i.topic):
loc = f"{item.source_file}:{item.source_line}"
detail = f" ({item.detail})" if item.detail else ""
print(f" {item.topic}{detail}")
print(f" {loc}")
print()
if __name__ == "__main__":
main()