
Tdd
- 219 installs
- 339 repo stars
- Updated August 4, 2026
- glebis/claude-skills
Implement features with red-green-refactor loops so behavior is specified by failing tests first and regressions are caught before merge.
About
The tdd skill instructs Claude Code to follow test-driven development by writing failing tests first, implementing minimal passing code, and refactoring safely, giving teams a disciplined build-time workflow that locks behavior before features expand across backend services and CLIs.
- Enforces red-green-refactor implementation order
- Writes failing tests before production code
- Keeps refactors safe with immediate feedback
- Integrates with CI-friendly test harnesses
Tdd by the numbers
- 219 all-time installs (skills.sh)
- Ranked #790 of 2,153 Testing & QA skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/glebis/claude-skills --skill tddAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 219 |
|---|---|
| repo stars | ★ 339 |
| Last updated | August 4, 2026 |
| Repository | glebis/claude-skills ↗ |
What it does
Implement features with red-green-refactor loops so behavior is specified by failing tests first and regressions are caught before merge.
Files
Test-Driven Development — Multi-Agent Orchestration
Enforce disciplined RED-GREEN-REFACTOR cycles using separate subagents for test writing and implementation. The core innovation: the Test Writer never sees implementation code, and the Implementer never sees the specification. This prevents the LLM from leaking implementation intent into test design.
When to Use
- User requests TDD, test-first, or red-green-refactor workflow
- User says
/tddwith a feature description or bug report - User wants to add a feature with test coverage enforced from the start
- User wants to fix a bug by first writing a reproducing test
Invocation Modes
| Invocation | Behavior |
|---|---|
/tdd <feature> | Interactive mode — pause for approval at slices and each RED checkpoint |
/tdd --auto <feature> | Autonomous mode — run all slices without pausing; stop ONLY on unrecoverable errors |
/tdd --resume | Resume from .tdd-state.json in project root |
/tdd --dry-run <feature> | Validation mode — runs Phase 0 + Phase 1 fully, renders all prompts, but skips Task() calls. No code is written. |
In --auto mode, skip all [HUMAN CHECKPOINT] steps. Print status lines instead:
[auto] RED slice 1/4: "validates email format" — test failing as expected
[auto] GREEN slice 1/4: passing (attempt 1)
[auto] REFACTOR slice 1/4: 1 suggestion applied, 0 skippedStop and ask the user ONLY when:
- Implementation fails after 5 attempts
- Regressions cannot be auto-fixed after 3 attempts
- A script error makes it impossible to continue (missing binary, permission denied, etc.)
In --dry-run mode, validate the entire orchestration pipeline without executing any subagents or writing any code:
1. Phase 0 runs fully: detect framework, verify baseline, extract API, discover docs, create state file 2. Phase 1 runs fully: decompose into slices (still requires user approval) 3. For each slice: render all three agent prompts (Test Writer, Implementer, Refactorer) with actual variables. Print rendered prompts to the user with character counts. 4. No `Task()` calls are made. No test files are written. No implementation code is generated. 5. Validate: check that all template variables resolve (no {UNRESOLVED} placeholders), all scripts execute without error, and the state file is well-formed. 6. Report summary:
DRY RUN COMPLETE: {feature name}
Phase 0:
Framework: {framework}
Language: {language}
Baseline: {pass|greenfield}
API surface: {line count} lines
Doc context: {line count} lines (or "none")
Phase 1:
Slices: {N} ({layer breakdown})
Prompts rendered: {N * 3} (all variables resolved)
Test Writer: {char count} chars
Implementer: {char count} chars
Refactorer: {char count} chars
State file: .tdd-state.json written
No code was modified.This mode is useful for:
- Validating that scripts work in the project's environment
- Reviewing prompt content before committing to a full TDD run
- Testing skill changes without side effects
Architecture Overview
ORCHESTRATOR (you, reading this file)
├─ Phase 0: Setup — detect framework, extract API, create state file
├─ Phase 1: Decompose into vertical slices → user approves
│
├─ FOR EACH SLICE:
│ ├─ Phase 2 (RED): Task(Test Writer) ← spec + API only
│ ├─ Phase 3 (GREEN): Task(Implementer) ← failing test + error only
│ └─ Phase 4 (REFACTOR): Task(Refactorer) ← all code + green results
│
└─ SummaryContext Boundaries (the key constraint)
| Agent | Sees | Does NOT See |
|---|---|---|
| Test Writer | Slice spec, public API signatures, framework conventions, layer constraints | Implementation code, other slices, implementation plans |
| Implementer | Failing test code, test failure output, file tree, existing source, layer constraints | Original spec, slice descriptions, future plans |
| Refactorer | All implementation + all tests + green results, layers touched | Original spec, decomposition rationale |
Workflow
Phase 0: Setup (once per session)
Step 1: Detect framework and test runner.
Check for: package.json (jest/vitest), pyproject.toml/pytest.ini (pytest),
go.mod (go test), Cargo.toml (cargo test), Gemfile (rspec), composer.json (phpunit)If ambiguous, ask: "What command runs your tests? (e.g., npm test, pytest)"
Step 2: Detect language from source files (for agent prompts):
TypeScript (.ts/.tsx), JavaScript (.js/.jsx), Python (.py), Go (.go), Rust (.rs), Ruby (.rb), PHP (.php)Step 3: Verify green baseline.
bash ~/.claude/skills/tdd/scripts/run_tests.sh {FRAMEWORK} "{TEST_COMMAND}"Parse the JSON output.
- If
statusis"pass": proceed. - If
statusis"fail": stop — "Existing tests are failing. TDD starts from a green baseline." - If
statusis"error"ANDtotalis 0: greenfield project — no tests exist yet. This is fine. Proceed.
Step 4: Extract the public API surface.
bash ~/.claude/skills/tdd/scripts/extract_api.sh {SOURCE_DIR}Save the output — this is what the Test Writer will see. If empty (greenfield), that's expected.
Step 5: Discover project documentation.
bash ~/.claude/skills/tdd/scripts/discover_docs.sh {PROJECT_ROOT} --lang {LANGUAGE}This searches for:
- Documentation files: README, ARCHITECTURE.md, docs/ folder, DESIGN.md, SPEC files, ADRs
- API specifications: OpenAPI/Swagger, GraphQL schemas, .proto files
- Source docstrings: JSDoc, Python docstrings, Go doc comments, Rust
///comments
Save the output as {DOC_CONTEXT}. This feeds into:
- Phase 1 — so slice decomposition is informed by documented behavior and API contracts
- Phase 2 — so the Test Writer writes tests aligned with documented intent, not just code signatures
If empty (no docs found), that's fine — proceed without doc context.
Step 6: Create the state file .tdd-state.json in the project root:
{
"feature": "user's feature description",
"framework": "jest|vitest|pytest|go|cargo|rspec|phpunit",
"language": "typescript|javascript|python|go|rust|ruby|php",
"test_command": "the full test command",
"source_dir": "src/",
"doc_context": "output from discover_docs.sh (or empty string)",
"auto_mode": false,
"dry_run": false,
"slices": [],
"current_slice": 0,
"phase": "setup",
"layer_map": {},
"files_modified": [],
"test_files_created": []
}Each slice in the slices array includes a layer field: "domain", "domain-service", "application", or "infrastructure". See Phase 1 for how layers are assigned.
The layer_map maps directory prefixes to layers. Built during Phase 1 from project structure:
{
"layer_map": {
"src/domain/": "domain",
"src/services/": "domain-service",
"src/application/": "application",
"src/infrastructure/": "infrastructure",
"src/adapters/": "infrastructure",
"src/controllers/": "infrastructure"
}
}If the project has no clear directory-layer mapping (flat structure), set layer_map to {} and skip path-based validation.
Step 5a (auto-detect layer_map): If layer_map is empty, scan the source directory for common DDD/layered architecture directory names and auto-populate:
Common directory → layer mappings (check if directories exist):
*/domain/ → "domain"
*/models/ → "domain" (ORM models often serve as domain entities)
*/entities/ → "domain"
*/value_objects/ → "domain"
*/services/ → "application" (unless clearly infrastructure)
*/application/ → "application"
*/use_cases/ → "application"
*/core/ → "application"
*/infrastructure/ → "infrastructure"
*/adapters/ → "infrastructure"
*/controllers/ → "infrastructure"
*/api/ → "infrastructure"
*/bot/ → "infrastructure" (Telegram/Discord bot handlers)
*/handlers/ → "infrastructure"
*/repositories/ → "infrastructure" (concrete repo implementations)Only add entries for directories that actually exist in the source tree. If fewer than 2 directories match, leave layer_map empty (flat project). Present the auto-detected map to the user for confirmation:
Auto-detected layer map from directory structure:
src/models/ → domain
src/services/ → application
src/core/ → application
src/bot/ → infrastructure
src/api/ → infrastructure
Does this mapping look correct? (adjust if needed)Update state: "phase": "setup". Write state file immediately.
---
Phase 1: Specification Decomposition
Take the user's feature request and decompose into ordered vertical slices. Each slice is one testable behavior.
Use doc context: When decomposing, cross-reference {DOC_CONTEXT} from Phase 0 Step 5. Documentation often describes intended behaviors, edge cases, and API contracts that should inform slice boundaries. If docs mention specific error cases, validation rules, or behavioral requirements, consider them as slice candidates.
Inside-Out Slice Ordering
After identifying all slices, sort them inside-out by architectural layer. This ensures each slice can build on real (not mocked) implementations from previous slices:
1. Domain model slices first — pure logic, no dependencies, no mocks needed 2. Domain service slices — cross-aggregate operations using real domain objects 3. Application service / use case slices — orchestration using in-memory fakes for ports 4. Infrastructure adapter slices last — repos, external APIs, framework adapters
Assign each slice a layer tag: domain, domain-service, application, or infrastructure. Use the heuristics from references/layer_guide.md to classify.
Why inside-out? Domain slices produce real objects that later slices use directly. This minimizes mocking and catches integration issues early. It also ensures business rules are implemented and tested before any infrastructure decisions are made.
For simple projects where all code lives in one layer, all slices get layer: "application" and the ordering doesn't change — the guidance degrades gracefully.
Edge Cases in Slice Ordering
Infrastructure-only features (e.g., "add email provider retry logic", "switch from Postgres to MySQL"):
- If a feature has NO domain or application behavior changes, all slices may be
infrastructure. This is valid — skip the inner layers entirely. - Present as: "This is a pure infrastructure change. All slices are infrastructure-layer."
Missing port interface (domain-service needs a port that doesn't exist yet):
- The first slice that needs the port should create the interface as part of its implementation. The Implementer is allowed to create files in inner layers (domain/domain-service can define their own ports).
- Example: a
domain-serviceslice forRegistrationServicecreatesdomain/ports/UserRepositoryinterface as part of GREEN.
Cross-cutting slices (a slice touches multiple layers):
- Tag with the INNERMOST layer it touches. The Implementer may create files in that layer and any inner layers.
- Example: a use case that also introduces a new domain event is tagged
applicationbut creates a file indomain/events/.
Present to the user:
I've broken this into N vertical slices (ordered inside-out):
Domain:
1. [behavior] — [what the test verifies]
Domain Services:
2. [behavior] — [what the test verifies]
Application:
3. [behavior] — [what the test verifies]
Infrastructure:
4. [behavior] — [what the test verifies]
Each slice follows RED -> GREEN -> REFACTOR before moving to the next.
Does this decomposition look right?If all slices fall in one layer, skip the layer headings and present as a flat list.
Wait for user approval (even in --auto mode — slice decomposition always needs sign-off).
Update state: Write slices array (each with layer field), set "phase": "decomposed".
---
Dry-Run Phase Override (Phase 2–4)
In --dry-run mode, replace Phases 2–4 entirely with the following for each slice:
1. Refresh API surface (extract_api.sh) 2. Render the Test Writer prompt with all variables filled in. Print it under a ### Test Writer Prompt (slice N) heading. 3. Render the Implementer prompt using placeholder test code: "(dry-run: test code would be generated by Test Writer)" for {FAILING_TEST_CODE} and "(dry-run: no test output)" for {TEST_FAILURE_OUTPUT}. 4. Render the Refactorer prompt using placeholder values: "(dry-run: no green output)" for {GREEN_TEST_OUTPUT}, "(dry-run: code from Test Writer)" for {ALL_TEST_CODE}, "(dry-run: code from Implementer)" for {ALL_IMPLEMENTATION_CODE}. 5. For each rendered prompt, verify no {UNRESOLVED_VARIABLE} patterns remain (regex: \{[A-Z][A-Z_]+\}). Report any unresolved variables as errors. 6. Print character counts for each prompt. 7. Move to next slice (no Task() calls, no file writes, no test runs).
After all slices are processed, print the dry-run summary and exit. Do NOT clean up the state file — it's useful for subsequent --resume.
---
Phase 2: RED — Write One Failing Test
Step 1: Refresh the API surface (it changes as slices are implemented):
bash ~/.claude/skills/tdd/scripts/extract_api.sh {SOURCE_DIR}Step 2: Read the prompt template from references/agent_prompts.md -> "Test Writer Agent" section. Construct the prompt by filling in:
{SLICE_SPEC}: The current slice's behavior description{LANGUAGE}: Detected language from Phase 0{FRAMEWORK}: Detected framework name{API_SURFACE}: Output from extract_api.sh{DOC_CONTEXT}: Output from discover_docs.sh (Phase 0 Step 5). Include only sections relevant to the current slice — filter by keyword match if the full output is large.{TEST_FILE_PATH}: Where the test should go (follow project conventions){EXISTING_TEST_CONTENT}: Current content of the test file (if it exists), or "No test file exists yet."{FRAMEWORK_SKELETON}: The relevant skeleton fromreferences/framework_configs.md{LAYER}: The slice's layer tag from Phase 1{LAYER_TEST_CONSTRAINTS}: Layer-specific test constraints (see agent_prompts.md -> Layer-Specific Constraint Lookup)
Step 3: Launch the Test Writer agent:
Task(subagent_type="general-purpose", prompt=<constructed prompt>)Step 4: Parse the JSON response using the parse_agent_json logic from agent_prompts.md: 1. Strip markdown fences if present 2. Try direct JSON parse 3. If that fails, find first { and last }, try that substring 4. If still invalid: retry the Task call once with appended "Return ONLY a JSON object." 5. If still failing: extract test code manually from the raw response
Step 5: Write the test code to the test file. If the file exists, append the test function (and merge imports). If new, create with the agent's imports_needed + test_code.
Step 5a (post-write test smell scan): Scan the test code for common smells before running:
| Smell | Detection | Action |
|---|---|---|
| Assertion Roulette | Multiple bare assert statements without messages in the same test function (3+) | Warn the user (don't block): "Test has N bare assertions — consider adding failure messages for easier debugging." |
| Unknown Test | Test name is generic: matches test_1, test_it, test_works, test_example, test_thing | Re-launch Test Writer with appended: "Use a descriptive test name that reads as a behavior spec (e.g., test_rejects_empty_email)." |
| Tautological assertion | assert True, assert result is not None when function has no None return path, assert isinstance(result, X) as sole assertion | Re-launch Test Writer with appended: "The assertion is tautological — test the actual behavior/value, not just that the function returns something." |
Step 5b (post-write layer lint): Scan the test code for layer-violating patterns:
| Layer | Forbidden patterns in test code |
|---|---|
domain | jest.mock(, vi.mock(, Mock(, mock.patch, unittest.mock, gomock, mockery — domain tests must not use mocking libraries |
domain-service | Same mocking patterns for domain objects (mocking ports/repos is OK) |
application | No forbidden patterns (mocking ports is expected) |
infrastructure | No forbidden patterns |
If forbidden patterns found: 1. Remove the offending mock/pattern from the test 2. Re-launch Test Writer with appended: "Do NOT use mocking libraries. This is a {LAYER} layer test. Use real domain objects." 3. If second attempt still uses forbidden patterns, ask user
Step 6: Run the test to confirm it FAILS (expect an assertion failure, not a setup error):
bash ~/.claude/skills/tdd/scripts/run_tests.sh {FRAMEWORK} "{TEST_COMMAND_FOR_SPECIFIC_TEST}"Step 7: Evaluate the result with semantic validation:
| Result | Action |
|---|---|
status: "fail", assertion error | Proper RED — test fails because the expected behavior doesn't exist yet. Proceed. |
status: "fail", ImportError / ModuleNotFoundError | Setup problem, not a proper RED. The test can't even import the module under test. Fix: create a minimal stub (empty class/function) so the import resolves, then re-run. The test should now fail on the assertion instead. |
status: "fail", AttributeError on missing method | Similar to import error — the class exists but the method doesn't. This is an acceptable RED if the assertion would also fail. Proceed. |
status: "pass" | Behavior already exists. Log: "Test passes — skipping slice (already implemented)." Increment current_slice, move to next slice. |
status: "error", SyntaxError | Fix: the test has a typo. Read the raw_tail, fix the test file directly. Re-run. If still erroring after 2 fix attempts, ask user. |
status: "error", compile/framework error | Fix: bad import, missing fixture, or framework misconfiguration. Read the raw_tail, fix the test file directly. Re-run. If still erroring after 2 fix attempts, ask user. |
Step 8 (interactive mode only — skip in --auto): Present to the user:
RED: Test written and failing as expected.
Test: {test_name}
File: {test_file_path}
Failure: {failure message from JSON}
This test verifies: {test_description from agent response}
Proceed to GREEN phase? (or adjust the test?)Wait for user approval before proceeding to GREEN.
Update state: "phase": "red", add test file to test_files_created. Write state immediately.
---
Phase 3: GREEN — Minimal Implementation
Step 1: Read the failing test file and the test failure output (the full raw_tail from the RED phase run_tests.sh result).
Step 2: Build the file tree of source files (not test files, not node_modules, etc.):
find {SOURCE_DIR} -type f \( -name '*.ts' -o -name '*.js' -o -name '*.py' -o -name '*.go' -o -name '*.rs' -o -name '*.rb' -o -name '*.php' \) | grep -v test | grep -v spec | grep -v node_modules | grep -v __pycache__ | grep -v vendor | grep -v target | grep -v dist | grep -v build | head -50Step 3: Read existing source files that the test imports or references.
Step 4: Read the prompt template from references/agent_prompts.md -> "Implementer Agent" section. Fill in:
{LANGUAGE}: Detected language{FAILING_TEST_CODE}: The complete test file content{TEST_FAILURE_OUTPUT}: Theraw_tailfrom run_tests.sh JSON output{FILE_TREE}: Source file listing from Step 2{EXISTING_SOURCE}: Content of relevant source files (if any — may be empty for greenfield){LAYER}: The slice's layer tag from Phase 1{LAYER_DEPENDENCY_CONSTRAINT}: Layer-specific dependency constraint (see agent_prompts.md -> Layer-Specific Constraint Lookup)
On retries (attempt > 1), also fill in the {?PREVIOUS_ATTEMPT} section:
{PREVIOUS_ATTEMPT_DESCRIPTION}: theexplanationfield from the failed attempt{PREVIOUS_ATTEMPT_ERROR}: theraw_tailfrom the test run after the failed attempt
CRITICAL: Do NOT include the slice specification, feature description, or any future plans. The Implementer works from the test alone.
Step 5: Launch the Implementer agent:
Task(subagent_type="general-purpose", prompt=<constructed prompt>)Step 6: Parse the JSON response. Validate layer boundaries, then apply file changes.
Step 6a (Layer path validation): If layer_map is not empty, check each file path in the response against the current slice's layer:
For each file in response.files:
inferred_layer = lookup file.path against layer_map (longest prefix match)
if inferred_layer exists AND inferred_layer != current_slice.layer:
if inferred_layer is OUTER relative to current_slice.layer:
REJECT: "Implementer created/modified {file.path} which belongs to
the {inferred_layer} layer, but this is a {current_slice.layer} slice.
Inner layers must not depend on outer layers."
→ Re-launch Implementer with appended constraint:
"Do NOT create or modify files in {inferred_layer} directories.
This slice is {current_slice.layer} only."
if inferred_layer is INNER relative to current_slice.layer:
ALLOW: outer layers may touch inner-layer files (e.g., adding a port interface)Layer ordering for "outer" check: domain < domain-service < application < infrastructure.
If layer_map is empty (flat project), skip this validation.
Step 6b: Apply validated file changes:
For each file in the response files array:
- If
actionis"create"or"overwrite": Use the Write tool to create or overwrite the file with the complete content - If
actionis"edit"(used for existing files over 200 lines): Use the Edit tool withold_string→new_stringto apply the changes. The Implementer returns only the changed functions with surrounding context — identify the insertion point or the function being replaced, and use Edit tool accordingly. If the edit target is ambiguous, fall back to reading the full file and using Write. - For existing files over 200 lines where the Implementer returned full content anyway (action = "overwrite"), prefer using Edit tool to apply only the diff — this prevents accidental reformatting of untouched code
Step 7: Run the specific test:
bash ~/.claude/skills/tdd/scripts/run_tests.sh {FRAMEWORK} "{TEST_COMMAND_FOR_SPECIFIC_TEST}"Step 8: RETRY LOOP (if test still fails):
attempt = 1
max_attempts = 5
previous_explanation = null
previous_error = null
while status != "pass" AND attempt <= max_attempts:
previous_explanation = explanation from last Implementer response
previous_error = raw_tail from last test run
Launch FRESH Task(Implementer) with:
- same test code + file tree + existing source (re-read!)
- NEW failure output
- PREVIOUS_ATTEMPT section filled in
Apply changes (Write tool for each file)
Re-run test
attempt += 1
if still failing after max_attempts:
STOP. Present to user:
"Implementation failed after 5 attempts. Last error: {raw_tail}"
Ask: "Adjust the test, try a different approach, or debug manually?"Each retry is a fresh Task call with only the previous attempt's explanation and error. This prevents the Implementer from going down rabbit holes while giving it enough context to try a different strategy.
Step 9: Once the specific test passes, run the FULL test suite:
bash ~/.claude/skills/tdd/scripts/run_tests.sh {FRAMEWORK} "{FULL_TEST_COMMAND}" --allStep 10: Handle regressions:
| Result | Action |
|---|---|
| All pass | Proceed to REFACTOR |
| Regressions found | Auto-fix: launch a fresh Implementer with the regression test failures. Apply. Re-run full suite. Repeat up to 3 times. If still failing after 3 regression-fix attempts, STOP and present to user. |
Step 11 (interactive mode only — skip in --auto): Present to the user:
GREEN: Test passing with minimal implementation.
Implementation: {explanation from agent response}
Files changed: {list}
All tests: {passed} passing, {failed} failing
Proceed to REFACTOR phase? (or adjust?)Update state: "phase": "green", update files_modified. Write state immediately.
Step 12 (domain/domain-service slices only): Layer purity check before REFACTOR:
For each new/modified file in a domain or domain-service layer slice:
- Import scan: Read all import/require statements. Check each imported module against
layer_map. Flag any import from an outer layer as a violation. - Constructor check: Verify constructor takes NO parameters typed from outer layers (no ORM sessions, HTTP clients, framework configs)
- Static call check: No static method calls to outer-layer code
- If violations found, fix them now (move the dependency to a port interface) before entering REFACTOR
Step 13: Full-repo import scan (all layers, runs once per slice):
Scan ALL source files (not just session-modified) for dependency direction violations:
# For each source file, extract imports and check against layer_map
# Language-specific patterns:
# Python: from X import Y, import X
# TypeScript/JS: import ... from 'X', require('X')
# Go: import "X"For each file: 1. Determine its layer from layer_map (skip if no match) 2. For each import, determine the imported module's layer from layer_map 3. If imported layer is OUTER relative to file's layer → violation
Report violations to the user before REFACTOR:
Layer scan found N dependency direction violation(s):
- domain/user.py imports infrastructure/db.py (domain → infrastructure)
- domain/services/registration.py imports adapters/email.py (domain-service → infrastructure)In --auto mode: attempt auto-fix (replace concrete import with port interface). In interactive mode: present violations and ask user how to proceed.
This supplements the Refactorer's import checking (which only sees session files) with a repo-wide scan. Static tools miss ~23% of violations (Pruijt et al., 2017) — combining textual + structural checks improves coverage.
---
Phase 4: REFACTOR
Step 1: Gather all context:
- All test files created/modified during this session
- All source files modified during this session
- The green test output
Step 2: Read the prompt template from references/agent_prompts.md -> "Refactorer Agent" section. Fill in:
{LANGUAGE}: Detected language{GREEN_TEST_OUTPUT}: Full test output showing all green{ALL_TEST_CODE}: Content of all test files{ALL_IMPLEMENTATION_CODE}: Content of all modified source files{SLICE_LAYERS}: Comma-separated list of unique layers from all slices completed so far
Step 3: Launch the Refactorer agent:
Task(subagent_type="general-purpose", prompt=<constructed prompt>)Step 4: Parse the JSON response. If suggestions is empty, skip to Step 6.
Apply suggestions one at a time, in priority order (high first):
For each suggestion: 1. Apply the code change (Edit tool, using old_code -> new_code for each file) 2. Run the project linter/formatter check (detect from project config):
- Python:
python -m black --check {files} && python -m flake8 {files} && python -m mypy {files} - TypeScript/JS:
npx eslint {files}ornpx tsc --noEmit - Go:
go vet ./... - Rust:
cargo clippy - If lint fails -> revert immediately and skip this suggestion (same as test failure)
3. Run the full test suite 4. If any test fails -> revert immediately (re-read the file from before the edit and Write it back) and skip this suggestion 5. If all tests pass and lint passes -> keep the change
Step 5 (interactive mode only — skip in --auto): Present:
REFACTOR: Code improved, all tests still passing.
Applied: {list of accepted suggestions}
Skipped: {list of reverted suggestions, if any}
All tests: {count} passing
[Moving to slice N of M] or [All slices complete]In --auto mode, print one-liner:
[auto] REFACTOR slice N/M: {applied_count} applied, {skipped_count} skippedUpdate state: "phase": "refactor". Write state immediately.
---
Phase 5: Next Slice or Complete
If more slices remain -> increment current_slice in state, return to Phase 2.
If all slices complete -> present summary:
TDD Complete: {feature name}
Slices implemented: N
Tests written: N
Files created/modified: {list}
All tests passing: yesClean up: remove .tdd-state.json (in --auto mode, remove silently; in interactive, ask user).
---
Resume Support
When user invokes /tdd --resume:
1. Read .tdd-state.json from project root 2. Report current state: "Found TDD session for '{feature}'. Currently at slice {N}/{total}, phase: {phase}." 3. Resume from the current phase of the current slice 4. If auto_mode is true in state, continue in auto mode
---
Edge Cases
Greenfield Projects
No source files, no tests, no test configuration. Handle gracefully:
1. Phase 0 Step 3: If run_tests.sh returns status: "error" with total: 0, check if any test files exist. If none, this is greenfield — proceed. 2. Phase 0 Step 4: extract_api.sh will return empty output. Pass "(No existing API — this is a new project)" to the Test Writer. 3. Phase 2: The Test Writer will create test files from scratch. May need to set up the test framework config (e.g., jest.config.js, pytest.ini). If the first test run fails with a framework error (not a test failure), create minimal framework config and retry.
Bug Fix TDD
1. Write a test demonstrating the bug (should FAIL showing the bug exists) 2. Confirm failure matches the reported bug — human checkpoint 3. Fix: minimal code to make test pass (GREEN phase as normal) 4. Verify: no regressions
Existing Code (Characterization Tests)
1. Write a test for CURRENT behavior (should PASS — this is a characterization test) 2. Modify the test for DESIRED behavior (should FAIL) 3. Proceed with GREEN -> REFACTOR
User-Provided Tests
If user provides test code: 1. Run to confirm it fails (RED confirmed) 2. Skip to Phase 3 (GREEN) — user-provided tests are authoritative 3. Do not modify without asking
Flaky Tests
If a test sometimes passes/fails: stop, report, fix the flaky test before continuing.
---
Failure Recovery Reference
| Failure | Phase | Recovery |
|---|---|---|
| Test Writer returns invalid JSON | RED | Parse with fence-stripping + substring extraction. Retry once with "Return ONLY JSON." Fall back to manual extraction. |
| Test passes when it should fail | RED | Log "already implemented", skip slice, move to next. |
| Test has syntax/compile error | RED | Read raw_tail, fix test file directly. Retry up to 2 times. Then ask user. |
| Implementer returns invalid JSON | GREEN | Same JSON recovery as Test Writer. |
| Test still fails after implementation | GREEN | Retry loop: up to 5 fresh Implementer calls with previous-attempt context. Then ask user. |
| Full suite has regressions | GREEN | Auto-fix: fresh Implementer with regression failures. Up to 3 attempts. Then ask user. |
| Refactorer suggestion breaks tests | REFACTOR | Revert immediately, skip suggestion, continue with next. |
| run_tests.sh timeout | Any | Increase timeout. If persistent, ask user about test performance. |
run_tests.sh returns "error" | Any | Read raw_tail for cause. Script error (missing binary, bad path) -> fix and retry. Compilation error -> treat as implementation error. |
| extract_api.sh returns empty | RED | Normal for greenfield. Pass "(No existing API)" message. |
| Agent response is completely empty | Any | Retry the Task call once. If still empty, ask user. |
---
Layer Reference
See references/layer_guide.md for layer definitions, dependency rules, test strategies by layer, and detection heuristics.
Anti-Patterns to Avoid
See references/anti_patterns.md. Critical ones:
- Never modify a test to make it pass (change implementation, not tests)
- Never write implementation before tests
- Never write all tests at once (vertical slicing)
- Never test implementation details
- Never skip the RED phase
- Never let domain code import infrastructure (dependency direction violation)
- Never mock domain objects — construct real instances instead
---
Framework Quick Reference
See references/framework_configs.md for setup details.
| Framework | Run single test | Run all | Watch mode |
|---|---|---|---|
| Jest | npx jest --testPathPattern=<file> -t "<name>" | npx jest | npx jest --watch |
| Vitest | npx vitest run <file> -t "<name>" | npx vitest run | npx vitest |
| pytest | pytest <file>::<test_name> -v | pytest -v | pytest-watch |
| Go | go test -run <TestName> ./... | go test ./... | — |
| Cargo | cargo test <test_name> | cargo test | cargo watch -x test |
| RSpec | rspec <file>:<line> | rspec | guard |
| PHPUnit | phpunit --filter <test_name> | phpunit | — |
{
"name": "tdd",
"description": "This skill should be used when the user wants to implement features or fix bugs using test-driven development. Enforces ",
"author": {
"name": "Gleb Kalinin"
},
"repository": "https://github.com/glebis/claude-skills",
"license": "MIT"
}TDD Skill for Claude Code
Multi-agent TDD orchestrator that enforces RED-GREEN-REFACTOR cycles with architecturally isolated context between test writing and implementation. The Test Writer never sees implementation code; the Implementer never sees the specification.
Usage
/tdd <feature description> # Interactive mode with human checkpoints
/tdd --auto <feature description> # Autonomous mode, stops only on errors
/tdd --resume # Resume from .tdd-state.jsonFeatures
- Context isolation: Separate agents for test writing, implementation, and refactoring with strict information boundaries
- Vertical slicing: Features decomposed into independently testable behavior slices
- DDD/Onion layer awareness: Slices sorted inside-out (domain first, infrastructure last) with layer-specific test constraints and dependency direction enforcement
- Auto-retry: Up to 5 implementation attempts per slice with fresh agent context on each retry
- Greenfield support: Works on projects with zero existing tests or source code
- Framework detection: Jest, Vitest, pytest, Go test, cargo test, RSpec, PHPUnit
Architecture
ORCHESTRATOR (SKILL.md)
|-- Phase 0: Setup -- detect framework, extract API, create state file
|-- Phase 1: Decompose into vertical slices (inside-out by layer)
|
|-- FOR EACH SLICE:
| |-- Phase 2 (RED): Task(Test Writer) <- spec + API + layer constraints
| | |-- Post-RED lint: block mocking libs in domain tests
| |-- Phase 3 (GREEN): Task(Implementer) <- failing test + error + layer deps
| | |-- Layer path validation: reject files outside slice's layer
| | |-- Domain purity check (constructors, imports, statics)
| | |-- Full-repo import scan: catch violations in untouched files
| |-- Phase 4 (REFACTOR): Task(Refactorer) <- all code + dependency direction audit
|
|-- SummaryDDD / Onion Layer Support
Each slice is tagged with a layer (domain, domain-service, application, infrastructure) which determines:
| Layer | Test constraints | Dependency rule |
|---|---|---|
| domain | No mocks, no framework imports, pure logic | Imports nothing from outer layers |
| domain-service | In-memory fakes for ports only | Imports domain only |
| application | In-memory fakes for all ports | Imports domain + domain-service |
| infrastructure | Integration tests, real deps allowed | Implements inner-layer interfaces |
Enforcement is multi-layered (not just textual reminders):
1. Path validation (GREEN phase): Implementer output files checked against layer_map — rejects writes to outer-layer directories 2. Post-RED test lint: Scans test code for mocking libraries (jest.mock, Mock(), etc.) in domain/domain-service tests 3. Domain purity check: Verifies constructors take no outer-layer types, no static calls to infrastructure 4. Full-repo import scan: Checks ALL source files (not just session-modified) for dependency direction violations 5. Refactorer audit: Checks direct + transitive dependency violations, flagging as high-priority suggestions
Port interface rule: the consumer defines the contract — ports live in the layer that needs them, not the layer that implements them.
This layer awareness degrades gracefully: for simple projects where everything lives in one layer, all slices get layer: "application" and the constraints don't add overhead.
Edge cases handled: infrastructure-only features, missing port interfaces (created by first slice that needs them), and cross-cutting slices (tagged by innermost layer touched).
File Structure
tdd/
|-- SKILL.md # Main orchestrator (read by Claude Code)
|-- README.md # This file
|-- scripts/
| |-- run_tests.sh # Test runner wrapper (JSON output)
| |-- extract_api.sh # Public API surface extractor
|-- references/
|-- agent_prompts.md # Agent prompt templates + constraint lookup
|-- anti_patterns.md # TDD and layer anti-pattern reference
|-- framework_configs.md # Per-framework test skeletons
|-- layer_guide.md # DDD/Onion layer definitions + test strategyResearch-Informed Design
The layer-aware testing approach is informed by empirical software engineering research:
- AI code + no arch constraints = 80% violation rate: LLM-generated code violates hexagonal architecture boundaries 80% of the time without explicit enforcement. Layer constraints in agent prompts directly counteract this. (arXiv:2412.02883 — TDD-Bench Verified, 2024)
- Static tools miss ~23% of dependency violations: Architecture compliance checking tools detect only 77% of dependencies on average. The refactorer supplements tooling by checking imports + transitive deps during code review. (Pruijt et al., Software: Practice and Experience, 2017)
- Test-driven prompting +38-45% accuracy: Using tests as specification improves LLM code generation accuracy by 38-45% over instruction-only prompting. RED-first is empirically better than spec-first for AI agents. (Naik et al., ICSE-Companion / IEEE TSE, 2024)
- TDD alone doesn't improve design: TDD's effect on design metrics is not as evident as expected -- the REFACTOR phase with dependency checks is where architectural quality emerges. (Turhan et al., IEEE, 2010/2017)
- TDD reduces defects 40-90%: Industrial teams using TDD saw 40-90% defect density reduction with 15-35% initial development time increase, offset by reduced maintenance. (Nagappan et al., Empirical Software Engineering, 2008)
- Over-mocking degrades LLM-generated tests: LLM-generated tests over-use mocking by 2-3x compared to human-written tests, leading to tests that pass despite broken implementations. In-memory fakes and Protocol-based test doubles produce more reliable test suites. (arXiv:2602.00409, 2025)
Full research survey: see references/layer_guide.md for citations applied to specific design decisions.
Installation
Copy tdd/ to ~/.claude/skills/tdd/ and ensure scripts/*.sh are executable:
cp -r tdd/ ~/.claude/skills/tdd/
chmod +x ~/.claude/skills/tdd/scripts/*.shAgent Prompt Templates
These templates are used by the orchestrator to construct Task tool calls with strict context isolation. Each agent receives ONLY the information listed -- nothing else.
Placeholders use {VARIABLE_NAME} syntax. Optional sections wrapped in {?SECTION}...{/SECTION} -- include only when the variable has content, omit entirely otherwise.
---
Test Writer Agent
subagent_type: general-purpose
Context boundary: Sees specification + API surface. Does NOT see implementation code, other slices, or implementation plans.
You are a TDD Test Writer. Your ONLY job is to write ONE failing test for a specific behavior.
## Specification for this slice
{SLICE_SPEC}
## Language and framework
- Language: {LANGUAGE}
- Framework: {FRAMEWORK}
- Test file location: {TEST_FILE_PATH}
## Public API surface (signatures only, no implementations)
{API_SURFACE}
Note: If the API surface is empty, the function/class does not exist yet. Write the test assuming the import path and function signature based on the specification. The Implementer will create the code.
{?DOC_CONTEXT}
## Project documentation (relevant excerpts)
{DOC_CONTEXT}
Use this documentation to understand intended behavior, API contracts, edge cases, and validation rules. Tests should align with documented behavior, not just inferred behavior from code signatures.
{/DOC_CONTEXT}
## Existing test file content (if any)
{EXISTING_TEST_CONTENT}
## Framework-specific test skeleton
{FRAMEWORK_SKELETON}
## Architectural layer for this slice
This slice belongs to the **{LAYER}** layer.
{LAYER_TEST_CONSTRAINTS}
## Mocking guidance (application layer)
When the slice is `application` layer and the code under test depends on async interfaces:
- Prefer writing a simple in-memory fake over using Mock/MagicMock with complex `.return_value` chains
- Use `AsyncMock` for any async method (not `MagicMock`)
- If a `@runtime_checkable Protocol` exists for the dependency, implement it with a fake class:
Instead of:
mock_repo = AsyncMock() mock_repo.get_by_id.return_value = User(id=1, name="test") mock_repo.save.return_value = None
Prefer (when the mock setup becomes non-trivial):
class FakeUserRepo: def __init__(self): self.saved = [] async def get_by_id(self, id: int) -> User: return User(id=id, name="test") async def save(self, user: User) -> None: self.saved.append(user)
For simple cases (1-2 methods, no state tracking), `AsyncMock` is fine. Use fakes when:
- The mock needs 3+ configured return values
- You need to track call history beyond simple assert_called
- The Protocol has methods that interact with each other
## Rules
1. Write EXACTLY ONE test function for the specified behavior
2. The test MUST fail because the implementation does not yet exist
3. Test through the public interface only -- no internal/private access
4. Use descriptive test names that read as behavior specs
5. Do NOT plan or think about implementation -- reason only from the specification
6. Do NOT write implementation code
7. Do NOT write helper functions beyond minimal test setup
8. Include all necessary imports in the test code
9. Follow the layer-specific test constraints above
## Output
Return a single JSON object. Do NOT wrap in markdown fences. Do NOT include any text before or after the JSON.
{"test_code": "the COMPLETE test code to add (including describe/it blocks, not just the assertion)", "test_name": "name of the test function", "test_description": "what behavior this test verifies", "imports_needed": "any import statements needed at the top of the file, or empty string if none"}---
Implementer Agent
subagent_type: general-purpose
Context boundary: Sees failing test + error output + existing source. Does NOT see the original specification, slice descriptions, or future plans.
You are a TDD Implementer. Your ONLY job is to write the MINIMUM code to make a failing test pass.
## Language: {LANGUAGE}
## Failing test code
{FAILING_TEST_CODE}
## Test failure output
{TEST_FAILURE_OUTPUT}
{?PREVIOUS_ATTEMPT}
## Previous attempt (failed)
The following approach was tried and still failed:
{PREVIOUS_ATTEMPT_DESCRIPTION}
Error after previous attempt:
{PREVIOUS_ATTEMPT_ERROR}
Do NOT repeat the same approach. Try a different strategy.
{/PREVIOUS_ATTEMPT}
## File tree (source files only)
{FILE_TREE}
## Existing source code (files relevant to the failing test)
{EXISTING_SOURCE}
## Architectural layer
This code belongs to the **{LAYER}** layer.
{LAYER_DEPENDENCY_CONSTRAINT}
## Rules
1. Write the MINIMUM code to make the failing test pass
2. No code beyond what the test requires
3. No premature abstractions or extra error handling
4. No optimization -- simple and direct
5. Hardcoded values are acceptable if they satisfy the test
6. Do NOT modify the test file
7. Do NOT add features or behaviors not tested
8. For NEW files or files under 200 lines: return the COMPLETE file content
9. For EXISTING files over 200 lines: return ONLY the new/changed functions plus enough surrounding context (imports, class declaration line) for the orchestrator to apply as an edit. Set `"action": "edit"` for these files.
10. Respect the layer dependency constraint above -- do NOT import from outer layers
## Output
Return a single JSON object. Do NOT wrap in markdown fences. Do NOT include any text before or after the JSON.
{"files": [{"path": "relative/path/to/file.ext", "action": "create | overwrite | edit", "content": "COMPLETE file content for create/overwrite, or ONLY changed functions with context for edit", "description": "what this file does"}], "explanation": "brief explanation of the implementation approach"}
Use `"action": "edit"` for existing files over 200 lines. For edit actions, include the function(s) being added/changed with their imports and class context -- the orchestrator will use Edit tool (old_string → new_string) to apply.Key change from v1: always return complete file content (no partial patches). The orchestrator uses the Write tool for creates, and for existing files it compares the returned content against the current content to determine what changed.
---
Refactorer Agent
subagent_type: general-purpose
Context boundary: Sees all implementation + all tests + green test results. Does NOT see the original specification or decomposition rationale.
You are a TDD Refactorer. All tests are currently passing. Your job is to suggest code improvements that preserve behavior.
## Language: {LANGUAGE}
## Current test results (all green)
{GREEN_TEST_OUTPUT}
## All test code
{ALL_TEST_CODE}
## All implementation code
{ALL_IMPLEMENTATION_CODE}
## Layers touched in this session
{SLICE_LAYERS}
## Rules
1. Do NOT change behavior -- all existing tests must continue to pass
2. Focus on: extracting duplication, improving naming, simplifying logic, applying appropriate patterns
3. Do NOT add new features, new tests, or new error handling
4. Each suggestion must be independently applicable (revert-safe)
5. Prefer small, targeted improvements over large restructurings
6. Apply the Rule of Three -- don't extract abstractions unless a pattern appears 3+ times
7. If no meaningful refactoring is needed, say so -- that's a valid outcome
8. Check all import statements for dependency direction violations: inner layers must NOT import from outer layers. Direction: domain → domain-service → application → infrastructure. Flag any violation as a HIGH priority suggestion.
9. Check for TRANSITIVE dependency violations: if file A imports file B, and B imports from an outer layer, then A has an indirect dependency on that outer layer. Trace one level deep: for each import in domain/domain-service code, check what THAT module imports. Flag transitive violations as HIGH priority with a note explaining the chain (e.g., "domain/User imports domain/validators which imports infrastructure/db — indirect violation").
10. Domain purity check: verify domain layer classes take NO constructor parameters whose types come from outer layers (no ORM session, no HTTP client, no framework config objects). Flag as HIGH priority.
## Output
Return a single JSON object. Do NOT wrap in markdown fences. Do NOT include any text before or after the JSON.
If refactoring is suggested:
{"suggestions": [{"description": "what this refactoring does", "priority": "high or medium or low", "files": [{"path": "relative/path/to/file.ext", "old_code": "exact code to find and replace", "new_code": "replacement code"}]}], "summary": "overall assessment of code quality"}
If no refactoring is needed:
{"suggestions": [], "summary": "Code is clean. No refactoring needed at this stage."}---
Notes for the Orchestrator
JSON Response Parsing
Agents frequently wrap output in markdown fences despite instructions. Parse robustly:
def parse_agent_json(response_text):
text = response_text.strip()
# Strip markdown fences
if text.startswith("```"):
lines = text.split("\n")
text = "\n".join(lines[1:]) # remove first line (```json or ```)
if text.rstrip().endswith("```"):
text = text.rstrip()[:-3]
text = text.strip()
# Try direct parse
try:
return json.loads(text)
except json.JSONDecodeError:
pass
# Find JSON substring
first_brace = text.find("{")
last_brace = text.rfind("}")
if first_brace != -1 and last_brace != -1:
return json.loads(text[first_brace:last_brace + 1])
raise ValueError(f"Could not parse JSON from agent response")Context Construction Checklist
When building each agent's prompt, verify these constraints:
Test Writer:
- [x] Contains slice spec
- [x] Contains API surface from extract_api.sh (run fresh each time)
- [x] Contains doc context from discover_docs.sh (if available — omit section if empty)
- [x] Contains language and framework
- [x] Contains existing test file content (if any)
- [x] Contains layer tag and layer-specific test constraints
- [ ] Does NOT contain any implementation source code
- [ ] Does NOT contain other slice descriptions
- [ ] Does NOT contain the feature description beyond the current slice
Implementer:
- [x] Contains the complete test file
- [x] Contains the test failure output (raw_tail from run_tests.sh)
- [x] Contains the file tree
- [x] Contains relevant existing source files
- [x] Contains previous attempt context (on retries)
- [x] Contains layer tag and layer-specific dependency constraint
- [ ] Does NOT contain the slice spec or feature description
- [ ] Does NOT contain future slice plans
Refactorer:
- [x] Contains ALL test files from this session
- [x] Contains ALL modified source files
- [x] Contains the green test output
- [x] Contains layer list for dependency direction checking
- [ ] Does NOT contain the original specification
- [ ] Does NOT contain the decomposition rationale
Retry Strategy for Implementer
On each retry (attempt > 1), add the {?PREVIOUS_ATTEMPT} section with:
PREVIOUS_ATTEMPT_DESCRIPTION: theexplanationfield from the failed attempt's JSON responsePREVIOUS_ATTEMPT_ERROR: theraw_tailfrom the new test run after applying the failed attempt
This gives the fresh agent enough context to avoid the same mistake without accumulating a long failure history.
Error Recovery
If an agent returns invalid JSON: 1. Apply the parse_agent_json logic above (handles fences + substring extraction) 2. If still invalid, retry the same Task call once with an appended instruction: "IMPORTANT: Your previous response was not valid JSON. Return ONLY a JSON object, nothing else." 3. If still failing after retry, fall back to the orchestrator reading the raw response and attempting to extract the relevant information manually (read the test code, file paths, etc.) 4. If all extraction fails, present the raw response to the user and ask how to proceed
Layer-Specific Constraint Lookup
When constructing agent prompts, substitute {LAYER_TEST_CONSTRAINTS} and {LAYER_DEPENDENCY_CONSTRAINT} based on the slice's layer:
{LAYER_TEST_CONSTRAINTS} for the Test Writer:
domain: "Write tests using only domain types. NO database mocks, NO HTTP mocks, NO file system. Test pure business logic through public methods. Construct real domain objects directly — never mock them."domain-service: "Use in-memory fakes that implement repository/port interfaces. Test the domain service's coordination logic. NO real I/O, NO database, NO HTTP. Use real domain objects from the domain layer."application: "Use in-memory fakes for all ports and repositories. Prefer writing a 5-line in-memory fake class that implements the Protocol/interface over configuring a complexMock(spec=...)with multiple return values. UseAsyncMockfor async interfaces. Test the orchestration flow — that the use case calls the right domain operations in the right order. NO real infrastructure."infrastructure: "Test that the adapter correctly translates between domain types and external formats (SQL rows, HTTP responses, file contents). May use integration test patterns with real dependencies or test containers."
{LAYER_DEPENDENCY_CONSTRAINT} for the Implementer:
domain: "This is the innermost layer. It MUST NOT import anything from domain-service, application, or infrastructure layers. No ORM imports, no HTTP clients, no framework imports. Only standard library and domain types."domain-service: "This layer may import from the domain model only. It MUST NOT import from application or infrastructure layers. If this service needs an external dependency (e.g., repository), define the port interface in the domain or domain-service layer — the consumer defines the contract."application: "This layer may import from domain model and domain services. It MUST NOT import from infrastructure. If this use case needs an external dependency not covered by an existing port, define the port interface here — the consumer defines the contract."infrastructure: "This layer may import from all inner layers. It implements interfaces defined in inner layers. Framework and external library imports are expected here."
{SLICE_LAYERS} for the Refactorer:
Comma-separated list of unique layers from all slices completed so far. Example: "domain, application, infrastructure".
Applying Implementer Output
For each file in the response files array, check the action field:
1. "create": Use Write tool to create the new file 2. "overwrite" (file ≤ 200 lines): Use Write tool to overwrite with the returned content 3. "overwrite" (file > 200 lines): Prefer using Edit tool to apply only the actual changes — diff the returned content against the current file and apply targeted edits. This prevents accidental reformatting of large files. 4. "edit": The Implementer returned only changed/new functions with context. Use the Edit tool with old_string → new_string:
- For new functions: identify the insertion point (end of class, after last import, etc.) and use Edit to insert
- For modified functions: use the existing function as
old_stringand the modified version asnew_string - For new imports: prepend to the existing import block
After applying all files, run the test immediately. If the full test suite catches regressions from a Write-based overwrite, consider re-applying via Edit instead.
TDD Anti-Patterns Reference
Phase Violations
Writing implementation before tests
Symptom: Code appears in source files before a corresponding test exists. Fix: Delete the implementation. Write the test first. Period. Why it matters: Implementation-first means the test is verifying what was built, not specifying what should be built. The test becomes a rubber stamp, not a design tool.
Writing all tests at once (horizontal slicing)
Symptom: Multiple test functions written in a batch before any implementation. Fix: Write exactly one test. Make it fail. Make it pass. Refactor. Then write the next test. Why it matters: Batch testing leads to batch implementation. The feedback loop widens, errors compound, and refactoring becomes risky because too many things change at once.
Skipping the RED phase
Symptom: A test is written and passes immediately without implementation. Fix: If the test passes without new code, either (a) the behavior already exists, (b) the test is trivially passing (wrong assertion), or (c) the test setup is wrong. Investigate. Why it matters: A test that was never seen failing provides no confidence. It might pass for the wrong reasons.
Modifying tests to match implementation
Symptom: After writing implementation, the test is changed to match what the code does rather than what the spec requires. Fix: The test encodes the REQUIREMENT. If the implementation doesn't match, fix the implementation. Only change the test if the user confirms the requirement was wrong. Why it matters: This inverts the authority chain. Tests are specifications; implementation serves them, not the other way around.
Test Quality Anti-Patterns
Testing implementation details
Symptom: Tests assert on private methods, internal state, call counts of mocked internals, or specific algorithm steps. Examples:
expect(service._cache).toHaveLength(3)-- testing private cacheexpect(mockDb.query).toHaveBeenCalledTimes(2)-- testing internal query patternexpect(result.__internal_flag).toBe(true)-- testing private state
Fix: Test through public interfaces only. Assert on return values, side effects visible to callers, or observable state changes. Why it matters: Implementation-detail tests break on every refactor, even when behavior is preserved. They test HOW the code works, not WHAT it does.
Testing the framework, not the code
Symptom: Tests that verify the test framework, mocking library, or ORM works correctly. Examples:
- Mocking a database then asserting the mock returns the mocked value
- Testing that
JSON.parse(JSON.stringify(x))round-trips correctly
Fix: Tests should verify YOUR code's behavior, not third-party behavior.
Tautological tests
Symptom: Tests where the assertion is trivially true regardless of implementation. Examples:
expect(true).toBe(true)expect(result).toBeDefined()(where result is always defined by the function signature)- Asserting a function returns without throwing when it has no throw paths
Fix: Every assertion must be capable of failing given a plausible incorrect implementation.
Over-mocking
Symptom: More mock setup code than actual test code. Every dependency is mocked. Fix: Use real implementations where practical. Mock only at system boundaries (network, filesystem, clock). Prefer integration tests with in-memory fakes over unit tests with extensive mocks. Why it matters: Over-mocked tests pass even when integration is broken. They test the wiring, not the behavior.
Structural Anti-Patterns
God test
Symptom: A single test function that tests multiple behaviors with multiple assertions and complex setup. Fix: Split into one test per behavior. Each test should have one reason to fail. Pattern: Arrange-Act-Assert, each section clearly delineated.
Test interdependence
Symptom: Tests that depend on execution order, shared mutable state, or other tests' side effects. Fix: Each test sets up its own state and tears it down. Tests must pass when run in isolation or in any order.
Fragile test fixtures
Symptom: A change in test setup code breaks many unrelated tests. Fix: Use builder patterns or factory functions that provide sensible defaults. Each test overrides only what it cares about.
Testing trivial code
Symptom: Tests for getters, setters, constructors, or obvious one-liners. Fix: Skip tests for code with zero logic. Focus testing on code with conditionals, loops, transformations, or business rules.
Process Anti-Patterns
Gold plating during GREEN
Symptom: Implementation during the GREEN phase includes extra features, optimization, or error handling not required by the current test. Fix: Write the absolute minimum to make the test pass. If you want to add more, write a test for it first.
Skipping REFACTOR
Symptom: After GREEN, immediately writing the next test without cleaning up. Fix: Always assess the code after GREEN. Even if no refactoring is needed, consciously evaluate. Refactoring is where design emerges.
Premature refactoring
Symptom: Extracting abstractions after only one or two instances of a pattern. Fix: Wait for the "Rule of Three" -- extract abstractions only after seeing a pattern three times. In early TDD cycles, duplication is acceptable.
Ignoring test failures in the full suite
Symptom: A new test passes but existing tests break, and the broken tests are dismissed as "unrelated." Fix: Every test failure after GREEN is a regression until proven otherwise. Investigate and fix before moving to REFACTOR.
Layer & Dependency Anti-Patterns
Domain importing infrastructure
Symptom: Domain model code imports ORM classes, HTTP clients, file system modules, or framework utilities. Examples:
from sqlalchemy.orm import Sessionin a domain entityimport axios from 'axios'in a domain serviceuse Illuminate\Database\Eloquent\Modelin a domain value object
Fix: Domain code must have zero external dependencies. If the domain needs to persist or communicate, define an interface (port) in the domain layer and let infrastructure implement it. Why it matters: Domain code that imports infrastructure cannot be tested without that infrastructure. It also locks business logic to a specific technology choice.
Business logic in handlers/controllers
Symptom: Validation rules, calculations, state transitions, or conditional logic living in HTTP handlers, CLI commands, or event listeners instead of domain objects or services. Examples:
- Price calculation in an Express route handler
- Email format validation in a controller
- Order state machine transitions in a message consumer
Fix: Extract the logic into a domain entity, value object, or domain service. The handler should only translate HTTP/CLI/event input into domain calls and translate domain output back. Why it matters: Business logic in handlers is untestable without spinning up the framework. It also gets duplicated when you add a second entry point (API + CLI + queue consumer).
Mocking domain objects
Symptom: Using jest.mock(), unittest.mock.Mock(), or similar to create fake domain entities or value objects instead of constructing real instances. Examples:
const user = { validate: jest.fn().mockReturnValue(true) }instead ofnew User("valid@email.com")mock_order = Mock(spec=Order)instead ofOrder(items=[item1, item2])
Fix: Domain objects are pure and cheap to construct. Use real instances in tests. Only mock at boundaries (repositories, external services). Why it matters: Mocking domain objects defeats the purpose of testing — you're testing your mocks, not your domain logic. If a domain object is hard to construct, that's a design smell.
Anemic domain model
Symptom: Entities with only getters/setters, all logic in services. Tests pass but the design is wrong — behavior is disconnected from the data it operates on. Examples:
Userclass with onlyname,emailproperties;UserService.validateUser(user)does all validationOrderwithitemslist;OrderCalculator.calculateTotal(order)computes the total externally
Fix: Move behavior onto the entity that owns the data. user.validate(), order.calculateTotal(). Services coordinate; entities compute. Why it matters: Anemic models scatter related logic across services, making it harder to find, test, and enforce invariants. It's procedural code wearing OOP clothing.
Repository interface in wrong layer
Symptom: The repository interface (UserRepository, OrderRepository) is defined alongside its implementation in the infrastructure layer, rather than in the domain layer. Examples:
infrastructure/repositories/user_repository.pycontains both the interface and the PostgreSQL implementationsrc/database/UserRepository.tsdefines the interface and exports it
Fix: Define the interface in the consuming layer (domain/ports/user_repository.py if consumed by domain services, application/ports/ if consumed by use cases). The infrastructure layer imports and implements it. The consumer defines the contract. Why it matters: If the interface lives in infrastructure, domain code must import infrastructure to reference it — breaking the dependency rule.
Service locator / static global container
Symptom: Domain or application code obtains dependencies through a global registry, static container, or service locator instead of constructor injection. Examples:
Container.resolve(UserRepository)called inside a domain service methodServiceLocator.get('emailService')in a use caseapp.make('UserRepository')(Laravel) inside domain code@injectdecorators that resolve from a global container at import time
Fix: Accept dependencies through the constructor (or function parameters). The composition root (in infrastructure) wires everything together. Domain and application code never knows where implementations come from. Why it matters: Service locators hide dependencies — the class signature doesn't reveal what it needs. Tests require configuring a global container instead of passing fakes directly. It also makes dependency direction invisible: a domain class appears independent but actually reaches into infrastructure at runtime.
Active Record bleed (ORM entity as domain object)
Symptom: The same class serves as both the database model (ORM entity) and the domain object. Framework-specific annotations, base classes, or conventions leak into domain logic. Examples:
class User extends Model(Eloquent/ActiveRecord) used directly in domain services@Entity() class Order(TypeORM) with both column decorators and business logic- Domain code calling
.save(),.delete(), or.query()on domain objects - SQLAlchemy
Basesubclass used as a domain entity
Fix: Separate the persistence model from the domain model. The infrastructure layer maps between them. Domain entities have zero ORM awareness. Why it matters: Active Record couples business logic to the database schema and ORM framework. Domain tests require a database (or complex mocking). Schema changes break business logic. The domain layer becomes untestable without infrastructure — violating the core DDD/Onion principle.
Test Framework Configuration Reference
JavaScript / TypeScript
Jest
Detection: jest in package.json devDependencies, or jest.config.js/ts/mjs
Run commands:
# All tests
npx jest
# Single file
npx jest path/to/file.test.ts
# Single test by name
npx jest --testPathPattern=file.test.ts -t "should calculate total"
# Watch mode
npx jest --watch
# With coverage
npx jest --coverageTest file conventions: *.test.ts, *.test.js, *.spec.ts, *.spec.js, or files in __tests__/
Minimal test skeleton:
describe("ModuleName", () => {
it("should [behavior]", () => {
// Arrange
const input = ...;
// Act
const result = functionUnderTest(input);
// Assert
expect(result).toBe(expected);
});
});Vitest
Detection: vitest in package.json devDependencies, or vitest.config.ts/js
Run commands:
# All tests (run once)
npx vitest run
# Single file
npx vitest run path/to/file.test.ts
# Single test by name
npx vitest run path/to/file.test.ts -t "should calculate total"
# Watch mode (default)
npx vitest
# With coverage
npx vitest run --coverageTest file conventions: Same as Jest. Vitest is API-compatible with Jest.
Minimal test skeleton: Same as Jest (uses same describe/it/expect API).
Mocha + Chai
Detection: mocha in package.json, .mocharc.yml
Run commands:
npx mocha "test/**/*.test.js"
npx mocha --grep "should calculate"---
Python
pytest
Detection: pytest in pyproject.toml/setup.cfg, pytest.ini, conftest.py
Run commands:
# All tests
pytest -v
# Single file
pytest tests/test_module.py -v
# Single test
pytest tests/test_module.py::test_function_name -v
# By keyword match
pytest -k "calculate and total" -v
# With coverage
pytest --cov=src -v
# Stop on first failure
pytest -x -vTest file conventions: test_*.py or *_test.py, functions prefixed with test_
Minimal test skeleton:
def test_should_calculate_total():
# Arrange
items = [Item(price=10), Item(price=20)]
# Act
result = calculate_total(items)
# Assert
assert result == 30With classes:
class TestCalculator:
def test_should_add_numbers(self):
calc = Calculator()
assert calc.add(2, 3) == 5
def test_should_handle_negative(self):
calc = Calculator()
assert calc.add(-1, 1) == 0Fixtures:
import pytest
@pytest.fixture
def calculator():
return Calculator()
def test_should_add(calculator):
assert calculator.add(2, 3) == 5pytest-asyncio
Detection: pytest-asyncio in pyproject.toml/requirements, or asyncio_mode in pytest.ini/pyproject.toml
asyncio_mode detection: Check pytest.ini and pyproject.toml for asyncio_mode = auto. When auto mode is set, @pytest.mark.asyncio is not needed — all async def test_* functions are automatically collected as async tests.
Run commands: Same as pytest (no difference for async tests).
Async test skeleton (asyncio_mode = auto):
from unittest.mock import AsyncMock, MagicMock
async def test_should_process_message():
# Arrange
service = MessageService()
mock_repo = AsyncMock()
mock_repo.save.return_value = None
# Act
result = await service.process("hello", repo=mock_repo)
# Assert
assert result.status == "processed"
mock_repo.save.assert_awaited_once()Async test skeleton (asyncio_mode = strict, or not set):
import pytest
from unittest.mock import AsyncMock
@pytest.mark.asyncio
async def test_should_process_message():
service = MessageService()
result = await service.process("hello")
assert result.status == "processed"Async fixtures:
import pytest
from unittest.mock import AsyncMock, MagicMock
@pytest.fixture
def mock_bot():
bot = MagicMock()
bot.send_message = AsyncMock()
return bot
@pytest.fixture
def mock_update():
update = MagicMock()
update.effective_chat.id = 12345
update.message.text = "test"
return update
@pytest.fixture
async def db_session():
"""Async fixture with setup and teardown."""
session = await create_test_session()
yield session
await session.close()Key patterns:
- Use
AsyncMockfor any method that isasync def— it returns an awaitable - Use
MagicMockfor synchronous attributes/properties on async objects assert_awaited_once()/assert_awaited_once_with(...)— async equivalents ofassert_called_once- Async fixtures use
async def+yieldfor setup/teardown
---
Go
go test
Detection: go.mod in project root
Run commands:
# All tests
go test ./...
# Single package
go test ./pkg/calculator/
# Single test
go test -run TestCalculateTotal ./pkg/calculator/
# Verbose
go test -v ./...
# With coverage
go test -cover ./...
# Race detection
go test -race ./...Test file conventions: *_test.go in same package
Minimal test skeleton:
func TestShouldCalculateTotal(t *testing.T) {
// Arrange
items := []Item{{Price: 10}, {Price: 20}}
// Act
result := CalculateTotal(items)
// Assert
if result != 30 {
t.Errorf("expected 30, got %d", result)
}
}Table-driven tests (idiomatic Go):
func TestCalculateTotal(t *testing.T) {
tests := []struct {
name string
items []Item
expected int
}{
{"empty list", []Item{}, 0},
{"single item", []Item{{Price: 10}}, 10},
{"multiple items", []Item{{Price: 10}, {Price: 20}}, 30},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := CalculateTotal(tt.items)
if result != tt.expected {
t.Errorf("expected %d, got %d", tt.expected, result)
}
})
}
}---
Rust
cargo test
Detection: Cargo.toml
Run commands:
# All tests
cargo test
# Single test
cargo test test_name
# Single module
cargo test module_name::
# With output
cargo test -- --nocapture
# Watch mode (requires cargo-watch)
cargo watch -x testTest file conventions: #[cfg(test)] module in source files, or tests/ directory for integration tests
Minimal test skeleton:
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn should_calculate_total() {
let items = vec![Item { price: 10 }, Item { price: 20 }];
let result = calculate_total(&items);
assert_eq!(result, 30);
}
}---
Ruby
RSpec
Detection: rspec in Gemfile, .rspec file, spec/ directory
Run commands:
# All tests
rspec
# Single file
rspec spec/calculator_spec.rb
# Single test by line
rspec spec/calculator_spec.rb:15
# By description
rspec -e "should calculate total"
# With documentation format
rspec --format documentationTest file conventions: spec/**/*_spec.rb
Minimal test skeleton:
RSpec.describe Calculator do
describe "#total" do
it "should calculate total for multiple items" do
items = [Item.new(price: 10), Item.new(price: 20)]
calculator = Calculator.new(items)
result = calculator.total
expect(result).to eq(30)
end
end
end---
PHP
PHPUnit
Detection: phpunit in composer.json, phpunit.xml
Run commands:
# All tests
./vendor/bin/phpunit
# Single file
./vendor/bin/phpunit tests/CalculatorTest.php
# Single test
./vendor/bin/phpunit --filter testShouldCalculateTotal
# With coverage
./vendor/bin/phpunit --coverage-textTest file conventions: tests/**/*Test.php, classes extend TestCase
Minimal test skeleton:
class CalculatorTest extends TestCase
{
public function testShouldCalculateTotal(): void
{
$items = [new Item(price: 10), new Item(price: 20)];
$calculator = new Calculator($items);
$result = $calculator->total();
$this->assertEquals(30, $result);
}
}DDD / Onion Layer Reference
Compact reference for layer-aware TDD. Not a DDD textbook — just enough for agents to enforce dependency direction and choose the right test strategy.
Why This Matters (Research-Backed)
- AI-generated code without architectural constraints produces 80% dependency violation rates in hexagonal architecture (arXiv:2512.04273, 2025). Layer constraints in agent prompts directly counteract this.
- Static tools detect only ~77% of dependency violations on average (Pruijt et al., 2017). Our refactorer supplements tooling by checking imports + transitive deps during code review.
- Test-driven prompting improves LLM code generation accuracy by 38-45% (Naik et al., 2024). RED-first with focused prompts is empirically better than spec-first.
- TDD alone doesn't automatically improve design (Turhan et al., 2017). The REFACTOR phase with dependency checks is essential, not optional — it's where architectural quality emerges.
Layer Definitions (Inside → Outside)
Domain Model (Core)
- Entities, Value Objects, Aggregates, Domain Events
- ZERO external dependencies (no ORM, no HTTP, no frameworks)
- Test with: direct construction, no mocks
- Example:
Userentity with email validation,Moneyvalue object with currency math
Domain Services
- Complex operations spanning multiple aggregates
- Depends on: Domain Model only (+ interfaces for ports it needs)
- Test with: real domain objects, in-memory fakes for repository interfaces
- Example:
RegistrationServicechecking uniqueness via a repository interface
Application Services (Use Cases)
- Orchestrate domain logic, handle transactions, coordinate side effects
- Depends on: Domain Model + Domain Services
- Defines port interfaces that infrastructure will implement
- Test with: in-memory fakes for all ports/repositories
- Example:
RegisterUserUseCasecoordinating validation, persistence, email notification
Infrastructure & Presentation
- DB access, external APIs, controllers, CLI handlers, adapters
- Depends on: all inner layers (implements their interfaces)
- Test with: integration tests, may use real dependencies or test containers
- Example:
PostgresUserRepositoryimplementingUserRepositoryinterface
Port Interface Placement
Ports (interfaces that abstract external dependencies) are defined by the layer that consumes them, not the layer that implements them:
- A
UserRepositoryinterface consumed by a domain service lives indomain/ports/ordomain-service/ports/ - A
NotificationServiceinterface consumed by an application use case lives inapplication/ports/ - Infrastructure implements these interfaces but never defines them
Rule: the consumer defines the contract; the provider fulfills it. This is the Dependency Inversion Principle.
Dependency Rule
Dependencies flow INWARD only. Inner layers define interfaces (ports); outer layers implement them (adapters).
Infrastructure → Application → Domain Services → Domain Model
↓ ↓ ↓ ↓
implements orchestrates uses real pure logic
interfaces via ports domain objects no importsViolation example: domain/user.py importing from infrastructure.db import Session — domain must never import infrastructure.
Layer Detection Heuristics
When classifying a slice, ask: 1. Does it involve only business rules with no I/O? → domain 2. Does it coordinate multiple domain objects but still no I/O? → domain-service 3. Does it orchestrate a workflow (validate, persist, notify)? → application 4. Does it talk to a database, HTTP API, file system, or framework? → infrastructure
Test Strategy by Layer
| Layer | Mocks/Fakes | Framework imports | I/O allowed |
|---|---|---|---|
| domain | None | None | No |
| domain-service | In-memory fakes for ports | None | No |
| application | In-memory fakes for all ports | Minimal (DI container) | No |
| infrastructure | Optional (real deps or test containers) | Yes | Yes |
#!/usr/bin/env bash
# Discover and extract project documentation relevant to TDD spec writing.
# Usage: discover_docs.sh <project_dir> [--lang <language>]
#
# Searches for:
# 1. Documentation files (README, docs/, ARCHITECTURE, CONTRIBUTING, ADRs)
# 2. API specification files (OpenAPI/Swagger, GraphQL schemas, .proto)
# 3. Inline docstrings from source code (JSDoc, Google-style, Rust doc comments)
#
# Output: structured text summary of discovered documentation,
# fed to Phase 1 (decomposition) and Phase 2 (Test Writer) as context.
set -euo pipefail
PROJECT_DIR="${1:?Usage: discover_docs.sh <project_dir> [--lang <language>]}"
LANG_OVERRIDE=""
shift
while [[ $# -gt 0 ]]; do
case "$1" in
--lang) LANG_OVERRIDE="$2"; shift 2 ;;
*) shift ;;
esac
done
MAX_DOC_LINES=200 # cap per-file extraction to keep context reasonable
MAX_TOTAL_CHARS=15000 # hard cap on total output
# Track total chars emitted
TOTAL_CHARS=0
emit() {
local text="$1"
local len=${#text}
if (( TOTAL_CHARS + len > MAX_TOTAL_CHARS )); then
local remaining=$((MAX_TOTAL_CHARS - TOTAL_CHARS))
if (( remaining > 50 )); then
echo "${text:0:$remaining}"
echo "... (truncated — doc discovery capped at ${MAX_TOTAL_CHARS} chars)"
fi
TOTAL_CHARS=$MAX_TOTAL_CHARS
return 1 # signal to stop
fi
echo "$text"
TOTAL_CHARS=$((TOTAL_CHARS + len + 1)) # +1 for newline
return 0
}
# ── Section 1: Documentation files ──────────────────────────────────
emit "# Project Documentation" || exit 0
emit "" || exit 0
# Find markdown/text docs (not in node_modules, vendor, etc.)
DOC_FILES=$(find "$PROJECT_DIR" -maxdepth 3 -type f \
\( -iname 'README*' -o -iname 'ARCHITECTURE*' -o -iname 'CONTRIBUTING*' \
-o -iname 'DESIGN*' -o -iname 'SPEC*' -o -iname 'API*' \
-o -iname 'CHANGELOG*' \) \
-not -path '*/node_modules/*' -not -path '*/vendor/*' \
-not -path '*/.git/*' -not -path '*/target/*' \
-not -path '*/dist/*' -not -path '*/build/*' \
-not -path '*/venv/*' -not -path '*/__pycache__/*' \
2>/dev/null | sort || true)
# Also check docs/ directory
if [[ -d "$PROJECT_DIR/docs" ]]; then
DOCS_DIR_FILES=$(find "$PROJECT_DIR/docs" -maxdepth 2 -type f \
\( -name '*.md' -o -name '*.txt' -o -name '*.rst' \) \
2>/dev/null | sort || true)
DOC_FILES=$(printf '%s\n%s' "$DOC_FILES" "$DOCS_DIR_FILES" | sort -u)
fi
# Also check doc/ directory
if [[ -d "$PROJECT_DIR/doc" ]]; then
DOC_DIR_FILES=$(find "$PROJECT_DIR/doc" -maxdepth 2 -type f \
\( -name '*.md' -o -name '*.txt' -o -name '*.rst' \) \
2>/dev/null | sort || true)
DOC_FILES=$(printf '%s\n%s' "$DOC_FILES" "$DOC_DIR_FILES" | sort -u)
fi
if [[ -n "$DOC_FILES" ]]; then
while IFS= read -r file; do
[[ -z "$file" ]] && continue
rel="${file#$PROJECT_DIR/}"
emit "## $rel" || exit 0
head -n "$MAX_DOC_LINES" "$file" | while IFS= read -r line; do
emit "$line" || exit 0
done
emit "" || exit 0
done <<< "$DOC_FILES"
else
emit "(No documentation files found)" || exit 0
emit "" || exit 0
fi
# ── Section 2: API specification files ──────────────────────────────
emit "# API Specifications" || exit 0
emit "" || exit 0
API_SPECS=$(find "$PROJECT_DIR" -maxdepth 4 -type f \
\( -name 'openapi.*' -o -name 'swagger.*' \
-o -name '*.openapi.json' -o -name '*.openapi.yaml' -o -name '*.openapi.yml' \
-o -name 'schema.graphql' -o -name '*.graphqls' \
-o -name '*.proto' \
-o -name 'api-spec.*' \) \
-not -path '*/node_modules/*' -not -path '*/vendor/*' \
-not -path '*/.git/*' \
2>/dev/null | sort || true)
if [[ -n "$API_SPECS" ]]; then
while IFS= read -r file; do
[[ -z "$file" ]] && continue
rel="${file#$PROJECT_DIR/}"
emit "## $rel" || exit 0
head -n "$MAX_DOC_LINES" "$file" | while IFS= read -r line; do
emit "$line" || exit 0
done
emit "" || exit 0
done <<< "$API_SPECS"
else
emit "(No API specification files found)" || exit 0
emit "" || exit 0
fi
# ── Section 3: Docstrings from source code ──────────────────────────
emit "# Source Docstrings" || exit 0
emit "" || exit 0
# Auto-detect language if not overridden
if [[ -z "$LANG_OVERRIDE" ]]; then
ts_count=$(find "$PROJECT_DIR" -name '*.ts' -not -path '*/node_modules/*' 2>/dev/null | head -20 | wc -l | tr -d ' ')
py_count=$(find "$PROJECT_DIR" -name '*.py' -not -path '*/venv/*' -not -path '*/__pycache__/*' 2>/dev/null | head -20 | wc -l | tr -d ' ')
go_count=$(find "$PROJECT_DIR" -name '*.go' -not -path '*/vendor/*' 2>/dev/null | head -20 | wc -l | tr -d ' ')
rs_count=$(find "$PROJECT_DIR" -name '*.rs' -not -path '*/target/*' 2>/dev/null | head -20 | wc -l | tr -d ' ')
max=0; LANG="unknown"
for pair in "typescript:$ts_count" "python:$py_count" "go:$go_count" "rust:$rs_count"; do
l="${pair%%:*}"; c="${pair##*:}"
if [[ "$c" -gt "$max" ]]; then max="$c"; LANG="$l"; fi
done
[[ "$ts_count" -gt 0 ]] && LANG="typescript"
else
LANG="$LANG_OVERRIDE"
fi
extract_docstrings_python() {
python3 -c "
import ast, sys, os
project = sys.argv[1]
max_lines = int(sys.argv[2])
count = 0
for root, dirs, files in os.walk(project):
# Skip irrelevant dirs
dirs[:] = [d for d in dirs if d not in ('__pycache__', 'venv', '.venv', 'node_modules', '.git', 'tests', 'test')]
for f in sorted(files):
if not f.endswith('.py') or f.startswith('test_') or f.endswith('_test.py'):
continue
path = os.path.join(root, f)
rel = os.path.relpath(path, project)
try:
with open(path) as fh:
tree = ast.parse(fh.read())
except Exception:
continue
file_docs = []
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)):
doc = ast.get_docstring(node)
if doc and not node.name.startswith('_'):
# Truncate long docstrings
lines = doc.strip().split('\n')
if len(lines) > 8:
lines = lines[:8] + ['...']
file_docs.append(f' {node.name}: {chr(10).join(\" \" + l for l in lines)}')
if file_docs:
print(f'## {rel}')
for d in file_docs:
print(d)
count += 1
print()
if count > max_lines:
print('... (truncated)')
sys.exit(0)
" "$PROJECT_DIR" "$MAX_DOC_LINES" 2>/dev/null || true
}
extract_docstrings_typescript() {
# Extract JSDoc comments (/** ... */) attached to exports
find "$PROJECT_DIR" -type f \( -name '*.ts' -o -name '*.tsx' -o -name '*.js' -o -name '*.jsx' \) \
-not -name '*.test.*' -not -name '*.spec.*' -not -name '*.d.ts' \
-not -path '*/node_modules/*' -not -path '*/dist/*' -not -path '*/build/*' \
-not -path '*/__tests__/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
rel="${file#$PROJECT_DIR/}"
# Use python to extract JSDoc + following signature
docs=$(python3 -c "
import re, sys
with open(sys.argv[1]) as f:
content = f.read()
# Match JSDoc blocks followed by export declarations
pattern = r'(/\*\*[\s\S]*?\*/)\s*\n\s*(export\s+(?:default\s+)?(?:async\s+)?(?:function|class|const|let|var|type|interface|enum)\s+\w+)'
matches = re.findall(pattern, content)
for jsdoc, sig in matches[:15]:
# Compact the JSDoc
lines = jsdoc.strip().split('\n')
if len(lines) > 6:
lines = lines[:6] + [' * ...', ' */']
print(' ' + sig.split('{')[0].split('(')[0].strip())
for l in lines:
print(' ' + l.strip())
print()
" "$file" 2>/dev/null || true)
if [[ -n "$docs" ]]; then
emit "## $rel" || exit 0
echo "$docs" | while IFS= read -r line; do
emit "$line" || exit 0
done
emit "" || exit 0
fi
done
}
extract_docstrings_go() {
# Go doc comments: // comments directly above exported functions/types
find "$PROJECT_DIR" -type f -name '*.go' \
-not -name '*_test.go' -not -path '*/vendor/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
rel="${file#$PROJECT_DIR/}"
docs=$(python3 -c "
import re, sys
with open(sys.argv[1]) as f:
content = f.read()
# Match comment blocks before exported declarations
pattern = r'((?://[^\n]*\n)+)\s*(func [A-Z]\w*|type [A-Z]\w*)'
matches = re.findall(pattern, content)
for comments, sig in matches[:15]:
lines = comments.strip().split('\n')
if len(lines) > 6:
lines = lines[:6] + ['// ...']
print(' ' + sig)
for l in lines:
print(' ' + l.strip())
print()
" "$file" 2>/dev/null || true)
if [[ -n "$docs" ]]; then
emit "## $rel" || exit 0
echo "$docs" | while IFS= read -r line; do
emit "$line" || exit 0
done
emit "" || exit 0
fi
done
}
extract_docstrings_rust() {
# Rust doc comments: /// before pub items
find "$PROJECT_DIR" -type f -name '*.rs' \
-not -path '*/target/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
rel="${file#$PROJECT_DIR/}"
docs=$(python3 -c "
import re, sys
with open(sys.argv[1]) as f:
content = f.read()
pattern = r'((?:///[^\n]*\n)+)\s*(pub (?:fn|struct|enum|trait|type)\s+\w+)'
matches = re.findall(pattern, content)
for comments, sig in matches[:15]:
lines = comments.strip().split('\n')
if len(lines) > 6:
lines = lines[:6] + ['/// ...']
print(' ' + sig)
for l in lines:
print(' ' + l.strip())
print()
" "$file" 2>/dev/null || true)
if [[ -n "$docs" ]]; then
emit "## $rel" || exit 0
echo "$docs" | while IFS= read -r line; do
emit "$line" || exit 0
done
emit "" || exit 0
fi
done
}
case "$LANG" in
typescript|javascript) extract_docstrings_typescript ;;
python) extract_docstrings_python ;;
go) extract_docstrings_go ;;
rust) extract_docstrings_rust ;;
*) emit "(Docstring extraction not supported for $LANG)" || true ;;
esac
#!/usr/bin/env bash
# Extract public API signatures from source files (no function bodies).
# Usage: extract_api.sh <source_dir> [--lang <language>]
#
# Auto-detects language from file extensions. Override with --lang.
# Supported: typescript, javascript, python, go, rust, ruby, php
#
# Output: public function/class/type signatures, one per line.
# This output is fed to the Test Writer agent so it knows what
# interfaces exist without seeing implementation details.
set -euo pipefail
SOURCE_DIR="${1:?Usage: extract_api.sh <source_dir> [--lang <language>]}"
LANG_OVERRIDE=""
shift
while [[ $# -gt 0 ]]; do
case "$1" in
--lang) LANG_OVERRIDE="$2"; shift 2 ;;
*) shift ;;
esac
done
# Auto-detect language from most common file extension
detect_language() {
local dir="$1"
local ts_count js_count py_count go_count rs_count rb_count php_count
ts_count=$(find "$dir" -name '*.ts' -not -name '*.d.ts' -not -path '*/node_modules/*' -not -path '*/.next/*' | head -50 | wc -l | tr -d ' ')
js_count=$(find "$dir" -name '*.js' -not -path '*/node_modules/*' -not -path '*/.next/*' | head -50 | wc -l | tr -d ' ')
py_count=$(find "$dir" -name '*.py' -not -path '*/__pycache__/*' -not -path '*/venv/*' | head -50 | wc -l | tr -d ' ')
go_count=$(find "$dir" -name '*.go' -not -path '*/vendor/*' | head -50 | wc -l | tr -d ' ')
rs_count=$(find "$dir" -name '*.rs' -not -path '*/target/*' | head -50 | wc -l | tr -d ' ')
rb_count=$(find "$dir" -name '*.rb' -not -path '*/vendor/*' | head -50 | wc -l | tr -d ' ')
php_count=$(find "$dir" -name '*.php' -not -path '*/vendor/*' | head -50 | wc -l | tr -d ' ')
local max=0 lang="unknown"
for pair in "typescript:$ts_count" "javascript:$js_count" "python:$py_count" "go:$go_count" "rust:$rs_count" "ruby:$rb_count" "php:$php_count"; do
local l="${pair%%:*}" c="${pair##*:}"
if [[ "$c" -gt "$max" ]]; then max="$c"; lang="$l"; fi
done
# If ts and js both exist, prefer ts
if [[ "$ts_count" -gt 0 ]]; then lang="typescript"; fi
echo "$lang"
}
LANG="${LANG_OVERRIDE:-$(detect_language "$SOURCE_DIR")}"
extract_typescript() {
local dir="$1"
echo "# TypeScript/JavaScript API Surface"
echo "# Source: $dir"
echo ""
find "$dir" -type f \( -name '*.ts' -o -name '*.tsx' -o -name '*.js' -o -name '*.jsx' \) \
-not -name '*.test.*' -not -name '*.spec.*' -not -name '*.d.ts' \
-not -path '*/node_modules/*' -not -path '*/.next/*' -not -path '*/dist/*' \
-not -path '*/__tests__/*' -not -path '*/build/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
local rel="${file#$dir/}"
# Extract export lines (includes re-exports and default exports)
local exports
exports=$(grep -nE '^export ' "$file" 2>/dev/null | grep -vE '^\s*//' || true)
if [[ -n "$exports" ]]; then
echo "## $rel"
while IFS= read -r line; do
local cleaned
cleaned=$(echo "$line" | sed -E 's/\{[^}]*$/\{...}/' | sed -E 's/= .+$/= ...;/')
echo " $cleaned"
done <<< "$exports"
echo ""
fi
done
}
extract_python() {
local dir="$1"
echo "# Python API Surface"
echo "# Source: $dir"
echo ""
find "$dir" -type f -name '*.py' \
-not -name 'test_*' -not -name '*_test.py' -not -name 'conftest.py' \
-not -path '*/venv/*' -not -path '*/__pycache__/*' -not -path '*/tests/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
local rel="${file#$dir/}"
local output=""
# Extract __all__ exports if present
local all_exports
all_exports=$(grep -n '^__all__' "$file" 2>/dev/null || true)
if [[ -n "$all_exports" ]]; then
output+="$all_exports"$'\n'
fi
# Get top-level functions, classes, and decorators
local top_level
top_level=$(grep -nE '^(def [a-zA-Z][a-zA-Z0-9_]*|class [a-zA-Z][a-zA-Z0-9_]*|async def [a-zA-Z][a-zA-Z0-9_]*|@(property|staticmethod|classmethod|dataclass|runtime_checkable))' "$file" 2>/dev/null | grep -v '^\s*def _' || true)
if [[ -n "$top_level" ]]; then
output+="$top_level"$'\n'
fi
# Get class methods (1-level indent: 4 spaces or 1 tab)
local methods
methods=$(grep -nE '^( |\t)(def [a-zA-Z][a-zA-Z0-9_]*|async def [a-zA-Z][a-zA-Z0-9_]*)' "$file" 2>/dev/null | grep -v 'def _[a-zA-Z]' || true)
if [[ -n "$methods" ]]; then
output+="$methods"$'\n'
fi
output=$(echo "$output" | sort -t: -k1,1n | uniq)
if [[ -n "$output" ]]; then
echo "## $rel"
while IFS= read -r line; do
[[ -z "$line" ]] && continue
echo " $line"
done <<< "$output"
echo ""
fi
done
}
extract_go() {
local dir="$1"
echo "# Go API Surface"
echo "# Source: $dir"
echo ""
find "$dir" -type f -name '*.go' \
-not -name '*_test.go' \
-not -path '*/vendor/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
local rel="${file#$dir/}"
# Exported: functions/types starting with uppercase
local signatures
signatures=$(grep -nE '^(func [A-Z]|type [A-Z]|var [A-Z]|const [A-Z])' "$file" 2>/dev/null || true)
if [[ -n "$signatures" ]]; then
echo "## $rel"
while IFS= read -r line; do
echo " $line"
done <<< "$signatures"
echo ""
fi
done
}
extract_rust() {
local dir="$1"
echo "# Rust API Surface"
echo "# Source: $dir"
echo ""
find "$dir" -type f -name '*.rs' \
-not -path '*/target/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
local rel="${file#$dir/}"
local signatures
signatures=$(grep -nE '^pub (fn|struct|enum|trait|type|const|static|mod|use)' "$file" 2>/dev/null || true)
if [[ -n "$signatures" ]]; then
echo "## $rel"
while IFS= read -r line; do
echo " $line"
done <<< "$signatures"
echo ""
fi
done
}
extract_ruby() {
local dir="$1"
echo "# Ruby API Surface"
echo "# Source: $dir"
echo ""
find "$dir" -type f -name '*.rb' \
-not -name '*_spec.rb' -not -name '*_test.rb' \
-not -path '*/spec/*' -not -path '*/test/*' -not -path '*/vendor/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
local rel="${file#$dir/}"
local signatures
signatures=$(grep -nE '^\s*(class |module |def [a-z]|def self\.)' "$file" 2>/dev/null || true)
if [[ -n "$signatures" ]]; then
echo "## $rel"
while IFS= read -r line; do
echo " $line"
done <<< "$signatures"
echo ""
fi
done
}
extract_php() {
local dir="$1"
echo "# PHP API Surface"
echo "# Source: $dir"
echo ""
find "$dir" -type f -name '*.php' \
-not -name '*Test.php' \
-not -path '*/vendor/*' -not -path '*/tests/*' \
-print0 2>/dev/null | sort -z | while IFS= read -r -d '' file; do
local rel="${file#$dir/}"
local signatures
signatures=$(grep -nE '^\s*(public |protected |private )?(static )?(function |class |interface |trait |enum )' "$file" 2>/dev/null || true)
if [[ -n "$signatures" ]]; then
echo "## $rel"
while IFS= read -r line; do
echo " $line"
done <<< "$signatures"
echo ""
fi
done
}
# Dispatch
case "$LANG" in
typescript|javascript) extract_typescript "$SOURCE_DIR" ;;
python) extract_python "$SOURCE_DIR" ;;
go) extract_go "$SOURCE_DIR" ;;
rust) extract_rust "$SOURCE_DIR" ;;
ruby) extract_ruby "$SOURCE_DIR" ;;
php) extract_php "$SOURCE_DIR" ;;
*)
echo "# Unknown language: $LANG"
echo "# Could not auto-detect from files in $SOURCE_DIR"
echo "# Use --lang to specify: typescript, python, go, rust, ruby, php"
exit 1
;;
esac
#!/usr/bin/env bash
# Universal test runner — wraps framework output into structured JSON.
# Usage: run_tests.sh <framework> <test_command> [--all] [--timeout <seconds>]
#
# Examples:
# run_tests.sh jest "npx jest src/sum.test.ts"
# run_tests.sh pytest "pytest tests/test_sum.py -v"
# run_tests.sh jest "npx jest" --all
# run_tests.sh jest "npx jest" --timeout 120
#
# Output: single JSON object on stdout:
# {"status":"pass|fail|error","total":N,"passed":N,"failed":N,
# "failures":[{"test_name":"...","message":"...","stack":"..."}],
# "raw_tail":"last 30 lines of output"}
#
# Status values:
# pass — all tests passed (exit 0)
# fail — one or more tests failed (exit non-zero, parseable output)
# error — script/compilation/infra error (exit non-zero, no parseable test results)
set -uo pipefail
# NOTE: intentionally NOT using set -e. Parsing steps may fail on unexpected
# output formats; we always want to produce JSON, even degraded.
FRAMEWORK="${1:?Usage: run_tests.sh <framework> <test_command> [--all] [--timeout <seconds>]}"
TEST_CMD="${2:?Usage: run_tests.sh <framework> <test_command> [--all] [--timeout <seconds>]}"
shift 2
TIMEOUT=300 # default 5 minutes
while [[ $# -gt 0 ]]; do
case "$1" in
--all) shift ;; # informational only, doesn't change behavior
--timeout) TIMEOUT="$2"; shift 2 ;;
*) shift ;;
esac
done
TMPFILE=$(mktemp)
trap 'rm -f "$TMPFILE"' EXIT
# Run the test command with timeout, capture output and exit code
EXIT_CODE=0
if command -v timeout &>/dev/null; then
timeout "$TIMEOUT" bash -c "$TEST_CMD" > "$TMPFILE" 2>&1 || EXIT_CODE=$?
elif command -v gtimeout &>/dev/null; then
gtimeout "$TIMEOUT" bash -c "$TEST_CMD" > "$TMPFILE" 2>&1 || EXIT_CODE=$?
else
# No timeout command available — run directly
bash -c "$TEST_CMD" > "$TMPFILE" 2>&1 || EXIT_CODE=$?
fi
# Exit code 124 = timeout killed the process
if [[ "$EXIT_CODE" -eq 124 ]]; then
echo '{"status":"error","total":0,"passed":0,"failed":0,"failures":[{"test_name":"TIMEOUT","message":"Test command exceeded '"$TIMEOUT"'s timeout","stack":""}],"raw_tail":"killed by timeout after '"$TIMEOUT"' seconds"}'
exit 0
fi
# Escape raw_tail safely via python3 (handles all JSON-special chars)
RAW_TAIL=$(tail -30 "$TMPFILE" | python3 -c '
import sys, json
text = sys.stdin.read()
# json.dumps produces a quoted string with all escaping handled
print(json.dumps(text))
' 2>/dev/null || echo '"(could not read output)"')
# RAW_TAIL is now a JSON-quoted string like "\"line1\\nline2\""
# Emit valid JSON. Uses python3 for safe assembly to avoid printf % issues.
emit_json() {
local status="$1" total="$2" passed="$3" failed="$4" failures="$5"
python3 -c "
import json, sys
obj = {
'status': sys.argv[1],
'total': int(sys.argv[2]),
'passed': int(sys.argv[3]),
'failed': int(sys.argv[4]),
'failures': json.loads(sys.argv[5]),
'raw_tail': json.loads(sys.argv[6])
}
print(json.dumps(obj))
" "$status" "$total" "$passed" "$failed" "$failures" "$RAW_TAIL" 2>/dev/null || \
echo '{"status":"error","total":0,"passed":0,"failed":0,"failures":[],"raw_tail":"JSON assembly failed"}'
}
# Parse based on framework
parse_jest_vitest() {
local total=0 passed=0 failed=0 failures="[]"
local summary_line
summary_line=$(grep -E '(Tests|Test Suites):.*total' "$TMPFILE" | tail -1 || true)
if [[ -n "$summary_line" ]]; then
total=$(echo "$summary_line" | grep -oE '[0-9]+ total' | grep -oE '[0-9]+' || echo 0)
passed=$(echo "$summary_line" | grep -oE '[0-9]+ passed' | grep -oE '[0-9]+' || echo 0)
failed=$(echo "$summary_line" | grep -oE '[0-9]+ failed' | grep -oE '[0-9]+' || echo 0)
fi
if [[ "$failed" -gt 0 ]] || [[ "$EXIT_CODE" -ne 0 ]]; then
failures=$(python3 -c "
import re, json, sys
text = open(sys.argv[1]).read()
pattern = r'● (.+?)(?:\n\n|\n\s*\n)([\s\S]*?)(?=\n\s*●|\n\s*Test Suites:|\Z)'
matches = re.findall(pattern, text)
results = []
for name, body in matches[:10]:
msg_lines = body.strip().split('\n')
msg = msg_lines[0] if msg_lines else ''
stack = '\n'.join(msg_lines[1:4]) if len(msg_lines) > 1 else ''
results.append({'test_name': name.strip(), 'message': msg.strip(), 'stack': stack.strip()})
if not results:
for line in text.split('\n'):
if line.strip().startswith('FAIL'):
results.append({'test_name': line.strip(), 'message': 'See raw output', 'stack': ''})
print(json.dumps(results))
" "$TMPFILE" 2>/dev/null || echo '[]')
fi
local status="pass"
[[ "$EXIT_CODE" -ne 0 ]] && status="fail"
[[ "$total" -eq 0 && "$EXIT_CODE" -ne 0 ]] && status="error"
emit_json "$status" "$total" "$passed" "$failed" "$failures"
}
parse_pytest() {
local total=0 passed=0 failed=0 failures="[]"
local summary_line
summary_line=$(grep -E '=+ .*(passed|failed|error).*=+' "$TMPFILE" | tail -1 || true)
if [[ -n "$summary_line" ]]; then
passed=$(echo "$summary_line" | grep -oE '[0-9]+ passed' | grep -oE '[0-9]+' || echo 0)
failed=$(echo "$summary_line" | grep -oE '[0-9]+ failed' | grep -oE '[0-9]+' || echo 0)
local errors
errors=$(echo "$summary_line" | grep -oE '[0-9]+ error' | grep -oE '[0-9]+' || echo 0)
total=$((passed + failed + errors))
fi
if [[ "$failed" -gt 0 ]] || [[ "$EXIT_CODE" -ne 0 ]]; then
failures=$(python3 -c "
import re, json, sys
text = open(sys.argv[1]).read()
pattern = r'FAILED (.+?)(?:\s*-\s*(.+))?$'
results = []
for m in re.finditer(pattern, text, re.MULTILINE):
name = m.group(1).strip()
msg = m.group(2).strip() if m.group(2) else 'See raw output'
results.append({'test_name': name, 'message': msg, 'stack': ''})
if not results:
pattern2 = r'___+ (.+?) ___+\n([\s\S]*?)(?=___+|\Z)'
for m in re.finditer(pattern2, text):
name = m.group(1).strip()
body = m.group(2).strip().split('\n')
msg = next((l for l in body if 'assert' in l.lower() or 'Error' in l), body[0] if body else '')
results.append({'test_name': name, 'message': msg.strip(), 'stack': ''})
print(json.dumps(results[:10]))
" "$TMPFILE" 2>/dev/null || echo '[]')
fi
local status="pass"
[[ "$EXIT_CODE" -ne 0 ]] && status="fail"
[[ "$total" -eq 0 && "$EXIT_CODE" -ne 0 ]] && status="error"
emit_json "$status" "$total" "$passed" "$failed" "$failures"
}
parse_go() {
local total=0 passed=0 failed=0 failures="[]"
passed=$(grep -cE '^--- PASS:' "$TMPFILE" 2>/dev/null) || passed=0
failed=$(grep -cE '^--- FAIL:' "$TMPFILE" 2>/dev/null) || failed=0
total=$((passed + failed))
if [[ "$failed" -gt 0 ]] || [[ "$EXIT_CODE" -ne 0 ]]; then
failures=$(python3 -c "
import re, json, sys
text = open(sys.argv[1]).read()
results = []
for m in re.finditer(r'^--- FAIL: (\S+)', text, re.MULTILINE):
name = m.group(1)
start = m.end()
end_match = re.search(r'^---', text[start:], re.MULTILINE)
block = text[start:start + end_match.start()] if end_match else text[start:start+500]
lines = [l.strip() for l in block.strip().split('\n') if l.strip()]
msg = lines[0] if lines else 'See raw output'
results.append({'test_name': name, 'message': msg, 'stack': ''})
print(json.dumps(results[:10]))
" "$TMPFILE" 2>/dev/null || echo '[]')
fi
local status="pass"
[[ "$EXIT_CODE" -ne 0 ]] && status="fail"
[[ "$total" -eq 0 && "$EXIT_CODE" -ne 0 ]] && status="error"
emit_json "$status" "$total" "$passed" "$failed" "$failures"
}
parse_cargo() {
local total=0 passed=0 failed=0 failures="[]"
local summary_line
summary_line=$(grep -E '^test result:' "$TMPFILE" | tail -1 || true)
if [[ -n "$summary_line" ]]; then
passed=$(echo "$summary_line" | grep -oE '[0-9]+ passed' | grep -oE '[0-9]+' || echo 0)
failed=$(echo "$summary_line" | grep -oE '[0-9]+ failed' | grep -oE '[0-9]+' || echo 0)
total=$((passed + failed))
fi
if [[ "$failed" -gt 0 ]] || [[ "$EXIT_CODE" -ne 0 ]]; then
failures=$(python3 -c "
import re, json, sys
text = open(sys.argv[1]).read()
results = []
for m in re.finditer(r'^---- (.+?) stdout ----\n([\s\S]*?)(?=^----|\Z)', text, re.MULTILINE):
name = m.group(1).strip()
body = m.group(2).strip().split('\n')
msg = next((l for l in body if 'panicked' in l or 'assert' in l), body[0] if body else '')
results.append({'test_name': name, 'message': msg.strip(), 'stack': ''})
print(json.dumps(results[:10]))
" "$TMPFILE" 2>/dev/null || echo '[]')
fi
local status="pass"
[[ "$EXIT_CODE" -ne 0 ]] && status="fail"
[[ "$total" -eq 0 && "$EXIT_CODE" -ne 0 ]] && status="error"
emit_json "$status" "$total" "$passed" "$failed" "$failures"
}
parse_rspec() {
local total=0 passed=0 failed=0 failures="[]"
local summary_line
summary_line=$(grep -E '[0-9]+ examples' "$TMPFILE" | tail -1 || true)
if [[ -n "$summary_line" ]]; then
total=$(echo "$summary_line" | grep -oE '[0-9]+ examples' | grep -oE '[0-9]+' || echo 0)
failed=$(echo "$summary_line" | grep -oE '[0-9]+ failures?' | grep -oE '[0-9]+' || echo 0)
passed=$((total - failed))
fi
if [[ "$failed" -gt 0 ]] || [[ "$EXIT_CODE" -ne 0 ]]; then
failures=$(python3 -c "
import re, json, sys
text = open(sys.argv[1]).read()
results = []
for m in re.finditer(r'^\s+\d+\) (.+?)\n\s+Failure/Error: (.+?)$', text, re.MULTILINE):
results.append({'test_name': m.group(1).strip(), 'message': m.group(2).strip(), 'stack': ''})
print(json.dumps(results[:10]))
" "$TMPFILE" 2>/dev/null || echo '[]')
fi
local status="pass"
[[ "$EXIT_CODE" -ne 0 ]] && status="fail"
emit_json "$status" "$total" "$passed" "$failed" "$failures"
}
parse_phpunit() {
local total=0 passed=0 failed=0 failures="[]"
if grep -qE '^OK \(' "$TMPFILE"; then
total=$(grep -oE 'OK \([0-9]+ tests' "$TMPFILE" | grep -oE '[0-9]+' || echo 0)
passed=$total
elif grep -qE 'Tests: [0-9]+' "$TMPFILE"; then
total=$(grep -oE 'Tests: [0-9]+' "$TMPFILE" | grep -oE '[0-9]+' || echo 0)
failed=$(grep -oE 'Failures: [0-9]+' "$TMPFILE" | grep -oE '[0-9]+' || echo 0)
passed=$((total - failed))
fi
if [[ "$failed" -gt 0 ]] || [[ "$EXIT_CODE" -ne 0 ]]; then
failures=$(python3 -c "
import re, json, sys
text = open(sys.argv[1]).read()
results = []
for m in re.finditer(r'^\d+\) (.+?)$\n(.+?)$', text, re.MULTILINE):
results.append({'test_name': m.group(1).strip(), 'message': m.group(2).strip(), 'stack': ''})
print(json.dumps(results[:10]))
" "$TMPFILE" 2>/dev/null || echo '[]')
fi
local status="pass"
[[ "$EXIT_CODE" -ne 0 ]] && status="fail"
emit_json "$status" "$total" "$passed" "$failed" "$failures"
}
# Generic fallback for unknown frameworks
parse_generic() {
local status="pass"
[[ "$EXIT_CODE" -ne 0 ]] && status="fail"
emit_json "$status" "0" "0" "0" "[]"
}
# Dispatch
case "$FRAMEWORK" in
jest|vitest) parse_jest_vitest ;;
pytest) parse_pytest ;;
go) parse_go ;;
cargo) parse_cargo ;;
rspec) parse_rspec ;;
phpunit) parse_phpunit ;;
*) parse_generic ;;
esac
"""Shared fixtures for TDD skill script tests."""
import os
from pathlib import Path
import pytest
SKILL_DIR = Path(__file__).parent.parent
SCRIPTS_DIR = SKILL_DIR / "scripts"
FIXTURES_DIR = Path(__file__).parent / "fixtures"
@pytest.fixture
def scripts_dir():
return SCRIPTS_DIR
@pytest.fixture
def python_project():
return FIXTURES_DIR / "python_project"
@pytest.fixture
def ts_project():
return FIXTURES_DIR / "ts_project"
@pytest.fixture
def go_project():
return FIXTURES_DIR / "go_project"
@pytest.fixture
def empty_project():
return FIXTURES_DIR / "empty_project"
@pytest.fixture
def tmp_project(tmp_path):
"""A writable temp directory for tests that need to create files."""
return tmp_path
package calculator
// Add returns the sum of two integers.
// It handles overflow by wrapping.
func Add(a, b int) int {
return a + b
}
// Divide returns a/b. Returns error if b is zero.
func Divide(a, b int) (int, error) {
if b == 0 {
return 0, fmt.Errorf("division by zero")
}
return a / b, nil
}
// Calculator holds state for chained operations.
type Calculator struct {
Result int
}
func internal() {}
API Reference
Calculator
Methods
add(a, b)— Returns sum of a and bsubtract(a, b)— Returns a minus bdivide(a, b)— Returns a divided by b, raises ValueError on zero
Standalone Functions
factorial(n)— Returns n!, raises ValueError for negative input
openapi: "3.0.0"
info:
title: Calculator API
version: "1.0.0"
paths:
/calculate:
post:
summary: Perform a calculation
requestBody:
content:
application/json:
schema:
type: object
properties:
operation:
type: string
a:
type: number
b:
type: number
Calculator Project
A simple calculator library for arithmetic operations.
Features
- Basic arithmetic: add, subtract, multiply, divide
- Factorial computation
- Input validation with descriptive errors
Usage
from src.calculator import Calculator
calc = Calculator()
result = calc.add(2, 3) # returns 5"""Calculator module for arithmetic operations."""
class Calculator:
"""A simple calculator with basic arithmetic.
Supports add, subtract, multiply, divide with
error handling for division by zero.
"""
def add(self, a: float, b: float) -> float:
"""Add two numbers and return the result."""
return a + b
def subtract(self, a: float, b: float) -> float:
"""Subtract b from a."""
return a - b
def divide(self, a: float, b: float) -> float:
"""Divide a by b.
Raises:
ValueError: If b is zero.
"""
if b == 0:
raise ValueError("Cannot divide by zero")
return a / b
def factorial(n: int) -> int:
"""Compute factorial of n.
Args:
n: Non-negative integer.
Returns:
n! as an integer.
"""
if n < 0:
raise ValueError("n must be non-negative")
if n <= 1:
return 1
return n * factorial(n - 1)
def _internal_helper():
"""This is private and should not appear in API surface."""
pass
from dataclasses import dataclass
@dataclass
class User:
"""Represents a user in the system."""
name: str
email: str
def validate_email(self) -> bool:
"""Check if email contains @ symbol."""
return "@" in self.email
TypeScript Calculator
Simple calculator with type safety.
/**
* Add two numbers together.
* @param a - First operand
* @param b - Second operand
* @returns The sum of a and b
*/
export function add(a: number, b: number): number {
return a + b;
}
/**
* Divide a by b with zero-check.
* @throws Error if b is zero
*/
export function divide(a: number, b: number): number {
if (b === 0) throw new Error("Division by zero");
return a / b;
}
export class Calculator {
private history: number[] = [];
add(a: number, b: number): number {
const result = a + b;
this.history.push(result);
return result;
}
}
export type Operation = "add" | "subtract" | "multiply" | "divide";
export interface CalculatorConfig {
precision: number;
strict: boolean;
}
You are a TDD Implementer. Your ONLY job is to write the MINIMUM code to make a failing test pass.
## Language: python
## Failing test code
def test_should_reject_email_without_at():
user = User(name='Test', email='invalid')
assert user.validate_email() is False
## Test failure output
FAILED tests/test_user.py::test_should_reject_email_without_at
ModuleNotFoundError: No module named 'src.models'
## File tree (source files only)
src/
src/models.py
src/__init__.py
## Existing source code (files relevant to the failing test)
# src/models.py
class User:
pass
## Architectural layer
This code belongs to the **domain** layer.
This is the innermost layer. It MUST NOT import anything from domain-service, application, or infrastructure layers. No ORM imports, no HTTP clients, no framework imports. Only standard library and domain types.
## Rules
1. Write the MINIMUM code to make the failing test pass
2. No code beyond what the test requires
3. No premature abstractions or extra error handling
4. No optimization -- simple and direct
5. Hardcoded values are acceptable if they satisfy the test
6. Do NOT modify the test file
7. Do NOT add features or behaviors not tested
8. ALWAYS return the COMPLETE file content for every file you change or create
9. Respect the layer dependency constraint above -- do NOT import from outer layers
## Output
Return a single JSON object. Do NOT wrap in markdown fences. Do NOT include any text before or after the JSON.
{"files": [{"path": "relative/path/to/file.ext", "action": "create or overwrite", "content": "COMPLETE file content -- the entire file from first line to last", "description": "what this file does"}], "explanation": "brief explanation of the implementation approach"}You are a TDD Refactorer. All tests are currently passing. Your job is to suggest code improvements that preserve behavior.
## Language: python
## Current test results (all green)
======================== 2 passed in 0.03s ========================
## All test code
def test_should_reject_email_without_at():
user = User(name='Test', email='invalid')
assert user.validate_email() is False
def test_should_accept_valid_email():
user = User(name='Test', email='test@example.com')
assert user.validate_email() is True
## All implementation code
class User:
def __init__(self, name: str, email: str):
self.name = name
self.email = email
def validate_email(self) -> bool:
return '@' in self.email
## Layers touched in this session
domain
## Rules
1. Do NOT change behavior -- all existing tests must continue to pass
2. Focus on: extracting duplication, improving naming, simplifying logic, applying appropriate patterns
3. Do NOT add new features, new tests, or new error handling
4. Each suggestion must be independently applicable (revert-safe)
5. Prefer small, targeted improvements over large restructurings
6. Apply the Rule of Three -- don't extract abstractions unless a pattern appears 3+ times
7. If no meaningful refactoring is needed, say so -- that's a valid outcome
8. Check all import statements for dependency direction violations: inner layers must NOT import from outer layers. Direction: domain → domain-service → application → infrastructure. Flag any violation as a HIGH priority suggestion.
9. Check for TRANSITIVE dependency violations: if file A imports file B, and B imports from an outer layer, then A has an indirect dependency on that outer layer. Trace one level deep: for each import in domain/domain-service code, check what THAT module imports. Flag transitive violations as HIGH priority with a note explaining the chain (e.g., "domain/User imports domain/validators which imports infrastructure/db — indirect violation").
10. Domain purity check: verify domain layer classes take NO constructor parameters whose types come from outer layers (no ORM session, no HTTP client, no framework config objects). Flag as HIGH priority.
## Output
Return a single JSON object. Do NOT wrap in markdown fences. Do NOT include any text before or after the JSON.
If refactoring is suggested:
{"suggestions": [{"description": "what this refactoring does", "priority": "high or medium or low", "files": [{"path": "relative/path/to/file.ext", "old_code": "exact code to find and replace", "new_code": "replacement code"}]}], "summary": "overall assessment of code quality"}
If no refactoring is needed:
{"suggestions": [], "summary": "Code is clean. No refactoring needed at this stage."}You are a TDD Test Writer. Your ONLY job is to write ONE failing test for a specific behavior.
## Specification for this slice
User email validation: should reject emails without @ symbol
## Language and framework
- Language: python
- Framework: pytest
- Test file location: tests/test_user.py
## Public API surface (signatures only, no implementations)
# Python API Surface
## src/models.py
5:class User:
10:def validate_email(self) -> bool:
Note: If the API surface is empty, the function/class does not exist yet. Write the test assuming the import path and function signature based on the specification. The Implementer will create the code.
## Project documentation (relevant excerpts)
# Project Documentation
## README.md
User model validates email format on creation.
# Source Docstrings
## models.py
User: Represents a user in the system.
validate_email: Check if email contains @ symbol.
Use this documentation to understand intended behavior, API contracts, edge cases, and validation rules. Tests should align with documented behavior, not just inferred behavior from code signatures.
## Existing test file content (if any)
No test file exists yet.
## Framework-specific test skeleton
def test_should_behavior():
# Arrange
...
# Act
result = function_under_test()
# Assert
assert result == expected
## Architectural layer for this slice
This slice belongs to the **domain** layer.
Write tests using only domain types. NO database mocks, NO HTTP mocks, NO file system. Test pure business logic through public methods. Construct real domain objects directly — never mock them.
## Rules
1. Write EXACTLY ONE test function for the specified behavior
2. The test MUST fail because the implementation does not yet exist
3. Test through the public interface only -- no internal/private access
4. Use descriptive test names that read as behavior specs
5. Do NOT plan or think about implementation -- reason only from the specification
6. Do NOT write implementation code
7. Do NOT write helper functions beyond minimal test setup
8. Include all necessary imports in the test code
9. Follow the layer-specific test constraints above
## Output
Return a single JSON object. Do NOT wrap in markdown fences. Do NOT include any text before or after the JSON.
{"test_code": "the COMPLETE test code to add (including describe/it blocks, not just the assertion)", "test_name": "name of the test function", "test_description": "what behavior this test verifies", "imports_needed": "any import statements needed at the top of the file, or empty string if none"}"""Tests for discover_docs.sh — project documentation discovery."""
import subprocess
import pytest
def run_discover(scripts_dir, project_dir, lang=None):
cmd = ["bash", str(scripts_dir / "discover_docs.sh"), str(project_dir)]
if lang:
cmd += ["--lang", lang]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
return result
class TestDocumentationFiles:
def test_finds_readme(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project)
assert result.returncode == 0
assert "README.md" in result.stdout
assert "Calculator Project" in result.stdout
def test_finds_docs_folder(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project)
assert "API.md" in result.stdout
assert "Calculator" in result.stdout
def test_ts_readme(self, scripts_dir, ts_project):
result = run_discover(scripts_dir, ts_project)
assert "TypeScript Calculator" in result.stdout
class TestAPISpecifications:
def test_finds_openapi_yaml(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project)
assert "openapi" in result.stdout.lower()
assert "/calculate" in result.stdout
def test_no_specs_in_ts_project(self, scripts_dir, ts_project):
result = run_discover(scripts_dir, ts_project)
assert "No API specification files found" in result.stdout
class TestPythonDocstrings:
def test_extracts_class_docstrings(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project, lang="python")
assert "Calculator" in result.stdout
assert "arithmetic" in result.stdout.lower()
def test_extracts_function_docstrings(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project, lang="python")
assert "factorial" in result.stdout
def test_excludes_private_docstrings(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project, lang="python")
assert "_internal_helper" not in result.stdout
def test_extracts_model_docstrings(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project, lang="python")
assert "User" in result.stdout
class TestTypeScriptDocstrings:
def test_extracts_jsdoc(self, scripts_dir, ts_project):
result = run_discover(scripts_dir, ts_project, lang="typescript")
assert "Source Docstrings" in result.stdout
# JSDoc for exported functions
assert "add" in result.stdout.lower() or "calculator" in result.stdout.lower()
class TestGoDocstrings:
def test_extracts_doc_comments(self, scripts_dir, go_project):
result = run_discover(scripts_dir, go_project, lang="go")
assert "Source Docstrings" in result.stdout
class TestEmptyProject:
def test_handles_empty_gracefully(self, scripts_dir, empty_project):
result = run_discover(scripts_dir, empty_project)
assert result.returncode == 0
assert "No documentation files found" in result.stdout
assert "No API specification files found" in result.stdout
class TestOutputStructure:
def test_has_three_sections(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project)
assert "# Project Documentation" in result.stdout
assert "# API Specifications" in result.stdout
assert "# Source Docstrings" in result.stdout
def test_output_not_empty(self, scripts_dir, python_project):
result = run_discover(scripts_dir, python_project)
assert len(result.stdout) > 100
class TestTruncation:
def test_respects_char_limit(self, scripts_dir, tmp_project):
"""Create a project with very large docs to test truncation."""
readme = tmp_project / "README.md"
readme.write_text("# Big Doc\n" + ("x" * 200 + "\n") * 100)
result = run_discover(scripts_dir, tmp_project, lang="python")
assert len(result.stdout) <= 16000 # 15k + some overhead
"""Tests for extract_api.sh — public API signature extraction.
Note: Python/TS fixtures are copied to tmp_path to avoid extract_api.sh's
`-not -path '*/tests/*'` exclusion (fixtures live under tests/).
"""
import shutil
import subprocess
import pytest
from conftest import FIXTURES_DIR
def run_extract(scripts_dir, source_dir, lang=None):
cmd = ["bash", str(scripts_dir / "extract_api.sh"), str(source_dir)]
if lang:
cmd += ["--lang", lang]
result = subprocess.run(cmd, capture_output=True, text=True, timeout=30)
return result
@pytest.fixture
def py_src(tmp_path):
"""Copy Python fixture to tmp_path so it's not under tests/."""
src = tmp_path / "src"
shutil.copytree(FIXTURES_DIR / "python_project" / "src", src)
return src
@pytest.fixture
def ts_src(tmp_path):
src = tmp_path / "src"
shutil.copytree(FIXTURES_DIR / "ts_project" / "src", src)
return src
@pytest.fixture
def go_src(tmp_path):
dst = tmp_path / "go_project"
shutil.copytree(FIXTURES_DIR / "go_project", dst)
return dst
class TestPythonExtraction:
def test_extracts_class_signatures(self, scripts_dir, py_src):
result = run_extract(scripts_dir, py_src)
assert result.returncode == 0
assert "class Calculator" in result.stdout
def test_extracts_function_signatures(self, scripts_dir, py_src):
result = run_extract(scripts_dir, py_src)
# extract_api.sh matches top-level definitions only (^def)
# methods inside classes are indented and won't match
assert "def factorial" in result.stdout
def test_extracts_dataclass_decorator(self, scripts_dir, py_src):
result = run_extract(scripts_dir, py_src)
assert "@dataclass" in result.stdout
def test_excludes_private_functions(self, scripts_dir, py_src):
result = run_extract(scripts_dir, py_src)
assert "_internal_helper" not in result.stdout
def test_shows_file_paths(self, scripts_dir, py_src):
result = run_extract(scripts_dir, py_src)
assert "calculator.py" in result.stdout
assert "models.py" in result.stdout
def test_header_present(self, scripts_dir, py_src):
result = run_extract(scripts_dir, py_src)
assert "Python API Surface" in result.stdout
class TestTypeScriptExtraction:
def test_extracts_exports(self, scripts_dir, ts_src):
result = run_extract(scripts_dir, ts_src)
assert result.returncode == 0
assert "export function add" in result.stdout
assert "export function divide" in result.stdout
def test_extracts_class_exports(self, scripts_dir, ts_src):
result = run_extract(scripts_dir, ts_src)
assert "export class Calculator" in result.stdout
def test_extracts_type_exports(self, scripts_dir, ts_src):
result = run_extract(scripts_dir, ts_src)
assert "export type Operation" in result.stdout
assert "export interface CalculatorConfig" in result.stdout
def test_header_present(self, scripts_dir, ts_src):
result = run_extract(scripts_dir, ts_src)
assert "TypeScript" in result.stdout
class TestGoExtraction:
def test_extracts_exported_functions(self, scripts_dir, go_src):
result = run_extract(scripts_dir, go_src)
assert result.returncode == 0
assert "func Add" in result.stdout
assert "func Divide" in result.stdout
def test_extracts_exported_types(self, scripts_dir, go_src):
result = run_extract(scripts_dir, go_src)
assert "type Calculator" in result.stdout
def test_excludes_unexported(self, scripts_dir, go_src):
result = run_extract(scripts_dir, go_src)
assert "internal" not in result.stdout
def test_header_present(self, scripts_dir, go_src):
result = run_extract(scripts_dir, go_src)
assert "Go API Surface" in result.stdout
class TestLanguageDetection:
def test_auto_detects_python(self, scripts_dir, py_src):
result = run_extract(scripts_dir, py_src)
assert "Python" in result.stdout
def test_auto_detects_typescript(self, scripts_dir, ts_src):
result = run_extract(scripts_dir, ts_src)
assert "TypeScript" in result.stdout
def test_lang_override(self, scripts_dir, py_src):
result = run_extract(scripts_dir, py_src, lang="go")
assert "Go API Surface" in result.stdout
class TestEmptyProject:
def test_empty_returns_successfully(self, scripts_dir, empty_project):
result = run_extract(scripts_dir, empty_project)
assert result.returncode in (0, 1)
def test_empty_no_crash(self, scripts_dir, empty_project):
result = run_extract(scripts_dir, empty_project, lang="python")
assert result.returncode == 0
assert "Python API Surface" in result.stdout