Test

Test Skill

Quick Ref: Generate tests, analyze coverage, fill gaps, run TDD loops. Output: passing tests + coverage report in .agents/test/.

YOU MUST EXECUTE THIS WORKFLOW. Do not just describe it.

Generate real tests, run them, verify they pass, and produce coverage artifacts. Do not output a plan and stop.

Modes

Default mode is generate when unspecified. Detect from user intent.

Step 0a: Scenarios-First (when the work has acceptance scenarios)

When the task is tied to a bead or .feature file, author tests FORWARD from the Gherkin scenarios — not backward from a coverage gap. This is the C2 contract (ag-9jle.4): the unit of test authoring is one scenario, one covering test.

1. Read the scenarios: the bead's ## Scenarios block (BEADS_DIR="$(ao beads dir)" br show <bead-id>) or a .feature under skills/<skill>/references/. 2. For each Scenario:, locate or write the test that exercises its Given/When/Then. Name it after the behavior. 3. Declare the linkage by adding @covered-by:<test-path> (optionally ::<TestName>) directly above the scenario in its source — so the leaf coverage gate can prove the mapping. 4. Run the leaf coverage gate and require it to pass before considering the slice tested:

bash scripts/check-bead-scenario-coverage.sh --bead <bead-id> --run      # every scenario -> a PASSING test
# (--bead fetches the body via `br show`; BEADS_DIR defaults to <repo>/_beads — bd is retired)
bash scripts/check-bead-scenario-coverage.sh skills/<skill>/references/<name>.feature --run

A scenario with no covering test is a FAIL — "tests exist" or a coverage percentage is not sufficient. Then continue with coverage gap-fill (Steps 1–5) for everything the scenarios don't reach. When there are no scenarios, skip this step and use the coverage-driven flow below.

Step 0: Detect Language and Load Standards

Scan the project root for language markers. Stop at the first match:

Load /standards for the detected language. Apply all testing conventions from the standards skill (naming, assertion style, structural rules).

Go-specific rules (from project CLAUDE.md):

• Test file naming: <source>_test.go. NEVER cov*_test.go or *_extra_test.go.
• Test function naming: Test<Uppercase> (e.g., TestParseConfig_EmptyInput).
• Prefer table-driven tests for multi-case functions.
• Use captureStdout for output functions and assert content.

Python-specific rules:

• Use pytest with conftest.py for shared fixtures.
• Use @pytest.mark.parametrize for multi-case functions.
• Type hints on test helpers.

JS/TS-specific rules:

• Use describe/it blocks with clear names.
• Group by function or module under test.
• Mock external services, not internal code.

Step 1: Analyze Existing Test Coverage

Run the coverage command for the detected language:

# Go
go test -coverprofile=coverage.out ./... 2>&1 | tee .agents/test/coverage-raw.txt
go tool cover -func=coverage.out > .agents/test/coverage-func.txt

# Python
pytest --cov --cov-report=term-missing --cov-report=json:.agents/test/coverage.json 2>&1 | tee .agents/test/coverage-raw.txt

# JS/TS
npx jest --coverage --coverageReporters=text 2>&1 | tee .agents/test/coverage-raw.txt

# Rust
cargo tarpaulin --out Lcov 2>&1 | tee .agents/test/coverage-raw.txt

Parse the output. Build a ranked list of files by coverage percentage (lowest first).

If /refactor is available, cross-reference: high-complexity + low-coverage = highest priority targets.

Step 2: Identify Gaps

From the coverage data, identify:

1. Untested files -- source files with no corresponding test file. 2. Uncovered functions -- exported/public functions with 0% coverage. 3. Uncovered branches -- functions with partial coverage (conditionals, error paths). 4. Missing edge cases -- functions that only test the happy path.

Produce a gap list sorted by risk (high complexity + low coverage first):

File                        | Coverage | Functions Missing Tests | Risk
----------------------------|----------|------------------------|------
internal/parser/parse.go    | 23%      | ParseConfig, Validate  | HIGH
internal/goals/measure.go   | 45%      | MeasureFitness         | MEDIUM
lib/utils.go                | 89%      | (edge cases only)      | LOW

Write gap list to .agents/test/gaps.md.

Step 3: Generate Tests

For each gap (highest risk first), generate tests following language-specific patterns.

If the target needs a specialized test pattern, load the matching reference before writing tests:

• Contract/API/CLI compatibility: references/conformance-harnesses.md
• Parsers, serializers, and untrusted inputs: references/fuzzing.md
• Generated files, snapshots, and rendered output: references/golden-artifacts.md
• Metamorphic/invariant-heavy behavior: references/metamorphic-testing.md
• Golden artifact update review: references/golden-artifact-strategy.md
• Real databases, services, queues, or APIs where mocks would hide failures: references/real-service-e2e.md

Go: Table-Driven Tests

func TestParseConfig_Variants(t *testing.T) {
    tests := []struct {
        name    string
        input   string
        want    *Config
        wantErr bool
    }{
        {name: "valid minimal", input: `name: foo`, want: &Config{Name: "foo"}},
        {name: "empty input", input: "", want: nil, wantErr: true},
        {name: "invalid yaml", input: `: bad`, want: nil, wantErr: true},
    }
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got, err := ParseConfig(tt.input)
            if (err != nil) != tt.wantErr {
                t.Errorf("ParseConfig() error = %v, wantErr %v", err, tt.wantErr)
                return
            }
            if !reflect.DeepEqual(got, tt.want) {
                t.Errorf("ParseConfig() = %v, want %v", got, tt.want)
            }
        })
    }
}

Python: Parametrized Tests

@pytest.mark.parametrize("input_val,expected", [
    ("valid", Config(name="valid")),
    ("", None),
    (None, None),
])
def test_parse_config(input_val: str, expected: Config | None) -> None:
    result = parse_config(input_val)
    assert result == expected

JS/TS: Describe Blocks

describe("parseConfig", () => {
  it("parses valid input", () => {
    expect(parseConfig("valid")).toEqual({ name: "valid" });
  });

  it("returns null for empty input", () => {
    expect(parseConfig("")).toBeNull();
  });

  it("throws on malformed input", () => {
    expect(() => parseConfig(": bad")).toThrow();
  });
});

Generation Rules

1. Read the source function first. Understand inputs, outputs, error conditions, and branches. 2. Cover every branch. Each if, switch, error return, and edge case gets at least one test case. 3. Assert exact expected values. Use == expected, not != nil or != "". 4. Name tests descriptively. The test name should describe the scenario: "empty input returns error", not "test1". 5. Test error paths explicitly. Verify error messages or error types, not just that an error occurred. 6. One assertion focus per test case. Each table row or parametrized case tests one specific behavior.

Step 4: Run Generated Tests

After writing each test file, immediately run it:

# Go
go test -v -run TestParseConfig ./internal/parser/

# Python
pytest -xvs tests/test_parser.py

# JS/TS
npx jest --verbose tests/parser.test.ts

# Rust
cargo test test_parse_config -- --nocapture

If tests fail: 1. Read the failure output. 2. Determine if the test is wrong or the code has a bug. 3. If the test is wrong: fix the test assertion or setup. 4. If the code has a bug: report it but fix the test to match current behavior, noting the bug in the output. 5. Re-run until green.

Never commit failing tests (unless in TDD mode, Step red).

Step 5: Output Coverage Report

Re-run coverage after adding tests:

# Same commands as Step 1

Compare before/after. Write summary to .agents/test/summary.md:

# Test Generation Summary

**Language:** Go
**Date:** 2026-03-27
**Mode:** generate

## Coverage Delta

| Metric | Before | After | Delta |
|--------|--------|-------|-------|
| Overall | 52.3%  | 71.8% | +19.5% |
| internal/parser | 23.0% | 85.2% | +62.2% |
| internal/goals  | 45.0% | 67.3% | +22.3% |

## Tests Added

- `internal/parser/parse_test.go` -- 12 test cases (3 existing + 9 new)
- `internal/goals/measure_test.go` -- 8 test cases (new file)

## Remaining Gaps

- `internal/render/` -- 34% coverage, complex template logic
- `cmd/root.go` -- integration test needed

## Bugs Found

- `ParseConfig` does not validate empty name field (passes silently)

Create .agents/test/ directory if it does not exist. All artifacts go there.

TDD Mode

When mode is tdd, follow the red-green-refactor cycle:

Red: Write a Failing Test

1. Understand the feature requirement from the user. 2. Write a test that describes the desired behavior. 3. Run the test -- it MUST fail. If it passes, the test is not testing new behavior.

# Verify the test fails
go test -v -run TestNewFeature ./...
# Expected: FAIL

Green: Minimal Implementation

4. Write the minimum code to make the test pass. No extra logic, no optimization. 5. Run the test -- it MUST pass now.

go test -v -run TestNewFeature ./...
# Expected: PASS

Refactor

6. Clean up the implementation. Remove duplication, improve naming, simplify logic. 7. Run ALL tests -- everything must still pass.

go test ./...
# Expected: all PASS

Repeat

8. Pick the next behavior. Write the next failing test. Continue the cycle.

Log each cycle to .agents/test/tdd-log.md:

## Cycle 1: Parse empty config returns error
- RED: TestParseConfig_EmptyInput -- FAIL (function not implemented)
- GREEN: Added nil check in ParseConfig -- PASS
- REFACTOR: Extracted validation to validateConfig() -- PASS

## Cycle 2: Parse config validates name field
- RED: TestParseConfig_MissingName -- FAIL (no name validation)
- GREEN: Added name check -- PASS
- REFACTOR: None needed -- PASS

Strategy Mode

When mode is strategy, analyze and recommend (no code generation):

1. Inventory existing tests. Count test files, test functions, assertion density. 2. Classify test types. Unit, integration, end-to-end, benchmark. 3. Identify structural gaps. Missing test directories, no CI integration, no fixtures. 4. Recommend architecture:

• Test directory structure matching source layout.
• Shared fixtures and helpers.
• Integration test separation (build tags in Go, markers in pytest).
• CI pipeline integration.

5. Output to `.agents/test/strategy.md`.

What Makes Good Tests vs Bad Tests

Good Tests

• Assert behavioral correctness. Test what the function does, not that it exists.
• Use exact expected values. assert result == Config(name="foo", count=3) -- verifies the full output.
• Cover error paths. Error conditions are where bugs hide. Test them explicitly.
• Descriptive names. TestParseConfig_InvalidYAML_ReturnsError tells you what broke when it fails.
• Independent. Each test runs in isolation. No shared mutable state between tests.
• Fast. Unit tests should run in milliseconds. Slow tests get skipped.

Bad Tests (Banned)

• Coverage-padding. Tests that assert != nil or != "" solely to inflate coverage metrics. Every test must assert a specific expected value.
• Zero-assertion smoke tests. func TestFoo(t *testing.T) { Foo() } -- proves nothing except "doesn't panic."
• Tautological assertions. assert foo(x) == foo(x) -- tests the test framework, not the code.
• Implementation-coupled tests. Tests that break when you refactor internals without changing behavior. Test the interface, not the implementation.
• Flaky tests. Tests that depend on timing, network, or ordering. Mock external dependencies. Use deterministic inputs.

If you find existing tests that match the "bad" patterns, flag them in the summary but do not delete them without user confirmation.

Specialized Test References

• references/conformance-harnesses.md -- Contract, compatibility, schema, and process conformance patterns
• references/fuzzing.md -- Fuzz targets, seed corpora, invariants, and crash triage
• references/golden-artifacts.md -- Golden file modes, update discipline, and artifact diff review
• references/real-service-e2e.md -- Real-service integration tests with non-production safety gates

Integration with Other Skills

Flags

Output Artifacts

All artifacts are written to .agents/test/:

Reference Documents

• references/test.feature — Executable spec: load standards + detect language, generate real passing tests (not a plan), coverage gap-fill, artifacts in .agents/test/ (soc-qk4b)

Conformance Harnesses

Use this reference when /test needs to prove an implementation follows an external contract, compatibility surface, schema, protocol, CLI behavior, or generated artifact shape.

Trigger

Choose conformance testing when the target has a contract that can be exercised repeatedly:

• JSON schema, OpenAPI, protobuf, or CLI output schema.
• Golden behavior from an existing implementation.
• Round-trip parse/render/serialize behavior.
• Cross-runtime compatibility claims.
• Generated artifacts that must remain in sync with source files.

Harness Patterns

Required Loop

1. Identify the contract source of truth. 2. Build fixtures from the contract, not from the implementation under test. 3. Run the current implementation and capture output. 4. Canonicalize dynamic fields before comparing. 5. Fail loudly on unknown fields, missing fields, wrong exit codes, or silently skipped cases. 6. Write a coverage matrix showing contract rows covered and uncovered.

Output

Add a short conformance section to .agents/test/summary.md:

## Conformance Coverage

| Contract | Cases | Covered | Gaps |
|---|---:|---:|---|
| <schema or spec> | <n> | <n> | <missing cases> |

Stop Criteria

Stop when every must-support contract row has a mechanical test or a documented exclusion with owner and rationale.

---

Source: Adapted from an external skill corpus / testing-conformance-harnesses. Pattern-only, no verbatim text.

Fuzzing

Use this reference when /test targets parsers, serializers, file readers, protocol decoders, untrusted input handlers, state machines, or security-sensitive validation logic.

Trigger

Prioritize fuzzing for code that accepts:

• Raw bytes, strings, JSON, YAML, XML, CSV, or custom formats.
• Network request bodies, CLI arguments, file contents, or environment input.
• Deserialized objects from untrusted boundaries.
• State transition sequences where operation order matters.

Rules

• Keep fuzz targets deterministic.
• Minimize external I/O inside fuzz functions.
• Add seed corpus entries for known edge cases before relying on random discovery.
• Assert invariants, not implementation details.
• Save every crash or regression input as a stable fixture.

Target Template

Every fuzz target needs:

1. A small wrapper around the real public function. 2. A seed corpus with valid, invalid, empty, boundary, and previously broken inputs. 3. At least one invariant:

• no panic
• valid input round-trips
• invalid input returns an error
• output remains canonical
• resource use stays bounded

Triage

When fuzzing finds a failure:

1. Minimize the input. 2. Add it as a named regression fixture. 3. Write a normal unit test for the minimized case. 4. Fix the bug. 5. Re-run fuzzing and the regression test.

Output

Record fuzz coverage in .agents/test/summary.md:

## Fuzz Targets

| Target | Corpus Seeds | Duration | Findings |
|---|---:|---:|---|
| <target> | <n> | <time> | <none or issue> |

---

Source: Adapted from an external skill corpus / testing-fuzzing. Pattern-only, no verbatim text.

Golden Artifact Strategy

Use this reference before accepting changes to snapshots, generated reports, rendered docs, CLI output fixtures, or other checked-in artifacts that define a behavior contract.

Strategy

Golden artifacts are useful when the artifact itself is the interface. They are weak when the output is volatile, host-specific, or easier to assert through a structured parser.

Choose one comparison mode up front:

Update Rules

Do not refresh a golden artifact just to make a test pass.

1. Run the current test and inspect the diff. 2. Identify the source change that produced the diff. 3. Decide whether the diff is intended behavior, fixture drift, or a bug. 4. Update the artifact only for intended behavior or fixture drift. 5. Add a short note explaining why the new artifact is accepted.

Review Checklist

• Dynamic fields are scrubbed or parsed away.
• The fixture input is checked in near the artifact or named in the test.
• The test fails on missing fields, extra unknown fields, and wrong exit codes

when those are part of the contract.

• The update command is documented in the test or nearby fixture comment.

Output

Add an acceptance table to .agents/test/summary.md when golden files change:

## Golden Artifact Review

| Artifact | Decision | Evidence |
|---|---|---|
| <path> | accepted/rejected | <diff or command> |

---

Source: Adapted from an external skill corpus / testing-golden-artifacts. Pattern-only, no verbatim text.

Golden Artifacts

Use this reference when /test must protect generated files, rendered output, snapshots, reports, command output, or serialized data.

When Golden Tests Fit

Golden tests are useful when humans care about the exact artifact or when downstream tooling depends on stable shape.

Good targets:

• Generated markdown, JSON, YAML, CLI help, reports, and manifests.
• Rendered templates after dynamic fields are scrubbed.
• Cross-platform output after path and newline normalization.
• Structured output that can be canonicalized before comparison.

Avoid golden tests for volatile output that has no stable contract.

Golden Modes

Update Discipline

Never update golden files as the first move.

1. Run the test and inspect the diff. 2. Decide if the diff is intended. 3. If intended, update the golden file and mention why in the summary. 4. If unintended, fix the generator or source data.

Output

Add this to .agents/test/summary.md when golden files change:

## Golden Changes

| Artifact | Verdict | Reason |
|---|---|---|
| <path> | accepted/rejected | <why> |

---

Source: Adapted from an external skill corpus / testing-golden-artifacts. Pattern-only, no verbatim text.

Metamorphic Testing

Use this reference when /test needs stronger evidence than example-based assertions can provide, especially for ranking, transforms, parsers, planners, and other behavior where one exact expected answer is too narrow.

When To Use It

Metamorphic tests fit when the target should preserve or transform properties across related inputs.

Good targets:

• Parsers and serializers with round-trip or normalization behavior.
• Search, ranking, scoring, or filtering code with monotonicity rules.
• Refactors where old and new paths should agree on observable output.
• Data transforms where field order, whitespace, or batching should not matter.
• CLI wrappers where equivalent flags or input forms should converge.

Avoid metamorphic tests when the contract is a single fixed artifact. Use a golden artifact strategy for that case.

Relation Patterns

Test Loop

1. Name the invariant before writing cases. 2. Generate or hand-pick related inputs that differ in one controlled way. 3. Run the real public API, CLI, or script on every related input. 4. Compare the property that must hold, not private implementation details. 5. Add any failure input as a stable regression fixture.

Output

Record the invariant and generated cases in .agents/test/summary.md:

## Metamorphic Coverage

| Target | Relation | Cases | Findings |
|---|---|---:|---|
| <target> | <relation> | <n> | <none or issue> |

---

Source: Adapted from an external skill corpus / testing-metamorphic. Pattern-only, no verbatim text.

Real-Service E2E

Use this reference when mocks would hide the failure mode: auth flows, payment/webhook flows, queues, databases, storage, third-party APIs in sandbox mode, or multiple services with serialization boundaries.

Safety Gate

Before running real-service tests, prove the target is non-production:

• Test or sandbox credentials only.
• Dedicated test database, bucket, queue, project, or tenant.
• Destructive operations isolated by namespace or transaction rollback.
• Clear cleanup path.
• No live customer data.

If any safety check is unknown, stop and ask for an explicit test environment.

Pattern

1. Create test-owned resources with unique names. 2. Exercise the full boundary through the public interface. 3. Assert durable state, emitted events, logs, and API responses. 4. Clean up in defer, fixture teardown, or transaction rollback. 5. Capture enough evidence to debug failures without rerunning blindly.

What To Avoid

• Mocking the component whose integration is under test.
• Sharing mutable fixtures across tests.
• Sleeping for fixed durations when polling with timeouts would work.
• Running against production by default.
• Skipping cleanup on failure.

Output

Record real-service test safety in .agents/test/summary.md:

## Real-Service Safety

| Check | Result |
|---|---|
| Non-production credentials | PASS/FAIL |
| Isolated namespace | PASS/FAIL |
| Cleanup verified | PASS/FAIL |

---

Source: Adapted from an external skill corpus / testing-real-service-e2e-no-mocks. Pattern-only, no verbatim text.