Test Guard

Test Guard

You are reviewing generated or changed test code before it ships. Enforce the rules below after the first test-writing pass and before the tests are presented, committed, or merged. Be a sharp reviewer, not a pedantic one: flag what wastes maintenance effort or hides real bugs, ignore cosmetic preferences.

These rules exist because coding agents over-generate tests. The common failure modes: mock-heavy unit tests that assert implementation details, near-duplicate test bodies that differ by one value, and tests that re-verify the framework instead of the project's logic. Each looks productive in a diff and costs maintenance forever.

When this skill activates

• A coding agent has just written new test functions or test files, in any language
• You are editing existing tests
• You are reviewing a diff that contains test changes
• The user asks you to write, add, or review tests

Adapt to the project first

These rules are universal, but their application is not. Before reviewing:

1. Check the project's own agent instructions (CLAUDE.md, AGENTS.md) and testing docs. Project-specific testing rules win over this skill when they conflict. 2. Identify the test stack, then read the matching reference for concrete patterns:

• Python / pytest → references/pytest.md
• PHP / PHPUnit / Pest / WordPress → references/phpunit.md
• JavaScript / TypeScript / Jest / Vitest → references/jest.md

3. If the project calls LLM APIs, uses agent frameworks, or wires up observability/telemetry, also read references/llm-app-testing.md — it adds three rules specific to LLM applications. 4. Map the project's system boundaries: network calls, databases, filesystem, clock and randomness, third-party SDKs, LLM APIs. Existing fixtures and test helpers usually reveal where the project already draws these lines.

What to do

1. Read the test code: the diff, the new file, or the section being modified. 2. Check each test against the rules below. 3. Report violations concisely: rule number, location, why it violates, suggested fix. 4. If the user explicitly invokes this skill before test writing, apply the rules as you write — don't write violations and then flag them.

When writing new tests, ask for each test: "What specific bug does this catch that no other test in this suite catches?" If you can't answer clearly, don't write it.

The Nine Rules

Rule 1: Test behavior, not implementation

Test what code does from the caller's perspective. Assert return values and observable side effects. Never assert that an internal helper was called with specific arguments — that test breaks on every refactor while catching nothing.

Violation pattern: asserting a mock of an internal function was called, where that function is not a system boundary. Fix: assert the return value or the state change the caller observes.

Rule 2: Every mock must be justified

Mock only at system boundaries: network and HTTP calls, LLM APIs, databases, filesystem I/O on external files, clock and randomness, third-party SDKs. Never mock internal classes or helper functions to isolate a "unit" — the seams you create hide the integration bugs worth catching.

When you mock a boundary, assert what the caller does with the response, not that the mock received specific arguments.

Rule 3: One scenario per test, data-driven for variants

If two or more tests share identical setup and differ only in input/output values, merge them into one data-driven test (@pytest.mark.parametrize, PHPUnit #[DataProvider], Jest test.each).

When separate tests ARE correct: different setup, different assertions, different mock configurations, or genuinely different scenarios that happen to exercise the same function.

Rule 4: Every test must justify its existence

Ask: "What bug does this catch that no other test catches?" Delete tests that only catch typos, verify default values of data classes, or test trivial pass-through logic.

Common unjustified tests: constructors setting attributes, a function rejecting input the type system already forbids, string formatting of log messages, a constant equaling its literal value.

Rule 5: Name tests for the scenario

Pattern: test_<scenario>_<expected_outcome>. The name should read like a requirement, not echo the function signature.

Rule 6: Production regression tests are sacred

Tests that reproduce a real production bug are always justified. Reference the incident (date, issue ID, or short description) in the name or a comment, and never delete them. They are exempt from Rule 4 — their justification is the incident.

Rule 7: No tests for framework guarantees

Don't test that the validation library validates, the ORM commits, the router returns 404, or the test framework's fixtures work. Test your logic that sits on top of the framework.

Violation pattern: a test that would still pass if you deleted all the project's custom code and kept only framework defaults.

Rule 8: State and value objects are real, never mocked

Never mock a data model, DTO, entity, or state object. Construct a real instance. Mocking state hides field-name typos and validation errors — exactly the bugs worth catching. If constructing the real object is painful, that is design feedback, not a reason to mock; add a small builder or factory helper.

Rule 9: Infrastructure under test gets real infrastructure

When database queries, schema behavior, or persistence logic is the subject of the test, run against a real test database with real migrations applied via fixtures. Mocking the session there tests nothing. Mocking the database is fine when persistence is only a side effect of the behavior under test.

Reporting format

When flagging violations, use this format:

**Rule N violation** in `tests/path/file.ext::<test_name>`
- What: <one sentence describing the violation>
- Fix: <one sentence describing what to do instead>

Group violations by file. If a file has no violations, don't mention it.

Severity guide

Not all violations are equal. Use judgment:

• Must fix: Rules 1, 2, 8 — these hide real bugs or make tests brittle
• Should fix: Rules 3, 4, 5, 7 — these cause bloat and maintenance drag
• Sacred: Rule 6 — never delete, always allow
• Worth noting: Rule 9 — test architecture; flag it, but don't block small changes on it

References

• references/pytest.md — Python/pytest patterns: parametrize, fixtures, mock boundaries, real Pydantic instances
• references/phpunit.md — PHP/PHPUnit/Pest patterns, including WordPress and WooCommerce test boundaries
• references/jest.md — Jest/Vitest patterns: test.each, module mocks, msw, snapshot discipline
• references/llm-app-testing.md — three extra rules for LLM applications: prompt contracts, observability wiring, agent-flow transitions

What this skill does NOT do

• It does not run tests. Use the project's test runner for that.
• It does not enforce code style — that's the linter's job.
• It does not decide what to test — only how to test it.
• It does not flag pre-existing violations in files you're not touching, unless asked to audit.

Test Guard — JavaScript / TypeScript / Jest / Vitest Patterns

Concrete applications of the nine rules for JS/TS projects. Read this when reviewing or writing Jest or Vitest tests.

Rule 2: Mock boundaries in JS/TS

Justified mock targets:

• Network: prefer msw (Mock Service Worker) over jest.mock-ing your own fetch wrapper — it mocks at the true boundary
• LLM / third-party SDK clients (openai, @anthropic-ai/sdk, Stripe, etc.)
• Timers and randomness: vi.useFakeTimers() / jest.useFakeTimers(), seeded RNG
• Filesystem and process env in Node code

Unjustified mocks:

• jest.mock('../utils/helpers') — mocking your own internal module to isolate a "unit" (Rule 2)
• Mocking a class's private method via prototype patching (Rule 1)
• Hand-built object literals pretending to be domain entities when a real constructor or factory exists (Rule 8)

Rule 3: test.each

// Violation: three near-identical it() blocks
// Fix:
test.each([
  ['Hello World', 'hello-world'],
  ['  padded  ', 'padded'],
  ['Café Menu', 'cafe-menu'],
])('slugify(%s) → %s', (raw, expected) => {
  expect(slugify(raw)).toBe(expected);
});

Snapshot discipline

Snapshot tests are implementation tests in disguise unless the snapshot is the contract (e.g., a public JSON output, a CLI's help text). Avoid snapshots of:

• Full component trees that change on every styling tweak (Rule 1 — brittle, asserts implementation)
• Large objects nobody reviews — an unread snapshot approves itself (Rule 4)

Prefer targeted assertions: expect(screen.getByRole('button')).toHaveTextContent('Save').

UI component tests

• Test what the user sees and does (Testing Library queries by role/label), not component internals or state hooks (Rule 1).
• Don't test that React renders, routes resolve, or props propagate — framework guarantees (Rule 7).

Rule 9: Real persistence

For data-layer logic (Prisma/Drizzle/Knex queries), run against a real test database — testcontainers, a Dockerized Postgres, or SQLite where compatible. Mocking the query builder to test the query builder tests nothing.

Test Guard — LLM Application Rules

Three additional rules for projects that call LLM APIs, use agent/workflow frameworks (LangGraph, CrewAI, custom state machines), or wire up observability/telemetry (Langfuse, LangSmith, OpenTelemetry). Apply these on top of the nine core rules.

Rule 10: Prompt tests — test the contract, not the content

Prompt text changes constantly; tests pinned to wording rot within a week. Don't assert specific phrasing.

Do test:

• The prompt template exists and loads without error (smoke test)
• Template variables are substituted correctly — no leftover {placeholder} markers
• The prompt contains required structural markers if the caller parses them (e.g., a JSON schema block, a delimiter the parser splits on)

Rule 11: Observability is infrastructure

Don't unit-test telemetry wiring. The violation pattern is asserting a tracing/analytics mock's call arguments:

# Violation — tests wiring, not behavior
mock_tracer.assert_called_once_with(session_id=..., tags=[...])

Mocking observability calls to prevent side effects during tests is fine and often necessary. Just don't assert on the mock's call args. If telemetry breaks, dashboards show it; a unit test asserting wiring only breaks refactors.

Rule 12: Agent and flow tests test transitions

For agent frameworks and state machines: test that given a state plus an event, the flow reaches the correct next state with the correct fields set. Mock the LLM calls to return controlled responses.

Test: state in → state out.

Don't test: the exact prompt string passed to the LLM, the number of LLM calls made, or internal retry logic — those are implementation details (Rule 1) that change with every model upgrade.

A useful pattern is a table of transition cases (data-driven, Rule 3): starting state, mocked LLM response, expected resulting state.

Severity

• Must fix: Rule 12 violations that assert prompt strings or call counts — they break on every model/prompt change
• Should fix: Rule 10 wording assertions
• Worth noting: Rule 11 — flag it, but don't block a small change on it

Test Guard — PHP / PHPUnit / Pest Patterns

Concrete applications of the nine rules for PHP projects, including WordPress and WooCommerce. Read this when reviewing or writing PHP tests.

Rule 2: Mock boundaries in PHP

Justified mock targets:

• HTTP: Guzzle handlers/middleware, pre_http_request filter in WordPress
• External SDKs: payment gateways, mail providers, LLM API clients
• Clock: inject a clock (psr/clock) instead of calling time() directly
• Filesystem on external paths (prefer vfsStream or temp dirs over mocking)

Unjustified mocks:

• Mockery/Prophecy doubles for the project's own value objects, DTOs, or entities — construct real instances (Rule 8)
• Mocking internal services just to isolate a class — if wiring is painful, fix the constructor, don't fake the collaborator
• Partial mocks of the class under test — you are no longer testing the class

Rule 3: Data providers

/**
 * @see Rule 3 — variants of one scenario belong in a data provider.
 */
#[DataProvider('provideSlugCases')]
public function test_slugify_normalizes_input( string $raw, string $expected ): void {
    $this->assertSame( $expected, slugify( $raw ) );
}

public static function provideSlugCases(): array {
    return array(
        'lowercases'     => array( 'Hello World', 'hello-world' ),
        'strips padding' => array( '  padded  ', 'padded' ),
        'transliterates' => array( 'Café Menu', 'cafe-menu' ),
    );
}

Pest equivalent: it('normalizes slug', ...)->with([...]).

WordPress-specific boundaries

• Integration tests (WP_UnitTestCase / wp-env / wp-cli scaffold): use the real WordPress test framework with factories — self::factory()->post->create(), self::factory()->user->create(). Don't mock WP_Post or WP_User; the factories exist precisely so you don't have to (Rule 8).
• Unit tests without WordPress loaded (Brain Monkey / WP_Mock): mocking WordPress functions like get_option() or apply_filters() is a boundary mock and justified. But assert what your code does with the values, not that get_option was called with specific args (Rule 1).
• Mock outbound HTTP with the pre_http_request filter rather than patching wp_remote_get internals.
• Don't test that WordPress sanitizes, escapes, or that hooks fire — that's core's guarantee (Rule 7). Test your callback's behavior given an input.

WooCommerce notes

• Build real WC_Product / WC_Order objects via WC_Helper_Product and WC_Helper_Order in integration tests — never MagicMock-style doubles of them (Rule 8).
• Cart and checkout logic is stateful: prefer integration tests over heavily mocked unit tests; mocked carts hide hook-ordering bugs.

Rule 9: Real database

WP_UnitTestCase already wraps each test in a transaction against a real schema — use it for query, meta, and persistence logic instead of mocking $wpdb. Mocking $wpdb->prepare or $wpdb->get_results to test a query builder tests nothing.

Test Guard — Python / pytest Patterns

Concrete applications of the nine rules for pytest projects. Read this when reviewing or writing Python tests.

Rule 2: Mock boundaries in Python

Justified mock targets:

• HTTP clients: httpx, requests, aiohttp (or use respx / responses instead of raw mocks)
• LLM SDK calls: openai, anthropic, litellm.completion and friends
• Database sessions, when the database is not the subject (see Rule 9)
• Filesystem I/O on external paths (tmp_path fixture is often better than mocking)
• Clock and randomness: time.time, datetime.now, random (prefer freezegun or injected clocks)

Unjustified mocks (common agent-generated violations):

• MagicMock() standing in for a Pydantic model or dataclass — construct the real thing
• Mocking internal utility functions to isolate a "unit"
• Mocking json.loads / json.dumps or other stdlib pure functions

Rule 3: Parametrize

# Violation: three copy-pasted tests differing by one value
def test_slug_lowercase(): ...
def test_slug_strips_spaces(): ...
def test_slug_handles_unicode(): ...

# Fix
@pytest.mark.parametrize(
    ("raw", "expected"),
    [
        ("Hello World", "hello-world"),
        ("  padded  ", "padded"),
        ("Café Menu", "cafe-menu"),
    ],
)
def test_slugify_normalizes_input(raw, expected):
    assert slugify(raw) == expected

Rule 8: Real Pydantic/dataclass instances

# Wrong — hides field typos and validation errors
state = MagicMock()
state.user_id = "123"
state.status = "ACTIVE"

# Right — Pydantic validates the construction itself
state = UserState(user_id="123", status="ACTIVE")

If a model needs many fields, add a factory fixture or use factory_boy — don't fall back to MagicMock.

Rule 9: Real database via fixtures

Use a fixture that applies real migrations (e.g., a session-scoped test database with alembic upgrade head), and function-scoped transactions rolled back per test. pytest-postgresql, testcontainers, or an SQLite-compatible fallback all work; the point is real schema, not a mocked session, whenever query or persistence logic is the subject.

pytest-specific smells

• assert mock.call_count == N on anything internal — Rule 1 violation
• @patch stacks three or more deep — the test is coupled to implementation; restructure
• Asserting log output via caplog for messages no caller parses — Rule 4 violation
• Fixtures that build mocks of project classes — Rule 8 violation, make the fixture build real objects