Testing

Testing

Test-driven by default. Prove the problem before solving it.

Core Workflow: Bug First

1. Reproduce — Run the failing scenario and observe the actual behavior 2. Write a failing test — Minimal reproduction as a test case 3. Commit the failing test — Use Skill(commit) with message like test: reproduce #issue-description 4. Fix the code — Only now attempt the fix 5. Verify — Run pnpm test and confirm the test passes 6. Commit the fix — Separate commit from the test

This applies to all bug fixes. The failing test proves the bug exists and prevents regressions.

Fixture Tests

Noodle uses directory-based fixtures in testdata/. Read references/fixtures.md for full format, helper API, and code patterns.

Creating a New Fixture

./scripts/scaffold-fixture.sh <package-dir> <fixture-name> [state-count]

Fill in the generated input.json/input.ndjson and expected.md, then:

pnpm fixtures:hash:sync

Bug Fixtures

See references/fixtures.md for full bug fixture format (frontmatter fields, expected error sections, and the fix workflow).

Key Commands

Running Specific Tests

# Single package
go test ./loop

# Single test
go test ./loop -run TestLoopDirectoryFixtures

# Bypass cache
go test -count=1 ./parse

# With race detector
go test -race ./...

# Verbose
go test -v ./tui

Integration Tests

Long-running or external-dependency tests use testing.Short():

if testing.Short() {
    t.Skip("skipping integration test in short mode")
}

pnpm test:short skips these. pnpm test runs everything.

Fixture Test Reference

Directory Structure

package/testdata/
└── fixture-name/
    ├── expected.md          # Expected output with YAML frontmatter
    ├── .noodle.toml         # Optional config override
    ├── state-01/
    │   ├── input.json       # or input.ndjson
    │   └── .noodle/         # Runtime state (queue.json, etc.)
    └── state-02/            # Additional states numbered sequentially

expected.md Format

The file starts with YAML frontmatter delimited by ---, then markdown sections containing fenced JSON blocks.

Frontmatter fields:

--- schema_version: 1 expected_failure: false bug: false source_hash: pending ---

Sections (as H2 headings with fenced json blocks):

• ## Expected — Expected output (format varies by package)
• ## Expected Error — Error expectations (optional)
• ## Runtime Dump — Loop package uses this for multi-state transitions

Frontmatter Fields

Marking a Fixture as a Known Bug

Set both fields in frontmatter:

expected_failure: true
bug: true

Add a placeholder ## Expected Error section with JSON body {"any": true}.

Verify it appears in pnpm bugs.

Scaffold Script

./scripts/scaffold-fixture.sh <package-dir> <fixture-name> [state-count]

# Examples:
./scripts/scaffold-fixture.sh loop my-new-test 2
./scripts/scaffold-fixture.sh parse edge-case-input 1

Creates the directory, empty input files, and expected.md with source_hash: pending. Runs fixturehash sync automatically.

Recording Loop Fixtures

# Record new expected output
pnpm fixtures:loop:record

# Or manually:
NOODLE_LOOP_FIXTURE_MODE=record go test ./loop -run TestLoopDirectoryFixtures -count=1

After recording, sync hashes:

pnpm fixtures:hash:sync

Test Helper API (internal/testutil/fixturedir)

// Discover and load
inventory := fixturedir.LoadInventory(t, "testdata")

// Validate structure
fixturedir.AssertValidFixtureRoot(t, "testdata")

// Parse state data
input := fixturedir.ParseStateJSON[MyInput](t, state, "input.json")

// Get expected section
expected := fixturedir.MustSection(t, fixtureCase, "Expected")

// Assert errors
fixturedir.AssertError(t, "context", err, fixtureCase.ExpectedError)

Typical Fixture Test Pattern

func TestDirectoryFixtures(t *testing.T) {
    fixturedir.AssertValidFixtureRoot(t, "testdata")
    inventory := fixturedir.LoadInventory(t, "testdata")

    for _, fc := range inventory.Cases {
        fc := fc
        t.Run(fc.Name, func(t *testing.T) {
            state := fc.States[0]
            input := fixturedir.ParseStateJSON[MyType](t, state, "input.json")

            actual, err := MyFunction(input)

            fixturedir.AssertError(t, "test", err, fc.ExpectedError)
            if fc.ExpectedError != nil {
                return
            }

            expected := fixturedir.MustSection(t, fc, "Expected")
            // ... compare actual vs expected
        })
    }
}