Code Testing Extensions

Code Testing Extensions

This skill provides access to language-specific guidance files used by the code-testing pipeline. Call this skill to get the file paths, then read the relevant file for your target language.

Available Extension Files

Usage

Read the appropriate extension file for the target language before writing test code. When an <language>-examples.md file exists for the target language, read it alongside the base extension to see a concrete end-to-end pipeline walkthrough (research output, plan, generated tests, fix cycles, final report).

C++ Extension

Language-specific guidance for C++ test generation.

Testing Internals

If types are not well suited for testing only through their public surface, consider exposing internals to tests using a preprocessor-guarded friend declaration:

class MyClass {
#ifdef UNIT_TESTING
    friend class MyClassTest;
#endif
    // ...
};

Define UNIT_TESTING only in the test build configuration so production builds remain unaffected.

.NET Pipeline Examples

Concrete input→output examples for the test generation pipeline targeting a .NET/C# codebase. These show what each pipeline phase produces for a small project.

Source Under Test

A simple InvoiceService in a .NET 9 project using MSTest:

src/
  Contoso.Billing/
    Contoso.Billing.csproj
    InvoiceService.cs
    Invoice.cs
    IInvoiceRepository.cs
tests/
  Contoso.Billing.Tests/
    Contoso.Billing.Tests.csproj    (exists, references Contoso.Billing)
Contoso.Billing.sln

// InvoiceService.cs
namespace Contoso.Billing;

public class InvoiceService(IInvoiceRepository repository)
{
    public decimal CalculateTotal(Invoice invoice)
    {
        if (invoice is null) throw new ArgumentNullException(nameof(invoice));
        if (invoice.LineItems.Count == 0) throw new InvalidOperationException("Invoice has no line items.");

        var subtotal = invoice.LineItems.Sum(li => li.Quantity * li.UnitPrice);
        var tax = subtotal * invoice.TaxRate;
        return Math.Round(subtotal + tax, 2);
    }

    public async Task<Invoice> GetByIdAsync(int id)
    {
        var invoice = await repository.FindAsync(id);
        return invoice ?? throw new KeyNotFoundException($"Invoice {id} not found.");
    }

    public async Task MarkAsPaidAsync(int id)
    {
        var invoice = await repository.FindAsync(id)
            ?? throw new KeyNotFoundException($"Invoice {id} not found.");
        if (invoice.Status == InvoiceStatus.Paid)
            throw new InvalidOperationException("Invoice is already paid.");
        invoice.Status = InvoiceStatus.Paid;
        invoice.PaidDate = DateTime.UtcNow;
        await repository.UpdateAsync(invoice);
    }
}

Sample Research Output

What code-testing-researcher produces in .testagent/research.md:

# Test Generation Research

## Project Overview
- **Path**: C:\src\Contoso.Billing
- **Language**: C# (.NET 9)
- **Framework**: .NET 9 (net9.0)
- **Test Framework**: MSTest 3.8

## Coverage Baseline
- **Initial Line Coverage**: unknown
- **Strategy**: broad
- **Existing Test Count**: 0 tests across 0 files

## Build & Test Commands
- **Build**: `dotnet build Contoso.Billing.sln`
- **Test**: `dotnet test Contoso.Billing.sln`
- **Lint**: `dotnet format Contoso.Billing.sln`

## Project Structure
- Source: `src/Contoso.Billing/`
- Tests: `tests/Contoso.Billing.Tests/` (exists, empty)

## Files to Test

### High Priority
| File | Classes/Functions | Testability | Notes |
|------|-------------------|-------------|-------|
| src/Contoso.Billing/InvoiceService.cs | InvoiceService: CalculateTotal, GetByIdAsync, MarkAsPaidAsync | High | Core business logic, repository dependency needs mocking |

### Low Priority / Skip
| File | Reason |
|------|--------|
| src/Contoso.Billing/Invoice.cs | Data model, no logic |
| src/Contoso.Billing/IInvoiceRepository.cs | Interface, no implementation |

## Existing Tests
- No existing tests found

## Existing Test Projects
- **Project file**: `tests/Contoso.Billing.Tests/Contoso.Billing.Tests.csproj`
- **Target source project**: `src/Contoso.Billing/Contoso.Billing.csproj`
- **Test files**: none

## Testing Patterns
- No existing patterns; recommend sealed test classes, AAA structure, `Moq` for mocking IInvoiceRepository

## Recommendations
- Start with InvoiceService.CalculateTotal (pure logic, easy to test)
- Then async methods (require mocking IInvoiceRepository)

Sample Plan Output

What code-testing-planner produces in .testagent/plan.md:

# Test Implementation Plan

## Overview
Generate MSTest tests for the Contoso.Billing InvoiceService, covering all three
public methods across happy path, edge case, and error scenarios. Single phase
since there is only one source file.

## Commands
- **Build**: `dotnet build tests/Contoso.Billing.Tests/Contoso.Billing.Tests.csproj`
- **Test**: `dotnet test tests/Contoso.Billing.Tests/Contoso.Billing.Tests.csproj`
- **Lint**: `dotnet format --include tests/Contoso.Billing.Tests/`

## Phase Summary
| Phase | Focus | Files | Est. Tests |
|-------|-------|-------|------------|
| 1 | InvoiceService | 1 | 9-12 |

---

## Phase 1: InvoiceService

### Overview
Cover all public methods of InvoiceService. CalculateTotal is pure logic tested
with DataRow. Async methods require a mocked IInvoiceRepository.

### Files to Test

#### 1. InvoiceService.cs
- **Source**: `src/Contoso.Billing/InvoiceService.cs`
- **Test File**: `tests/Contoso.Billing.Tests/InvoiceServiceTests.cs`
- **Test Class**: `InvoiceServiceTests`

**Methods to Test**:
1. `CalculateTotal` — Pure calculation logic
   - Happy path: single line item returns quantity × price + tax
   - Happy path: multiple line items summed correctly
   - Edge case: zero tax rate returns subtotal only
   - Error case: null invoice throws ArgumentNullException
   - Error case: empty line items throws InvalidOperationException

2. `GetByIdAsync` — Repository lookup
   - Happy path: existing ID returns invoice
   - Error case: non-existent ID throws KeyNotFoundException

3. `MarkAsPaidAsync` — State transition
   - Happy path: unpaid invoice transitions to Paid with PaidDate set
   - Error case: already paid throws InvalidOperationException
   - Error case: non-existent ID throws KeyNotFoundException

### Success Criteria
- [ ] All test files created
- [ ] Tests compile with `dotnet build`
- [ ] All tests pass with `dotnet test`

Sample Generated Test File

What code-testing-implementer produces:

using Microsoft.VisualStudio.TestTools.UnitTesting;
using Moq;
using Contoso.Billing;

namespace Contoso.Billing.Tests;

[TestClass]
public sealed class InvoiceServiceTests
{
    private readonly Mock<IInvoiceRepository> _repositoryMock = new();
    private readonly InvoiceService _sut;

    public InvoiceServiceTests()
    {
        _sut = new InvoiceService(_repositoryMock.Object);
    }

    // --- CalculateTotal ---

    [TestMethod]
    [DataRow(1, 100.00, 0.10, 110.00, DisplayName = "Single item with 10% tax")]
    [DataRow(3, 25.00, 0.0, 75.00, DisplayName = "Multiple quantity, zero tax")]
    public void CalculateTotal_ValidLineItems_ReturnsExpectedTotal(
        int quantity, double unitPrice, double taxRate, double expected)
    {
        // Arrange
        var invoice = new Invoice
        {
            TaxRate = (decimal)taxRate,
            LineItems = [new LineItem { Quantity = quantity, UnitPrice = (decimal)unitPrice }]
        };

        // Act
        var total = _sut.CalculateTotal(invoice);

        // Assert
        Assert.AreEqual((decimal)expected, total);
    }

    [TestMethod]
    public void CalculateTotal_NullInvoice_ThrowsArgumentNullException()
    {
        Assert.ThrowsExactly<ArgumentNullException>(() => _sut.CalculateTotal(null!));
    }

    [TestMethod]
    public void CalculateTotal_EmptyLineItems_ThrowsInvalidOperationException()
    {
        // Arrange
        var invoice = new Invoice { LineItems = [] };

        // Act & Assert
        Assert.ThrowsExactly<InvalidOperationException>(() => _sut.CalculateTotal(invoice));
    }

    // --- GetByIdAsync ---

    [TestMethod]
    public async Task GetByIdAsync_ExistingId_ReturnsInvoice()
    {
        // Arrange
        var expected = new Invoice { Id = 42 };
        _repositoryMock.Setup(r => r.FindAsync(42)).ReturnsAsync(expected);

        // Act
        var result = await _sut.GetByIdAsync(42);

        // Assert
        Assert.AreSame(expected, result);
    }

    [TestMethod]
    public async Task GetByIdAsync_NonExistentId_ThrowsKeyNotFoundException()
    {
        // Arrange
        _repositoryMock.Setup(r => r.FindAsync(999)).ReturnsAsync((Invoice?)null);

        // Act & Assert
        await Assert.ThrowsExactlyAsync<KeyNotFoundException>(
            () => _sut.GetByIdAsync(999));
    }

    // --- MarkAsPaidAsync ---

    [TestMethod]
    public async Task MarkAsPaidAsync_UnpaidInvoice_SetsStatusAndDate()
    {
        // Arrange
        var invoice = new Invoice { Id = 1, Status = InvoiceStatus.Pending };
        _repositoryMock.Setup(r => r.FindAsync(1)).ReturnsAsync(invoice);

        // Act
        await _sut.MarkAsPaidAsync(1);

        // Assert
        Assert.AreEqual(InvoiceStatus.Paid, invoice.Status);
        Assert.IsNotNull(invoice.PaidDate);
        _repositoryMock.Verify(r => r.UpdateAsync(invoice), Times.Once);
    }

    [TestMethod]
    public async Task MarkAsPaidAsync_AlreadyPaid_ThrowsInvalidOperationException()
    {
        // Arrange
        var invoice = new Invoice { Id = 1, Status = InvoiceStatus.Paid };
        _repositoryMock.Setup(r => r.FindAsync(1)).ReturnsAsync(invoice);

        // Act & Assert
        await Assert.ThrowsExactlyAsync<InvalidOperationException>(
            () => _sut.MarkAsPaidAsync(1));
    }
}

Sample Fix Cycle

When the implementer encounters a build error, the fixer agent diagnoses and resolves it:

Build output:

error CS0246: The type or namespace name 'Moq' could not be found (are you missing a using directive or an assembly reference?)

Fixer diagnosis: The test project is missing the Moq NuGet package.

Fix applied:

dotnet add tests/Contoso.Billing.Tests/Contoso.Billing.Tests.csproj package Moq

Rebuild: dotnet build tests/Contoso.Billing.Tests/Contoso.Billing.Tests.csproj → SUCCESS

---

Another common cycle:

Build output:

error CS7036: There is no argument given that corresponds to the required parameter 'repository' of 'InvoiceService.InvoiceService(IInvoiceRepository)'

Fixer diagnosis: Test code instantiated new InvoiceService() without passing the required constructor parameter. The source uses a primary constructor with an IInvoiceRepository dependency.

Fix applied:

// Before (wrong)
var sut = new InvoiceService();

// After (fixed)
var repositoryMock = new Mock<IInvoiceRepository>();
var sut = new InvoiceService(repositoryMock.Object);

Rebuild: SUCCESS

Sample Final Report

What code-testing-generator produces at Step 9:

## Test Generation Report

**Project**: Contoso.Billing
**Strategy**: Single pass

### Results
| Metric         | Value |
|----------------|-------|
| Tests created  | 9     |
| Tests passing  | 9     |
| Tests failing  | 0     |
| Files created  | 1     |

### Files Created
- `tests/Contoso.Billing.Tests/InvoiceServiceTests.cs` (9 tests)

### Coverage
- InvoiceService.CalculateTotal — 3 happy path, 2 error cases
- InvoiceService.GetByIdAsync — 1 happy path, 1 error case
- InvoiceService.MarkAsPaidAsync — 1 happy path, 1 error case

### Build Validation
- Scoped build: ✅ passed
- Full solution build (`dotnet build --no-incremental`): ✅ passed

### Next Steps
- Add integration tests for repository layer if needed
- Consider testing with multiple line items for CalculateTotal

.NET Extension

Language-specific guidance for .NET (C#/F#/VB) test generation.

Build Commands

• Use --no-restore if dependencies are already restored
• Use -v:q (quiet) to reduce output noise
• Always use --no-incremental for the final validation build — incremental builds hide errors like CS7036

Test Commands

• Use --no-build if already built
• Use -v:q for quieter output

Lint Command

dotnet format --include path/to/file.cs
dotnet format MySolution.sln         # full solution

Project Reference Validation

Before writing test code, read the test project's .csproj to verify it has <ProjectReference> entries for the assemblies your tests will use. If a reference is missing, add it:

<ItemGroup>
    <ProjectReference Include="../SourceProject/SourceProject.csproj" />
</ItemGroup>

This prevents CS0234 ("namespace not found") and CS0246 ("type not found") errors.

Common CS Error Codes

.csproj / .sln Handling

• During phase implementation, build only the specific test .csproj for speed
• For the final validation, build the full .sln with --no-incremental
• Full-solution builds catch cross-project reference errors invisible in scoped builds

Registering a new test project (MANDATORY when dotnet new was used)

A new .csproj is invisible to dotnet test <solution>, to dotnet test run from the repo root, and to any CI/benchmark harness until it is added to the solution. Run dotnet sln add immediately after creating the project as part of Step 3 ("Register Test Project with Build System") — do not defer it to a later step.

1. Use the exact solution or solution-filter target identified in .testagent/research.md or .testagent/plan.md — do not search for or substitute a different .sln, .slnx, or .slnf target. 2. If that target is a .sln or .slnx, run dotnet sln <solution> add <test-project.csproj>. 3. If the target is a .slnf (solution filter), also ensure the new project is included in the filter; adding only to the underlying .sln may not be enough for test discovery. 4. Skip this if the project is already included in the solution or solution filter used for testing. 5. Prefer the researched test command. If you need to run the solution directly, use dotnet test --solution <solution> only for repos on .NET SDK 10+ with MTP-style syntax; otherwise use the standard positional form dotnet test <solution>.

Harness Discovery Check

Before reporting success, run the harness-equivalent discovery command from the repo root and confirm the test count went up by at least the number of tests you generated. The harness (CI, msbench, coverage tools) does not know which .csproj you targeted — it runs the solution-level command, so a test that passes via dotnet test MyProject.Tests.csproj is still worthless if dotnet test <solution> --list-tests doesn't enumerate it.

# From repo root, against the solution identified in .testagent/research.md
dotnet test <solution> --list-tests --no-build 2>&1 | grep -c '^    [A-Za-z]'

If the delta is 0, the new project isn't in the solution. Run dotnet sln <solution> add <test-project.csproj> and re-run the check. Do not report success until the harness command sees your new tests.

Test Framework Detection

Detect the framework from the test project's .csproj package references and match its conventions:

Use the repo's existing framework — do not introduce a different one.

MSTest Template

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace ProjectName.Tests;

[TestClass]
public sealed class ClassNameTests
{
    [TestMethod]
    public void MethodName_Scenario_ExpectedResult()
    {
        // Arrange
        var sut = new ClassName();

        // Act
        var result = sut.MethodName(input);

        // Assert
        Assert.AreEqual(expected, result);
    }

    [TestMethod]
    [DataRow(2, 3, 5, DisplayName = "Positive numbers")]
    [DataRow(-1, 1, 0, DisplayName = "Negative and positive")]
    public void Add_ValidInputs_ReturnsSum(int a, int b, int expected)
    {
        // Act
        var result = _sut.Add(a, b);

        // Assert
        Assert.AreEqual(expected, result);
    }
}

Skip Coverage Tools

Do not configure or run code coverage measurement tools (coverlet, dotnet-coverage, XPlat Code Coverage) by default. These tools have inconsistent cross-configuration behavior and waste significant time. Coverage is measured separately by the evaluation harness.

Exception: if the user or evaluation harness explicitly requires a Cobertura/XML coverage artifact (e.g., they ask for coverlet.collector or a --collect:"XPlat Code Coverage" run), add the coverlet.collector PackageReference to the generated .NET test csproj so the harness's coverage command can produce output. Do not run the coverage command yourself; leave that to the validation step.

Go Pipeline Examples

Concrete input→output examples for the test generation pipeline targeting a Go codebase. These show what each pipeline phase produces for a small package.

Source Under Test

A simple InvoiceService in a Go module:

go.mod                                 (module github.com/contoso/billing)
internal/billing/
  invoice.go
  invoice_repository.go               (defines the InvoiceRepository interface)
  invoice_service.go

// internal/billing/invoice_service.go
package billing

import (
    "context"
    "errors"
    "fmt"
    "math"
    "time"
)

type InvoiceService struct {
    repository InvoiceRepository
    now        func() time.Time
}

func NewInvoiceService(repo InvoiceRepository) *InvoiceService {
    return &InvoiceService{repository: repo, now: time.Now}
}

func (s *InvoiceService) CalculateTotal(invoice *Invoice) (float64, error) {
    if invoice == nil {
        return 0, errors.New("invoice must not be nil")
    }
    if len(invoice.LineItems) == 0 {
        return 0, errors.New("invoice has no line items")
    }
    var subtotal float64
    for _, li := range invoice.LineItems {
        subtotal += float64(li.Quantity) * li.UnitPrice
    }
    tax := subtotal * invoice.TaxRate
    return math.Round((subtotal+tax)*100) / 100, nil
}

func (s *InvoiceService) GetByID(ctx context.Context, id int) (*Invoice, error) {
    invoice, err := s.repository.Find(ctx, id)
    if err != nil {
        return nil, err
    }
    if invoice == nil {
        return nil, fmt.Errorf("invoice %d not found", id)
    }
    return invoice, nil
}

func (s *InvoiceService) MarkAsPaid(ctx context.Context, id int) error {
    invoice, err := s.repository.Find(ctx, id)
    if err != nil {
        return err
    }
    if invoice == nil {
        return fmt.Errorf("invoice %d not found", id)
    }
    if invoice.Status == StatusPaid {
        return errors.New("invoice is already paid")
    }
    invoice.Status = StatusPaid
    invoice.PaidDate = s.now()
    return s.repository.Update(ctx, invoice)
}

Sample Research Output

What code-testing-researcher produces in .testagent/research.md:

# Test Generation Research

## Project Overview
- **Path**: /work/billing
- **Language**: Go 1.22 (from go.mod)
- **Module**: github.com/contoso/billing
- **Test Framework**: standard `testing` package (no testify/gomock detected in go.sum)

## Coverage Baseline
- **Initial Line Coverage**: unknown
- **Strategy**: broad
- **Existing Test Count**: 0 tests across 0 files

## Build & Test Commands
- **Vet**: `go vet ./...`
- **Build**: `go build ./...`
- **Compile tests**: `go test -count=1 -run=^$ ./internal/billing`
- **Test**: `go test -count=1 ./internal/billing`

## Project Structure
- Source: `internal/billing/`
- Tests: none

## Files to Test

### High Priority
| File | Functions | Testability | Notes |
|------|-----------|-------------|-------|
| internal/billing/invoice_service.go | InvoiceService.CalculateTotal, GetByID, MarkAsPaid | High | Uses InvoiceRepository interface — easy to fake with a hand-written struct |

## Existing Tests
- No existing tests found

## Testing Patterns
- No existing patterns; recommend white-box `package billing` tests with hand-written fake repository (no testify since the repo doesn't use it), table-driven `t.Run` subtests for CalculateTotal, and an injected `now func() time.Time` for MarkAsPaid.

## Recommendations
- Inject `now` instead of stubbing `time.Now` globally — the struct already supports it
- Place tests in `internal/billing/invoice_service_test.go` (same package, white-box)

Sample Plan Output

# Test Implementation Plan

## Overview
Generate standard-library Go tests for InvoiceService using table-driven subtests
and a hand-written fake repository. Single phase since there is only one source file.

## Commands
- **Compile tests**: `go test -count=1 -run=^$ ./internal/billing`
- **Test**: `go test -count=1 -v ./internal/billing`

## Phase 1: InvoiceService

### Files to Test

#### 1. invoice_service.go
- **Source**: `internal/billing/invoice_service.go`
- **Test File**: `internal/billing/invoice_service_test.go`

**Functions to Test**:
1. `CalculateTotal` — Table-driven
   - Happy paths: single item, multi-item, rounding
   - Error cases: nil invoice, empty line items
2. `GetByID` — happy + missing + repo error
3. `MarkAsPaid` — happy (verifies timestamp via injected clock) + already-paid + missing + repo error

Sample Generated Test File

// internal/billing/invoice_service_test.go
package billing

import (
    "context"
    "errors"
    "strings"
    "testing"
    "time"
)

type fakeRepository struct {
    findFunc   func(ctx context.Context, id int) (*Invoice, error)
    updateFunc func(ctx context.Context, invoice *Invoice) error
    updated    *Invoice
}

func (f *fakeRepository) Find(ctx context.Context, id int) (*Invoice, error) {
    if f.findFunc != nil {
        return f.findFunc(ctx, id)
    }
    return nil, nil
}

func (f *fakeRepository) Update(ctx context.Context, invoice *Invoice) error {
    f.updated = invoice
    if f.updateFunc != nil {
        return f.updateFunc(ctx, invoice)
    }
    return nil
}

func TestInvoiceService_CalculateTotal(t *testing.T) {
    tests := []struct {
        name    string
        invoice *Invoice
        want    float64
        wantErr string
    }{
        {
            name:    "single item with 10% tax",
            invoice: &Invoice{TaxRate: 0.10, LineItems: []LineItem{{Quantity: 1, UnitPrice: 100}}},
            want:    110,
        },
        {
            name:    "multi quantity zero tax",
            invoice: &Invoice{TaxRate: 0, LineItems: []LineItem{{Quantity: 3, UnitPrice: 25}}},
            want:    75,
        },
        {
            name:    "rounds half up",
            invoice: &Invoice{TaxRate: 0.07, LineItems: []LineItem{{Quantity: 2, UnitPrice: 9.99}}},
            want:    21.38,
        },
        {
            name:    "nil invoice errors",
            invoice: nil,
            wantErr: "invoice must not be nil",
        },
        {
            name:    "empty line items errors",
            invoice: &Invoice{TaxRate: 0, LineItems: []LineItem{}},
            wantErr: "no line items",
        },
    }
    sut := NewInvoiceService(&fakeRepository{})
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            got, err := sut.CalculateTotal(tt.invoice)
            if tt.wantErr != "" {
                if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
                    t.Fatalf("expected error containing %q, got %v", tt.wantErr, err)
                }
                return
            }
            if err != nil {
                t.Fatalf("unexpected error: %v", err)
            }
            if got != tt.want {
                t.Errorf("CalculateTotal = %v, want %v", got, tt.want)
            }
        })
    }
}

func TestInvoiceService_GetByID(t *testing.T) {
    ctx := context.Background()
    want := &Invoice{ID: 42}

    t.Run("returns invoice when found", func(t *testing.T) {
        repo := &fakeRepository{findFunc: func(_ context.Context, _ int) (*Invoice, error) { return want, nil }}
        sut := NewInvoiceService(repo)
        got, err := sut.GetByID(ctx, 42)
        if err != nil || got != want {
            t.Fatalf("got (%v, %v), want (%v, nil)", got, err, want)
        }
    })

    t.Run("returns not-found error when missing", func(t *testing.T) {
        repo := &fakeRepository{findFunc: func(_ context.Context, _ int) (*Invoice, error) { return nil, nil }}
        sut := NewInvoiceService(repo)
        _, err := sut.GetByID(ctx, 999)
        if err == nil || !strings.Contains(err.Error(), "999") {
            t.Fatalf("expected error mentioning 999, got %v", err)
        }
    })

    t.Run("propagates repository error", func(t *testing.T) {
        boom := errors.New("boom")
        repo := &fakeRepository{findFunc: func(_ context.Context, _ int) (*Invoice, error) { return nil, boom }}
        sut := NewInvoiceService(repo)
        _, err := sut.GetByID(ctx, 1)
        if !errors.Is(err, boom) {
            t.Fatalf("expected boom error, got %v", err)
        }
    })
}

func TestInvoiceService_MarkAsPaid(t *testing.T) {
    ctx := context.Background()
    fixedTime := time.Date(2025, 1, 1, 12, 0, 0, 0, time.UTC)

    t.Run("transitions pending invoice to paid", func(t *testing.T) {
        invoice := &Invoice{ID: 1, Status: StatusPending}
        repo := &fakeRepository{findFunc: func(_ context.Context, _ int) (*Invoice, error) { return invoice, nil }}
        sut := NewInvoiceService(repo)
        sut.now = func() time.Time { return fixedTime }

        if err := sut.MarkAsPaid(ctx, 1); err != nil {
            t.Fatalf("unexpected error: %v", err)
        }
        if invoice.Status != StatusPaid {
            t.Errorf("status = %v, want %v", invoice.Status, StatusPaid)
        }
        if !invoice.PaidDate.Equal(fixedTime) {
            t.Errorf("paid date = %v, want %v", invoice.PaidDate, fixedTime)
        }
        if repo.updated != invoice {
            t.Errorf("repository was not updated with the invoice")
        }
    })

    t.Run("rejects already-paid invoice", func(t *testing.T) {
        invoice := &Invoice{ID: 1, Status: StatusPaid}
        repo := &fakeRepository{findFunc: func(_ context.Context, _ int) (*Invoice, error) { return invoice, nil }}
        sut := NewInvoiceService(repo)
        if err := sut.MarkAsPaid(ctx, 1); err == nil || !strings.Contains(err.Error(), "already paid") {
            t.Fatalf("expected already-paid error, got %v", err)
        }
        if repo.updated != nil {
            t.Errorf("update should not be called for already-paid invoice")
        }
    })

    t.Run("returns not-found when missing", func(t *testing.T) {
        repo := &fakeRepository{findFunc: func(_ context.Context, _ int) (*Invoice, error) { return nil, nil }}
        sut := NewInvoiceService(repo)
        if err := sut.MarkAsPaid(ctx, 999); err == nil || !strings.Contains(err.Error(), "999") {
            t.Fatalf("expected not-found error, got %v", err)
        }
    })
}

Sample Fix Cycle

When the implementer hits a compile or test-runner issue, the fixer agent diagnoses and resolves it.

Test output:

internal/billing/invoice_service_test.go:14:6: cannot use &fakeRepository{} (value of type *fakeRepository) as type InvoiceRepository in argument to NewInvoiceService:
        *fakeRepository does not implement InvoiceRepository (missing method Update)

Fixer diagnosis: The fake repository only implemented Find. Go enforces full interface implementation at compile time. Add the missing method.

Fix applied: Add the Update method to fakeRepository (shown in the test file above).

Rebuild + rerun: go test -count=1 ./internal/billing → SUCCESS

---

Another common cycle — wrong test selection regex:

Test output:

testing: warning: no tests to run

Fixer diagnosis: The agent used go test -run TestInvoiceService_CalculateTotal/single_item without ^...$ anchors. The Go test runner treats -run as a regex; the underscore makes the match too narrow.

Fix applied:

# Before — bare name without anchors, and an unquoted space would be parsed
# by the shell as two separate arguments
go test -run 'TestInvoiceService_CalculateTotal/single_item'

# After — anchor the subtest name, replace spaces with underscores
go test -run '^TestInvoiceService_CalculateTotal$/^single_item_with_10%_tax$' ./internal/billing

Rerun: SUCCESS

Sample Final Report

## Test Generation Report

**Project**: billing (Go)
**Strategy**: Direct (single source file in scope)

### Results
| Metric         | Value |
|----------------|-------|
| Tests created  | 11    |
| Tests passing  | 11    |
| Tests failing  | 0     |
| Files created  | 1     |

### Files Created
- `internal/billing/invoice_service_test.go` (3 top-level tests, 11 subtests including 5 table cases)

### Coverage
- InvoiceService.CalculateTotal — 3 happy + 2 error cases (table-driven)
- InvoiceService.GetByID — happy + missing + repo-error
- InvoiceService.MarkAsPaid — happy (with fixed clock) + already-paid + missing

### Build / Test Validation
- `go vet ./...`: ✅
- `go test -count=1 ./internal/billing`: ✅ PASS

### Next Steps
- Add fuzz test (`FuzzCalculateTotal`) if rounding correctness is critical
- Consider extracting a `Clock` interface if more time-dependent logic appears

Go Extension

Language-specific guidance for Go test generation.

Rule #1: Investigate the Repo First

Before writing any test or running any command, read:

1. Existing tests — find *_test.go files and copy their style (table-driven layout, helper usage, assertion library, build tags) 2. `go.mod` / `go.sum` — module path, Go version, dependencies (e.g. testify, gomock, mockery) 3. Build/CI scripts — Makefile, magefile.go, Taskfile.yml, .github/workflows/*.yml 4. `go.work` — if present, you are in a workspace; tests for a module must run from that module's directory or use -C (Go 1.20+)

Use whatever assertion style and test layout the repo already uses. Do not introduce testify if the repo uses the standard library only.

Toolchain Detection

Build Commands

go build ./... is the closest thing to a "does it compile" gate. It does not exercise test files — use go test -run=^$ to type-check tests as well.

Test Commands

• -run arguments are regular expressions anchored with ^...$; without anchors the pattern matches as a substring
• go test -count=1 is the canonical way to bypass the test result cache; never use a fake -count=2 or environment hacks
• -race significantly slows tests and requires CGO — only enable if the repo's CI does

Lint Command

Use the repo's lint script first (make lint, task lint). Otherwise detect from .golangci.yml/.golangci.yaml:

• .golangci.yml present → golangci-lint run ./...
• No config → gofmt -w . and go vet ./...
• goimports config / pre-commit hook → goimports -w path/to/file.go

Never disable existing linters in the test files you generate.

Project Layout and Imports

Go uses package paths derived from the module path in go.mod.

• Test files must end with _test.go — the toolchain ignores other names
• A package directory may contain both package foo and package foo_test test files simultaneously
• Helpers shared across tests in one package go in helpers_test.go — do not export them; put them in the _test package only if integration tests in another package need them
• Imports use the full module path: import "github.com/org/module/pkg" — copy the exact module path from go.mod

Test Function Signatures

Use table-driven tests when generating multiple cases for the same behavior — this is idiomatic Go and matches what most repos already use:

func TestAdd(t *testing.T) {
    tests := []struct {
        name string
        a, b int
        want int
    }{
        {"positives", 2, 3, 5},
        {"negatives", -1, -1, -2},
    }
    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            if got := Add(tt.a, tt.b); got != tt.want {
                t.Errorf("Add(%d,%d) = %d, want %d", tt.a, tt.b, got, tt.want)
            }
        })
    }
}

When iterating with t.Run over a loop variable on Go < 1.22, capture it with tt := tt to avoid closure-over-loop-variable bugs.

Common Errors

Mocking Rules

Go has no reflection-based mocking framework that's universally adopted. Pick what the repo already uses:

• Interfaces + hand-written fakes (most idiomatic) — define a small interface in the consumer package and pass a struct that implements it
• `gomock` / `mockgen` — if the repo has //go:generate mockgen ... directives or mocks/ directories, regenerate via go generate ./... rather than editing generated files
• `testify/mock` — used in many repos; instantiate with new(MockX) and chain .On("Method", ...).Return(...)
• `httptest` — for HTTP clients/servers; spin up httptest.NewServer instead of mocking http.Client

Always prefer dependency injection over global function patching. If a test needs more than 3 mocks, flag it as a design smell.

Concurrency and Cleanup

• Use t.Cleanup(func() { ... }) instead of deferring in test bodies — runs even if t.FailNow fires
• Use t.TempDir() for temp files — auto-cleaned at test end
• Use t.Context() (Go 1.24+) or pass an explicit context.Background() — never call real network or filesystem APIs without one in long-running tests

Dependency Installation (Last Resort)

Only install packages after investigation confirms they are missing:

go get github.com/stretchr/testify@latest
go mod tidy

Run go mod tidy after any go get to keep go.sum consistent. Never edit go.sum by hand.

Skip Coverage Tools

Do not configure or run coverage tools (-cover, -coverprofile, go tool cover). Coverage is measured separately by the evaluation harness.

Java Pipeline Examples

Concrete input→output examples for the test generation pipeline targeting a Java codebase using JUnit 5 + Mockito. These show what each pipeline phase produces for a small project.

Source Under Test

A simple InvoiceService in a Maven project using JUnit 5:

pom.xml
src/main/java/com/contoso/billing/
  InvoiceService.java
  Invoice.java                       (mutable POJO with status, taxRate, lineItems and setStatus / setPaidDate mutators)
  InvoiceStatus.java                 (enum: PENDING, PAID)
  InvoiceRepository.java             (interface)
src/test/java/com/contoso/billing/   (exists, empty)

// src/main/java/com/contoso/billing/InvoiceService.java
package com.contoso.billing;

import java.math.BigDecimal;
import java.math.RoundingMode;
import java.time.Clock;
import java.time.LocalDateTime;
import java.util.Optional;

public class InvoiceService {

    private final InvoiceRepository repository;
    private final Clock clock;

    public InvoiceService(InvoiceRepository repository) {
        this(repository, Clock.systemUTC());
    }

    public InvoiceService(InvoiceRepository repository, Clock clock) {
        this.repository = repository;
        this.clock = clock;
    }

    public BigDecimal calculateTotal(Invoice invoice) {
        if (invoice == null) {
            throw new IllegalArgumentException("invoice must not be null");
        }
        if (invoice.lineItems().isEmpty()) {
            throw new IllegalStateException("Invoice has no line items.");
        }
        BigDecimal subtotal = invoice.lineItems().stream()
            .map(li -> li.unitPrice().multiply(BigDecimal.valueOf(li.quantity())))
            .reduce(BigDecimal.ZERO, BigDecimal::add);
        BigDecimal tax = subtotal.multiply(invoice.taxRate());
        return subtotal.add(tax).setScale(2, RoundingMode.HALF_UP);
    }

    public Invoice getById(int id) {
        Optional<Invoice> invoice = repository.find(id);
        return invoice.orElseThrow(
            () -> new IllegalArgumentException("Invoice " + id + " not found."));
    }

    public void markAsPaid(int id) {
        Invoice invoice = repository.find(id)
            .orElseThrow(() -> new IllegalArgumentException("Invoice " + id + " not found."));
        if (invoice.status() == InvoiceStatus.PAID) {
            throw new IllegalStateException("Invoice is already paid.");
        }
        invoice.setStatus(InvoiceStatus.PAID);
        invoice.setPaidDate(LocalDateTime.now(clock));
        repository.update(invoice);
    }
}

Sample Research Output

What code-testing-researcher produces in .testagent/research.md:

# Test Generation Research

## Project Overview
- **Path**: /work/billing
- **Language**: Java 21 (`<maven.compiler.release>21</maven.compiler.release>`)
- **Build Tool**: Maven (wrapper `./mvnw` present)
- **Test Framework**: JUnit 5 (Jupiter 5.10) + Mockito 5.x (detected in pom.xml)
- **Assertion library**: built-in `Assertions` (no AssertJ/Hamcrest in deps)

## Coverage Baseline
- **Initial Line Coverage**: unknown
- **Strategy**: broad
- **Existing Test Count**: 0 tests across 0 files

## Build & Test Commands
- **Compile**: `./mvnw -q test-compile`
- **Test**: `./mvnw -q test`
- **Single class**: `./mvnw -q test -Dtest=InvoiceServiceTest`
- **Single method**: `./mvnw -q test -Dtest=InvoiceServiceTest#calculateTotal_validLineItems_returnsExpectedTotal`

## Project Structure
- Source: `src/main/java/com/contoso/billing/`
- Tests: `src/test/java/com/contoso/billing/` (exists, empty)

## Files to Test

### High Priority
| File | Classes/Methods | Testability | Notes |
|------|-----------------|-------------|-------|
| InvoiceService.java | calculateTotal, getById, markAsPaid | High | Repository dependency mockable via Mockito; clock injection available for time-dependent test |

## Testing Patterns
- No existing patterns; recommend JUnit 5 + Mockito with `@ExtendWith(MockitoExtension.class)`, `@Mock` / `@InjectMocks` fields, `@ParameterizedTest` + `@CsvSource` for table-driven `calculateTotal`, and `Clock.fixed(...)` for `markAsPaid` timestamp.

## Recommendations
- Test class lives in the same package (`com.contoso.billing`) for package-private access if needed
- Inject `Clock.fixed(...)` rather than mocking `LocalDateTime.now(...)` — the service already accepts a Clock

Sample Plan Output

# Test Implementation Plan

## Overview
Generate JUnit 5 + Mockito tests for InvoiceService, covering all three public
methods across happy path, edge case, and error scenarios. Single phase since
there is only one source file.

## Commands
- **Compile**: `./mvnw -q test-compile`
- **Test**: `./mvnw -q test -Dtest=InvoiceServiceTest`

## Phase 1: InvoiceService

### Files to Test

#### 1. InvoiceService.java
- **Source**: `src/main/java/com/contoso/billing/InvoiceService.java`
- **Test File**: `src/test/java/com/contoso/billing/InvoiceServiceTest.java`

**Methods to Test**:
1. `calculateTotal` — pure logic (parameterized)
   - Happy paths: single item w/ tax, multi-quantity zero tax, rounding-half-up
   - Error cases: null invoice → IllegalArgumentException; empty line items → IllegalStateException
2. `getById` — happy + missing
3. `markAsPaid` — happy (verify status + paid date via fixed clock + verify update) + already-paid + missing

Sample Generated Test File

// src/test/java/com/contoso/billing/InvoiceServiceTest.java
package com.contoso.billing;

import java.math.BigDecimal;
import java.time.Clock;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneOffset;
import java.util.List;
import java.util.Optional;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;

import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;

@ExtendWith(MockitoExtension.class)
class InvoiceServiceTest {

    @Mock
    InvoiceRepository repository;

    @InjectMocks
    InvoiceService sut;

    // --- calculateTotal ---

    @ParameterizedTest(name = "qty={0} unitPrice={1} taxRate={2} -> {3}")
    @CsvSource({
        "1, 100.00, 0.10, 110.00",
        "3, 25.00,  0.00,  75.00",
        "2,  9.99,  0.07,  21.38"
    })
    void calculateTotal_validLineItems_returnsExpectedTotal(
        int quantity, BigDecimal unitPrice, BigDecimal taxRate, BigDecimal expected
    ) {
        Invoice invoice = new Invoice(1, InvoiceStatus.PENDING, taxRate,
            List.of(new LineItem(quantity, unitPrice)));

        BigDecimal total = sut.calculateTotal(invoice);

        assertEquals(0, total.compareTo(expected),
            () -> "expected " + expected + " but got " + total);
    }

    @Test
    @DisplayName("null invoice throws IllegalArgumentException")
    void calculateTotal_nullInvoice_throws() {
        assertThrows(IllegalArgumentException.class, () -> sut.calculateTotal(null));
    }

    @Test
    void calculateTotal_emptyLineItems_throws() {
        Invoice invoice = new Invoice(1, InvoiceStatus.PENDING, BigDecimal.ZERO, List.of());

        IllegalStateException ex = assertThrows(IllegalStateException.class,
            () -> sut.calculateTotal(invoice));
        assertEquals("Invoice has no line items.", ex.getMessage());
    }

    // --- getById ---

    @Test
    void getById_existingId_returnsInvoice() {
        Invoice expected = new Invoice(42, InvoiceStatus.PENDING, BigDecimal.ZERO, List.of());
        when(repository.find(42)).thenReturn(Optional.of(expected));

        assertSame(expected, sut.getById(42));
    }

    @Test
    void getById_missingId_throws() {
        when(repository.find(999)).thenReturn(Optional.empty());

        IllegalArgumentException ex = assertThrows(IllegalArgumentException.class,
            () -> sut.getById(999));
        assertEquals("Invoice 999 not found.", ex.getMessage());
    }

    // --- markAsPaid (uses an injected fixed Clock instead of @InjectMocks) ---

    @Test
    void markAsPaid_pendingInvoice_transitionsToPaidAndPersists() {
        Clock fixed = Clock.fixed(Instant.parse("2025-01-01T12:00:00Z"), ZoneOffset.UTC);
        InvoiceService service = new InvoiceService(repository, fixed);
        Invoice invoice = new Invoice(1, InvoiceStatus.PENDING, BigDecimal.ZERO, List.of());
        when(repository.find(1)).thenReturn(Optional.of(invoice));

        service.markAsPaid(1);

        assertEquals(InvoiceStatus.PAID, invoice.status());
        assertEquals(LocalDateTime.ofInstant(fixed.instant(), ZoneOffset.UTC), invoice.paidDate());
        verify(repository).update(invoice);
    }

    @Test
    void markAsPaid_alreadyPaid_throwsAndDoesNotUpdate() {
        Invoice invoice = new Invoice(1, InvoiceStatus.PAID, BigDecimal.ZERO, List.of());
        when(repository.find(1)).thenReturn(Optional.of(invoice));

        assertThrows(IllegalStateException.class, () -> sut.markAsPaid(1));
        verify(repository, never()).update(any());
    }

    @Test
    void markAsPaid_missingId_throws() {
        when(repository.find(999)).thenReturn(Optional.empty());

        assertThrows(IllegalArgumentException.class, () -> sut.markAsPaid(999));
    }
}

Sample Fix Cycle

When the implementer hits a compile or runtime error, the fixer agent diagnoses and resolves it.

Test output:

[ERROR] No tests found for given includes: [com.contoso.billing.InvoiceServiceTest]

Fixer diagnosis: Surefire only includes **/*Test.class (default). The class is InvoiceServiceTest (correct) but it was created under src/test/java/com/contoso/billing/ with no package declaration. Maven compiles it into the default package, so -Dtest=com.contoso.billing.InvoiceServiceTest doesn't match.

Fix applied: Add package com.contoso.billing; at the top of the test file so it lands in the expected package.

Rebuild + rerun: ./mvnw -q test -Dtest=InvoiceServiceTest → SUCCESS

---

Another common cycle — wrong Mockito setup:

Test output:

org.mockito.exceptions.misusing.UnnecessaryStubbingException:
Unnecessary stubbings detected.
  1. -> at InvoiceServiceTest.calculateTotal_nullInvoice_throws(InvoiceServiceTest.java:55)

Fixer diagnosis: @MockitoExtension runs in strict mode by default — stubbed calls (when(repository.find(...)).thenReturn(...)) must be used. The test stubbed repository in a @BeforeEach for every test, but calculateTotal_nullInvoice_throws never touches the repository.

Fix applied: Move stubs into the tests that actually need them (as shown in the generated file above), rather than a single shared @BeforeEach.

Rebuild + rerun: SUCCESS

Sample Final Report

## Test Generation Report

**Project**: billing (Java / Maven)
**Strategy**: Direct (single source file in scope)

### Results
| Metric         | Value |
|----------------|-------|
| Tests created  | 8     |
| Tests passing  | 8     |
| Tests failing  | 0     |
| Files created  | 1     |

### Files Created
- `src/test/java/com/contoso/billing/InvoiceServiceTest.java` (8 tests, 3 parameterized cases via @CsvSource)

### Coverage
- InvoiceService.calculateTotal — 3 happy path, 2 error cases
- InvoiceService.getById — happy + missing
- InvoiceService.markAsPaid — happy (fixed Clock) + already-paid + missing

### Build / Test Validation
- `./mvnw -q test-compile`: ✅
- `./mvnw -q test`: ✅ Tests run: 8, Failures: 0, Errors: 0

### Next Steps
- Add AssertJ if the team standardises on it (more expressive assertions)
- Consider Testcontainers for true repository integration tests

Java Extension

Language-specific guidance for Java test generation.

Rule #1: Investigate the Repo First

Before writing any test or running any command, read:

1. Existing tests — find *Test.java / *Tests.java / *IT.java (integration) files and copy their style (JUnit version, assertion library, mock library, lifecycle methods) 2. Build file — pom.xml (Maven), build.gradle / build.gradle.kts (Gradle), BUILD / BUILD.bazel (Bazel) 3. Java version — <maven.compiler.release>, sourceCompatibility, or toolchains block 4. Wrapper scripts — always prefer ./mvnw or ./gradlew over a system-installed Maven/Gradle so you match the project's pinned version

Use whatever framework the repo already uses (JUnit 4, JUnit 5/Jupiter, TestNG). Do not migrate to a different framework as a side effect of writing tests.

Build Tool Detection

If both pom.xml and build.gradle exist, pick the one used by CI.

Build Commands

• Use -q (Maven) / --console=plain (Gradle) to reduce output noise
• For Gradle, prefer --no-daemon only in CI; locally the daemon makes incremental builds far faster

Test Commands

• Surefire runs unit tests (*Test.java); Failsafe runs integration tests (*IT.java) — do not put long integration tests under Surefire
• Gradle's --tests accepts wildcards: --tests "*MyMethod*"
• Use --rerun-tasks (Gradle) or -DforkCount=... (Surefire) only when troubleshooting cache issues

Lint Command

Use the repo's existing lint task first. Otherwise check for:

• Checkstyle (checkstyle.xml, <plugin>checkstyle</plugin>) → ./mvnw checkstyle:check or ./gradlew checkstyleMain
• Spotless (spotless block / plugin) → ./mvnw spotless:apply or ./gradlew spotlessApply
• ErrorProne / NullAway → integrated into compilation; run a normal build
• google-java-format / palantir-java-format → use the repo's configured formatter

Never disable existing checks in the test files you generate.

Project Layout and Imports

Maven/Gradle conventional layout:

src/
├── main/java/com/example/foo/Bar.java
├── main/resources/
├── test/java/com/example/foo/BarTest.java
└── test/resources/

• Test classes must mirror the production class's package to access package-private members
• Avoid wildcard imports unless the repo already uses them — match the explicit imports shown in the templates below
• For JUnit 5: import org.junit.jupiter.api.Test (and other annotations as needed) and org.junit.jupiter.api.Assertions.assertEquals etc. as static imports
• For JUnit 4: import org.junit.Test, org.junit.Before, etc., and org.junit.Assert.assertEquals etc. as static imports

Test Framework Detection

Argument order matters: JUnit/AssertJ use (expected, actual); TestNG uses (actual, expected). Reversing them produces confusing failure messages.

JUnit 5 Template

package com.example.foo;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

class CalculatorTest {

    @Test
    @DisplayName("add returns sum of two positive numbers")
    void add_positiveNumbers_returnsSum() {
        Calculator sut = new Calculator();
        assertEquals(5, sut.add(2, 3));
    }

    @ParameterizedTest
    @CsvSource({
        "2, 3, 5",
        "-1, 1, 0"
    })
    void add_validInputs_returnsSum(int a, int b, int expected) {
        assertEquals(expected, new Calculator().add(a, b));
    }

    @Test
    void divide_byZero_throws() {
        Calculator sut = new Calculator();
        assertThrows(ArithmeticException.class, () -> sut.divide(1, 0));
    }
}

Common Errors

Mocking Rules

• Use whatever the repo already uses: Mockito (most common), EasyMock, JMockit, or hand-written fakes
• For JUnit 5 + Mockito, use @ExtendWith(MockitoExtension.class) with @Mock / @InjectMocks fields
• For JUnit 4 + Mockito, use @RunWith(MockitoJUnitRunner.class) or MockitoAnnotations.openMocks(this) in @Before
• Use when(mock.method(...)).thenReturn(...) for stubs and verify(mock).method(...) for interactions
• Use ArgumentCaptor to assert on complex argument values rather than over-specifying matchers
• Prefer constructor injection so production code stays testable without @InjectMocks
• If a test needs more than 3 mocks, flag it as a design smell

Spring Boot

If the repo uses Spring Boot:

• @SpringBootTest loads the full context — slow; use only when needed
• Slice tests are faster: @WebMvcTest, @DataJpaTest, @JsonTest
• Use @MockBean (Spring) only inside Spring tests; in plain unit tests use @Mock
• Use @Testcontainers for real-DB integration tests if the repo already has it on the classpath

Dependency Installation (Last Resort)

Only add dependencies after investigation confirms they are missing.

Maven (pom.xml):

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter</artifactId>
    <version>5.10.2</version>
    <scope>test</scope>
</dependency>

Gradle (build.gradle.kts):

testImplementation("org.junit.jupiter:junit-jupiter:5.10.2")
testRuntimeOnly("org.junit.platform:junit-platform-launcher")

If the repo uses BOMs (<dependencyManagement> or Gradle platforms), reuse them — don't pin a different version than the BOM publishes.

Skip Coverage Tools

Do not configure or run coverage tools (JaCoCo, Cobertura, OpenClover). Coverage is measured separately by the evaluation harness.

Kotlin Extension

Language-specific guidance for Kotlin test generation. For pure-Java codebases, use java.md instead.

Rule #1: Investigate the Repo First

Before writing any test or running any command, read:

1. Existing tests — find files in src/test/kotlin/, src/commonTest/kotlin/, src/jvmTest/kotlin/, etc., and copy their style (framework, assertion library, mock library, coroutine helpers) 2. Build file — build.gradle.kts / build.gradle — note Kotlin version, plugins (kotlin("jvm"), kotlin("multiplatform"), kotlin("android")), and dependencies { testImplementation(...) } 3. `gradle/libs.versions.toml` — the version catalog if the repo uses one; reference aliases instead of hard-coded versions 4. Wrapper script — always invoke ./gradlew (Unix) or .\gradlew.bat (Windows), never a system-installed Gradle 5. Multiplatform layout — src/<sourceSet>/kotlin/ indicates KMP; tests live in matching *Test source sets (commonTest, jvmTest, nativeTest)

Use whatever framework the repo already uses (JUnit Jupiter, JUnit 4, Kotest, kotlin.test). Do not switch.

Project Type Detection

For Android, see also platform-specific test types: src/test/ for unit tests on the JVM, src/androidTest/ for instrumented tests on a device/emulator. They use different runners and gradle tasks.

Build Commands

• Use --console=plain to suppress Gradle's animated output
• Use --build-cache (often default in CI) to reuse outputs
• For Android: ./gradlew assembleDebug (build APK) and ./gradlew testDebugUnitTest (run unit tests)

Test Commands

• --tests accepts wildcards: --tests "*Widget*". Method names with spaces or backticks must be quoted: --tests "com.example.WidgetTest.creates a widget"
• Use --rerun-tasks only when troubleshooting cache issues
• For Kotest, the runner is registered with JUnit Platform — the standard ./gradlew test and --tests flags work the same way

Lint Command

Use the repo's lint tooling first:

• ./gradlew ktlintCheck (autoformat: ./gradlew ktlintFormat) when ktlint is configured
• ./gradlew detekt when detekt is configured
• ./gradlew spotlessCheck / spotlessApply for the Spotless plugin
• Android Studio's IDE inspections; ./gradlew lint (Android-only) for the Android Lint task

Project Layout

src/
├── main/kotlin/com/example/foo/Bar.kt
├── main/resources/
├── test/kotlin/com/example/foo/BarTest.kt   # mirrors production package
└── test/resources/

KMP layout:

src/
├── commonMain/kotlin/...        # shared
├── commonTest/kotlin/...        # shared tests using kotlin.test
├── jvmMain/kotlin/...
├── jvmTest/kotlin/...
├── jsMain/kotlin/...
└── jsTest/kotlin/...

• Test classes mirror the production class's package so they can access internal members (Kotlin's internal is module-scoped — within the same Gradle module, including the test source set)
• For KMP common tests, you can only import from kotlin.test and other multiplatform-aware libraries (e.g. kotlinx.coroutines.test, Kotest multiplatform, MockK on JVM only)

Test Framework Detection

For Kotest, stick to the spec style the repo already uses — mixing styles is confusing.

Test Templates

JUnit 5

package com.example.foo

import org.junit.jupiter.api.DisplayName
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.assertThrows
import kotlin.test.assertEquals

class CalculatorTest {

    @Test
    @DisplayName("add returns sum of two positive numbers")
    fun `add returns sum of two positives`() {
        val sut = Calculator()
        assertEquals(5, sut.add(2, 3))
    }

    @Test
    fun `divide by zero throws`() {
        val sut = Calculator()
        assertThrows<ArithmeticException> { sut.divide(1, 0) }
    }
}

Backticked method names (` like this `) are idiomatic for Kotlin tests because they read better in failure messages.

Kotest (StringSpec)

package com.example.foo

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe
import io.kotest.assertions.throwables.shouldThrow

class CalculatorSpec : StringSpec({
    "add returns sum of two positive numbers" {
        Calculator().add(2, 3) shouldBe 5
    }

    "divide by zero throws" {
        shouldThrow<ArithmeticException> { Calculator().divide(1, 0) }
    }
})

Coroutines

• Use kotlinx-coroutines-test when it's already on the classpath; otherwise add it as a testImplementation only after confirming it is missing (see Dependency Installation)
• Use runTest { ... } (replaces the older runBlockingTest) for suspend test bodies
• For virtual time advance, use a TestDispatcher built from testScheduler — e.g. StandardTestDispatcher(testScheduler) or UnconfinedTestDispatcher(testScheduler) — rather than calling delay and waiting in real time
• Inject a CoroutineDispatcher into production code instead of using Dispatchers.Main/IO directly — then swap it in tests via Dispatchers.setMain(testDispatcher)

@Test
fun `loads data eventually`() = runTest {
    val repo = FakeRepo()
    val dispatcher = StandardTestDispatcher(testScheduler)
    val sut = Loader(repo, dispatcher)
    sut.start()
    advanceUntilIdle()
    assertEquals(LoadState.Done, sut.state.value)
}

Common Errors

Mocking Rules

• MockK is the de-facto standard for Kotlin (final classes, coroutine support): every { mock.foo() } returns 1, coEvery { mock.suspendFn() } returns 1, verify { mock.foo() }, coVerify { ... }
• Mockito works on Kotlin too with mockito-kotlin extensions, but Kotlin classes are final by default — use Mockito's inline mock maker (default in Mockito 5+; the mockito-inline artifact for Mockito 3.x/4.x). mockito-subclass cannot mock final classes
• Avoid mockkStatic/mockkObject for production code you control — refactor to a wrapper instead
• Prefer constructor injection so you don't need framework annotations (@InjectMocks) at all
• If a test needs more than 3 mocks, flag it as a design smell

Android Specifics

• Robolectric tests live under src/test/ and emulate the Android framework on the JVM — fast but imperfect
• Instrumented tests live under src/androidTest/, require a connected device/emulator, and are slow — use sparingly
• Compose UI tests use createComposeRule() and composeTestRule.onNodeWithText(...).performClick() — match the existing test setup if Compose is in the project
• Hilt: use @HiltAndroidTest and HiltAndroidRule for instrumented tests; for unit tests pass fakes directly to ViewModels

Dependency Installation (Last Resort)

Only add dependencies after investigation confirms they are missing.

build.gradle.kts:

dependencies {
    testImplementation("org.junit.jupiter:junit-jupiter:5.10.2")
    testImplementation("io.mockk:mockk:1.13.10")
    testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.8.0")
}

tasks.test {
    useJUnitPlatform()
}

If the repo uses a version catalog, add to gradle/libs.versions.toml and reference via libs.junit.jupiter etc. Match the major versions already in use.

Skip Coverage Tools

Do not configure or run coverage tools (JaCoCo, Kover). Coverage is measured separately by the evaluation harness.

PowerShell Extension

Language-specific guidance for PowerShell test generation using Pester v5.

Rule #0: Confirm the Test Target

If the prompt does not name a specific file (e.g. "test the repository", "cover one core module", "comprehensive suite"), do not assume the largest or top-level upstream code is the intended target. In real workflows the user usually wants to test code they have just added, and large upstream repos contain hundreds of scripts already covered by existing *.Tests.ps1 files.

Run these read-only discovery commands first — they are the deliberate exception to Rule #1's "before writing any test or running any command" rule, and their output is the ground truth Rule #1's reading is meant to interpret. Do not write or execute any tests until Rule #0 and Rule #1 are both complete.

Prefer targets that match all of:

1. Untracked or recently added (git status / git log --diff-filter=A). 2. Small and pure (a few hundred lines, no external state, no Invoke-WebRequest/registry/filesystem side effects). 3. Located under a conventional source root (tools/, src/, Public/, Private/, or the module root next to a .psd1). 4. Have no existing matching *.Tests.ps1 file.

If a .psd1 manifest's RootModule (or ModuleToProcess) points at a specific .psm1, that module is almost certainly the target — start there.

Test Placement Contract

Pester only discovers tests under the path passed to Invoke-Pester -Path (or the current directory when no path is given). Verification harnesses (CI, msbench, coverage tools) typically scope discovery to a single directory such as tools/ or tests/. Place every test file there, matching the existing convention in the repo:

A *.Tests.ps1 file placed outside the discovery root will be invisible to both Invoke-Pester and the harness.

First-Test Sanity Loop

After writing the first *.Tests.ps1 file — before writing any others:

1. Run Invoke-Pester -Path <dir> -PassThru and confirm the TotalCount is > 0. If it is 0, Pester is not discovering your file; fix the location, filename, or Describe/It structure before continuing. 2. Run the test (Invoke-Pester -Path <your.Tests.ps1> -Output Detailed); fix Import-Module / dot-source / BeforeAll errors before adding more tests. 3. Only then expand to cover the remaining functions.

This catches placement and discovery mistakes on turn 1 instead of after dozens of failed-test iterations.

Harness Discovery Check

Before reporting success, run the harness-equivalent discovery command from the repo root and confirm the test count went up by at least the number of tests you generated. CI/msbench/coverage harnesses do not know which directory you targeted with -Path; they invoke Pester from the repo root with default discovery, so a test that passes via Invoke-Pester -Path ./tools/Foo.Tests.ps1 is still worthless if Invoke-Pester from the repo root does not enumerate it.

# From repo root — mirrors what a generic harness sees
$result = Invoke-Pester -Configuration @{ Run = @{ PassThru = $true; SkipRun = $true } }
"$($result.TotalCount) tests discovered"

If the count did not increase, your *.Tests.ps1 file is outside the harness discovery root. Move it to the convention the repo's existing tests use (or, if there are no existing tests, prefer the repo root's tests/, Tests/, tst/, test/, or co-locate next to the source). Do not report success until the harness-equivalent command sees your new tests.

Rule #1: Investigate the Repo First

Before writing any test or running any command, read:

1. Existing tests — find *.Tests.ps1 files and copy their style (structure, assertions, mock approach, import method) 2. Module structure — look for .psd1 (manifest), .psm1 (root module), Public//Private/ organization 3. Build/test scripts — check for build.ps1, Invoke-Build (*.build.ps1), psake, or CI scripts 4. Shell target — check .psd1 for PowerShellVersion/CompatiblePSEditions, CI matrix for pwsh vs powershell.exe

Use the repo's existing test conventions. Only add Pester if the repo has no tests at all.

Build Commands

PowerShell is interpreted — no build step. If the repo has a build script, use it. Otherwise validate with:

• Module loads: Import-Module ./MyModule.psd1 -Force -ErrorAction Stop
• Script analyzer: Invoke-ScriptAnalyzer -Path ./src -Recurse (if PSScriptAnalyzer is available)
• Lint: Invoke-ScriptAnalyzer -Path path/to/file.ps1 -Fix

Test Commands

• Prefer the repo's build/test script over raw Invoke-Pester
• Use -Output Detailed during fix cycles, -Output Minimal for final validation

Project Layout and Imports

• All imports go in `BeforeAll` — never at script top level
• Use `$PSScriptRoot` or `$PSCommandPath` — never $MyInvocation.MyCommand.Path (returns empty in BeforeAll)
• Use -Force on Import-Module to pick up changes between runs

Test File Naming

• Files: *.Tests.ps1 — match existing convention (co-located vs Tests/ directory)

Pester v5 Discovery vs Run (Critical)

Pester v5 runs in two phases: Discovery (collects test metadata) then Run (executes tests). This is the #1 source of agent errors.

Rules:

• All setup code goes in BeforeAll or BeforeEach — never at script top level or loose inside Describe/Context
• Code directly inside Describe/Context (but outside It/Before*/After*) runs during Discovery — do not put setup, imports, or variable assignments there
• Data for -ForEach / -TestCases must be set in BeforeDiscovery, not BeforeAll (BeforeAll runs after discovery)
• -Skip:$condition evaluates at Discovery time — conditions from BeforeAll will be $null
• Use foreach loops for dynamic test generation only with BeforeDiscovery data
• Use TestDrive: for file-based tests instead of touching repo files — Pester cleans it up automatically

Common Errors

Mocking Rules

• Place mocks in BeforeAll (shared) or BeforeEach (reset per test)
• Mock where the command is called from — use -ModuleName to mock inside a module's scope
• Use -ParameterFilter for selective mocking (no param() block needed in v5)
• Verify calls with Should -Invoke — default scope inside It counts only that test's calls
• Use InModuleScope sparingly and as narrowly as possible — prefer Mock -ModuleName for testing via public API
• Inside mock bodies, use $PesterBoundParameters not $PSBoundParameters
• If a test needs more than 3 mocks, flag it as a design smell

Non-Obvious Assertions

Most Should operators are self-explanatory. These are the ones agents get wrong:

• Should -Throw requires a scriptblock: { risky-op } | Should -Throw — not a direct call
• Should -Contain is for collections — use Should -Be for scalar equality
• Should -HaveParameter validates cmdlet signatures: Get-Command X | Should -HaveParameter 'Name' -Mandatory
• Should -Invoke verifies mock calls: Should -Invoke Get-Item -Times 1 -Exactly

Cross-Platform

• Prefer pwsh (PowerShell 7+) unless the repo explicitly targets Windows PowerShell 5.1
• Use Join-Path for paths — never string concatenation with \
• Linux/macOS file systems are case-sensitive — match exact casing in imports and paths
• Windows ships Pester 3.4.0 — if v5 is needed: Install-Module Pester -Force -SkipPublisherCheck
• Check $PSVersionTable.PSEdition to detect Core vs Desktop

Skip Coverage Tools

Do not configure or run coverage tools (Pester CodeCoverage, JaCoCo export). Coverage is measured separately by the evaluation harness.

Python Pipeline Examples

Concrete input→output examples for the test generation pipeline targeting a Python codebase using pytest. These show what each pipeline phase produces for a small project.

Source Under Test

A simple InvoiceService in a Python package using pytest:

src/
  contoso_billing/
    __init__.py
    invoice_service.py
    invoice.py
    invoice_repository.py
tests/
  __init__.py
  conftest.py             (empty, just marks tests/ as a package root)
pyproject.toml

# src/contoso_billing/invoice_service.py
from decimal import Decimal, ROUND_HALF_UP
from .invoice import Invoice, InvoiceStatus
from .invoice_repository import InvoiceRepository


class InvoiceService:
    def __init__(self, repository: InvoiceRepository) -> None:
        self._repository = repository

    def calculate_total(self, invoice: Invoice) -> Decimal:
        if invoice is None:
            raise ValueError("invoice must not be None")
        if not invoice.line_items:
            raise ValueError("Invoice has no line items.")

        subtotal = sum(
            (li.quantity * li.unit_price for li in invoice.line_items),
            start=Decimal("0"),
        )
        tax = subtotal * invoice.tax_rate
        return (subtotal + tax).quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)

    def get_by_id(self, invoice_id: int) -> Invoice:
        invoice = self._repository.find(invoice_id)
        if invoice is None:
            raise KeyError(f"Invoice {invoice_id} not found.")
        return invoice

    def mark_as_paid(self, invoice_id: int) -> None:
        invoice = self._repository.find(invoice_id)
        if invoice is None:
            raise KeyError(f"Invoice {invoice_id} not found.")
        if invoice.status == InvoiceStatus.PAID:
            raise ValueError("Invoice is already paid.")
        invoice.status = InvoiceStatus.PAID
        invoice.paid_date = _utcnow()
        self._repository.update(invoice)


def _utcnow():
    from datetime import datetime, timezone
    return datetime.now(timezone.utc)

Sample Research Output

What code-testing-researcher produces in .testagent/research.md:

# Test Generation Research

## Project Overview
- **Path**: /work/contoso-billing
- **Language**: Python 3.11
- **Framework**: pure library (no Flask/Django)
- **Test Framework**: pytest 8.x (declared in pyproject.toml [project.optional-dependencies].test)
- **Package Layout**: `src/` layout — production package imports as `contoso_billing`

## Coverage Baseline
- **Initial Line Coverage**: unknown
- **Strategy**: broad
- **Existing Test Count**: 0 tests across 0 files

## Build & Test Commands
- **Install (editable)**: `python -m pip install -e ".[test]"`
- **Build/Type-check**: none configured
- **Test**: `python -m pytest`
- **Lint**: none configured

## Project Structure
- Source: `src/contoso_billing/`
- Tests: `tests/` (exists, empty besides `conftest.py`)

## Files to Test

### High Priority
| File | Classes/Functions | Testability | Notes |
|------|-------------------|-------------|-------|
| src/contoso_billing/invoice_service.py | InvoiceService: calculate_total, get_by_id, mark_as_paid | High | Core business logic, repository dependency needs mocking |

### Low Priority / Skip
| File | Reason |
|------|--------|
| src/contoso_billing/invoice.py | Dataclass, no logic |
| src/contoso_billing/invoice_repository.py | Interface/protocol, no implementation |

## Existing Tests
- No existing tests found

## Testing Patterns
- No existing patterns; recommend pytest function-style tests in `tests/test_invoice_service.py`, `unittest.mock.Mock(spec=InvoiceRepository)` for repository fakes, and `@pytest.mark.parametrize` for table-driven cases.

## Recommendations
- Start with `calculate_total` (pure logic, easy to parametrize)
- Then `get_by_id` and `mark_as_paid` (require mocking the repository)
- Use `unittest.mock.patch("contoso_billing.invoice_service._utcnow")` to control the timestamp in `mark_as_paid`

Sample Plan Output

What code-testing-planner produces in .testagent/plan.md:

# Test Implementation Plan

## Overview
Generate pytest tests for the Contoso Billing InvoiceService, covering all three
public methods across happy path, edge case, and error scenarios. Single phase
since there is only one source file.

## Commands
- **Install**: `python -m pip install -e ".[test]"`
- **Test**: `python -m pytest tests/test_invoice_service.py -q`
- **Test (file-scoped during dev)**: `python -m pytest tests/test_invoice_service.py::test_calculate_total_valid_line_items_returns_expected_total -q`

## Phase Summary
| Phase | Focus | Files | Est. Tests |
|-------|-------|-------|------------|
| 1 | InvoiceService | 1 | 9-12 |

---

## Phase 1: InvoiceService

### Overview
Cover all public methods of InvoiceService. `calculate_total` is pure logic tested
with `@pytest.mark.parametrize`. The async-looking methods are synchronous but
require a mocked InvoiceRepository.

### Files to Test

#### 1. invoice_service.py
- **Source**: `src/contoso_billing/invoice_service.py`
- **Test File**: `tests/test_invoice_service.py`

**Methods to Test**:
1. `calculate_total` — Pure calculation logic
   - Happy path: single line item returns quantity × price + tax
   - Happy path: multiple line items summed correctly
   - Edge case: zero tax rate returns subtotal only
   - Error case: None invoice raises ValueError
   - Error case: empty line items raises ValueError

2. `get_by_id` — Repository lookup
   - Happy path: existing ID returns invoice
   - Error case: missing ID raises KeyError

3. `mark_as_paid` — State transition
   - Happy path: pending invoice transitions to PAID with `paid_date` set
   - Error case: already-paid raises ValueError
   - Error case: missing ID raises KeyError

### Success Criteria
- [ ] Test file created at `tests/test_invoice_service.py`
- [ ] `python -m pytest` reports all tests passed
- [ ] No real network/IO; repository is mocked with `Mock(spec=InvoiceRepository)`

Sample Generated Test File

What code-testing-implementer produces:

# tests/test_invoice_service.py
from datetime import datetime, timezone
from decimal import Decimal
from unittest.mock import Mock, patch

import pytest

from contoso_billing.invoice import Invoice, InvoiceStatus, LineItem
from contoso_billing.invoice_repository import InvoiceRepository
from contoso_billing.invoice_service import InvoiceService


@pytest.fixture
def repository() -> Mock:
    return Mock(spec=InvoiceRepository)


@pytest.fixture
def sut(repository: Mock) -> InvoiceService:
    return InvoiceService(repository)


# --- calculate_total ---

@pytest.mark.parametrize(
    "quantity, unit_price, tax_rate, expected",
    [
        (1, "100.00", "0.10", "110.00"),
        (3, "25.00", "0.00", "75.00"),
        (2, "9.99", "0.07", "21.38"),
    ],
    ids=["single-item-10pct-tax", "multi-quantity-zero-tax", "rounds-half-up"],
)
def test_calculate_total_valid_line_items_returns_expected_total(
    sut: InvoiceService, quantity: int, unit_price: str, tax_rate: str, expected: str
) -> None:
    invoice = Invoice(
        tax_rate=Decimal(tax_rate),
        line_items=[LineItem(quantity=quantity, unit_price=Decimal(unit_price))],
    )

    total = sut.calculate_total(invoice)

    assert total == Decimal(expected)


def test_calculate_total_none_invoice_raises_value_error(sut: InvoiceService) -> None:
    with pytest.raises(ValueError, match="invoice must not be None"):
        sut.calculate_total(None)


def test_calculate_total_empty_line_items_raises_value_error(sut: InvoiceService) -> None:
    invoice = Invoice(tax_rate=Decimal("0"), line_items=[])

    with pytest.raises(ValueError, match="no line items"):
        sut.calculate_total(invoice)


# --- get_by_id ---

def test_get_by_id_existing_id_returns_invoice(
    sut: InvoiceService, repository: Mock
) -> None:
    expected = Invoice(id=42, tax_rate=Decimal("0"), line_items=[])
    repository.find.return_value = expected

    result = sut.get_by_id(42)

    assert result is expected
    repository.find.assert_called_once_with(42)


def test_get_by_id_missing_id_raises_key_error(
    sut: InvoiceService, repository: Mock
) -> None:
    repository.find.return_value = None

    with pytest.raises(KeyError, match="999"):
        sut.get_by_id(999)


# --- mark_as_paid ---

def test_mark_as_paid_pending_invoice_sets_status_and_date(
    sut: InvoiceService, repository: Mock
) -> None:
    invoice = Invoice(id=1, status=InvoiceStatus.PENDING, tax_rate=Decimal("0"), line_items=[])
    repository.find.return_value = invoice
    fixed_now = datetime(2025, 1, 1, 12, 0, tzinfo=timezone.utc)

    with patch("contoso_billing.invoice_service._utcnow", return_value=fixed_now):
        sut.mark_as_paid(1)

    assert invoice.status == InvoiceStatus.PAID
    assert invoice.paid_date == fixed_now
    repository.update.assert_called_once_with(invoice)


def test_mark_as_paid_already_paid_raises_value_error(
    sut: InvoiceService, repository: Mock
) -> None:
    invoice = Invoice(id=1, status=InvoiceStatus.PAID, tax_rate=Decimal("0"), line_items=[])
    repository.find.return_value = invoice

    with pytest.raises(ValueError, match="already paid"):
        sut.mark_as_paid(1)

    repository.update.assert_not_called()


def test_mark_as_paid_missing_id_raises_key_error(
    sut: InvoiceService, repository: Mock
) -> None:
    repository.find.return_value = None

    with pytest.raises(KeyError, match="999"):
        sut.mark_as_paid(999)

Sample Fix Cycle

When the implementer encounters an import or attribute error, the fixer agent diagnoses and resolves it.

Test output:

ModuleNotFoundError: No module named 'contoso_billing'

Fixer diagnosis: The package is not installed in editable mode, so the src/ layout's package is not on sys.path.

Fix applied:

python -m pip install -e ".[test]"

Rerun: python -m pytest tests/test_invoice_service.py -q → SUCCESS

---

Another common cycle — patch target wrong:

Test output:

AttributeError: <module 'datetime'> does not have the attribute '_utcnow'

Fixer diagnosis: The test patched datetime._utcnow but the production code defines its own _utcnow helper inside contoso_billing.invoice_service. Patches must target the lookup site, not the definition site.

Fix applied:

# Before (wrong)
with patch("datetime._utcnow", return_value=fixed_now):

# After (fixed) — patch where the name is looked up
with patch("contoso_billing.invoice_service._utcnow", return_value=fixed_now):

Rerun: SUCCESS

---

Another common cycle — Mock without spec:

Test output:

AttributeError: Mock object has no attribute 'find_by_id'

(but the actual repository method is find, not find_by_id)

Fixer diagnosis: Mock() happily creates any attribute on access, so a typo in the test went undetected until the production code called repository.find(...). Using Mock(spec=InvoiceRepository) would have failed at setup time.

Fix applied:

# Before
repository = Mock()
repository.find_by_id.return_value = expected   # typo, silently accepted

# After
repository = Mock(spec=InvoiceRepository)
repository.find.return_value = expected         # typos now raise AttributeError

Rerun: SUCCESS

Sample Final Report

What code-testing-generator produces at Step 9:

## Test Generation Report

**Project**: contoso-billing
**Strategy**: Direct (single source file in scope)

### Results
| Metric         | Value |
|----------------|-------|
| Tests created  | 9     |
| Tests passing  | 9     |
| Tests failing  | 0     |
| Files created  | 1     |

### Files Created
- `tests/test_invoice_service.py` (9 tests, 3 parametrized)

### Coverage
- InvoiceService.calculate_total — 3 happy path, 2 error cases
- InvoiceService.get_by_id — 1 happy path, 1 error case
- InvoiceService.mark_as_paid — 1 happy path, 2 error cases

### Build / Install Validation
- Editable install: ✅ `python -m pip install -e ".[test]"`
- Test run: ✅ `python -m pytest` — 9 passed in 0.12s

### Next Steps
- Add tests for repository implementations if any exist
- Consider snapshot/property-based testing (`hypothesis`) for `calculate_total` rounding behaviour

Python Extension

Language-specific guidance for Python test generation.

Rule #1: Investigate the Repo First

Before writing any test or running any command, discover what the repo already does:

1. Find ALL existing test files — search broadly: test_*.py, *_test.py, *.uts, test/*.sh, or any other test format. Do not assume pytest. 2. Identify the test framework — look for:

• Custom test runners (e.g. UTscapy for scapy, project-specific harnesses)
• Standard frameworks (pytest, unittest, nose2)
• Test runner scripts in Makefile, tox.ini, nox, scripts/
• Config entries in pyproject.toml, setup.cfg, pytest.ini, conftest.py

3. Read existing tests thoroughly — copy their exact style: file format, imports, fixtures, assertion patterns, helper utilities, setup/teardown conventions 4. Package layout — determine import paths from existing code, not guesswork

Use whatever framework and conventions the repo already uses. If the repo uses a custom test framework (custom file formats, custom runners, domain-specific test utilities), adopt it fully — do not layer pytest on top. Only introduce pytest if the repo has no tests at all.

Environment Detection

Detect the runner from lockfiles/config and prefix all commands accordingly:

If Makefile, tox.ini, or nox config exists, prefer those scripts over raw commands.

Build Commands

Python has no separate build step. Validate with the type checker if one is configured:

Test Commands

If the repo uses a custom test framework (custom file formats, custom runner), use its native commands — do not wrap them in pytest. Examples:

For pytest projects (the most common case), use the detected <prefix>:

• Prefer python -m pytest over bare pytest to ensure the correct interpreter
• If the project uses unittest only (no pytest in deps), use python -m unittest discover

Lint Command

Use the repo's existing lint script first (make lint, tox -e lint). Otherwise detect tools from config:

• ruff.toml or [tool.ruff] → <prefix> ruff check --fix && <prefix> ruff format
• [tool.black] → <prefix> black
• .flake8 → <prefix> flake8

Project Layout and Imports

• Match existing test imports exactly — do not invent src. prefixes unless existing tests use them
• Check pyproject.toml [tool.setuptools.package-dir] for layout hints
• Default test placement: tests/ mirroring source structure (src/billing/service.py → tests/billing/test_service.py)

Test File Naming

Match the repo's existing conventions. Common patterns:

• pytest: Files test_*.py or *_test.py, functions test_ prefix, classes Test prefix
• Custom frameworks: Use whatever format existing tests use (e.g. .uts for UTscapy, custom extensions)

If writing new tests in a repo with no tests, default to pytest conventions.

Common Errors

Mocking Rules

• Use unittest.mock (stdlib) — no extra dependency needed
• Patch where the name is looked up, not where it is defined: @patch("mypackage.module.datetime") not @patch("datetime.datetime")
• Use Mock(spec=RealClass) to catch attribute errors
• Use AsyncMock for async functions
• Prefer dependency injection over @patch
• If a test needs more than 3 mocks, flag it as a design smell

Dependency Installation (Last Resort)

Only install packages after investigation confirms they are missing. Use the detected prefix:

Never run bare pip install in a Poetry/PDM/uv project — it bypasses the lockfile.

Skip Coverage Tools

Do not configure or run coverage tools (coverage.py, pytest-cov). Coverage is measured separately by the evaluation harness.

Ruby Extension

Language-specific guidance for Ruby test generation.

Rule #0: Confirm the Test Target

If the prompt does not name a specific file (e.g. "test the repository", "cover one core module", "comprehensive suite"), do not assume the largest or top-level upstream code is the intended target. In real workflows the user usually wants to test code they have just added, and large upstream repos contain hundreds of modules already covered by existing specs.

Run these read-only discovery commands first — they are the deliberate exception to Rule #1's "before writing any test or running any command" rule, and their output is the ground truth Rule #1's reading is meant to interpret. Do not write or execute any tests until Rule #0 and Rule #1 are both complete.

Prefer targets that match all of:

1. Untracked or recently added (git status / git log --diff-filter=A). 2. Small and pure (a few hundred lines, no I/O, no global state). 3. Located under a conventional source root (lib/, app/models/, app/services/). 4. Have no existing matching *_spec.rb / *_test.rb.

If spec/spec_helper.rb already requires one specific file (e.g. require "string_utils"), that file is almost certainly the target — start there.

Test Placement Contract

RSpec only discovers specs under spec/ by default, and verification harnesses (CI, msbench, coverage tools) typically scope discovery to spec/ alone. Place every spec there, mirroring the source layout:

A spec placed anywhere outside spec/ (e.g. next to the source under lib/) will be invisible to bundle exec rspec and to the harness. The same applies to Minitest: place tests under test/ and use *_test.rb naming.

Gem-monorepo trap (fastlane, ruby/ruby, large gems with sub-gems): if the repo contains multiple */spec/ directories (each sub-gem with its own specs), bundle exec rspec from the repo root only loads ./spec/ by default — sub-gem specs are invisible to the harness. Either:

• place the new spec inside the root ./spec/ (with a require_relative to the sub-gem's lib/), or
• run the sub-gem's bundle exec rspec from the sub-gem dir AND verify in the Harness Discovery Check below that the root command also enumerates it (often it won't — you'll need to extend .rspec with --default-path or the root Rakefile's test task).

For interpreter-build repos (ruby/ruby itself) the test runner requires make test-all after make miniruby — ruby test/foo_test.rb alone is not what the harness runs.

First-Test Sanity Loop

After writing the first spec — before writing any others:

1. Run bundle exec rspec --dry-run and confirm the example count is > 0. If it is 0, RSpec is not seeing your file; fix the location, filename, or $LOAD_PATH before continuing. 2. Run the spec (bundle exec rspec spec/<your_spec>.rb); fix LoadError, missing require, or constant errors before adding more tests. 3. Only then expand to cover the remaining methods.

This catches placement and load-path mistakes on turn 1 instead of after dozens of failed-test iterations.

Harness Discovery Check

Before reporting success, run the harness-equivalent discovery command from the repo root and confirm the example count went up by at least the number of tests you generated. CI/msbench/coverage harnesses do not know which file or sub-gem dir you targeted; they run the framework's default discovery from the repo root, so a spec that passes via bundle exec rspec fastlane_core/spec/foo_spec.rb is still worthless if bundle exec rspec --dry-run from the repo root doesn't enumerate it.

# RSpec — from repo root
bundle exec rspec --dry-run 2>&1 | grep -E '^[0-9]+ example'

# Minitest (Rails)
{ bundle exec rake test --dry-run 2>/dev/null || bin/rails test --list-tests; } | wc -l

# Custom runner (Homebrew, ruby/ruby, etc.)
# Use the repo's own runner — `./bin/brew tests --list`, `make test-all`, etc.
# If no `--list`/`--dry-run` mode exists, run a single matching test by name and confirm exit 0.

If the count did not increase, your spec is invisible to the harness. Move it into ./spec/, extend .rspec/Rakefile so the harness picks up the sub-gem dir, or switch to a require_relative strategy from a root-level spec. Do not report success until the harness-equivalent command sees your new tests.

Rule #1: Investigate the Repo First

Before writing any test or running any command, read:

1. Existing tests — find spec/**/*_spec.rb (RSpec) or test/**/*_test.rb (Minitest) and copy their style (matchers, helpers, factories, contexts) 2. `Gemfile` / `Gemfile.lock` — Ruby version, test framework, supporting gems (rspec, minitest, factory_bot, webmock, vcr, rails) 3. `.ruby-version` / .tool-versions — pinned Ruby version 4. Test helpers — spec/spec_helper.rb, spec/rails_helper.rb, test/test_helper.rb — these dictate the load path, requires, and global config 5. Rake tasks — Rakefile may define a default task that runs the full test suite

Use the framework the repo already uses. Do not introduce RSpec into a Minitest project (or vice versa).

Toolchain Detection

Always run inside bundle exec if a Gemfile.lock is present — otherwise you may pick up a system gem version that disagrees with the lockfile.

Build Commands

Ruby is interpreted — there is no compile step. The closest validations:

For Rails: load all classes once with bundle exec rails zeitwerk:check to catch missing constants before running tests.

Test Commands

RSpec

Minitest

Rails (any framework)

Always prefer the wrapper script (bin/rails, bin/rspec) when present — they enforce the project's loader/setup.

Lint Command

• bundle exec rubocop — autocorrect with bundle exec rubocop -A (only if existing tests already conform; do not autocorrect unrelated files)
• bundle exec standardrb --fix if standard is in the Gemfile
• Some Rails projects add rubocop-rails, rubocop-rspec, rubocop-performance — they enforce extra rules

Project Layout and Loading

Loading source code:

• RSpec: spec/spec_helper.rb typically does require 'my_gem' or sets $LOAD_PATH. Match its pattern in new specs by require 'spec_helper' (or require 'rails_helper' in Rails)
• Minitest: each _test.rb typically require 'test_helper'
• Rails uses Zeitwerk autoloading — do not add require_relative '../../app/models/widget'; just require 'rails_helper' and reference the constant

Test File Naming

RSpec Template

require 'spec_helper'
require 'calculator'

RSpec.describe Calculator do
  subject(:calculator) { described_class.new }

  describe '#add' do
    it 'returns the sum of two positive numbers' do
      expect(calculator.add(2, 3)).to eq(5)
    end

    context 'with negative numbers' do
      it 'returns the correct sum' do
        expect(calculator.add(-1, 1)).to eq(0)
      end
    end

    it 'raises when given non-numeric input' do
      expect { calculator.add('a', 1) }.to raise_error(TypeError)
    end
  end
end

Common Errors

Mocking Rules (RSpec)

• Use instance_double(Klass) and class_double(Klass) — they verify that the method actually exists, unlike double
• allow(obj).to receive(:method).and_return(value) for stubs; expect(obj).to receive(:method) for interaction expectations
• Prefer instance_double over plain double; prefer dependency injection over allow_any_instance_of
• Use let for memoized helpers; use let! only when the side effect must run before each example
• Avoid global state mutation in tests — wrap in around blocks or use ClimateControl for env vars
• For HTTP, use webmock (stub_request(:get, ...)) or vcr cassettes if the project already uses them
• If a test needs more than 3 mocks, flag it as a design smell

Mocking Rules (Minitest)

• Use Minitest::Mock for simple cases: mock = Minitest::Mock.new; mock.expect(:method, return_value, [arg])
• For richer mocking, projects commonly add mocha: obj.expects(:method).returns(value) (in test_helper.rb: require 'mocha/minitest')
• Always verify mocks at end of test (mock.verify for Minitest::Mock); Mocha verifies automatically

Rails Specifics

• Use the smallest spec type that covers the behavior: model spec for pure logic, request spec for HTTP, system spec only when JS/UI matters
• rails-controller-testing gem must be present for assigns(:foo) and assert_template
• ActiveJob::TestHelper and ActiveSupport::Testing::TimeHelpers (travel_to) come with Rails — use them instead of Timecop if Rails ≥ 5
• Use fixtures only if the project already uses them; factory_bot is more common in modern Rails apps
• Database transactions wrap each test by default — for system tests with browser drivers, use DatabaseCleaner strategies the project already configures

Dependency Installation (Last Resort)

Only add gems after investigation confirms they are missing. Edit Gemfile:

group :test do
  gem 'rspec'
  gem 'webmock'
end

Then run:

bundle install

Never gem install outside Bundler — it bypasses the lockfile and changes the global Ruby environment.

Skip Coverage Tools

Do not configure or run coverage tools (SimpleCov). Coverage is measured separately by the evaluation harness.

Rust Extension

Language-specific guidance for Rust test generation.

Rule #1: Investigate the Repo First

Before writing any test or running any command, read:

1. Existing tests — look at #[cfg(test)] mod tests blocks inside src/, integration tests in tests/, doc tests in source comments, and any examples/ that double as smoke tests 2. `Cargo.toml` — workspace layout ([workspace]), edition, dev-dependencies, feature flags, [[bench]] / [[test]] declarations 3. `Cargo.lock` — if checked in, you must not break it without intent 4. Toolchain — rust-toolchain.toml pins the channel (stable / nightly / specific version) 5. `build.rs` — custom build scripts may set cfg flags or generate code that tests rely on

Match the repo's existing conventions — assertion macros, mock approach, feature-gating — exactly. Do not introduce tokio::test if the repo uses async-std, etc.

Toolchain Detection

Build Commands

cargo check is far faster than cargo build and catches almost the same errors. Prefer it during the fix loop; use cargo build --tests (or cargo test --no-run) before declaring tests compilable.

Test Commands

• Arguments before -- are for cargo; arguments after -- go to the test binary
• cargo test foo runs every test with foo in its full path (module::tests::foo_does_a_thing) — to avoid surprise matches use --exact
• cargo nextest run is significantly faster if the repo already uses it (Cargo.toml [profile.nextest...] or .config/nextest.toml) — match the repo's choice

Lint Command

Use the repo's lint script first. Otherwise:

• cargo fmt --all -- --check (CI), cargo fmt (apply)
• cargo clippy --all-targets --all-features -- -D warnings
• If clippy.toml / rustfmt.toml exists, the project has opinions — never override them in your tests

Project Layout

my_crate/
├── Cargo.toml
├── src/
│   ├── lib.rs        # library crate root
│   ├── main.rs       # binary crate root (mutually OK with lib.rs)
│   └── module.rs     # private/public module
├── tests/            # integration tests — each .rs is a separate crate
│   └── widget.rs
├── benches/          # cargo bench targets
└── examples/         # cargo run --example name

• Unit tests at the bottom of module.rs:

  #[cfg(test)]
  mod tests {
      use super::*;

      #[test]
      fn name_scenario_expected() {
          // ...
      }
  }

• Integration tests import the crate by name: use my_crate::PublicType;
• Helpers shared between integration tests must live in tests/common/mod.rs (the mod.rs form prevents cargo from treating them as a top-level test crate)

Test Function Patterns

Pick the async harness the repo already uses. Do not mix tokio and async-std in tests.

Common Errors

Mocking Rules

Rust has no single dominant mocking framework. Match the repo:

• Trait + struct fakes (most idiomatic): define a trait, pass Arc<dyn Trait> or generic T: Trait, implement a fake struct in tests
• `mockall` crate: #[automock] on a trait generates MockTrait for use in tests
• `mockito` / `wiremock`: HTTP server mocks for client tests
• `tempfile`: scoped temp directories that auto-clean (tempfile::tempdir())

Avoid unsafe patches to "mock" free functions. Refactor to inject a trait instead. If a test needs more than 3 mocks, flag it as a design smell.

Features and cfg

• Tests behind a feature flag run only when that feature is enabled — use #[cfg(feature = "foo")] on the mod tests or individual #[test] functions
• --all-features exercises everything but may pull conflicting features in some workspaces; check cargo test --all-features is part of CI before relying on it
• Use #[cfg(test)] to gate test-only helpers in production source files — not #[cfg(feature = "test")]

Concurrency, IO, and unsafe

• Tests run in parallel by default. If your tests share global state (env vars, current dir, statics), serialize them with the serial_test crate (if present) or move state into the test
• Never write to /tmp or the repo dir directly — use tempfile::tempdir() so cleanup is automatic
• Tests in unsafe code should also run under Miri (cargo +nightly miri test) if the repo's CI does

Dependency Installation (Last Resort)

Only add dependencies after investigation confirms they are missing:

[dev-dependencies]
mockall = "0.12"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

Or via cargo:

cargo add --dev mockall
cargo add --dev tokio --features macros,rt-multi-thread

Match the major version of any tokio/serde/etc. already pinned by the workspace.

Skip Coverage Tools

Do not configure or run coverage tools (cargo tarpaulin, cargo llvm-cov, grcov). Coverage is measured separately by the evaluation harness.