Test First Bugs

Test-first bug fixing

Enforce a disciplined bug-fixing workflow that prevents regression and parallelizes fix attempts.

Core workflow

When a bug is reported, follow these steps in order:

Phase 1: Reproduce and document

1. Understand the bug — Gather details about expected vs actual behavior 2. Identify the test location — Determine where tests live in the project (check for tests/, __tests__/, spec/, *.test.*, *.spec.* patterns) 3. Write a failing test — Create a test that demonstrates the bug

Phase 2: Fix with subagents

4. Launch fix subagents — Use the Task tool with subagent_type=general-purpose to attempt fixes 5. Run the test — Verify the fix by running the specific test 6. Iterate if needed — If test still fails, launch additional subagents with new approaches

Phase 3: Verify and complete

7. Run full test suite — Ensure no regressions were introduced 8. Report success — Confirm the bug is fixed with passing test as proof

Writing the failing test

Test naming convention

Name the test to describe the bug:

# Python (pytest)
def test_user_login_fails_when_email_has_uppercase():
    ...

# Python (unittest)
def test_should_handle_empty_input_without_crashing(self):
    ...

// JavaScript (Jest/Vitest)
it('should not crash when input array is empty', () => { ... });
test('handles special characters in username', () => { ... });

// TypeScript
describe('UserService', () => {
  it('returns null when user not found instead of throwing', () => { ... });
});

Test structure

Every bug reproduction test follows this pattern:

def test_bug_description():
    # 1. ARRANGE - Set up the conditions that trigger the bug
    input_data = create_problematic_input()

    # 2. ACT - Perform the action that causes the bug
    result = function_under_test(input_data)

    # 3. ASSERT - Verify the expected (correct) behavior
    assert result == expected_value  # This should FAIL initially

Finding the right test file

Check the project structure for existing test patterns:

# Find test files
find . -name "*.test.*" -o -name "*.spec.*" -o -name "test_*.py" | head -20

# Find test directories
ls -la tests/ __tests__/ spec/ test/ 2>/dev/null

# Check package.json for test command
grep -A5 '"test"' package.json

Launching fix subagents

Use the Task tool to parallelize fix attempts:

Task tool parameters:
- subagent_type: "general-purpose"
- description: "Fix [bug description]"
- prompt: Include:
  1. The bug description
  2. The failing test location and contents
  3. Suspected cause (if known)
  4. Constraint: "Run the test to verify your fix works"

Parallel fix strategies

Launch multiple subagents with different approaches:

1. Direct fix agent — Focus on the immediate code causing the bug 2. Root cause agent — Investigate deeper architectural issues 3. Edge case agent — Look for similar bugs in related code

When projects lack tests

If the project has no test infrastructure:

1. Set up minimal test framework first 2. Create the test file in a sensible location 3. Document the test setup for future use

Quick test setup commands

# Python
pip install pytest
mkdir -p tests && touch tests/__init__.py

# JavaScript/TypeScript
npm install --save-dev jest
# or
npm install --save-dev vitest

# Go
# Tests are built-in, create *_test.go files

Verifying the fix

After subagent reports completion:

# Run the specific test
pytest tests/test_module.py::test_bug_description -v
npm test -- --grep "bug description"
go test -run TestBugDescription -v

# Run full suite to check for regressions
pytest
npm test
go test ./...

Example workflow

User reports: "The login function crashes when email has spaces"

Phase 1 — Write failing test:

# tests/test_auth.py
def test_login_handles_email_with_spaces():
    """Bug: Login crashes when email contains spaces"""
    auth = AuthService()

    # This should return an error, not crash
    result = auth.login("user @example.com", "password")

    assert result.success == False
    assert "invalid email" in result.error.lower()

Run test to confirm it fails:

pytest tests/test_auth.py::test_login_handles_email_with_spaces -v
# Expected: FAILED (demonstrates the bug)

Phase 2 — Launch subagent:

Task tool:
- subagent_type: "general-purpose"
- description: "Fix email space crash"
- prompt: "Fix the login crash when email contains spaces.

  Bug: AuthService.login() crashes instead of returning error when email has spaces.

  Failing test: tests/test_auth.py::test_login_handles_email_with_spaces

  After fixing, run: pytest tests/test_auth.py::test_login_handles_email_with_spaces -v

  The test must pass to confirm the fix."

Phase 3 — Verify:

# Specific test passes
pytest tests/test_auth.py::test_login_handles_email_with_spaces -v
# PASSED

# No regressions
pytest tests/test_auth.py -v
# All tests pass

Integration with hooks

The bug-report-detector hook in this plugin automatically: 1. Detects when a user reports a bug 2. Reminds Claude to follow the test-first workflow 3. Blocks Edit/Write tools until a test file has been created or modified

Additional resources

Reference files

• `references/test-frameworks.md` — Framework-specific test patterns
• `references/common-bugs.md` — Common bug patterns and test strategies

Example files

• `examples/python-bug-test.py` — Python pytest example
• `examples/js-bug-test.js` — JavaScript Jest example

Scripts

• `scripts/find-tests.sh` — Locate test infrastructure in a project

Common bug patterns and test strategies

Null / None / Undefined handling

Symptoms: TypeError, NullPointerException, "undefined is not a function"

Test strategy:

def test_handles_none_input():
    result = function(None)
    assert result is not None  # or appropriate default

def test_handles_missing_key():
    data = {}  # Missing expected key
    result = function(data)
    assert result == default_value

Common fixes:

• Add null checks at function entry
• Use optional chaining (?. in JS/TS)
• Provide default values
• Use get() with defaults for dict/object access

Off-by-one errors

Symptoms: IndexError, missing first/last item, extra iteration

Test strategy:

def test_first_element():
    result = function([1, 2, 3])
    assert result[0] == 1  # Verify first element handled

def test_last_element():
    result = function([1, 2, 3])
    assert result[-1] == 3  # Verify last element handled

def test_single_element():
    result = function([1])
    assert len(result) == 1

def test_empty_collection():
    result = function([])
    assert result == []

String encoding issues

Symptoms: UnicodeDecodeError, garbled text, "?" characters

Test strategy:

def test_handles_unicode():
    result = function("café ñ 日本語")
    assert "café" in result

def test_handles_emoji():
    result = function("Hello 👋 World")
    assert "👋" in result

def test_handles_special_chars():
    result = function("test@#$%^&*()")
    assert result is not None

Race conditions

Symptoms: Intermittent failures, data corruption, deadlocks

Test strategy:

import threading
import concurrent.futures

def test_concurrent_access():
    results = []

    def worker():
        results.append(function())

    threads = [threading.Thread(target=worker) for _ in range(10)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()

    assert len(results) == 10
    assert all(r is not None for r in results)

Date/time bugs

Symptoms: Wrong timezone, off-by-one day, DST issues

Test strategy:

from datetime import datetime, timezone
import freezegun  # or time-machine

@freezegun.freeze_time("2024-03-10 02:30:00")  # DST transition
def test_handles_dst_transition():
    result = function()
    assert result.hour in (2, 3)  # Depends on expected behavior

def test_handles_timezone():
    utc_time = datetime.now(timezone.utc)
    result = function(utc_time)
    # Verify timezone preserved or converted correctly

def test_handles_leap_year():
    date = datetime(2024, 2, 29)  # Leap year
    result = function(date)
    assert result is not None

Floating point precision

Symptoms: 0.1 + 0.2 != 0.3, comparison failures

Test strategy:

import math

def test_float_calculation():
    result = function(0.1, 0.2)
    assert math.isclose(result, 0.3, rel_tol=1e-9)

def test_currency_calculation():
    # Use Decimal for money
    from decimal import Decimal
    result = function(Decimal("10.99"), Decimal("5.01"))
    assert result == Decimal("16.00")

Memory leaks / resource exhaustion

Symptoms: OOM errors, file handle exhaustion, slow degradation

Test strategy:

import tracemalloc

def test_no_memory_leak():
    tracemalloc.start()

    for _ in range(1000):
        function()

    current, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()

    assert peak < 100_000_000  # 100MB threshold

def test_file_handles_closed():
    import resource
    soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)

    for _ in range(100):
        function()

    # Should not leak file descriptors

SQL injection / input validation

Symptoms: Security vulnerability, unexpected query results

Test strategy:

def test_sql_injection_attempt():
    malicious_input = "'; DROP TABLE users; --"
    result = function(malicious_input)
    # Should sanitize or reject, not execute
    assert "DROP" not in str(result)

def test_xss_attempt():
    malicious_input = "<script>alert('xss')</script>"
    result = function(malicious_input)
    assert "<script>" not in result

Async / Promise handling

Symptoms: Unhandled promise rejection, callback not called

Test strategy:

it('handles async error correctly', async () => {
  // Bug: unhandled rejection
  await expect(asyncFunction()).rejects.toThrow('Expected error');
});

it('resolves within timeout', async () => {
  const result = await Promise.race([
    asyncFunction(),
    new Promise((_, reject) =>
      setTimeout(() => reject(new Error('Timeout')), 5000)
    )
  ]);
  expect(result).toBeDefined();
});

State mutation bugs

Symptoms: Unexpected side effects, stale data, wrong order

Test strategy:

def test_does_not_mutate_input():
    original = [1, 2, 3]
    input_copy = original.copy()

    function(original)

    assert original == input_copy  # Input unchanged

def test_returns_new_object():
    obj = {"key": "value"}
    result = function(obj)

    assert result is not obj  # Different object
    assert result == expected

Configuration / environment bugs

Symptoms: Works locally, fails in production

Test strategy:

import os
from unittest.mock import patch

def test_handles_missing_env_var():
    with patch.dict(os.environ, {}, clear=True):
        result = function()
        assert result == default_value

def test_handles_different_env():
    with patch.dict(os.environ, {"ENV": "production"}):
        result = function()
        # Verify production behavior

Error message preservation

Symptoms: Generic error, lost context, unhelpful message

Test strategy:

def test_error_includes_context():
    try:
        function(bad_input)
        assert False, "Should have raised"
    except CustomError as e:
        assert "bad_input" in str(e)
        assert e.original_error is not None

Pagination / limit bugs

Symptoms: Missing items, duplicates, infinite loop

Test strategy:

def test_pagination_no_duplicates():
    all_results = []
    page = 1
    while True:
        results = function(page=page, limit=10)
        if not results:
            break
        all_results.extend(results)
        page += 1

    # No duplicates
    assert len(all_results) == len(set(r.id for r in all_results))

def test_large_offset():
    result = function(page=10000, limit=10)
    assert isinstance(result, list)  # Should not error

Test framework patterns

Quick reference for writing bug reproduction tests in common frameworks.

Python

pytest (recommended)

# tests/test_module.py
import pytest
from myapp.module import function_under_test

def test_bug_description():
    """
    Bug: [describe the bug]
    Expected: [expected behavior]
    Actual: [actual buggy behavior]
    """
    # Arrange
    input_data = "problematic input"

    # Act
    result = function_under_test(input_data)

    # Assert
    assert result == expected_value

# For exceptions
def test_should_not_crash_on_bad_input():
    with pytest.raises(ValueError, match="expected error"):
        function_under_test(bad_input)

# For async
@pytest.mark.asyncio
async def test_async_bug():
    result = await async_function()
    assert result is not None

# Parametrized for multiple cases
@pytest.mark.parametrize("input,expected", [
    ("case1", "result1"),
    ("case2", "result2"),
])
def test_multiple_cases(input, expected):
    assert function_under_test(input) == expected

Run commands:

pytest tests/test_module.py::test_bug_description -v
pytest tests/test_module.py -v  # All tests in file
pytest -x  # Stop on first failure
pytest --tb=short  # Shorter tracebacks

# Watch-mode loop (re-runs on file change). Install with: pip install pytest-watcher
ptw -- tests/test_module.py -v

For async tests, install pytest-asyncio and either decorate each test with @pytest.mark.asyncio (above) or set asyncio_mode = auto in pytest.ini / pyproject.toml to skip the per-test decorator.

unittest

import unittest
from myapp.module import function_under_test

class TestBugFix(unittest.TestCase):
    def test_bug_description(self):
        """Bug: [description]"""
        result = function_under_test("input")
        self.assertEqual(result, expected)

    def test_should_raise_on_invalid(self):
        with self.assertRaises(ValueError):
            function_under_test(None)

Run commands:

python -m unittest tests.test_module.TestBugFix.test_bug_description
python -m unittest discover tests/

JavaScript / TypeScript

Jest

// __tests__/module.test.js
const { functionUnderTest } = require('../src/module');

describe('Module', () => {
  describe('functionUnderTest', () => {
    it('should handle edge case without crashing', () => {
      // Bug: crashes on empty input
      const result = functionUnderTest('');

      expect(result).toBeDefined();
      expect(result.error).toBeNull();
    });

    it('should throw on invalid input', () => {
      expect(() => {
        functionUnderTest(null);
      }).toThrow('Invalid input');
    });
  });
});

// Async
it('should fetch data correctly', async () => {
  const result = await asyncFunction();
  expect(result.data).toHaveLength(3);
});

// With mocks
jest.mock('../src/api');
it('should handle API error', async () => {
  api.fetch.mockRejectedValue(new Error('Network error'));

  const result = await functionUnderTest();

  expect(result.error).toBe('Network error');
});

Run commands:

# Jest uses -t / --testNamePattern (NOT --grep, which is Mocha)
npm test -- -t "should handle edge case"
npm test -- __tests__/module.test.js
npm test -- --watch  # Watch mode

Vitest

// src/module.test.ts
import { describe, it, expect, vi } from 'vitest';
import { functionUnderTest } from './module';

describe('functionUnderTest', () => {
  it('handles empty array without crashing', () => {
    // Bug: TypeError when array is empty
    const result = functionUnderTest([]);

    expect(result).toEqual([]);
  });
});

Run commands:

npx vitest run src/module.test.ts
npx vitest --reporter=verbose

Mocha + Chai

const { expect } = require('chai');
const { functionUnderTest } = require('../src/module');

describe('Module', function() {
  it('should handle special characters', function() {
    const result = functionUnderTest('test@#$%');
    expect(result).to.be.a('string');
    expect(result).to.not.include('undefined');
  });
});

Go

// module_test.go
package mypackage

import (
    "testing"
)

func TestBugDescription(t *testing.T) {
    // Bug: function panics on nil input
    result, err := FunctionUnderTest(nil)

    if err == nil {
        t.Error("expected error for nil input")
    }
    if result != nil {
        t.Errorf("expected nil result, got %v", result)
    }
}

// Table-driven tests
func TestMultipleCases(t *testing.T) {
    tests := []struct {
        name     string
        input    string
        expected string
    }{
        {"empty string", "", "default"},
        {"special chars", "@#$", "sanitized"},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            result := FunctionUnderTest(tt.input)
            if result != tt.expected {
                t.Errorf("got %s, want %s", result, tt.expected)
            }
        })
    }
}

Run commands:

go test -v -run TestBugDescription
go test ./... -v
go test -race ./...  # Check for race conditions

Rust

// src/module.rs or tests/module_test.rs
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_bug_description() {
        // Bug: panics on empty vec
        let result = function_under_test(vec![]);

        assert!(result.is_ok());
        assert_eq!(result.unwrap(), expected);
    }

    #[test]
    #[should_panic(expected = "invalid input")]
    fn test_panics_on_invalid() {
        function_under_test(invalid_input);
    }
}

Run commands:

cargo test test_bug_description -- --nocapture
cargo test -- --test-threads=1

Ruby (RSpec)

# spec/module_spec.rb
require 'module'

RSpec.describe Module do
  describe '#function_under_test' do
    it 'handles nil input without crashing' do
      # Bug: NoMethodError on nil
      result = described_class.function_under_test(nil)

      expect(result).to be_nil
    end

    it 'raises ArgumentError for invalid type' do
      expect {
        described_class.function_under_test(123)
      }.to raise_error(ArgumentError, /expected string/)
    end
  end
end

Run commands:

rspec spec/module_spec.rb:10  # Line number
rspec --example "handles nil"

PHP (PHPUnit)

<?php
// tests/ModuleTest.php
use PHPUnit\Framework\TestCase;

class ModuleTest extends TestCase
{
    public function testBugDescription(): void
    {
        // Bug: returns null instead of empty array
        $result = functionUnderTest([]);

        $this->assertIsArray($result);
        $this->assertEmpty($result);
    }

    public function testThrowsOnInvalid(): void
    {
        $this->expectException(InvalidArgumentException::class);
        functionUnderTest(null);
    }
}

Run commands:

./vendor/bin/phpunit --filter testBugDescription
./vendor/bin/phpunit tests/ModuleTest.php