Test Master

Test Master

Comprehensive testing specialist ensuring software quality through functional, performance, and security testing.

Core Workflow

1. Define scope — Identify what to test and which testing types apply 2. Create strategy — Plan the test approach across functional, performance, and security perspectives 3. Write tests — Implement tests with proper assertions (see example below) 4. Execute — Run tests and collect results

• If tests fail: classify the failure (assertion error vs. environment/flakiness), fix root cause, re-run
• If tests are flaky: isolate ordering dependencies, check async handling, add retry or stabilization logic

5. Report — Document findings with severity ratings and actionable fix recommendations

• Verify coverage targets are met before closing; flag gaps explicitly

Quick-Start Example

A minimal Jest unit test illustrating the key patterns this skill enforces:

// ✅ Good: meaningful description, specific assertion, isolated dependency
describe('calculateDiscount', () => {
  it('applies 10% discount for premium users', () => {
    const result = calculateDiscount({ price: 100, userTier: 'premium' });
    expect(result).toBe(90); // specific outcome, not just truthy
  });

  it('throws on negative price', () => {
    expect(() => calculateDiscount({ price: -1, userTier: 'standard' }))
      .toThrow('Price must be non-negative');
  });
});

Apply the same structure for pytest (def test_…, assert result == expected) and other frameworks.

Reference Guide

Load detailed guidance based on context:

<!-- TDD Iron Laws and Testing Anti-Patterns adapted from obra/superpowers by Jesse Vincent (@obra), MIT License -->

Constraints

MUST DO

• Test happy paths AND error/edge cases (e.g., empty input, null, boundary values)
• Mock external dependencies — never call real APIs or databases in unit tests
• Use meaningful it('…') descriptions that read as plain-English specifications
• Assert specific outcomes (expect(result).toBe(90)), not just truthiness
• Run tests in CI/CD; document and remediate coverage gaps

MUST NOT

• Skip error-path testing (e.g., don't test only the success branch of a try/catch)
• Use production data in tests — use fixtures or factories instead
• Create order-dependent tests — each test must be independently runnable
• Ignore flaky tests — quarantine and fix them; don't just re-run until green
• Test implementation details (internal method calls) — test observable behaviour

Output Templates

When creating test plans, provide: 1. Test scope and approach 2. Test cases with expected outcomes 3. Coverage analysis 4. Findings with severity (Critical/High/Medium/Low) 5. Specific fix recommendations

Documentation

Automation Frameworks

Advanced Framework Patterns

Screenplay Pattern

// Better separation of concerns than POM
export class Actor {
  constructor(private page: Page) {}
  attemptsTo(...tasks: Task[]) {
    return Promise.all(tasks.map(t => t.performAs(this)));
  }
}

class Login implements Task {
  constructor(private email: string, private password: string) {}
  async performAs(actor: Actor) {
    await actor.page.getByLabel('Email').fill(this.email);
    await actor.page.getByLabel('Password').fill(this.password);
    await actor.page.getByRole('button', { name: 'Login' }).click();
  }
}

// Clear, maintainable test code
await new Actor(page).attemptsTo(new Login('user@test.com', 'pass'));

Keyword-Driven Testing

const keywords = {
  NAVIGATE: (page, url) => page.goto(url),
  CLICK: (page, selector) => page.click(selector),
  TYPE: (page, selector, text) => page.fill(selector, text),
  VERIFY: (page, selector) => expect(page.locator(selector)).toBeVisible(),
};

// Data drives execution - ideal for non-technical authors
const steps = [
  { keyword: 'NAVIGATE', args: ['/login'] },
  { keyword: 'TYPE', args: ['#email', 'user@test.com'] },
  { keyword: 'CLICK', args: ['#submit'] },
];

for (const step of steps) await keywords[step.keyword](page, ...step.args);

Model-Based Testing

// State machine defines valid transitions
const cartModel = {
  empty: { addItem: 'hasItems' },
  hasItems: { addItem: 'hasItems', removeItem: 'hasItems|empty', checkout: 'checkingOut' },
  checkingOut: { confirm: 'complete', cancel: 'hasItems' },
};

// Generate comprehensive test paths automatically
const testPaths = generatePathsFromModel(cartModel);

Maintenance Strategies

Self-Healing Locators

// Multi-strategy finder with automatic fallback
async function findElement(page: Page, strategies: string[]): Promise<Locator> {
  for (const selector of strategies) {
    const el = page.locator(selector);
    if (await el.count() > 0) return el;
  }
  throw new Error(`Not found: ${strategies.join(', ')}`);
}

// Usage: tries best -> good -> fallback
const submit = await findElement(page, [
  '[data-testid="submit"]',     // Best: stable test ID
  'button:has-text("Submit")',  // Good: semantic
  'button.primary',             // Fallback: CSS
]);

Error Recovery & Smart Retry

// Auto-retry with recovery actions
async function clickWithRecovery(page: Page, selector: string, retries = 3) {
  for (let i = 0; i < retries; i++) {
    try {
      await page.click(selector, { timeout: 5000 });
      return;
    } catch (e) {
      if (i === retries - 1) throw e;
      await page.reload();
      await page.waitForLoadState('networkidle');
    }
  }
}

// Exponential backoff for flaky operations
async function retryWithBackoff<T>(fn: () => Promise<T>, retries = 3): Promise<T> {
  for (let i = 0; i < retries; i++) {
    try {
      return await fn();
    } catch (e) {
      if (i === retries - 1) throw e;
      await new Promise(r => setTimeout(r, 1000 * Math.pow(2, i)));
    }
  }
}

Scaling Strategies

Parallel & Distributed Execution

// playwright.config.ts
export default defineConfig({
  workers: process.env.CI ? 8 : 4,
  fullyParallel: true,
  retries: process.env.CI ? 2 : 0,
  
  // Shard tests across multiple machines
  shard: process.env.SHARD ? {
    current: parseInt(process.env.SHARD_INDEX),
    total: parseInt(process.env.SHARD_TOTAL),
  } : undefined,
});

# GitHub Actions: distribute across 5 workers
strategy:
  matrix:
    shard: [1, 2, 3, 4, 5]
steps:
  - run: npx playwright test --shard=${{ matrix.shard }}/5

Resource Optimization

// Reuse browser contexts for faster execution
let browser: Browser;
let context: BrowserContext;

test.beforeAll(async () => {
  browser = await chromium.launch();
  context = await browser.newContext();
});

test('test 1', async () => {
  const page = await context.newPage();
  // Test logic
  await page.close();
});

test.afterAll(async () => {
  await context.close();
  await browser.close();
});

CI/CD Integration

Complete Pipeline

name: E2E Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - run: npm ci
      - run: npx playwright install --with-deps
      
      - run: npx playwright test --shard=${{ matrix.shard }}/4
        env:
          CI: true
      
      - uses: actions/upload-artifact@v3
        if: always()
        with:
          name: report-${{ matrix.shard }}
          path: playwright-report/

Test Data Factories

export class UserFactory {
  static create(overrides?: Partial<User>): User {
    return {
      id: faker.string.uuid(),
      email: faker.internet.email(),
      name: faker.person.fullName(),
      role: 'user',
      ...overrides,
    };
  }

  static createMany(count: number) {
    return Array.from({ length: count }, () => this.create());
  }
}

// Seed test data
test.beforeEach(async ({ page }) => {
  await page.request.post('/api/test/seed', {
    data: { users: UserFactory.createMany(10) },
  });
});

Team Enablement

Training Program

**Week 1-2**: Framework basics, page objects, first test
**Week 3-4**: Data-driven, API integration, CI/CD
**Week 5-6**: Performance, error handling, scaling
**Ongoing**: Code reviews, knowledge sharing

Code Review Checklist

- [ ] Independent tests (no order dependency)
- [ ] Semantic locators (getByRole, getByLabel)
- [ ] Proper waits (no arbitrary timeouts)
- [ ] Error cases tested
- [ ] Test data cleanup
- [ ] Meaningful test names
- [ ] Page objects updated

Automation Strategy

ROI Calculation

const manual = { timePerRun: 30, runsPerSprint: 10 };
const automation = { development: 120, maintenance: 5 };

const timeSaved = (manual.timePerRun * manual.runsPerSprint) - automation.maintenance;
const breakEven = Math.ceil(automation.development / timeSaved);
const annualSavings = (timeSaved * 26 - automation.development) / 60; // hours

// Example: Break-even in 1 sprint, save 110 hours/year

Selection Criteria

**Automate**: Repetitive, stable UI, critical paths, data-driven, positive ROI
**Don't Automate**: Exploratory, changing UI, one-time, usability, negative ROI

Reporting & Metrics

Custom Reporter

class MetricsReporter implements Reporter {
  onTestEnd(test: TestCase, result: TestResult) {
    this.sendMetrics({
      name: test.title,
      duration: result.duration,
      status: result.status,
      retries: result.retry,
    });
  }
}

Quick Reference

E2E Testing

E2E Test Strategy

// Critical user paths to test
const criticalPaths = [
  'User registration and login',
  'Core product/service workflow',
  'Payment/checkout flow',
  'Settings and profile management',
];

User Flow Testing

import { test, expect } from '@playwright/test';

test.describe('User Registration Flow', () => {
  test('complete registration', async ({ page }) => {
    await page.goto('/register');

    await page.getByLabel('Email').fill('new@example.com');
    await page.getByLabel('Password').fill('SecurePass123!');
    await page.getByLabel('Confirm Password').fill('SecurePass123!');
    await page.getByRole('button', { name: 'Register' }).click();

    await expect(page).toHaveURL(/dashboard/);
    await expect(page.getByText('Welcome')).toBeVisible();
  });

  test('shows validation errors', async ({ page }) => {
    await page.goto('/register');

    await page.getByLabel('Email').fill('invalid');
    await page.getByRole('button', { name: 'Register' }).click();

    await expect(page.getByText('Invalid email')).toBeVisible();
  });
});

Checkout Flow

test.describe('Checkout Flow', () => {
  test('complete purchase', async ({ page }) => {
    // Add to cart
    await page.goto('/products/123');
    await page.getByRole('button', { name: 'Add to Cart' }).click();
    await expect(page.getByTestId('cart-count')).toHaveText('1');

    // Checkout
    await page.goto('/cart');
    await page.getByRole('button', { name: 'Checkout' }).click();

    // Payment
    await page.getByLabel('Card Number').fill('4242424242424242');
    await page.getByLabel('Expiry').fill('12/25');
    await page.getByLabel('CVC').fill('123');
    await page.getByRole('button', { name: 'Pay' }).click();

    // Confirmation
    await expect(page).toHaveURL(/order-confirmation/);
    await expect(page.getByText('Order Confirmed')).toBeVisible();
  });
});

Test Data Management

// fixtures/testData.ts
export const testUsers = {
  standard: {
    email: 'standard@test.com',
    password: 'TestPass123!',
  },
  admin: {
    email: 'admin@test.com',
    password: 'AdminPass123!',
  },
};

// Test setup
test.beforeEach(async ({ page }) => {
  // Seed test data
  await page.request.post('/api/test/seed');
});

test.afterEach(async ({ page }) => {
  // Clean up
  await page.request.post('/api/test/cleanup');
});

Cross-Browser Testing

// playwright.config.ts
export default defineConfig({
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
    { name: 'mobile-chrome', use: { ...devices['Pixel 5'] } },
    { name: 'mobile-safari', use: { ...devices['iPhone 13'] } },
  ],
});

Quick Reference

Integration Testing

API Testing (Supertest)

import request from 'supertest';
import { app } from '../app';

describe('POST /api/users', () => {
  it('creates user with valid data', async () => {
    const response = await request(app)
      .post('/api/users')
      .send({ email: 'test@test.com', name: 'Test' })
      .expect(201);

    expect(response.body).toMatchObject({
      email: 'test@test.com',
      name: 'Test',
    });
    expect(response.body.id).toBeDefined();
  });

  it('returns 400 for invalid email', async () => {
    const response = await request(app)
      .post('/api/users')
      .send({ email: 'invalid', name: 'Test' })
      .expect(400);

    expect(response.body.error).toContain('email');
  });

  it('returns 401 without auth token', async () => {
    await request(app)
      .get('/api/users/me')
      .expect(401);
  });
});

Authenticated Requests

describe('Protected endpoints', () => {
  let authToken: string;

  beforeAll(async () => {
    const response = await request(app)
      .post('/api/auth/login')
      .send({ email: 'test@test.com', password: 'password' });
    authToken = response.body.token;
  });

  it('accesses protected route', async () => {
    await request(app)
      .get('/api/users/me')
      .set('Authorization', `Bearer ${authToken}`)
      .expect(200);
  });
});

Database Testing

import { db } from '../database';

describe('UserRepository', () => {
  beforeEach(async () => {
    await db.query('DELETE FROM users');
  });

  afterAll(async () => {
    await db.end();
  });

  it('creates and retrieves user', async () => {
    const user = await userRepo.create({
      email: 'test@test.com',
      name: 'Test',
    });

    const found = await userRepo.findById(user.id);
    expect(found).toEqual(user);
  });
});

pytest API Testing

import pytest
from httpx import AsyncClient

@pytest.mark.asyncio
async def test_create_user(client: AsyncClient):
    response = await client.post("/api/users/", json={
        "email": "test@example.com",
        "name": "Test"
    })
    assert response.status_code == 201
    assert response.json()["email"] == "test@example.com"

@pytest.mark.asyncio
async def test_invalid_email(client: AsyncClient):
    response = await client.post("/api/users/", json={
        "email": "invalid",
        "name": "Test"
    })
    assert response.status_code == 422

Quick Reference

Performance Testing

k6 Load Test

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 20 },   // Ramp up to 20 users
    { duration: '1m', target: 20 },    // Stay at 20 users
    { duration: '30s', target: 0 },    // Ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'],  // 95% requests under 500ms
    http_req_failed: ['rate<0.01'],    // <1% errors
  },
};

export default function () {
  const res = http.get('http://localhost:3000/api/users');

  check(res, {
    'status is 200': (r) => r.status === 200,
    'response time < 200ms': (r) => r.timings.duration < 200,
  });

  sleep(1);
}

Stress Test

export const options = {
  stages: [
    { duration: '2m', target: 100 },   // Ramp to 100 users
    { duration: '5m', target: 100 },   // Stay at 100
    { duration: '2m', target: 200 },   // Push to 200
    { duration: '5m', target: 200 },   // Stay at 200
    { duration: '2m', target: 0 },     // Ramp down
  ],
};

Spike Test

export const options = {
  stages: [
    { duration: '10s', target: 10 },   // Normal load
    { duration: '1m', target: 10 },
    { duration: '10s', target: 200 },  // Spike!
    { duration: '3m', target: 200 },
    { duration: '10s', target: 10 },   // Scale down
    { duration: '3m', target: 10 },
    { duration: '10s', target: 0 },
  ],
};

API Testing with Auth

import http from 'k6/http';

export function setup() {
  const loginRes = http.post('http://localhost:3000/api/login', {
    email: 'test@test.com',
    password: 'password',
  });
  return { token: loginRes.json('token') };
}

export default function (data) {
  const params = {
    headers: { Authorization: `Bearer ${data.token}` },
  };

  http.get('http://localhost:3000/api/protected', params);
}

Thresholds Reference

thresholds: {
  // Response time
  http_req_duration: ['p(95)<500', 'p(99)<1000'],

  // Error rate
  http_req_failed: ['rate<0.01'],

  // Throughput
  http_reqs: ['rate>100'],

  // Custom metrics
  'http_req_duration{name:login}': ['p(95)<200'],
}

Quick Reference

QA Methodology

Manual Testing Types

Exploratory Testing

**Charter**: Explore {feature} with focus on {aspect}
**Duration**: 60-90 min
**Mission**: Find defects in {specific functionality}

Test Ideas:
- Boundary conditions & edge cases
- Error handling & recovery
- User workflow variations
- Integration points

Findings:
1. [HIGH] {Issue + impact}
2. [MED] {Issue + impact}

Coverage: {Areas explored} | Risks: {Identified risks}

Usability Testing

**Task**: Can users complete {action} intuitively?
**Metrics**: Time to complete, errors made, satisfaction (1-5)
**Success**: 80% complete without help in <5 min

Observations:
- Navigation confusing at {step}
- Users expect {A} but get {B}
- Positive: {feature feedback}

Accessibility Testing (WCAG 2.1 AA)

test('accessibility compliance', async ({ page }) => {
  // Keyboard navigation
  await page.keyboard.press('Tab');
  expect(['A', 'BUTTON', 'INPUT']).toContain(
    await page.evaluate(() => document.activeElement.tagName)
  );
  
  // ARIA labels
  expect(await page.getByRole('button').first().getAttribute('aria-label')).toBeTruthy();
  
  // Color contrast (axe-core)
  const violations = await page.evaluate(async () => {
    const axe = await import('axe-core');
    return (await axe.run()).violations;
  });
  expect(violations).toHaveLength(0);
});

Localization Testing

**Test**: {Feature} in {language/locale}
- [ ] Text displays without truncation
- [ ] Date/time/currency formats correct
- [ ] Right-to-left layout (Arabic, Hebrew)
- [ ] Character encoding UTF-8
- [ ] Sort order respects locale

Compatibility Matrix

| Browser | Version | OS | Status |
|---------|---------|----|----- --|
| Chrome | Latest | Win/Mac | ✓ |
| Firefox | Latest | Win/Mac | ✓ |
| Safari | Latest | macOS/iOS | ✓ |
| Edge | Latest | Windows | ✓ |

Test Design Techniques

Pairwise Testing

// Test all parameter pairs efficiently
const pairwiseTests = [
  { browser: 'chrome', os: 'windows', lang: 'en' },
  { browser: 'firefox', os: 'mac', lang: 'es' },
  { browser: 'safari', os: 'windows', lang: 'fr' },
  // Covers all pairs with minimal tests
];

Risk-Based Testing

| Risk | Probability | Impact | Priority | Test Effort |
|------|-------------|--------|----------|-------------|
| Critical | High | High | P0 | Exhaustive |
| High | Med-High | High | P1 | Comprehensive |
| Medium | Low-Med | Med | P2 | Standard |
| Low | Low | Low | P3 | Smoke only |

Defect Management

Root Cause Analysis (5 Whys)

1. Why did defect occur? {User input not validated}
2. Why wasn't it validated? {Validation logic missing}
3. Why was it missing? {Requirement unclear}
4. Why was requirement unclear? {Acceptance criteria incomplete}
5. Why incomplete? {No QA review in planning}

**Root Cause**: QA not involved in requirements phase
**Prevention**: Add QA to all planning meetings

Defect Report Template

## [CRITICAL] {Defect Title}

**Steps to Reproduce**:
1. {Step 1}
2. {Step 2}

**Expected**: {Should happen}
**Actual**: {Actually happens}
**Impact**: {Business/user impact}
**Root Cause**: {Why it happened}
**Fix**: {Recommended solution}

Quality Metrics

Key Calculations

// Defect Removal Efficiency (target: >95%)
const dre = (defectsInTesting / (defectsInTesting + defectsInProd)) * 100;

// Defect Leakage (target: <5%)
const leakage = (defectsInProd / totalDefects) * 100;

// Test Effectiveness (target: >90%)
const effectiveness = (defectsFoundByTests / totalDefects) * 100;

// Automation ROI
const roi = (timeSaved - maintenanceCost - developmentCost) / developmentCost;

Quality Dashboard

| Metric | Target | Actual | Trend | Status |
|--------|--------|--------|-------|--------|
| Coverage | >80% | 87% | ↑ | ✓ |
| Defect Leakage | <5% | 3% | ↓ | ✓ |
| Automation | >70% | 68% | ↑ | ⚠ |
| Critical Defects | 0 | 0 | → | ✓ |
| MTTR | <48h | 36h | ↓ | ✓ |

Continuous Testing & Shift-Left

Shift-Left Activities

**Early Testing**:
- Review requirements for testability
- Create test cases during design
- TDD: unit tests with code
- Automated tests in CI pipeline
- Static analysis on commit
- Security scanning pre-merge

**Benefits**: 10x cheaper defect fixes, faster feedback

Feedback Cycle Targets

const feedbackCycle = {
  unitTests: '< 5 min',       // On save
  integration: '< 15 min',    // On commit
  e2e: '< 30 min',            // On PR
  regression: '< 2 hours',    // Nightly
};

Quality Advocacy

Quality Gates

## Production Release Gate

**Must Pass (Blockers)**:
- [ ] Zero critical defects
- [ ] Coverage >80%
- [ ] All P0/P1 tests passing
- [ ] Performance SLA met
- [ ] Security scan clean
- [ ] Accessibility WCAG AA

**Decision**: GO | NO-GO | GO with exceptions

Team Education Program

**Week 1-2**: Test fundamentals
**Week 3-4**: Automation basics
**Week 5-6**: Advanced topics (perf, security, API)
**Ongoing**: Best practices, tool updates

Test Planning

Test Plan Template

## Test Plan: {Feature}

**Scope**: {What to test}
**Types**: Unit, Integration, E2E, Perf, Security
**Resources**: {Team allocation}
**Dependencies**: {Prerequisites}
**Schedule**: {Timeline}
**Entry Criteria**: {Start conditions}
**Exit Criteria**: {Completion conditions}
**Risks**: {Identified risks + mitigation}

Environment Strategy

| Env | Purpose | Data | Refresh | Access |
|-----|---------|------|---------|--------|
| Dev | Development | Synthetic | On-demand | All |
| Test | QA testing | Test data | Daily | QA |
| Stage | Pre-prod | Prod-like | Weekly | Limited |
| Prod | Live | Real | N/A | Ops |

Quick Reference

Security Testing

Authentication Tests

describe('Authentication Security', () => {
  it('rejects invalid credentials', async () => {
    await request(app)
      .post('/api/login')
      .send({ email: 'user@test.com', password: 'wrong' })
      .expect(401);
  });

  it('rejects expired tokens', async () => {
    const expiredToken = createExpiredToken();
    await request(app)
      .get('/api/protected')
      .set('Authorization', `Bearer ${expiredToken}`)
      .expect(401);
  });

  it('rejects tampered tokens', async () => {
    const tamperedToken = validToken.slice(0, -5) + 'xxxxx';
    await request(app)
      .get('/api/protected')
      .set('Authorization', `Bearer ${tamperedToken}`)
      .expect(401);
  });

  it('enforces rate limiting on login', async () => {
    for (let i = 0; i < 6; i++) {
      await request(app)
        .post('/api/login')
        .send({ email: 'user@test.com', password: 'wrong' });
    }

    await request(app)
      .post('/api/login')
      .send({ email: 'user@test.com', password: 'correct' })
      .expect(429);
  });
});

Authorization Tests

describe('Authorization', () => {
  it('denies access to other users resources', async () => {
    await request(app)
      .get('/api/users/other-user-id/data')
      .set('Authorization', `Bearer ${userAToken}`)
      .expect(403);
  });

  it('denies admin routes to regular users', async () => {
    await request(app)
      .delete('/api/admin/users/123')
      .set('Authorization', `Bearer ${regularUserToken}`)
      .expect(403);
  });
});

Input Validation Tests

describe('Input Validation', () => {
  it('rejects SQL injection attempts', async () => {
    await request(app)
      .get('/api/users')
      .query({ search: "'; DROP TABLE users; --" })
      .expect(400);
  });

  it('rejects XSS in input fields', async () => {
    const response = await request(app)
      .post('/api/posts')
      .send({ title: '<script>alert("xss")</script>' })
      .expect(201);

    expect(response.body.title).not.toContain('<script>');
  });

  it('validates file upload types', async () => {
    await request(app)
      .post('/api/upload')
      .attach('file', 'malicious.exe')
      .expect(400);
  });
});

Security Headers Test

describe('Security Headers', () => {
  it('sets security headers', async () => {
    const response = await request(app).get('/');

    expect(response.headers['x-content-type-options']).toBe('nosniff');
    expect(response.headers['x-frame-options']).toBe('DENY');
    expect(response.headers['strict-transport-security']).toBeDefined();
  });
});

Security Test Checklist

Quick Reference

TDD Iron Laws

---

The Fundamental Principle

NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST.

This is non-negotiable. If you wrote production code before writing a failing test, delete it and start over. No exceptions.

---

The Three Iron Laws

Iron Law 1: The Fundamental Rule

"You shall not write any production code unless it is to make a failing test pass."

Every line of production code must have a corresponding test that: 1. Was written first 2. Was observed to fail 3. Now passes because of that code

Iron Law 2: Proof Through Observation

"If you didn't watch the test fail, you don't know if it tests the right thing."

Mandatory verification steps:

• Write the test
• Run it and observe the failure
• Verify the failure message is meaningful
• Only then implement the fix

A test you've never seen fail proves nothing.

Iron Law 3: The Final Rule

"Production code exists → A test exists that failed first. Otherwise → It's not TDD."

There is no middle ground. Code written without a prior failing test is not test-driven development, regardless of how many tests exist afterward.

---

The RED-GREEN-REFACTOR Cycle

RED: Write One Minimal Failing Test

// Start with the smallest possible failing test
it('should return 0 for empty array', () => {
  expect(sum([])).toBe(0);
});
// Run: ✗ FAIL - sum is not defined

Requirements:

• One test at a time
• Minimal scope
• Clear failure message
• Observe the red

GREEN: Implement Simplest Passing Code

// Write only enough code to pass this specific test
function sum(numbers: number[]): number {
  return 0;
}
// Run: ✓ PASS

Requirements:

• Simplest possible implementation
• No extra features
• No optimization
• Just make it pass

REFACTOR: Improve While Keeping Tests Green

// Now improve the code while tests stay green
function sum(numbers: number[]): number {
  return numbers.reduce((acc, n) => acc + n, 0);
}
// Run: ✓ PASS (still)

Requirements:

• Tests must stay green
• Remove duplication
• Improve clarity
• No new functionality

---

Common Rationalizations to Reject

These thoughts indicate you're about to violate TDD:

---

Practical Application

Starting a New Feature

// 1. RED: Write failing test for simplest behavior
describe('UserValidator', () => {
  it('should reject empty email', () => {
    expect(validateEmail('')).toBe(false);
  });
});

// 2. GREEN: Implement minimal passing code
function validateEmail(email: string): boolean {
  return email.length > 0;
}

// 3. RED: Add next failing test
it('should reject email without @', () => {
  expect(validateEmail('invalid')).toBe(false);
});

// 4. GREEN: Extend to pass both tests
function validateEmail(email: string): boolean {
  return email.length > 0 && email.includes('@');
}

// Continue cycle...

Fixing a Bug

// 1. RED: Write test that exposes the bug
it('should handle negative numbers in sum', () => {
  expect(sum([-1, -2, -3])).toBe(-6);
});
// Run: ✗ FAIL - got 0 instead of -6

// 2. GREEN: Fix the bug
function sum(numbers: number[]): number {
  return numbers.reduce((acc, n) => acc + n, 0);
}
// Run: ✓ PASS

// Bug is now fixed AND protected against regression

---

Verification Checklist

Before claiming any code is complete:

• [ ] Every production function has corresponding tests
• [ ] Each test was written before its implementation
• [ ] Each test was observed to fail first
• [ ] Tests verify behavior, not implementation
• [ ] Refactoring kept all tests green
• [ ] No production code exists without a test

---

Content adapted from [obra/superpowers](https://github.com/obra/superpowers) by Jesse Vincent (@obra), MIT License.

Test Reports

Test Report Template

# Test Report: {Feature Name}

**Date**: YYYY-MM-DD
**Tester**: {Name}
**Version**: {App Version}

## Summary

| Metric | Value |
|--------|-------|
| Total Tests | X |
| Passed | X |
| Failed | X |
| Skipped | X |
| Coverage | X% |

## Test Scope

- [x] Unit tests
- [x] Integration tests
- [x] E2E tests
- [ ] Performance tests
- [ ] Security tests

## Findings

### [CRITICAL] {Issue Title}
- **Location**: src/api/users.ts:45
- **Steps to Reproduce**:
  1. Send POST to /api/users without auth
  2. Request succeeds with 201
- **Expected**: 401 Unauthorized
- **Actual**: 201 Created
- **Impact**: Unauthorized user creation
- **Fix**: Add auth middleware

### [HIGH] {Issue Title}
- **Location**: src/services/orders.ts:123
- **Description**: N+1 query in order list
- **Impact**: 3s response time with 100 orders
- **Fix**: Add eager loading for order items

### [MEDIUM] {Issue Title}
- **Details**: ...

### [LOW] {Issue Title}
- **Details**: ...

## Coverage Analysis

| Module | Lines | Branches | Functions |
|--------|-------|----------|-----------|
| api/ | 85% | 78% | 90% |
| services/ | 92% | 85% | 95% |
| utils/ | 100% | 100% | 100% |

### Coverage Gaps
- `src/api/admin.ts` - 0% (no tests)
- `src/services/payment.ts:45-60` - Error handling untested

## Recommendations

1. **Immediate**: Add auth middleware to admin routes
2. **High Priority**: Optimize order queries
3. **Medium Priority**: Add tests for payment error handling
4. **Low Priority**: Increase branch coverage in api/

## Performance Results

| Endpoint | p50 | p95 | p99 |
|----------|-----|-----|-----|
| GET /users | 45ms | 120ms | 250ms |
| POST /orders | 150ms | 400ms | 800ms |

## Sign-off

- [ ] All critical issues addressed
- [ ] Coverage meets threshold (80%)
- [ ] Performance meets SLA

Severity Definitions

Quick Reference

Testing Anti-Patterns

---

Core Principle

"Test what the code does, not what the mocks do."

When tests verify mock behavior instead of actual functionality, they provide false confidence while catching zero real bugs.

---

The Five Anti-Patterns

Anti-Pattern 1: Testing Mock Behavior

The Problem: Verifying that mocks exist and were called, rather than testing actual component output.

// ❌ BAD: Testing the mock, not the behavior
it('should call the API', () => {
  const mockApi = jest.fn().mockResolvedValue({ data: 'test' });
  const service = new UserService(mockApi);

  service.getUser(1);

  expect(mockApi).toHaveBeenCalledWith(1); // Testing mock, not result
});

// ✅ GOOD: Testing actual behavior
it('should return user data from API', async () => {
  const mockApi = jest.fn().mockResolvedValue({ id: 1, name: 'Alice' });
  const service = new UserService(mockApi);

  const user = await service.getUser(1);

  expect(user.name).toBe('Alice'); // Testing actual output
});

Solution: Test the genuine component output. If you can only verify mock calls, reconsider whether the test adds value.

---

Anti-Pattern 2: Test-Only Methods in Production

The Problem: Adding methods to production classes solely for test setup or cleanup.

// ❌ BAD: Production code polluted with test concerns
class UserCache {
  private cache: Map<number, User> = new Map();

  getUser(id: number): User | undefined {
    return this.cache.get(id);
  }

  // This method exists ONLY for tests
  _resetForTesting(): void {
    this.cache.clear();
  }
}

// ✅ GOOD: Test utilities separate from production
// production/UserCache.ts
class UserCache {
  private cache: Map<number, User> = new Map();

  getUser(id: number): User | undefined {
    return this.cache.get(id);
  }
}

// test/helpers.ts
function createFreshCache(): UserCache {
  return new UserCache(); // Fresh instance per test
}

Solution: Relocate cleanup logic to test utility functions. Use fresh instances per test instead of reset methods.

---

Anti-Pattern 3: Mocking Without Understanding

The Problem: Over-mocking without grasping side effects, leading to tests that pass but hide real issues.

// ❌ BAD: Mocking everything without understanding
it('should process order', async () => {
  jest.mock('./inventory');
  jest.mock('./payment');
  jest.mock('./shipping');
  jest.mock('./notifications');

  const result = await processOrder(order);

  expect(result.success).toBe(true); // What did we actually test?
});

// ✅ GOOD: Strategic mocking with real components where possible
it('should process order with real inventory check', async () => {
  // Real inventory service against test database
  const inventory = new InventoryService(testDb);

  // Mock only external services
  const payment = mockPaymentGateway();

  const processor = new OrderProcessor(inventory, payment);
  const result = await processor.process(order);

  expect(result.success).toBe(true);
  expect(await inventory.getStock(order.itemId)).toBe(originalStock - 1);
});

Solution: Run tests with real implementations first to understand behavior. Then mock at the appropriate level - external services, not internal logic.

---

Anti-Pattern 4: Incomplete Mocks

The Problem: Partial mock responses missing downstream fields that production code expects.

// ❌ BAD: Incomplete mock response
const mockUserApi = jest.fn().mockResolvedValue({
  id: 1,
  name: 'Test User'
  // Missing: email, createdAt, permissions, settings...
});

// Test passes, but production crashes when accessing user.email

// ✅ GOOD: Complete mock matching real API response
const mockUserApi = jest.fn().mockResolvedValue({
  id: 1,
  name: 'Test User',
  email: 'test@example.com',
  createdAt: '2024-01-01T00:00:00Z',
  permissions: ['read', 'write'],
  settings: {
    theme: 'light',
    notifications: true
  }
});

// Or use a factory
const mockUserApi = jest.fn().mockResolvedValue(
  createMockUser({ name: 'Test User' }) // Factory fills defaults
);

Solution: Mirror complete real API response structure. Use factories to generate complete mock objects with sensible defaults.

---

Anti-Pattern 5: Integration Tests as Afterthought

The Problem: Treating testing as optional follow-up work rather than integral to development.

// ❌ BAD: "We'll add tests later"
// Day 1: Write 500 lines of code
// Day 2: Write 500 more lines
// Day 3: "We need to ship, tests can wait"
// Day 30: Catastrophic bug in production
// Day 31: "Why didn't we have tests?"

// ✅ GOOD: Tests are part of implementation
// Write failing test
it('should reject duplicate usernames', async () => {
  await createUser({ username: 'alice' });

  await expect(createUser({ username: 'alice' }))
    .rejects.toThrow('Username already exists');
});

// Make it pass
async function createUser(data: UserInput): Promise<User> {
  const existing = await db.users.findByUsername(data.username);
  if (existing) {
    throw new Error('Username already exists');
  }
  return db.users.create(data);
}

// Feature AND test ship together

Solution: Follow TDD - testing is implementation, not documentation. No feature is "done" without tests.

---

Detection Checklist

Review your tests for these warning signs:

---

Quick Reference

---

Content adapted from [obra/superpowers](https://github.com/obra/superpowers) by Jesse Vincent (@obra), MIT License.

Unit Testing

Jest/Vitest Pattern

describe('UserService', () => {
  let service: UserService;
  let mockRepo: jest.Mocked<UserRepository>;

  beforeEach(() => {
    mockRepo = { findById: jest.fn(), save: jest.fn() } as any;
    service = new UserService(mockRepo);
  });

  afterEach(() => jest.clearAllMocks());

  describe('getUser', () => {
    it('returns user when found', async () => {
      const user = { id: '1', name: 'Test' };
      mockRepo.findById.mockResolvedValue(user);

      const result = await service.getUser('1');

      expect(result).toEqual(user);
      expect(mockRepo.findById).toHaveBeenCalledWith('1');
    });

    it('throws NotFoundError when user not found', async () => {
      mockRepo.findById.mockResolvedValue(null);

      await expect(service.getUser('1')).rejects.toThrow(NotFoundError);
    });
  });
});

pytest Pattern

import pytest
from unittest.mock import Mock, AsyncMock

class TestUserService:
    @pytest.fixture
    def mock_repo(self):
        return Mock()

    @pytest.fixture
    def service(self, mock_repo):
        return UserService(mock_repo)

    async def test_get_user_returns_user(self, service, mock_repo):
        mock_repo.find_by_id = AsyncMock(return_value={"id": "1", "name": "Test"})

        result = await service.get_user("1")

        assert result == {"id": "1", "name": "Test"}
        mock_repo.find_by_id.assert_called_once_with("1")

    async def test_get_user_raises_not_found(self, service, mock_repo):
        mock_repo.find_by_id = AsyncMock(return_value=None)

        with pytest.raises(NotFoundError):
            await service.get_user("1")

Mocking Patterns

// Mock functions
const mockFn = jest.fn();
mockFn.mockReturnValue('value');
mockFn.mockResolvedValue('async value');
mockFn.mockRejectedValue(new Error('error'));

// Mock modules
jest.mock('./database', () => ({
  query: jest.fn(),
}));

// Spy on existing methods
jest.spyOn(console, 'log').mockImplementation(() => {});

Test Organization

describe('Feature', () => {
  describe('happy path', () => {
    it('does expected behavior', () => {});
  });

  describe('edge cases', () => {
    it('handles empty input', () => {});
    it('handles max values', () => {});
  });

  describe('error cases', () => {
    it('throws on invalid input', () => {});
  });
});

Quick Reference