Qa Testing Strategy

QA Testing Strategy (Jan 2026)

Risk-based quality engineering strategy for modern software delivery.

Core references: curated links in data/sources.json (SLOs/error budgets, contracts, E2E, OpenTelemetry). Start with references/operational-playbook.md for a compact, navigable overview.

Scope

• Create or update a risk-based test strategy (what to test, where, and why)
• Define quality gates and release criteria (merge vs deploy)
• Select the smallest effective layer (unit → integration → contract → E2E)
• Make failures diagnosable (artifacts, logs/traces, ownership)
• Operationalize reliability (flake SLO, quarantines, suite budgets)

Use Instead

Quick Reference

Default Workflow

1. Clarify scope and risk: critical journeys, failure modes, and non-functional risks (latency, data loss, auth). 2. Define quality signals: SLOs/error budgets, contract/schema checks, and what blocks merge vs blocks deploy. 3. Choose the smallest effective layer (unit → integration → contract → E2E). 4. Make failures diagnosable: artifacts + correlation IDs (logs/traces/screenshots), clear ownership, deflake runbook. 5. Operationalize: flake SLO, quarantine with expiry, suite budgets (PR gate vs scheduled), dashboards.

Test Pyramid

           /\
          /E2E\          5-10% - Critical journeys
         /------\
        /Integr. \       15-25% - API, DB, queues
       /----------\
      /Component \       20-30% - UI modules
     /------------\
    /   Unit      \      40-60% - Logic and invariants
   /--------------\

Decision Tree: Test Strategy

Need to test: [Feature Type]
    │
    ├─ Pure business logic/invariants? → Unit tests (mock boundaries)
    │
    ├─ UI component/state transitions? → Component tests
    │   └─ Cross-page user journey? → E2E tests
    │
    ├─ API Endpoint?
    │   ├─ Single service boundary? → Integration tests (real DB/deps)
    │   └─ Cross-service compatibility? → Contract tests (schema/versioning)
    │
    ├─ Event-driven/API schema evolution? → Contract + backward-compat tests
    │
    └─ Performance-critical? → k6 load testing

Core QA Principles

Definition of Done

• Strategy is risk-based: critical journeys + failure modes explicit
• Test portfolio is layered: fast checks catch most defects
• CI is economical: fast pre-merge gates, heavy suites scheduled
• Failures are diagnosable: actionable artifacts (logs/trace/screenshots)
• Flakes managed with SLO and deflake runbook

Shift-Left Gates (Pre-Merge)

• Contracts: OpenAPI/AsyncAPI/JSON Schema validation
• Static checks: lint, typecheck, secret scanning
• Fast tests: unit + key integration (avoid full E2E as PR gate)

Shift-Right (Post-Deploy)

• Synthetic checks for critical paths (monitoring-as-tests)
• Canary analysis: compare SLO signals and key metrics before ramping
• Feature flags for safe rollouts and fast rollback
• Convert incidents into regression tests (prefer lower layers first)

CI Economics

Flake Management

• Define: test fails without product change, passes on rerun
• Track weekly: flaky_failures / total_test_executions (where flaky_failure = fail_then_pass_on_rerun)
• SLO: Suite flake rate ≤ 1% weekly
• Quarantine policy with owner and expiry
• Use the deflake runbook: template-flaky-test-triage-deflake-runbook.md

Common Patterns

AAA Pattern

it('should apply discount', () => {
  // Arrange
  const order = { total: 150 };
  // Act
  const result = calculateDiscount(order);
  // Assert
  expect(result.discount).toBe(15);
});

Page Object Model (E2E)

class LoginPage {
  async login(email: string, password: string) {
    await this.page.fill('[data-testid="email"]', email);
    await this.page.fill('[data-testid="password"]', password);
    await this.page.click('[data-testid="submit"]');
  }
}

Anti-Patterns

Do / Avoid

Do

• Write tests against stable contracts and user-visible behavior
• Treat flaky tests as P1 reliability work
• Make "how to debug this failure" part of every suite

Avoid

• "Everything E2E" as default
• Sleeps/time-based waits (use event-based)
• Coverage % as primary quality KPI

Feature Matrix vs Test Matrix Gate (Release Blocking)

Before release, run a coverage audit that maps product features/backlog IDs to direct test evidence.

Gate Rules

• Every release-scoped feature must map to at least one direct automated test, or an explicit waiver with owner/date.
• Evidence must include file path and test identifier (suite/spec/case).
• "Covered indirectly" is not accepted without written rationale and risk acknowledgment.
• If critical features have no direct evidence, release is blocked.

Minimal Audit Output

• feature/backlog id
• coverage status (direct, indirect, none)
• evidence reference
• risk level
• owner and due date for gaps

Resources

Templates

Data

Related Skills

• qa-debugging — Debugging failing tests
• qa-agent-testing — Testing AI agents
• software-backend — API patterns to test
• ops-devops-platform — CI/CD pipelines

Ops Gate: Release-Safe Verification Sequence

Use this sequence for feature branches that touch user flows, pricing, localization, or analytics.

# 1) Static checks
npm run lint
npm run typecheck

# 2) Fast correctness
npm run test:unit

# 3) Critical path checks
npm run test:e2e -- --grep "@critical"

# 4) Instrumentation gate (if configured)
npm run test:analytics-gate

# 5) Production build
npm run build

If a Gate Fails

1. Capture exact failing command and first error line. 2. Classify: environment issue, baseline known failure, or regression. 3. Re-run only the failed gate once after fix. 4. Do not continue to later gates while earlier required gates are red.

Agent Output Contract for QA Handoff

Always report:

• commands run,
• pass/fail per gate,
• whether failures are pre-existing or introduced,
• next blocking action.

Fact-Checking

• Use web search/web fetch to verify current external facts, versions, pricing, deadlines, regulations, or platform behavior before final answers.
• Prefer primary sources; report source links and dates for volatile information.
• If web access is unavailable, state the limitation and mark guidance as unverified.

Automation Pipeline Template

• Triggering events: PR open/update, nightly, release branch, hotfix path
• Stages: Lint → unit → component/contract → integration → E2E → performance → security scans
• Parallelization: Which suites can parallelize; shard strategy; caching plan
• Environment setup: Containers/services required, seeding scripts, secrets handling
• Quality gates: Required checks, coverage thresholds, allowed flake rate, blocking vs warning jobs
• Artifacts: Test reports, coverage, screenshots/videos, traces, SBOMs
• Rollbacks: What to do on failure; auto-revert, feature flag toggles, chat notifications
• Governance: Owners, escalation path, maintenance cadence for dependencies and flaky tests

BDD Testing Template: Cucumber & Gherkin

Use this template for behavior-driven development (BDD) with Cucumber and Gherkin syntax to create executable specifications.

Why BDD (2024 Best Practices)

Benefits:

• Living documentation (scenarios are always up-to-date)
• Collaboration between technical and non-technical stakeholders
• Clear acceptance criteria before development
• Executable specifications
• Shared understanding of requirements

When to use: Acceptance tests, E2E critical paths, stakeholder-facing features

When NOT to use: Unit tests (use code directly), implementation details

Basic Gherkin Syntax

# features/user-login.feature
Feature: User Login
  As a registered user
  I want to log in to my account
  So that I can access my personalized dashboard

  Background:
    Given the application is running
    And I am on the login page

  Scenario: Successful login with valid credentials
    When I enter email "user@example.com"
    And I enter password "SecurePass123"
    And I click the "Login" button
    Then I should see my dashboard
    And I should see "Welcome back, John"

  Scenario: Failed login with invalid password
    When I enter email "user@example.com"
    And I enter password "WrongPassword"
    And I click the "Login" button
    Then I should see an error "Invalid credentials"
    And I should remain on the login page

  Scenario: Account lockout after multiple failed attempts
    When I enter email "user@example.com"
    And I enter password "WrongPassword"
    And I click the "Login" button 3 times
    Then I should see an error "Account temporarily locked"
    And I should not be able to log in for 15 minutes

Step Definitions (TypeScript)

// step-definitions/login.steps.ts
import { Given, When, Then, Before, After } from '@cucumber/cucumber'
import { expect } from '@playwright/test'
import { LoginPage } from '../pages/login.page'

let page: Page
let loginPage: LoginPage

Before(async function () {
  page = await this.browser.newPage()
  loginPage = new LoginPage(page)
})

After(async function () {
  await page.close()
})

Given('the application is running', async function () {
  // Verify app health endpoint
  const response = await page.request.get('https://api.example.com/health')
  expect(response.status()).toBe(200)
})

Given('I am on the login page', async function () {
  await page.goto('/login')
  await expect(page.getByRole('heading', { name: 'Login' })).toBeVisible()
})

When('I enter email {string}', async function (email: string) {
  await page.getByLabel('Email').fill(email)
})

When('I enter password {string}', async function (password: string) {
  await page.getByLabel('Password').fill(password)
})

When('I click the {string} button', async function (buttonText: string) {
  await page.getByRole('button', { name: buttonText }).click()
})

When('I click the {string} button {int} times', async function (buttonText: string, times: number) {
  for (let i = 0; i < times; i++) {
    await page.getByRole('button', { name: buttonText }).click()
    await page.waitForTimeout(1000)
  }
})

Then('I should see my dashboard', async function () {
  await expect(page).toHaveURL('/dashboard')
  await expect(page.getByTestId('dashboard')).toBeVisible()
})

Then('I should see {string}', async function (text: string) {
  await expect(page.getByText(text)).toBeVisible()
})

Then('I should see an error {string}', async function (errorMessage: string) {
  await expect(page.getByRole('alert')).toContainText(errorMessage)
})

Then('I should remain on the login page', async function () {
  await expect(page).toHaveURL('/login')
})

Then('I should not be able to log in for {int} minutes', async function (minutes: number) {
  // Store context for future validation
  this.lockoutDuration = minutes
  const lockoutMessage = await page.getByTestId('lockout-message').textContent()
  expect(lockoutMessage).toContain(`${minutes} minutes`)
})

Scenario Outlines (Data-Driven Tests)

Feature: Shopping Cart

  Scenario Outline: Apply discount codes
    Given I have "<item>" in my cart with price <price>
    When I apply discount code "<code>"
    Then the total should be <total>
    And I should see discount message "<message>"

    Examples:
      | item      | price | code     | total | message                |
      | Laptop    | 1000  | SAVE10   | 900   | 10% discount applied   |
      | Mouse     | 50    | SAVE10   | 45    | 10% discount applied   |
      | Laptop    | 1000  | SAVE50   | 500   | 50% discount applied   |
      | Mouse     | 50    | INVALID  | 50    | Invalid discount code  |
      | Keyboard  | 100   |          | 100   | No discount applied    |

  Scenario Outline: Validate product search
    Given I am on the products page
    When I search for "<query>"
    Then I should see <result_count> results
    And the first result should be "<first_result>"

    Examples:
      | query     | result_count | first_result       |
      | laptop    | 15           | MacBook Pro        |
      | mouse     | 42           | Logitech MX Master |
      | keyboard  | 28           | Mechanical Keyboard|
      | monitor   | 31           | Dell UltraSharp    |
      | invalid   | 0            |                    |

Tags for Organization

@smoke @critical
Feature: User Authentication

  @happy-path
  Scenario: Successful login
    # ...

  @error-handling
  Scenario: Invalid credentials
    # ...

  @security @slow
  Scenario: Account lockout
    # ...

  @wip
  Scenario: Two-factor authentication
    # Work in progress

# Run specific tags
npm run test:cucumber -- --tags "@smoke"
npm run test:cucumber -- --tags "@critical and not @slow"
npm run test:cucumber -- --tags "@smoke or @regression"

Best Practices: Writing Good Gherkin

GOOD: Declarative (Focus on WHAT, not HOW)

# Good - Describes behavior from user perspective
Scenario: User completes checkout
  Given I have items in my cart
  When I complete the checkout process
  Then my order should be confirmed

# Bad - Implementation details (HOW)
Scenario: User completes checkout
  Given I click the cart icon
  And I see the cart page
  When I click the "Checkout" button
  And I fill in field "address" with "123 Main St"
  And I fill in field "city" with "San Francisco"
  And I click the "Submit" button
  Then I should see element with id "confirmation"

GOOD: Independent Scenarios

# Good - Self-contained
Scenario: Delete user account
  Given I have a user account
  When I request account deletion
  Then my account should be deleted

# Bad - Depends on previous scenario
Scenario: Delete user account
  # Assumes account was created in previous scenario
  When I request account deletion
  Then my account should be deleted

GOOD: Use Background for Common Setup

Feature: Product Management

  Background:
    Given I am logged in as an admin
    And I am on the products page

  Scenario: Add new product
    When I create a product with name "Laptop"
    Then I should see "Laptop" in the product list

  Scenario: Edit product
    Given I have a product "Mouse"
    When I edit the product name to "Wireless Mouse"
    Then I should see "Wireless Mouse" in the product list

Data Tables

Scenario: Create user with complete profile
  When I create a user with the following details:
    | field      | value                |
    | name       | John Doe             |
    | email      | john@example.com     |
    | age        | 30                   |
    | country    | USA                  |
    | role       | admin                |
  Then the user should be created successfully

Scenario: Bulk create users
  When I create the following users:
    | name       | email                | role   |
    | Alice      | alice@example.com    | user   |
    | Bob        | bob@example.com      | admin  |
    | Charlie    | charlie@example.com  | user   |
  Then all users should be created successfully

// Step definition for data tables
When('I create a user with the following details:', async function (dataTable) {
  const userData = dataTable.rowsHash()
  await this.api.post('/users', userData)
})

When('I create the following users:', async function (dataTable) {
  const users = dataTable.hashes()
  for (const user of users) {
    await this.api.post('/users', user)
  }
})

Hooks for Setup/Teardown

// support/hooks.ts
import { Before, After, BeforeAll, AfterAll, Status } from '@cucumber/cucumber'

BeforeAll(async function () {
  // Global setup (runs once before all scenarios)
  console.log('Starting test suite')
})

AfterAll(async function () {
  // Global teardown (runs once after all scenarios)
  console.log('Test suite completed')
})

Before(async function () {
  // Setup before each scenario
  this.startTime = Date.now()
})

After(async function (scenario) {
  // Teardown after each scenario
  const duration = Date.now() - this.startTime
  console.log(`Scenario "${scenario.pickle.name}" took ${duration}ms`)

  // Take screenshot on failure
  if (scenario.result?.status === Status.FAILED) {
    const screenshot = await this.page.screenshot()
    this.attach(screenshot, 'image/png')
  }
})

// Tagged hooks
Before({ tags: '@database' }, async function () {
  await this.db.clear()
})

After({ tags: '@database' }, async function () {
  await this.db.close()
})

Custom World (Shared Context)

// support/world.ts
import { setWorldConstructor, World, IWorldOptions } from '@cucumber/cucumber'
import { chromium, Browser, Page } from '@playwright/test'

export class CustomWorld extends World {
  browser?: Browser
  page?: Page
  apiResponse?: any
  testData: Map<string, any>

  constructor(options: IWorldOptions) {
    super(options)
    this.testData = new Map()
  }

  async init() {
    this.browser = await chromium.launch()
    const context = await this.browser.newContext()
    this.page = await context.newPage()
  }

  async cleanup() {
    await this.page?.close()
    await this.browser?.close()
  }

  // Helper methods
  async login(email: string, password: string) {
    await this.page!.goto('/login')
    await this.page!.getByLabel('Email').fill(email)
    await this.page!.getByLabel('Password').fill(password)
    await this.page!.getByRole('button', { name: 'Login' }).click()
  }

  storeData(key: string, value: any) {
    this.testData.set(key, value)
  }

  getData(key: string) {
    return this.testData.get(key)
  }
}

setWorldConstructor(CustomWorld)

Configuration

// cucumber.config.ts
export default {
  require: ['step-definitions/**/*.ts'],
  requireModule: ['ts-node/register'],
  format: [
    'progress-bar',
    'html:test-results/cucumber-report.html',
    'json:test-results/cucumber-report.json',
    'junit:test-results/cucumber-report.xml'
  ],
  formatOptions: {
    snippetInterface: 'async-await'
  },
  parallel: 2,
  retry: 1,
  retryTagFilter: '@flaky'
}

Integration with CI/CD

# .github/workflows/bdd-tests.yml
name: BDD Tests

on: [push, pull_request]

jobs:
  cucumber-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Run Cucumber tests
        run: npm run test:cucumber

      - name: Publish test results
        uses: EnricoMi/publish-unit-test-result-action@v2
        if: always()
        with:
          files: test-results/cucumber-report.xml

      - name: Upload HTML report
        uses: actions/upload-artifact@v3
        if: always()
        with:
          name: cucumber-report
          path: test-results/cucumber-report.html

Common Patterns Checklist

• [ ] Write scenarios from user perspective (declarative)
• [ ] Keep scenarios independent (no dependencies)
• [ ] Use Background for common setup
• [ ] Use Scenario Outline for data-driven tests
• [ ] Use tags for organization (@smoke, @regression, @wip)
• [ ] Implement reusable step definitions
• [ ] Use Custom World for shared context
• [ ] Add screenshots on failure
• [ ] Write one assertion per Then step
• [ ] Avoid brittle implementation details

Anti-Patterns to Avoid

[FAIL] Overly specific scenarios:

# Too detailed
When I click the button with id "submit-btn-123"
And I wait 2 seconds
Then I should see element with class "success-message"

[OK] User-focused scenarios:

# Better
When I submit the form
Then I should see a success message

[FAIL] Reusing steps inappropriately:

# Confusing reuse
Given I am on the login page
And I am on the products page  # Which page am I on?

[FAIL] Testing too much in one scenario:

# Too much in one scenario (split into 3 scenarios)
Scenario: Complete user journey
  Given I register a new account
  And I log in
  And I add products to cart
  And I checkout
  And I view order history
  And I update my profile
  # ... 20 more steps

Related Resources

See ../../references/shift-left-testing.md for writing scenarios in requirements phase, and ../e2e/template-playwright.md for implementing step definitions with Playwright.

E2E Testing Template: Playwright

Use this template for end-to-end testing with Playwright for cross-browser automation.

Why Playwright (2024-2025)

Advantages:

• Cross-browser support (Chromium, Firefox, WebKit)
• Auto-wait and retry mechanisms
• Parallel execution by default
• Network interception and mocking
• Multiple contexts (auth, sessions)
• Mobile device emulation
• Video/screenshot/trace recording

Basic Test Structure

// tests/checkout.spec.ts
import { test, expect } from '@playwright/test'

test.describe('Checkout Flow', () => {
  test.beforeEach(async ({ page }) => {
    // Setup: Navigate and authenticate
    await page.goto('/')
    await page.getByRole('button', { name: 'Sign In' }).click()
    await page.getByLabel('Email').fill('test@example.com')
    await page.getByLabel('Password').fill('password123')
    await page.getByRole('button', { name: 'Login' }).click()
    await expect(page.getByText('Welcome back')).toBeVisible()
  })

  test('user can complete purchase', async ({ page }) => {
    // Add item to cart
    await page.goto('/products/laptop')
    await page.getByRole('button', { name: 'Add to Cart' }).click()
    await expect(page.getByTestId('cart-count')).toHaveText('1')

    // Go to checkout
    await page.getByRole('link', { name: 'Cart' }).click()
    await page.getByRole('button', { name: 'Proceed to Checkout' }).click()

    // Fill shipping info
    await page.getByLabel('Address').fill('123 Main St')
    await page.getByLabel('City').fill('San Francisco')
    await page.getByLabel('ZIP Code').fill('94102')
    await page.getByRole('button', { name: 'Continue' }).click()

    // Fill payment info
    await page.getByLabel('Card Number').fill('4242424242424242')
    await page.getByLabel('Expiry').fill('12/25')
    await page.getByLabel('CVV').fill('123')

    // Submit order
    await page.getByRole('button', { name: 'Place Order' }).click()

    // Verify success
    await expect(page.getByRole('heading', { name: 'Order Confirmed' })).toBeVisible()
    await expect(page.getByTestId('order-number')).toContainText('ORDER-')
  })

  test('should validate required fields', async ({ page }) => {
    await page.goto('/checkout')

    // Try to proceed without filling fields
    await page.getByRole('button', { name: 'Continue' }).click()

    // Expect validation errors
    await expect(page.getByText('Address is required')).toBeVisible()
    await expect(page.getByText('City is required')).toBeVisible()
  })
})

Page Object Model (POM)

// pages/checkout.page.ts
import { Page, Locator } from '@playwright/test'

export class CheckoutPage {
  readonly page: Page
  readonly addressInput: Locator
  readonly cityInput: Locator
  readonly zipInput: Locator
  readonly continueButton: Locator
  readonly cardNumberInput: Locator
  readonly expiryInput: Locator
  readonly cvvInput: Locator
  readonly placeOrderButton: Locator
  readonly orderConfirmation: Locator

  constructor(page: Page) {
    this.page = page
    this.addressInput = page.getByLabel('Address')
    this.cityInput = page.getByLabel('City')
    this.zipInput = page.getByLabel('ZIP Code')
    this.continueButton = page.getByRole('button', { name: 'Continue' })
    this.cardNumberInput = page.getByLabel('Card Number')
    this.expiryInput = page.getByLabel('Expiry')
    this.cvvInput = page.getByLabel('CVV')
    this.placeOrderButton = page.getByRole('button', { name: 'Place Order' })
    this.orderConfirmation = page.getByRole('heading', { name: 'Order Confirmed' })
  }

  async fillShippingInfo(address: string, city: string, zip: string) {
    await this.addressInput.fill(address)
    await this.cityInput.fill(city)
    await this.zipInput.fill(zip)
    await this.continueButton.click()
  }

  async fillPaymentInfo(cardNumber: string, expiry: string, cvv: string) {
    await this.cardNumberInput.fill(cardNumber)
    await this.expiryInput.fill(expiry)
    await this.cvvInput.fill(cvv)
  }

  async placeOrder() {
    await this.placeOrderButton.click()
    await this.orderConfirmation.waitFor()
  }
}

// Usage in tests
import { CheckoutPage } from './pages/checkout.page'

test('checkout with POM', async ({ page }) => {
  const checkoutPage = new CheckoutPage(page)
  await page.goto('/checkout')

  await checkoutPage.fillShippingInfo('123 Main St', 'San Francisco', '94102')
  await checkoutPage.fillPaymentInfo('4242424242424242', '12/25', '123')
  await checkoutPage.placeOrder()

  await expect(checkoutPage.orderConfirmation).toBeVisible()
})

Authentication State Reuse

// auth.setup.ts
import { test as setup } from '@playwright/test'

const authFile = 'playwright/.auth/user.json'

setup('authenticate', async ({ page }) => {
  await page.goto('/login')
  await page.getByLabel('Email').fill('test@example.com')
  await page.getByLabel('Password').fill('password123')
  await page.getByRole('button', { name: 'Sign In' }).click()
  await page.waitForURL('/dashboard')

  // Save storage state
  await page.context().storageState({ path: authFile })
})

// playwright.config.ts
export default defineConfig({
  projects: [
    { name: 'setup', testMatch: /.*\.setup\.ts/ },
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
        storageState: authFile
      },
      dependencies: ['setup']
    }
  ]
})

// Now all tests run with authenticated state
test('access protected page', async ({ page }) => {
  await page.goto('/dashboard') // Already authenticated
  await expect(page.getByText('Welcome back')).toBeVisible()
})

API Mocking

test('mock API responses', async ({ page }) => {
  // Mock API call
  await page.route('**/api/products', async route => {
    await route.fulfill({
      status: 200,
      contentType: 'application/json',
      body: JSON.stringify([
        { id: 1, name: 'Laptop', price: 999 },
        { id: 2, name: 'Mouse', price: 29 }
      ])
    })
  })

  await page.goto('/products')
  await expect(page.getByText('Laptop')).toBeVisible()
  await expect(page.getByText('$999')).toBeVisible()
})

test('simulate API error', async ({ page }) => {
  await page.route('**/api/products', route => {
    route.fulfill({ status: 500, body: 'Internal Server Error' })
  })

  await page.goto('/products')
  await expect(page.getByText('Failed to load products')).toBeVisible()
})

Mobile Testing

import { devices } from '@playwright/test'

test.use({ ...devices['iPhone 13 Pro'] })

test('mobile navigation', async ({ page }) => {
  await page.goto('/')

  // Mobile menu
  await page.getByRole('button', { name: 'Menu' }).click()
  await expect(page.getByRole('navigation')).toBeVisible()

  // Test mobile-specific features
  await page.getByRole('link', { name: 'Products' }).click()
  await expect(page).toHaveURL(/.*products/)
})

Visual Testing

test('visual regression', async ({ page }) => {
  await page.goto('/products')

  // Full page screenshot
  await expect(page).toHaveScreenshot('products-page.png')

  // Element screenshot
  const product = page.getByTestId('product-1')
  await expect(product).toHaveScreenshot('product-card.png')
})

Parallel Execution

// playwright.config.ts
export default defineConfig({
  workers: process.env.CI ? 2 : undefined, // 2 workers in CI, all CPU cores locally
  fullyParallel: true,
  retries: process.env.CI ? 2 : 0
})

// Shard tests across machines
// Machine 1:
// npx playwright test --shard=1/3

// Machine 2:
// npx playwright test --shard=2/3

// Machine 3:
// npx playwright test --shard=3/3

Tracing and Debugging

// playwright.config.ts
export default defineConfig({
  use: {
    trace: 'on-first-retry', // or 'on', 'off', 'retain-on-failure'
    video: 'retain-on-failure',
    screenshot: 'only-on-failure'
  }
})

// View trace
// npx playwright show-trace trace.zip

// Debug mode
// npx playwright test --debug

// Headed mode
// npx playwright test --headed

Accessibility Testing

import { test, expect } from '@playwright/test'
import AxeBuilder from '@axe-core/playwright'

test('homepage should be accessible', async ({ page }) => {
  await page.goto('/')

  const accessibilityScanResults = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa'])
    .analyze()

  expect(accessibilityScanResults.violations).toEqual([])
})

Network Interception

test('track network requests', async ({ page }) => {
  const requests: string[] = []

  page.on('request', request => {
    if (request.url().includes('/api/')) {
      requests.push(request.url())
    }
  })

  await page.goto('/dashboard')

  expect(requests).toContain('https://api.example.com/api/user')
  expect(requests).toContain('https://api.example.com/api/products')
})

test('wait for specific API call', async ({ page }) => {
  await page.goto('/products')

  // Wait for API response
  const responsePromise = page.waitForResponse(
    response => response.url().includes('/api/products') && response.status() === 200
  )

  await page.getByRole('button', { name: 'Load More' }).click()
  await responsePromise // Wait for API call to complete
})

Best Practices

// GOOD: Use data-testid for stable selectors
await page.getByTestId('submit-button').click()

// GOOD: Use role and accessible name
await page.getByRole('button', { name: 'Submit' }).click()

// BAD: Avoid CSS selectors (brittle)
await page.locator('.btn.btn-primary').click()

// GOOD: Use auto-waiting (built-in retries)
await page.getByText('Loading...').waitFor({ state: 'hidden' })
await page.getByText('Content loaded').waitFor()

// GOOD: Test user journeys, not implementation
test('user can purchase product', async ({ page }) => {
  // Focus on user actions and outcomes
})

// BAD: Don't test internal state
test('cart state updated', async ({ page }) => {
  // Avoid testing Redux store or internal state
})

Configuration Template

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test'

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 2 : undefined,
  reporter: [
    ['html'],
    ['json', { outputFile: 'test-results.json' }],
    ['junit', { outputFile: 'junit.xml' }]
  ],
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
    trace: 'on-first-retry',
    video: 'retain-on-failure',
    screenshot: 'only-on-failure'
  },
  projects: [
    { name: 'setup', testMatch: /.*\.setup\.ts/ },
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'], storageState: 'playwright/.auth/user.json' },
      dependencies: ['setup']
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'], storageState: 'playwright/.auth/user.json' },
      dependencies: ['setup']
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'], storageState: 'playwright/.auth/user.json' },
      dependencies: ['setup']
    },
    {
      name: 'Mobile Chrome',
      use: { ...devices['Pixel 5'], storageState: 'playwright/.auth/user.json' },
      dependencies: ['setup']
    }
  ],
  webServer: {
    command: 'npm run dev',
    url: 'http://localhost:3000',
    reuseExistingServer: !process.env.CI
  }
})

Common Patterns Checklist

• [ ] Use Page Object Model for reusable page interactions
• [ ] Reuse authentication state across tests
• [ ] Mock external APIs for reliability
• [ ] Test critical user journeys only (not every page)
• [ ] Use data-testid or ARIA roles for selectors
• [ ] Enable trace/video for debugging failures
• [ ] Run tests in parallel
• [ ] Test across multiple browsers
• [ ] Include mobile device testing
• [ ] Set up retry logic for flaky tests

Related Resources

See ../../references/comprehensive-testing-guide.md for E2E testing strategies and ../bdd/template-cucumber-gherkin.md for BDD integration.

Integration Testing Template: API Integration Tests

Use this template for testing API integrations, service communication, and database interactions.

Framework Selection

Supertest + Jest/Vitest - Best for:

• REST API testing with Express/Fastify
• HTTP request/response validation
• Middleware testing
• Integration with existing Jest/Vitest setup

Playwright/Puppeteer - Best for:

• Full-stack integration tests
• Browser-based API interactions
• Testing with real authentication flows
• Visual verification alongside API calls

Testcontainers - Best for:

• Testing with real databases (PostgreSQL, MongoDB, Redis)
• Message queue integration (RabbitMQ, Kafka)
• Isolated test environments
• CI/CD compatibility

Basic API Integration Test

// api/users.integration.test.ts
import request from 'supertest'
import { app } from '../app'
import { db } from '../database'
import { UserFactory } from '../test-factories/user.factory'

describe('User API Integration', () => {
  beforeAll(async () => {
    // Setup: Start test database
    await db.connect()
    await db.migrate.latest()
  })

  afterAll(async () => {
    // Teardown: Close connections
    await db.destroy()
  })

  beforeEach(async () => {
    // Reset database state before each test
    await db('users').truncate()
  })

  describe('POST /api/users', () => {
    it('should create user and return 201', async () => {
      // Arrange
      const userData = {
        email: 'test@example.com',
        password: 'SecurePass123!',
        name: 'Test User'
      }

      // Act
      const response = await request(app)
        .post('/api/users')
        .send(userData)
        .expect('Content-Type', /json/)
        .expect(201)

      // Assert
      expect(response.body).toMatchObject({
        id: expect.any(String),
        email: 'test@example.com',
        name: 'Test User'
      })
      expect(response.body.password).toBeUndefined() // Never return password

      // Verify database state
      const dbUser = await db('users').where({ id: response.body.id }).first()
      expect(dbUser).toBeDefined()
      expect(dbUser.email).toBe('test@example.com')
    })

    it('should return 409 for duplicate email', async () => {
      // Arrange
      const userData = { email: 'test@example.com', password: 'pass', name: 'Test' }
      await request(app).post('/api/users').send(userData)

      // Act
      const response = await request(app)
        .post('/api/users')
        .send(userData)
        .expect(409)

      // Assert
      expect(response.body.error).toBe('Email already exists')
    })

    it('should validate required fields', async () => {
      // Act
      const response = await request(app)
        .post('/api/users')
        .send({ email: 'test@example.com' }) // Missing password and name
        .expect(400)

      // Assert
      expect(response.body.errors).toContainEqual(
        expect.objectContaining({ field: 'password', message: expect.any(String) })
      )
      expect(response.body.errors).toContainEqual(
        expect.objectContaining({ field: 'name', message: expect.any(String) })
      )
    })
  })

  describe('GET /api/users/:id', () => {
    it('should return user by id', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db)

      // Act
      const response = await request(app)
        .get(`/api/users/${user.id}`)
        .expect(200)

      // Assert
      expect(response.body).toMatchObject({
        id: user.id,
        email: user.email,
        name: user.name
      })
    })

    it('should return 404 for non-existent user', async () => {
      // Act
      const response = await request(app)
        .get('/api/users/non-existent-id')
        .expect(404)

      // Assert
      expect(response.body.error).toBe('User not found')
    })
  })

  describe('PUT /api/users/:id', () => {
    it('should update user', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db)
      const updates = { name: 'Updated Name' }

      // Act
      const response = await request(app)
        .put(`/api/users/${user.id}`)
        .send(updates)
        .expect(200)

      // Assert
      expect(response.body.name).toBe('Updated Name')

      // Verify database state
      const dbUser = await db('users').where({ id: user.id }).first()
      expect(dbUser.name).toBe('Updated Name')
    })

    it('should not allow email update', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db)

      // Act
      const response = await request(app)
        .put(`/api/users/${user.id}`)
        .send({ email: 'newemail@example.com' })
        .expect(400)

      // Assert
      expect(response.body.error).toContain('cannot change email')
    })
  })

  describe('DELETE /api/users/:id', () => {
    it('should soft delete user', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db)

      // Act
      await request(app)
        .delete(`/api/users/${user.id}`)
        .expect(204)

      // Assert - User should still exist but be marked deleted
      const dbUser = await db('users').where({ id: user.id }).first()
      expect(dbUser.deleted_at).toBeDefined()
    })
  })
})

Authentication Integration Tests

describe('Authentication Flow', () => {
  let authToken: string

  describe('POST /api/auth/login', () => {
    it('should authenticate user and return token', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db, { password: 'TestPass123!' })

      // Act
      const response = await request(app)
        .post('/api/auth/login')
        .send({ email: user.email, password: 'TestPass123!' })
        .expect(200)

      // Assert
      expect(response.body).toMatchObject({
        token: expect.any(String),
        user: {
          id: user.id,
          email: user.email
        }
      })

      authToken = response.body.token
    })

    it('should reject invalid credentials', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db, { password: 'TestPass123!' })

      // Act
      const response = await request(app)
        .post('/api/auth/login')
        .send({ email: user.email, password: 'WrongPassword' })
        .expect(401)

      // Assert
      expect(response.body.error).toBe('Invalid credentials')
    })

    it('should lock account after 5 failed attempts', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db, { password: 'TestPass123!' })

      // Act - 5 failed attempts
      for (let i = 0; i < 5; i++) {
        await request(app)
          .post('/api/auth/login')
          .send({ email: user.email, password: 'WrongPassword' })
      }

      // Act - 6th attempt should be locked
      const response = await request(app)
        .post('/api/auth/login')
        .send({ email: user.email, password: 'TestPass123!' }) // Even correct password
        .expect(423)

      // Assert
      expect(response.body.error).toContain('Account locked')
    })
  })

  describe('Protected Routes', () => {
    it('should allow access with valid token', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db)
      const token = await generateAuthToken(user)

      // Act
      const response = await request(app)
        .get('/api/users/me')
        .set('Authorization', `Bearer ${token}`)
        .expect(200)

      // Assert
      expect(response.body.id).toBe(user.id)
    })

    it('should reject invalid token', async () => {
      // Act
      const response = await request(app)
        .get('/api/users/me')
        .set('Authorization', 'Bearer invalid-token')
        .expect(401)

      // Assert
      expect(response.body.error).toBe('Invalid token')
    })

    it('should reject expired token', async () => {
      // Arrange
      const user = await UserFactory.createInDb(db)
      const expiredToken = await generateAuthToken(user, { expiresIn: '-1h' })

      // Act
      const response = await request(app)
        .get('/api/users/me')
        .set('Authorization', `Bearer ${expiredToken}`)
        .expect(401)

      // Assert
      expect(response.body.error).toContain('expired')
    })
  })
})

Database Integration Tests

import { PostgreSqlContainer } from '@testcontainers/postgresql'

describe('Database Integration', () => {
  let container: PostgreSqlContainer
  let testDb: Database

  beforeAll(async () => {
    // Start PostgreSQL container
    container = await new PostgreSqlContainer('postgres:15')
      .withDatabase('test_db')
      .withUsername('test_user')
      .withPassword('test_pass')
      .start()

    // Connect to test database
    testDb = await connectToDatabase({
      host: container.getHost(),
      port: container.getPort(),
      database: container.getDatabase(),
      username: container.getUsername(),
      password: container.getPassword()
    })

    // Run migrations
    await testDb.migrate.latest()
  }, 60000) // Increased timeout for container startup

  afterAll(async () => {
    await testDb.destroy()
    await container.stop()
  })

  describe('Transaction Handling', () => {
    it('should commit transaction on success', async () => {
      // Act
      await testDb.transaction(async (trx) => {
        await trx('users').insert({ email: 'test@example.com', name: 'Test' })
        await trx('profiles').insert({ user_email: 'test@example.com', bio: 'Test bio' })
      })

      // Assert
      const user = await testDb('users').where({ email: 'test@example.com' }).first()
      const profile = await testDb('profiles').where({ user_email: 'test@example.com' }).first()

      expect(user).toBeDefined()
      expect(profile).toBeDefined()
    })

    it('should rollback transaction on error', async () => {
      // Act
      await expect(
        testDb.transaction(async (trx) => {
          await trx('users').insert({ email: 'test@example.com', name: 'Test' })
          throw new Error('Simulated error')
        })
      ).rejects.toThrow()

      // Assert - User should not exist
      const user = await testDb('users').where({ email: 'test@example.com' }).first()
      expect(user).toBeUndefined()
    })
  })

  describe('Complex Queries', () => {
    it('should perform join queries', async () => {
      // Arrange
      await testDb('users').insert([
        { id: '1', email: 'user1@example.com', name: 'User 1' },
        { id: '2', email: 'user2@example.com', name: 'User 2' }
      ])
      await testDb('posts').insert([
        { id: '1', user_id: '1', title: 'Post 1' },
        { id: '2', user_id: '1', title: 'Post 2' },
        { id: '3', user_id: '2', title: 'Post 3' }
      ])

      // Act
      const results = await testDb('users')
        .select('users.name', testDb.raw('COUNT(posts.id) as post_count'))
        .leftJoin('posts', 'users.id', 'posts.user_id')
        .groupBy('users.id')
        .orderBy('post_count', 'desc')

      // Assert
      expect(results).toHaveLength(2)
      expect(results[0]).toMatchObject({ name: 'User 1', post_count: '2' })
      expect(results[1]).toMatchObject({ name: 'User 2', post_count: '1' })
    })
  })
})

External Service Integration Tests

import { WireMock } from 'wiremock'

describe('External Service Integration', () => {
  let wireMock: WireMock

  beforeAll(async () => {
    // Start WireMock server for stubbing external APIs
    wireMock = new WireMock({ host: 'localhost', port: 8080 })
    await wireMock.start()
  })

  afterAll(async () => {
    await wireMock.stop()
  })

  beforeEach(async () => {
    await wireMock.resetAll()
  })

  describe('Payment Service Integration', () => {
    it('should process payment successfully', async () => {
      // Arrange - Stub external payment API
      await wireMock.stub({
        request: {
          method: 'POST',
          url: '/api/payments'
        },
        response: {
          status: 200,
          jsonBody: {
            transactionId: 'TX123456',
            status: 'approved'
          }
        }
      })

      const order = await OrderFactory.createInDb(db)

      // Act
      const response = await request(app)
        .post(`/api/orders/${order.id}/pay`)
        .send({ amount: 100, currency: 'USD' })
        .expect(200)

      // Assert
      expect(response.body).toMatchObject({
        transactionId: 'TX123456',
        status: 'approved'
      })

      // Verify WireMock received request
      const requests = await wireMock.getRequests()
      expect(requests).toHaveLength(1)
      expect(requests[0].body).toContain('amount')
    })

    it('should handle payment service timeout', async () => {
      // Arrange - Stub with delay
      await wireMock.stub({
        request: {
          method: 'POST',
          url: '/api/payments'
        },
        response: {
          status: 200,
          fixedDelayMilliseconds: 10000 // 10 second delay
        }
      })

      const order = await OrderFactory.createInDb(db)

      // Act
      const response = await request(app)
        .post(`/api/orders/${order.id}/pay`)
        .send({ amount: 100, currency: 'USD' })
        .expect(504)

      // Assert
      expect(response.body.error).toContain('timeout')
    })

    it('should retry on service failure', async () => {
      // Arrange - First call fails, second succeeds
      await wireMock.stub({
        request: {
          method: 'POST',
          url: '/api/payments'
        },
        response: {
          status: 500
        }
      })

      setTimeout(async () => {
        await wireMock.resetAll()
        await wireMock.stub({
          request: {
            method: 'POST',
            url: '/api/payments'
          },
          response: {
            status: 200,
            jsonBody: { transactionId: 'TX123456', status: 'approved' }
          }
        })
      }, 1000)

      const order = await OrderFactory.createInDb(db)

      // Act
      const response = await request(app)
        .post(`/api/orders/${order.id}/pay`)
        .send({ amount: 100, currency: 'USD' })
        .expect(200)

      // Assert
      expect(response.body.transactionId).toBe('TX123456')

      // Verify retries happened
      const requests = await wireMock.getRequests()
      expect(requests.length).toBeGreaterThan(1)
    })
  })
})

Message Queue Integration Tests

import { RabbitMQContainer } from '@testcontainers/rabbitmq'
import amqp from 'amqplib'

describe('Message Queue Integration', () => {
  let container: RabbitMQContainer
  let connection: amqp.Connection
  let channel: amqp.Channel

  beforeAll(async () => {
    // Start RabbitMQ container
    container = await new RabbitMQContainer().start()

    // Connect to RabbitMQ
    connection = await amqp.connect(container.getAmqpUrl())
    channel = await connection.createChannel()
  }, 60000)

  afterAll(async () => {
    await channel.close()
    await connection.close()
    await container.stop()
  })

  describe('Order Processing Queue', () => {
    it('should publish and consume messages', async () => {
      // Arrange
      const queueName = 'order_processing'
      await channel.assertQueue(queueName, { durable: false })

      const orderData = {
        orderId: '123',
        userId: 'user-456',
        total: 99.99
      }

      // Act - Publish message
      channel.sendToQueue(queueName, Buffer.from(JSON.stringify(orderData)))

      // Assert - Consume message
      const message = await new Promise<any>((resolve) => {
        channel.consume(queueName, (msg) => {
          if (msg) {
            resolve(JSON.parse(msg.content.toString()))
            channel.ack(msg)
          }
        })
      })

      expect(message).toMatchObject(orderData)
    })

    it('should handle message rejection and retry', async () => {
      // Arrange
      const queueName = 'order_processing_retry'
      await channel.assertQueue(queueName, { durable: true })
      await channel.assertQueue(`${queueName}_dlq`, { durable: true })

      const invalidOrder = { orderId: 'invalid' }
      channel.sendToQueue(queueName, Buffer.from(JSON.stringify(invalidOrder)))

      // Act - Consumer rejects invalid message
      let attempts = 0
      await new Promise<void>((resolve) => {
        channel.consume(queueName, (msg) => {
          if (msg) {
            attempts++
            if (attempts < 3) {
              channel.nack(msg, false, true) // Requeue
            } else {
              channel.sendToQueue(`${queueName}_dlq`, msg.content) // Dead letter
              channel.ack(msg)
              resolve()
            }
          }
        })
      })

      // Assert
      expect(attempts).toBe(3)
    })
  })
})

Best Practices Checklist

• [ ] Test the entire request/response cycle (not just business logic)
• [ ] Use real database instances (Testcontainers) for accuracy
• [ ] Reset database state between tests (truncate or transactions)
• [ ] Test authentication and authorization flows
• [ ] Verify database state after operations (not just API responses)
• [ ] Stub external services (WireMock, MSW) for reliability
• [ ] Test error scenarios (timeouts, retries, failures)
• [ ] Test transaction rollbacks and commits
• [ ] Use factories for complex test data setup
• [ ] Test message queue processing (if applicable)
• [ ] Validate response headers and status codes
• [ ] Test rate limiting and throttling
• [ ] Verify side effects (emails sent, events published)

Common Pitfalls

[FAIL] Using in-memory databases for integration tests:

// Bad - SQLite in-memory doesn't match production PostgreSQL
const db = new SQLite(':memory:')

// Good - Use Testcontainers with real PostgreSQL
const container = await new PostgreSqlContainer('postgres:15').start()

[FAIL] Not cleaning up between tests:

// Bad - Tests interfere with each other
beforeAll(async () => {
  await db.seed.run() // Only runs once
})

// Good - Fresh state for each test
beforeEach(async () => {
  await db('users').truncate()
})

[FAIL] Testing external APIs directly:

// Bad - Tests depend on external service availability
await fetch('https://api.stripe.com/v1/charges')

// Good - Stub external services
await wireMock.stub({ ... })

Configuration

package.json

{
  "scripts": {
    "test:integration": "vitest run --config vitest.integration.config.ts",
    "test:integration:watch": "vitest --config vitest.integration.config.ts"
  },
  "devDependencies": {
    "@testcontainers/postgresql": "^10.0.0",
    "@testcontainers/rabbitmq": "^10.0.0",
    "supertest": "^6.3.3",
    "wiremock": "^3.0.0"
  }
}

vitest.integration.config.ts

import { defineConfig } from 'vitest/config'

export default defineConfig({
  test: {
    include: ['**/*.integration.test.ts'],
    testTimeout: 30000, // Longer timeout for containers
    hookTimeout: 60000, // Container startup can be slow
    globalSetup: './test/integration-setup.ts',
    pool: 'forks', // Isolation for database tests
    poolOptions: {
      forks: {
        singleFork: false // Run tests in parallel
      }
    }
  }
})

Related Resources

See ../../references/comprehensive-testing-guide.md for complete testing guide across all layers.

Performance Testing Template: k6

Use this template for load and performance testing with k6, a modern developer-centric load testing tool.

Why k6 (2024-2025)

Advantages over JMeter/Gatling:

• JavaScript DSL (familiar syntax)
• CLI-first approach (no GUI overhead)
• Built-in Grafana Cloud integration
• Excellent CI/CD integration
• Real-time metrics and thresholds
• Protocol Buffers and gRPC support

Basic Load Test

// load-test.js
import http from 'k6/http'
import { check, sleep } from 'k6'
import { Rate } from 'k6/metrics'

// Custom metrics
const errorRate = new Rate('errors')

export const options = {
  stages: [
    { duration: '2m', target: 100 },   // Ramp up to 100 users over 2 minutes
    { duration: '5m', target: 100 },   // Stay at 100 users for 5 minutes
    { duration: '2m', target: 200 },   // Ramp up to 200 users
    { duration: '5m', target: 200 },   // Stay at 200 users
    { duration: '2m', target: 0 },     // Ramp down to 0 users
  ],
  thresholds: {
    http_req_duration: ['p(95)<500', 'p(99)<1000'],  // 95% under 500ms, 99% under 1s
    http_req_failed: ['rate<0.01'],                   // Error rate under 1%
    errors: ['rate<0.1'],                             // Custom error rate under 10%
  }
}

export default function () {
  // GET request
  const response = http.get('https://api.example.com/products')

  // Validate response
  const checkResult = check(response, {
    'status is 200': (r) => r.status === 200,
    'response time < 500ms': (r) => r.timings.duration < 500,
    'body contains products': (r) => r.json('products') !== undefined
  })

  // Track errors
  errorRate.add(!checkResult)

  // Think time (simulate user behavior)
  sleep(1)
}

API Testing with Authentication

import http from 'k6/http'
import { check } from 'k6'

export const options = {
  vus: 50, // 50 virtual users
  duration: '5m'
}

// Setup: Authenticate once per VU
export function setup() {
  const loginRes = http.post('https://api.example.com/auth/login', {
    email: 'test@example.com',
    password: 'password123'
  })

  const token = loginRes.json('token')
  return { token }
}

export default function (data) {
  const headers = {
    'Authorization': `Bearer ${data.token}`,
    'Content-Type': 'application/json'
  }

  // Authenticated requests
  const productsRes = http.get('https://api.example.com/products', { headers })
  check(productsRes, {
    'products loaded': (r) => r.status === 200
  })

  const ordersRes = http.get('https://api.example.com/orders', { headers })
  check(ordersRes, {
    'orders loaded': (r) => r.status === 200
  })
}

User Scenarios (Realistic Workflows)

import http from 'k6/http'
import { check, sleep, group } from 'k6'
import { randomItem } from 'https://jslib.k6.io/k6-utils/1.2.0/index.js'

export const options = {
  scenarios: {
    // 80% of users browse products
    browsers: {
      executor: 'ramping-vus',
      startVUs: 0,
      stages: [
        { duration: '5m', target: 80 },
        { duration: '10m', target: 80 },
        { duration: '2m', target: 0 }
      ],
      exec: 'browseProducts'
    },
    // 20% of users make purchases
    buyers: {
      executor: 'ramping-vus',
      startVUs: 0,
      stages: [
        { duration: '5m', target: 20 },
        { duration: '10m', target: 20 },
        { duration: '2m', target: 0 }
      ],
      exec: 'purchaseFlow'
    }
  },
  thresholds: {
    'group_duration{group:::Browse Products}': ['p(95)<2000'],
    'group_duration{group:::Purchase Flow}': ['p(95)<5000']
  }
}

export function browseProducts() {
  group('Browse Products', () => {
    // Homepage
    http.get('https://api.example.com/')
    sleep(2)

    // Category page
    const categories = ['laptops', 'phones', 'tablets']
    http.get(`https://api.example.com/products?category=${randomItem(categories)}`)
    sleep(3)

    // Product detail
    const productId = Math.floor(Math.random() * 100) + 1
    http.get(`https://api.example.com/products/${productId}`)
    sleep(2)
  })
}

export function purchaseFlow() {
  group('Purchase Flow', () => {
    // Login
    const loginRes = http.post('https://api.example.com/auth/login', {
      email: 'buyer@example.com',
      password: 'password'
    })
    const token = loginRes.json('token')
    const headers = { 'Authorization': `Bearer ${token}` }

    sleep(1)

    // Add to cart
    http.post('https://api.example.com/cart', {
      productId: 42,
      quantity: 1
    }, { headers })

    sleep(2)

    // Checkout
    http.post('https://api.example.com/orders', {
      cartId: '123',
      paymentMethod: 'card'
    }, { headers })

    sleep(1)
  })
}

Spike Testing

export const options = {
  stages: [
    { duration: '10s', target: 100 },   // Normal load
    { duration: '1m', target: 100 },    // Sustain normal load
    { duration: '10s', target: 1000 },  // SPIKE to 10x load
    { duration: '3m', target: 1000 },   // Sustain spike
    { duration: '10s', target: 100 },   // Drop back to normal
    { duration: '3m', target: 100 },    // Recovery period
    { duration: '10s', target: 0 }      // Ramp down
  ],
  thresholds: {
    http_req_duration: ['p(99)<3000'],  // Allow higher latency during spike
    http_req_failed: ['rate<0.05']      // Allow 5% errors during spike
  }
}

Stress Testing (Find Breaking Point)

export const options = {
  stages: [
    { duration: '2m', target: 100 },
    { duration: '5m', target: 100 },
    { duration: '2m', target: 200 },
    { duration: '5m', target: 200 },
    { duration: '2m', target: 300 },
    { duration: '5m', target: 300 },
    { duration: '2m', target: 400 },  // Continue increasing until system breaks
    { duration: '5m', target: 400 },
    { duration: '10m', target: 0 }
  ]
}

Soak Testing (Endurance)

export const options = {
  stages: [
    { duration: '5m', target: 100 },   // Ramp up
    { duration: '8h', target: 100 },   // Sustained load for 8 hours
    { duration: '5m', target: 0 }      // Ramp down
  ],
  thresholds: {
    http_req_duration: ['p(99)<1000'],
    http_req_failed: ['rate<0.001']    // Very low error rate for sustained test
  }
}

Custom Metrics and Trends

import { Trend, Counter, Gauge } from 'k6/metrics'

// Custom metrics
const productLoadTime = new Trend('product_load_time')
const cartItemsCount = new Gauge('cart_items')
const ordersPlaced = new Counter('orders_placed')

export default function () {
  const start = Date.now()
  const res = http.get('https://api.example.com/products/42')
  const duration = Date.now() - start

  productLoadTime.add(duration)

  if (res.status === 200) {
    const product = res.json()
    // Track cart items
    cartItemsCount.add(product.quantity || 0)
  }

  // Simulate order placement
  if (Math.random() < 0.1) {  // 10% of users place order
    http.post('https://api.example.com/orders', {})
    ordersPlaced.add(1)
  }
}

Thresholds and SLOs

export const options = {
  thresholds: {
    // HTTP metrics
    http_req_duration: [
      'p(50)<200',    // 50% under 200ms
      'p(90)<400',    // 90% under 400ms
      'p(95)<500',    // 95% under 500ms
      'p(99)<1000'    // 99% under 1s
    ],
    'http_req_duration{name:ProductPage}': ['p(95)<300'],
    'http_req_duration{name:Checkout}': ['p(95)<1000'],

    // Error rates
    http_req_failed: ['rate<0.01'],  // Total error rate < 1%
    'http_req_failed{name:Checkout}': ['rate<0.001'],  // Checkout errors < 0.1%

    // Custom metrics
    'product_load_time': ['p(95)<500'],
    'orders_placed': ['count>100']  // At least 100 orders during test
  }
}

Data-Driven Testing

import { SharedArray } from 'k6/data'
import papaparse from 'https://jslib.k6.io/papaparse/5.1.1/index.js'

// Load test data once (shared across VUs)
const testData = new SharedArray('users', function () {
  return papaparse.parse(open('./test-users.csv'), { header: true }).data
})

export default function () {
  // Each VU gets a different user
  const user = testData[__VU % testData.length]

  const loginRes = http.post('https://api.example.com/auth/login', {
    email: user.email,
    password: user.password
  })

  check(loginRes, {
    'login successful': (r) => r.status === 200
  })
}

CI/CD Integration

GitHub Actions

name: Load Tests

on:
  schedule:
    - cron: '0 2 * * *'  # Run nightly
  workflow_dispatch:

jobs:
  k6-load-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run k6 load test
        uses: grafana/k6-action@v0.3.0
        with:
          filename: load-test.js
          cloud: true
          token: ${{ secrets.K6_CLOUD_TOKEN }}

      - name: Check thresholds
        if: failure()
        run: |
          echo "Load test thresholds failed!"
          exit 1

Docker

# Run k6 in Docker
docker run --rm -i grafana/k6 run - <load-test.js

# With output to InfluxDB
docker run --rm \
  -e K6_OUT=influxdb=http://influxdb:8086/k6 \
  grafana/k6 run /scripts/load-test.js

Running Tests

# Basic run
k6 run load-test.js

# With custom VUs and duration
k6 run --vus 100 --duration 30s load-test.js

# Output to file
k6 run --out json=results.json load-test.js

# Cloud execution
k6 cloud load-test.js

# With environment variables
k6 run -e BASE_URL=https://staging.example.com load-test.js

Analyzing Results

// Example summary output
data_received..................: 148 MB  2.5 MB/s
data_sent......................: 13 MB   219 kB/s
http_req_blocked...............: avg=1.46ms   min=1µs    med=5µs     max=1.03s   p(90)=11µs   p(95)=15µs
http_req_connecting............: avg=700µs    min=0s     med=0s      max=608ms   p(90)=0s     p(95)=0s
http_req_duration..............: avg=145.12ms min=100ms  med=124ms   max=2.35s   p(90)=203ms  p(95)=232ms
http_req_failed................: 0.52%   [check] 52    [x] 9948
http_req_receiving.............: avg=332µs    min=22µs   med=108µs   max=117ms   p(90)=214µs  p(95)=278µs
http_req_sending...............: avg=88µs     min=7µs    med=29µs    max=5.12ms  p(90)=149µs  p(95)=186µs
http_req_tls_handshaking.......: avg=0s       min=0s     med=0s      max=0s      p(90)=0s     p(95)=0s
http_req_waiting...............: avg=144.7ms  min=100ms  med=124ms   max=2.35s   p(90)=202ms  p(95)=232ms
http_reqs......................: 10000   166.666667/s
iteration_duration.............: avg=1.14s    min=1s     med=1.12s   max=3.37s   p(90)=1.2s   p(95)=1.23s
iterations.....................: 10000   166.666667/s
vus............................: 100     min=100 max=100
vus_max........................: 100     min=100 max=100

Best Practices Checklist

• [ ] Define realistic load scenarios (not just max load)
• [ ] Use stages for gradual ramp-up
• [ ] Set meaningful thresholds (based on SLOs)
• [ ] Include think time to simulate real users
• [ ] Test authentication flows separately
• [ ] Monitor backend metrics during tests (CPU, memory, DB connections)
• [ ] Run tests from multiple geographic regions
• [ ] Test during expected peak hours
• [ ] Gradually increase load to find breaking point
• [ ] Include soak tests for long-running stability

Common Patterns

// Pattern: Weighted scenarios
export const options = {
  scenarios: {
    light_load: {
      executor: 'constant-vus',
      vus: 50,
      duration: '10m',
      exec: 'browsing'
    },
    heavy_load: {
      executor: 'ramping-arrival-rate',
      startRate: 10,
      timeUnit: '1s',
      preAllocatedVUs: 100,
      stages: [
        { duration: '5m', target: 50 },
        { duration: '10m', target: 50 }
      ],
      exec: 'checkout'
    }
  }
}

// Pattern: Smoke test (quick validation)
export const options = {
  vus: 1,
  duration: '1m',
  thresholds: {
    http_req_failed: ['rate<0.01']
  }
}

// Pattern: Breakpoint test (find max capacity)
export const options = {
  executor: 'ramping-arrival-rate',
  startRate: 1,
  timeUnit: '1s',
  preAllocatedVUs: 500,
  maxVUs: 1000,
  stages: [
    { duration: '2h', target: 100 }  // Slowly ramp up until system breaks
  ]
}

Related Resources

See ../../references/comprehensive-testing-guide.md for performance testing strategies and ../../references/shift-left-testing.md for early performance validation.

Flaky Test Triage & Deflake Runbook

Use this runbook to reduce CI noise, prevent silent regressions, and restore confidence.

Core

Definitions

• Flaky test: fails without product change and passes on rerun.
• “Rerun-pass” is a defect signal, not a success.

Flake SLOs (Example Targets)

• Suite flake rate <= 1% weekly.
• Time-to-deflake: p50 <= 2 business days, p95 <= 7 business days.
• Mainline health: >= 99% green builds/day.

Intake Checklist (First 5 Minutes)

• Identify the failing test(s): name/path, suite, owner.
• Collect context:
• Build URL, commit SHA, branch, runner type (self-hosted vs hosted)
• Timestamp, region, parallel shard/worker ID
• Retry count and whether it passed on retry
• Correlation IDs (request/trace IDs) and artifacts (logs/screenshots/traces)

Triage Flow (Reproduce → Classify → Fix → Prevent)

Reproduce:

• Run the single test N times (example: 20) on the same runner profile.
• If CI-only: reproduce in a container/runner that matches CI resources.

Classify (pick the dominant class):

Fix:

• Prefer product fixes for real races over test-only band-aids.
• Add/upgrade observability for the failing path (logs/traces) to catch it next time.

Prevent:

• Add a pre-merge check that would have caught the issue earlier (unit/integration/contract).

Quarantine Policy (If You Must)

Quarantine is a temporary safety valve, not a solution.

REQUIRED fields:

• Owner: ______________________
• Ticket: _____________________
• Reason: _____________________
• Expiry date: ________________
• Impact: blocks PRs? yes/no

Rules:

• No quarantines without expiry and an assigned owner.
• Quarantined tests must still run and report; they just don’t block merges.
• If a quarantined test starts failing consistently, escalate as a product defect.

CI Economics (Contain Blast Radius)

• Split suites by layer and cost: fast PR gate vs slow scheduled suites.
• Shard long-running suites; keep PR feedback under a fixed budget.

Anti-Patterns (Deflake Smells)

• Adding sleeps to “stabilize” without proving the race.
• Increasing timeouts globally instead of fixing the slow step.
• Weakening assertions so failures disappear.
• Marking rerun-pass as success without tracking flake rate.

Optional: AI / Automation

Do:

• Use AI to cluster failures across builds and summarize common signatures, but require evidence links (logs/traces/stack traces).
• Use AI to propose candidate root causes; validate via targeted instrumentation and reproduction.

Avoid:

• Letting AI auto-edit tests to “heal” flakes by reducing assertions or switching to brittle selectors.

Template: Release Coverage Audit (Feature Matrix vs Test Matrix)

Release Context

• Release tag/branch: ________________________
• Audit date: YYYY-MM-DD
• Auditor: _______________________________

Coverage Table

Summary

• Critical features total: ___
• Directly covered: ___
• Indirectly covered: ___
• Uncovered: ___

Decision

• [ ] GO
• [ ] NO-GO

Reason: _____________________________________________________________

Required Follow-ups

1. ___________________________________________________________ 2. ___________________________________________________________

Test Case Design Template (Given/When/Then + Oracles)

Use this template for any test layer (unit/integration/contract/E2E) by filling only what applies.

Core

Metadata

• ID: __________________________
• Title: _______________________
• Owner: _______________________
• Layer: unit / component / contract / integration / E2E / exploratory
• Priority: P0 / P1 / P2 / P3
• Risk addressed: journey + failure mode(s)

Goal (What This Test Proves)

• Hypothesis: _______________________________________________
• Why now: _________________________________________________

Preconditions

• Environment: local / CI / staging
• Feature flags/config: _____________________________________
• Auth/user roles: __________________________________________

Test Data

• Data setup method: fixtures / factories / seed / API setup
• Data identifiers (IDs/keys): _______________________________
• Cleanup/reset plan: _______________________________________

Steps (Given / When / Then)

Given:

• ___________________________________________________________

When:

• ___________________________________________________________

Then:

• ___________________________________________________________

Oracles (How You Know It’s Correct)

Functional oracles:

• Expected state/output: _____________________________________
• Contract/schema: __________________________________________

Quality oracles (if applicable):

• Security: authz/authn, sensitive data not exposed
• Accessibility: roles/labels, focus order, keyboard paths
• Performance: budget (p95/p99) and no significant regression

Negative oracles:

• What must NOT happen: _____________________________________

Observability (Debugging Ergonomics)

• Correlation IDs captured: request ID / trace ID / build URL
• Failure artifacts expected:
• Logs
• Traces
• Screenshots/video (UI)
• Crash reports/core dumps (if relevant)

Flake Control (Determinism)

• Time control: timezone/locale/frozen time? ________________
• Network control: mocked/stubbed boundaries? _______________
• Retries policy: ___________________________________________
• Timeout budget: ___________________________________________

Automation Notes

• What to mock vs keep real: ________________________________
• Lowest layer alternative: can this be tested lower? ________
• CI execution: PR gate / nightly / release _________________

Pass/Fail Criteria

• Pass criteria: ____________________________________________
• Fail criteria: ____________________________________________

Optional: AI / Automation

Do:

• Use AI to propose edge cases and variations (boundaries, auth roles, locales).
• Use AI to draft Given/When/Then steps and candidate oracles, then validate manually.

Avoid:

• Copying AI-generated assertions without verifying the oracle and failure mode.
• Generating large combinatorial suites without a risk-based selection.

QA Test Strategy One-Pager (Risk-Based)

Use this template to define a minimal, high-signal quality plan that balances risk coverage, CI economics, and debuggability.

Core

Context

• Product/area: ________________________________
• What is changing (scope): _____________________
• Release cadence: ______________________________
• Environments: local / CI / staging / prod
• Key dependencies: _____________________________

Quality Goals (Measurable)

• Reliability: SLIs/SLOs (latency/error/availability) and error budget policy
• Performance: budgets (p95/p99), frontend budgets (if applicable)
• Security: required checks (SAST/DAST/dependency scanning)
• Accessibility: target level and automation scope

Risk Model (Journeys x Failure Modes)

List top user journeys and likely failure modes.

Test Portfolio (Layered)

Define what runs where, and why.

• Unit: business logic, validators, pure functions
• Component: UI logic and accessibility checks
• Contract: OpenAPI/AsyncAPI/JSON schema validations
• Integration: API + DB + key dependencies (mock third parties)
• E2E: thin, critical user journeys only
• Exploratory: discovery and usability; convert high-ROI findings to automation
• Performance/resilience: scheduled or canary-gated, not every PR

Shift-Left (Pre-Merge Gates)

• Required: lint, typecheck, unit tests, contract validation
• Conditional: integration smoke for affected areas
• Avoid: full E2E as a default PR gate (unless E2E-only product)

CI/CD Stages (Economics)

• PR gate: ________________________________
• Post-merge: _____________________________
• Nightly: ________________________________
• Release: ________________________________
• Budgets (example targets):
• PR gate p50 <= 10 min, p95 <= 20 min
• Mainline health >= 99% green builds/day

Flake Management

• Flake definition: fails without product change and passes on rerun.
• SLO examples (example targets):
• Suite flake rate <= 1% weekly
• Time-to-deflake p50 <= 2 business days, p95 <= 7 business days
• Quarantine rules: owner + ticket + expiry; never “ignore forever”.
• Runbook: runbooks/template-flaky-test-triage-deflake-runbook.md

Observability for QA (Debugging Ergonomics)

• Required correlation IDs: request ID, trace ID
• Failure artifacts: logs, traces, screenshots/videos (UI), crash reports
• Where artifacts live: __________________________

Owners and Cadence

• Suite owners: _________________________________
• Review cadence: _______________________________
• Deprecation policy for low-value tests: ________

Optional: AI / Automation

Do:

• Use AI to draft the initial risk register and candidate test ideas; validate against domain knowledge and telemetry.
• Use AI to summarize test failures (log/trace clustering) while retaining evidence links.

Avoid:

• Accepting generated assertions/oracles without validation.
• Using AI to “heal” tests by weakening assertions.

Unit Testing Template: Jest / Vitest

Use this template for writing unit tests with Jest or Vitest for JavaScript/TypeScript projects.

Framework Selection

Jest - Best for:

• React applications (built-in React Testing Library support)
• Projects already using Jest (migration cost)
• Teams needing extensive mocking capabilities
• Zero-config setup preference

Vitest - Best for:

• Vite-based projects (instant compatibility)
• Projects prioritizing speed (native ESM, parallel execution)
• TypeScript/JSX without transpilation
• Modern tooling (watch mode, UI mode)

Test File Structure

Basic Test Structure (AAA Pattern)

// user.service.test.ts
import { describe, it, expect, beforeEach, afterEach } from 'vitest' // or '@jest/globals'
import { UserService } from './user.service'
import { DatabaseMock } from '../__mocks__/database.mock'

describe('UserService', () => {
  let service: UserService
  let dbMock: DatabaseMock

  beforeEach(() => {
    // Arrange: Setup for each test
    dbMock = new DatabaseMock()
    service = new UserService(dbMock)
  })

  afterEach(() => {
    // Cleanup after each test
    dbMock.clear()
  })

  describe('createUser', () => {
    it('should hash password before saving', async () => {
      // Arrange
      const userData = {
        email: 'test@example.com',
        password: 'PlainPassword123',
        name: 'Test User'
      }

      // Act
      const user = await service.createUser(userData)

      // Assert
      expect(user.password).not.toBe('PlainPassword123')
      expect(user.password).toMatch(/^\$2[aby]\$.{56}$/) // bcrypt pattern
      expect(user.email).toBe('test@example.com')
    })

    it('should throw error for duplicate email', async () => {
      // Arrange
      const userData = { email: 'test@example.com', password: 'pass', name: 'Test' }
      await service.createUser(userData)

      // Act & Assert
      await expect(service.createUser(userData))
        .rejects
        .toThrow('Email already exists')
    })

    it('should validate email format', async () => {
      // Arrange
      const invalidData = { email: 'invalid-email', password: 'pass', name: 'Test' }

      // Act & Assert
      await expect(service.createUser(invalidData))
        .rejects
        .toThrow('Invalid email format')
    })
  })

  describe('findUserById', () => {
    it('should return user when exists', async () => {
      // Arrange
      const user = await service.createUser({
        email: 'test@example.com',
        password: 'pass',
        name: 'Test User'
      })

      // Act
      const found = await service.findUserById(user.id)

      // Assert
      expect(found).toBeDefined()
      expect(found?.id).toBe(user.id)
      expect(found?.email).toBe('test@example.com')
    })

    it('should return null when user not found', async () => {
      // Act
      const found = await service.findUserById('non-existent-id')

      // Assert
      expect(found).toBeNull()
    })
  })
})

Testing Edge Cases

describe('Edge Cases', () => {
  describe('boundary values', () => {
    it('should handle minimum age', () => {
      expect(service.isAdult(18)).toBe(true)
      expect(service.isAdult(17)).toBe(false)
    })

    it('should handle maximum string length', () => {
      const maxName = 'a'.repeat(100)
      const tooLong = 'a'.repeat(101)

      expect(service.validateName(maxName)).toBe(true)
      expect(() => service.validateName(tooLong)).toThrow('Name too long')
    })
  })

  describe('null and undefined handling', () => {
    it('should handle null input', () => {
      expect(() => service.processData(null)).toThrow('Invalid input')
    })

    it('should handle undefined input', () => {
      expect(() => service.processData(undefined)).toThrow('Invalid input')
    })

    it('should handle empty string', () => {
      expect(() => service.processData('')).toThrow('Invalid input')
    })
  })

  describe('special characters', () => {
    it('should escape SQL injection attempts', () => {
      const malicious = "'; DROP TABLE users; --"
      expect(() => service.searchUsers(malicious)).not.toThrow()
    })

    it('should sanitize XSS attempts', () => {
      const xss = '<script>alert("XSS")</script>'
      const sanitized = service.sanitizeInput(xss)
      expect(sanitized).not.toContain('<script>')
    })
  })
})

Mocking Dependencies

Mock Functions

import { vi } from 'vitest' // or jest.fn()

describe('Mocking', () => {
  it('should call external service', async () => {
    // Arrange
    const emailService = {
      send: vi.fn().mockResolvedValue({ success: true })
    }
    const service = new UserService(db, emailService)

    // Act
    await service.createUser({ email: 'test@example.com', password: 'pass', name: 'Test' })

    // Assert
    expect(emailService.send).toHaveBeenCalledTimes(1)
    expect(emailService.send).toHaveBeenCalledWith({
      to: 'test@example.com',
      subject: 'Welcome',
      template: 'welcome'
    })
  })

  it('should handle service failure gracefully', async () => {
    // Arrange
    const emailService = {
      send: vi.fn().mockRejectedValue(new Error('Email service down'))
    }
    const service = new UserService(db, emailService)

    // Act & Assert
    await expect(service.createUser({ email: 'test@example.com', password: 'pass', name: 'Test' }))
      .rejects
      .toThrow('Failed to send welcome email')
  })
})

Mock Modules

// Vitest module mocking
vi.mock('../services/payment.service', () => ({
  PaymentService: vi.fn().mockImplementation(() => ({
    processPayment: vi.fn().mockResolvedValue({ transactionId: 'TX123' })
  }))
}))

// Jest module mocking
jest.mock('../services/payment.service', () => ({
  PaymentService: jest.fn().mockImplementation(() => ({
    processPayment: jest.fn().mockResolvedValue({ transactionId: 'TX123' })
  }))
}))

Snapshot Testing

describe('Snapshot Testing', () => {
  it('should match user profile snapshot', () => {
    const user = service.getUserProfile('user-123')
    expect(user).toMatchSnapshot()
  })

  it('should match inline snapshot', () => {
    const config = service.getConfig()
    expect(config).toMatchInlineSnapshot(`
      {
        "apiUrl": "https://api.example.com",
        "timeout": 5000,
        "retries": 3
      }
    `)
  })
})

Test Data Factories

// test-factories/user.factory.ts
import { faker } from '@faker-js/faker'

export class UserFactory {
  static create(overrides: Partial<User> = {}): User {
    return {
      id: faker.string.uuid(),
      email: faker.internet.email(),
      name: faker.person.fullName(),
      age: faker.number.int({ min: 18, max: 80 }),
      createdAt: faker.date.past(),
      ...overrides
    }
  }

  static createMany(count: number, overrides: Partial<User> = {}): User[] {
    return Array.from({ length: count }, () => this.create(overrides))
  }

  static createAdmin(): User {
    return this.create({ role: 'admin', permissions: ['read', 'write', 'delete'] })
  }
}

// Usage in tests
describe('UserService', () => {
  it('should process batch of users', () => {
    const users = UserFactory.createMany(10)
    const result = service.processBatch(users)
    expect(result.processed).toBe(10)
  })

  it('should grant admin access', () => {
    const admin = UserFactory.createAdmin()
    expect(service.hasPermission(admin, 'delete')).toBe(true)
  })
})

Async Testing

describe('Async Operations', () => {
  it('should resolve promise', async () => {
    const result = await service.fetchData()
    expect(result).toBeDefined()
  })

  it('should reject promise', async () => {
    await expect(service.fetchInvalidData()).rejects.toThrow('Not found')
  })

  it('should timeout after delay', async () => {
    vi.useFakeTimers()
    const promise = service.delayedOperation()
    vi.advanceTimersByTime(5000)
    await expect(promise).resolves.toBe('completed')
    vi.useRealTimers()
  })
})

Coverage Configuration

Vitest (vite.config.ts)

import { defineConfig } from 'vitest/config'

export default defineConfig({
  test: {
    coverage: {
      provider: 'v8', // or 'istanbul'
      reporter: ['text', 'json', 'html', 'lcov'],
      exclude: [
        '**/node_modules/**',
        '**/dist/**',
        '**/*.test.ts',
        '**/*.config.ts',
        '**/types/**'
      ],
      thresholds: {
        lines: 80,
        functions: 80,
        branches: 80,
        statements: 80
      }
    }
  }
})

Jest (jest.config.js)

module.exports = {
  collectCoverageFrom: [
    'src/**/*.{ts,tsx}',
    '!src/**/*.d.ts',
    '!src/**/*.test.{ts,tsx}',
    '!src/**/__mocks__/**'
  ],
  coverageThresholds: {
    global: {
      lines: 80,
      functions: 80,
      branches: 80,
      statements: 80
    },
    './src/services/': {
      lines: 90,
      functions: 90,
      branches: 90,
      statements: 90
    }
  },
  coverageReporters: ['text', 'lcov', 'html']
}

Best Practices Checklist

• [ ] Use descriptive test names (what is being tested + expected outcome)
• [ ] Follow AAA pattern (Arrange, Act, Assert)
• [ ] Test one thing per test
• [ ] Use factories for test data (avoid magic values)
• [ ] Mock external dependencies (APIs, databases)
• [ ] Test edge cases and error conditions
• [ ] Keep tests independent (no shared state)
• [ ] Use beforeEach/afterEach for setup/cleanup
• [ ] Aim for 80%+ coverage on business logic
• [ ] Run tests in parallel where possible
• [ ] Use snapshot testing sparingly (for stable output only)

Common Pitfalls

[FAIL] Testing implementation details:

// Bad
expect(service.internalHelperMethod()).toBe(true)

// Good
expect(service.publicMethod()).toBe(expectedResult)

[FAIL] Shared mutable state:

// Bad
let sharedUser: User
beforeAll(() => {
  sharedUser = createUser() // Shared across tests
})

// Good
let user: User
beforeEach(() => {
  user = createUser() // Fresh for each test
})

[FAIL] Not cleaning up after tests:

// Bad
afterEach(() => {
  // No cleanup
})

// Good
afterEach(() => {
  vi.clearAllMocks()
  dbMock.clear()
})

Running Tests

# Vitest
npm run test              # Run once
npm run test:watch        # Watch mode
npm run test:ui           # UI mode
npm run test:coverage     # With coverage

# Jest
npm test                  # Run once
npm test -- --watch       # Watch mode
npm test -- --coverage    # With coverage
npm test -- UserService   # Run specific file

Related Resources

See ../../references/comprehensive-testing-guide.md for complete testing guide across all layers.

Visual Regression Testing Template

Use this template for catching unintended visual changes in UI components, pages, and design systems.

Framework Selection

Playwright Visual Comparisons - Best for:

• Full-page screenshots across browsers
• Component screenshot testing
• Built-in pixel-diff comparison
• CI/CD integration out of the box

Chromatic (Storybook) - Best for:

• Design system visual testing
• Component library regression
• Automated visual review workflow
• Cloud-based baseline management

Percy (BrowserStack) - Best for:

• Cross-browser visual testing
• Responsive design validation
• Integration with existing E2E tests
• Advanced diff algorithms

BackstopJS - Best for:

• Lightweight visual regression
• JSON configuration
• Headless browser testing
• Open-source, self-hosted

Playwright Visual Testing

Basic Screenshot Testing

// components/Button.visual.test.ts
import { test, expect } from '@playwright/test'

test.describe('Button Visual Tests', () => {
  test('default button renders correctly', async ({ page }) => {
    await page.goto('/components/button')

    // Take screenshot of specific element
    const button = page.locator('[data-testid="default-button"]')
    await expect(button).toHaveScreenshot('button-default.png')
  })

  test('button states', async ({ page }) => {
    await page.goto('/components/button')

    // Hover state
    const button = page.locator('[data-testid="default-button"]')
    await button.hover()
    await expect(button).toHaveScreenshot('button-hover.png')

    // Focus state
    await button.focus()
    await expect(button).toHaveScreenshot('button-focus.png')

    // Disabled state
    const disabledButton = page.locator('[data-testid="disabled-button"]')
    await expect(disabledButton).toHaveScreenshot('button-disabled.png')
  })

  test('button variants', async ({ page }) => {
    await page.goto('/components/button')

    const variants = ['primary', 'secondary', 'outline', 'ghost', 'destructive']

    for (const variant of variants) {
      const button = page.locator(`[data-testid="button-${variant}"]`)
      await expect(button).toHaveScreenshot(`button-${variant}.png`)
    }
  })

  test('button sizes', async ({ page }) => {
    await page.goto('/components/button')

    const sizes = ['sm', 'md', 'lg']

    for (const size of sizes) {
      const button = page.locator(`[data-testid="button-${size}"]`)
      await expect(button).toHaveScreenshot(`button-size-${size}.png`)
    }
  })
})

Full Page Screenshots

// pages/Dashboard.visual.test.ts
import { test, expect } from '@playwright/test'

test.describe('Dashboard Visual Tests', () => {
  test.beforeEach(async ({ page }) => {
    // Setup: Login and navigate
    await page.goto('/login')
    await page.fill('[name="email"]', 'test@example.com')
    await page.fill('[name="password"]', 'password')
    await page.click('button[type="submit"]')
    await page.waitForURL('/dashboard')
  })

  test('dashboard initial state', async ({ page }) => {
    // Wait for dynamic content to load
    await page.waitForLoadState('networkidle')

    // Take full page screenshot
    await expect(page).toHaveScreenshot('dashboard-initial.png', {
      fullPage: true,
      animations: 'disabled' // Disable animations for consistent screenshots
    })
  })

  test('dashboard with filters applied', async ({ page }) => {
    // Apply filters
    await page.click('[data-testid="filter-button"]')
    await page.click('[data-testid="filter-last-30-days"]')
    await page.click('[data-testid="apply-filters"]')

    // Wait for filtered data
    await page.waitForResponse(resp => resp.url().includes('/api/analytics'))
    await page.waitForLoadState('networkidle')

    await expect(page).toHaveScreenshot('dashboard-filtered.png', {
      fullPage: true
    })
  })

  test('dashboard responsive layouts', async ({ page }) => {
    const viewports = [
      { name: 'mobile', width: 375, height: 667 },
      { name: 'tablet', width: 768, height: 1024 },
      { name: 'desktop', width: 1920, height: 1080 }
    ]

    for (const viewport of viewports) {
      await page.setViewportSize({ width: viewport.width, height: viewport.height })
      await page.waitForLoadState('networkidle')

      await expect(page).toHaveScreenshot(`dashboard-${viewport.name}.png`, {
        fullPage: true
      })
    }
  })
})

Cross-Browser Visual Testing

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test'

export default defineConfig({
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] }
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] }
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] }
    },
    {
      name: 'mobile-chrome',
      use: { ...devices['Pixel 5'] }
    },
    {
      name: 'mobile-safari',
      use: { ...devices['iPhone 13'] }
    }
  ],

  // Visual comparison settings
  expect: {
    toHaveScreenshot: {
      maxDiffPixels: 100, // Allow up to 100 pixels difference
      threshold: 0.2, // 20% threshold for pixel color difference
      animations: 'disabled'
    }
  }
})

Advanced Visual Testing Techniques

// components/Chart.visual.test.ts
import { test, expect } from '@playwright/test'

test.describe('Chart Visual Tests', () => {
  test('chart with stable mock data', async ({ page }) => {
    // Mock API to return consistent data
    await page.route('**/api/chart-data', route => {
      route.fulfill({
        status: 200,
        body: JSON.stringify({
          labels: ['Jan', 'Feb', 'Mar', 'Apr', 'May'],
          data: [10, 20, 15, 25, 30]
        })
      })
    })

    await page.goto('/dashboard/charts')

    // Wait for chart to render
    await page.waitForSelector('canvas.chart-canvas')

    // Take screenshot with mask for dynamic elements
    await expect(page).toHaveScreenshot('chart-stable.png', {
      mask: [page.locator('[data-testid="timestamp"]')] // Hide timestamp
    })
  })

  test('chart with animations complete', async ({ page }) => {
    await page.goto('/dashboard/charts')

    // Wait for animations to complete
    await page.waitForTimeout(1000) // Wait for chart animation

    await expect(page.locator('.chart-container')).toHaveScreenshot('chart-animated.png')
  })

  test('chart theme variations', async ({ page }) => {
    const themes = ['light', 'dark', 'high-contrast']

    for (const theme of themes) {
      await page.goto('/dashboard/charts')
      await page.evaluate((t) => {
        document.documentElement.setAttribute('data-theme', t)
      }, theme)

      await page.waitForTimeout(500) // Wait for theme transition

      await expect(page.locator('.chart-container')).toHaveScreenshot(`chart-${theme}.png`)
    }
  })
})

Chromatic Visual Testing (Storybook)

Story Configuration

// Button.stories.tsx
import type { Meta, StoryObj } from '@storybook/react'
import { Button } from './Button'

const meta: Meta<typeof Button> = {
  title: 'Components/Button',
  component: Button,
  parameters: {
    chromatic: {
      viewports: [375, 768, 1200], // Test multiple viewports
      delay: 300, // Wait 300ms before screenshot
      pauseAnimationAtEnd: true
    }
  }
}

export default meta
type Story = StoryObj<typeof Button>

export const Primary: Story = {
  args: {
    variant: 'primary',
    children: 'Click me'
  }
}

export const AllVariants: Story = {
  render: () => (
    <div style={{ display: 'flex', gap: '1rem', flexDirection: 'column' }}>
      <Button variant="primary">Primary</Button>
      <Button variant="secondary">Secondary</Button>
      <Button variant="outline">Outline</Button>
      <Button variant="ghost">Ghost</Button>
      <Button variant="destructive">Destructive</Button>
    </div>
  ),
  parameters: {
    chromatic: { disableSnapshot: false }
  }
}

export const InteractiveStates: Story = {
  render: () => (
    <div style={{ display: 'flex', gap: '1rem' }}>
      <Button>Default</Button>
      <Button className="hover">Hover</Button>
      <Button className="focus">Focus</Button>
      <Button disabled>Disabled</Button>
    </div>
  ),
  parameters: {
    pseudo: { hover: ['.hover'], focus: ['.focus'] } // Simulate states
  }
}

export const DarkMode: Story = {
  args: {
    variant: 'primary',
    children: 'Dark Mode'
  },
  parameters: {
    backgrounds: { default: 'dark' },
    chromatic: { modes: { dark: { theme: 'dark' } } }
  }
}

Chromatic Configuration

// .storybook/main.js
module.exports = {
  stories: ['../src/**/*.stories.@(ts|tsx)'],
  addons: [
    '@storybook/addon-essentials',
    '@storybook/addon-interactions'
  ],
  framework: {
    name: '@storybook/react-vite',
    options: {}
  }
}

// chromatic.config.json
{
  "projectToken": "your-project-token",
  "buildScriptName": "build-storybook",
  "exitZeroOnChanges": true,
  "exitOnceUploaded": true,
  "onlyChanged": true, // Only test changed components
  "skip": "dependabot/**", // Skip bot PRs
  "ignoreLastBuildOnBranch": "main"
}

Percy Visual Testing

Percy with Playwright

// tests/visual/HomePage.percy.test.ts
import { test } from '@playwright/test'
import percySnapshot from '@percy/playwright'

test.describe('Home Page Visual Tests', () => {
  test('homepage renders correctly', async ({ page }) => {
    await page.goto('/')
    await page.waitForLoadState('networkidle')

    // Take Percy snapshot
    await percySnapshot(page, 'Homepage - Desktop')
  })

  test('homepage responsive', async ({ page }) => {
    await page.goto('/')
    await page.waitForLoadState('networkidle')

    // Percy automatically tests configured breakpoints
    await percySnapshot(page, 'Homepage - Responsive', {
      widths: [375, 768, 1280, 1920]
    })
  })

  test('homepage with user logged in', async ({ page, context }) => {
    // Set auth cookie
    await context.addCookies([{
      name: 'session',
      value: 'test-session-token',
      domain: 'localhost',
      path: '/'
    }])

    await page.goto('/')
    await page.waitForLoadState('networkidle')

    await percySnapshot(page, 'Homepage - Logged In')
  })

  test('homepage dark mode', async ({ page }) => {
    await page.goto('/')
    await page.evaluate(() => {
      document.documentElement.setAttribute('data-theme', 'dark')
    })
    await page.waitForTimeout(300) // Theme transition

    await percySnapshot(page, 'Homepage - Dark Mode')
  })
})

Percy Configuration

# .percy.yml
version: 2
static:
  cleanUrls: true
  include: '**/*.{html,htm}'
  exclude: '**/node_modules/**'

snapshot:
  widths:
    - 375  # Mobile
    - 768  # Tablet
    - 1280 # Desktop
    - 1920 # Large Desktop

  min-height: 1024

  # Enable Percy-specific features
  enable-javascript: true

  # CSS for stabilizing screenshots
  percy-css: |
    * {
      animation-duration: 0s !important;
      transition-duration: 0s !important;
    }
    [data-percy-hide] {
      visibility: hidden !important;
    }

discovery:
  allowed-hostnames:
    - localhost
    - '*.yourdomain.com'

  network-idle-timeout: 750

BackstopJS Visual Regression

BackstopJS Configuration

// backstop.config.js
module.exports = {
  id: 'visual_regression_test',
  viewports: [
    {
      label: 'phone',
      width: 375,
      height: 667
    },
    {
      label: 'tablet',
      width: 768,
      height: 1024
    },
    {
      label: 'desktop',
      width: 1920,
      height: 1080
    }
  ],

  scenarios: [
    {
      label: 'Homepage',
      url: 'http://localhost:3000',
      delay: 1000,
      misMatchThreshold: 0.1,
      requireSameDimensions: true
    },
    {
      label: 'Button Component',
      url: 'http://localhost:3000/components/button',
      selectors: ['[data-testid="button-showcase"]'],
      delay: 500,
      hoverSelector: '[data-testid="button-primary"]',
      clickSelector: '[data-testid="button-toggle"]'
    },
    {
      label: 'Dashboard - Logged In',
      url: 'http://localhost:3000/dashboard',
      cookiePath: 'backstop_data/cookies.json',
      delay: 2000,
      removeSelectors: [
        '[data-testid="timestamp"]', // Hide dynamic timestamp
        '[data-testid="live-data"]'   // Hide live updating data
      ]
    },
    {
      label: 'Form Validation',
      url: 'http://localhost:3000/contact',
      onBeforeScript: 'puppet/onBefore.js',
      onReadyScript: 'puppet/fillForm.js',
      delay: 500
    }
  ],

  paths: {
    bitmaps_reference: 'backstop_data/bitmaps_reference',
    bitmaps_test: 'backstop_data/bitmaps_test',
    engine_scripts: 'backstop_data/engine_scripts',
    html_report: 'backstop_data/html_report',
    ci_report: 'backstop_data/ci_report'
  },

  report: ['browser', 'CI'],
  engine: 'puppeteer',
  engineOptions: {
    args: ['--no-sandbox']
  },

  asyncCaptureLimit: 5,
  asyncCompareLimit: 50,

  debug: false,
  debugWindow: false
}

BackstopJS Custom Scripts

// backstop_data/engine_scripts/puppet/fillForm.js
module.exports = async (page, scenario, viewport) => {
  console.log('Filling form for scenario:', scenario.label)

  // Fill form fields
  await page.type('[name="email"]', 'test@example.com')
  await page.type('[name="name"]', 'Test User')
  await page.type('[name="message"]', 'This is a test message')

  // Trigger validation by clicking submit
  await page.click('button[type="submit"]')

  // Wait for validation messages
  await page.waitForSelector('.validation-message', { timeout: 1000 })
}

// backstop_data/engine_scripts/puppet/onBefore.js
module.exports = async (page, scenario, viewport) => {
  console.log('Running onBefore for:', scenario.label)

  // Set cookies for authenticated scenarios
  if (scenario.cookiePath) {
    const cookies = require(scenario.cookiePath)
    await page.setCookie(...cookies)
  }

  // Hide dynamic elements
  await page.evaluateOnNewDocument(() => {
    window.localStorage.setItem('disable-animations', 'true')
  })
}

CI/CD Integration

GitHub Actions with Playwright

# .github/workflows/visual-tests.yml
name: Visual Regression Tests

on:
  pull_request:
    branches: [main, develop]

jobs:
  visual-tests:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 18
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Install Playwright Browsers
        run: npx playwright install --with-deps

      - name: Run visual tests
        run: npm run test:visual

      - name: Upload test results
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: visual-test-results
          path: test-results/
          retention-days: 30

      - name: Upload screenshots
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: failed-screenshots
          path: test-results/**/*-diff.png

GitHub Actions with Chromatic

# .github/workflows/chromatic.yml
name: Chromatic Visual Tests

on: push

jobs:
  chromatic:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0 # Full git history for Chromatic

      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 18

      - name: Install dependencies
        run: npm ci

      - name: Run Chromatic
        uses: chromaui/action@v1
        with:
          projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
          buildScriptName: 'build-storybook'
          exitZeroOnChanges: true
          onlyChanged: true # Only test changed stories

Best Practices Checklist

• [ ] Disable animations and transitions in visual tests
• [ ] Wait for network idle before taking screenshots
• [ ] Use data-testid attributes for stable selectors
• [ ] Mask or hide dynamic content (timestamps, live data)
• [ ] Test multiple viewports (mobile, tablet, desktop)
• [ ] Test interactive states (hover, focus, disabled)
• [ ] Test theme variations (light, dark, high-contrast)
• [ ] Use consistent mock data for charts and dynamic content
• [ ] Set appropriate mismatch thresholds (0.1% - 1%)
• [ ] Store reference screenshots in version control or cloud
• [ ] Review visual diffs in CI/CD pipeline
• [ ] Test cross-browser compatibility (Chrome, Firefox, Safari)
• [ ] Isolate component testing with Storybook

Common Pitfalls

[FAIL] Not waiting for content to load:

// Bad - Screenshot taken before content loads
await page.goto('/dashboard')
await expect(page).toHaveScreenshot()

// Good - Wait for network idle
await page.goto('/dashboard')
await page.waitForLoadState('networkidle')
await expect(page).toHaveScreenshot()

[FAIL] Testing with animations enabled:

// Bad - Animations cause flaky tests
await expect(page).toHaveScreenshot()

// Good - Disable animations
await expect(page).toHaveScreenshot({
  animations: 'disabled'
})

[FAIL] Not handling dynamic content:

// Bad - Timestamp causes every test to fail
await expect(page).toHaveScreenshot('dashboard.png')

// Good - Mask dynamic elements
await expect(page).toHaveScreenshot('dashboard.png', {
  mask: [page.locator('[data-testid="timestamp"]')]
})

[FAIL] Overly strict thresholds:

// Bad - Fails on minor anti-aliasing differences
await expect(page).toHaveScreenshot({
  maxDiffPixels: 0
})

// Good - Allow minor pixel differences
await expect(page).toHaveScreenshot({
  maxDiffPixels: 100,
  threshold: 0.2
})

Testing Workflow

1. Initial baseline: Run tests and accept all screenshots as baseline

npm run test:visual -- --update-snapshots

2. Development: Make UI changes and run tests

npm run test:visual

3. Review diffs: Check diff images for unintended changes

open test-results/*-diff.png

4. Update baselines: Accept intentional changes

npm run test:visual -- --update-snapshots

5. CI/CD: Automated visual testing on every PR

Related Resources

See ../../references/comprehensive-testing-guide.md for complete testing guide across all layers.

Chaos Engineering & Resilience Testing

Proactive reliability validation through controlled failure injection. Use chaos engineering to discover weaknesses before they cause production incidents.

Contents

• When to Use This Reference
• Core Principles
• Chaos Engineering Tools (2026)
• Experiment Categories
• CI/CD Integration
• Compliance: DORA & SOC 2
• Chaos Experiment Report
• Steady State Metrics
• Best Practices
• Quick Start Checklist
• Related References
• External Resources

---

When to Use This Reference

• Validating system resilience before major releases
• Preparing for compliance audits (DORA, SOC 2)
• Building confidence in disaster recovery plans
• Testing failover mechanisms and circuit breakers
• Validating auto-scaling and self-healing infrastructure

---

Core Principles

Build-Measure-Learn Cycle

1. STEADY STATE
   Define normal behavior metrics (latency, error rate, throughput)

2. HYPOTHESIS
   "The system will maintain <metric> within <threshold> when <failure>"

3. EXPERIMENT
   Inject controlled failure in staging/production

4. OBSERVE
   Measure deviation from steady state

5. LEARN
   Fix weaknesses, update runbooks, repeat

Blast Radius Containment

Rule: Start small, expand gradually, always have a kill switch.

---

Chaos Engineering Tools (2026)

Tool Comparison

LitmusChaos Example

# litmus-experiment.yaml
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosExperiment
metadata:
  name: pod-delete
spec:
  definition:
    scope: Namespaced
    permissions:
      - apiGroups: [""]
        resources: ["pods"]
        verbs: ["delete", "list", "get"]
    image: litmuschaos/go-runner:latest
    args:
      - -c
      - ./experiments -name pod-delete
    env:
      - name: TOTAL_CHAOS_DURATION
        value: "30"
      - name: CHAOS_INTERVAL
        value: "10"
      - name: FORCE
        value: "false"

Gremlin Attack Types

Resource Attacks:
├── CPU              # Consume CPU cycles
├── Memory           # Consume memory
├── Disk             # Fill disk space
├── IO               # Slow disk I/O
└── Process Killer   # Kill specific processes

Network Attacks:
├── Latency          # Add network delay
├── Packet Loss      # Drop packets
├── Blackhole        # Drop all traffic
├── DNS              # DNS failures
└── Certificate      # TLS/SSL failures

State Attacks:
├── Shutdown         # Graceful shutdown
├── Time Travel      # Change system clock
└── Process Killer   # Kill by name/PID

---

Experiment Categories

1. Infrastructure Failures

2. Network Failures

3. Application Failures

---

CI/CD Integration

GitHub Actions Example

name: Chaos Testing
on:
  schedule:
    - cron: '0 2 * * 1-5'  # Weekday nights
  workflow_dispatch:
    inputs:
      experiment:
        description: 'Chaos experiment to run'
        required: true
        type: choice
        options:
          - pod-delete
          - network-latency
          - cpu-stress

jobs:
  chaos-test:
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - uses: actions/checkout@v4

      - name: Setup kubectl
        uses: azure/setup-kubectl@v3

      - name: Install LitmusChaos
        run: |
          kubectl apply -f https://litmuschaos.github.io/litmus/litmus-operator-v3.0.0.yaml
          kubectl wait --for=condition=Ready pods -l app=chaos-operator -n litmus

      - name: Run Chaos Experiment
        run: |
          kubectl apply -f chaos-experiments/${{ inputs.experiment }}.yaml
          kubectl wait --for=condition=ChaosResultVerdict=Pass \
            chaosresult/${{ inputs.experiment }}-result -n default --timeout=300s

      - name: Collect Results
        if: always()
        run: |
          kubectl get chaosresult -n default -o yaml > chaos-results.yaml

      - name: Upload Results
        uses: actions/upload-artifact@v4
        with:
          name: chaos-results
          path: chaos-results.yaml

Game Day Automation

#!/bin/bash
# game-day-runner.sh

set -euo pipefail

EXPERIMENTS=(
  "pod-delete:api-service"
  "network-latency:database:500ms"
  "cpu-stress:worker:80%"
)

echo "Starting Game Day: $(date)"

for exp in "${EXPERIMENTS[@]}"; do
  IFS=':' read -r type target params <<< "$exp"

  echo "Running: $type on $target with $params"

  # Run experiment
  litmus run --experiment "$type" --target "$target" --params "$params"

  # Collect metrics during experiment
  prometheus-query "rate(http_requests_total{status=~'5..'}[1m])" > "metrics-$type.json"

  # Wait for recovery
  sleep 60

  # Verify steady state restored
  if ! verify-steady-state; then
    echo "ALERT: System did not recover from $type"
    exit 1
  fi
done

echo "Game Day Complete: All experiments passed"

---

Compliance: DORA & SOC 2

DORA (Digital Operational Resilience Act)

DORA requires financial entities to regularly test ICT resilience. Chaos engineering provides:

SOC 2 Alignment

Audit Documentation

## Chaos Experiment Report

**Experiment ID:** CHX-2026-001
**Date:** 2026-01-18
**Environment:** Production (5% canary)
**Conducted By:** SRE Team

### Hypothesis
The payment service will maintain <100ms p99 latency when
the primary database fails over to replica.

### Experiment Details
- **Attack Type:** Database primary termination
- **Duration:** 5 minutes
- **Blast Radius:** 5% of production traffic
- **Kill Switch:** Immediate rollback via feature flag

### Results
| Metric | Baseline | During Experiment | Pass/Fail |
|--------|----------|-------------------|-----------|
| p99 Latency | 45ms | 120ms | FAIL |
| Error Rate | 0.01% | 0.8% | FAIL |
| Failover Time | N/A | 45s | N/A |

### Findings
1. Connection pool not warming on failover
2. DNS TTL too high (300s → should be 30s)
3. Health checks not detecting stale connections

### Remediation
- [ ] Implement connection pool pre-warming
- [ ] Reduce DNS TTL to 30s
- [ ] Add active health checks to connection pool

### Sign-off
- Engineering Lead: _____________ Date: _______
- SRE Lead: _____________ Date: _______

---

Steady State Metrics

Define Before Experimenting

steady_state:
  metrics:
    - name: error_rate
      query: "sum(rate(http_requests_total{status=~'5..'}[5m])) / sum(rate(http_requests_total[5m]))"
      threshold: "< 0.01"  # 1%

    - name: p99_latency
      query: "histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))"
      threshold: "< 0.2"  # 200ms

    - name: throughput
      query: "sum(rate(http_requests_total[5m]))"
      threshold: "> 1000"  # 1000 RPS

    - name: saturation
      query: "avg(container_memory_usage_bytes / container_spec_memory_limit_bytes)"
      threshold: "< 0.8"  # 80%

SLO-Based Thresholds

---

Best Practices

Do

• Start with read-only experiments (latency, not data corruption)
• Run experiments during business hours (team available)
• Have clear rollback procedures before starting
• Document hypotheses and results
• Share learnings across teams
• Automate recurring experiments in CI/CD

Avoid

• Running in production without staging validation first
• Experiments without clear success criteria
• Chaos without observability (you won't know what happened)
• Skipping post-mortems after failures
• Running during incident response or high-traffic events

---

Quick Start Checklist

• [ ] Define steady state metrics (error rate, latency, throughput)
• [ ] Choose chaos tool (LitmusChaos for K8s, Gremlin for enterprise)
• [ ] Start in development/staging environment
• [ ] Write first hypothesis: "System X will maintain Y when Z fails"
• [ ] Run experiment with minimal blast radius
• [ ] Document findings and remediation
• [ ] Schedule recurring experiments (weekly/monthly)
• [ ] Integrate with CI/CD for pre-release validation

---

Related References

• operational-playbook.md — Test pyramid and CI gates
• ../SKILL.md — Main testing strategy overview
• ../../ops-devops-platform/SKILL.md — CI/CD and infrastructure
• ../../software-security-appsec/SKILL.md — Security testing

---

External Resources

• Gremlin Chaos Engineering
• LitmusChaos
• AWS Fault Injection Simulator
• Steadybit
• Principles of Chaos Engineering