
Senior Qa
- 197 installs
- 451 repo stars
- Updated July 21, 2026
- borghei/claude-skills
Plan test strategy, write cases, automate regression, triage defects, and sign off releases across web, mobile, and API surfaces.
About
Senior QA skill for shipping reliable software: defines test pyramids, authors detailed cases, advises on automation and CI hooks, reproduces edge-case bugs, and produces release sign-off guidance for web, mobile, and backend integrations.
- Risk-based test planning
- Automation framework guidance
- API contract and E2E coverage
- Defect triage and severity rules
- Release readiness checklists
Senior Qa by the numbers
- 197 all-time installs (skills.sh)
- Ranked #811 of 2,153 Testing & QA skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/borghei/claude-skills --skill senior-qaAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 197 |
|---|---|
| repo stars | ★ 451 |
| Last updated | July 21, 2026 |
| Repository | borghei/claude-skills ↗ |
What it does
Plan test strategy, write cases, automate regression, triage defects, and sign off releases across web, mobile, and API surfaces.
Files
Senior QA Engineer
Test automation, coverage analysis, and quality assurance patterns for React and Next.js applications. Generates Jest + React Testing Library unit test stubs, analyzes Istanbul/LCOV coverage for gaps, and scaffolds Playwright E2E suites for App Router and Pages Router projects.
Core Capabilities
- Unit test generation — scan React/TypeScript components and emit Jest + RTL stubs (render, interaction, state, optional
jest-axea11y). - Coverage analysis — parse Istanbul/LCOV reports, identify gaps by severity, flag critical business-logic paths, and emit text/HTML/JSON recommendations.
- E2E scaffolding — detect Next.js routes (dynamic segments, route groups, auth pages) and generate Playwright specs, Page Object Models, fixtures, and config.
- Test strategy — the testing pyramid, coverage targets by code type, and test organization patterns.
- Automation patterns — Page Object Model, data factories, MSW mocking, fixtures, and custom render utilities.
- Quality practices — testable code design, AAA structure, test isolation, flaky-test elimination, and quality metrics.
When to Use
- Setting up unit tests for new or existing React components.
- Improving test coverage or preparing for release.
- Setting up Playwright E2E tests for a Next.js project.
- Configuring Jest thresholds or improving overall test quality.
Clarify First
Before generating tests, confirm these inputs. If any is unknown or vague, ASK — do not assume:
- [ ] Test layer — unit (Jest+RTL) vs E2E (Playwright) (selects
test_suite_generatorvse2e_test_scaffolder) - [ ] Source path — the components or Next.js routes to generate tests from (the scaffolder's input)
- [ ] Coverage focus — happy path only vs interaction/state/a11y (
jest-axe) (sets which test stubs are emitted)
Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.
Tools
| Tool | Purpose | Command |
|---|---|---|
test_suite_generator.py | Generate Jest + RTL test stubs from React components | python scripts/test_suite_generator.py src/components/ --output __tests__/ |
coverage_analyzer.py | Analyze Istanbul/LCOV coverage and report gaps | python scripts/coverage_analyzer.py coverage/coverage-final.json --threshold 80 |
e2e_test_scaffolder.py | Scaffold Playwright E2E tests from Next.js routes | python scripts/e2e_test_scaffolder.py src/app/ --output e2e/ |
References
Load the reference that matches the task — keep this file lean and pull detail on demand:
- [references/qa-workflows-and-tools.md](references/qa-workflows-and-tools.md) — quick start, the three tool overviews with sample output, the unit-test/coverage/E2E workflows, common pattern snippets (RTL queries, async, MSW, Playwright locators, jest thresholds), common commands, troubleshooting table, and success criteria. Read when running a workflow.
- [references/tool-cli-reference.md](references/tool-cli-reference.md) — full flag tables, examples, output formats, and generated artifacts for all three scripts. Read before running the scripts.
- [references/testing_strategies.md](references/testing_strategies.md) — the testing pyramid, testing types, coverage targets and thresholds, and test organization patterns. Read when designing test strategy.
- [references/test_automation_patterns.md](references/test_automation_patterns.md) — Page Object Model, data factories, fixture management, mocking strategies (MSW), and custom test utilities. Read when writing test code.
- [references/qa_best_practices.md](references/qa_best_practices.md) — writing testable code, naming conventions, AAA pattern, test isolation, flaky-test debugging, and quality metrics. Read when improving test quality.
Scope & Limitations
This skill covers:
- Unit test stub generation for React/TypeScript functional and class components using Jest and React Testing Library.
- Coverage analysis and gap identification from Istanbul JSON and LCOV report formats.
- E2E test scaffolding for Next.js App Router and Pages Router projects using Playwright.
- Accessibility test generation via
jest-axeintegration.
This skill does NOT cover:
- Backend API testing (see
senior-backendfor Express/Node.js testing patterns). - Performance or load testing (see
senior-devopsfor infrastructure and performance tooling). - Visual regression testing or screenshot comparison workflows.
- Mobile-native testing (React Native, Flutter) -- this skill targets web browser-based testing only.
Integration Points
| Skill | Integration | Data Flow |
|---|---|---|
senior-frontend | Generated test stubs align with component patterns from the frontend skill | Frontend components --> test_suite_generator.py --> test files |
senior-fullstack | Code quality analyzer consumes the same coverage reports produced here | coverage_analyzer.py output --> fullstack quality dashboard |
senior-devops | E2E test scaffolder generates CI-ready Playwright configs that plug into DevOps pipelines | e2e_test_scaffolder.py --> playwright.config.ts --> GitHub Actions workflow |
code-reviewer | Coverage gaps feed directly into code review checklists for untested changes | coverage_analyzer.py gaps --> review checklist items |
tdd-guide | TDD workflow references this skill's test generator for initial red-phase stub creation | TDD cycle --> test_suite_generator.py --scan-only --> write tests --> implement |
qa-browser-automation | Page Object Models generated here are consumed by the browser automation skill for advanced E2E scenarios | e2e_test_scaffolder.py --include-pom --> POM classes --> browser automation flows |
Senior QA Testing Engineer Skill
Production-ready quality assurance and test automation skill for React/Next.js applications.
Tech Stack Focus
| Category | Technologies |
|---|---|
| Unit/Integration | Jest, React Testing Library |
| E2E Testing | Playwright |
| Coverage Analysis | Istanbul, NYC, LCOV |
| API Mocking | MSW (Mock Service Worker) |
| Accessibility | jest-axe, @axe-core/playwright |
Quick Start
# Generate component tests
python scripts/test_suite_generator.py src/components --include-a11y
# Analyze coverage gaps
python scripts/coverage_analyzer.py coverage/coverage-final.json --threshold 80 --strict
# Scaffold E2E tests for Next.js
python scripts/e2e_test_scaffolder.py src/app --page-objectsScripts
test_suite_generator.py
Scans React/TypeScript components and generates Jest + React Testing Library test stubs.
Features:
- Detects functional, class, memo, and forwardRef components
- Generates render, interaction, and accessibility tests
- Identifies props requiring mock data
- Optional
--include-a11yfor jest-axe assertions
Usage:
python scripts/test_suite_generator.py <component-dir> [options]
Options:
--scan-only List components without generating tests
--include-a11y Add accessibility test assertions
--output DIR Output directory for test filescoverage_analyzer.py
Parses Istanbul JSON or LCOV coverage reports and identifies testing gaps.
Features:
- Calculates line, branch, function, and statement coverage
- Identifies critical untested paths (auth, payment, API routes)
- Generates text and HTML reports
- Threshold enforcement with
--strictflag
Usage:
python scripts/coverage_analyzer.py <coverage-file> [options]
Options:
--threshold N Minimum coverage percentage (default: 80)
--strict Exit with error if below threshold
--format FORMAT Output format: text, json, html
--output FILE Output file pathe2e_test_scaffolder.py
Scans Next.js App Router or Pages Router directories and generates Playwright tests.
Features:
- Detects routes, dynamic parameters, and layouts
- Generates test files per route with navigation and content checks
- Optional Page Object Model class generation
- Generates
playwright.config.tsand auth fixtures
Usage:
python scripts/e2e_test_scaffolder.py <app-dir> [options]
Options:
--page-objects Generate Page Object Model classes
--output DIR Output directory for E2E tests
--base-url URL Base URL for tests (default: http://localhost:3000)References
testing_strategies.md (650 lines)
Comprehensive testing strategy guide covering:
- Test pyramid and distribution (70% unit, 20% integration, 10% E2E)
- Coverage targets by project type
- Testing types (unit, integration, E2E, visual, accessibility)
- CI/CD integration patterns
- Testing decision framework
test_automation_patterns.md (1010 lines)
React/Next.js test automation patterns:
- Page Object Model implementation for Playwright
- Test data factories and builder patterns
- Fixture management (Playwright and Jest)
- Mocking strategies (MSW, Jest module mocking)
- Custom test utilities (
renderWithProviders) - Async testing patterns
- Snapshot testing guidelines
qa_best_practices.md (965 lines)
Quality assurance best practices:
- Writing testable React code
- Test naming conventions (Describe-It pattern)
- Arrange-Act-Assert structure
- Test isolation principles
- Handling flaky tests
- Debugging failed tests
- Quality metrics and KPIs
Workflows
Workflow 1: New Component Testing
1. Create component in src/components/ 2. Run test_suite_generator.py to generate test stub 3. Fill in test assertions based on component behavior 4. Run npm test to verify tests pass 5. Check coverage with coverage_analyzer.py
Workflow 2: E2E Test Setup
1. Run e2e_test_scaffolder.py on your Next.js app directory 2. Review generated tests in e2e/ directory 3. Customize Page Objects for complex interactions 4. Run npx playwright test to execute 5. Configure CI/CD with generated playwright.config.ts
Workflow 3: Coverage Gap Analysis
1. Run tests with coverage: npm test -- --coverage 2. Analyze with coverage_analyzer.py --strict --threshold 80 3. Review critical untested paths in report 4. Prioritize tests for auth, payment, and API routes 5. Re-run analysis to verify improvement
Test Pyramid Targets
| Test Type | Ratio | Focus |
|---|---|---|
| Unit | 70% | Individual functions, utilities, hooks |
| Integration | 20% | Component interactions, API calls, state |
| E2E | 10% | Critical user journeys, happy paths |
Coverage Targets
| Project Type | Line | Branch | Function |
|---|---|---|---|
| Startup/MVP | 60% | 50% | 70% |
| Production | 80% | 70% | 85% |
| Enterprise | 90% | 85% | 95% |
CI/CD Integration
# .github/workflows/test.yml
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install dependencies
run: npm ci
- name: Run unit tests
run: npm test -- --coverage
- name: Run E2E tests
run: npx playwright test
- name: Upload coverage
uses: codecov/codecov-action@v4Related Skills
- senior-frontend - React/Next.js component development
- senior-fullstack - Full application architecture
- senior-devops - CI/CD pipeline setup
- code-reviewer - Code review with testing focus
---
Version: 2.0.0 Last Updated: January 2026 Tech Focus: React 18+, Next.js 14+, Jest 29+, Playwright 1.40+
QA Best Practices for React and Next.js
Guidelines for writing maintainable tests, debugging failures, and measuring test quality.
---
Table of Contents
- Writing Testable Code
- Test Naming Conventions
- Arrange-Act-Assert Pattern
- Test Isolation Principles
- Handling Flaky Tests
- Code Review for Testability
- Test Maintenance Strategies
- Debugging Failed Tests
- Quality Metrics and KPIs
---
Writing Testable Code
Testable code is easy to understand, has clear boundaries, and minimizes dependencies.
Dependency Injection
Instead of creating dependencies inside functions, pass them as parameters.
Hard to Test:
// src/services/userService.ts
import { prisma } from '../lib/prisma';
import { sendEmail } from '../lib/email';
export async function createUser(data: UserInput) {
const user = await prisma.user.create({ data });
await sendEmail(user.email, 'Welcome!');
return user;
}Easy to Test:
// src/services/userService.ts
export function createUserService(
db: PrismaClient,
emailService: EmailService
) {
return {
async createUser(data: UserInput) {
const user = await db.user.create({ data });
await emailService.send(user.email, 'Welcome!');
return user;
},
};
}
// Usage in app
const userService = createUserService(prisma, emailService);
// Usage in tests
const mockDb = { user: { create: jest.fn() } };
const mockEmail = { send: jest.fn() };
const testService = createUserService(mockDb, mockEmail);Pure Functions
Pure functions are deterministic and have no side effects, making them trivial to test.
Impure (Hard to Test):
function formatTimestamp() {
const now = new Date();
return `${now.getFullYear()}-${now.getMonth() + 1}-${now.getDate()}`;
}Pure (Easy to Test):
function formatTimestamp(date: Date): string {
return `${date.getFullYear()}-${date.getMonth() + 1}-${date.getDate()}`;
}
// Test
expect(formatTimestamp(new Date('2024-03-15'))).toBe('2024-3-15');Separation of Concerns
Separate business logic from UI and I/O operations.
Mixed Concerns (Hard to Test):
// Component with embedded business logic
function CheckoutForm() {
const [total, setTotal] = useState(0);
const handleSubmit = async (items: CartItem[]) => {
// Business logic mixed with UI
let sum = 0;
for (const item of items) {
sum += item.price * item.quantity;
if (item.category === 'electronics') {
sum *= 0.9; // 10% discount
}
}
const tax = sum * 0.08;
const finalTotal = sum + tax;
// API call
await fetch('/api/orders', {
method: 'POST',
body: JSON.stringify({ items, total: finalTotal }),
});
setTotal(finalTotal);
};
return <form onSubmit={handleSubmit}>...</form>;
}Separated Concerns (Easy to Test):
// Pure business logic (easy to unit test)
export function calculateOrderTotal(items: CartItem[]): number {
return items.reduce((sum, item) => {
const subtotal = item.price * item.quantity;
const discount = item.category === 'electronics' ? 0.9 : 1;
return sum + subtotal * discount;
}, 0);
}
export function calculateTax(subtotal: number, rate = 0.08): number {
return subtotal * rate;
}
// Custom hook for order logic (testable with renderHook)
export function useCheckout() {
const [total, setTotal] = useState(0);
const mutation = useMutation(createOrder);
const checkout = async (items: CartItem[]) => {
const subtotal = calculateOrderTotal(items);
const tax = calculateTax(subtotal);
const finalTotal = subtotal + tax;
await mutation.mutateAsync({ items, total: finalTotal });
setTotal(finalTotal);
};
return { checkout, total, isLoading: mutation.isLoading };
}
// Component (integration testable)
function CheckoutForm() {
const { checkout, total, isLoading } = useCheckout();
return <form onSubmit={() => checkout(items)}>...</form>;
}Component Design for Testability
| Pattern | Testability | Example |
|---|---|---|
| Props over context | High | <Button disabled={!valid}> |
| Callbacks over side effects | High | onSubmit={handleSubmit} |
| Controlled components | High | <Input value={value} onChange={...}> |
| Render props | Medium | <DataProvider render={data => ...}> |
| Internal state | Low | const [x, setX] = useState() |
| Global state | Low | useGlobalStore() |
---
Test Naming Conventions
Good test names document expected behavior and help diagnose failures.
Naming Patterns
Pattern 1: should [expected behavior] when [condition]
describe('LoginForm', () => {
it('should display error message when credentials are invalid', () => {});
it('should redirect to dashboard when login succeeds', () => {});
it('should disable submit button when form is submitting', () => {});
});Pattern 2: [method/action] [expected result]
describe('calculateDiscount', () => {
it('returns 0 for orders under $50', () => {});
it('returns 10% for orders $50-$99', () => {});
it('returns 20% for orders $100+', () => {});
});Pattern 3: given [context], when [action], then [result]
describe('ShoppingCart', () => {
it('given an empty cart, when adding an item, then cart count is 1', () => {});
it('given items in cart, when removing all, then cart is empty', () => {});
});Describe Block Organization
describe('UserService', () => {
describe('createUser', () => {
describe('with valid input', () => {
it('creates user in database', () => {});
it('sends welcome email', () => {});
it('returns user with id', () => {});
});
describe('with invalid input', () => {
it('throws ValidationError for missing email', () => {});
it('throws ValidationError for invalid email format', () => {});
it('throws ConflictError for duplicate email', () => {});
});
});
describe('deleteUser', () => {
it('removes user from database', () => {});
it('throws NotFoundError for non-existent user', () => {});
});
});Anti-patterns to Avoid
| Bad | Good | Why |
|---|---|---|
it('works') | it('returns sum of two numbers') | Describes behavior |
it('test 1') | it('handles empty array') | Specific scenario |
it('should do stuff') | it('should validate email format') | Clear expectation |
| Duplicating code in name | Describing behavior | Readable output |
---
Arrange-Act-Assert Pattern
The AAA pattern structures tests into three clear phases.
Structure
it('calculates total with discount', () => {
// Arrange - Set up test data and conditions
const items = [
{ name: 'Widget', price: 100, quantity: 2 },
{ name: 'Gadget', price: 50, quantity: 1 },
];
const discountRate = 0.1;
// Act - Execute the code being tested
const result = calculateTotal(items, discountRate);
// Assert - Verify the outcome
expect(result).toBe(225); // (200 + 50) * 0.9
});Async Example
it('fetches user profile', async () => {
// Arrange
const userId = '123';
server.use(
rest.get('/api/users/:id', (req, res, ctx) =>
res(ctx.json({ id: userId, name: 'John' }))
)
);
// Act
render(<UserProfile userId={userId} />);
// Assert
await expect(screen.findByText('John')).resolves.toBeInTheDocument();
});Component Testing Example
it('submits form with user input', async () => {
// Arrange
const user = userEvent.setup();
const onSubmit = jest.fn();
render(<ContactForm onSubmit={onSubmit} />);
// Act
await user.type(screen.getByLabelText('Name'), 'John Doe');
await user.type(screen.getByLabelText('Email'), 'john@example.com');
await user.type(screen.getByLabelText('Message'), 'Hello!');
await user.click(screen.getByRole('button', { name: 'Send' }));
// Assert
expect(onSubmit).toHaveBeenCalledWith({
name: 'John Doe',
email: 'john@example.com',
message: 'Hello!',
});
});Guidelines
1. One Act per test - Test one behavior at a time 2. Multiple assertions OK - If they verify the same behavior 3. Avoid logic in tests - No if/else, loops in test code 4. Setup in Arrange, not beforeEach - Unless truly shared
---
Test Isolation Principles
Isolated tests are independent, repeatable, and can run in any order.
State Isolation
describe('CartService', () => {
let cartService: CartService;
// Fresh instance for each test
beforeEach(() => {
cartService = new CartService();
});
it('adds item to empty cart', () => {
cartService.addItem({ id: '1', quantity: 1 });
expect(cartService.getItems()).toHaveLength(1);
});
it('starts with empty cart', () => {
// Not affected by previous test
expect(cartService.getItems()).toHaveLength(0);
});
});Database Isolation
describe('UserRepository', () => {
beforeAll(async () => {
// Connect to test database
await db.connect(process.env.TEST_DATABASE_URL);
});
beforeEach(async () => {
// Clean database before each test
await db.query('TRUNCATE users CASCADE');
});
afterAll(async () => {
await db.disconnect();
});
it('creates user', async () => {
const user = await userRepo.create({ email: 'test@example.com' });
expect(user.id).toBeDefined();
});
});API Mocking Isolation
describe('ProductList', () => {
// Reset handlers after each test
afterEach(() => server.resetHandlers());
it('shows products from API', async () => {
// Default handler returns products
render(<ProductList />);
await expect(screen.findByText('Widget')).resolves.toBeInTheDocument();
});
it('shows error on API failure', async () => {
// Override handler for this test only
server.use(
rest.get('/api/products', (req, res, ctx) =>
res(ctx.status(500))
)
);
render(<ProductList />);
await expect(screen.findByText('Error')).resolves.toBeInTheDocument();
});
it('shows products again', async () => {
// Back to default handler (server.resetHandlers ran)
render(<ProductList />);
await expect(screen.findByText('Widget')).resolves.toBeInTheDocument();
});
});Isolation Checklist
| Aspect | Solution |
|---|---|
| Global state | Reset in beforeEach |
| Timers | jest.useFakeTimers() + jest.useRealTimers() |
| DOM | RTL's cleanup (automatic) |
| Database | Truncate tables or use transactions |
| API mocks | server.resetHandlers() |
| File system | Use temp directories, clean up in afterEach |
| Environment vars | Restore in afterEach |
---
Handling Flaky Tests
Flaky tests pass and fail intermittently without code changes.
Common Causes and Fixes
1. Timing Issues
// Flaky - race condition
it('shows loading then data', () => {
render(<UserProfile />);
expect(screen.getByText('Loading')).toBeInTheDocument();
expect(screen.getByText('John')).toBeInTheDocument(); // May fail
});
// Fixed - proper async handling
it('shows loading then data', async () => {
render(<UserProfile />);
expect(screen.getByText('Loading')).toBeInTheDocument();
await waitFor(() => {
expect(screen.getByText('John')).toBeInTheDocument();
});
});2. Non-deterministic Data
// Flaky - random data
it('sorts users alphabetically', () => {
const users = [createUser(), createUser(), createUser()];
// Names are random, order unpredictable
});
// Fixed - deterministic data
it('sorts users alphabetically', () => {
const users = [
createUser({ name: 'Charlie' }),
createUser({ name: 'Alice' }),
createUser({ name: 'Bob' }),
];
const sorted = sortUsers(users);
expect(sorted.map(u => u.name)).toEqual(['Alice', 'Bob', 'Charlie']);
});3. Test Order Dependencies
// Flaky - relies on previous test
describe('Counter', () => {
const counter = new Counter(); // Shared instance!
it('increments', () => {
counter.increment();
expect(counter.value).toBe(1);
});
it('starts at zero', () => {
expect(counter.value).toBe(0); // Fails! Value is 1
});
});
// Fixed - fresh instance per test
describe('Counter', () => {
let counter: Counter;
beforeEach(() => {
counter = new Counter();
});
it('increments', () => {
counter.increment();
expect(counter.value).toBe(1);
});
it('starts at zero', () => {
expect(counter.value).toBe(0); // Passes
});
});4. Network/External Dependencies
// Flaky - real network call
it('fetches data', async () => {
const data = await fetch('https://api.example.com/data');
expect(data).toBeDefined();
});
// Fixed - mock the network
it('fetches data', async () => {
server.use(
rest.get('https://api.example.com/data', (req, res, ctx) =>
res(ctx.json({ value: 42 }))
)
);
const data = await fetchData();
expect(data.value).toBe(42);
});Flaky Test Detection
// jest.config.js
module.exports = {
// Run each test multiple times to detect flakiness
testEnvironment: 'jsdom',
// Add reporters to track flaky tests
reporters: [
'default',
['jest-junit', { outputDirectory: './reports' }],
],
};
// Run tests multiple times
// npx jest --runInBand --testTimeout=10000 --repeat=5Quarantine Strategy
1. Identify - Track tests that fail randomly 2. Quarantine - Move to separate suite, run separately 3. Fix - Investigate and fix root cause 4. Restore - Move back to main suite
// Temporarily skip flaky test
it.skip('flaky test to fix', () => {
// TODO: Fix timing issue in #123
});
// Or run only when investigating
it.todo('investigate flaky behavior');---
Code Review for Testability
Questions to ask during code review to ensure testable code.
Testability Checklist
Functions and Methods:
- [ ] Does it have a single responsibility?
- [ ] Are dependencies injected?
- [ ] Can it be tested without mocking internals?
- [ ] Does it return a value or have observable side effects?
Components:
- [ ] Are props descriptive and minimal?
- [ ] Can behavior be triggered via user events?
- [ ] Are loading/error states exposed?
- [ ] Can it be rendered without a full app context?
State Management:
- [ ] Is state minimal and derived where possible?
- [ ] Can state changes be triggered and observed?
- [ ] Are side effects separated from reducers?
Review Comments
Before:
// Hard to test - embedded dependency
function processPayment(order: Order) {
const stripe = new Stripe(process.env.STRIPE_KEY);
return stripe.charges.create({
amount: order.total,
currency: 'usd',
});
}Review Comment:
Consider injecting the payment processor to improve testability:
```typescript
function processPayment(order: Order, processor: PaymentProcessor) {
return processor.charge(order.total, 'usd');
}
```
This allows testing with a mock processor without hitting Stripe's API.
---
Test Maintenance Strategies
Keep tests maintainable as the codebase evolves.
Reducing Duplication
Use helpers for common assertions:
// __tests__/helpers/assertions.ts
export function expectLoadingState(container: HTMLElement) {
expect(within(container).getByRole('progressbar')).toBeInTheDocument();
}
export function expectErrorState(container: HTMLElement, message: string) {
expect(within(container).getByRole('alert')).toHaveTextContent(message);
}
// Usage
it('shows loading state', () => {
render(<DataList />);
expectLoadingState(screen.getByTestId('data-list'));
});Use factory functions:
// Instead of repeating setup
function renderWithUser(ui: ReactElement, user = createUser()) {
return {
user,
...render(<AuthProvider user={user}>{ui}</AuthProvider>),
};
}Updating Tests When Code Changes
Scenario: Renaming a prop
// Old component
<Button onClick={handleClick} />
// New component
<Button onPress={handleClick} />
// Find and update all tests
// grep -r "onClick" __tests__/ --include="*.test.tsx"Scenario: Changing API response shape
// Update factory first
export function createUserResponse(overrides = {}) {
return {
user: { // New nested structure
id: '1',
name: 'Test User',
...overrides,
},
};
}
// Tests automatically get new shapeWhen to Delete Tests
- Redundant coverage - Multiple tests testing the same thing
- Testing implementation - Tests that break on refactor
- Obsolete features - Tests for removed functionality
- Flaky beyond repair - Tests that can't be stabilized
Test Documentation
/**
* @group integration
* @requires database
*
* Tests for the order processing workflow.
* These tests require a running PostgreSQL instance.
*
* Setup: docker-compose up -d postgres
*/
describe('OrderProcessor', () => {
/**
* Verifies that orders with backordered items
* are split into separate fulfillment batches.
*
* Related: JIRA-1234
*/
it('splits orders with backordered items', () => {});
});---
Debugging Failed Tests
Techniques for investigating test failures.
Jest Debugging
Run single test:
# By name pattern
npx jest -t "should validate email"
# By file
npx jest src/utils/__tests__/validation.test.ts
# Watch mode for iteration
npx jest --watchDebug with Node inspector:
node --inspect-brk node_modules/.bin/jest --runInBand
# Open chrome://inspect in ChromeVerbose output:
npx jest --verbose --no-coverageReact Testing Library Debugging
it('renders user profile', async () => {
render(<UserProfile userId="123" />);
// Print current DOM
screen.debug();
// Print specific element
screen.debug(screen.getByRole('heading'));
// Log accessible roles
screen.logTestingPlaygroundURL(); // Opens interactive playground
// Check what queries would match
const element = screen.getByRole('button');
console.log(prettyDOM(element));
});Playwright Debugging
# Debug mode - opens browser with inspector
npx playwright test --debug
# UI mode - visual test runner
npx playwright test --ui
# Headed mode - see browser
npx playwright test --headed
# Trace viewer after failure
npx playwright show-trace trace.zipPause in test:
test('debug this', async ({ page }) => {
await page.goto('/');
await page.pause(); // Opens inspector
await page.click('button');
});Common Failure Patterns
| Symptom | Likely Cause | Debug Approach |
|---|---|---|
| "Unable to find element" | Wrong query or element not rendered | screen.debug(), check async |
| "Expected X, received Y" | Logic error or stale mock | Log intermediate values |
| "Timeout exceeded" | Slow async or missing await | Increase timeout, check promises |
| "Cannot read property of undefined" | Missing mock or setup | Check beforeEach, mock returns |
| Passes locally, fails in CI | Environment difference | Check env vars, timing |
Investigating Flaky Failures
// Add logging for intermittent failures
it('processes order', async () => {
console.log('Test started at', Date.now());
const order = await createOrder();
console.log('Order created:', order.id);
const result = await processOrder(order);
console.log('Process result:', result);
expect(result.status).toBe('completed');
});---
Quality Metrics and KPIs
Measure test suite effectiveness and track quality improvements.
Key Metrics
Coverage Metrics:
| Metric | Target | Measurement |
|---|---|---|
| Line coverage | 80% | jest --coverage |
| Branch coverage | 75% | jest --coverage |
| Function coverage | 80% | jest --coverage |
| Critical path coverage | 95% | Custom tracking |
Test Suite Health:
| Metric | Target | Measurement |
|---|---|---|
| Test pass rate | 100% | CI reports |
| Flaky test rate | <1% | Track retries |
| Test execution time | <5 min | CI timing |
| Tests per component | ≥3 | Test count / components |
Defect Metrics:
| Metric | Target | Measurement |
|---|---|---|
| Defects found in testing | >70% | Bug tracking |
| Defects escaped to prod | <10% | Production bugs |
| Regression rate | <5% | Bugs reintroduced |
| Mean time to detect | <1 day | Bug timestamps |
Dashboard Example
// scripts/test-metrics.ts
import { readCoverageReport } from './utils';
const coverage = readCoverageReport('./coverage/coverage-summary.json');
const testResults = readTestReport('./reports/jest-results.json');
const metrics = {
coverage: {
lines: coverage.total.lines.pct,
branches: coverage.total.branches.pct,
functions: coverage.total.functions.pct,
},
tests: {
total: testResults.numTotalTests,
passed: testResults.numPassedTests,
failed: testResults.numFailedTests,
passRate: (testResults.numPassedTests / testResults.numTotalTests) * 100,
},
execution: {
duration: testResults.testResults.reduce((sum, r) => sum + r.duration, 0),
},
};
console.log('Test Metrics:', JSON.stringify(metrics, null, 2));CI Quality Gates
# .github/workflows/quality.yml
name: Quality Gates
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
- run: npm ci
- run: npm test -- --coverage
# Coverage gate
- name: Check coverage
run: |
coverage=$(jq '.total.lines.pct' coverage/coverage-summary.json)
if (( $(echo "$coverage < 80" | bc -l) )); then
echo "Coverage $coverage% is below 80% threshold"
exit 1
fi
# Test count gate
- name: Check test count
run: |
tests=$(jq '.numTotalTests' reports/test-results.json)
if [ "$tests" -lt 100 ]; then
echo "Test count $tests is below minimum of 100"
exit 1
fiTrend Tracking
Track metrics over time to identify trends:
// Weekly metrics collection
{
"week": "2024-W03",
"coverage": {
"lines": 82.4,
"branches": 76.1,
"trend": "+1.2%" // vs previous week
},
"tests": {
"total": 487,
"new": 23,
"removed": 5
},
"execution": {
"avgDuration": 245, // seconds
"trend": "-12s"
},
"flaky": {
"count": 3,
"rate": 0.6
}
}---
Summary
1. Write testable code - Inject dependencies, use pure functions, separate concerns 2. Name tests clearly - Describe behavior, not implementation 3. Follow AAA pattern - Arrange, Act, Assert for clear structure 4. Isolate tests - Fresh state, reset mocks, no dependencies between tests 5. Fix flaky tests - Handle timing, use deterministic data, mock externals 6. Review for testability - Check during code review, not after 7. Maintain tests - Reduce duplication, update with code changes 8. Debug systematically - Use debug tools, log strategically 9. Measure quality - Track coverage, pass rate, execution time
QA Workflows and Tools
Read this when generating tests, analyzing coverage, scaffolding E2E suites, or looking up the common test patterns and commands — the quick start, tool overviews, step-by-step workflows, pattern snippets, troubleshooting, and success criteria.
Quick Start
# Generate Jest test stubs for React components
python scripts/test_suite_generator.py src/components/ --output __tests__/
# Analyze test coverage from Jest/Istanbul reports
python scripts/coverage_analyzer.py coverage/coverage-final.json --threshold 80
# Scaffold Playwright E2E tests for Next.js routes
python scripts/e2e_test_scaffolder.py src/app/ --output e2e/---
Tools Overview
1. Test Suite Generator
Scans React/TypeScript components and generates Jest + React Testing Library test stubs with proper structure.
Input: Source directory containing React components Output: Test files with describe blocks, render tests, interaction tests
Usage:
# Basic usage - scan components and generate tests
python scripts/test_suite_generator.py src/components/ --output __tests__/
# Output:
# Scanning: src/components/
# Found 24 React components
#
# Generated tests:
# __tests__/Button.test.tsx (render, click handler, disabled state)
# __tests__/Modal.test.tsx (render, open/close, keyboard events)
# __tests__/Form.test.tsx (render, validation, submission)
# ...
#
# Summary: 24 test files, 87 test cases
# Include accessibility tests
python scripts/test_suite_generator.py src/ --output __tests__/ --include-a11y
# Generate with custom template
python scripts/test_suite_generator.py src/ --template custom-template.tsxSupported Patterns:
- Functional components with hooks
- Components with Context providers
- Components with data fetching
- Form components with validation
---
2. Coverage Analyzer
Parses Jest/Istanbul coverage reports and identifies gaps, uncovered branches, and provides actionable recommendations.
Input: Coverage report (JSON or LCOV format) Output: Coverage analysis with recommendations
Usage:
# Analyze coverage report
python scripts/coverage_analyzer.py coverage/coverage-final.json
# Output:
# === Coverage Analysis Report ===
# Overall: 72.4% (target: 80%)
#
# BY TYPE:
# Statements: 74.2%
# Branches: 68.1%
# Functions: 71.8%
# Lines: 73.5%
#
# CRITICAL GAPS (uncovered business logic):
# src/services/payment.ts:45-67 - Payment processing
# src/hooks/useAuth.ts:23-41 - Authentication flow
#
# RECOMMENDATIONS:
# 1. Add tests for payment service error handling
# 2. Cover authentication edge cases
# 3. Test form validation branches
#
# Files below threshold (80%):
# src/components/Checkout.tsx: 45%
# src/services/api.ts: 62%
# Enforce threshold (exit 1 if below)
python scripts/coverage_analyzer.py coverage/ --threshold 80 --strict
# Generate HTML report
python scripts/coverage_analyzer.py coverage/ --format html --output report.html---
3. E2E Test Scaffolder
Scans Next.js pages/app directory and generates Playwright test files with common interactions.
Input: Next.js pages or app directory Output: Playwright test files organized by route
Usage:
# Scaffold E2E tests for Next.js App Router
python scripts/e2e_test_scaffolder.py src/app/ --output e2e/
# Output:
# Scanning: src/app/
# Found 12 routes
#
# Generated E2E tests:
# e2e/home.spec.ts (navigation, hero section)
# e2e/auth/login.spec.ts (form submission, validation)
# e2e/auth/register.spec.ts (registration flow)
# e2e/dashboard.spec.ts (authenticated routes)
# e2e/products/[id].spec.ts (dynamic routes)
# ...
#
# Generated: playwright.config.ts
# Generated: e2e/fixtures/auth.ts
# Include Page Object Model classes
python scripts/e2e_test_scaffolder.py src/app/ --output e2e/ --include-pom
# Generate for specific routes
python scripts/e2e_test_scaffolder.py src/app/ --routes "/login,/dashboard,/checkout"---
QA Workflows
Unit Test Generation Workflow
Use when setting up tests for new or existing React components.
Step 1: Scan project for untested components
python scripts/test_suite_generator.py src/components/ --scan-onlyStep 2: Generate test stubs
python scripts/test_suite_generator.py src/components/ --output __tests__/Step 3: Review and customize generated tests
// __tests__/Button.test.tsx (generated)
import { render, screen, fireEvent } from '@testing-library/react';
import { Button } from '../src/components/Button';
describe('Button', () => {
it('renders with label', () => {
render(<Button>Click me</Button>);
expect(screen.getByRole('button', { name: /click me/i })).toBeInTheDocument();
});
it('calls onClick when clicked', () => {
const handleClick = jest.fn();
render(<Button onClick={handleClick}>Click</Button>);
fireEvent.click(screen.getByRole('button'));
expect(handleClick).toHaveBeenCalledTimes(1);
});
// TODO: Add your specific test cases
});Step 4: Run tests and check coverage
npm test -- --coverage
python scripts/coverage_analyzer.py coverage/coverage-final.json---
Coverage Analysis Workflow
Use when improving test coverage or preparing for release.
Step 1: Generate coverage report
npm test -- --coverage --coverageReporters=jsonStep 2: Analyze coverage gaps
python scripts/coverage_analyzer.py coverage/coverage-final.json --threshold 80Step 3: Identify critical paths
python scripts/coverage_analyzer.py coverage/ --critical-pathsStep 4: Generate missing test stubs
python scripts/test_suite_generator.py src/ --uncovered-only --output __tests__/Step 5: Verify improvement
npm test -- --coverage
python scripts/coverage_analyzer.py coverage/ --compare previous-coverage.json---
E2E Test Setup Workflow
Use when setting up Playwright for a Next.js project.
Step 1: Initialize Playwright (if not installed)
npm init playwright@latestStep 2: Scaffold E2E tests from routes
python scripts/e2e_test_scaffolder.py src/app/ --output e2e/Step 3: Configure authentication fixtures
// e2e/fixtures/auth.ts (generated)
import { test as base } from '@playwright/test';
export const test = base.extend({
authenticatedPage: async ({ page }, use) => {
await page.goto('/login');
await page.fill('[name="email"]', 'test@example.com');
await page.fill('[name="password"]', 'password');
await page.click('button[type="submit"]');
await page.waitForURL('/dashboard');
await use(page);
},
});Step 4: Run E2E tests
npx playwright test
npx playwright show-reportStep 5: Add to CI pipeline
# .github/workflows/e2e.yml
- name: Run E2E tests
run: npx playwright test
- name: Upload report
uses: actions/upload-artifact@v3
with:
name: playwright-report
path: playwright-report/---
Common Patterns Quick Reference
React Testing Library Queries
// Preferred (accessible)
screen.getByRole('button', { name: /submit/i })
screen.getByLabelText(/email/i)
screen.getByPlaceholderText(/search/i)
// Fallback
screen.getByTestId('custom-element')Async Testing
// Wait for element
await screen.findByText(/loaded/i);
// Wait for removal
await waitForElementToBeRemoved(() => screen.queryByText(/loading/i));
// Wait for condition
await waitFor(() => {
expect(mockFn).toHaveBeenCalled();
});Mocking with MSW
import { rest } from 'msw';
import { setupServer } from 'msw/node';
const server = setupServer(
rest.get('/api/users', (req, res, ctx) => {
return res(ctx.json([{ id: 1, name: 'John' }]));
})
);
beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());Playwright Locators
// Preferred
page.getByRole('button', { name: 'Submit' })
page.getByLabel('Email')
page.getByText('Welcome')
// Chaining
page.getByRole('listitem').filter({ hasText: 'Product' })Coverage Thresholds (jest.config.js)
module.exports = {
coverageThreshold: {
global: {
branches: 80,
functions: 80,
lines: 80,
statements: 80,
},
},
};---
Common Commands
# Jest
npm test # Run all tests
npm test -- --watch # Watch mode
npm test -- --coverage # With coverage
npm test -- Button.test.tsx # Single file
# Playwright
npx playwright test # Run all E2E tests
npx playwright test --ui # UI mode
npx playwright test --debug # Debug mode
npx playwright codegen # Generate tests
# Coverage
npm test -- --coverage --coverageReporters=lcov,json
python scripts/coverage_analyzer.py coverage/coverage-final.json---
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
| Test suite generator finds 0 components | Source directory contains no .tsx/.jsx files, or components use non-standard export patterns | Verify the source path points to a directory with React components. Check that components start with an uppercase letter and use standard export const or export function syntax. |
| Coverage analyzer exits with "Could not find or parse coverage data" | The coverage file path is incorrect or the format is unsupported | Ensure you pass a valid coverage-final.json (Istanbul) or lcov.info file. Run npm test -- --coverage --coverageReporters=json first to generate the report. |
| E2E scaffolder detects no routes | The scanned directory lacks page.tsx (App Router) or index.tsx (Pages Router) files | Confirm you are pointing at the correct Next.js directory (e.g., src/app/ for App Router or pages/ for Pages Router). |
| Generated tests fail to compile | Import paths in generated stubs do not match actual project structure | Adjust the relative import paths in each generated .test.tsx file to match your project's alias or directory layout. |
| Coverage report shows 100% on files with no executable code | Istanbul counts files with zero statements as fully covered by default | Exclude non-code files (e.g., type declarations, barrel exports) using the coveragePathIgnorePatterns option in jest.config.js. |
| Flaky E2E tests in CI | Playwright tests depend on a running dev server that is not ready before tests start | Use the webServer option in playwright.config.ts (generated by the scaffolder) and increase the timeout value if the app is slow to start. |
--strict mode fails despite seemingly adequate coverage | Branch coverage is calculated separately and may be below the threshold even when line coverage passes | Run coverage_analyzer.py without --strict first to review the full breakdown, then add targeted branch-coverage tests for conditional logic. |
---
Success Criteria
- Code coverage above 80% across statements, branches, functions, and lines for all non-trivial source files.
- Zero critical coverage gaps in authentication, payment, and security modules as reported by
coverage_analyzer.py --critical-paths. - 100% of React components have at least a render test confirming they mount without crashing.
- All E2E tests pass on Chromium, Firefox, and WebKit in CI before merge to main.
- Branch coverage above 75% for service layers, API handlers, and middleware modules.
- No flaky tests -- every test in the suite produces deterministic results across 3 consecutive CI runs.
- Test generation covers all detected routes -- the E2E scaffolder accounts for every page route in the Next.js application, including dynamic segments.
Test Automation Patterns for React and Next.js
Reusable patterns for structuring test code, mocking dependencies, and handling async operations.
---
Table of Contents
- Page Object Model for React
- Test Data Factories
- Fixture Management
- Mocking Strategies
- Custom Test Utilities
- Async Testing Patterns
- Snapshot Testing Guidelines
---
Page Object Model for React
The Page Object Model (POM) encapsulates page interactions into reusable classes, reducing test maintenance.
Playwright Page Objects
// e2e/pages/LoginPage.ts
import { Page, Locator, expect } from '@playwright/test';
export class LoginPage {
readonly page: Page;
readonly emailInput: Locator;
readonly passwordInput: Locator;
readonly submitButton: Locator;
readonly errorMessage: Locator;
constructor(page: Page) {
this.page = page;
this.emailInput = page.getByLabel('Email');
this.passwordInput = page.getByLabel('Password');
this.submitButton = page.getByRole('button', { name: 'Sign in' });
this.errorMessage = page.getByRole('alert');
}
async goto() {
await this.page.goto('/login');
}
async login(email: string, password: string) {
await this.emailInput.fill(email);
await this.passwordInput.fill(password);
await this.submitButton.click();
}
async expectError(message: string) {
await expect(this.errorMessage).toContainText(message);
}
async expectRedirectToDashboard() {
await expect(this.page).toHaveURL('/dashboard');
}
}Usage in Tests:
// e2e/auth.spec.ts
import { test, expect } from '@playwright/test';
import { LoginPage } from './pages/LoginPage';
test.describe('Authentication', () => {
let loginPage: LoginPage;
test.beforeEach(async ({ page }) => {
loginPage = new LoginPage(page);
await loginPage.goto();
});
test('successful login redirects to dashboard', async () => {
await loginPage.login('user@example.com', 'password123');
await loginPage.expectRedirectToDashboard();
});
test('invalid credentials show error', async () => {
await loginPage.login('user@example.com', 'wrongpassword');
await loginPage.expectError('Invalid credentials');
});
});Component Object Model (React Testing Library)
// __tests__/objects/LoginFormObject.ts
import { screen, fireEvent, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
export class LoginFormObject {
get emailInput() {
return screen.getByLabelText(/email/i);
}
get passwordInput() {
return screen.getByLabelText(/password/i);
}
get submitButton() {
return screen.getByRole('button', { name: /sign in/i });
}
get errorMessage() {
return screen.queryByRole('alert');
}
async fillEmail(email: string) {
await userEvent.type(this.emailInput, email);
}
async fillPassword(password: string) {
await userEvent.type(this.passwordInput, password);
}
async submit() {
await userEvent.click(this.submitButton);
}
async login(email: string, password: string) {
await this.fillEmail(email);
await this.fillPassword(password);
await this.submit();
}
async expectError(message: string) {
await waitFor(() => {
expect(this.errorMessage).toHaveTextContent(message);
});
}
}When to Use POM
| Scenario | Use POM? |
|---|---|
| Complex pages with many interactions | Yes |
| Reusable components tested across suites | Yes |
| Simple single-use tests | No (overkill) |
| E2E tests with shared flows | Yes |
---
Test Data Factories
Factories create test data with sensible defaults, reducing boilerplate and improving maintainability.
Basic Factory Pattern
// __tests__/factories/userFactory.ts
interface User {
id: string;
email: string;
name: string;
role: 'admin' | 'user' | 'guest';
createdAt: Date;
preferences: {
theme: 'light' | 'dark';
notifications: boolean;
};
}
let idCounter = 0;
export function createUser(overrides: Partial<User> = {}): User {
return {
id: `user-${++idCounter}`,
email: `user${idCounter}@example.com`,
name: `Test User ${idCounter}`,
role: 'user',
createdAt: new Date('2024-01-01'),
preferences: {
theme: 'light',
notifications: true,
},
...overrides,
// Deep merge preferences if provided
preferences: {
theme: 'light',
notifications: true,
...overrides.preferences,
},
};
}
// Specialized builders
export function createAdmin(overrides: Partial<User> = {}): User {
return createUser({ role: 'admin', ...overrides });
}
export function createGuest(overrides: Partial<User> = {}): User {
return createUser({
role: 'guest',
name: 'Guest',
email: '',
...overrides,
});
}Builder Pattern for Complex Objects
// __tests__/factories/orderBuilder.ts
interface OrderItem {
productId: string;
quantity: number;
price: number;
}
interface Order {
id: string;
userId: string;
items: OrderItem[];
status: 'pending' | 'processing' | 'shipped' | 'delivered';
total: number;
shippingAddress: Address;
createdAt: Date;
}
export class OrderBuilder {
private order: Partial<Order> = {};
private items: OrderItem[] = [];
withId(id: string): this {
this.order.id = id;
return this;
}
forUser(userId: string): this {
this.order.userId = userId;
return this;
}
withItem(productId: string, quantity: number, price: number): this {
this.items.push({ productId, quantity, price });
return this;
}
withStatus(status: Order['status']): this {
this.order.status = status;
return this;
}
shippedTo(address: Address): this {
this.order.shippingAddress = address;
return this;
}
build(): Order {
const total = this.items.reduce(
(sum, item) => sum + item.price * item.quantity,
0
);
return {
id: this.order.id || `order-${Date.now()}`,
userId: this.order.userId || 'user-1',
items: this.items,
status: this.order.status || 'pending',
total,
shippingAddress: this.order.shippingAddress || createAddress(),
createdAt: new Date(),
};
}
}
// Usage
const order = new OrderBuilder()
.forUser('user-123')
.withItem('product-1', 2, 29.99)
.withItem('product-2', 1, 49.99)
.withStatus('processing')
.build();Factory with Faker
// __tests__/factories/productFactory.ts
import { faker } from '@faker-js/faker';
interface Product {
id: string;
name: string;
description: string;
price: number;
category: string;
inStock: boolean;
imageUrl: string;
}
export function createProduct(overrides: Partial<Product> = {}): Product {
return {
id: faker.string.uuid(),
name: faker.commerce.productName(),
description: faker.commerce.productDescription(),
price: parseFloat(faker.commerce.price({ min: 10, max: 500 })),
category: faker.commerce.department(),
inStock: faker.datatype.boolean({ probability: 0.8 }),
imageUrl: faker.image.url(),
...overrides,
};
}
export function createProducts(count: number): Product[] {
return Array.from({ length: count }, () => createProduct());
}---
Fixture Management
Fixtures provide consistent test data and setup across test suites.
Playwright Fixtures
// e2e/fixtures/auth.ts
import { test as base, Page } from '@playwright/test';
import { createUser } from '../factories/userFactory';
interface AuthFixtures {
authenticatedPage: Page;
adminPage: Page;
testUser: ReturnType<typeof createUser>;
}
export const test = base.extend<AuthFixtures>({
testUser: async ({}, use) => {
const user = createUser();
await use(user);
},
authenticatedPage: async ({ page, testUser }, use) => {
// Login via API to skip UI
await page.request.post('/api/auth/login', {
data: {
email: testUser.email,
password: 'testpassword',
},
});
// Get session cookie
const cookies = await page.context().cookies();
await page.context().addCookies(cookies);
await use(page);
},
adminPage: async ({ page }, use) => {
const admin = createUser({ role: 'admin' });
await page.request.post('/api/auth/login', {
data: {
email: admin.email,
password: 'adminpassword',
},
});
await use(page);
},
});
export { expect } from '@playwright/test';Using Custom Fixtures:
// e2e/dashboard.spec.ts
import { test, expect } from './fixtures/auth';
test('dashboard shows user name', async ({ authenticatedPage, testUser }) => {
await authenticatedPage.goto('/dashboard');
await expect(authenticatedPage.getByText(testUser.name)).toBeVisible();
});
test('admin sees admin panel', async ({ adminPage }) => {
await adminPage.goto('/dashboard');
await expect(adminPage.getByText('Admin Panel')).toBeVisible();
});Jest Test Setup
// jest.setup.ts
import '@testing-library/jest-dom';
import { server } from './__tests__/mocks/server';
// Start MSW server before all tests
beforeAll(() => server.listen({ onUnhandledRequest: 'error' }));
// Reset handlers after each test
afterEach(() => server.resetHandlers());
// Clean up after all tests
afterAll(() => server.close());
// Mock window.matchMedia
Object.defineProperty(window, 'matchMedia', {
writable: true,
value: jest.fn().mockImplementation(query => ({
matches: false,
media: query,
onchange: null,
addListener: jest.fn(),
removeListener: jest.fn(),
addEventListener: jest.fn(),
removeEventListener: jest.fn(),
dispatchEvent: jest.fn(),
})),
});
// Mock IntersectionObserver
global.IntersectionObserver = class IntersectionObserver {
constructor() {}
observe() {}
unobserve() {}
disconnect() {}
};Shared Test Data Files
// __tests__/fixtures/products.json
{
"products": [
{
"id": "prod-1",
"name": "Widget Pro",
"price": 29.99,
"category": "Electronics"
},
{
"id": "prod-2",
"name": "Gadget Plus",
"price": 49.99,
"category": "Electronics"
}
]
}
// __tests__/fixtures/index.ts
import productsData from './products.json';
import usersData from './users.json';
export const fixtures = {
products: productsData.products,
users: usersData.users,
};---
Mocking Strategies
MSW (Mock Service Worker) for API Mocking
MSW intercepts network requests at the service worker level, working in both browser and Node.
Handler Setup:
// __tests__/mocks/handlers.ts
import { rest } from 'msw';
import { createUser } from '../factories/userFactory';
import { createProduct } from '../factories/productFactory';
export const handlers = [
// GET /api/users/:id
rest.get('/api/users/:id', (req, res, ctx) => {
const { id } = req.params;
const user = createUser({ id: id as string });
return res(ctx.json(user));
}),
// GET /api/products
rest.get('/api/products', (req, res, ctx) => {
const category = req.url.searchParams.get('category');
const products = Array.from({ length: 10 }, () => createProduct());
const filtered = category
? products.filter(p => p.category === category)
: products;
return res(ctx.json(filtered));
}),
// POST /api/orders
rest.post('/api/orders', async (req, res, ctx) => {
const body = await req.json();
return res(
ctx.status(201),
ctx.json({
id: `order-${Date.now()}`,
...body,
status: 'pending',
})
);
}),
// Error simulation
rest.get('/api/error', (req, res, ctx) => {
return res(
ctx.status(500),
ctx.json({ error: 'Internal Server Error' })
);
}),
];Server Setup:
// __tests__/mocks/server.ts
import { setupServer } from 'msw/node';
import { handlers } from './handlers';
export const server = setupServer(...handlers);Overriding Handlers in Tests:
// __tests__/components/ProductList.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import { rest } from 'msw';
import { server } from '../mocks/server';
import { ProductList } from '../../src/components/ProductList';
describe('ProductList', () => {
it('shows loading state', () => {
render(<ProductList />);
expect(screen.getByText('Loading...')).toBeInTheDocument();
});
it('renders products', async () => {
render(<ProductList />);
await waitFor(() => {
expect(screen.getAllByTestId('product-card')).toHaveLength(10);
});
});
it('shows error state on API failure', async () => {
server.use(
rest.get('/api/products', (req, res, ctx) => {
return res(ctx.status(500));
})
);
render(<ProductList />);
await waitFor(() => {
expect(screen.getByText(/error loading products/i)).toBeInTheDocument();
});
});
it('shows empty state when no products', async () => {
server.use(
rest.get('/api/products', (req, res, ctx) => {
return res(ctx.json([]));
})
);
render(<ProductList />);
await waitFor(() => {
expect(screen.getByText('No products found')).toBeInTheDocument();
});
});
});Jest Module Mocking
// Mocking a module
jest.mock('../../src/services/analytics', () => ({
trackEvent: jest.fn(),
trackPageView: jest.fn(),
setUser: jest.fn(),
}));
// Mocking with implementation
jest.mock('next/router', () => ({
useRouter: jest.fn().mockReturnValue({
pathname: '/test',
push: jest.fn(),
replace: jest.fn(),
query: {},
}),
}));
// Partial mock (keep some real implementations)
jest.mock('../../src/utils/helpers', () => ({
...jest.requireActual('../../src/utils/helpers'),
sendEmail: jest.fn().mockResolvedValue({ success: true }),
}));Mocking Hooks
// __tests__/hooks/useAuth.test.tsx
import { renderHook, act } from '@testing-library/react';
import { useAuth } from '../../src/hooks/useAuth';
import * as authService from '../../src/services/auth';
jest.mock('../../src/services/auth');
const mockAuthService = authService as jest.Mocked<typeof authService>;
describe('useAuth', () => {
beforeEach(() => {
jest.clearAllMocks();
});
it('logs in user successfully', async () => {
const mockUser = { id: '1', email: 'test@example.com' };
mockAuthService.login.mockResolvedValue(mockUser);
const { result } = renderHook(() => useAuth());
await act(async () => {
await result.current.login('test@example.com', 'password');
});
expect(result.current.user).toEqual(mockUser);
expect(result.current.isAuthenticated).toBe(true);
});
it('handles login error', async () => {
mockAuthService.login.mockRejectedValue(new Error('Invalid credentials'));
const { result } = renderHook(() => useAuth());
await act(async () => {
try {
await result.current.login('test@example.com', 'wrong');
} catch (e) {
// Expected
}
});
expect(result.current.user).toBeNull();
expect(result.current.error).toBe('Invalid credentials');
});
});---
Custom Test Utilities
Render with Providers
// __tests__/utils/renderWithProviders.tsx
import React, { ReactElement } from 'react';
import { render, RenderOptions } from '@testing-library/react';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { ThemeProvider } from '../../src/contexts/ThemeContext';
import { AuthProvider } from '../../src/contexts/AuthContext';
interface ExtendedRenderOptions extends Omit<RenderOptions, 'wrapper'> {
initialUser?: User | null;
theme?: 'light' | 'dark';
}
export function renderWithProviders(
ui: ReactElement,
{
initialUser = null,
theme = 'light',
...renderOptions
}: ExtendedRenderOptions = {}
) {
const queryClient = new QueryClient({
defaultOptions: {
queries: {
retry: false, // Disable retries in tests
},
},
});
function Wrapper({ children }: { children: React.ReactNode }) {
return (
<QueryClientProvider client={queryClient}>
<AuthProvider initialUser={initialUser}>
<ThemeProvider initialTheme={theme}>
{children}
</ThemeProvider>
</AuthProvider>
</QueryClientProvider>
);
}
return {
...render(ui, { wrapper: Wrapper, ...renderOptions }),
queryClient,
};
}
// Re-export everything from RTL
export * from '@testing-library/react';
export { renderWithProviders as render };Usage:
// __tests__/components/Dashboard.test.tsx
import { render, screen } from '../utils/renderWithProviders';
import { Dashboard } from '../../src/components/Dashboard';
import { createUser } from '../factories/userFactory';
describe('Dashboard', () => {
it('shows user greeting when authenticated', () => {
const user = createUser({ name: 'John Doe' });
render(<Dashboard />, { initialUser: user });
expect(screen.getByText('Hello, John Doe')).toBeInTheDocument();
});
it('shows login prompt when not authenticated', () => {
render(<Dashboard />, { initialUser: null });
expect(screen.getByText('Please log in')).toBeInTheDocument();
});
it('applies dark theme', () => {
render(<Dashboard />, { theme: 'dark' });
expect(document.body).toHaveClass('dark');
});
});Custom Matchers
// __tests__/utils/customMatchers.ts
import { expect } from '@playwright/test';
expect.extend({
async toHaveLoadedSuccessfully(page) {
const hasNoErrors = await page.evaluate(() => {
return !document.querySelector('[data-error]');
});
const isLoaded = await page.evaluate(() => {
return document.readyState === 'complete';
});
return {
pass: hasNoErrors && isLoaded,
message: () =>
hasNoErrors
? 'Page loaded with errors'
: 'Page did not finish loading',
};
},
toBeWithinRange(received, floor, ceiling) {
const pass = received >= floor && received <= ceiling;
return {
pass,
message: () =>
`expected ${received} ${pass ? 'not ' : ''}to be within range ${floor} - ${ceiling}`,
};
},
});
// Type declarations
declare global {
namespace PlaywrightTest {
interface Matchers<R> {
toHaveLoadedSuccessfully(): Promise<R>;
}
}
}---
Async Testing Patterns
Waiting for Elements
// Preferred: Use findBy* (waits automatically)
const element = await screen.findByText('Loaded');
// Wait for element to appear
await waitFor(() => {
expect(screen.getByText('Loaded')).toBeInTheDocument();
});
// Wait for element to disappear
await waitForElementToBeRemoved(() => screen.queryByText('Loading...'));
// Wait with custom timeout
await waitFor(
() => {
expect(mockFn).toHaveBeenCalled();
},
{ timeout: 5000 }
);Testing Async State Changes
// __tests__/components/AsyncButton.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { AsyncButton } from '../../src/components/AsyncButton';
describe('AsyncButton', () => {
it('shows loading state during async operation', async () => {
const user = userEvent.setup();
const onClickMock = jest.fn().mockImplementation(
() => new Promise(resolve => setTimeout(resolve, 100))
);
render(<AsyncButton onClick={onClickMock}>Submit</AsyncButton>);
// Initial state
expect(screen.getByRole('button')).toHaveTextContent('Submit');
expect(screen.getByRole('button')).not.toBeDisabled();
// Click and verify loading state
await user.click(screen.getByRole('button'));
expect(screen.getByRole('button')).toHaveTextContent('Loading...');
expect(screen.getByRole('button')).toBeDisabled();
// Wait for completion
await waitFor(() => {
expect(screen.getByRole('button')).toHaveTextContent('Submit');
expect(screen.getByRole('button')).not.toBeDisabled();
});
});
});Testing Debounced/Throttled Functions
// __tests__/components/SearchInput.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { SearchInput } from '../../src/components/SearchInput';
// Use fake timers for debounce testing
jest.useFakeTimers();
describe('SearchInput', () => {
it('debounces search calls', async () => {
const user = userEvent.setup({ advanceTimers: jest.advanceTimersByTime });
const onSearchMock = jest.fn();
render(<SearchInput onSearch={onSearchMock} debounceMs={300} />);
// Type quickly
await user.type(screen.getByRole('textbox'), 'test');
// No calls yet (debouncing)
expect(onSearchMock).not.toHaveBeenCalled();
// Advance timers past debounce threshold
jest.advanceTimersByTime(300);
// Now it should be called once with final value
expect(onSearchMock).toHaveBeenCalledTimes(1);
expect(onSearchMock).toHaveBeenCalledWith('test');
});
});Playwright Async Patterns
// e2e/async-patterns.spec.ts
import { test, expect } from '@playwright/test';
test('waits for API response', async ({ page }) => {
// Wait for specific response
const responsePromise = page.waitForResponse('/api/data');
await page.click('button.load-data');
const response = await responsePromise;
expect(response.status()).toBe(200);
});
test('waits for navigation', async ({ page }) => {
await page.goto('/');
await Promise.all([
page.waitForURL('/dashboard'),
page.click('a.dashboard-link'),
]);
});
test('waits for network idle', async ({ page }) => {
await page.goto('/', { waitUntil: 'networkidle' });
});
test('retries assertion until pass', async ({ page }) => {
// Auto-retrying assertion
await expect(page.locator('.counter')).toHaveText('10', { timeout: 5000 });
});---
Snapshot Testing Guidelines
When to Use Snapshots
| Good Use Cases | Bad Use Cases |
|---|---|
| Static UI components | Dynamic content |
| Error messages | Timestamps/IDs |
| Configuration objects | Large component trees |
| Serializable data | Interactive components |
Component Snapshots
// __tests__/components/Button.test.tsx
import { render } from '@testing-library/react';
import { Button } from '../../src/components/Button';
describe('Button snapshots', () => {
it('renders primary variant', () => {
const { container } = render(
<Button variant="primary">Click me</Button>
);
expect(container.firstChild).toMatchSnapshot();
});
it('renders secondary variant', () => {
const { container } = render(
<Button variant="secondary">Click me</Button>
);
expect(container.firstChild).toMatchSnapshot();
});
it('renders disabled state', () => {
const { container } = render(
<Button disabled>Click me</Button>
);
expect(container.firstChild).toMatchSnapshot();
});
});Inline Snapshots
// Good for small, stable outputs
it('formats date correctly', () => {
const result = formatDate(new Date('2024-01-15'));
expect(result).toMatchInlineSnapshot(`"January 15, 2024"`);
});
it('generates expected error message', () => {
const error = new ValidationError('email', 'Invalid format');
expect(error.message).toMatchInlineSnapshot(
`"Validation failed for 'email': Invalid format"`
);
});Snapshot Best Practices
1. Keep snapshots small - Snapshot specific elements, not entire pages 2. Use inline snapshots for small outputs - Easier to review in code 3. Review snapshot changes carefully - Don't blindly update 4. Avoid snapshots for dynamic content - Filter out timestamps, IDs 5. Combine with other assertions - Snapshots complement, not replace
// Filtering dynamic content from snapshots
it('renders user card', () => {
const { container } = render(<UserCard user={mockUser} />);
// Remove dynamic elements before snapshot
const card = container.firstChild;
const timestamp = card.querySelector('.timestamp');
timestamp?.remove();
expect(card).toMatchSnapshot();
});---
Summary
1. Use Page Objects for complex, reusable page interactions 2. Build factories for consistent test data creation 3. Leverage MSW for realistic API mocking 4. Create custom render utilities for provider wrapping 5. Master async patterns to avoid flaky tests 6. Use snapshots wisely for stable, static content only
Testing Strategies for React and Next.js Applications
Comprehensive guide to test architecture, coverage targets, and CI/CD integration patterns.
---
Table of Contents
- The Testing Pyramid
- Testing Types Deep Dive
- Coverage Targets and Thresholds
- Test Organization Patterns
- CI/CD Integration Strategies
- Testing Decision Framework
---
The Testing Pyramid
The testing pyramid guides how to distribute testing effort across different test types for optimal ROI.
Classic Pyramid Structure
/\
/ \ E2E Tests (5-10%)
/----\ - User journey validation
/ \ - Critical path coverage
/--------\ Integration Tests (20-30%)
/ \ - Component interactions
/ \ - API integration
/--------------\ Unit Tests (60-70%)
/ \ - Individual functions
------------------ - Isolated componentsReact/Next.js Adapted Pyramid
For frontend applications, the pyramid shifts slightly:
| Level | Percentage | Tools | Focus |
|---|---|---|---|
| Unit | 50-60% | Jest, RTL | Pure functions, hooks, isolated components |
| Integration | 25-35% | RTL, MSW | Component trees, API calls, context |
| E2E | 10-15% | Playwright | Critical user flows, cross-page navigation |
Why This Distribution?
Unit tests are fast and cheap:
- Execute in milliseconds
- Pinpoint failures precisely
- Easy to maintain
- Run on every commit
Integration tests balance coverage and cost:
- Test realistic scenarios
- Catch component interaction bugs
- Moderate execution time
- Run on every PR
E2E tests are expensive but essential:
- Validate real user experience
- Catch deployment issues
- Slow and brittle
- Run on staging/production
---
Testing Types Deep Dive
Unit Testing
Purpose: Verify individual units of code work correctly in isolation.
What to Unit Test:
- Pure utility functions
- Custom hooks (with renderHook)
- Individual component rendering
- State reducers
- Validation logic
- Data transformers
Example: Testing a Pure Function
// utils/formatPrice.ts
export function formatPrice(cents: number, currency = 'USD'): string {
const formatter = new Intl.NumberFormat('en-US', {
style: 'currency',
currency,
});
return formatter.format(cents / 100);
}
// utils/formatPrice.test.ts
describe('formatPrice', () => {
it('formats cents to USD by default', () => {
expect(formatPrice(1999)).toBe('$19.99');
});
it('handles zero', () => {
expect(formatPrice(0)).toBe('$0.00');
});
it('supports different currencies', () => {
expect(formatPrice(1999, 'EUR')).toContain('€');
});
it('handles large numbers', () => {
expect(formatPrice(100000000)).toBe('$1,000,000.00');
});
});Example: Testing a Custom Hook
// hooks/useCounter.ts
export function useCounter(initial = 0) {
const [count, setCount] = useState(initial);
const increment = () => setCount(c => c + 1);
const decrement = () => setCount(c => c - 1);
const reset = () => setCount(initial);
return { count, increment, decrement, reset };
}
// hooks/useCounter.test.ts
import { renderHook, act } from '@testing-library/react';
import { useCounter } from './useCounter';
describe('useCounter', () => {
it('starts with initial value', () => {
const { result } = renderHook(() => useCounter(5));
expect(result.current.count).toBe(5);
});
it('increments count', () => {
const { result } = renderHook(() => useCounter(0));
act(() => result.current.increment());
expect(result.current.count).toBe(1);
});
it('decrements count', () => {
const { result } = renderHook(() => useCounter(5));
act(() => result.current.decrement());
expect(result.current.count).toBe(4);
});
it('resets to initial value', () => {
const { result } = renderHook(() => useCounter(10));
act(() => result.current.increment());
act(() => result.current.reset());
expect(result.current.count).toBe(10);
});
});Integration Testing
Purpose: Verify multiple units work together correctly.
What to Integration Test:
- Component trees with multiple children
- Components with context providers
- Form submission flows
- API call and response handling
- State management interactions
- Router-dependent components
Example: Testing Component with API Call
// components/UserProfile.tsx
export function UserProfile({ userId }: { userId: string }) {
const [user, setUser] = useState<User | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
useEffect(() => {
fetch(`/api/users/${userId}`)
.then(res => res.json())
.then(data => setUser(data))
.catch(err => setError(err.message))
.finally(() => setLoading(false));
}, [userId]);
if (loading) return <div>Loading...</div>;
if (error) return <div>Error: {error}</div>;
return <div>{user?.name}</div>;
}
// components/UserProfile.test.tsx
import { render, screen, waitFor } from '@testing-library/react';
import { rest } from 'msw';
import { setupServer } from 'msw/node';
import { UserProfile } from './UserProfile';
const server = setupServer(
rest.get('/api/users/:id', (req, res, ctx) => {
return res(ctx.json({ id: req.params.id, name: 'John Doe' }));
})
);
beforeAll(() => server.listen());
afterEach(() => server.resetHandlers());
afterAll(() => server.close());
describe('UserProfile', () => {
it('shows loading state initially', () => {
render(<UserProfile userId="123" />);
expect(screen.getByText('Loading...')).toBeInTheDocument();
});
it('displays user name after loading', async () => {
render(<UserProfile userId="123" />);
await waitFor(() => {
expect(screen.getByText('John Doe')).toBeInTheDocument();
});
});
it('displays error on API failure', async () => {
server.use(
rest.get('/api/users/:id', (req, res, ctx) => {
return res(ctx.status(500));
})
);
render(<UserProfile userId="123" />);
await waitFor(() => {
expect(screen.getByText(/Error/)).toBeInTheDocument();
});
});
});End-to-End Testing
Purpose: Verify complete user flows work in a real browser environment.
What to E2E Test:
- Critical business flows (checkout, signup, login)
- Cross-page navigation sequences
- Authentication flows
- Third-party integrations
- Payment processing
- Form wizards
Example: Testing Checkout Flow
// e2e/checkout.spec.ts
import { test, expect } from '@playwright/test';
test.describe('Checkout Flow', () => {
test.beforeEach(async ({ page }) => {
await page.goto('/');
});
test('completes purchase successfully', async ({ page }) => {
// Add product to cart
await page.goto('/products/widget-pro');
await page.getByRole('button', { name: 'Add to Cart' }).click();
// Verify cart updated
await expect(page.getByTestId('cart-count')).toHaveText('1');
// Go to checkout
await page.getByRole('link', { name: 'Checkout' }).click();
// Fill shipping info
await page.getByLabel('Email').fill('test@example.com');
await page.getByLabel('Address').fill('123 Test St');
await page.getByLabel('City').fill('Test City');
await page.getByLabel('Zip').fill('12345');
// Fill payment info (test card)
await page.getByLabel('Card Number').fill('4242424242424242');
await page.getByLabel('Expiry').fill('12/25');
await page.getByLabel('CVC').fill('123');
// Submit order
await page.getByRole('button', { name: 'Place Order' }).click();
// Verify confirmation
await expect(page).toHaveURL(/\/orders\/\w+/);
await expect(page.getByText('Order Confirmed')).toBeVisible();
});
test('shows validation errors for invalid input', async ({ page }) => {
await page.goto('/checkout');
await page.getByRole('button', { name: 'Place Order' }).click();
await expect(page.getByText('Email is required')).toBeVisible();
await expect(page.getByText('Address is required')).toBeVisible();
});
});Visual Regression Testing
Purpose: Catch unintended visual changes to UI components.
Tools: Playwright visual comparisons, Percy, Chromatic
Example: Visual Snapshot Test
// e2e/visual/components.spec.ts
import { test, expect } from '@playwright/test';
test.describe('Visual Regression', () => {
test('button variants render correctly', async ({ page }) => {
await page.goto('/storybook/button');
await expect(page).toHaveScreenshot('button-variants.png');
});
test('responsive header', async ({ page }) => {
// Desktop
await page.setViewportSize({ width: 1280, height: 720 });
await page.goto('/');
await expect(page.locator('header')).toHaveScreenshot('header-desktop.png');
// Mobile
await page.setViewportSize({ width: 375, height: 667 });
await expect(page.locator('header')).toHaveScreenshot('header-mobile.png');
});
});Accessibility Testing
Purpose: Ensure application is usable by people with disabilities.
Tools: jest-axe, @axe-core/playwright
Example: Automated A11y Testing
// Unit/Integration level with jest-axe
import { render } from '@testing-library/react';
import { axe, toHaveNoViolations } from 'jest-axe';
import { Button } from './Button';
expect.extend(toHaveNoViolations);
describe('Button accessibility', () => {
it('has no accessibility violations', async () => {
const { container } = render(<Button>Click me</Button>);
const results = await axe(container);
expect(results).toHaveNoViolations();
});
});
// E2E level with Playwright + Axe
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test('homepage has no a11y violations', async ({ page }) => {
await page.goto('/');
const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
});---
Coverage Targets and Thresholds
Recommended Thresholds by Project Type
| Project Type | Statements | Branches | Functions | Lines |
|---|---|---|---|---|
| Startup/MVP | 60% | 50% | 60% | 60% |
| Growing Product | 75% | 70% | 75% | 75% |
| Enterprise | 85% | 80% | 85% | 85% |
| Safety Critical | 95% | 90% | 95% | 95% |
Coverage by Code Type
High Coverage Priority (80%+):
- Business logic
- State management
- API handlers
- Form validation
- Authentication/authorization
- Payment processing
Medium Coverage Priority (60-80%):
- UI components
- Utility functions
- Data transformers
- Custom hooks
Lower Coverage Priority (40-60%):
- Static pages
- Simple wrappers
- Configuration files
- Types/interfaces
Jest Coverage Configuration
// jest.config.js
module.exports = {
collectCoverageFrom: [
'src/**/*.{ts,tsx}',
'!src/**/*.d.ts',
'!src/**/*.stories.{ts,tsx}',
'!src/**/index.{ts,tsx}', // barrel files
'!src/types/**',
],
coverageThreshold: {
global: {
statements: 80,
branches: 75,
functions: 80,
lines: 80,
},
// Higher thresholds for critical paths
'./src/services/payment/': {
statements: 95,
branches: 90,
functions: 95,
lines: 95,
},
'./src/services/auth/': {
statements: 90,
branches: 85,
functions: 90,
lines: 90,
},
},
coverageReporters: ['text', 'lcov', 'html', 'json'],
};---
Test Organization Patterns
Co-located Tests (Recommended for React)
src/
├── components/
│ ├── Button/
│ │ ├── Button.tsx
│ │ ├── Button.test.tsx # Unit tests
│ │ ├── Button.stories.tsx # Storybook
│ │ └── index.ts
│ └── Form/
│ ├── Form.tsx
│ ├── Form.test.tsx
│ └── Form.integration.test.tsx # Integration tests
├── hooks/
│ ├── useAuth.ts
│ └── useAuth.test.ts
└── utils/
├── formatters.ts
└── formatters.test.tsSeparate Test Directory
src/
├── components/
├── hooks/
└── utils/
__tests__/
├── unit/
│ ├── components/
│ ├── hooks/
│ └── utils/
├── integration/
│ └── flows/
└── fixtures/
├── users.json
└── products.json
e2e/
├── specs/
│ ├── auth.spec.ts
│ └── checkout.spec.ts
├── fixtures/
│ └── auth.ts
└── pages/ # Page Object Models
├── LoginPage.ts
└── CheckoutPage.tsTest File Naming Conventions
| Pattern | Use Case |
|---|---|
*.test.ts | Unit tests |
*.spec.ts | Integration/E2E tests |
*.integration.test.ts | Explicit integration tests |
*.e2e.spec.ts | Explicit E2E tests |
*.a11y.test.ts | Accessibility tests |
*.visual.spec.ts | Visual regression tests |
---
CI/CD Integration Strategies
Pipeline Stages
# .github/workflows/test.yml
name: Test Pipeline
on:
push:
branches: [main, dev]
pull_request:
branches: [main, dev]
jobs:
unit:
name: Unit Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npm run test:unit -- --coverage
- uses: codecov/codecov-action@v4
with:
files: coverage/lcov.info
fail_ci_if_error: true
integration:
name: Integration Tests
runs-on: ubuntu-latest
needs: unit
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npm run test:integration
e2e:
name: E2E Tests
runs-on: ubuntu-latest
needs: integration
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npx playwright install --with-deps
- run: npm run build
- run: npm run test:e2e
- uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-report
path: playwright-report/Test Splitting for Speed
# Run E2E tests in parallel across multiple machines
e2e:
strategy:
matrix:
shard: [1, 2, 3, 4]
steps:
- run: npx playwright test --shard=${{ matrix.shard }}/4PR Gating Rules
| Test Type | When to Run | Block Merge? |
|---|---|---|
| Unit | Every commit | Yes |
| Integration | Every PR | Yes |
| E2E (smoke) | Every PR | Yes |
| E2E (full) | Merge to main | No (alert only) |
| Visual | Every PR | No (review required) |
| Performance | Weekly/Release | No (alert only) |
---
Testing Decision Framework
When to Write Which Test
Is it a pure function with no side effects?
├── Yes → Unit test
└── No
├── Does it make API calls or use context?
│ ├── Yes → Integration test with mocking
│ └── No
│ ├── Is it a critical user flow?
│ │ ├── Yes → E2E test
│ │ └── No → Integration test
└── Is it UI-focused with many visual states?
├── Yes → Storybook + Visual test
└── No → Component unit testTest ROI Matrix
| Test Type | Write Time | Run Time | Maintenance | Confidence |
|---|---|---|---|---|
| Unit | Low | Very Fast | Low | Medium |
| Integration | Medium | Fast | Medium | High |
| E2E | High | Slow | High | Very High |
| Visual | Low | Medium | Medium | High (UI) |
When NOT to Test
- Generated code (GraphQL types, Prisma client)
- Third-party library internals
- Implementation details (internal state, private methods)
- Simple pass-through wrappers
- Type definitions
Red Flags in Testing Strategy
| Red Flag | Problem | Solution |
|---|---|---|
| E2E tests > 30% | Slow CI, flaky tests | Push logic down to integration |
| Only unit tests | Missing interaction bugs | Add integration tests |
| Testing mocks | Not testing real behavior | Test behavior, not implementation |
| 100% coverage goal | Diminishing returns | Focus on critical paths |
| No E2E tests | Missing deployment issues | Add smoke tests for critical flows |
---
Summary
1. Follow the pyramid: 60% unit, 30% integration, 10% E2E 2. Set thresholds by risk: Higher coverage for critical paths 3. Co-locate tests: Keep tests close to source code 4. Automate in CI: Run tests on every PR, gate merges on failure 5. Decide wisely: Not everything needs every type of test
Tool CLI Reference
Read this when running test_suite_generator.py, coverage_analyzer.py, or e2e_test_scaffolder.py — full flag tables, usage examples, output formats, and generated artifacts.
1. test_suite_generator.py
Purpose: Scans React/TypeScript source directories for components and generates Jest + React Testing Library test stubs with render tests, prop tests, interaction tests, state tests, and optional accessibility tests.
Usage:
python scripts/test_suite_generator.py <source> [options]Flags:
| Flag | Short | Type | Default | Description |
|---|---|---|---|---|
source | -- | positional | (required) | Source directory containing React components |
--output | -o | string | <source>/__tests__/ | Output directory for generated test files |
--include-a11y | -- | flag | false | Include accessibility tests using jest-axe |
--scan-only | -- | flag | false | Scan and report components without generating test files |
--template | -- | string | None | Path to a custom template file for test generation |
--verbose | -v | flag | false | Enable verbose output showing each detected component |
--json | -- | flag | false | Output results as JSON after generation |
Example:
python scripts/test_suite_generator.py src/components/ \
--output __tests__/ \
--include-a11y \
--verbose
# Scanning: src/components/
# Found 24 React components
# Button.test.tsx (4 test cases)
# Modal.test.tsx (3 test cases)
# ...
# Summary: 24 test files, 87 test casesOutput Formats:
- Default (text): Human-readable summary printed to stdout listing each generated file and test case count.
- JSON (`--json`): Structured object with
status,components(array of detected component metadata),generated_files(array of output paths), andsummary(totals).
---
2. coverage_analyzer.py
Purpose: Parses Jest/Istanbul coverage reports (JSON or LCOV format), identifies coverage gaps by severity, flags critical business-logic paths, and generates actionable recommendations in text, HTML, or JSON format.
Usage:
python scripts/coverage_analyzer.py <coverage> [options]Flags:
| Flag | Short | Type | Default | Description |
|---|---|---|---|---|
coverage | -- | positional | (required) | Path to coverage file (.json, .info) or directory containing coverage data |
--threshold | -t | int | 80 | Coverage threshold percentage for pass/fail determination |
--strict | -- | flag | false | Exit with code 1 if overall coverage is below threshold |
--critical-paths | -- | flag | false | Focus analysis on critical business paths (auth, payment, security) |
--format | -f | choice | text | Output format: text, html, or json |
--output | -o | string | None | Write report to file instead of stdout |
--verbose | -v | flag | false | Enable verbose output with detailed parsing information |
--json | -- | flag | false | Output summary results as JSON (independent of --format) |
Example:
python scripts/coverage_analyzer.py coverage/coverage-final.json \
--threshold 80 \
--strict \
--format html \
--output report.html
# Analyzing coverage from: coverage/coverage-final.json
# Found coverage data for 42 files
# Report written to: report.htmlOutput Formats:
- Text (`--format text`): Structured report with overall percentages, threshold pass/fail, critical gaps, files below threshold, and prioritized recommendations.
- HTML (`--format html`): Styled HTML report with color-coded coverage stats, gap severity table, and per-file breakdown. Suitable for CI artifact upload.
- JSON (`--json`): Summary object with
status(pass/fail),threshold,coverage(statement/branch/function/line percentages),files_analyzed,files_below_threshold,total_gaps, andcritical_gaps.
---
3. e2e_test_scaffolder.py
Purpose: Scans Next.js App Router or Pages Router directories, detects routes (including dynamic segments, route groups, and authenticated pages), and generates Playwright test files with navigation, form, auth, and interaction test stubs. Optionally generates Page Object Model classes and Playwright configuration.
Usage:
python scripts/e2e_test_scaffolder.py <source> [options]Flags:
| Flag | Short | Type | Default | Description |
|---|---|---|---|---|
source | -- | positional | (required) | Source directory (src/app/ for App Router or pages/ for Pages Router) |
--output | -o | string | e2e/ | Output directory for generated test and fixture files |
--include-pom | -- | flag | false | Generate Page Object Model classes in <output>/pages/ |
--routes | -- | string | None | Comma-separated list of routes to generate (e.g., "/login,/dashboard") |
--verbose | -v | flag | false | Enable verbose output showing each detected route |
--json | -- | flag | false | Output results as JSON |
Example:
python scripts/e2e_test_scaffolder.py src/app/ \
--output e2e/ \
--include-pom \
--routes "/login,/dashboard,/checkout"
# Scanning: src/app/
# Found 3 routes
# auth-login.spec.ts
# pages/AuthLoginPage.ts
# dashboard.spec.ts
# pages/DashboardPage.ts
# checkout.spec.ts
# pages/CheckoutPage.ts
# fixtures/auth.ts
#
# Summary: 3 routes, 8 files generatedOutput Formats:
- Default (text): Human-readable list of generated files and a route/file count summary.
- JSON (`--json`): Structured object with
status,routes(array of route metadata including path, type, params, form/auth detection),generated_files(array with type, route, and path), andsummary(totals and configuration flags).
Generated Artifacts:
<route>.spec.ts-- Playwright test file per route with contextual test cases.pages/<RouteName>Page.ts-- Page Object Model class (when--include-pomis set).playwright.config.ts-- Multi-browser configuration with dev server integration (generated once if not already present).fixtures/auth.ts-- Authentication fixture with UI and API login patterns (generated once if not already present).
#!/usr/bin/env python3
"""
Coverage Analyzer
Parses Jest/Istanbul coverage reports and identifies gaps, uncovered branches,
and provides actionable recommendations for improving test coverage.
Usage:
python coverage_analyzer.py coverage/coverage-final.json --threshold 80
python coverage_analyzer.py coverage/ --format html --output report.html
python coverage_analyzer.py coverage/ --critical-paths
"""
import os
import sys
import json
import argparse
import re
from pathlib import Path
from typing import Dict, List, Optional, Tuple, Any
from dataclasses import dataclass, field, asdict
from datetime import datetime
from collections import defaultdict
@dataclass
class FileCoverage:
"""Coverage data for a single file"""
path: str
statements: Tuple[int, int] # (covered, total)
branches: Tuple[int, int]
functions: Tuple[int, int]
lines: Tuple[int, int]
uncovered_lines: List[int] = field(default_factory=list)
uncovered_branches: List[str] = field(default_factory=list)
@property
def statement_pct(self) -> float:
return (self.statements[0] / self.statements[1] * 100) if self.statements[1] > 0 else 100
@property
def branch_pct(self) -> float:
return (self.branches[0] / self.branches[1] * 100) if self.branches[1] > 0 else 100
@property
def function_pct(self) -> float:
return (self.functions[0] / self.functions[1] * 100) if self.functions[1] > 0 else 100
@property
def line_pct(self) -> float:
return (self.lines[0] / self.lines[1] * 100) if self.lines[1] > 0 else 100
@dataclass
class CoverageGap:
"""An identified coverage gap"""
file: str
gap_type: str # 'statements', 'branches', 'functions', 'lines'
lines: List[int]
severity: str # 'critical', 'high', 'medium', 'low'
description: str
recommendation: str
@dataclass
class CoverageSummary:
"""Overall coverage summary"""
statements: Tuple[int, int]
branches: Tuple[int, int]
functions: Tuple[int, int]
lines: Tuple[int, int]
files_analyzed: int
files_below_threshold: int = 0
class CoverageParser:
"""Parses various coverage report formats"""
def __init__(self, verbose: bool = False):
self.verbose = verbose
def parse(self, path: Path) -> Tuple[Dict[str, FileCoverage], CoverageSummary]:
"""Parse coverage data from file or directory"""
if path.is_file():
if path.suffix == '.json':
return self._parse_istanbul_json(path)
elif path.suffix == '.info' or 'lcov' in path.name:
return self._parse_lcov(path)
elif path.is_dir():
# Look for common coverage files
for filename in ['coverage-final.json', 'coverage-summary.json', 'lcov.info']:
candidate = path / filename
if candidate.exists():
return self.parse(candidate)
# Check for coverage-final.json in coverage directory
coverage_json = path / 'coverage-final.json'
if coverage_json.exists():
return self._parse_istanbul_json(coverage_json)
raise ValueError(f"Could not find or parse coverage data at: {path}")
def _parse_istanbul_json(self, path: Path) -> Tuple[Dict[str, FileCoverage], CoverageSummary]:
"""Parse Istanbul/Jest JSON coverage format"""
with open(path, 'r') as f:
data = json.load(f)
files = {}
total_statements = [0, 0]
total_branches = [0, 0]
total_functions = [0, 0]
total_lines = [0, 0]
for file_path, file_data in data.items():
# Skip node_modules
if 'node_modules' in file_path:
continue
# Parse statement coverage
s_map = file_data.get('statementMap', {})
s_hits = file_data.get('s', {})
covered_statements = sum(1 for h in s_hits.values() if h > 0)
total_statements[0] += covered_statements
total_statements[1] += len(s_map)
# Parse branch coverage
b_map = file_data.get('branchMap', {})
b_hits = file_data.get('b', {})
covered_branches = sum(
sum(1 for h in hits if h > 0)
for hits in b_hits.values()
)
total_branch_count = sum(len(b['locations']) for b in b_map.values())
total_branches[0] += covered_branches
total_branches[1] += total_branch_count
# Parse function coverage
fn_map = file_data.get('fnMap', {})
fn_hits = file_data.get('f', {})
covered_functions = sum(1 for h in fn_hits.values() if h > 0)
total_functions[0] += covered_functions
total_functions[1] += len(fn_map)
# Determine uncovered lines
uncovered_lines = []
for stmt_id, hits in s_hits.items():
if hits == 0 and stmt_id in s_map:
stmt = s_map[stmt_id]
start_line = stmt.get('start', {}).get('line', 0)
if start_line not in uncovered_lines:
uncovered_lines.append(start_line)
# Count lines
line_coverage = self._calculate_line_coverage(s_map, s_hits)
total_lines[0] += line_coverage[0]
total_lines[1] += line_coverage[1]
# Identify uncovered branches
uncovered_branches = []
for branch_id, hits in b_hits.items():
for idx, hit in enumerate(hits):
if hit == 0:
uncovered_branches.append(f"{branch_id}:{idx}")
files[file_path] = FileCoverage(
path=file_path,
statements=(covered_statements, len(s_map)),
branches=(covered_branches, total_branch_count),
functions=(covered_functions, len(fn_map)),
lines=line_coverage,
uncovered_lines=sorted(uncovered_lines)[:50], # Limit
uncovered_branches=uncovered_branches[:20]
)
summary = CoverageSummary(
statements=tuple(total_statements),
branches=tuple(total_branches),
functions=tuple(total_functions),
lines=tuple(total_lines),
files_analyzed=len(files)
)
return files, summary
def _calculate_line_coverage(self, s_map: Dict, s_hits: Dict) -> Tuple[int, int]:
"""Calculate line coverage from statement data"""
lines = set()
covered_lines = set()
for stmt_id, stmt in s_map.items():
start_line = stmt.get('start', {}).get('line', 0)
end_line = stmt.get('end', {}).get('line', start_line)
for line in range(start_line, end_line + 1):
lines.add(line)
if s_hits.get(stmt_id, 0) > 0:
covered_lines.add(line)
return (len(covered_lines), len(lines))
def _parse_lcov(self, path: Path) -> Tuple[Dict[str, FileCoverage], CoverageSummary]:
"""Parse LCOV format coverage data"""
with open(path, 'r') as f:
content = f.read()
files = {}
current_file = None
current_data = {}
total = {
'statements': [0, 0],
'branches': [0, 0],
'functions': [0, 0],
'lines': [0, 0]
}
for line in content.split('\n'):
line = line.strip()
if line.startswith('SF:'):
current_file = line[3:]
current_data = {
'lines_hit': 0, 'lines_total': 0,
'functions_hit': 0, 'functions_total': 0,
'branches_hit': 0, 'branches_total': 0,
'uncovered_lines': []
}
elif line.startswith('DA:'):
parts = line[3:].split(',')
if len(parts) >= 2:
line_num = int(parts[0])
hits = int(parts[1])
current_data['lines_total'] += 1
if hits > 0:
current_data['lines_hit'] += 1
else:
current_data['uncovered_lines'].append(line_num)
elif line.startswith('FN:'):
current_data['functions_total'] += 1
elif line.startswith('FNDA:'):
parts = line[5:].split(',')
if len(parts) >= 1 and int(parts[0]) > 0:
current_data['functions_hit'] += 1
elif line.startswith('BRDA:'):
parts = line[5:].split(',')
current_data['branches_total'] += 1
if len(parts) >= 4 and parts[3] != '-' and int(parts[3]) > 0:
current_data['branches_hit'] += 1
elif line == 'end_of_record' and current_file:
# Skip node_modules
if 'node_modules' not in current_file:
files[current_file] = FileCoverage(
path=current_file,
statements=(current_data['lines_hit'], current_data['lines_total']),
branches=(current_data['branches_hit'], current_data['branches_total']),
functions=(current_data['functions_hit'], current_data['functions_total']),
lines=(current_data['lines_hit'], current_data['lines_total']),
uncovered_lines=current_data['uncovered_lines'][:50]
)
for key in total:
if key == 'statements' or key == 'lines':
total[key][0] += current_data['lines_hit']
total[key][1] += current_data['lines_total']
elif key == 'branches':
total[key][0] += current_data['branches_hit']
total[key][1] += current_data['branches_total']
elif key == 'functions':
total[key][0] += current_data['functions_hit']
total[key][1] += current_data['functions_total']
current_file = None
summary = CoverageSummary(
statements=tuple(total['statements']),
branches=tuple(total['branches']),
functions=tuple(total['functions']),
lines=tuple(total['lines']),
files_analyzed=len(files)
)
return files, summary
class CoverageAnalyzer:
"""Analyzes coverage data and generates recommendations"""
CRITICAL_PATTERNS = [
r'auth', r'payment', r'security', r'login', r'register',
r'checkout', r'order', r'transaction', r'billing'
]
SERVICE_PATTERNS = [
r'service', r'api', r'handler', r'controller', r'middleware'
]
def __init__(
self,
threshold: int = 80,
critical_paths: bool = False,
verbose: bool = False
):
self.threshold = threshold
self.critical_paths = critical_paths
self.verbose = verbose
def analyze(
self,
files: Dict[str, FileCoverage],
summary: CoverageSummary
) -> Tuple[List[CoverageGap], Dict[str, Any]]:
"""Analyze coverage and return gaps and recommendations"""
gaps = []
recommendations = {
'critical': [],
'high': [],
'medium': [],
'low': []
}
# Analyze each file
for file_path, coverage in files.items():
file_gaps = self._analyze_file(file_path, coverage)
gaps.extend(file_gaps)
# Sort gaps by severity
severity_order = {'critical': 0, 'high': 1, 'medium': 2, 'low': 3}
gaps.sort(key=lambda g: (severity_order[g.severity], -len(g.lines)))
# Generate recommendations
for gap in gaps:
recommendations[gap.severity].append({
'file': gap.file,
'type': gap.gap_type,
'lines': gap.lines[:10], # Limit
'description': gap.description,
'recommendation': gap.recommendation
})
# Add summary stats
stats = {
'overall_statement_pct': (summary.statements[0] / summary.statements[1] * 100) if summary.statements[1] > 0 else 100,
'overall_branch_pct': (summary.branches[0] / summary.branches[1] * 100) if summary.branches[1] > 0 else 100,
'overall_function_pct': (summary.functions[0] / summary.functions[1] * 100) if summary.functions[1] > 0 else 100,
'overall_line_pct': (summary.lines[0] / summary.lines[1] * 100) if summary.lines[1] > 0 else 100,
'files_analyzed': summary.files_analyzed,
'files_below_threshold': sum(
1 for f in files.values()
if f.line_pct < self.threshold
),
'total_gaps': len(gaps),
'critical_gaps': len(recommendations['critical']),
'threshold': self.threshold,
'meets_threshold': (summary.lines[0] / summary.lines[1] * 100) >= self.threshold if summary.lines[1] > 0 else True
}
return gaps, {
'recommendations': recommendations,
'stats': stats
}
def _analyze_file(self, file_path: str, coverage: FileCoverage) -> List[CoverageGap]:
"""Analyze a single file for coverage gaps"""
gaps = []
# Determine if file is critical
is_critical = any(
re.search(pattern, file_path.lower())
for pattern in self.CRITICAL_PATTERNS
)
is_service = any(
re.search(pattern, file_path.lower())
for pattern in self.SERVICE_PATTERNS
)
# Determine severity based on file type and coverage level
if is_critical:
base_severity = 'critical'
target_threshold = 95
elif is_service:
base_severity = 'high'
target_threshold = 85
else:
base_severity = 'medium'
target_threshold = self.threshold
# Check line coverage
if coverage.line_pct < target_threshold:
severity = base_severity if coverage.line_pct < 50 else self._lower_severity(base_severity)
gaps.append(CoverageGap(
file=file_path,
gap_type='lines',
lines=coverage.uncovered_lines[:20],
severity=severity,
description=f"Line coverage at {coverage.line_pct:.1f}% (target: {target_threshold}%)",
recommendation=self._get_line_recommendation(coverage)
))
# Check branch coverage
if coverage.branch_pct < target_threshold - 5: # Allow 5% less for branches
severity = base_severity if coverage.branch_pct < 40 else self._lower_severity(base_severity)
gaps.append(CoverageGap(
file=file_path,
gap_type='branches',
lines=[],
severity=severity,
description=f"Branch coverage at {coverage.branch_pct:.1f}%",
recommendation=f"Add tests for conditional logic. {len(coverage.uncovered_branches)} uncovered branches."
))
# Check function coverage
if coverage.function_pct < target_threshold:
severity = self._lower_severity(base_severity)
gaps.append(CoverageGap(
file=file_path,
gap_type='functions',
lines=[],
severity=severity,
description=f"Function coverage at {coverage.function_pct:.1f}%",
recommendation="Add tests for uncovered functions/methods."
))
return gaps
def _lower_severity(self, severity: str) -> str:
"""Lower severity by one level"""
mapping = {
'critical': 'high',
'high': 'medium',
'medium': 'low',
'low': 'low'
}
return mapping[severity]
def _get_line_recommendation(self, coverage: FileCoverage) -> str:
"""Generate recommendation for line coverage gaps"""
if coverage.line_pct < 30:
return "This file has very low coverage. Consider adding basic render/unit tests first."
elif coverage.line_pct < 60:
return "Add tests covering the main functionality and happy paths."
else:
return "Focus on edge cases and error handling paths."
class ReportGenerator:
"""Generates coverage reports in various formats"""
def __init__(self, verbose: bool = False):
self.verbose = verbose
def generate_text_report(
self,
files: Dict[str, FileCoverage],
summary: CoverageSummary,
analysis: Dict[str, Any],
threshold: int
) -> str:
"""Generate a text report"""
lines = []
# Header
lines.append("=" * 60)
lines.append("COVERAGE ANALYSIS REPORT")
lines.append(f"Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
lines.append("=" * 60)
lines.append("")
# Overall summary
stats = analysis['stats']
lines.append("OVERALL COVERAGE:")
lines.append(f" Statements: {stats['overall_statement_pct']:.1f}%")
lines.append(f" Branches: {stats['overall_branch_pct']:.1f}%")
lines.append(f" Functions: {stats['overall_function_pct']:.1f}%")
lines.append(f" Lines: {stats['overall_line_pct']:.1f}%")
lines.append("")
# Threshold check
threshold_status = "PASS" if stats['meets_threshold'] else "FAIL"
lines.append(f"Threshold ({threshold}%): {threshold_status}")
lines.append(f"Files analyzed: {stats['files_analyzed']}")
lines.append(f"Files below threshold: {stats['files_below_threshold']}")
lines.append("")
# Critical gaps
recs = analysis['recommendations']
if recs['critical']:
lines.append("-" * 60)
lines.append("CRITICAL GAPS (requires immediate attention):")
for rec in recs['critical'][:5]:
lines.append(f" - {rec['file']}")
lines.append(f" {rec['description']}")
if rec['lines']:
lines.append(f" Uncovered lines: {', '.join(map(str, rec['lines'][:5]))}")
lines.append("")
# High priority gaps
if recs['high']:
lines.append("-" * 60)
lines.append("HIGH PRIORITY GAPS:")
for rec in recs['high'][:5]:
lines.append(f" - {rec['file']}")
lines.append(f" {rec['description']}")
lines.append("")
# Files below threshold
below_threshold = [
(path, cov) for path, cov in files.items()
if cov.line_pct < threshold
]
below_threshold.sort(key=lambda x: x[1].line_pct)
if below_threshold:
lines.append("-" * 60)
lines.append(f"FILES BELOW {threshold}% THRESHOLD:")
for path, cov in below_threshold[:10]:
short_path = path.split('/')[-1] if '/' in path else path
lines.append(f" {cov.line_pct:5.1f}% {short_path}")
if len(below_threshold) > 10:
lines.append(f" ... and {len(below_threshold) - 10} more files")
lines.append("")
# Recommendations
lines.append("-" * 60)
lines.append("RECOMMENDATIONS:")
all_recs = (
recs['critical'][:2] + recs['high'][:2] + recs['medium'][:2]
)
for i, rec in enumerate(all_recs[:5], 1):
lines.append(f" {i}. {rec['recommendation']}")
lines.append(f" File: {rec['file']}")
lines.append("")
lines.append("=" * 60)
return '\n'.join(lines)
def generate_html_report(
self,
files: Dict[str, FileCoverage],
summary: CoverageSummary,
analysis: Dict[str, Any],
threshold: int
) -> str:
"""Generate an HTML report"""
stats = analysis['stats']
recs = analysis['recommendations']
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Coverage Analysis Report</title>
<style>
body {{ font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; margin: 40px; }}
h1 {{ color: #333; }}
.summary {{ display: grid; grid-template-columns: repeat(4, 1fr); gap: 20px; margin: 20px 0; }}
.stat {{ background: #f5f5f5; padding: 20px; border-radius: 8px; text-align: center; }}
.stat-value {{ font-size: 2em; font-weight: bold; }}
.pass {{ color: #22c55e; }}
.fail {{ color: #ef4444; }}
.warn {{ color: #f59e0b; }}
table {{ width: 100%; border-collapse: collapse; margin: 20px 0; }}
th, td {{ padding: 12px; text-align: left; border-bottom: 1px solid #ddd; }}
th {{ background: #f5f5f5; }}
.gap-critical {{ background: #fef2f2; }}
.gap-high {{ background: #fffbeb; }}
.progress {{ background: #e5e7eb; border-radius: 4px; height: 8px; }}
.progress-bar {{ height: 100%; border-radius: 4px; }}
</style>
</head>
<body>
<h1>Coverage Analysis Report</h1>
<p>Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}</p>
<div class="summary">
<div class="stat">
<div class="stat-value {'pass' if stats['overall_statement_pct'] >= threshold else 'fail'}">{stats['overall_statement_pct']:.1f}%</div>
<div>Statements</div>
</div>
<div class="stat">
<div class="stat-value {'pass' if stats['overall_branch_pct'] >= threshold - 5 else 'fail'}">{stats['overall_branch_pct']:.1f}%</div>
<div>Branches</div>
</div>
<div class="stat">
<div class="stat-value {'pass' if stats['overall_function_pct'] >= threshold else 'fail'}">{stats['overall_function_pct']:.1f}%</div>
<div>Functions</div>
</div>
<div class="stat">
<div class="stat-value {'pass' if stats['overall_line_pct'] >= threshold else 'fail'}">{stats['overall_line_pct']:.1f}%</div>
<div>Lines</div>
</div>
</div>
<h2>Threshold Status: <span class="{'pass' if stats['meets_threshold'] else 'fail'}">{'PASS' if stats['meets_threshold'] else 'FAIL'}</span></h2>
<p>Target: {threshold}% | Files Analyzed: {stats['files_analyzed']} | Below Threshold: {stats['files_below_threshold']}</p>
<h2>Coverage Gaps</h2>
<table>
<thead>
<tr>
<th>Severity</th>
<th>File</th>
<th>Issue</th>
<th>Recommendation</th>
</tr>
</thead>
<tbody>
"""
# Add gaps to table
all_gaps = (
[(g, 'critical') for g in recs['critical']] +
[(g, 'high') for g in recs['high']] +
[(g, 'medium') for g in recs['medium'][:5]]
)
for gap, severity in all_gaps[:15]:
row_class = f"gap-{severity}" if severity in ['critical', 'high'] else ""
html += f""" <tr class="{row_class}">
<td>{severity.upper()}</td>
<td>{gap['file'].split('/')[-1]}</td>
<td>{gap['description']}</td>
<td>{gap['recommendation']}</td>
</tr>
"""
html += """ </tbody>
</table>
<h2>File Coverage Details</h2>
<table>
<thead>
<tr>
<th>File</th>
<th>Statements</th>
<th>Branches</th>
<th>Functions</th>
<th>Lines</th>
</tr>
</thead>
<tbody>
"""
# Sort files by line coverage
sorted_files = sorted(files.items(), key=lambda x: x[1].line_pct)
for path, cov in sorted_files[:20]:
short_path = path.split('/')[-1] if '/' in path else path
html += f""" <tr>
<td>{short_path}</td>
<td>{cov.statement_pct:.1f}%</td>
<td>{cov.branch_pct:.1f}%</td>
<td>{cov.function_pct:.1f}%</td>
<td>{cov.line_pct:.1f}%</td>
</tr>
"""
html += """ </tbody>
</table>
</body>
</html>
"""
return html
class CoverageAnalyzerTool:
"""Main tool class"""
def __init__(
self,
coverage_path: str,
threshold: int = 80,
critical_paths: bool = False,
strict: bool = False,
output_format: str = 'text',
output_path: Optional[str] = None,
verbose: bool = False
):
self.coverage_path = Path(coverage_path)
self.threshold = threshold
self.critical_paths = critical_paths
self.strict = strict
self.output_format = output_format
self.output_path = output_path
self.verbose = verbose
def run(self) -> Dict[str, Any]:
"""Run the coverage analysis"""
print(f"Analyzing coverage from: {self.coverage_path}")
# Parse coverage data
parser = CoverageParser(self.verbose)
files, summary = parser.parse(self.coverage_path)
print(f"Found coverage data for {len(files)} files")
# Analyze coverage
analyzer = CoverageAnalyzer(
threshold=self.threshold,
critical_paths=self.critical_paths,
verbose=self.verbose
)
gaps, analysis = analyzer.analyze(files, summary)
# Generate report
reporter = ReportGenerator(self.verbose)
if self.output_format == 'html':
report = reporter.generate_html_report(files, summary, analysis, self.threshold)
else:
report = reporter.generate_text_report(files, summary, analysis, self.threshold)
# Output report
if self.output_path:
with open(self.output_path, 'w') as f:
f.write(report)
print(f"Report written to: {self.output_path}")
else:
print(report)
# Return results
results = {
'status': 'pass' if analysis['stats']['meets_threshold'] else 'fail',
'threshold': self.threshold,
'coverage': {
'statements': analysis['stats']['overall_statement_pct'],
'branches': analysis['stats']['overall_branch_pct'],
'functions': analysis['stats']['overall_function_pct'],
'lines': analysis['stats']['overall_line_pct']
},
'files_analyzed': summary.files_analyzed,
'files_below_threshold': analysis['stats']['files_below_threshold'],
'total_gaps': analysis['stats']['total_gaps'],
'critical_gaps': analysis['stats']['critical_gaps']
}
# Exit with error if strict mode and below threshold
if self.strict and not analysis['stats']['meets_threshold']:
print(f"\nFailed: Coverage {analysis['stats']['overall_line_pct']:.1f}% below threshold {self.threshold}%")
sys.exit(1)
return results
def main():
"""Main entry point"""
parser = argparse.ArgumentParser(
description="Analyze Jest/Istanbul coverage reports and identify gaps",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Basic analysis
python coverage_analyzer.py coverage/coverage-final.json
# With threshold enforcement
python coverage_analyzer.py coverage/ --threshold 80 --strict
# Generate HTML report
python coverage_analyzer.py coverage/ --format html --output report.html
# Focus on critical paths
python coverage_analyzer.py coverage/ --critical-paths
"""
)
parser.add_argument(
'coverage',
help='Path to coverage file or directory'
)
parser.add_argument(
'--threshold', '-t',
type=int,
default=80,
help='Coverage threshold percentage (default: 80)'
)
parser.add_argument(
'--strict',
action='store_true',
help='Exit with error if coverage is below threshold'
)
parser.add_argument(
'--critical-paths',
action='store_true',
help='Focus analysis on critical business paths'
)
parser.add_argument(
'--format', '-f',
choices=['text', 'html', 'json'],
default='text',
help='Output format (default: text)'
)
parser.add_argument(
'--output', '-o',
help='Output file path'
)
parser.add_argument(
'--verbose', '-v',
action='store_true',
help='Enable verbose output'
)
parser.add_argument(
'--json',
action='store_true',
help='Output results as JSON (summary only)'
)
args = parser.parse_args()
try:
tool = CoverageAnalyzerTool(
coverage_path=args.coverage,
threshold=args.threshold,
critical_paths=args.critical_paths,
strict=args.strict,
output_format=args.format,
output_path=args.output,
verbose=args.verbose
)
results = tool.run()
if args.json:
print(json.dumps(results, indent=2))
except Exception as e:
print(f"Error: {e}")
if args.verbose:
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == '__main__':
main()