Fix

Fix Failing or Flaky Tests

Diagnose and fix a Playwright test that fails or passes intermittently using a systematic taxonomy.

Input

$ARGUMENTS contains:

• A test file path: e2e/login.spec.ts
• A test name: ""should redirect after login"`
• A description: "the checkout test fails in CI but passes locally"

Steps

1. Reproduce the Failure

Run the test to capture the error:

npx playwright test <file> --reporter=list

If the test passes, it's likely flaky. Run burn-in:

npx playwright test <file> --repeat-each=10 --reporter=list

If it still passes, try with parallel workers:

npx playwright test --fully-parallel --workers=4 --repeat-each=5

2. Capture Trace

Run with full tracing:

npx playwright test <file> --trace=on --retries=0

Read the trace output. Use /debug to analyze trace files if available.

3. Categorize the Failure

Load flaky-taxonomy.md from this skill directory.

Every failing test falls into one of four categories:

4. Apply Targeted Fix

Timing/Async:

• Replace waitForTimeout() with web-first assertions
• Add await to missing Playwright calls
• Wait for specific network responses before asserting
• Use toBeVisible() before interacting with elements

Test Isolation:

• Remove shared mutable state between tests
• Create test data per-test via API or fixtures
• Use unique identifiers (timestamps, random strings) for test data
• Check for database state leaks

Environment:

• Match viewport sizes between local and CI
• Account for font rendering differences in screenshots
• Use docker locally to match CI environment
• Check for timezone-dependent assertions

Infrastructure:

• Increase timeout for slow CI runners
• Add retries in CI config (retries: 2)
• Check for browser OOM (reduce parallel workers)
• Ensure browser dependencies are installed

5. Verify the Fix

Run the test 10 times to confirm stability:

npx playwright test <file> --repeat-each=10 --reporter=list

All 10 must pass. If any fail, go back to step 3.

6. Prevent Recurrence

Suggest:

• Add to CI with retries: 2 if not already
• Enable trace: 'on-first-retry' in config
• Add the fix pattern to project's test conventions doc

Output

• Root cause category and specific issue
• The fix applied (with diff)
• Verification result (10/10 passes)
• Prevention recommendation

Flaky Test Taxonomy

Decision Tree

Test is flaky
│
├── Fails locally with --repeat-each=20?
│   ├── YES → TIMING / ASYNC
│   │   ├── Missing await? → Add await
│   │   ├── waitForTimeout? → Replace with assertion
│   │   ├── Race condition? → Wait for specific event
│   │   └── Animation? → Wait for animation end or disable
│   │
│   └── NO → Continue...
│
├── Passes alone, fails in suite?
│   ├── YES → TEST ISOLATION
│   │   ├── Shared variable? → Make per-test
│   │   ├── Database state? → Reset per-test
│   │   ├── localStorage? → Clear in beforeEach
│   │   └── Cookie leak? → Use isolated contexts
│   │
│   └── NO → Continue...
│
├── Fails in CI, passes locally?
│   ├── YES → ENVIRONMENT
│   │   ├── Viewport? → Set explicit size
│   │   ├── Fonts? → Use Docker locally
│   │   ├── Timezone? → Use UTC everywhere
│   │   └── Network? → Mock external services
│   │
│   └── NO → INFRASTRUCTURE
│       ├── Browser crash? → Reduce workers
│       ├── OOM? → Limit parallel tests
│       ├── DNS? → Add retry config
│       └── File system? → Use unique temp dirs

Common Fixes by Category

Timing / Async

Missing await:

// BAD — race condition
page.goto('/dashboard');
expect(page.getByText('Welcome')).toBeVisible();

// GOOD
await page.goto('/dashboard');
await expect(page.getByText('Welcome')).toBeVisible();

Clicking before visible:

// BAD — element may not be ready
await page.getByRole('button', { name: 'Submit' }).click();

// GOOD — ensure visible first
const submitBtn = page.getByRole('button', { name: 'Submit' });
await expect(submitBtn).toBeVisible();
await submitBtn.click();

Race with network:

// BAD — data might not be loaded
await page.goto('/users');
await expect(page.getByRole('table')).toBeVisible();

// GOOD — wait for API response
const responsePromise = page.waitForResponse('**/api/users');
await page.goto('/users');
await responsePromise;
await expect(page.getByRole('table')).toBeVisible();

Test Isolation

Shared state fix:

// BAD — tests share userId
let userId: string;
test('create', async () => { userId = '123'; });
test('read', async () => { /* uses userId */ });

// GOOD — each test is independent
test('read user', async ({ request }) => {
  const response = await request.post('/api/users', { data: { name: 'Test' } });
  const { id } = await response.json();
  // Use id within this test
});

localStorage cleanup:

test.beforeEach(async ({ page }) => {
  await page.goto('/');
  await page.evaluate(() => localStorage.clear());
});

Environment

Explicit viewport:

test.use({ viewport: { width: 1280, height: 720 } });

Timezone-safe dates:

// BAD
expect(dateText).toBe('March 5, 2026');

// GOOD — timezone independent
expect(dateText).toMatch(/\d{1,2}\/\d{1,2}\/\d{4}/);

Infrastructure

Retry config:

// playwright.config.ts
export default defineConfig({
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 2 : undefined,
});

Increase timeout for CI:

test.setTimeout(60_000); // 60s for slow CI