Playwright Testing

Frontend Testing

Unlock reliable confidence fast: enable safe refactors by choosing the right test layer, making the app observable, and eliminating nondeterminism so failures are actionable.

Philosophy: Confidence Per Minute

Frontend tests fail for two reasons: the product is broken, or the test is lying. Your job is to maximize signal and minimize "test is lying".

Before writing a test, ask:

• What user risk am I covering (money, progression, auth, data loss, crashes)?
• What's the narrowest layer that catches this bug class (pure logic vs UI vs full browser)?
• What nondeterminism exists (time, RNG, async loading, network, animations, fonts, GPU)?
• What "ready" signal can I wait on besides setTimeout?
• What should a failure print/screenshot so it's diagnosable in CI?

Core principles: 1. Test the contract, not the implementation: assert stable user-meaningful outcomes and public seams. 2. Prefer determinism over retries: make time/RNG/network controllable; remove flake at the source. 3. Observe like a debugger: console errors, network failures, screenshots, and state dumps on failure. 4. One critical flow first: a reliable smoke test beats 50 flaky tests.

Test Layer Decision Tree

Pick the cheapest layer that provides needed confidence:

Quick Start: First Smoke Test

1. Define 1 critical flow: "page loads → user can start → one key action works" 2. Add a test seam to the app (see below) 3. Choose runner: Playwright MCP for E2E, unit tests for logic 4. Fail loudly: treat console errors and failed requests as test failures 5. Stabilize: seed RNG, freeze time, fix viewport, disable animations

Concrete MCP Workflow: Testing a Game

Step-by-step sequence for testing a Phaser/canvas game:

1. mcp__playwright__browser_navigate
   → http://localhost:3000?test=1&seed=42

2. mcp__playwright__browser_evaluate
   → () => new Promise(r => { const c = () => window.__TEST__?.ready ? r(true) : setTimeout(c, 100); c(); })
   (Wait for game ready)

3. mcp__playwright__browser_console_messages
   → level: "error"
   (Fail if any errors)

4. mcp__playwright__browser_snapshot
   → Get UI state and refs

5. mcp__playwright__browser_click
   → element: "Start Button", ref: [from snapshot]

6. mcp__playwright__browser_evaluate
   → () => window.__TEST__.state()
   (Assert game state is correct)

7. mcp__playwright__browser_press_key
   → key: "ArrowRight" (or WASD for movement)

8. mcp__playwright__browser_evaluate
   → () => window.__TEST__.state().player.x
   (Verify movement happened)

9. mcp__playwright__browser_take_screenshot
   → filename: "gameplay-state.png"
   (Visual evidence after deterministic setup)

Recommended Test Seams

Add to the app for testability (read-only, stable, minimal):

window.__TEST__ = {
  ready: false,           // true after first interactive frame
  seed: null,             // current RNG seed
  sceneKey: null,         // current scene/route
  state: () => ({         // JSON-serializable snapshot
    scene: this.sceneKey,
    player: { x, y, hp },
    score: gameState.score,
    entities: entities.map(e => ({ id: e.id, type: e.type, x: e.x, y: e.y }))
  }),
  commands: {             // optional mutation commands
    reset: () => {},
    seed: (n) => {},
    skipIntro: () => {}
  }
};

Rule: Expose IDs + essential fields, not raw Phaser/engine objects.

Anti-Patterns to Avoid

❌ Testing the wrong layer: E2E tests for pure logic Why tempting: "Let's just test everything through the browser" Better: Unit tests for logic; reserve E2E for integration contracts

❌ Testing implementation details: Asserting DOM structure/classnames Why tempting: Easy to assert what you can see in DevTools Better: Assert user-meaningful outputs (text, score, HP changes)

❌ Sleep-driven tests: wait 2s then click Why tempting: Simple and "works on my machine" Better: Wait on explicit readiness (DOM marker, window.__TEST__.ready)

❌ Uncontrolled randomness: RNG/time in assertions Why tempting: "The game uses random, so the test should too" Better: Seed RNG (?seed=42), freeze time, assert stable invariants

❌ Pixel snapshots without determinism: Canvas screenshots that flake Why tempting: "I'll catch visual bugs automatically" Better: Deterministic mode first; then screenshot at known stable frames

❌ Retries as a strategy: "Just bump retries to 3" Why tempting: Quick fix that makes CI green Better: Fix the flake source; retries hide real problems

Debugging Failed Tests

When a test fails, gather evidence in this order:

1. Console errors: mcp__playwright__browser_console_messages({ level: "error" }) 2. Network failures: mcp__playwright__browser_network_requests() → check for non-2xx 3. Screenshot: mcp__playwright__browser_take_screenshot() → visual state at failure 4. App state: mcp__playwright__browser_evaluate({ function: "() => window.__TEST__.state()" }) 5. Classify the flake (see references/flake-reduction.md):

• Readiness? → add explicit wait
• Timing? → control animation/physics
• Environment? → lock viewport/DPR
• Data? → isolate test data

Graduation Criteria: When Is Testing "Enough"?

Minimum viable test suite:

• [ ] 1 smoke test that proves the app loads and primary action works
• [ ] Test seam exists (window.__TEST__ with ready flag and state)
• [ ] Deterministic mode for canvas/games (?test=1 enables seeding)
• [ ] Console errors fail tests (no silent failures)
• [ ] CI runs tests on every push

Level up when:

• Critical paths (auth, payment, save/load) have dedicated E2E
• Unit tests cover complex logic (pathfinding, damage calc, state machines)
• Visual regression on key screens (menu, HUD) with locked determinism

Visual Regression with imgdiff.py

For pixel comparison of screenshots:

# Compare baseline to current
python scripts/imgdiff.py baseline.png current.png --out diff.png

# Allow small tolerance (anti-aliasing differences)
python scripts/imgdiff.py baseline.png current.png --max-rms 2.0

Exit codes: 0 = identical, 1 = different, 2 = error

UI Slicing Regressions (Nine-Slice / Ribbons / Bars)

Canvas UI issues (panel seams, segmented ribbons, invisible HUD fills) are best caught with a dedicated UI harness instead of the full gameplay flow.

1. Build a simple test.html/scene that loads only the UI assets. 2. Render raw slices next to assembled panels (multi-size), and include ribbon/bars with both “raw crop + scale” and “stitched multi-slice” views. 3. Expose window.__TEST__ with .commands.showTest(n) so Playwright can toggle each mode deterministically. 4. Capture targeted screenshots (panels, ribbons, bars) and diff them in CI.

See references/phaser-canvas-testing.md for the deterministic setup + screenshot workflow.

Variation Guidance

Adapt approach based on context:

• DOM app: Standard Playwright selectors, wait for text/elements
• Canvas game: Test seams mandatory, wait via window.__TEST__.ready
• Hybrid: DOM for menus, test seams for gameplay
• CI-only GPU: May need software rendering flags or skip visual tests
• UI slicing regressions: For nine-slice/ribbon/bar artifacts, prefer a small UI harness scene/page with deterministic modes and targeted screenshots (references/phaser-canvas-testing.md).

Bundled Resources

Read these when needed:

• references/playwright-mcp-cheatsheet.md: Detailed MCP tool patterns
• references/phaser-canvas-testing.md: Deterministic mode for Phaser games
• references/flake-reduction.md: Flake classification and fixes

Remember

You can make almost any frontend (including canvas/WebGL games) testable by adding a tiny, stable seam for readiness + state. One reliable smoke test is the foundation. Aim for tests that are boring to maintain: deterministic, explicit about readiness, and rich in failure evidence. The goal is confidence, not coverage numbers.

Flake Reduction (Frontend)

First: Classify the Flake

Before fixing, identify the category:

Triage Workflow

1. Reproduce locally with CI-identical flags:

   # Match CI environment
   HEADLESS=true VIEWPORT=1280x720 npm test

2. Capture evidence on failure:

• Console messages (via browser_console_messages)
• Network requests with status codes
• Screenshot at failure moment
• State dump from window.__TEST__.state()

3. Check for patterns:

• Always fails on first run? → Readiness flake
• Fails only in parallel? → Concurrency flake
• Fails on specific machine? → Environment flake

Fix Patterns (In Order of Leverage)

1. Readiness Fixes (Most Common)

Problem: Test acts before app is ready.

Solutions:

Add explicit ready signal:

// In app
window.__TEST__ = { ready: false };
// After initialization complete
window.__TEST__.ready = true;

Wait on ready signal:

// In test via browser_evaluate
() => window.__TEST__?.ready === true

Avoid waitForTimeout—prefer waitForFunction:

// Bad
await page.waitForTimeout(2000);

// Good
await page.waitForFunction(() => window.__TEST__.ready);

2. Determinism Fixes

Problem: RNG/time/animation causes different outcomes.

Solutions:

Seed RNG:

// Deterministic random
function seededRandom(seed) {
  return function() {
    seed = (seed * 9301 + 49297) % 233280;
    return seed / 233280;
  };
}
Math.random = seededRandom(12345);

Control time:

// Fixed timestep for game loop
const FIXED_DT = 1000 / 60;
function gameLoop() {
  update(FIXED_DT); // Always same dt
  render();
  requestAnimationFrame(gameLoop);
}

Disable animations in test mode:

if (window.__TEST__) {
  CSS.supports('animation', 'none') &&
    document.body.classList.add('no-animations');
}

3. Isolation Fixes

Problem: Tests affect each other.

Solutions:

Reset storage between tests:

// Before each test
localStorage.clear();
sessionStorage.clear();
indexedDB.deleteDatabase('myapp');

Use unique test data:

// Bad: shared account
const user = { email: 'test@example.com' };

// Good: unique per test
const user = { email: `test-${Date.now()}@example.com` };

Avoid order-dependent tests:

// Each test should work standalone
// Don't rely on previous test creating data

4. Environment Fixes

Problem: Different machines produce different results.

Solutions:

Lock viewport and DPR:

// In test setup
await page.setViewportSize({ width: 1280, height: 720 });
await page.emulateMedia({ colorScheme: 'light' });

Lock locale/timezone:

// Via browser context
const context = await browser.newContext({
  locale: 'en-US',
  timezoneId: 'America/New_York'
});

Handle font differences:

// Either: use system fonts only
// Or: preload and wait for web fonts
await document.fonts.ready;

5. Temporary Guardrails

Use retries ONLY as temporary measure:

// Tag flaky tests for tracking
test.describe('flaky-wip', () => {
  test.retry(2); // Temporary until fixed

  test('intermittent test', async () => {
    // TODO: Fix readiness issue in #123
  });
});

Track flaky tests:

• Create ticket for each flaky test
• Set deadline for fix
• Remove retry once fixed

Flake Diagnosis Checklist

When a test fails:

• [ ] What exact error message?
• [ ] Does it reproduce locally with same config?
• [ ] What was the app state at failure? (screenshot/state dump)
• [ ] Were there console errors?
• [ ] Were there failed network requests?
• [ ] Did other tests run before this one?
• [ ] Is there shared state being modified?
• [ ] Does it pass with increased timeouts? (readiness issue)
• [ ] Does it pass when run alone? (concurrency issue)

Red Flags in Test Code

// Red flag: magic sleep
await page.waitForTimeout(3000);
// Fix: wait for specific condition

// Red flag: retry loop
for (let i = 0; i < 3; i++) {
  try { await test(); break; } catch {}
}
// Fix: make test deterministic

// Red flag: order-dependent
test('B depends on A', () => { /* uses data from test A */ });
// Fix: set up own data

// Red flag: time-sensitive assertion
expect(performance.now() - start).toBeLessThan(100);
// Fix: mock time or use range

// Red flag: DOM structure assertion
expect(wrapper.find('.btn-primary-v2')).toExist();
// Fix: assert on text/role/behavior

Phaser / Canvas / WebGL Testing

Why Canvas/WebGL Tests Get Flaky

Common nondeterminism sources:

• Variable frame times (CPU load, headless rendering)
• Time-based movement/physics without fixed timestep
• RNG for loot/spawns/AI decisions
• Async asset loading and "first frame" races
• Font loading differences affecting mixed DOM+canvas layouts
• GPU/driver differences in rendering

The fix is not "more retries"—it's deterministic mode + explicit readiness.

Deterministic Mode Pattern

When ?test=1 query param (or build-time flag) is enabled:

// In game initialization
const isTestMode = new URLSearchParams(window.location.search).has('test');

if (isTestMode) {
  // 1. Seed RNG
  const seed = parseInt(params.get('seed')) || 12345;
  Math.random = seededRandom(seed);

  // 2. Fixed timestep
  game.loop.targetFps = 60;
  game.loop.forceSetTimeOut = true; // Consistent frame timing

  // 3. Disable visual noise
  disableCameraShake();
  disableParticles();
  disableScreenFlash();

  // 4. Ensure assets preloaded before interaction
  await preloadAllAssets();
}

Implementing the Test Seam

// Add to your game's boot or create phase
window.__TEST__ = {
  ready: false,
  seed: null,
  sceneKey: null,
  frameCount: 0,

  state: () => ({
    scene: game.scene.getScenes(true)[0]?.scene.key,
    player: getPlayerState(),
    enemies: getEnemyStates(),
    score: gameState.score,
    resources: gameState.resources
  }),

  commands: {
    reset: () => game.scene.start('MainMenu'),
    seed: (n) => { seedRNG(n); window.__TEST__.seed = n; },
    skipIntro: () => game.scene.start('Gameplay'),
    advanceFrame: () => game.loop.step(16.67)
  }
};

// Set ready after:
// 1. Preload completed
// 2. First scene created
// 3. First render tick occurred
game.events.on('ready', () => {
  window.__TEST__.ready = true;
});

What to Assert (Avoid Brittle Assertions)

Good assertions (match player-visible behavior):

• "Player can start" → scene key is correct, UI state is interactive
• "Attack damages enemy" → enemy HP decreased after attack action
• "Collecting coin increments score" → score increased by expected amount
• "Player dies at 0 HP" → death state triggered, game over UI shown

Brittle assertions to avoid:

• Exact pixel positions without fixed dt and RNG
• Internal array/map ordering
• Sprite instance properties directly
• Animation frame indices

Screenshot Testing: Making It Reliable

Before comparing screenshots:

1. Lock viewport + DPR:

   mcp__playwright__browser_resize({ width: 1280, height: 720 })

2. Set deterministic mode:

   Navigate to: http://localhost:3000?test=1&seed=42

3. Wait for stable frame:

   // Via browser_evaluate
   () => window.__TEST__.ready && window.__TEST__.frameCount >= 10

4. Target screenshots strategically:

• Menu screens (static, predictable)
• First gameplay frame after deterministic setup
• Specific game states (pause menu, game over)

NOT every frame or random gameplay moments.

UI Slicing Regressions (Nine-Slice / Ribbons / Bars)

For visual bugs in UI panels, ribbons, or HUD bars, stop relying on the full game flow—use a dedicated UI harness scene.

Harness Pattern

1. Load only the UI assets (papers, ribbons, bars) into test.html. 2. Present each element twice: raw frame/tile views and final assembled render at different sizes. 3. Add keyboard controls (1..N) plus window.__TEST__.commands.showTest(n) so Playwright can flip modes. 4. Capture targeted screenshots (panels, ribbons, bars) deterministically; diff them in CI with scripts/imgdiff.py.

This makes nine-slice/trimming issues and segmented ribbons easy to spot without the noise of gameplay.

See the UI harness instructions in the main skill and docs/postmortem-ui-panel-rendering.md for reference.

Phaser-Specific Patterns

Fixed Timestep for Physics

const config = {
  physics: {
    default: 'arcade',
    arcade: {
      // Fixed timestep for deterministic physics
      fps: 60,
      timeScale: 1
    }
  }
};

Seeding Phaser's RNG

// Phaser has built-in seeded random
const rnd = new Phaser.Math.RandomDataGenerator([seed.toString()]);
// Use rnd.frac(), rnd.between(), etc. instead of Math.random()

Waiting for Asset Load

// In preload scene
this.load.on('complete', () => {
  window.__TEST__.assetsLoaded = true;
});

Exposing Entity State Safely

// Don't expose: this.player (Sprite instance)
// Do expose:
window.__TEST__.state = () => ({
  player: {
    x: Math.round(this.player.x),
    y: Math.round(this.player.y),
    hp: this.player.getData('hp'),
    state: this.player.getData('state')
  }
});

Test Workflow for Phaser Games

1. Navigate with ?test=1&seed=<number> 2. Wait for window.__TEST__.ready === true 3. Drive input via browser_press_key for WASD/arrows, browser_click for UI 4. Wait for game state change (poll window.__TEST__.state()) 5. Assert state matches expected outcome 6. Screenshot at known deterministic points only

Playwright MCP Cheatsheet

Patterns for using Playwright MCP tools during frontend testing tasks (especially canvas/WebGL games).

Mental Model

• Use MCP to reproduce a user flow and collect evidence: console, network, screenshots, and state
• Prefer explicit readiness over time-based waits
• Treat any console error (or failed asset request) as a product failure unless explicitly allowed

Tool Patterns by Task

Navigate + Wait for App Readiness

For DOM apps:

1. mcp__playwright__browser_navigate({ url: "http://localhost:3000" })
2. mcp__playwright__browser_wait_for({ text: "Welcome" })

For canvas/game apps:

1. mcp__playwright__browser_navigate({ url: "http://localhost:3000?test=1" })
2. mcp__playwright__browser_evaluate({
     function: "() => new Promise(resolve => { const check = () => window.__TEST__?.ready ? resolve(true) : setTimeout(check, 100); check(); })"
   })

Assert State (White-Box via Test Seams)

Read app state through exposed test API:

mcp__playwright__browser_evaluate({
  function: "() => window.__TEST__.state()"
})

Common assertions:

• Scene/route: window.__TEST__.sceneKey === "MainMenu"
• Score/resources: window.__TEST__.state().score >= 100
• Entity state: window.__TEST__.state().player.hp > 0

Drive User Input

Click interactions:

mcp__playwright__browser_click({ element: "Start Button", ref: "[ref-from-snapshot]" })

Keyboard input (games):

mcp__playwright__browser_press_key({ key: "ArrowRight" })
mcp__playwright__browser_press_key({ key: "Space" })  // attack/jump

Drag operations:

mcp__playwright__browser_drag({
  startElement: "Tower icon", startRef: "[ref]",
  endElement: "Map tile", endRef: "[ref]"
})

Text input:

mcp__playwright__browser_type({
  element: "Player name field",
  ref: "[ref]",
  text: "TestPlayer"
})

Catch Silent Failures

Check for console errors:

mcp__playwright__browser_console_messages({ level: "error" })
// Fail test if any errors returned

Check for failed network requests:

mcp__playwright__browser_network_requests()
// Fail if any required asset returned non-2xx/3xx

Visual Evidence

Take screenshot (after determinism enforced):

mcp__playwright__browser_take_screenshot({
  filename: "game-main-menu.png",
  type: "png"
})

Element screenshot:

mcp__playwright__browser_take_screenshot({
  element: "Game canvas",
  ref: "[ref]",
  filename: "canvas-state.png"
})

Workflow: Complete Test Sequence

1. Navigate to app URL (with ?test=1 for deterministic mode) 2. Wait for readiness signal 3. Check console for pre-existing errors 4. Drive user input sequence 5. Assert state via test seams 6. Screenshot if visual verification needed 7. Check console/network for errors introduced by actions

Test Seam Recommendations

Minimal, stable, read-only seams to add to the app:

window.__TEST__ = {
  ready: false,           // Set true after first interactive frame
  version: "1.0.0",       // For cache invalidation
  seed: null,             // Current RNG seed (if seeded)
  sceneKey: null,         // Current scene/route
  state: () => ({         // Returns JSON-serializable snapshot
    scene: this.sceneKey,
    player: { x, y, hp, state },
    score: currentScore,
    entities: [...entityList.map(e => ({ id, type, x, y }))]
  }),
  commands: {             // Optional mutation commands
    reset: () => {},      // Reset to initial state
    seed: (n) => {},      // Set RNG seed
    skipIntro: () => {},  // Jump past animations
    setTime: (t) => {}    // Control game clock
  }
};

Key principle: Expose IDs + essential fields, not raw engine objects.

Common Gotchas

1. Race on navigate: Always wait for readiness after navigation, never assume immediate availability 2. Stale refs: Snapshot refs become invalid after navigation or major DOM changes—re-snapshot 3. Animation timing: Screenshots during animations will be inconsistent—wait for animation completion or disable animations 4. Canvas click coordinates: For canvas, clicking via MCP clicks DOM position—ensure canvas fills expected area