
Playwright Skill
- 4.9k installs
- 343 repo stars
- Updated July 2, 2026
- testdino-hq/playwright-skill
playwright-skill is an agent skill for Battle-tested Playwright patterns for writing, debugging, and scaling reliable test suites. Use when you need guidance f
About
playwright-skill shares battle-tested Playwright patterns for writing, debugging, and scaling reliable end-to-end test suites. It covers locator strategies, auto-waiting, trace and video capture, parallel sharding, fixture design, network mocking, authentication storage state, CI retries, and flake triage workflows. Agents apply it when users need stable selectors, debugging failing specs, configuring playwright.config.ts, or structuring Page Object patterns without brittle timeouts. The skill emphasizes role and test-id locators, web-first assertions, and isolating tests from shared state. See SKILL.md for setup, examples, and guardrails before production use. See SKILL.md for setup, examples, and guardrails before production use. See SKILL.md for setup, examples, and guardrails before production use. See SKILL.md for setup, examples, and guardrails before production use. See SKILL.md for setup, examples, and guardrails before production use.
- Battle-tested Playwright patterns for writing, debugging, and scaling reliable test suites. Use when you need guidance f
- Organizing test suites
- API testing (REST/GraphQL)
- File uploads/downloads
- General debugging workflow
Playwright Skill by the numbers
- 4,906 all-time installs (skills.sh)
- +129 installs in the week ending Aug 5, 2026 (Skillselion tracking)
- Ranked #125 of 2,203 Security skills by installs in the Skillselion catalog
- Security screen: HIGH risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
playwright-skill capabilities & compatibility
- Capabilities
- battle tested playwright patterns for writing, d · organizing test suites · api testing (rest/graphql)
npx skills add https://github.com/testdino-hq/playwright-skill --skill playwright-skillAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 4.9k |
|---|---|
| repo stars | ★ 343 |
| Security audit | 3 / 3 scanners passed |
| Last updated | July 2, 2026 |
| Repository | testdino-hq/playwright-skill ↗ |
How do I run playwright-skill tasks with correct setup and documented commands?
Battle-tested Playwright patterns for writing, debugging, and scaling reliable test suites. Use when you need guidance for E2E, API, component, visual, accessibility, or security testing, plus CI/CD,
Who is it for?
Developers automating playwright skill via agent-guided SKILL.md workflows.
Skip if: Skip when unrelated tooling already covers the task without this skill's documented flow.
When should I use this skill?
Battle-tested Playwright patterns for writing, debugging, and scaling reliable test suites. Use when you need guidance for E2E, API, component, visual, accessibility, or security testing, plus CI/CD,
What you get
Repeatable playwright-skill workflows with grounded commands and expected outputs.
- Playwright spec patterns
- Page-object templates
- CI integration guidance
By the numbers
- 50+ reference guides in SKILL.md
- Version 2.3.0 in skill metadata
- Covers 6 test surfaces: E2E, API, component, visual, accessibility, security
Files
Playwright CI/CD
Ship reliable tests in every pipeline — CI-specific patterns for speed, stability, and actionable reports.
9 guides covering CI/CD setup, parallel execution, containerized runs, reporting, and infrastructure patterns for all major CI providers.
Golden Rules
1. `retries: 2` in CI only — surface flakiness in pipelines, not locally 2. `traces: 'on-first-retry'` — capture rich debugging artifacts without slowing every run 3. Shard across runners — --shard=N/M splits tests evenly; scale horizontally, not vertically 4. Cache browser binaries — ~/.cache/ms-playwright keyed on Playwright version 5. Upload artifacts on failure — traces, screenshots, and HTML reports as CI artifacts 6. Use the official Docker image — mcr.microsoft.com/playwright:v* has all OS deps pre-installed 7. Global setup for auth — run login once in globalSetup, reuse storageState across workers 8. Fail fast, debug later — keep CI runs short; use trace viewer and HTML reports to investigate
Guide Index
CI Providers
| Provider | Guide |
|---|---|
| GitHub Actions | ci-github-actions.md |
| GitLab CI | ci-gitlab.md |
| CircleCI / Azure DevOps / Jenkins | ci-other.md |
Execution & Scaling
| Topic | Guide |
|---|---|
| Parallel execution & sharding | parallel-and-sharding.md |
| Docker & containers | docker-and-containers.md |
| Multi-project config | projects-and-dependencies.md |
Reporting & Setup
| Topic | Guide |
|---|---|
| Reports & artifacts | reporting-and-artifacts.md |
| Code coverage | test-coverage.md |
| Global setup/teardown | global-setup-teardown.md |
CI: GitHub Actions
When to use: Running Playwright tests automatically on pull requests, merges to main, or on a schedule. GitHub Actions is the most common CI for Playwright projects.
Quick Reference
# Key CLI flags for CI
npx playwright install --with-deps # install browsers + OS deps
npx playwright test --shard=1/4 # run 1 of 4 shards
npx playwright test --reporter=github # annotate PR with failures
npx playwright merge-reports ./blob-report # merge shard reportsPatterns
Pattern 1: Production-Ready Workflow (Copy-Paste Starter)
Use when: Any project using GitHub Actions. This is the complete, battle-tested workflow.
# .github/workflows/playwright.yml
name: Playwright Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
# Cancel in-progress runs for the same PR/branch
concurrency:
group: playwright-${{ github.ref }}
cancel-in-progress: true
env:
CI: true
jobs:
test:
timeout-minutes: 30
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
- name: Run Playwright tests
run: npx playwright test
- name: Upload HTML report
uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 14
- name: Upload test traces
uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-traces
path: test-results/
retention-days: 7Pattern 2: Sharded Execution with Matrix Strategy
Use when: Test suite takes more than 10 minutes. Split across parallel runners to cut wall-clock time. Avoid when: Suite runs under 5 minutes -- sharding overhead (checkout, install, merge) negates the benefit.
# .github/workflows/playwright-sharded.yml
name: Playwright Tests (Sharded)
on:
push:
branches: [main]
pull_request:
branches: [main]
concurrency:
group: playwright-${{ github.ref }}
cancel-in-progress: true
env:
CI: true
jobs:
test:
timeout-minutes: 20
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1/4, 2/4, 3/4, 4/4]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
- name: Run Playwright tests (shard ${{ matrix.shard }})
run: npx playwright test --shard=${{ matrix.shard }}
- name: Upload blob report
uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: blob-report-${{ strategy.job-index }}
path: blob-report/
retention-days: 1
merge-reports:
if: ${{ !cancelled() }}
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Download all blob reports
uses: actions/download-artifact@v4
with:
path: all-blob-reports
pattern: blob-report-*
merge-multiple: true
- name: Merge reports
run: npx playwright merge-reports --reporter=html ./all-blob-reports
- name: Upload merged HTML report
uses: actions/upload-artifact@v4
with:
name: playwright-report
path: playwright-report/
retention-days: 14Config for sharding -- add blob reporter so shard output can be merged:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['github']]
: [['html', { open: 'on-failure' }]],
});Pattern 3: Reusable Workflow
Use when: Multiple repositories or multiple workflow files need the same Playwright setup. Avoid when: Single repo with one workflow.
# .github/workflows/playwright-reusable.yml
name: Playwright Reusable
on:
workflow_call:
inputs:
node-version:
type: string
default: '20'
test-command:
type: string
default: 'npx playwright test'
shard-total:
type: number
default: 1
secrets:
BASE_URL:
required: false
TEST_PASSWORD:
required: false
jobs:
test:
timeout-minutes: 30
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: ${{ fromJson(format('[{0}]', join(fromJson(format('[{0}]', inputs.shard-total == 1 && '"1/1"' || '"1/4","2/4","3/4","4/4"')), ','))) }}
env:
CI: true
BASE_URL: ${{ secrets.BASE_URL }}
TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
cache: 'npm'
- run: npm ci
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
- name: Run tests
run: ${{ inputs.test-command }} --shard=${{ matrix.shard }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report-${{ strategy.job-index }}
path: playwright-report/
retention-days: 14Calling the reusable workflow:
# .github/workflows/ci.yml
name: CI
on:
pull_request:
branches: [main]
jobs:
e2e:
uses: ./.github/workflows/playwright-reusable.yml
with:
node-version: '20'
shard-total: 4
secrets:
BASE_URL: ${{ secrets.STAGING_URL }}
TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}Pattern 4: Running in a Container
Use when: You need a reproducible environment identical to local Docker runs, or the runner's OS dependencies cause issues. Avoid when: Standard ubuntu-latest with --with-deps works fine (the common case).
# .github/workflows/playwright-container.yml
name: Playwright (Container)
on:
pull_request:
branches: [main]
jobs:
test:
timeout-minutes: 30
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.52.0-noble
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
# No browser install needed -- they're in the container image
- name: Run Playwright tests
run: npx playwright test
env:
HOME: /root # required when running as root in container
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 14Pattern 5: Environment Secrets and Deployment Targets
Use when: Tests run against staging/production environments that require authentication credentials. Avoid when: Tests only run against a locally started dev server.
# .github/workflows/playwright-staging.yml
name: Playwright (Staging)
on:
push:
branches: [main]
workflow_dispatch:
jobs:
test:
timeout-minutes: 30
runs-on: ubuntu-latest
environment: staging # GitHub Environment with protection rules
env:
CI: true
BASE_URL: ${{ vars.STAGING_URL }}
TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
API_KEY: ${{ secrets.API_KEY }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
- name: Run tests against staging
run: npx playwright test --grep @smoke
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: staging-report
path: playwright-report/
retention-days: 14Pattern 6: Scheduled Runs (Nightly Regression)
Use when: Full regression suite is too slow for every PR. Run it nightly against main. Avoid when: Suite runs in under 15 minutes and can run on every PR.
# .github/workflows/playwright-nightly.yml
name: Nightly Regression
on:
schedule:
- cron: '0 3 * * 1-5' # 3 AM UTC, Mon-Fri
workflow_dispatch: # allow manual trigger
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
env:
CI: true
BASE_URL: ${{ vars.STAGING_URL }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- name: Install Playwright browsers
run: npx playwright install --with-deps
- name: Run full regression suite
run: npx playwright test --grep @regression
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: nightly-report-${{ github.run_number }}
path: playwright-report/
retention-days: 30
- name: Notify on failure
if: failure()
uses: slackapi/slack-github-action@v1.27.0
with:
payload: |
{
"text": "Nightly Playwright regression failed: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}Decision Guide
| Scenario | Approach | Why |
|---|---|---|
| Small suite (< 5 min) | Single job, no sharding | Overhead of sharding exceeds time saved |
| Medium suite (5-20 min) | 2-4 shards with matrix | Cut wall-clock time by ~60-75% |
| Large suite (20+ min) | 4-8 shards + blob report merge | Keep PR feedback under 10 minutes |
| Cross-browser on PRs | Chromium only on PRs; all browsers on main | 3x fewer minutes burned on PRs |
| Staging/prod smoke tests | Separate workflow with environment: | Isolate secrets, add approval gates |
| Nightly full regression | schedule trigger + workflow_dispatch | Full coverage without blocking PRs |
| Multiple repos, same setup | Reusable workflow with workflow_call | DRY; update one file, all repos benefit |
| Reproducible env needed | Container job with Playwright image | Identical to local Docker environment |
Security: Pinning Actions to Commit SHAs
Version tags like actions/checkout@v4 are mutable — the tag can be moved to a different commit without warning, introducing unverified code into your CI pipeline (supply-chain risk W012).
Best practice: pin every action to its full commit SHA.
# Instead of:
- uses: actions/checkout@v4
# Pin to a specific immutable commit SHA:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2How to find the SHA for any action:
# Look up the SHA for a tagged release on GitHub:
# https://github.com/<owner>/<action>/releases
# Click the tag → copy the full commit SHA from the URL or commit details
# Or use the gh CLI:
gh api repos/actions/checkout/git/ref/tags/v4.2.2 --jq '.object.sha'Example pinned workflow step set (verify SHAs at release pages before use):
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
- uses: actions/cache@d4323d4df104b026a6aa633fdb11d772146be0bf # v4.2.2
- uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
- uses: actions/download-artifact@fa0a91b85d4f404e444306234b4f18a22b3d1e57 # v4.1.8Always verify these SHAs against the official release pages at github.com/actions before adding them to production workflows.
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
No concurrency group | Duplicate runs waste minutes on every push | Add concurrency: { group: ..., cancel-in-progress: true } |
fail-fast: true with sharding | One shard failure cancels others; you lose their results | Set fail-fast: false to collect all failures |
| Installing browsers without caching | 60-90 seconds wasted every run | Cache ~/.cache/ms-playwright keyed on lockfile hash |
timeout-minutes not set | Stuck jobs run for 6 hours (GitHub default) | Set explicit timeout: 20-30 minutes |
| Uploading artifacts only on failure | No report when tests pass; can't verify results | Use if: ${{ !cancelled() }} to always upload |
| Hardcoding secrets in workflow files | Security breach | Use GitHub Secrets and Environments |
| Running all browsers on every PR | 3x CI cost for marginal benefit | Chromium on PR; cross-browser on main merge |
actions/upload-artifact with no retention | Default 90-day retention fills storage | Set retention-days: 7-14 for reports |
No --with-deps on browser install | Missing OS libraries cause browser launch failures | Always use npx playwright install --with-deps |
Using mutable action version tags (@v4) | Tag can be silently re-pointed to a different commit (supply-chain risk) | Pin to full commit SHA; see Security section above |
Troubleshooting
Browser launch fails: "Missing dependencies"
Cause: Browsers installed from cache but OS dependencies were not cached (they live in system directories, not ~/.cache).
Fix: Always run npx playwright install-deps on cache hit:
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-depsTests pass locally but fail in CI with timeouts
Cause: CI runners have fewer CPU cores and less RAM than your dev machine. Default workers and timeouts are too aggressive.
Fix: Reduce workers and increase timeouts for CI in your config:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
workers: process.env.CI ? '50%' : undefined,
use: {
actionTimeout: process.env.CI ? 15_000 : 10_000,
navigationTimeout: process.env.CI ? 30_000 : 15_000,
},
});Sharded reports are incomplete -- some shards missing from merged report
Cause: Using actions/download-artifact@v4 without merge-multiple: true, or artifact names collide across shards.
Fix: Give each shard a unique artifact name and use merge-multiple:
# Upload in each shard job
- uses: actions/upload-artifact@v4
with:
name: blob-report-${{ strategy.job-index }}
path: blob-report/
# Download in merge job
- uses: actions/download-artifact@v4
with:
path: all-blob-reports
pattern: blob-report-*
merge-multiple: truewebServer fails in CI: "port 3000 already in use"
Cause: Previous run left a zombie process, or another job step is using the port.
Fix: Ensure reuseExistingServer: false in CI and add a pre-step to kill stale processes:
- name: Kill stale processes
run: lsof -ti:3000 | xargs kill -9 2>/dev/null || trueGitHub annotations not appearing on PR
Cause: The github reporter is not configured.
Fix: Add github reporter for CI runs:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['html', { open: 'never' }], ['github']]
: [['html', { open: 'on-failure' }]],
});Related
- ci/parallel-and-sharding.md -- sharding strategies and blob report merging
- ci/reporting-and-artifacts.md -- reporter configuration and artifact management
- ci/docker-and-containers.md -- container images for CI
- ci/ci-gitlab.md -- GitLab CI equivalent
- ci/ci-other.md -- CircleCI, Azure DevOps, Jenkins
- core/configuration.md -- CI-aware config settings
CI: GitLab CI/CD
When to use: Running Playwright tests in GitLab pipelines on merge requests, merges to main, or scheduled pipelines.
Quick Reference
# Key commands used in GitLab pipelines
npx playwright install --with-deps # install browsers + OS deps
npx playwright test --shard=1/4 # run 1 of 4 parallel shards
npx playwright merge-reports ./blob-report # merge shard results
npx playwright test --reporter=dot # minimal output for CI logsPatterns
Pattern 1: Production-Ready Pipeline (Copy-Paste Starter)
Use when: Any GitLab project with Playwright tests. This is the complete, recommended configuration.
# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.52.0-noble
stages:
- install
- test
- report
variables:
CI: "true"
npm_config_cache: "$CI_PROJECT_DIR/.npm"
# Cache node_modules and npm cache across pipelines
cache:
key:
files:
- package-lock.json
paths:
- .npm/
- node_modules/
install:
stage: install
script:
- npm ci
artifacts:
paths:
- node_modules/
expire_in: 1 hour
test:
stage: test
needs: [install]
script:
- npx playwright test
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHPattern 2: Parallel Sharded Execution
Use when: Test suite exceeds 10 minutes. Use GitLab's parallel keyword to split across jobs automatically. Avoid when: Suite runs under 5 minutes.
# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.52.0-noble
stages:
- install
- test
- report
variables:
CI: "true"
npm_config_cache: "$CI_PROJECT_DIR/.npm"
cache:
key:
files:
- package-lock.json
paths:
- .npm/
- node_modules/
install:
stage: install
script:
- npm ci
artifacts:
paths:
- node_modules/
expire_in: 1 hour
test:
stage: test
needs: [install]
parallel: 4
script:
- npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
artifacts:
when: always
paths:
- blob-report/
expire_in: 1 hour
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
merge-report:
stage: report
needs: [test]
when: always
script:
- npx playwright merge-reports --reporter=html ./blob-report
artifacts:
when: always
paths:
- playwright-report/
expire_in: 14 daysConfig for sharded pipelines:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['dot']]
: [['html', { open: 'on-failure' }]],
});Pattern 3: Merge Request Pipelines with Environment Variables
Use when: Tests need secrets (API keys, passwords) and should only run on merge requests or the default branch. Avoid when: Tests are fully self-contained with no external dependencies.
# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.52.0-noble
stages:
- test
variables:
CI: "true"
test:e2e:
stage: test
variables:
BASE_URL: $STAGING_URL
TEST_PASSWORD: $TEST_PASSWORD
API_KEY: $API_KEY
before_script:
- npm ci
script:
- npx playwright test
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
# Run on merge requests
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# Run on default branch pushes
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
# Allow manual trigger
- when: manual
allow_failure: trueSetting variables in GitLab: Navigate to Settings > CI/CD > Variables and add:
STAGING_URL-- not masked, not protectedTEST_PASSWORD-- masked, protectedAPI_KEY-- masked, protected
Pattern 4: Multi-Browser Testing with Child Pipelines
Use when: Running Chromium on MRs and all browsers on the default branch. Avoid when: You only test one browser.
# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.52.0-noble
stages:
- install
- test
variables:
CI: "true"
install:
stage: install
script:
- npm ci
artifacts:
paths:
- node_modules/
expire_in: 1 hour
# Chromium only on merge requests (fast feedback)
test:chromium:
stage: test
needs: [install]
script:
- npx playwright test --project=chromium
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# All browsers on default branch
test:all-browsers:
stage: test
needs: [install]
parallel:
matrix:
- PROJECT: [chromium, firefox, webkit]
script:
- npx playwright test --project=$PROJECT
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHPattern 5: Custom Docker Image with Application
Use when: Tests need the application running alongside Playwright, or you need custom system dependencies. Avoid when: The official Playwright image plus webServer in config handles your use case.
# .gitlab-ci.yml
stages:
- test
test:e2e:
stage: test
image: mcr.microsoft.com/playwright:v1.52.0-noble
services:
- name: postgres:16-alpine
alias: db
- name: redis:7-alpine
alias: cache
variables:
CI: "true"
DATABASE_URL: "postgresql://postgres:postgres@db:5432/test"
REDIS_URL: "redis://cache:6379"
POSTGRES_PASSWORD: "postgres"
POSTGRES_DB: "test"
before_script:
- npm ci
- npx prisma db push
- npx prisma db seed
script:
- npx playwright test
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHPattern 6: Scheduled Nightly Regression
Use when: Full regression is too slow for every MR. Run it on a schedule.
# .gitlab-ci.yml (add to existing config)
test:nightly:
stage: test
image: mcr.microsoft.com/playwright:v1.52.0-noble
before_script:
- npm ci
script:
- npx playwright test --grep @regression
artifacts:
when: always
paths:
- playwright-report/
expire_in: 30 days
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"Set up the schedule in CI/CD > Schedules: 0 3 * * 1-5 (3 AM UTC, weekdays).
Decision Guide
| Scenario | Approach | Why |
|---|---|---|
| Simple project, < 5 min suite | Single test job using Playwright Docker image | No sharding overhead; artifacts capture report |
| Suite > 10 min | parallel: N with --shard | GitLab auto-assigns CI_NODE_INDEX/CI_NODE_TOTAL |
| Merge request fast feedback | Chromium only on MRs; all browsers on main | 3x fewer pipeline minutes on MRs |
| External services needed (DB, Redis) | services: keyword with Postgres/Redis images | GitLab manages service lifecycle |
| Secrets for staging environment | GitLab CI/CD Variables (masked + protected) | Never hardcode secrets in .gitlab-ci.yml |
| Full nightly regression | Pipeline schedule (CI_PIPELINE_SOURCE == "schedule") | Avoids blocking MR pipelines |
| Report browsing | artifacts: with paths: [playwright-report/] | Browse directly in GitLab job artifacts UI |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
| Not using the Playwright Docker image | Installing browsers every run adds 1-2 minutes | Use mcr.microsoft.com/playwright:v1.52.0-noble as base image |
artifacts: when: on_failure only | No report when tests pass; can't verify results | Use when: always to capture reports regardless |
No expire_in on artifacts | Artifacts accumulate and consume storage | Set expire_in: 14 days for reports, 1 hour for intermediate artifacts |
parallel: without fail-fast: false equivalent | GitLab does not cancel siblings by default (good), but allow_failure: false means the pipeline fails fast | Acceptable default behavior; no change needed |
Hardcoding CI_NODE_TOTAL in shard flag | Breaks when you change parallel: value | Use --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL |
Skipping needs: between stages | Jobs wait for all previous stage jobs, not just their dependencies | Use needs: for precise dependency graphs |
Large cache: including node_modules/ without key | Stale cache causes version conflicts | Key cache on package-lock.json hash |
Troubleshooting
Browser launch fails: "Failed to launch browser"
Cause: Not using the Playwright Docker image, or using a version that doesn't match your @playwright/test version.
Fix: Match the Docker image tag to your Playwright version:
# Check your version
# npm ls @playwright/test -> @playwright/test@1.52.0
image: mcr.microsoft.com/playwright:v1.52.0-nobleTests hang in GitLab runner: "Navigation timeout exceeded"
Cause: GitLab shared runners may have limited resources. Default timeouts too tight.
Fix: Reduce workers and increase timeouts:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
workers: process.env.CI ? 2 : undefined,
use: {
navigationTimeout: process.env.CI ? 30_000 : 15_000,
},
});Pipeline runs on every push, not just merge requests
Cause: Missing rules: configuration. Default GitLab behavior runs on every push.
Fix: Add explicit rules:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHServices (Postgres/Redis) not reachable from tests
Cause: Using localhost instead of the service alias.
Fix: Use the service alias as hostname:
services:
- name: postgres:16-alpine
alias: db # <-- use "db" as hostname
variables:
DATABASE_URL: "postgresql://postgres:postgres@db:5432/test" # not localhostMerged report is empty after sharded run
Cause: Each shard job needs the blob reporter, not html. The merge step creates the HTML report.
Fix: Configure blob reporter for CI:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['dot']]
: [['html', { open: 'on-failure' }]],
});Related
- ci/ci-github-actions.md -- GitHub Actions equivalent
- ci/ci-other.md -- CircleCI, Azure DevOps, Jenkins
- ci/parallel-and-sharding.md -- sharding strategies
- ci/docker-and-containers.md -- Docker image details
- ci/reporting-and-artifacts.md -- reporter configuration
CI: CircleCI, Azure DevOps, and Jenkins
When to use: Running Playwright tests in CI platforms other than GitHub Actions or GitLab. Each section provides a production-ready config you can copy and adapt.
Quick Reference
# Common across all CI platforms
npx playwright install --with-deps # install browsers + OS deps
npx playwright test --shard=1/4 # shard for parallelism
npx playwright merge-reports ./blob-report # merge shard results
npx playwright test --reporter=dot,html # multiple reportersPatterns
Pattern 1: CircleCI
Use when: Your project runs on CircleCI.
Basic Pipeline
# .circleci/config.yml
version: 2.1
executors:
playwright:
docker:
- image: mcr.microsoft.com/playwright:v1.52.0-noble
working_directory: ~/project
jobs:
install:
executor: playwright
steps:
- checkout
- restore_cache:
keys:
- npm-deps-{{ checksum "package-lock.json" }}
- run: npm ci
- save_cache:
key: npm-deps-{{ checksum "package-lock.json" }}
paths:
- node_modules
- persist_to_workspace:
root: .
paths:
- node_modules
test:
executor: playwright
parallelism: 4
steps:
- checkout
- attach_workspace:
at: .
- run:
name: Run Playwright tests
command: |
npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
- store_artifacts:
path: playwright-report
destination: playwright-report
- store_artifacts:
path: test-results
destination: test-results
- store_test_results:
path: test-results/junit.xml
workflows:
test:
jobs:
- install
- test:
requires:
- installConfig for CircleCI JUnit integration:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
['dot'],
['html', { open: 'never' }],
['junit', { outputFile: 'test-results/junit.xml' }],
]
: [['html', { open: 'on-failure' }]],
});CircleCI with Orbs (Simplified)
# .circleci/config.yml
version: 2.1
orbs:
node: circleci/node@6.1
executors:
playwright:
docker:
- image: mcr.microsoft.com/playwright:v1.52.0-noble
jobs:
e2e:
executor: playwright
parallelism: 4
steps:
- checkout
- node/install-packages
- run:
name: Run tests
command: npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
- store_artifacts:
path: playwright-report
- store_test_results:
path: test-results/junit.xml
workflows:
main:
jobs:
- e2e---
Pattern 2: Azure DevOps
Use when: Your project runs on Azure DevOps Pipelines.
Basic Pipeline
# azure-pipelines.yml
trigger:
branches:
include:
- main
pr:
branches:
include:
- main
pool:
vmImage: 'ubuntu-latest'
variables:
CI: 'true'
npm_config_cache: $(Pipeline.Workspace)/.npm
steps:
- task: NodeTool@0
inputs:
versionSpec: '20.x'
displayName: 'Install Node.js'
- task: Cache@2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
path: $(npm_config_cache)
displayName: 'Cache npm'
- script: npm ci
displayName: 'Install dependencies'
- script: npx playwright install --with-deps
displayName: 'Install Playwright browsers'
- script: npx playwright test
displayName: 'Run Playwright tests'
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'test-results/junit.xml'
mergeTestResults: true
testRunTitle: 'Playwright Tests'
displayName: 'Publish test results'
- task: PublishPipelineArtifact@1
condition: always()
inputs:
targetPath: playwright-report
artifact: playwright-report
publishLocation: 'pipeline'
displayName: 'Upload report'Config for Azure DevOps JUnit integration:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
['dot'],
['html', { open: 'never' }],
['junit', { outputFile: 'test-results/junit.xml' }],
]
: [['html', { open: 'on-failure' }]],
});Azure DevOps with Sharding
# azure-pipelines.yml
trigger:
branches:
include:
- main
pr:
branches:
include:
- main
variables:
CI: 'true'
stages:
- stage: Test
jobs:
- job: Playwright
pool:
vmImage: 'ubuntu-latest'
strategy:
matrix:
shard1:
SHARD: '1/4'
shard2:
SHARD: '2/4'
shard3:
SHARD: '3/4'
shard4:
SHARD: '4/4'
steps:
- task: NodeTool@0
inputs:
versionSpec: '20.x'
- script: npm ci
displayName: 'Install dependencies'
- script: npx playwright install --with-deps
displayName: 'Install browsers'
- script: npx playwright test --shard=$(SHARD)
displayName: 'Run tests (shard $(SHARD))'
- task: PublishPipelineArtifact@1
condition: always()
inputs:
targetPath: blob-report
artifact: blob-report-$(System.JobPositionInPhase)
displayName: 'Upload blob report'
- stage: Report
dependsOn: Test
condition: always()
jobs:
- job: MergeReports
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool@0
inputs:
versionSpec: '20.x'
- script: npm ci
displayName: 'Install dependencies'
- task: DownloadPipelineArtifact@2
inputs:
patterns: 'blob-report-*/**'
path: all-blob-reports
displayName: 'Download all blob reports'
- script: npx playwright merge-reports --reporter=html ./all-blob-reports
displayName: 'Merge reports'
- task: PublishPipelineArtifact@1
inputs:
targetPath: playwright-report
artifact: playwright-report
displayName: 'Upload merged report'---
Pattern 3: Jenkins
Use when: Your project runs on Jenkins.
Jenkinsfile (Declarative Pipeline)
// Jenkinsfile
pipeline {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root' // Playwright needs root in container
}
}
environment {
CI = 'true'
HOME = '/root'
npm_config_cache = "${WORKSPACE}/.npm"
}
options {
timeout(time: 30, unit: 'MINUTES')
disableConcurrentBuilds()
}
stages {
stage('Install') {
steps {
sh 'npm ci'
}
}
stage('Test') {
steps {
sh 'npx playwright test'
}
post {
always {
// Publish JUnit results
junit allowEmptyResults: true,
testResults: 'test-results/junit.xml'
// Archive HTML report
archiveArtifacts artifacts: 'playwright-report/**',
allowEmptyArchive: true
// Archive traces on failure
archiveArtifacts artifacts: 'test-results/**',
allowEmptyArchive: true
}
}
}
}
post {
failure {
// Notify on failure (Slack, email, etc.)
echo 'Playwright tests failed!'
}
cleanup {
cleanWs()
}
}
}Config for Jenkins JUnit integration:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
['dot'],
['html', { open: 'never' }],
['junit', { outputFile: 'test-results/junit.xml' }],
]
: [['html', { open: 'on-failure' }]],
});Jenkins with Parallel Stages
// Jenkinsfile (sharded)
pipeline {
agent none
environment {
CI = 'true'
HOME = '/root'
}
options {
timeout(time: 30, unit: 'MINUTES')
}
stages {
stage('Test') {
parallel {
stage('Shard 1') {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
steps {
sh 'npm ci'
sh 'npx playwright test --shard=1/4'
}
post {
always {
archiveArtifacts artifacts: 'blob-report/**',
allowEmptyArchive: true
}
}
}
stage('Shard 2') {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
steps {
sh 'npm ci'
sh 'npx playwright test --shard=2/4'
}
post {
always {
archiveArtifacts artifacts: 'blob-report/**',
allowEmptyArchive: true
}
}
}
stage('Shard 3') {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
steps {
sh 'npm ci'
sh 'npx playwright test --shard=3/4'
}
post {
always {
archiveArtifacts artifacts: 'blob-report/**',
allowEmptyArchive: true
}
}
}
stage('Shard 4') {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
steps {
sh 'npm ci'
sh 'npx playwright test --shard=4/4'
}
post {
always {
archiveArtifacts artifacts: 'blob-report/**',
allowEmptyArchive: true
}
}
}
}
}
}
}Decision Guide
| CI Platform | Docker Image Support | Native Parallelism | Artifact Browsing | JUnit Integration |
|---|---|---|---|---|
| CircleCI | First-class (docker: executor) | parallelism: N with CIRCLE_NODE_INDEX | Via artifacts tab | store_test_results |
| Azure DevOps | Via vmImage or container jobs | strategy.matrix | Pipeline Artifacts UI | PublishTestResults@2 |
| Jenkins | Docker Pipeline plugin | parallel stages | Archived Artifacts | junit step |
| Scenario | CircleCI | Azure DevOps | Jenkins |
|---|---|---|---|
| Shard variable | $((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL | Define in matrix: SHARD: '1/4' | Hardcode per parallel stage |
| Cache key | checksum "package-lock.json" | Cache@2 with key template | stash/unstash or shared volume |
| Secrets | Context + environment variables | Variable groups + pipeline variables | Credentials plugin |
| Report upload | store_artifacts | PublishPipelineArtifact@1 | archiveArtifacts |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
Installing browsers on bare metal without --with-deps | Missing OS libs cause launch failures | Use Playwright Docker image or --with-deps flag |
| No JUnit reporter | CI platform can't display test results natively | Add ['junit', { outputFile: 'test-results/junit.xml' }] |
| Unlimited job timeout | Hung tests run indefinitely, consuming CI resources | Set explicit timeout (20-30 min) |
| No artifact upload on success | Can't verify results when tests pass | Always upload reports (condition: always() / when: always) |
| Running browsers as non-root in container without setup | Permission errors on browser binaries | Run as root or configure proper permissions |
| Hardcoding shard count in config instead of using CI variables | Must update two places when changing parallelism | Use CI-native variables (CI_NODE_TOTAL, CIRCLE_NODE_TOTAL) |
Troubleshooting
CircleCI: "Error: browserType.launch: Executable doesn't exist"
Cause: Not using the Playwright Docker image, or image version doesn't match @playwright/test version.
Fix: Match image tag to your Playwright version:
docker:
- image: mcr.microsoft.com/playwright:v1.52.0-noble # match package.json versionAzure DevOps: Test results not showing in Tests tab
Cause: JUnit reporter not configured, or PublishTestResults@2 task missing.
Fix: Add both the reporter and the publish task:
// playwright.config.ts
reporter: [['junit', { outputFile: 'test-results/junit.xml' }]],- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'test-results/junit.xml'Jenkins: "Browser closed unexpectedly" in Docker agent
Cause: Running as non-root user in container. Chromium's sandbox needs root or --no-sandbox.
Fix: Run as root in the Docker agent:
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
environment {
HOME = '/root'
}All platforms: Shard index off by one
Cause: CircleCI's CIRCLE_NODE_INDEX is 0-based, but Playwright's --shard is 1-based.
Fix: Add 1 to the index for CircleCI:
# CircleCI
command: npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
# GitLab (already 1-based)
command: npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTALRelated
- ci/ci-github-actions.md -- GitHub Actions configuration
- ci/ci-gitlab.md -- GitLab CI configuration
- ci/parallel-and-sharding.md -- sharding strategies
- ci/docker-and-containers.md -- Docker image details
- ci/reporting-and-artifacts.md -- reporter configuration for CI
Docker and Containers
When to use: Running Playwright tests in containers for reproducible environments, CI pipelines, or local development with consistent browser versions. Essential when your team needs identical test environments across machines.
Quick Reference
# Official Playwright Docker images
docker pull mcr.microsoft.com/playwright:v1.52.0-noble # Ubuntu 24.04, all browsers
docker pull mcr.microsoft.com/playwright:v1.52.0-jammy # Ubuntu 22.04, all browsers
# Run tests in container
docker run --rm -v $(pwd):/app -w /app mcr.microsoft.com/playwright:v1.52.0-noble \
npx playwright test
# Check your Playwright version (must match image tag)
npx playwright --versionPatterns
Pattern 1: Running Tests in the Official Image
Use when: Quick, reproducible test runs without building a custom image. Avoid when: You need application services (database, API) running alongside -- use docker-compose instead.
# Run all tests
docker run --rm \
-v $(pwd):/app \
-w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"
# Run with environment variables
docker run --rm \
-v $(pwd):/app \
-w /app \
-e CI=true \
-e BASE_URL=http://host.docker.internal:3000 \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"
# Run and extract report
docker run --rm \
-v $(pwd):/app \
-w /app \
-v $(pwd)/playwright-report:/app/playwright-report \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"Pattern 2: Custom Dockerfile
Use when: You need additional system dependencies, pre-installed npm packages, or a smaller image with only certain browsers. Avoid when: The official image works as-is.
# Dockerfile.playwright
FROM mcr.microsoft.com/playwright:v1.52.0-noble
WORKDIR /app
# Copy package files first for better layer caching
COPY package.json package-lock.json ./
RUN npm ci
# Copy the rest of the project
COPY . .
# Default command
CMD ["npx", "playwright", "test"]# Build and run
docker build -f Dockerfile.playwright -t my-e2e-tests .
docker run --rm my-e2e-tests
# Run with specific options
docker run --rm my-e2e-tests npx playwright test --project=chromium
# Extract report
docker run --rm -v $(pwd)/reports:/app/playwright-report my-e2e-testsSlim image with only Chromium:
# Dockerfile.playwright-chromium
FROM node:20-slim
# Install only Chromium dependencies
RUN npx playwright install --with-deps chromium
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
CMD ["npx", "playwright", "test", "--project=chromium"]Pattern 3: Docker Compose with Application Stack
Use when: Tests need the full application stack: web server, database, cache, and Playwright running together. Avoid when: Tests run against a remote environment (staging/prod) -- no local services needed.
# docker-compose.yml
services:
# Application under test
app:
build:
context: .
dockerfile: Dockerfile
ports:
- "3000:3000"
environment:
- NODE_ENV=test
- DATABASE_URL=postgresql://postgres:postgres@db:5432/test
- REDIS_URL=redis://cache:6379
depends_on:
db:
condition: service_healthy
cache:
condition: service_started
# Database
db:
image: postgres:16-alpine
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 5s
retries: 5
tmpfs:
- /var/lib/postgresql/data # RAM disk for speed
# Cache
cache:
image: redis:7-alpine
# Playwright test runner
e2e:
image: mcr.microsoft.com/playwright:v1.52.0-noble
working_dir: /app
volumes:
- .:/app
- /app/node_modules # prevent host node_modules from overriding
environment:
- CI=true
- BASE_URL=http://app:3000
depends_on:
- app
command: bash -c "npm ci && npx playwright test"
profiles:
- test # only start with: docker compose --profile test up# Run the full stack with tests
docker compose --profile test up --abort-on-container-exit --exit-code-from e2e
# Run just the app (for local dev)
docker compose up app
# Run tests against already-running stack
docker compose --profile test run --rm e2e npx playwright test
# Tear down everything
docker compose --profile test down -vPattern 4: Extracting Reports and Traces from Containers
Use when: You need test artifacts (HTML reports, traces, screenshots) accessible on the host after container tests complete. Avoid when: CI handles artifact collection natively (GitHub Actions, GitLab artifacts).
# Method 1: Bind mount the output directories
docker run --rm \
-v $(pwd):/app \
-v $(pwd)/playwright-report:/app/playwright-report \
-v $(pwd)/test-results:/app/test-results \
-w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"
# Method 2: Copy artifacts from a stopped container
docker run --name e2e-run \
-v $(pwd):/app \
-w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test" || true
docker cp e2e-run:/app/playwright-report ./playwright-report
docker cp e2e-run:/app/test-results ./test-results
docker rm e2e-run
# View the report
npx playwright show-report ./playwright-reportDocker Compose for report extraction:
# docker-compose.yml (add to e2e service)
services:
e2e:
image: mcr.microsoft.com/playwright:v1.52.0-noble
working_dir: /app
volumes:
- .:/app
- ./playwright-report:/app/playwright-report
- ./test-results:/app/test-results
# ... rest of configPattern 5: CI Container Strategies
Use when: Your CI environment benefits from containerized test execution.
GitHub Actions -- container job:
# .github/workflows/playwright.yml
jobs:
test:
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.52.0-noble
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npx playwright test
env:
HOME: /rootGitLab CI -- image directive:
# .gitlab-ci.yml
test:
image: mcr.microsoft.com/playwright:v1.52.0-noble
script:
- npm ci
- npx playwright testJenkins -- Docker agent:
// Jenkinsfile
pipeline {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
stages {
stage('Test') {
steps {
sh 'npm ci'
sh 'npx playwright test'
}
}
}
}Pattern 6: Development Container (devcontainer)
Use when: Your team uses VS Code Dev Containers or GitHub Codespaces and needs Playwright available in the development environment. Avoid when: Everyone installs Playwright locally and version differences don't cause issues.
// .devcontainer/devcontainer.json
{
"name": "Playwright Dev",
"image": "mcr.microsoft.com/playwright:v1.52.0-noble",
"features": {
"ghcr.io/devcontainers/features/node:1": {
"version": "20"
}
},
"postCreateCommand": "npm ci",
"customizations": {
"vscode": {
"extensions": [
"ms-playwright.playwright"
]
}
},
"forwardPorts": [3000, 9323],
"remoteUser": "root"
}Decision Guide
| Scenario | Approach | Why |
|---|---|---|
| Simple CI pipeline | Official Playwright image as CI image | Browsers pre-installed; zero browser install time |
| Tests need database + cache | Docker Compose with app, db, cache, e2e services | Full stack in isolated containers |
| Team needs identical environments | Dev Container or custom Dockerfile | Eliminate "works on my machine" |
| Only testing Chromium | Slim image: node:20-slim + install --with-deps chromium | Smaller image, faster pulls |
| Cross-browser testing | Official Playwright image (has all browsers) | All three engines pre-installed |
| Local development | Run directly on host, not in container | Faster iteration, easier debugging |
| CI with artifact extraction | Bind mount report/results dirs or use CI artifact upload | Reports accessible after container exits |
| Image | Size | Browsers | Base OS |
|---|---|---|---|
mcr.microsoft.com/playwright:v1.52.0-noble | ~2 GB | Chromium, Firefox, WebKit | Ubuntu 24.04 |
mcr.microsoft.com/playwright:v1.52.0-jammy | ~2 GB | Chromium, Firefox, WebKit | Ubuntu 22.04 |
| Custom slim (Chromium only) | ~800 MB | Chromium | Depends on base |
Security: Pinning Docker Images to Digest
Version tags like mcr.microsoft.com/playwright:v1.52.0-noble are mutable — the tag can be updated to point to a different image layer without changing the tag name. This is a supply-chain risk (W012): you may pull different code than you tested against.
Best practice: pin images to their immutable content digest.
# Get the digest for a specific tag
docker pull mcr.microsoft.com/playwright:v1.52.0-noble
docker inspect --format='{{index .RepoDigests 0}}' mcr.microsoft.com/playwright:v1.52.0-noble
# e.g. mcr.microsoft.com/playwright@sha256:abc123...
# Or use skopeo (no pull required):
skopeo inspect docker://mcr.microsoft.com/playwright:v1.52.0-noble | jq '.Digest'Use digest-pinned references in CI:
# Instead of:
image: mcr.microsoft.com/playwright:v1.52.0-noble
# Pin to digest (example — verify the actual digest for your version):
image: mcr.microsoft.com/playwright:v1.52.0-noble@sha256:<digest-from-inspect># Dockerfile
FROM mcr.microsoft.com/playwright:v1.52.0-noble@sha256:<digest-from-inspect>Digest values are specific to the exact image build. Verify the digest from the official Microsoft Artifact Registry (mcr.microsoft.com) before pinning.
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
Image tag doesn't match @playwright/test version | Browser binaries incompatible with Playwright library | Always match: v1.52.0 image for @playwright/test@1.52.0 |
Using latest tag | Unpredictable; image updates can break tests | Pin to exact version: v1.52.0-noble |
| Using only a version tag without digest | Tag is mutable; supply-chain risk if image is silently updated | Pin to digest: playwright:v1.52.0-noble@sha256:<digest> |
| Installing browsers inside container at runtime | Wastes 60-90 seconds on every run | Use official image (browsers pre-installed) or build custom image with browsers baked in |
| Running as non-root without configuring sandbox | Chromium sandbox fails with permission errors | Run as root (-u root) or disable sandbox (--no-sandbox in launch args) |
Bind-mounting node_modules from host | Platform-specific binaries (macOS vs Linux) cause crashes | Use anonymous volume: -v /app/node_modules |
| No health checks on dependent services | Tests start before database is ready | Add healthcheck to db service; use depends_on: condition: service_healthy |
| Building application inside the Playwright container | Large image, slow builds, wrong base for your app | Separate app and e2e containers in docker-compose |
Troubleshooting
"browserType.launch: Executable doesn't exist" in container
Cause: Playwright version in package.json doesn't match the Docker image version.
Fix: Ensure exact version match:
# Check your version
npm ls @playwright/test
# @playwright/test@1.52.0
# Use matching image
docker pull mcr.microsoft.com/playwright:v1.52.0-nobleTests fail with "net::ERR_CONNECTION_REFUSED" in docker-compose
Cause: Tests are trying to reach localhost:3000 but the app is in a different container.
Fix: Use the service name as hostname and configure baseURL:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
baseURL: process.env.BASE_URL || 'http://localhost:3000',
},
// Disable webServer in Docker -- app is managed by docker-compose
...(process.env.CI ? {} : {
webServer: {
command: 'npm run dev',
url: 'http://localhost:3000',
reuseExistingServer: true,
},
}),
});# docker-compose.yml
e2e:
environment:
- BASE_URL=http://app:3000 # "app" is the service namePermission denied on mounted volumes
Cause: Container runs as root but host files are owned by your user, or vice versa.
Fix: Match user IDs or run as root:
# Run as your host user
docker run --rm -u $(id -u):$(id -g) \
-v $(pwd):/app -w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
npx playwright test
# Or run as root (simpler, fine for CI)
docker run --rm \
-v $(pwd):/app -w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
npx playwright testContainer tests are much slower than local
Cause: Docker Desktop on macOS/Windows has I/O overhead for bind-mounted volumes.
Fix: Copy files into the container instead of mounting:
# Dockerfile.playwright
FROM mcr.microsoft.com/playwright:v1.52.0-noble
WORKDIR /app
COPY . .
RUN npm ci
CMD ["npx", "playwright", "test"]Or use delegated mount on macOS:
docker run --rm \
-v $(pwd):/app:delegated \
-w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"Related
- ci/ci-github-actions.md -- container jobs in GitHub Actions
- ci/ci-gitlab.md -- Docker images in GitLab CI
- ci/ci-other.md -- Docker agents in Jenkins, CircleCI
- ci/parallel-and-sharding.md -- sharding within containers
- ci/reporting-and-artifacts.md -- extracting reports from containers
Global Setup and Teardown
When to use: One-time operations that must run before or after the entire test suite -- database seeding, environment health checks, creating shared auth state, starting external services. Runs once per npx playwright test invocation, not once per test or per worker.Quick Reference
globalSetup → runs ONCE before all tests in all projects
↓
setup projects → runs before dependent projects (has browser context)
↓
test projects → your actual tests
↓
teardown projects → runs after dependent projects (has browser context)
↓
globalTeardown → runs ONCE after all tests in all projectsKey distinction:
globalSetup/globalTeardown: No browser, no Playwright fixtures. Pure Node.js.- Setup projects with
dependencies: Has full browser context, can usepage,request, etc.
Patterns
Pattern 1: Basic Global Setup and Teardown
Use when: One-time non-browser work like database seeding, environment validation, or external service preparation. Avoid when: You need a browser (use a setup project instead) or per-test isolation (use fixtures).
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
globalSetup: './tests/global-setup.ts',
globalTeardown: './tests/global-teardown.ts',
testDir: './tests',
});// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
console.log('Global setup: seeding database...');
// Seed the test database
const { execSync } = await import('child_process');
execSync('npx prisma db push --force-reset', { stdio: 'inherit' });
execSync('npx prisma db seed', { stdio: 'inherit' });
// Store run metadata for tests to use
process.env.TEST_RUN_ID = `run-${Date.now()}`;
}
export default globalSetup;// tests/global-teardown.ts
import type { FullConfig } from '@playwright/test';
async function globalTeardown(config: FullConfig) {
console.log('Global teardown: cleaning up...');
const { execSync } = await import('child_process');
execSync('npx prisma db push --force-reset', { stdio: 'inherit' });
}
export default globalTeardown;Pattern 2: Environment Health Check in Global Setup
Use when: Verifying the test environment is healthy before running any tests. Fails fast if services are down. Avoid when: Tests use webServer which already does a health check.
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
const maxRetries = 10;
const retryDelay = 2000;
console.log(`Checking if ${baseURL} is reachable...`);
for (let i = 0; i < maxRetries; i++) {
try {
const response = await fetch(`${baseURL}/api/health`);
if (response.ok) {
console.log(`Environment is healthy (attempt ${i + 1})`);
return;
}
} catch {
// Connection refused or timeout -- retry
}
console.log(`Waiting for environment... (attempt ${i + 1}/${maxRetries})`);
await new Promise((resolve) => setTimeout(resolve, retryDelay));
}
throw new Error(`Environment at ${baseURL} is not reachable after ${maxRetries} attempts`);
}
export default globalSetup;Pattern 3: Authentication State in Global Setup (Without Browser)
Use when: Creating auth tokens or session cookies via API, without needing a browser. Avoid when: Login requires browser interaction (use a setup project instead).
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
import * as fs from 'fs';
import * as path from 'path';
async function globalSetup(config: FullConfig) {
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
// Authenticate via API (no browser needed)
const response = await fetch(`${baseURL}/api/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
email: 'admin@example.com',
password: process.env.TEST_PASSWORD,
}),
});
if (!response.ok) {
throw new Error(`Authentication failed: ${response.status} ${response.statusText}`);
}
const { token } = await response.json();
// Save the token as storageState for browser tests to pick up
const authDir = path.resolve(process.cwd(), 'playwright/.auth');
fs.mkdirSync(authDir, { recursive: true });
const storageState = {
cookies: [],
origins: [
{
origin: baseURL,
localStorage: [
{ name: 'auth_token', value: token },
],
},
],
};
fs.writeFileSync(
path.join(authDir, 'user.json'),
JSON.stringify(storageState, null, 2)
);
}
export default globalSetup;// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
globalSetup: './tests/global-setup.ts',
projects: [
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/user.json',
},
},
],
});Pattern 4: Passing Data from Global Setup to Tests
Use when: Global setup generates values (IDs, tokens, URLs) that tests need. Avoid when: Each test should create its own data (the usual case).
Method 1: Environment variables (simplest):
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
process.env.TEST_RUN_ID = `run-${Date.now()}`;
process.env.SEED_USER_ID = 'user-12345';
}
export default globalSetup;// tests/dashboard.spec.ts
import { test, expect } from '@playwright/test';
test('dashboard shows seeded data', async ({ page }) => {
const userId = process.env.SEED_USER_ID;
await page.goto(`/users/${userId}/dashboard`);
await expect(page.getByRole('heading')).toBeVisible();
});Method 2: Shared file (for complex data):
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
import * as fs from 'fs';
import * as path from 'path';
const SETUP_DATA_PATH = path.resolve(process.cwd(), 'test-data/setup-data.json');
async function globalSetup(config: FullConfig) {
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
// Create test data via API
const res = await fetch(`${baseURL}/api/test/seed`, { method: 'POST' });
const seedData = await res.json();
// Write to shared file
const dir = path.dirname(SETUP_DATA_PATH);
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(SETUP_DATA_PATH, JSON.stringify(seedData, null, 2));
}
export default globalSetup;// tests/helpers/setup-data.ts
import * as fs from 'fs';
import * as path from 'path';
const SETUP_DATA_PATH = path.resolve(process.cwd(), 'test-data/setup-data.json');
export function getSetupData(): { userId: string; orgId: string; apiKey: string } {
const raw = fs.readFileSync(SETUP_DATA_PATH, 'utf8');
return JSON.parse(raw);
}// tests/org-settings.spec.ts
import { test, expect } from '@playwright/test';
import { getSetupData } from './helpers/setup-data';
test('org settings page loads', async ({ page }) => {
const { orgId } = getSetupData();
await page.goto(`/orgs/${orgId}/settings`);
await expect(page.getByRole('heading', { name: 'Settings' })).toBeVisible();
});Pattern 5: Global Setup with storageState (Browser-Based Auth)
Use when: Authentication requires browser interaction (form login, OAuth redirect, MFA). Avoid when: Auth can be done via API call (use Pattern 3 instead).
Important: globalSetup has no browser. For browser-based auth, use a setup project instead.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
projects: [
// Setup project: runs first, has a browser, saves auth state
{
name: 'setup',
testMatch: /global\.setup\.ts/,
},
// Test projects: depend on setup, reuse saved auth state
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
{
name: 'firefox',
use: {
...devices['Desktop Firefox'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
],
});// tests/global.setup.ts
import { test as setup, expect } from '@playwright/test';
const authFile = 'playwright/.auth/user.json';
setup('authenticate', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
// Wait for navigation to confirm login succeeded
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
// Save signed-in state (cookies + localStorage)
await page.context().storageState({ path: authFile });
});Pattern 6: Global Setup for External Services
Use when: Starting or configuring external services (mock servers, test containers, feature flags) before any tests run. Avoid when: Per-test or per-worker isolation is needed (use fixtures).
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
// Start a mock API server
const { createServer } = await import('../mocks/server');
const server = await createServer();
const port = await server.listen(0);
process.env.MOCK_API_URL = `http://localhost:${port}`;
// Configure feature flags for test environment
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
await fetch(`${baseURL}/api/admin/feature-flags`, {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${process.env.ADMIN_API_KEY}`,
},
body: JSON.stringify({
newCheckout: true,
darkMode: false,
betaFeatures: true,
}),
});
// Return a cleanup function (Playwright calls globalTeardown separately)
// For cleanup, use globalTeardown
}
export default globalSetup;// tests/global-teardown.ts
import type { FullConfig } from '@playwright/test';
async function globalTeardown(config: FullConfig) {
// Reset feature flags
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
await fetch(`${baseURL}/api/admin/feature-flags/reset`, {
method: 'POST',
headers: { Authorization: `Bearer ${process.env.ADMIN_API_KEY}` },
});
}
export default globalTeardown;Decision Guide
| Need | Use | Why |
|---|---|---|
| One-time DB seed | globalSetup | No browser needed; runs once before everything |
| Browser-based login (shared state) | Setup project with dependencies | Needs page and context (not available in globalSetup) |
| API-based auth token | globalSetup | Simple HTTP call, no browser needed |
| Per-test unique data | Custom fixture via test.extend() | Each test gets isolated data |
| Per-worker shared resource | Worker-scoped fixture ({ scope: 'worker' }) | Shared within worker, isolated between workers |
| Health check before tests | globalSetup | Fail fast if environment is down |
| Start mock server | globalSetup + globalTeardown | One-time server lifecycle |
| Clean up after all tests | globalTeardown | Runs once at the end regardless of pass/fail |
Do I need globalSetup?
│
├── Does the work need a browser (page, context)?
│ ├── YES → Use a setup project, not globalSetup
│ └── NO → globalSetup is appropriate
│
├── Does every test need unique/isolated data?
│ ├── YES → Use a fixture with test.extend()
│ └── NO → globalSetup for shared, read-only data
│
├── Is it per-worker (expensive resource, connection pool)?
│ ├── YES → Worker-scoped fixture
│ └── NO → globalSetup for truly global, one-time work
│
└── Is it cleanup?
├── After all tests → globalTeardown
├── After each test → Fixture teardown (after use())
└── After each worker → Worker-scoped fixture teardownAnti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
Browser login in globalSetup | No browser context available; complex workarounds | Use a setup project with dependencies |
Creating per-test data in globalSetup | All tests share the same data; not isolated | Use per-test fixtures for unique data |
globalSetup without globalTeardown | Database or services left in dirty state | Always pair setup with teardown |
| Storing setup results in module-level variables | Workers are separate processes; variables don't share | Use environment variables or files |
Complex logic in globalSetup | Hard to debug; runs outside normal test lifecycle | Keep it minimal: seed, verify, set env vars |
globalSetup that takes > 60 seconds | Slows every test run, even for a single test | Move heavy work to a separate script or CI step |
Relying on globalTeardown for critical cleanup | If the process crashes, globalTeardown might not run | Design tests to be idempotent; use beforeAll in setup project |
Troubleshooting
Global setup runs but environment variables are not available in tests
Cause: Each worker is a separate process. process.env mutations in globalSetup propagate to workers, but only if set before workers spawn.
Fix: Set environment variables at the top level of globalSetup, before any async work that might delay:
// tests/global-setup.ts
async function globalSetup() {
// This works -- set before returning
process.env.TEST_RUN_ID = `run-${Date.now()}`;
}
export default globalSetup;If environment variables are still missing, write data to a file instead (Pattern 4, Method 2).
Global setup fails with "Cannot find module"
Cause: The path in globalSetup is relative to the config file, but the module resolution is wrong.
Fix: Use a path relative to the project root:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
globalSetup: './tests/global-setup.ts', // relative to config location
globalTeardown: './tests/global-teardown.ts',
});Global teardown doesn't run after test failure
Cause: If the process is killed (SIGKILL, OOM) rather than exiting normally, teardown is skipped.
Fix: Design your setup to be idempotent. Global setup should handle a dirty state from a previous incomplete run:
// tests/global-setup.ts
async function globalSetup() {
// Always reset first, then seed -- handles dirty state
const { execSync } = await import('child_process');
execSync('npx prisma db push --force-reset', { stdio: 'inherit' });
execSync('npx prisma db seed', { stdio: 'inherit' });
}
export default globalSetup;Setup project runs every time, even when only running one test file
Cause: The dependencies configuration requires the setup project to run before any dependent project.
Fix: This is expected behavior. To skip setup during focused debugging:
# Skip setup by running without dependencies
npx playwright test --project=chromium --no-deps tests/specific-test.spec.tsstorageState file doesn't exist when tests start
Cause: The setup project or globalSetup that creates the file failed silently, or the path is wrong.
Fix: Add explicit error handling and verify the file exists:
// tests/global.setup.ts
import { test as setup, expect } from '@playwright/test';
import * as fs from 'fs';
const authFile = 'playwright/.auth/user.json';
setup('authenticate', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
await page.context().storageState({ path: authFile });
// Verify the file was created
if (!fs.existsSync(authFile)) {
throw new Error(`Auth state file was not created at ${authFile}`);
}
});Related
- core/fixtures-and-hooks.md -- per-test and per-worker fixtures (preferred over globalSetup for most cases)
- core/configuration.md --
globalSetup,globalTeardown,webServerconfig - ci/projects-and-dependencies.md -- setup projects with
dependencies - core/authentication.md -- authentication patterns using setup projects
- core/test-data-management.md -- seeding and managing test data
Parallel Execution and Sharding
When to use: Speeding up test suites by running tests concurrently within a single machine (parallelism) or across multiple machines (sharding). Essential once your suite exceeds 5 minutes.
Quick Reference
# Workers (parallelism within one machine)
npx playwright test --workers=4 # fixed worker count
npx playwright test --workers=50% # percentage of CPU cores
# Sharding (splitting across machines)
npx playwright test --shard=1/4 # run first quarter
npx playwright test --shard=2/4 # run second quarter
# Merging shard results
npx playwright merge-reports ./blob-report # merge to default HTML
npx playwright merge-reports --reporter=html,json ./blob-report # multiple formats
# Fully parallel mode
npx playwright test --fully-parallel # override config for this runPatterns
Pattern 1: Configuring Workers
Use when: Controlling how many tests run simultaneously on one machine. Avoid when: You only have 1-2 tests (parallelism has no effect).
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
// fullyParallel: true means tests WITHIN a file also run in parallel.
// Without it, only files run in parallel (tests within a file are serial).
fullyParallel: true,
// Workers: how many parallel processes
// - undefined: auto-detect (half CPU cores, capped at a reasonable number)
// - number: fixed count
// - string percentage: '50%' of CPU cores
workers: process.env.CI ? '50%' : undefined,
});What `fullyParallel` actually controls:
| Setting | Files run in parallel | Tests within a file run in parallel |
|---|---|---|
fullyParallel: false (default) | Yes | No -- serial within each file |
fullyParallel: true | Yes | Yes -- every test is independent |
Per-file override when one file needs serial execution:
// tests/onboarding.spec.ts
import { test, expect } from '@playwright/test';
// This file's tests run serially even with fullyParallel: true in config
test.describe.configure({ mode: 'serial' });
test('step 1: enter company name', async ({ page }) => {
// ...
});
test('step 2: choose plan', async ({ page }) => {
// ...
});Pattern 2: Sharding Across CI Machines
Use when: Suite is too slow for a single machine even with maximum workers. Split work across N separate CI jobs. Avoid when: Suite runs under 5 minutes on one machine.
Sharding splits the test file list into N equal groups. Each shard runs one group.
# Machine 1 Machine 2 Machine 3 Machine 4
--shard=1/4 --shard=2/4 --shard=3/4 --shard=4/4Config for sharded runs -- use blob reporter:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
fullyParallel: true,
workers: process.env.CI ? '50%' : undefined,
// Blob reporter outputs a binary file that can be merged later.
// In non-CI, use HTML for local viewing.
reporter: process.env.CI
? [['blob'], ['github']]
: [['html', { open: 'on-failure' }]],
});Pattern 3: Merging Blob Reports from Shards
Use when: You sharded your tests and need a single unified report. Avoid when: No sharding -- the regular HTML reporter works directly.
Each shard produces a .zip file in blob-report/. After all shards complete, merge them:
# Download all blob-report/ directories into one folder, then:
npx playwright merge-reports --reporter=html ./all-blob-reports
# Multiple output formats
npx playwright merge-reports --reporter=html,json,junit ./all-blob-reports
# Custom output directory
PLAYWRIGHT_HTML_REPORT=merged-report npx playwright merge-reports --reporter=html ./all-blob-reportsGitHub Actions example (merge job):
merge-reports:
if: ${{ !cancelled() }}
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- uses: actions/download-artifact@v4
with:
path: all-blob-reports
pattern: blob-report-*
merge-multiple: true
- run: npx playwright merge-reports --reporter=html ./all-blob-reports
- uses: actions/upload-artifact@v4
with:
name: playwright-report
path: playwright-report/
retention-days: 14Pattern 4: Worker-Scoped Fixtures for Shared Resources
Use when: Parallel workers each need an expensive resource (database connection, auth token) that should be created once per worker, not once per test. Avoid when: The resource is cheap to create -- use a regular test-scoped fixture.
// fixtures.ts
import { test as base } from '@playwright/test';
type WorkerFixtures = {
dbConnection: DatabaseClient;
workerAuthToken: string;
};
export const test = base.extend<{}, WorkerFixtures>({
// Created once per worker process, shared across all tests in that worker
dbConnection: [async ({}, use) => {
const db = await DatabaseClient.connect(process.env.DB_URL!);
await use(db);
await db.disconnect();
}, { scope: 'worker' }],
workerAuthToken: [async ({}, use, workerInfo) => {
// Each worker gets a unique user to avoid test interference
const response = await fetch(`${process.env.API_URL}/auth/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
username: `worker-user-${workerInfo.workerIndex}`,
password: process.env.TEST_PASSWORD,
}),
});
const { token } = await response.json();
await use(token);
}, { scope: 'worker' }],
});
export { expect } from '@playwright/test';Pattern 5: Test Isolation for Safe Parallelism
Use when: Preparing tests to run in parallel without interference. Avoid when: Never -- isolation is always required for reliable parallel execution.
The golden rule: Each test must create its own state and clean up after itself. No test should depend on or modify state that another test uses.
// BAD: Tests share a hardcoded user -- parallel runs collide
test('update profile', async ({ page }) => {
await page.goto('/users/shared-user/profile');
await page.getByLabel('Name').fill('New Name');
await page.getByRole('button', { name: 'Save' }).click();
// Another parallel test also editing "shared-user" -- race condition!
});
// GOOD: Each test creates its own user
test('update profile', async ({ page, request }) => {
// Create a unique user for this test
const res = await request.post('/api/test/users', {
data: { name: `user-${Date.now()}`, email: `${Date.now()}@test.com` },
});
const user = await res.json();
await page.goto(`/users/${user.id}/profile`);
await page.getByLabel('Name').fill('Updated Name');
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByLabel('Name')).toHaveValue('Updated Name');
// Cleanup
await request.delete(`/api/test/users/${user.id}`);
});Using `workerInfo` and `testInfo` for unique identifiers:
import { test, expect } from '@playwright/test';
test('create order', async ({ page }, testInfo) => {
const uniqueId = `order-${testInfo.workerIndex}-${Date.now()}`;
// Use uniqueId for any data this test creates
await page.goto(`/orders/new?ref=${uniqueId}`);
// ...
});Pattern 6: Dynamic Shard Count Based on Test Count
Use when: You want to automatically adjust shard count based on how many tests exist, rather than hardcoding. Avoid when: Your test count is stable and a fixed shard count works well.
# .github/workflows/playwright.yml -- dynamic shard calculation
jobs:
determine-shards:
runs-on: ubuntu-latest
outputs:
shard-count: ${{ steps.calc.outputs.count }}
shard-matrix: ${{ steps.calc.outputs.matrix }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- id: calc
run: |
TEST_COUNT=$(npx playwright test --list --reporter=json 2>/dev/null | node -e "
const data = require('fs').readFileSync('/dev/stdin', 'utf8');
const parsed = JSON.parse(data);
console.log(parsed.suites?.reduce((acc, s) => acc + (s.specs?.length || 0), 0) || 0);
")
# 1 shard per 20 tests, minimum 1, maximum 8
SHARDS=$(( (TEST_COUNT + 19) / 20 ))
SHARDS=$(( SHARDS > 8 ? 8 : SHARDS ))
SHARDS=$(( SHARDS < 1 ? 1 : SHARDS ))
# Build matrix array: ["1/N", "2/N", ...]
MATRIX="["
for i in $(seq 1 $SHARDS); do
[ $i -gt 1 ] && MATRIX+=","
MATRIX+="\"$i/$SHARDS\""
done
MATRIX+="]"
echo "count=$SHARDS" >> $GITHUB_OUTPUT
echo "matrix=$MATRIX" >> $GITHUB_OUTPUT
test:
needs: determine-shards
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: ${{ fromJson(needs.determine-shards.outputs.shard-matrix) }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npx playwright install --with-deps
- run: npx playwright test --shard=${{ matrix.shard }}Decision Guide
| Scenario | Workers | Shards | Why |
|---|---|---|---|
| < 50 tests, < 5 min | Auto (default) | None | No optimization needed |
| 50-200 tests, 5-15 min | '50%' in CI | 2-4 shards | Balance speed and CI cost |
| 200+ tests, > 15 min | '50%' in CI | 4-8 shards | Keep feedback under 10 min |
| Flaky tests due to resource contention | Reduce: workers: 2 | Keep current | Fewer workers = less CPU/memory pressure |
| Tests modify shared database | workers: 1 or isolate per worker | Still useful | Sharding splits files; workers run them |
| CI has limited CPU/RAM | workers: 1 or '25%' | More shards | Compensate fewer workers with more machines |
| Question | workers (in-process) | --shard (across machines) |
|---|---|---|
| What does it split? | Tests across CPU cores on one machine | Test files across separate CI jobs |
| Controlled by? | playwright.config.ts or --workers CLI | --shard=X/Y CLI flag |
| Shares memory? | Yes (same machine) | No (separate machines) |
| Report merging needed? | No (single process) | Yes (merge-reports) |
| Cost | Free (same machine) | More CI minutes (more machines) |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
fullyParallel: false with no reason | Tests within files run serially; slow suite | Set fullyParallel: true unless specific tests need serial |
workers: 1 in CI "to be safe" | Negates parallelism entirely | Fix isolation issues; use workers: '50%' |
| Tests sharing a hardcoded user account | Race conditions when parallel -- both tests modify same data | Each test creates unique data via API or fixture |
--shard=1/4 without blob reporter | Each shard produces its own HTML report; no merged view | Configure reporter: [['blob']] for sharded CI runs |
| Sharding with 3 tests | Overhead of shard setup exceeds time saved | Only shard when suite exceeds 5 minutes |
test.describe.serial() everywhere | Kills parallelism, creates hidden dependencies | Use only when tests genuinely depend on prior state |
| Worker count higher than CPU cores | Context switching overhead; slower, not faster | Use '50%' or let Playwright auto-detect |
Not using fail-fast: false in CI matrix | One shard failure cancels others; incomplete results | Always set fail-fast: false for sharded strategies |
Troubleshooting
Tests pass alone but fail when run together
Cause: Shared state between tests -- database rows, cookies, global variables, file system.
Fix: Isolate each test. Use unique data per test:
test('create order', async ({ page, request }, testInfo) => {
// Unique product per test -- no collision with parallel tests
const product = await request.post('/api/test/products', {
data: { name: `Widget-${testInfo.workerIndex}-${Date.now()}` },
});
// ...
});Shard produces no tests: "No tests found"
Cause: Shard count exceeds the number of test files. A shard gets zero files.
Fix: Reduce shard count to at most the number of test files:
# If you have 10 test files, max 10 shards
npx playwright test --shard=1/10 # OK
npx playwright test --shard=1/20 # Some shards will be emptyMerged report missing some test results
Cause: Blob report files from a shard were not downloaded or were overwritten due to name collision.
Fix: Give each shard's artifact a unique name:
# Each shard
- uses: actions/upload-artifact@v4
with:
name: blob-report-${{ strategy.job-index }} # unique per shard
path: blob-report/
# Merge step
- uses: actions/download-artifact@v4
with:
pattern: blob-report-*
merge-multiple: true
path: all-blob-reportsWorker-scoped fixture not shared -- recreated per test
Cause: Missing { scope: 'worker' } option, or the fixture depends on a test-scoped fixture.
Fix: Ensure the fixture uses worker scope and only depends on worker-scoped fixtures:
export const test = base.extend<{}, { sharedResource: Resource }>({
sharedResource: [async ({}, use) => {
const resource = await Resource.create();
await use(resource);
await resource.destroy();
}, { scope: 'worker' }], // Don't forget this
});Tests are slower with more workers
Cause: Machine is CPU- or memory-bound. More workers cause thrashing.
Fix: Reduce workers until you find the sweet spot:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
workers: process.env.CI ? 2 : undefined, // Start low, increase if stable
});Related
- ci/ci-github-actions.md -- sharded GitHub Actions workflow
- ci/ci-gitlab.md -- GitLab
parallel:keyword with sharding - ci/ci-other.md -- sharding on CircleCI, Azure DevOps, Jenkins
- ci/reporting-and-artifacts.md -- blob reporter and merge-reports
- core/fixtures-and-hooks.md -- worker-scoped fixtures
- core/test-organization.md -- parallel vs serial execution
Projects and Dependencies
When to use: Running tests across multiple browsers, devices, or environments from a single config. Projects let you define different test configurations that can depend on each other, share setup work, and run selectively.
Quick Reference
# Run all projects
npx playwright test
# Run a specific project
npx playwright test --project=chromium
npx playwright test --project="Mobile Safari"
# Run multiple projects
npx playwright test --project=chromium --project=firefox
# List all projects and their tests
npx playwright test --list
# Skip dependencies (e.g., skip setup during debugging)
npx playwright test --project=chromium --no-depsPatterns
Pattern 1: Multi-Browser Testing
Use when: Ensuring your application works across Chromium, Firefox, and WebKit. Avoid when: Early development -- start with Chromium only and add browsers later.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
fullyParallel: true,
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
],
});Selective browser testing in CI (fast on PRs, thorough on main):
# .github/workflows/ci.yml
jobs:
test-pr:
if: github.event_name == 'pull_request'
steps:
# Chromium only on PRs for fast feedback
- run: npx playwright test --project=chromium
test-main:
if: github.ref == 'refs/heads/main'
steps:
# All browsers on main branch
- run: npx playwright testPattern 2: Desktop and Mobile Projects
Use when: Testing responsive layouts, touch interactions, or mobile-specific behavior. Avoid when: Your application is desktop-only.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
fullyParallel: true,
projects: [
// Desktop browsers
{
name: 'Desktop Chrome',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'Desktop Firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'Desktop Safari',
use: { ...devices['Desktop Safari'] },
},
// Mobile devices
{
name: 'Mobile Chrome',
use: { ...devices['Pixel 7'] },
},
{
name: 'Mobile Safari',
use: { ...devices['iPhone 14'] },
},
// Tablet
{
name: 'iPad',
use: { ...devices['iPad Pro 11'] },
},
],
});Run only mobile or only desktop:
npx playwright test --project="Mobile Chrome" --project="Mobile Safari"
npx playwright test --project="Desktop Chrome" --project="Desktop Firefox"Pattern 3: Setup Project with Dependencies
Use when: Tests need shared state (authentication, seeded data) that should be created once before all test projects run. Avoid when: Tests are fully independent with no shared setup phase.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
projects: [
// Setup runs first -- no dependencies, so it runs immediately
{
name: 'setup',
testMatch: /global\.setup\.ts/,
},
// Browser projects depend on setup
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
{
name: 'firefox',
use: {
...devices['Desktop Firefox'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
],
});// tests/global.setup.ts
import { test as setup, expect } from '@playwright/test';
const authFile = 'playwright/.auth/user.json';
setup('authenticate', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
await page.context().storageState({ path: authFile });
});How dependencies work: 1. Playwright identifies projects with no dependencies and runs them first. 2. Once a dependency project completes successfully, dependent projects start. 3. If a dependency project fails, all dependent projects are skipped.
Pattern 4: Multiple Auth Roles
Use when: Tests need different user roles (admin, editor, viewer) with separate auth states. Avoid when: All tests use the same user -- use a single setup project.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
projects: [
// Auth setup for each role
{
name: 'auth-admin',
testMatch: /auth\.setup\.ts/,
use: {
userRole: 'admin',
storageStatePath: 'playwright/.auth/admin.json',
},
},
{
name: 'auth-editor',
testMatch: /auth\.setup\.ts/,
use: {
userRole: 'editor',
storageStatePath: 'playwright/.auth/editor.json',
},
},
// Admin tests
{
name: 'admin-tests',
testDir: './tests/admin',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/admin.json',
},
dependencies: ['auth-admin'],
},
// Editor tests
{
name: 'editor-tests',
testDir: './tests/editor',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/editor.json',
},
dependencies: ['auth-editor'],
},
// Unauthenticated tests (no dependencies, no storageState)
{
name: 'public-tests',
testDir: './tests/public',
use: { ...devices['Desktop Chrome'] },
},
],
});// tests/auth.setup.ts
import { test as setup, expect } from '@playwright/test';
const credentials: Record<string, { email: string; password: string }> = {
admin: { email: 'admin@example.com', password: process.env.ADMIN_PASSWORD! },
editor: { email: 'editor@example.com', password: process.env.EDITOR_PASSWORD! },
};
setup('authenticate', async ({ page }, testInfo) => {
const role = testInfo.project.use.userRole as string;
const authFile = testInfo.project.use.storageStatePath as string;
const { email, password } = credentials[role];
await page.goto('/login');
await page.getByLabel('Email').fill(email);
await page.getByLabel('Password').fill(password);
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
await page.context().storageState({ path: authFile });
});Pattern 5: Environment-Specific Projects
Use when: Running the same tests against different environments (dev, staging, production) from one config. Avoid when: You only test one environment.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
const ENV = process.env.TEST_ENV || 'local';
const envConfig: Record<string, { baseURL: string; retries: number }> = {
local: { baseURL: 'http://localhost:3000', retries: 0 },
staging: { baseURL: 'https://staging.example.com', retries: 2 },
production: { baseURL: 'https://www.example.com', retries: 2 },
};
const env = envConfig[ENV];
export default defineConfig({
testDir: './tests',
retries: env.retries,
use: {
baseURL: env.baseURL,
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
// Only run smoke tests in production
...(ENV === 'production'
? [{
name: 'smoke',
testMatch: '**/*smoke*.spec.ts',
use: { ...devices['Desktop Chrome'] },
grep: /@smoke/,
}]
: []),
],
});# Run against different environments
TEST_ENV=local npx playwright test
TEST_ENV=staging npx playwright test
TEST_ENV=production npx playwright test --project=smokePattern 6: Project with Custom testDir and testMatch
Use when: Different projects need different test directories or file patterns. Avoid when: All projects run the same tests.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
projects: [
// E2E tests -- run in all browsers
{
name: 'e2e-chromium',
testDir: './tests/e2e',
testMatch: '**/*.spec.ts',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'e2e-firefox',
testDir: './tests/e2e',
testMatch: '**/*.spec.ts',
use: { ...devices['Desktop Firefox'] },
},
// API tests -- no browser needed, run once
{
name: 'api',
testDir: './tests/api',
testMatch: '**/*.spec.ts',
use: {
baseURL: 'https://api.example.com',
},
},
// Visual regression -- Chromium only
{
name: 'visual',
testDir: './tests/visual',
testMatch: '**/*.spec.ts',
use: {
...devices['Desktop Chrome'],
// Lock viewport for consistent screenshots
viewport: { width: 1280, height: 720 },
},
},
],
});# Run only API tests
npx playwright test --project=api
# Run only visual tests
npx playwright test --project=visual
# Run all E2E tests
npx playwright test --project=e2e-chromium --project=e2e-firefoxPattern 7: Teardown Projects
Use when: You need browser-based cleanup after tests complete (e.g., deleting test data via the UI). Avoid when: Cleanup can be done via API in globalTeardown or fixtures (the usual case).
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
projects: [
{
name: 'setup',
testMatch: /global\.setup\.ts/,
teardown: 'teardown', // link to teardown project
},
{
name: 'teardown',
testMatch: /global\.teardown\.ts/,
},
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
],
});// tests/global.teardown.ts
import { test as teardown } from '@playwright/test';
teardown('clean up test data', async ({ request }) => {
await request.post('/api/test/cleanup', {
headers: { Authorization: `Bearer ${process.env.ADMIN_API_KEY}` },
});
});Execution order: 1. setup project runs 2. chromium project runs (depends on setup) 3. teardown project runs (linked via setup's teardown field)
Pattern 8: Grep and GrepInvert for Project Filtering
Use when: Different projects should run different subsets of tests based on tags. Avoid when: All projects run all tests.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
projects: [
// Smoke tests -- only @smoke tagged tests, Chromium only
{
name: 'smoke',
use: { ...devices['Desktop Chrome'] },
grep: /@smoke/,
},
// Full regression -- everything except @slow, all browsers
{
name: 'regression-chromium',
use: { ...devices['Desktop Chrome'] },
grepInvert: /@slow/,
},
{
name: 'regression-firefox',
use: { ...devices['Desktop Firefox'] },
grepInvert: /@slow/,
},
// Slow tests -- only @slow, Chromium only, higher timeout
{
name: 'slow',
use: { ...devices['Desktop Chrome'] },
grep: /@slow/,
timeout: 120_000,
},
],
});# CI: run smoke on PRs
npx playwright test --project=smoke
# CI: run full regression on main
npx playwright test --project=regression-chromium --project=regression-firefox
# CI: run slow tests nightly
npx playwright test --project=slowDecision Guide
| Scenario | Number of Projects | Configuration |
|---|---|---|
| Getting started | 1 (chromium) | Single project, no dependencies |
| Cross-browser testing | 3 (chromium, firefox, webkit) | One per browser engine |
| Responsive design | 5-6 (desktop + mobile + tablet) | Use devices presets |
| Authenticated tests | 1 setup + N browser projects | Setup with dependencies |
| Multiple auth roles | N setup + N test projects | One setup project per role |
| Different test types (E2E, API, visual) | One per type | Custom testDir per project |
| Environment targeting | Same projects, different baseURL | Use TEST_ENV env var |
| Smoke vs regression suites | Projects with grep / grepInvert | Tag-based filtering |
| Feature | dependencies | teardown | globalSetup |
|---|---|---|---|
| Has browser context | Yes | Yes | No |
| Runs when | Before dependent projects | After linked setup project's dependents | Before all projects |
| Use for | Auth state, data seeding with browser | Browser-based cleanup | Non-browser setup (DB, health check) |
| Skips dependents on failure | Yes | N/A | Yes (entire suite fails) |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
| All browsers on every PR | 3x CI time for marginal benefit | Chromium on PRs; all browsers on main |
dependencies on projects that don't need shared state | Forced serial execution; slower | Only use dependencies when projects truly need shared state |
| Duplicating config across projects | Hard to maintain; settings drift | Use shared use at the top level; override per project |
No --project filtering in CI | All projects always run; no control | Use --project to run subsets based on context |
| Setup project modifies database without teardown | Dirty state for next run | Always pair setup with teardown or make setup idempotent |
| Many projects with overlapping test directories | Same test runs multiple times unintentionally | Set explicit testDir and testMatch per project |
Not using --no-deps for debugging | Setup runs every time, even for one test | Use --no-deps to skip dependencies during focused debugging |
Troubleshooting
"No tests found" when running specific project
Cause: The project's testDir or testMatch doesn't match any files.
Fix: List tests for the project to see what matches:
npx playwright test --project=chromium --listCheck that testDir and testMatch are correct:
{
name: 'chromium',
testDir: './tests', // must contain test files
testMatch: '**/*.spec.ts', // must match your file naming
}Setup project runs every time, even for a single test
Cause: The test's project has dependencies: ['setup'], so setup always runs first.
Fix: Use --no-deps to skip dependencies during development:
npx playwright test --project=chromium --no-deps tests/specific-test.spec.tsstorageState file not found
Cause: Setup project failed or didn't create the file. Or the path in use.storageState doesn't match the path in the setup project.
Fix: Verify paths match exactly:
// In setup test:
await page.context().storageState({ path: 'playwright/.auth/user.json' });
// In project config:
use: { storageState: 'playwright/.auth/user.json' } // must match exactlyAdd .auth to .gitignore:
echo "playwright/.auth/" >> .gitignoreDependent projects run even when setup fails
Cause: This should not happen -- Playwright skips dependent projects when a dependency fails. If it seems like they run, check that dependencies is spelled correctly.
Fix: Verify the dependency name matches exactly:
// Setup project name
{ name: 'setup', ... }
// Dependency reference (must match exactly, case-sensitive)
{ dependencies: ['setup'] } // correct
{ dependencies: ['Setup'] } // WRONG -- case mismatchTests from multiple projects interfere with each other
Cause: Projects share a database or external state. Since projects run in parallel by default, they can collide.
Fix: Either: 1. Use per-project isolated data (different test users, different database schemas) 2. Run projects sequentially by adding artificial dependencies:
projects: [
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
dependencies: ['chromium'], // forces serial execution
},
],Custom use properties cause TypeScript errors
Cause: Playwright's use type doesn't include your custom properties.
Fix: Extend the type or use as any:
// Option 1: Declare custom properties
declare module '@playwright/test' {
interface PlaywrightTestOptions {
userRole: string;
storageStatePath: string;
}
}
// Option 2: Quick fix (less type-safe)
{
name: 'auth-admin',
use: {
userRole: 'admin',
storageStatePath: 'playwright/.auth/admin.json',
} as any,
}Related
- core/configuration.md -- base config,
projects,usesettings - ci/global-setup-teardown.md --
globalSetupvs setup projects - core/fixtures-and-hooks.md -- option fixtures configured per project
- core/authentication.md -- auth state via setup projects
- ci/parallel-and-sharding.md -- sharding across projects
- ci/ci-github-actions.md -- running specific projects in CI
Reporting and Artifacts
When to use: Configuring test output for local debugging, CI dashboards, and team visibility. Every project needs a reporting strategy from day one.
Quick Reference
# View the last HTML report
npx playwright show-report
# Run with specific reporter
npx playwright test --reporter=html
npx playwright test --reporter=dot # minimal CI output
npx playwright test --reporter=line # one line per test
npx playwright test --reporter=json # machine-readable
npx playwright test --reporter=junit # CI integration
# Multiple reporters via CLI
npx playwright test --reporter=dot,html
# Merge shard reports
npx playwright merge-reports --reporter=html ./blob-reportPatterns
Pattern 1: Multi-Reporter Configuration
Use when: Every project. You always want at least two reporters: one for humans, one for CI. Avoid when: Never -- always configure reporters.
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
// CI: machine-readable + human-readable + CI annotations
['dot'], // minimal console output
['html', { open: 'never' }], // browsable report (uploaded as artifact)
['junit', { outputFile: 'test-results/junit.xml' }], // CI test tab integration
['github'], // PR annotations (GitHub Actions only)
]
: [
// Local: detailed console + auto-opening report
['list'], // verbose console output
['html', { open: 'on-failure' }], // auto-open on failure
],
});Pattern 2: Built-in Reporters in Detail
Use when: Choosing the right reporter for your context.
| Reporter | Output | Best For |
|---|---|---|
list | One line per test with pass/fail | Local development |
line | Updates a single line as tests complete | Local, less verbose |
dot | Single dot per test: . pass, F fail | CI logs (minimal) |
html | Interactive HTML page with traces | Post-run analysis |
json | Machine-readable JSON to stdout or file | Custom tooling, dashboards |
junit | JUnit XML | CI platforms (Azure DevOps, Jenkins, CircleCI) |
github | GitHub Actions annotations | GitHub PRs |
blob | Binary archive for shard merging | Sharded CI runs |
JSON reporter -- write to file:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['json', { outputFile: 'test-results/results.json' }],
],
});JUnit reporter -- customize output:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['junit', {
outputFile: 'test-results/junit.xml',
stripANSIControlSequences: true,
includeProjectInTestName: true,
}],
],
});Pattern 2b: Trace Retention For Flaky Tests (Playwright 1.59+)
Use when: A test sometimes fails and sometimes passes on retry, and you need artifacts from both attempts to compare behavior. Avoid when: Trace size matters more than diagnosis depth. In that case, keep using 'on-first-retry'.
Playwright 1.59 adds a new trace mode, 'retain-on-failure-and-retries', which records each run and keeps all traces when any attempt fails.
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
trace: process.env.CI
? 'retain-on-failure-and-retries'
: 'on-first-retry',
},
});This is excellent for flaky-test analysis because you can compare the failing run against a passing retry instead of guessing what changed between attempts.
Pattern 2c: Scoped HAR Artifacts For a Flow (Playwright 1.60+)
Use when: You want a network archive (HAR) attached to the report for one specific flow — e.g. a flaky third-party integration — without recording HAR for the entire context. Avoid when: A trace already gives you the network waterfall you need (traces include network detail). Reach for a dedicated HAR only when you need a portable .har to replay or hand to another tool.
Playwright 1.60 adds tracing.startHar() / tracing.stopHar(), which record HAR on demand inside the test. Capture only the suspect flow and attach it to the report:
import { test } from '@playwright/test';
test('payment integration', async ({ context, page }, testInfo) => {
const harPath = testInfo.outputPath('payment.har');
await context.tracing.startHar({ path: harPath, urlFilter: '**/payments/**' });
await page.goto('/checkout');
await page.getByRole('button', { name: 'Pay' }).click();
await context.tracing.stopHar();
await testInfo.attach('payment-network', { path: harPath, contentType: 'application/json' });
});The attached HAR shows up alongside traces and screenshots in the HTML report and CI artifacts. See core/network-mocking.md for the full API and await using cleanup.
Pattern 3: Custom Reporter
Use when: Built-in reporters don't meet your needs -- you want Slack notifications, database logging, or custom dashboards. Avoid when: A built-in reporter or existing third-party reporter covers your case.
// reporters/slack-reporter.ts
import type {
FullConfig,
FullResult,
Reporter,
Suite,
TestCase,
TestResult,
} from '@playwright/test/reporter';
class SlackReporter implements Reporter {
private passed = 0;
private failed = 0;
private skipped = 0;
private failures: string[] = [];
onTestEnd(test: TestCase, result: TestResult) {
switch (result.status) {
case 'passed':
this.passed++;
break;
case 'failed':
case 'timedOut':
this.failed++;
this.failures.push(`${test.title}: ${result.error?.message?.split('\n')[0]}`);
break;
case 'skipped':
this.skipped++;
break;
}
}
async onEnd(result: FullResult) {
const total = this.passed + this.failed + this.skipped;
const emoji = this.failed > 0 ? ':red_circle:' : ':large_green_circle:';
const text = [
`${emoji} *Playwright Tests*: ${result.status}`,
`Passed: ${this.passed} | Failed: ${this.failed} | Skipped: ${this.skipped} | Total: ${total}`,
`Duration: ${(result.duration / 1000).toFixed(1)}s`,
];
if (this.failures.length > 0) {
text.push('', '*Failures:*');
this.failures.slice(0, 5).forEach((f) => text.push(` - ${f}`));
if (this.failures.length > 5) {
text.push(` ...and ${this.failures.length - 5} more`);
}
}
const webhookUrl = process.env.SLACK_WEBHOOK_URL;
if (webhookUrl) {
await fetch(webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ text: text.join('\n') }),
});
}
}
}
export default SlackReporter;Register the custom reporter:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['dot'],
['html', { open: 'never' }],
['./reporters/slack-reporter.ts'],
],
});Pattern 4: Trace File Management
Use when: Debugging test failures. Traces capture a complete timeline of actions, network requests, DOM snapshots, and console logs. Avoid when: Never disable traces entirely in CI -- the on-first-retry setting has minimal overhead.
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
// 'on-first-retry': records trace only when a test fails and retries.
// Minimal overhead on passing tests, full debugging on failures.
trace: 'on-first-retry',
},
});Trace options:
| Value | Records trace | When | Overhead |
|---|---|---|---|
'off' | Never | -- | None |
'on' | Every test | Always | High (large files) |
'on-first-retry' | On first retry after failure | Retries only | Minimal |
'retain-on-failure' | Every test, keeps only failures | Failures | Medium |
'retain-on-first-failure' | Every test, keeps only first failure | First failure | Medium |
Viewing traces:
# Open trace viewer locally
npx playwright show-trace test-results/my-test/trace.zip
# Open trace from HTML report (click "Traces" tab in the report)
npx playwright show-report
# Online trace viewer (upload trace.zip)
# https://trace.playwright.devPattern 5: Screenshot and Video Configuration
Use when: Visual evidence of test failures is valuable for debugging or bug reports. Avoid when: Never disable screenshots in CI -- the on-failure setting is cheap.
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
// Screenshots
screenshot: 'only-on-failure', // capture final state on failure
// Video
video: 'retain-on-failure', // record all, keep only failures
// Video size (optional -- smaller = less disk)
video: {
mode: 'retain-on-failure',
size: { width: 1280, height: 720 },
},
},
});Screenshot options:
| Value | Captures | Disk cost |
|---|---|---|
'off' | Never | None |
'on' | Every test (at end) | High |
'only-on-failure' | Failed tests only | Low |
Video options:
| Value | Records | Keeps | Disk cost |
|---|---|---|---|
'off' | Never | -- | None |
'on' | Every test | All | Very high |
'on-first-retry' | On retry | Retried tests | Low |
'retain-on-failure' | Every test | Failed only | Medium |
Pattern 6: Artifact Organization for CI
Use when: Keeping test artifacts organized and accessible in CI.
Recommended directory structure:
test-results/ # Playwright's default output directory
├── my-test-chromium/
│ ├── trace.zip # Trace file
│ ├── test-failed-1.png # Screenshot
│ └── video.webm # Video recording
├── another-test-firefox/
│ ├── trace.zip
│ └── test-failed-1.png
└── junit.xml # JUnit report (if configured)
playwright-report/ # HTML report directory
├── index.html
└── data/
└── ...
blob-report/ # Blob report for shard merging
└── report-1.zipGitHub Actions artifact upload:
# Upload HTML report (always -- useful even when tests pass)
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 14
# Upload traces and screenshots (only on failure -- saves storage)
- uses: actions/upload-artifact@v4
if: failure()
with:
name: test-traces
path: |
test-results/**/trace.zip
test-results/**/*.png
test-results/**/*.webm
retention-days: 7Pattern 7: Allure Integration
Use when: Your team uses Allure for test reporting across multiple test frameworks. Avoid when: The built-in HTML reporter meets your needs (it usually does).
# Install Allure reporter
npm install -D allure-playwright// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['line'],
['allure-playwright', {
detail: true,
outputFolder: 'allure-results',
suiteTitle: true,
}],
],
});# Generate and view Allure report
npx allure generate allure-results -o allure-report --clean
npx allure open allure-report
# Or use Allure CLI
allure serve allure-resultsAdd Allure metadata to tests:
import { test, expect } from '@playwright/test';
import { allure } from 'allure-playwright';
test('checkout flow', async ({ page }) => {
await allure.epic('E-Commerce');
await allure.feature('Checkout');
await allure.story('Credit Card Payment');
await allure.severity('critical');
await page.goto('/checkout');
// ... test implementation
});Decision Guide
| Scenario | Reporter Configuration | Why |
|---|---|---|
| Local development | [['list'], ['html', { open: 'on-failure' }]] | Verbose console + auto-opening report on failure |
| GitHub Actions | [['dot'], ['html'], ['github']] | Minimal logs + report artifact + PR annotations |
| GitLab CI | [['dot'], ['html'], ['junit']] | Minimal logs + report artifact + test tab |
| Azure DevOps / Jenkins | [['dot'], ['html'], ['junit']] | JUnit for native test results integration |
| Sharded CI | [['blob'], ['github']] | Blob for merging; github for PR annotations |
| Team uses Allure | [['line'], ['allure-playwright']] | Cross-framework reporting consistency |
| Custom dashboard | [['json', { outputFile: '...' }]] + custom reporter | JSON for data, custom for notifications |
| Artifact | When to Collect | Retention | Upload Condition |
|---|---|---|---|
| HTML report | Always | 14 days | if: ${{ !cancelled() }} |
Traces (.zip) | On failure | 7 days | if: failure() |
Screenshots (.png) | On failure | 7 days | if: failure() |
Videos (.webm) | On failure | 7 days | if: failure() |
| JUnit XML | Always | 14 days | if: ${{ !cancelled() }} |
| Blob report | Always (sharded) | 1 day | if: ${{ !cancelled() }} |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
| No reporter configured | Default list only; no persistent report | Always configure html + one CI reporter |
trace: 'on' in CI | Massive artifacts (50-100 MB per test), slow uploads | Use trace: 'on-first-retry' |
video: 'on' in CI | Enormous storage cost; slows test execution | Use video: 'retain-on-failure' |
| Only uploading artifacts on failure | No report when tests pass; can't verify results | Upload with if: ${{ !cancelled() }} (always) |
| No retention limits on artifacts | CI storage fills up within weeks | Set retention-days: 7-14 |
Using only dot reporter with no HTML | Can't drill into failures after the run | Always pair dot with html in CI |
| JUnit output to stdout | Interferes with console output; hard to parse | Write to file: ['junit', { outputFile: 'results/junit.xml' }] |
Custom reporter that blocks onEnd | Slow Slack/HTTP calls delay pipeline completion | Use Promise.race with a timeout in custom reporters |
Troubleshooting
HTML report is empty or missing tests
Cause: Another reporter is conflicting, or outputFolder was overridden to a non-default path.
Fix: Check your reporter config. The HTML report defaults to playwright-report/:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [['html', { outputFolder: 'playwright-report', open: 'never' }]],
});Traces are too large for CI artifact upload
Cause: trace: 'on' records every test, even passing ones.
Fix: Switch to 'on-first-retry' and ensure retries > 0 in CI:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
trace: 'on-first-retry',
},
});JUnit XML not recognized by CI platform
Cause: Output path doesn't match what the CI task expects, or the file is empty.
Fix: Ensure the path matches your CI configuration:
// playwright.config.ts -- the outputFile path
reporter: [['junit', { outputFile: 'test-results/junit.xml' }]],# GitHub Actions
- uses: dorny/test-reporter@v1
with:
path: test-results/junit.xml
reporter: java-junit
# Azure DevOps
- task: PublishTestResults@2
inputs:
testResultsFiles: 'test-results/junit.xml'
# Jenkins
junit 'test-results/junit.xml'merge-reports produces empty report
Cause: Shards are using html reporter instead of blob. Only blob output can be merged.
Fix: Use blob reporter for sharded runs:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['dot']] // blob for merge, dot for console
: [['html', { open: 'on-failure' }]],
});Screenshots not appearing in HTML report
Cause: screenshot: 'off' or screenshots are in test-results/ but not linked to the report.
Fix: Enable screenshots and ensure both directories are available:
use: {
screenshot: 'only-on-failure',
},The HTML report automatically embeds screenshots from test-results/. If you move or delete test-results/, screenshots will be missing from the report.
Related
- ci/ci-github-actions.md -- artifact upload in GitHub Actions
- ci/ci-gitlab.md -- artifact configuration in GitLab
- ci/parallel-and-sharding.md -- blob reporter for sharded runs
- core/configuration.md -- trace, screenshot, video settings
- core/debugging.md -- using traces and screenshots for debugging
MIT License
Copyright (c) 2026 TestDino
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Related skills
How it compares
Pick playwright-skill for opinionated Playwright pattern libraries; use generic testing skills for non-browser unit test frameworks.
FAQ
Who is playwright-skill for?
Developers using agents to execute playwright skill workflows from SKILL.md.
When should I use playwright-skill?
Battle-tested Playwright patterns for writing, debugging, and scaling reliable test suites. Use when you need guidance for E2E, API, component, visual, accessibility, or security t
Is playwright-skill safe to install?
Review the Security Audits panel on this page before installing in production.