
Playwright Ci
- 166 installs
- 343 repo stars
- Updated July 2, 2026
- testdino-hq/playwright-skill
Use playwright-ci for development tasks
About
playwright-ci: A skill for development. This provides functionality for development workflows.
- playwright-ci
Playwright Ci by the numbers
- 166 all-time installs (skills.sh)
- +7 installs in the week ending Jul 27, 2026 (Skillselion tracking)
- Ranked #2,291 of 4,347 Backend & APIs skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/testdino-hq/playwright-skill --skill playwright-ciAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 166 |
|---|---|
| repo stars | ★ 343 |
| Last updated | July 2, 2026 |
| Repository | testdino-hq/playwright-skill ↗ |
What it does
Use playwright-ci for development tasks
Files
Playwright CI/CD
Ship reliable tests in every pipeline — CI-specific patterns for speed, stability, and actionable reports.
9 guides covering CI/CD setup, parallel execution, containerized runs, reporting, and infrastructure patterns for all major CI providers.
Golden Rules
1. `retries: 2` in CI only — surface flakiness in pipelines, not locally 2. `traces: 'on-first-retry'` — capture rich debugging artifacts without slowing every run 3. Shard across runners — --shard=N/M splits tests evenly; scale horizontally, not vertically 4. Cache browser binaries — ~/.cache/ms-playwright keyed on Playwright version 5. Upload artifacts on failure — traces, screenshots, and HTML reports as CI artifacts 6. Use the official Docker image — mcr.microsoft.com/playwright:v* has all OS deps pre-installed 7. Global setup for auth — run login once in globalSetup, reuse storageState across workers 8. Fail fast, debug later — keep CI runs short; use trace viewer and HTML reports to investigate
Guide Index
CI Providers
| Provider | Guide |
|---|---|
| GitHub Actions | ci-github-actions.md |
| GitLab CI | ci-gitlab.md |
| CircleCI / Azure DevOps / Jenkins | ci-other.md |
Execution & Scaling
| Topic | Guide |
|---|---|
| Parallel execution & sharding | parallel-and-sharding.md |
| Docker & containers | docker-and-containers.md |
| Multi-project config | projects-and-dependencies.md |
Reporting & Setup
| Topic | Guide |
|---|---|
| Reports & artifacts | reporting-and-artifacts.md |
| Code coverage | test-coverage.md |
| Global setup/teardown | global-setup-teardown.md |
CI: GitHub Actions
When to use: Running Playwright tests automatically on pull requests, merges to main, or on a schedule. GitHub Actions is the most common CI for Playwright projects.
Quick Reference
# Key CLI flags for CI
npx playwright install --with-deps # install browsers + OS deps
npx playwright test --shard=1/4 # run 1 of 4 shards
npx playwright test --reporter=github # annotate PR with failures
npx playwright merge-reports ./blob-report # merge shard reportsPatterns
Pattern 1: Production-Ready Workflow (Copy-Paste Starter)
Use when: Any project using GitHub Actions. This is the complete, battle-tested workflow.
# .github/workflows/playwright.yml
name: Playwright Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
# Cancel in-progress runs for the same PR/branch
concurrency:
group: playwright-${{ github.ref }}
cancel-in-progress: true
env:
CI: true
jobs:
test:
timeout-minutes: 30
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
- name: Run Playwright tests
run: npx playwright test
- name: Upload HTML report
uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 14
- name: Upload test traces
uses: actions/upload-artifact@v4
if: failure()
with:
name: playwright-traces
path: test-results/
retention-days: 7Pattern 2: Sharded Execution with Matrix Strategy
Use when: Test suite takes more than 10 minutes. Split across parallel runners to cut wall-clock time. Avoid when: Suite runs under 5 minutes -- sharding overhead (checkout, install, merge) negates the benefit.
# .github/workflows/playwright-sharded.yml
name: Playwright Tests (Sharded)
on:
push:
branches: [main]
pull_request:
branches: [main]
concurrency:
group: playwright-${{ github.ref }}
cancel-in-progress: true
env:
CI: true
jobs:
test:
timeout-minutes: 20
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1/4, 2/4, 3/4, 4/4]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
- name: Run Playwright tests (shard ${{ matrix.shard }})
run: npx playwright test --shard=${{ matrix.shard }}
- name: Upload blob report
uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: blob-report-${{ strategy.job-index }}
path: blob-report/
retention-days: 1
merge-reports:
if: ${{ !cancelled() }}
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Download all blob reports
uses: actions/download-artifact@v4
with:
path: all-blob-reports
pattern: blob-report-*
merge-multiple: true
- name: Merge reports
run: npx playwright merge-reports --reporter=html ./all-blob-reports
- name: Upload merged HTML report
uses: actions/upload-artifact@v4
with:
name: playwright-report
path: playwright-report/
retention-days: 14Config for sharding -- add blob reporter so shard output can be merged:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['github']]
: [['html', { open: 'on-failure' }]],
});Pattern 3: Reusable Workflow
Use when: Multiple repositories or multiple workflow files need the same Playwright setup. Avoid when: Single repo with one workflow.
# .github/workflows/playwright-reusable.yml
name: Playwright Reusable
on:
workflow_call:
inputs:
node-version:
type: string
default: '20'
test-command:
type: string
default: 'npx playwright test'
shard-total:
type: number
default: 1
secrets:
BASE_URL:
required: false
TEST_PASSWORD:
required: false
jobs:
test:
timeout-minutes: 30
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: ${{ fromJson(format('[{0}]', join(fromJson(format('[{0}]', inputs.shard-total == 1 && '"1/1"' || '"1/4","2/4","3/4","4/4"')), ','))) }}
env:
CI: true
BASE_URL: ${{ secrets.BASE_URL }}
TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ inputs.node-version }}
cache: 'npm'
- run: npm ci
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
- name: Run tests
run: ${{ inputs.test-command }} --shard=${{ matrix.shard }}
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report-${{ strategy.job-index }}
path: playwright-report/
retention-days: 14Calling the reusable workflow:
# .github/workflows/ci.yml
name: CI
on:
pull_request:
branches: [main]
jobs:
e2e:
uses: ./.github/workflows/playwright-reusable.yml
with:
node-version: '20'
shard-total: 4
secrets:
BASE_URL: ${{ secrets.STAGING_URL }}
TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}Pattern 4: Running in a Container
Use when: You need a reproducible environment identical to local Docker runs, or the runner's OS dependencies cause issues. Avoid when: Standard ubuntu-latest with --with-deps works fine (the common case).
# .github/workflows/playwright-container.yml
name: Playwright (Container)
on:
pull_request:
branches: [main]
jobs:
test:
timeout-minutes: 30
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.52.0-noble
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
# No browser install needed -- they're in the container image
- name: Run Playwright tests
run: npx playwright test
env:
HOME: /root # required when running as root in container
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 14Pattern 5: Environment Secrets and Deployment Targets
Use when: Tests run against staging/production environments that require authentication credentials. Avoid when: Tests only run against a locally started dev server.
# .github/workflows/playwright-staging.yml
name: Playwright (Staging)
on:
push:
branches: [main]
workflow_dispatch:
jobs:
test:
timeout-minutes: 30
runs-on: ubuntu-latest
environment: staging # GitHub Environment with protection rules
env:
CI: true
BASE_URL: ${{ vars.STAGING_URL }}
TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
API_KEY: ${{ secrets.API_KEY }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- name: Cache Playwright browsers
id: playwright-cache
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install Playwright browsers
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: npx playwright install --with-deps
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-deps
- name: Run tests against staging
run: npx playwright test --grep @smoke
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: staging-report
path: playwright-report/
retention-days: 14Pattern 6: Scheduled Runs (Nightly Regression)
Use when: Full regression suite is too slow for every PR. Run it nightly against main. Avoid when: Suite runs in under 15 minutes and can run on every PR.
# .github/workflows/playwright-nightly.yml
name: Nightly Regression
on:
schedule:
- cron: '0 3 * * 1-5' # 3 AM UTC, Mon-Fri
workflow_dispatch: # allow manual trigger
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
env:
CI: true
BASE_URL: ${{ vars.STAGING_URL }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- name: Install Playwright browsers
run: npx playwright install --with-deps
- name: Run full regression suite
run: npx playwright test --grep @regression
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: nightly-report-${{ github.run_number }}
path: playwright-report/
retention-days: 30
- name: Notify on failure
if: failure()
uses: slackapi/slack-github-action@v1.27.0
with:
payload: |
{
"text": "Nightly Playwright regression failed: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}Decision Guide
| Scenario | Approach | Why |
|---|---|---|
| Small suite (< 5 min) | Single job, no sharding | Overhead of sharding exceeds time saved |
| Medium suite (5-20 min) | 2-4 shards with matrix | Cut wall-clock time by ~60-75% |
| Large suite (20+ min) | 4-8 shards + blob report merge | Keep PR feedback under 10 minutes |
| Cross-browser on PRs | Chromium only on PRs; all browsers on main | 3x fewer minutes burned on PRs |
| Staging/prod smoke tests | Separate workflow with environment: | Isolate secrets, add approval gates |
| Nightly full regression | schedule trigger + workflow_dispatch | Full coverage without blocking PRs |
| Multiple repos, same setup | Reusable workflow with workflow_call | DRY; update one file, all repos benefit |
| Reproducible env needed | Container job with Playwright image | Identical to local Docker environment |
Security: Pinning Actions to Commit SHAs
Version tags like actions/checkout@v4 are mutable — the tag can be moved to a different commit without warning, introducing unverified code into your CI pipeline (supply-chain risk W012).
Best practice: pin every action to its full commit SHA.
# Instead of:
- uses: actions/checkout@v4
# Pin to a specific immutable commit SHA:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2How to find the SHA for any action:
# Look up the SHA for a tagged release on GitHub:
# https://github.com/<owner>/<action>/releases
# Click the tag → copy the full commit SHA from the URL or commit details
# Or use the gh CLI:
gh api repos/actions/checkout/git/ref/tags/v4.2.2 --jq '.object.sha'Example pinned workflow step set (verify SHAs at release pages before use):
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
- uses: actions/cache@d4323d4df104b026a6aa633fdb11d772146be0bf # v4.2.2
- uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0
- uses: actions/download-artifact@fa0a91b85d4f404e444306234b4f18a22b3d1e57 # v4.1.8Always verify these SHAs against the official release pages at github.com/actions before adding them to production workflows.
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
No concurrency group | Duplicate runs waste minutes on every push | Add concurrency: { group: ..., cancel-in-progress: true } |
fail-fast: true with sharding | One shard failure cancels others; you lose their results | Set fail-fast: false to collect all failures |
| Installing browsers without caching | 60-90 seconds wasted every run | Cache ~/.cache/ms-playwright keyed on lockfile hash |
timeout-minutes not set | Stuck jobs run for 6 hours (GitHub default) | Set explicit timeout: 20-30 minutes |
| Uploading artifacts only on failure | No report when tests pass; can't verify results | Use if: ${{ !cancelled() }} to always upload |
| Hardcoding secrets in workflow files | Security breach | Use GitHub Secrets and Environments |
| Running all browsers on every PR | 3x CI cost for marginal benefit | Chromium on PR; cross-browser on main merge |
actions/upload-artifact with no retention | Default 90-day retention fills storage | Set retention-days: 7-14 for reports |
No --with-deps on browser install | Missing OS libraries cause browser launch failures | Always use npx playwright install --with-deps |
Using mutable action version tags (@v4) | Tag can be silently re-pointed to a different commit (supply-chain risk) | Pin to full commit SHA; see Security section above |
Troubleshooting
Browser launch fails: "Missing dependencies"
Cause: Browsers installed from cache but OS dependencies were not cached (they live in system directories, not ~/.cache).
Fix: Always run npx playwright install-deps on cache hit:
- name: Install Playwright OS dependencies
if: steps.playwright-cache.outputs.cache-hit == 'true'
run: npx playwright install-depsTests pass locally but fail in CI with timeouts
Cause: CI runners have fewer CPU cores and less RAM than your dev machine. Default workers and timeouts are too aggressive.
Fix: Reduce workers and increase timeouts for CI in your config:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
workers: process.env.CI ? '50%' : undefined,
use: {
actionTimeout: process.env.CI ? 15_000 : 10_000,
navigationTimeout: process.env.CI ? 30_000 : 15_000,
},
});Sharded reports are incomplete -- some shards missing from merged report
Cause: Using actions/download-artifact@v4 without merge-multiple: true, or artifact names collide across shards.
Fix: Give each shard a unique artifact name and use merge-multiple:
# Upload in each shard job
- uses: actions/upload-artifact@v4
with:
name: blob-report-${{ strategy.job-index }}
path: blob-report/
# Download in merge job
- uses: actions/download-artifact@v4
with:
path: all-blob-reports
pattern: blob-report-*
merge-multiple: truewebServer fails in CI: "port 3000 already in use"
Cause: Previous run left a zombie process, or another job step is using the port.
Fix: Ensure reuseExistingServer: false in CI and add a pre-step to kill stale processes:
- name: Kill stale processes
run: lsof -ti:3000 | xargs kill -9 2>/dev/null || trueGitHub annotations not appearing on PR
Cause: The github reporter is not configured.
Fix: Add github reporter for CI runs:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['html', { open: 'never' }], ['github']]
: [['html', { open: 'on-failure' }]],
});Related
- ci/parallel-and-sharding.md -- sharding strategies and blob report merging
- ci/reporting-and-artifacts.md -- reporter configuration and artifact management
- ci/docker-and-containers.md -- container images for CI
- ci/ci-gitlab.md -- GitLab CI equivalent
- ci/ci-other.md -- CircleCI, Azure DevOps, Jenkins
- core/configuration.md -- CI-aware config settings
CI: GitLab CI/CD
When to use: Running Playwright tests in GitLab pipelines on merge requests, merges to main, or scheduled pipelines.
Quick Reference
# Key commands used in GitLab pipelines
npx playwright install --with-deps # install browsers + OS deps
npx playwright test --shard=1/4 # run 1 of 4 parallel shards
npx playwright merge-reports ./blob-report # merge shard results
npx playwright test --reporter=dot # minimal output for CI logsPatterns
Pattern 1: Production-Ready Pipeline (Copy-Paste Starter)
Use when: Any GitLab project with Playwright tests. This is the complete, recommended configuration.
# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.52.0-noble
stages:
- install
- test
- report
variables:
CI: "true"
npm_config_cache: "$CI_PROJECT_DIR/.npm"
# Cache node_modules and npm cache across pipelines
cache:
key:
files:
- package-lock.json
paths:
- .npm/
- node_modules/
install:
stage: install
script:
- npm ci
artifacts:
paths:
- node_modules/
expire_in: 1 hour
test:
stage: test
needs: [install]
script:
- npx playwright test
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHPattern 2: Parallel Sharded Execution
Use when: Test suite exceeds 10 minutes. Use GitLab's parallel keyword to split across jobs automatically. Avoid when: Suite runs under 5 minutes.
# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.52.0-noble
stages:
- install
- test
- report
variables:
CI: "true"
npm_config_cache: "$CI_PROJECT_DIR/.npm"
cache:
key:
files:
- package-lock.json
paths:
- .npm/
- node_modules/
install:
stage: install
script:
- npm ci
artifacts:
paths:
- node_modules/
expire_in: 1 hour
test:
stage: test
needs: [install]
parallel: 4
script:
- npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
artifacts:
when: always
paths:
- blob-report/
expire_in: 1 hour
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
merge-report:
stage: report
needs: [test]
when: always
script:
- npx playwright merge-reports --reporter=html ./blob-report
artifacts:
when: always
paths:
- playwright-report/
expire_in: 14 daysConfig for sharded pipelines:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['dot']]
: [['html', { open: 'on-failure' }]],
});Pattern 3: Merge Request Pipelines with Environment Variables
Use when: Tests need secrets (API keys, passwords) and should only run on merge requests or the default branch. Avoid when: Tests are fully self-contained with no external dependencies.
# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.52.0-noble
stages:
- test
variables:
CI: "true"
test:e2e:
stage: test
variables:
BASE_URL: $STAGING_URL
TEST_PASSWORD: $TEST_PASSWORD
API_KEY: $API_KEY
before_script:
- npm ci
script:
- npx playwright test
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
# Run on merge requests
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# Run on default branch pushes
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
# Allow manual trigger
- when: manual
allow_failure: trueSetting variables in GitLab: Navigate to Settings > CI/CD > Variables and add:
STAGING_URL-- not masked, not protectedTEST_PASSWORD-- masked, protectedAPI_KEY-- masked, protected
Pattern 4: Multi-Browser Testing with Child Pipelines
Use when: Running Chromium on MRs and all browsers on the default branch. Avoid when: You only test one browser.
# .gitlab-ci.yml
image: mcr.microsoft.com/playwright:v1.52.0-noble
stages:
- install
- test
variables:
CI: "true"
install:
stage: install
script:
- npm ci
artifacts:
paths:
- node_modules/
expire_in: 1 hour
# Chromium only on merge requests (fast feedback)
test:chromium:
stage: test
needs: [install]
script:
- npx playwright test --project=chromium
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
# All browsers on default branch
test:all-browsers:
stage: test
needs: [install]
parallel:
matrix:
- PROJECT: [chromium, firefox, webkit]
script:
- npx playwright test --project=$PROJECT
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHPattern 5: Custom Docker Image with Application
Use when: Tests need the application running alongside Playwright, or you need custom system dependencies. Avoid when: The official Playwright image plus webServer in config handles your use case.
# .gitlab-ci.yml
stages:
- test
test:e2e:
stage: test
image: mcr.microsoft.com/playwright:v1.52.0-noble
services:
- name: postgres:16-alpine
alias: db
- name: redis:7-alpine
alias: cache
variables:
CI: "true"
DATABASE_URL: "postgresql://postgres:postgres@db:5432/test"
REDIS_URL: "redis://cache:6379"
POSTGRES_PASSWORD: "postgres"
POSTGRES_DB: "test"
before_script:
- npm ci
- npx prisma db push
- npx prisma db seed
script:
- npx playwright test
artifacts:
when: always
paths:
- playwright-report/
- test-results/
expire_in: 14 days
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHPattern 6: Scheduled Nightly Regression
Use when: Full regression is too slow for every MR. Run it on a schedule.
# .gitlab-ci.yml (add to existing config)
test:nightly:
stage: test
image: mcr.microsoft.com/playwright:v1.52.0-noble
before_script:
- npm ci
script:
- npx playwright test --grep @regression
artifacts:
when: always
paths:
- playwright-report/
expire_in: 30 days
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"Set up the schedule in CI/CD > Schedules: 0 3 * * 1-5 (3 AM UTC, weekdays).
Decision Guide
| Scenario | Approach | Why |
|---|---|---|
| Simple project, < 5 min suite | Single test job using Playwright Docker image | No sharding overhead; artifacts capture report |
| Suite > 10 min | parallel: N with --shard | GitLab auto-assigns CI_NODE_INDEX/CI_NODE_TOTAL |
| Merge request fast feedback | Chromium only on MRs; all browsers on main | 3x fewer pipeline minutes on MRs |
| External services needed (DB, Redis) | services: keyword with Postgres/Redis images | GitLab manages service lifecycle |
| Secrets for staging environment | GitLab CI/CD Variables (masked + protected) | Never hardcode secrets in .gitlab-ci.yml |
| Full nightly regression | Pipeline schedule (CI_PIPELINE_SOURCE == "schedule") | Avoids blocking MR pipelines |
| Report browsing | artifacts: with paths: [playwright-report/] | Browse directly in GitLab job artifacts UI |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
| Not using the Playwright Docker image | Installing browsers every run adds 1-2 minutes | Use mcr.microsoft.com/playwright:v1.52.0-noble as base image |
artifacts: when: on_failure only | No report when tests pass; can't verify results | Use when: always to capture reports regardless |
No expire_in on artifacts | Artifacts accumulate and consume storage | Set expire_in: 14 days for reports, 1 hour for intermediate artifacts |
parallel: without fail-fast: false equivalent | GitLab does not cancel siblings by default (good), but allow_failure: false means the pipeline fails fast | Acceptable default behavior; no change needed |
Hardcoding CI_NODE_TOTAL in shard flag | Breaks when you change parallel: value | Use --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL |
Skipping needs: between stages | Jobs wait for all previous stage jobs, not just their dependencies | Use needs: for precise dependency graphs |
Large cache: including node_modules/ without key | Stale cache causes version conflicts | Key cache on package-lock.json hash |
Troubleshooting
Browser launch fails: "Failed to launch browser"
Cause: Not using the Playwright Docker image, or using a version that doesn't match your @playwright/test version.
Fix: Match the Docker image tag to your Playwright version:
# Check your version
# npm ls @playwright/test -> @playwright/test@1.52.0
image: mcr.microsoft.com/playwright:v1.52.0-nobleTests hang in GitLab runner: "Navigation timeout exceeded"
Cause: GitLab shared runners may have limited resources. Default timeouts too tight.
Fix: Reduce workers and increase timeouts:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
workers: process.env.CI ? 2 : undefined,
use: {
navigationTimeout: process.env.CI ? 30_000 : 15_000,
},
});Pipeline runs on every push, not just merge requests
Cause: Missing rules: configuration. Default GitLab behavior runs on every push.
Fix: Add explicit rules:
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHServices (Postgres/Redis) not reachable from tests
Cause: Using localhost instead of the service alias.
Fix: Use the service alias as hostname:
services:
- name: postgres:16-alpine
alias: db # <-- use "db" as hostname
variables:
DATABASE_URL: "postgresql://postgres:postgres@db:5432/test" # not localhostMerged report is empty after sharded run
Cause: Each shard job needs the blob reporter, not html. The merge step creates the HTML report.
Fix: Configure blob reporter for CI:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['dot']]
: [['html', { open: 'on-failure' }]],
});Related
- ci/ci-github-actions.md -- GitHub Actions equivalent
- ci/ci-other.md -- CircleCI, Azure DevOps, Jenkins
- ci/parallel-and-sharding.md -- sharding strategies
- ci/docker-and-containers.md -- Docker image details
- ci/reporting-and-artifacts.md -- reporter configuration
CI: CircleCI, Azure DevOps, and Jenkins
When to use: Running Playwright tests in CI platforms other than GitHub Actions or GitLab. Each section provides a production-ready config you can copy and adapt.
Quick Reference
# Common across all CI platforms
npx playwright install --with-deps # install browsers + OS deps
npx playwright test --shard=1/4 # shard for parallelism
npx playwright merge-reports ./blob-report # merge shard results
npx playwright test --reporter=dot,html # multiple reportersPatterns
Pattern 1: CircleCI
Use when: Your project runs on CircleCI.
Basic Pipeline
# .circleci/config.yml
version: 2.1
executors:
playwright:
docker:
- image: mcr.microsoft.com/playwright:v1.52.0-noble
working_directory: ~/project
jobs:
install:
executor: playwright
steps:
- checkout
- restore_cache:
keys:
- npm-deps-{{ checksum "package-lock.json" }}
- run: npm ci
- save_cache:
key: npm-deps-{{ checksum "package-lock.json" }}
paths:
- node_modules
- persist_to_workspace:
root: .
paths:
- node_modules
test:
executor: playwright
parallelism: 4
steps:
- checkout
- attach_workspace:
at: .
- run:
name: Run Playwright tests
command: |
npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
- store_artifacts:
path: playwright-report
destination: playwright-report
- store_artifacts:
path: test-results
destination: test-results
- store_test_results:
path: test-results/junit.xml
workflows:
test:
jobs:
- install
- test:
requires:
- installConfig for CircleCI JUnit integration:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
['dot'],
['html', { open: 'never' }],
['junit', { outputFile: 'test-results/junit.xml' }],
]
: [['html', { open: 'on-failure' }]],
});CircleCI with Orbs (Simplified)
# .circleci/config.yml
version: 2.1
orbs:
node: circleci/node@6.1
executors:
playwright:
docker:
- image: mcr.microsoft.com/playwright:v1.52.0-noble
jobs:
e2e:
executor: playwright
parallelism: 4
steps:
- checkout
- node/install-packages
- run:
name: Run tests
command: npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
- store_artifacts:
path: playwright-report
- store_test_results:
path: test-results/junit.xml
workflows:
main:
jobs:
- e2e---
Pattern 2: Azure DevOps
Use when: Your project runs on Azure DevOps Pipelines.
Basic Pipeline
# azure-pipelines.yml
trigger:
branches:
include:
- main
pr:
branches:
include:
- main
pool:
vmImage: 'ubuntu-latest'
variables:
CI: 'true'
npm_config_cache: $(Pipeline.Workspace)/.npm
steps:
- task: NodeTool@0
inputs:
versionSpec: '20.x'
displayName: 'Install Node.js'
- task: Cache@2
inputs:
key: 'npm | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
npm | "$(Agent.OS)"
path: $(npm_config_cache)
displayName: 'Cache npm'
- script: npm ci
displayName: 'Install dependencies'
- script: npx playwright install --with-deps
displayName: 'Install Playwright browsers'
- script: npx playwright test
displayName: 'Run Playwright tests'
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'test-results/junit.xml'
mergeTestResults: true
testRunTitle: 'Playwright Tests'
displayName: 'Publish test results'
- task: PublishPipelineArtifact@1
condition: always()
inputs:
targetPath: playwright-report
artifact: playwright-report
publishLocation: 'pipeline'
displayName: 'Upload report'Config for Azure DevOps JUnit integration:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
['dot'],
['html', { open: 'never' }],
['junit', { outputFile: 'test-results/junit.xml' }],
]
: [['html', { open: 'on-failure' }]],
});Azure DevOps with Sharding
# azure-pipelines.yml
trigger:
branches:
include:
- main
pr:
branches:
include:
- main
variables:
CI: 'true'
stages:
- stage: Test
jobs:
- job: Playwright
pool:
vmImage: 'ubuntu-latest'
strategy:
matrix:
shard1:
SHARD: '1/4'
shard2:
SHARD: '2/4'
shard3:
SHARD: '3/4'
shard4:
SHARD: '4/4'
steps:
- task: NodeTool@0
inputs:
versionSpec: '20.x'
- script: npm ci
displayName: 'Install dependencies'
- script: npx playwright install --with-deps
displayName: 'Install browsers'
- script: npx playwright test --shard=$(SHARD)
displayName: 'Run tests (shard $(SHARD))'
- task: PublishPipelineArtifact@1
condition: always()
inputs:
targetPath: blob-report
artifact: blob-report-$(System.JobPositionInPhase)
displayName: 'Upload blob report'
- stage: Report
dependsOn: Test
condition: always()
jobs:
- job: MergeReports
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool@0
inputs:
versionSpec: '20.x'
- script: npm ci
displayName: 'Install dependencies'
- task: DownloadPipelineArtifact@2
inputs:
patterns: 'blob-report-*/**'
path: all-blob-reports
displayName: 'Download all blob reports'
- script: npx playwright merge-reports --reporter=html ./all-blob-reports
displayName: 'Merge reports'
- task: PublishPipelineArtifact@1
inputs:
targetPath: playwright-report
artifact: playwright-report
displayName: 'Upload merged report'---
Pattern 3: Jenkins
Use when: Your project runs on Jenkins.
Jenkinsfile (Declarative Pipeline)
// Jenkinsfile
pipeline {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root' // Playwright needs root in container
}
}
environment {
CI = 'true'
HOME = '/root'
npm_config_cache = "${WORKSPACE}/.npm"
}
options {
timeout(time: 30, unit: 'MINUTES')
disableConcurrentBuilds()
}
stages {
stage('Install') {
steps {
sh 'npm ci'
}
}
stage('Test') {
steps {
sh 'npx playwright test'
}
post {
always {
// Publish JUnit results
junit allowEmptyResults: true,
testResults: 'test-results/junit.xml'
// Archive HTML report
archiveArtifacts artifacts: 'playwright-report/**',
allowEmptyArchive: true
// Archive traces on failure
archiveArtifacts artifacts: 'test-results/**',
allowEmptyArchive: true
}
}
}
}
post {
failure {
// Notify on failure (Slack, email, etc.)
echo 'Playwright tests failed!'
}
cleanup {
cleanWs()
}
}
}Config for Jenkins JUnit integration:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
['dot'],
['html', { open: 'never' }],
['junit', { outputFile: 'test-results/junit.xml' }],
]
: [['html', { open: 'on-failure' }]],
});Jenkins with Parallel Stages
// Jenkinsfile (sharded)
pipeline {
agent none
environment {
CI = 'true'
HOME = '/root'
}
options {
timeout(time: 30, unit: 'MINUTES')
}
stages {
stage('Test') {
parallel {
stage('Shard 1') {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
steps {
sh 'npm ci'
sh 'npx playwright test --shard=1/4'
}
post {
always {
archiveArtifacts artifacts: 'blob-report/**',
allowEmptyArchive: true
}
}
}
stage('Shard 2') {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
steps {
sh 'npm ci'
sh 'npx playwright test --shard=2/4'
}
post {
always {
archiveArtifacts artifacts: 'blob-report/**',
allowEmptyArchive: true
}
}
}
stage('Shard 3') {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
steps {
sh 'npm ci'
sh 'npx playwright test --shard=3/4'
}
post {
always {
archiveArtifacts artifacts: 'blob-report/**',
allowEmptyArchive: true
}
}
}
stage('Shard 4') {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
steps {
sh 'npm ci'
sh 'npx playwright test --shard=4/4'
}
post {
always {
archiveArtifacts artifacts: 'blob-report/**',
allowEmptyArchive: true
}
}
}
}
}
}
}Decision Guide
| CI Platform | Docker Image Support | Native Parallelism | Artifact Browsing | JUnit Integration |
|---|---|---|---|---|
| CircleCI | First-class (docker: executor) | parallelism: N with CIRCLE_NODE_INDEX | Via artifacts tab | store_test_results |
| Azure DevOps | Via vmImage or container jobs | strategy.matrix | Pipeline Artifacts UI | PublishTestResults@2 |
| Jenkins | Docker Pipeline plugin | parallel stages | Archived Artifacts | junit step |
| Scenario | CircleCI | Azure DevOps | Jenkins |
|---|---|---|---|
| Shard variable | $((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL | Define in matrix: SHARD: '1/4' | Hardcode per parallel stage |
| Cache key | checksum "package-lock.json" | Cache@2 with key template | stash/unstash or shared volume |
| Secrets | Context + environment variables | Variable groups + pipeline variables | Credentials plugin |
| Report upload | store_artifacts | PublishPipelineArtifact@1 | archiveArtifacts |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
Installing browsers on bare metal without --with-deps | Missing OS libs cause launch failures | Use Playwright Docker image or --with-deps flag |
| No JUnit reporter | CI platform can't display test results natively | Add ['junit', { outputFile: 'test-results/junit.xml' }] |
| Unlimited job timeout | Hung tests run indefinitely, consuming CI resources | Set explicit timeout (20-30 min) |
| No artifact upload on success | Can't verify results when tests pass | Always upload reports (condition: always() / when: always) |
| Running browsers as non-root in container without setup | Permission errors on browser binaries | Run as root or configure proper permissions |
| Hardcoding shard count in config instead of using CI variables | Must update two places when changing parallelism | Use CI-native variables (CI_NODE_TOTAL, CIRCLE_NODE_TOTAL) |
Troubleshooting
CircleCI: "Error: browserType.launch: Executable doesn't exist"
Cause: Not using the Playwright Docker image, or image version doesn't match @playwright/test version.
Fix: Match image tag to your Playwright version:
docker:
- image: mcr.microsoft.com/playwright:v1.52.0-noble # match package.json versionAzure DevOps: Test results not showing in Tests tab
Cause: JUnit reporter not configured, or PublishTestResults@2 task missing.
Fix: Add both the reporter and the publish task:
// playwright.config.ts
reporter: [['junit', { outputFile: 'test-results/junit.xml' }]],- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'test-results/junit.xml'Jenkins: "Browser closed unexpectedly" in Docker agent
Cause: Running as non-root user in container. Chromium's sandbox needs root or --no-sandbox.
Fix: Run as root in the Docker agent:
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
environment {
HOME = '/root'
}All platforms: Shard index off by one
Cause: CircleCI's CIRCLE_NODE_INDEX is 0-based, but Playwright's --shard is 1-based.
Fix: Add 1 to the index for CircleCI:
# CircleCI
command: npx playwright test --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
# GitLab (already 1-based)
command: npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTALRelated
- ci/ci-github-actions.md -- GitHub Actions configuration
- ci/ci-gitlab.md -- GitLab CI configuration
- ci/parallel-and-sharding.md -- sharding strategies
- ci/docker-and-containers.md -- Docker image details
- ci/reporting-and-artifacts.md -- reporter configuration for CI
Docker and Containers
When to use: Running Playwright tests in containers for reproducible environments, CI pipelines, or local development with consistent browser versions. Essential when your team needs identical test environments across machines.
Quick Reference
# Official Playwright Docker images
docker pull mcr.microsoft.com/playwright:v1.52.0-noble # Ubuntu 24.04, all browsers
docker pull mcr.microsoft.com/playwright:v1.52.0-jammy # Ubuntu 22.04, all browsers
# Run tests in container
docker run --rm -v $(pwd):/app -w /app mcr.microsoft.com/playwright:v1.52.0-noble \
npx playwright test
# Check your Playwright version (must match image tag)
npx playwright --versionPatterns
Pattern 1: Running Tests in the Official Image
Use when: Quick, reproducible test runs without building a custom image. Avoid when: You need application services (database, API) running alongside -- use docker-compose instead.
# Run all tests
docker run --rm \
-v $(pwd):/app \
-w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"
# Run with environment variables
docker run --rm \
-v $(pwd):/app \
-w /app \
-e CI=true \
-e BASE_URL=http://host.docker.internal:3000 \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"
# Run and extract report
docker run --rm \
-v $(pwd):/app \
-w /app \
-v $(pwd)/playwright-report:/app/playwright-report \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"Pattern 2: Custom Dockerfile
Use when: You need additional system dependencies, pre-installed npm packages, or a smaller image with only certain browsers. Avoid when: The official image works as-is.
# Dockerfile.playwright
FROM mcr.microsoft.com/playwright:v1.52.0-noble
WORKDIR /app
# Copy package files first for better layer caching
COPY package.json package-lock.json ./
RUN npm ci
# Copy the rest of the project
COPY . .
# Default command
CMD ["npx", "playwright", "test"]# Build and run
docker build -f Dockerfile.playwright -t my-e2e-tests .
docker run --rm my-e2e-tests
# Run with specific options
docker run --rm my-e2e-tests npx playwright test --project=chromium
# Extract report
docker run --rm -v $(pwd)/reports:/app/playwright-report my-e2e-testsSlim image with only Chromium:
# Dockerfile.playwright-chromium
FROM node:20-slim
# Install only Chromium dependencies
RUN npx playwright install --with-deps chromium
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
CMD ["npx", "playwright", "test", "--project=chromium"]Pattern 3: Docker Compose with Application Stack
Use when: Tests need the full application stack: web server, database, cache, and Playwright running together. Avoid when: Tests run against a remote environment (staging/prod) -- no local services needed.
# docker-compose.yml
services:
# Application under test
app:
build:
context: .
dockerfile: Dockerfile
ports:
- "3000:3000"
environment:
- NODE_ENV=test
- DATABASE_URL=postgresql://postgres:postgres@db:5432/test
- REDIS_URL=redis://cache:6379
depends_on:
db:
condition: service_healthy
cache:
condition: service_started
# Database
db:
image: postgres:16-alpine
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 5s
retries: 5
tmpfs:
- /var/lib/postgresql/data # RAM disk for speed
# Cache
cache:
image: redis:7-alpine
# Playwright test runner
e2e:
image: mcr.microsoft.com/playwright:v1.52.0-noble
working_dir: /app
volumes:
- .:/app
- /app/node_modules # prevent host node_modules from overriding
environment:
- CI=true
- BASE_URL=http://app:3000
depends_on:
- app
command: bash -c "npm ci && npx playwright test"
profiles:
- test # only start with: docker compose --profile test up# Run the full stack with tests
docker compose --profile test up --abort-on-container-exit --exit-code-from e2e
# Run just the app (for local dev)
docker compose up app
# Run tests against already-running stack
docker compose --profile test run --rm e2e npx playwright test
# Tear down everything
docker compose --profile test down -vPattern 4: Extracting Reports and Traces from Containers
Use when: You need test artifacts (HTML reports, traces, screenshots) accessible on the host after container tests complete. Avoid when: CI handles artifact collection natively (GitHub Actions, GitLab artifacts).
# Method 1: Bind mount the output directories
docker run --rm \
-v $(pwd):/app \
-v $(pwd)/playwright-report:/app/playwright-report \
-v $(pwd)/test-results:/app/test-results \
-w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"
# Method 2: Copy artifacts from a stopped container
docker run --name e2e-run \
-v $(pwd):/app \
-w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test" || true
docker cp e2e-run:/app/playwright-report ./playwright-report
docker cp e2e-run:/app/test-results ./test-results
docker rm e2e-run
# View the report
npx playwright show-report ./playwright-reportDocker Compose for report extraction:
# docker-compose.yml (add to e2e service)
services:
e2e:
image: mcr.microsoft.com/playwright:v1.52.0-noble
working_dir: /app
volumes:
- .:/app
- ./playwright-report:/app/playwright-report
- ./test-results:/app/test-results
# ... rest of configPattern 5: CI Container Strategies
Use when: Your CI environment benefits from containerized test execution.
GitHub Actions -- container job:
# .github/workflows/playwright.yml
jobs:
test:
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.52.0-noble
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npx playwright test
env:
HOME: /rootGitLab CI -- image directive:
# .gitlab-ci.yml
test:
image: mcr.microsoft.com/playwright:v1.52.0-noble
script:
- npm ci
- npx playwright testJenkins -- Docker agent:
// Jenkinsfile
pipeline {
agent {
docker {
image 'mcr.microsoft.com/playwright:v1.52.0-noble'
args '-u root'
}
}
stages {
stage('Test') {
steps {
sh 'npm ci'
sh 'npx playwright test'
}
}
}
}Pattern 6: Development Container (devcontainer)
Use when: Your team uses VS Code Dev Containers or GitHub Codespaces and needs Playwright available in the development environment. Avoid when: Everyone installs Playwright locally and version differences don't cause issues.
// .devcontainer/devcontainer.json
{
"name": "Playwright Dev",
"image": "mcr.microsoft.com/playwright:v1.52.0-noble",
"features": {
"ghcr.io/devcontainers/features/node:1": {
"version": "20"
}
},
"postCreateCommand": "npm ci",
"customizations": {
"vscode": {
"extensions": [
"ms-playwright.playwright"
]
}
},
"forwardPorts": [3000, 9323],
"remoteUser": "root"
}Decision Guide
| Scenario | Approach | Why |
|---|---|---|
| Simple CI pipeline | Official Playwright image as CI image | Browsers pre-installed; zero browser install time |
| Tests need database + cache | Docker Compose with app, db, cache, e2e services | Full stack in isolated containers |
| Team needs identical environments | Dev Container or custom Dockerfile | Eliminate "works on my machine" |
| Only testing Chromium | Slim image: node:20-slim + install --with-deps chromium | Smaller image, faster pulls |
| Cross-browser testing | Official Playwright image (has all browsers) | All three engines pre-installed |
| Local development | Run directly on host, not in container | Faster iteration, easier debugging |
| CI with artifact extraction | Bind mount report/results dirs or use CI artifact upload | Reports accessible after container exits |
| Image | Size | Browsers | Base OS |
|---|---|---|---|
mcr.microsoft.com/playwright:v1.52.0-noble | ~2 GB | Chromium, Firefox, WebKit | Ubuntu 24.04 |
mcr.microsoft.com/playwright:v1.52.0-jammy | ~2 GB | Chromium, Firefox, WebKit | Ubuntu 22.04 |
| Custom slim (Chromium only) | ~800 MB | Chromium | Depends on base |
Security: Pinning Docker Images to Digest
Version tags like mcr.microsoft.com/playwright:v1.52.0-noble are mutable — the tag can be updated to point to a different image layer without changing the tag name. This is a supply-chain risk (W012): you may pull different code than you tested against.
Best practice: pin images to their immutable content digest.
# Get the digest for a specific tag
docker pull mcr.microsoft.com/playwright:v1.52.0-noble
docker inspect --format='{{index .RepoDigests 0}}' mcr.microsoft.com/playwright:v1.52.0-noble
# e.g. mcr.microsoft.com/playwright@sha256:abc123...
# Or use skopeo (no pull required):
skopeo inspect docker://mcr.microsoft.com/playwright:v1.52.0-noble | jq '.Digest'Use digest-pinned references in CI:
# Instead of:
image: mcr.microsoft.com/playwright:v1.52.0-noble
# Pin to digest (example — verify the actual digest for your version):
image: mcr.microsoft.com/playwright:v1.52.0-noble@sha256:<digest-from-inspect># Dockerfile
FROM mcr.microsoft.com/playwright:v1.52.0-noble@sha256:<digest-from-inspect>Digest values are specific to the exact image build. Verify the digest from the official Microsoft Artifact Registry (mcr.microsoft.com) before pinning.
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
Image tag doesn't match @playwright/test version | Browser binaries incompatible with Playwright library | Always match: v1.52.0 image for @playwright/test@1.52.0 |
Using latest tag | Unpredictable; image updates can break tests | Pin to exact version: v1.52.0-noble |
| Using only a version tag without digest | Tag is mutable; supply-chain risk if image is silently updated | Pin to digest: playwright:v1.52.0-noble@sha256:<digest> |
| Installing browsers inside container at runtime | Wastes 60-90 seconds on every run | Use official image (browsers pre-installed) or build custom image with browsers baked in |
| Running as non-root without configuring sandbox | Chromium sandbox fails with permission errors | Run as root (-u root) or disable sandbox (--no-sandbox in launch args) |
Bind-mounting node_modules from host | Platform-specific binaries (macOS vs Linux) cause crashes | Use anonymous volume: -v /app/node_modules |
| No health checks on dependent services | Tests start before database is ready | Add healthcheck to db service; use depends_on: condition: service_healthy |
| Building application inside the Playwright container | Large image, slow builds, wrong base for your app | Separate app and e2e containers in docker-compose |
Troubleshooting
"browserType.launch: Executable doesn't exist" in container
Cause: Playwright version in package.json doesn't match the Docker image version.
Fix: Ensure exact version match:
# Check your version
npm ls @playwright/test
# @playwright/test@1.52.0
# Use matching image
docker pull mcr.microsoft.com/playwright:v1.52.0-nobleTests fail with "net::ERR_CONNECTION_REFUSED" in docker-compose
Cause: Tests are trying to reach localhost:3000 but the app is in a different container.
Fix: Use the service name as hostname and configure baseURL:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
baseURL: process.env.BASE_URL || 'http://localhost:3000',
},
// Disable webServer in Docker -- app is managed by docker-compose
...(process.env.CI ? {} : {
webServer: {
command: 'npm run dev',
url: 'http://localhost:3000',
reuseExistingServer: true,
},
}),
});# docker-compose.yml
e2e:
environment:
- BASE_URL=http://app:3000 # "app" is the service namePermission denied on mounted volumes
Cause: Container runs as root but host files are owned by your user, or vice versa.
Fix: Match user IDs or run as root:
# Run as your host user
docker run --rm -u $(id -u):$(id -g) \
-v $(pwd):/app -w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
npx playwright test
# Or run as root (simpler, fine for CI)
docker run --rm \
-v $(pwd):/app -w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
npx playwright testContainer tests are much slower than local
Cause: Docker Desktop on macOS/Windows has I/O overhead for bind-mounted volumes.
Fix: Copy files into the container instead of mounting:
# Dockerfile.playwright
FROM mcr.microsoft.com/playwright:v1.52.0-noble
WORKDIR /app
COPY . .
RUN npm ci
CMD ["npx", "playwright", "test"]Or use delegated mount on macOS:
docker run --rm \
-v $(pwd):/app:delegated \
-w /app \
mcr.microsoft.com/playwright:v1.52.0-noble \
bash -c "npm ci && npx playwright test"Related
- ci/ci-github-actions.md -- container jobs in GitHub Actions
- ci/ci-gitlab.md -- Docker images in GitLab CI
- ci/ci-other.md -- Docker agents in Jenkins, CircleCI
- ci/parallel-and-sharding.md -- sharding within containers
- ci/reporting-and-artifacts.md -- extracting reports from containers
Global Setup and Teardown
When to use: One-time operations that must run before or after the entire test suite -- database seeding, environment health checks, creating shared auth state, starting external services. Runs once per npx playwright test invocation, not once per test or per worker.Quick Reference
globalSetup → runs ONCE before all tests in all projects
↓
setup projects → runs before dependent projects (has browser context)
↓
test projects → your actual tests
↓
teardown projects → runs after dependent projects (has browser context)
↓
globalTeardown → runs ONCE after all tests in all projectsKey distinction:
globalSetup/globalTeardown: No browser, no Playwright fixtures. Pure Node.js.- Setup projects with
dependencies: Has full browser context, can usepage,request, etc.
Patterns
Pattern 1: Basic Global Setup and Teardown
Use when: One-time non-browser work like database seeding, environment validation, or external service preparation. Avoid when: You need a browser (use a setup project instead) or per-test isolation (use fixtures).
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
globalSetup: './tests/global-setup.ts',
globalTeardown: './tests/global-teardown.ts',
testDir: './tests',
});// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
console.log('Global setup: seeding database...');
// Seed the test database
const { execSync } = await import('child_process');
execSync('npx prisma db push --force-reset', { stdio: 'inherit' });
execSync('npx prisma db seed', { stdio: 'inherit' });
// Store run metadata for tests to use
process.env.TEST_RUN_ID = `run-${Date.now()}`;
}
export default globalSetup;// tests/global-teardown.ts
import type { FullConfig } from '@playwright/test';
async function globalTeardown(config: FullConfig) {
console.log('Global teardown: cleaning up...');
const { execSync } = await import('child_process');
execSync('npx prisma db push --force-reset', { stdio: 'inherit' });
}
export default globalTeardown;Pattern 2: Environment Health Check in Global Setup
Use when: Verifying the test environment is healthy before running any tests. Fails fast if services are down. Avoid when: Tests use webServer which already does a health check.
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
const maxRetries = 10;
const retryDelay = 2000;
console.log(`Checking if ${baseURL} is reachable...`);
for (let i = 0; i < maxRetries; i++) {
try {
const response = await fetch(`${baseURL}/api/health`);
if (response.ok) {
console.log(`Environment is healthy (attempt ${i + 1})`);
return;
}
} catch {
// Connection refused or timeout -- retry
}
console.log(`Waiting for environment... (attempt ${i + 1}/${maxRetries})`);
await new Promise((resolve) => setTimeout(resolve, retryDelay));
}
throw new Error(`Environment at ${baseURL} is not reachable after ${maxRetries} attempts`);
}
export default globalSetup;Pattern 3: Authentication State in Global Setup (Without Browser)
Use when: Creating auth tokens or session cookies via API, without needing a browser. Avoid when: Login requires browser interaction (use a setup project instead).
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
import * as fs from 'fs';
import * as path from 'path';
async function globalSetup(config: FullConfig) {
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
// Authenticate via API (no browser needed)
const response = await fetch(`${baseURL}/api/auth/login`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
email: 'admin@example.com',
password: process.env.TEST_PASSWORD,
}),
});
if (!response.ok) {
throw new Error(`Authentication failed: ${response.status} ${response.statusText}`);
}
const { token } = await response.json();
// Save the token as storageState for browser tests to pick up
const authDir = path.resolve(process.cwd(), 'playwright/.auth');
fs.mkdirSync(authDir, { recursive: true });
const storageState = {
cookies: [],
origins: [
{
origin: baseURL,
localStorage: [
{ name: 'auth_token', value: token },
],
},
],
};
fs.writeFileSync(
path.join(authDir, 'user.json'),
JSON.stringify(storageState, null, 2)
);
}
export default globalSetup;// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
globalSetup: './tests/global-setup.ts',
projects: [
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/user.json',
},
},
],
});Pattern 4: Passing Data from Global Setup to Tests
Use when: Global setup generates values (IDs, tokens, URLs) that tests need. Avoid when: Each test should create its own data (the usual case).
Method 1: Environment variables (simplest):
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
process.env.TEST_RUN_ID = `run-${Date.now()}`;
process.env.SEED_USER_ID = 'user-12345';
}
export default globalSetup;// tests/dashboard.spec.ts
import { test, expect } from '@playwright/test';
test('dashboard shows seeded data', async ({ page }) => {
const userId = process.env.SEED_USER_ID;
await page.goto(`/users/${userId}/dashboard`);
await expect(page.getByRole('heading')).toBeVisible();
});Method 2: Shared file (for complex data):
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
import * as fs from 'fs';
import * as path from 'path';
const SETUP_DATA_PATH = path.resolve(process.cwd(), 'test-data/setup-data.json');
async function globalSetup(config: FullConfig) {
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
// Create test data via API
const res = await fetch(`${baseURL}/api/test/seed`, { method: 'POST' });
const seedData = await res.json();
// Write to shared file
const dir = path.dirname(SETUP_DATA_PATH);
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(SETUP_DATA_PATH, JSON.stringify(seedData, null, 2));
}
export default globalSetup;// tests/helpers/setup-data.ts
import * as fs from 'fs';
import * as path from 'path';
const SETUP_DATA_PATH = path.resolve(process.cwd(), 'test-data/setup-data.json');
export function getSetupData(): { userId: string; orgId: string; apiKey: string } {
const raw = fs.readFileSync(SETUP_DATA_PATH, 'utf8');
return JSON.parse(raw);
}// tests/org-settings.spec.ts
import { test, expect } from '@playwright/test';
import { getSetupData } from './helpers/setup-data';
test('org settings page loads', async ({ page }) => {
const { orgId } = getSetupData();
await page.goto(`/orgs/${orgId}/settings`);
await expect(page.getByRole('heading', { name: 'Settings' })).toBeVisible();
});Pattern 5: Global Setup with storageState (Browser-Based Auth)
Use when: Authentication requires browser interaction (form login, OAuth redirect, MFA). Avoid when: Auth can be done via API call (use Pattern 3 instead).
Important: globalSetup has no browser. For browser-based auth, use a setup project instead.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
projects: [
// Setup project: runs first, has a browser, saves auth state
{
name: 'setup',
testMatch: /global\.setup\.ts/,
},
// Test projects: depend on setup, reuse saved auth state
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
{
name: 'firefox',
use: {
...devices['Desktop Firefox'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
],
});// tests/global.setup.ts
import { test as setup, expect } from '@playwright/test';
const authFile = 'playwright/.auth/user.json';
setup('authenticate', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
// Wait for navigation to confirm login succeeded
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
// Save signed-in state (cookies + localStorage)
await page.context().storageState({ path: authFile });
});Pattern 6: Global Setup for External Services
Use when: Starting or configuring external services (mock servers, test containers, feature flags) before any tests run. Avoid when: Per-test or per-worker isolation is needed (use fixtures).
// tests/global-setup.ts
import type { FullConfig } from '@playwright/test';
async function globalSetup(config: FullConfig) {
// Start a mock API server
const { createServer } = await import('../mocks/server');
const server = await createServer();
const port = await server.listen(0);
process.env.MOCK_API_URL = `http://localhost:${port}`;
// Configure feature flags for test environment
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
await fetch(`${baseURL}/api/admin/feature-flags`, {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${process.env.ADMIN_API_KEY}`,
},
body: JSON.stringify({
newCheckout: true,
darkMode: false,
betaFeatures: true,
}),
});
// Return a cleanup function (Playwright calls globalTeardown separately)
// For cleanup, use globalTeardown
}
export default globalSetup;// tests/global-teardown.ts
import type { FullConfig } from '@playwright/test';
async function globalTeardown(config: FullConfig) {
// Reset feature flags
const baseURL = config.projects[0]?.use?.baseURL || 'http://localhost:3000';
await fetch(`${baseURL}/api/admin/feature-flags/reset`, {
method: 'POST',
headers: { Authorization: `Bearer ${process.env.ADMIN_API_KEY}` },
});
}
export default globalTeardown;Decision Guide
| Need | Use | Why |
|---|---|---|
| One-time DB seed | globalSetup | No browser needed; runs once before everything |
| Browser-based login (shared state) | Setup project with dependencies | Needs page and context (not available in globalSetup) |
| API-based auth token | globalSetup | Simple HTTP call, no browser needed |
| Per-test unique data | Custom fixture via test.extend() | Each test gets isolated data |
| Per-worker shared resource | Worker-scoped fixture ({ scope: 'worker' }) | Shared within worker, isolated between workers |
| Health check before tests | globalSetup | Fail fast if environment is down |
| Start mock server | globalSetup + globalTeardown | One-time server lifecycle |
| Clean up after all tests | globalTeardown | Runs once at the end regardless of pass/fail |
Do I need globalSetup?
│
├── Does the work need a browser (page, context)?
│ ├── YES → Use a setup project, not globalSetup
│ └── NO → globalSetup is appropriate
│
├── Does every test need unique/isolated data?
│ ├── YES → Use a fixture with test.extend()
│ └── NO → globalSetup for shared, read-only data
│
├── Is it per-worker (expensive resource, connection pool)?
│ ├── YES → Worker-scoped fixture
│ └── NO → globalSetup for truly global, one-time work
│
└── Is it cleanup?
├── After all tests → globalTeardown
├── After each test → Fixture teardown (after use())
└── After each worker → Worker-scoped fixture teardownAnti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
Browser login in globalSetup | No browser context available; complex workarounds | Use a setup project with dependencies |
Creating per-test data in globalSetup | All tests share the same data; not isolated | Use per-test fixtures for unique data |
globalSetup without globalTeardown | Database or services left in dirty state | Always pair setup with teardown |
| Storing setup results in module-level variables | Workers are separate processes; variables don't share | Use environment variables or files |
Complex logic in globalSetup | Hard to debug; runs outside normal test lifecycle | Keep it minimal: seed, verify, set env vars |
globalSetup that takes > 60 seconds | Slows every test run, even for a single test | Move heavy work to a separate script or CI step |
Relying on globalTeardown for critical cleanup | If the process crashes, globalTeardown might not run | Design tests to be idempotent; use beforeAll in setup project |
Troubleshooting
Global setup runs but environment variables are not available in tests
Cause: Each worker is a separate process. process.env mutations in globalSetup propagate to workers, but only if set before workers spawn.
Fix: Set environment variables at the top level of globalSetup, before any async work that might delay:
// tests/global-setup.ts
async function globalSetup() {
// This works -- set before returning
process.env.TEST_RUN_ID = `run-${Date.now()}`;
}
export default globalSetup;If environment variables are still missing, write data to a file instead (Pattern 4, Method 2).
Global setup fails with "Cannot find module"
Cause: The path in globalSetup is relative to the config file, but the module resolution is wrong.
Fix: Use a path relative to the project root:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
globalSetup: './tests/global-setup.ts', // relative to config location
globalTeardown: './tests/global-teardown.ts',
});Global teardown doesn't run after test failure
Cause: If the process is killed (SIGKILL, OOM) rather than exiting normally, teardown is skipped.
Fix: Design your setup to be idempotent. Global setup should handle a dirty state from a previous incomplete run:
// tests/global-setup.ts
async function globalSetup() {
// Always reset first, then seed -- handles dirty state
const { execSync } = await import('child_process');
execSync('npx prisma db push --force-reset', { stdio: 'inherit' });
execSync('npx prisma db seed', { stdio: 'inherit' });
}
export default globalSetup;Setup project runs every time, even when only running one test file
Cause: The dependencies configuration requires the setup project to run before any dependent project.
Fix: This is expected behavior. To skip setup during focused debugging:
# Skip setup by running without dependencies
npx playwright test --project=chromium --no-deps tests/specific-test.spec.tsstorageState file doesn't exist when tests start
Cause: The setup project or globalSetup that creates the file failed silently, or the path is wrong.
Fix: Add explicit error handling and verify the file exists:
// tests/global.setup.ts
import { test as setup, expect } from '@playwright/test';
import * as fs from 'fs';
const authFile = 'playwright/.auth/user.json';
setup('authenticate', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
await page.context().storageState({ path: authFile });
// Verify the file was created
if (!fs.existsSync(authFile)) {
throw new Error(`Auth state file was not created at ${authFile}`);
}
});Related
- core/fixtures-and-hooks.md -- per-test and per-worker fixtures (preferred over globalSetup for most cases)
- core/configuration.md --
globalSetup,globalTeardown,webServerconfig - ci/projects-and-dependencies.md -- setup projects with
dependencies - core/authentication.md -- authentication patterns using setup projects
- core/test-data-management.md -- seeding and managing test data
Parallel Execution and Sharding
When to use: Speeding up test suites by running tests concurrently within a single machine (parallelism) or across multiple machines (sharding). Essential once your suite exceeds 5 minutes.
Quick Reference
# Workers (parallelism within one machine)
npx playwright test --workers=4 # fixed worker count
npx playwright test --workers=50% # percentage of CPU cores
# Sharding (splitting across machines)
npx playwright test --shard=1/4 # run first quarter
npx playwright test --shard=2/4 # run second quarter
# Merging shard results
npx playwright merge-reports ./blob-report # merge to default HTML
npx playwright merge-reports --reporter=html,json ./blob-report # multiple formats
# Fully parallel mode
npx playwright test --fully-parallel # override config for this runPatterns
Pattern 1: Configuring Workers
Use when: Controlling how many tests run simultaneously on one machine. Avoid when: You only have 1-2 tests (parallelism has no effect).
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
// fullyParallel: true means tests WITHIN a file also run in parallel.
// Without it, only files run in parallel (tests within a file are serial).
fullyParallel: true,
// Workers: how many parallel processes
// - undefined: auto-detect (half CPU cores, capped at a reasonable number)
// - number: fixed count
// - string percentage: '50%' of CPU cores
workers: process.env.CI ? '50%' : undefined,
});What `fullyParallel` actually controls:
| Setting | Files run in parallel | Tests within a file run in parallel |
|---|---|---|
fullyParallel: false (default) | Yes | No -- serial within each file |
fullyParallel: true | Yes | Yes -- every test is independent |
Per-file override when one file needs serial execution:
// tests/onboarding.spec.ts
import { test, expect } from '@playwright/test';
// This file's tests run serially even with fullyParallel: true in config
test.describe.configure({ mode: 'serial' });
test('step 1: enter company name', async ({ page }) => {
// ...
});
test('step 2: choose plan', async ({ page }) => {
// ...
});Pattern 2: Sharding Across CI Machines
Use when: Suite is too slow for a single machine even with maximum workers. Split work across N separate CI jobs. Avoid when: Suite runs under 5 minutes on one machine.
Sharding splits the test file list into N equal groups. Each shard runs one group.
# Machine 1 Machine 2 Machine 3 Machine 4
--shard=1/4 --shard=2/4 --shard=3/4 --shard=4/4Config for sharded runs -- use blob reporter:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
fullyParallel: true,
workers: process.env.CI ? '50%' : undefined,
// Blob reporter outputs a binary file that can be merged later.
// In non-CI, use HTML for local viewing.
reporter: process.env.CI
? [['blob'], ['github']]
: [['html', { open: 'on-failure' }]],
});Pattern 3: Merging Blob Reports from Shards
Use when: You sharded your tests and need a single unified report. Avoid when: No sharding -- the regular HTML reporter works directly.
Each shard produces a .zip file in blob-report/. After all shards complete, merge them:
# Download all blob-report/ directories into one folder, then:
npx playwright merge-reports --reporter=html ./all-blob-reports
# Multiple output formats
npx playwright merge-reports --reporter=html,json,junit ./all-blob-reports
# Custom output directory
PLAYWRIGHT_HTML_REPORT=merged-report npx playwright merge-reports --reporter=html ./all-blob-reportsGitHub Actions example (merge job):
merge-reports:
if: ${{ !cancelled() }}
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- uses: actions/download-artifact@v4
with:
path: all-blob-reports
pattern: blob-report-*
merge-multiple: true
- run: npx playwright merge-reports --reporter=html ./all-blob-reports
- uses: actions/upload-artifact@v4
with:
name: playwright-report
path: playwright-report/
retention-days: 14Pattern 4: Worker-Scoped Fixtures for Shared Resources
Use when: Parallel workers each need an expensive resource (database connection, auth token) that should be created once per worker, not once per test. Avoid when: The resource is cheap to create -- use a regular test-scoped fixture.
// fixtures.ts
import { test as base } from '@playwright/test';
type WorkerFixtures = {
dbConnection: DatabaseClient;
workerAuthToken: string;
};
export const test = base.extend<{}, WorkerFixtures>({
// Created once per worker process, shared across all tests in that worker
dbConnection: [async ({}, use) => {
const db = await DatabaseClient.connect(process.env.DB_URL!);
await use(db);
await db.disconnect();
}, { scope: 'worker' }],
workerAuthToken: [async ({}, use, workerInfo) => {
// Each worker gets a unique user to avoid test interference
const response = await fetch(`${process.env.API_URL}/auth/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
username: `worker-user-${workerInfo.workerIndex}`,
password: process.env.TEST_PASSWORD,
}),
});
const { token } = await response.json();
await use(token);
}, { scope: 'worker' }],
});
export { expect } from '@playwright/test';Pattern 5: Test Isolation for Safe Parallelism
Use when: Preparing tests to run in parallel without interference. Avoid when: Never -- isolation is always required for reliable parallel execution.
The golden rule: Each test must create its own state and clean up after itself. No test should depend on or modify state that another test uses.
// BAD: Tests share a hardcoded user -- parallel runs collide
test('update profile', async ({ page }) => {
await page.goto('/users/shared-user/profile');
await page.getByLabel('Name').fill('New Name');
await page.getByRole('button', { name: 'Save' }).click();
// Another parallel test also editing "shared-user" -- race condition!
});
// GOOD: Each test creates its own user
test('update profile', async ({ page, request }) => {
// Create a unique user for this test
const res = await request.post('/api/test/users', {
data: { name: `user-${Date.now()}`, email: `${Date.now()}@test.com` },
});
const user = await res.json();
await page.goto(`/users/${user.id}/profile`);
await page.getByLabel('Name').fill('Updated Name');
await page.getByRole('button', { name: 'Save' }).click();
await expect(page.getByLabel('Name')).toHaveValue('Updated Name');
// Cleanup
await request.delete(`/api/test/users/${user.id}`);
});Using `workerInfo` and `testInfo` for unique identifiers:
import { test, expect } from '@playwright/test';
test('create order', async ({ page }, testInfo) => {
const uniqueId = `order-${testInfo.workerIndex}-${Date.now()}`;
// Use uniqueId for any data this test creates
await page.goto(`/orders/new?ref=${uniqueId}`);
// ...
});Pattern 6: Dynamic Shard Count Based on Test Count
Use when: You want to automatically adjust shard count based on how many tests exist, rather than hardcoding. Avoid when: Your test count is stable and a fixed shard count works well.
# .github/workflows/playwright.yml -- dynamic shard calculation
jobs:
determine-shards:
runs-on: ubuntu-latest
outputs:
shard-count: ${{ steps.calc.outputs.count }}
shard-matrix: ${{ steps.calc.outputs.matrix }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- id: calc
run: |
TEST_COUNT=$(npx playwright test --list --reporter=json 2>/dev/null | node -e "
const data = require('fs').readFileSync('/dev/stdin', 'utf8');
const parsed = JSON.parse(data);
console.log(parsed.suites?.reduce((acc, s) => acc + (s.specs?.length || 0), 0) || 0);
")
# 1 shard per 20 tests, minimum 1, maximum 8
SHARDS=$(( (TEST_COUNT + 19) / 20 ))
SHARDS=$(( SHARDS > 8 ? 8 : SHARDS ))
SHARDS=$(( SHARDS < 1 ? 1 : SHARDS ))
# Build matrix array: ["1/N", "2/N", ...]
MATRIX="["
for i in $(seq 1 $SHARDS); do
[ $i -gt 1 ] && MATRIX+=","
MATRIX+="\"$i/$SHARDS\""
done
MATRIX+="]"
echo "count=$SHARDS" >> $GITHUB_OUTPUT
echo "matrix=$MATRIX" >> $GITHUB_OUTPUT
test:
needs: determine-shards
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: ${{ fromJson(needs.determine-shards.outputs.shard-matrix) }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- run: npm ci
- run: npx playwright install --with-deps
- run: npx playwright test --shard=${{ matrix.shard }}Decision Guide
| Scenario | Workers | Shards | Why |
|---|---|---|---|
| < 50 tests, < 5 min | Auto (default) | None | No optimization needed |
| 50-200 tests, 5-15 min | '50%' in CI | 2-4 shards | Balance speed and CI cost |
| 200+ tests, > 15 min | '50%' in CI | 4-8 shards | Keep feedback under 10 min |
| Flaky tests due to resource contention | Reduce: workers: 2 | Keep current | Fewer workers = less CPU/memory pressure |
| Tests modify shared database | workers: 1 or isolate per worker | Still useful | Sharding splits files; workers run them |
| CI has limited CPU/RAM | workers: 1 or '25%' | More shards | Compensate fewer workers with more machines |
| Question | workers (in-process) | --shard (across machines) |
|---|---|---|
| What does it split? | Tests across CPU cores on one machine | Test files across separate CI jobs |
| Controlled by? | playwright.config.ts or --workers CLI | --shard=X/Y CLI flag |
| Shares memory? | Yes (same machine) | No (separate machines) |
| Report merging needed? | No (single process) | Yes (merge-reports) |
| Cost | Free (same machine) | More CI minutes (more machines) |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
fullyParallel: false with no reason | Tests within files run serially; slow suite | Set fullyParallel: true unless specific tests need serial |
workers: 1 in CI "to be safe" | Negates parallelism entirely | Fix isolation issues; use workers: '50%' |
| Tests sharing a hardcoded user account | Race conditions when parallel -- both tests modify same data | Each test creates unique data via API or fixture |
--shard=1/4 without blob reporter | Each shard produces its own HTML report; no merged view | Configure reporter: [['blob']] for sharded CI runs |
| Sharding with 3 tests | Overhead of shard setup exceeds time saved | Only shard when suite exceeds 5 minutes |
test.describe.serial() everywhere | Kills parallelism, creates hidden dependencies | Use only when tests genuinely depend on prior state |
| Worker count higher than CPU cores | Context switching overhead; slower, not faster | Use '50%' or let Playwright auto-detect |
Not using fail-fast: false in CI matrix | One shard failure cancels others; incomplete results | Always set fail-fast: false for sharded strategies |
Troubleshooting
Tests pass alone but fail when run together
Cause: Shared state between tests -- database rows, cookies, global variables, file system.
Fix: Isolate each test. Use unique data per test:
test('create order', async ({ page, request }, testInfo) => {
// Unique product per test -- no collision with parallel tests
const product = await request.post('/api/test/products', {
data: { name: `Widget-${testInfo.workerIndex}-${Date.now()}` },
});
// ...
});Shard produces no tests: "No tests found"
Cause: Shard count exceeds the number of test files. A shard gets zero files.
Fix: Reduce shard count to at most the number of test files:
# If you have 10 test files, max 10 shards
npx playwright test --shard=1/10 # OK
npx playwright test --shard=1/20 # Some shards will be emptyMerged report missing some test results
Cause: Blob report files from a shard were not downloaded or were overwritten due to name collision.
Fix: Give each shard's artifact a unique name:
# Each shard
- uses: actions/upload-artifact@v4
with:
name: blob-report-${{ strategy.job-index }} # unique per shard
path: blob-report/
# Merge step
- uses: actions/download-artifact@v4
with:
pattern: blob-report-*
merge-multiple: true
path: all-blob-reportsWorker-scoped fixture not shared -- recreated per test
Cause: Missing { scope: 'worker' } option, or the fixture depends on a test-scoped fixture.
Fix: Ensure the fixture uses worker scope and only depends on worker-scoped fixtures:
export const test = base.extend<{}, { sharedResource: Resource }>({
sharedResource: [async ({}, use) => {
const resource = await Resource.create();
await use(resource);
await resource.destroy();
}, { scope: 'worker' }], // Don't forget this
});Tests are slower with more workers
Cause: Machine is CPU- or memory-bound. More workers cause thrashing.
Fix: Reduce workers until you find the sweet spot:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
workers: process.env.CI ? 2 : undefined, // Start low, increase if stable
});Related
- ci/ci-github-actions.md -- sharded GitHub Actions workflow
- ci/ci-gitlab.md -- GitLab
parallel:keyword with sharding - ci/ci-other.md -- sharding on CircleCI, Azure DevOps, Jenkins
- ci/reporting-and-artifacts.md -- blob reporter and merge-reports
- core/fixtures-and-hooks.md -- worker-scoped fixtures
- core/test-organization.md -- parallel vs serial execution
Projects and Dependencies
When to use: Running tests across multiple browsers, devices, or environments from a single config. Projects let you define different test configurations that can depend on each other, share setup work, and run selectively.
Quick Reference
# Run all projects
npx playwright test
# Run a specific project
npx playwright test --project=chromium
npx playwright test --project="Mobile Safari"
# Run multiple projects
npx playwright test --project=chromium --project=firefox
# List all projects and their tests
npx playwright test --list
# Skip dependencies (e.g., skip setup during debugging)
npx playwright test --project=chromium --no-depsPatterns
Pattern 1: Multi-Browser Testing
Use when: Ensuring your application works across Chromium, Firefox, and WebKit. Avoid when: Early development -- start with Chromium only and add browsers later.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
fullyParallel: true,
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
],
});Selective browser testing in CI (fast on PRs, thorough on main):
# .github/workflows/ci.yml
jobs:
test-pr:
if: github.event_name == 'pull_request'
steps:
# Chromium only on PRs for fast feedback
- run: npx playwright test --project=chromium
test-main:
if: github.ref == 'refs/heads/main'
steps:
# All browsers on main branch
- run: npx playwright testPattern 2: Desktop and Mobile Projects
Use when: Testing responsive layouts, touch interactions, or mobile-specific behavior. Avoid when: Your application is desktop-only.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
fullyParallel: true,
projects: [
// Desktop browsers
{
name: 'Desktop Chrome',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'Desktop Firefox',
use: { ...devices['Desktop Firefox'] },
},
{
name: 'Desktop Safari',
use: { ...devices['Desktop Safari'] },
},
// Mobile devices
{
name: 'Mobile Chrome',
use: { ...devices['Pixel 7'] },
},
{
name: 'Mobile Safari',
use: { ...devices['iPhone 14'] },
},
// Tablet
{
name: 'iPad',
use: { ...devices['iPad Pro 11'] },
},
],
});Run only mobile or only desktop:
npx playwright test --project="Mobile Chrome" --project="Mobile Safari"
npx playwright test --project="Desktop Chrome" --project="Desktop Firefox"Pattern 3: Setup Project with Dependencies
Use when: Tests need shared state (authentication, seeded data) that should be created once before all test projects run. Avoid when: Tests are fully independent with no shared setup phase.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
projects: [
// Setup runs first -- no dependencies, so it runs immediately
{
name: 'setup',
testMatch: /global\.setup\.ts/,
},
// Browser projects depend on setup
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
{
name: 'firefox',
use: {
...devices['Desktop Firefox'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
],
});// tests/global.setup.ts
import { test as setup, expect } from '@playwright/test';
const authFile = 'playwright/.auth/user.json';
setup('authenticate', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill('user@example.com');
await page.getByLabel('Password').fill(process.env.TEST_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
await page.context().storageState({ path: authFile });
});How dependencies work: 1. Playwright identifies projects with no dependencies and runs them first. 2. Once a dependency project completes successfully, dependent projects start. 3. If a dependency project fails, all dependent projects are skipped.
Pattern 4: Multiple Auth Roles
Use when: Tests need different user roles (admin, editor, viewer) with separate auth states. Avoid when: All tests use the same user -- use a single setup project.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
projects: [
// Auth setup for each role
{
name: 'auth-admin',
testMatch: /auth\.setup\.ts/,
use: {
userRole: 'admin',
storageStatePath: 'playwright/.auth/admin.json',
},
},
{
name: 'auth-editor',
testMatch: /auth\.setup\.ts/,
use: {
userRole: 'editor',
storageStatePath: 'playwright/.auth/editor.json',
},
},
// Admin tests
{
name: 'admin-tests',
testDir: './tests/admin',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/admin.json',
},
dependencies: ['auth-admin'],
},
// Editor tests
{
name: 'editor-tests',
testDir: './tests/editor',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/editor.json',
},
dependencies: ['auth-editor'],
},
// Unauthenticated tests (no dependencies, no storageState)
{
name: 'public-tests',
testDir: './tests/public',
use: { ...devices['Desktop Chrome'] },
},
],
});// tests/auth.setup.ts
import { test as setup, expect } from '@playwright/test';
const credentials: Record<string, { email: string; password: string }> = {
admin: { email: 'admin@example.com', password: process.env.ADMIN_PASSWORD! },
editor: { email: 'editor@example.com', password: process.env.EDITOR_PASSWORD! },
};
setup('authenticate', async ({ page }, testInfo) => {
const role = testInfo.project.use.userRole as string;
const authFile = testInfo.project.use.storageStatePath as string;
const { email, password } = credentials[role];
await page.goto('/login');
await page.getByLabel('Email').fill(email);
await page.getByLabel('Password').fill(password);
await page.getByRole('button', { name: 'Sign in' }).click();
await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
await page.context().storageState({ path: authFile });
});Pattern 5: Environment-Specific Projects
Use when: Running the same tests against different environments (dev, staging, production) from one config. Avoid when: You only test one environment.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
const ENV = process.env.TEST_ENV || 'local';
const envConfig: Record<string, { baseURL: string; retries: number }> = {
local: { baseURL: 'http://localhost:3000', retries: 0 },
staging: { baseURL: 'https://staging.example.com', retries: 2 },
production: { baseURL: 'https://www.example.com', retries: 2 },
};
const env = envConfig[ENV];
export default defineConfig({
testDir: './tests',
retries: env.retries,
use: {
baseURL: env.baseURL,
},
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},
// Only run smoke tests in production
...(ENV === 'production'
? [{
name: 'smoke',
testMatch: '**/*smoke*.spec.ts',
use: { ...devices['Desktop Chrome'] },
grep: /@smoke/,
}]
: []),
],
});# Run against different environments
TEST_ENV=local npx playwright test
TEST_ENV=staging npx playwright test
TEST_ENV=production npx playwright test --project=smokePattern 6: Project with Custom testDir and testMatch
Use when: Different projects need different test directories or file patterns. Avoid when: All projects run the same tests.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
projects: [
// E2E tests -- run in all browsers
{
name: 'e2e-chromium',
testDir: './tests/e2e',
testMatch: '**/*.spec.ts',
use: { ...devices['Desktop Chrome'] },
},
{
name: 'e2e-firefox',
testDir: './tests/e2e',
testMatch: '**/*.spec.ts',
use: { ...devices['Desktop Firefox'] },
},
// API tests -- no browser needed, run once
{
name: 'api',
testDir: './tests/api',
testMatch: '**/*.spec.ts',
use: {
baseURL: 'https://api.example.com',
},
},
// Visual regression -- Chromium only
{
name: 'visual',
testDir: './tests/visual',
testMatch: '**/*.spec.ts',
use: {
...devices['Desktop Chrome'],
// Lock viewport for consistent screenshots
viewport: { width: 1280, height: 720 },
},
},
],
});# Run only API tests
npx playwright test --project=api
# Run only visual tests
npx playwright test --project=visual
# Run all E2E tests
npx playwright test --project=e2e-chromium --project=e2e-firefoxPattern 7: Teardown Projects
Use when: You need browser-based cleanup after tests complete (e.g., deleting test data via the UI). Avoid when: Cleanup can be done via API in globalTeardown or fixtures (the usual case).
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
projects: [
{
name: 'setup',
testMatch: /global\.setup\.ts/,
teardown: 'teardown', // link to teardown project
},
{
name: 'teardown',
testMatch: /global\.teardown\.ts/,
},
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
storageState: 'playwright/.auth/user.json',
},
dependencies: ['setup'],
},
],
});// tests/global.teardown.ts
import { test as teardown } from '@playwright/test';
teardown('clean up test data', async ({ request }) => {
await request.post('/api/test/cleanup', {
headers: { Authorization: `Bearer ${process.env.ADMIN_API_KEY}` },
});
});Execution order: 1. setup project runs 2. chromium project runs (depends on setup) 3. teardown project runs (linked via setup's teardown field)
Pattern 8: Grep and GrepInvert for Project Filtering
Use when: Different projects should run different subsets of tests based on tags. Avoid when: All projects run all tests.
// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';
export default defineConfig({
testDir: './tests',
projects: [
// Smoke tests -- only @smoke tagged tests, Chromium only
{
name: 'smoke',
use: { ...devices['Desktop Chrome'] },
grep: /@smoke/,
},
// Full regression -- everything except @slow, all browsers
{
name: 'regression-chromium',
use: { ...devices['Desktop Chrome'] },
grepInvert: /@slow/,
},
{
name: 'regression-firefox',
use: { ...devices['Desktop Firefox'] },
grepInvert: /@slow/,
},
// Slow tests -- only @slow, Chromium only, higher timeout
{
name: 'slow',
use: { ...devices['Desktop Chrome'] },
grep: /@slow/,
timeout: 120_000,
},
],
});# CI: run smoke on PRs
npx playwright test --project=smoke
# CI: run full regression on main
npx playwright test --project=regression-chromium --project=regression-firefox
# CI: run slow tests nightly
npx playwright test --project=slowDecision Guide
| Scenario | Number of Projects | Configuration |
|---|---|---|
| Getting started | 1 (chromium) | Single project, no dependencies |
| Cross-browser testing | 3 (chromium, firefox, webkit) | One per browser engine |
| Responsive design | 5-6 (desktop + mobile + tablet) | Use devices presets |
| Authenticated tests | 1 setup + N browser projects | Setup with dependencies |
| Multiple auth roles | N setup + N test projects | One setup project per role |
| Different test types (E2E, API, visual) | One per type | Custom testDir per project |
| Environment targeting | Same projects, different baseURL | Use TEST_ENV env var |
| Smoke vs regression suites | Projects with grep / grepInvert | Tag-based filtering |
| Feature | dependencies | teardown | globalSetup |
|---|---|---|---|
| Has browser context | Yes | Yes | No |
| Runs when | Before dependent projects | After linked setup project's dependents | Before all projects |
| Use for | Auth state, data seeding with browser | Browser-based cleanup | Non-browser setup (DB, health check) |
| Skips dependents on failure | Yes | N/A | Yes (entire suite fails) |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
| All browsers on every PR | 3x CI time for marginal benefit | Chromium on PRs; all browsers on main |
dependencies on projects that don't need shared state | Forced serial execution; slower | Only use dependencies when projects truly need shared state |
| Duplicating config across projects | Hard to maintain; settings drift | Use shared use at the top level; override per project |
No --project filtering in CI | All projects always run; no control | Use --project to run subsets based on context |
| Setup project modifies database without teardown | Dirty state for next run | Always pair setup with teardown or make setup idempotent |
| Many projects with overlapping test directories | Same test runs multiple times unintentionally | Set explicit testDir and testMatch per project |
Not using --no-deps for debugging | Setup runs every time, even for one test | Use --no-deps to skip dependencies during focused debugging |
Troubleshooting
"No tests found" when running specific project
Cause: The project's testDir or testMatch doesn't match any files.
Fix: List tests for the project to see what matches:
npx playwright test --project=chromium --listCheck that testDir and testMatch are correct:
{
name: 'chromium',
testDir: './tests', // must contain test files
testMatch: '**/*.spec.ts', // must match your file naming
}Setup project runs every time, even for a single test
Cause: The test's project has dependencies: ['setup'], so setup always runs first.
Fix: Use --no-deps to skip dependencies during development:
npx playwright test --project=chromium --no-deps tests/specific-test.spec.tsstorageState file not found
Cause: Setup project failed or didn't create the file. Or the path in use.storageState doesn't match the path in the setup project.
Fix: Verify paths match exactly:
// In setup test:
await page.context().storageState({ path: 'playwright/.auth/user.json' });
// In project config:
use: { storageState: 'playwright/.auth/user.json' } // must match exactlyAdd .auth to .gitignore:
echo "playwright/.auth/" >> .gitignoreDependent projects run even when setup fails
Cause: This should not happen -- Playwright skips dependent projects when a dependency fails. If it seems like they run, check that dependencies is spelled correctly.
Fix: Verify the dependency name matches exactly:
// Setup project name
{ name: 'setup', ... }
// Dependency reference (must match exactly, case-sensitive)
{ dependencies: ['setup'] } // correct
{ dependencies: ['Setup'] } // WRONG -- case mismatchTests from multiple projects interfere with each other
Cause: Projects share a database or external state. Since projects run in parallel by default, they can collide.
Fix: Either: 1. Use per-project isolated data (different test users, different database schemas) 2. Run projects sequentially by adding artificial dependencies:
projects: [
{ name: 'chromium', use: { ...devices['Desktop Chrome'] } },
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
dependencies: ['chromium'], // forces serial execution
},
],Custom use properties cause TypeScript errors
Cause: Playwright's use type doesn't include your custom properties.
Fix: Extend the type or use as any:
// Option 1: Declare custom properties
declare module '@playwright/test' {
interface PlaywrightTestOptions {
userRole: string;
storageStatePath: string;
}
}
// Option 2: Quick fix (less type-safe)
{
name: 'auth-admin',
use: {
userRole: 'admin',
storageStatePath: 'playwright/.auth/admin.json',
} as any,
}Related
- core/configuration.md -- base config,
projects,usesettings - ci/global-setup-teardown.md --
globalSetupvs setup projects - core/fixtures-and-hooks.md -- option fixtures configured per project
- core/authentication.md -- auth state via setup projects
- ci/parallel-and-sharding.md -- sharding across projects
- ci/ci-github-actions.md -- running specific projects in CI
Reporting and Artifacts
When to use: Configuring test output for local debugging, CI dashboards, and team visibility. Every project needs a reporting strategy from day one.
Quick Reference
# View the last HTML report
npx playwright show-report
# Run with specific reporter
npx playwright test --reporter=html
npx playwright test --reporter=dot # minimal CI output
npx playwright test --reporter=line # one line per test
npx playwright test --reporter=json # machine-readable
npx playwright test --reporter=junit # CI integration
# Multiple reporters via CLI
npx playwright test --reporter=dot,html
# Merge shard reports
npx playwright merge-reports --reporter=html ./blob-reportPatterns
Pattern 1: Multi-Reporter Configuration
Use when: Every project. You always want at least two reporters: one for humans, one for CI. Avoid when: Never -- always configure reporters.
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [
// CI: machine-readable + human-readable + CI annotations
['dot'], // minimal console output
['html', { open: 'never' }], // browsable report (uploaded as artifact)
['junit', { outputFile: 'test-results/junit.xml' }], // CI test tab integration
['github'], // PR annotations (GitHub Actions only)
]
: [
// Local: detailed console + auto-opening report
['list'], // verbose console output
['html', { open: 'on-failure' }], // auto-open on failure
],
});Pattern 2: Built-in Reporters in Detail
Use when: Choosing the right reporter for your context.
| Reporter | Output | Best For |
|---|---|---|
list | One line per test with pass/fail | Local development |
line | Updates a single line as tests complete | Local, less verbose |
dot | Single dot per test: . pass, F fail | CI logs (minimal) |
html | Interactive HTML page with traces | Post-run analysis |
json | Machine-readable JSON to stdout or file | Custom tooling, dashboards |
junit | JUnit XML | CI platforms (Azure DevOps, Jenkins, CircleCI) |
github | GitHub Actions annotations | GitHub PRs |
blob | Binary archive for shard merging | Sharded CI runs |
JSON reporter -- write to file:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['json', { outputFile: 'test-results/results.json' }],
],
});JUnit reporter -- customize output:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['junit', {
outputFile: 'test-results/junit.xml',
stripANSIControlSequences: true,
includeProjectInTestName: true,
}],
],
});Pattern 2b: Trace Retention For Flaky Tests (Playwright 1.59+)
Use when: A test sometimes fails and sometimes passes on retry, and you need artifacts from both attempts to compare behavior. Avoid when: Trace size matters more than diagnosis depth. In that case, keep using 'on-first-retry'.
Playwright 1.59 adds a new trace mode, 'retain-on-failure-and-retries', which records each run and keeps all traces when any attempt fails.
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
trace: process.env.CI
? 'retain-on-failure-and-retries'
: 'on-first-retry',
},
});This is excellent for flaky-test analysis because you can compare the failing run against a passing retry instead of guessing what changed between attempts.
Pattern 2c: Scoped HAR Artifacts For a Flow (Playwright 1.60+)
Use when: You want a network archive (HAR) attached to the report for one specific flow — e.g. a flaky third-party integration — without recording HAR for the entire context. Avoid when: A trace already gives you the network waterfall you need (traces include network detail). Reach for a dedicated HAR only when you need a portable .har to replay or hand to another tool.
Playwright 1.60 adds tracing.startHar() / tracing.stopHar(), which record HAR on demand inside the test. Capture only the suspect flow and attach it to the report:
import { test } from '@playwright/test';
test('payment integration', async ({ context, page }, testInfo) => {
const harPath = testInfo.outputPath('payment.har');
await context.tracing.startHar({ path: harPath, urlFilter: '**/payments/**' });
await page.goto('/checkout');
await page.getByRole('button', { name: 'Pay' }).click();
await context.tracing.stopHar();
await testInfo.attach('payment-network', { path: harPath, contentType: 'application/json' });
});The attached HAR shows up alongside traces and screenshots in the HTML report and CI artifacts. See core/network-mocking.md for the full API and await using cleanup.
Pattern 3: Custom Reporter
Use when: Built-in reporters don't meet your needs -- you want Slack notifications, database logging, or custom dashboards. Avoid when: A built-in reporter or existing third-party reporter covers your case.
// reporters/slack-reporter.ts
import type {
FullConfig,
FullResult,
Reporter,
Suite,
TestCase,
TestResult,
} from '@playwright/test/reporter';
class SlackReporter implements Reporter {
private passed = 0;
private failed = 0;
private skipped = 0;
private failures: string[] = [];
onTestEnd(test: TestCase, result: TestResult) {
switch (result.status) {
case 'passed':
this.passed++;
break;
case 'failed':
case 'timedOut':
this.failed++;
this.failures.push(`${test.title}: ${result.error?.message?.split('\n')[0]}`);
break;
case 'skipped':
this.skipped++;
break;
}
}
async onEnd(result: FullResult) {
const total = this.passed + this.failed + this.skipped;
const emoji = this.failed > 0 ? ':red_circle:' : ':large_green_circle:';
const text = [
`${emoji} *Playwright Tests*: ${result.status}`,
`Passed: ${this.passed} | Failed: ${this.failed} | Skipped: ${this.skipped} | Total: ${total}`,
`Duration: ${(result.duration / 1000).toFixed(1)}s`,
];
if (this.failures.length > 0) {
text.push('', '*Failures:*');
this.failures.slice(0, 5).forEach((f) => text.push(` - ${f}`));
if (this.failures.length > 5) {
text.push(` ...and ${this.failures.length - 5} more`);
}
}
const webhookUrl = process.env.SLACK_WEBHOOK_URL;
if (webhookUrl) {
await fetch(webhookUrl, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ text: text.join('\n') }),
});
}
}
}
export default SlackReporter;Register the custom reporter:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['dot'],
['html', { open: 'never' }],
['./reporters/slack-reporter.ts'],
],
});Pattern 4: Trace File Management
Use when: Debugging test failures. Traces capture a complete timeline of actions, network requests, DOM snapshots, and console logs. Avoid when: Never disable traces entirely in CI -- the on-first-retry setting has minimal overhead.
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
// 'on-first-retry': records trace only when a test fails and retries.
// Minimal overhead on passing tests, full debugging on failures.
trace: 'on-first-retry',
},
});Trace options:
| Value | Records trace | When | Overhead |
|---|---|---|---|
'off' | Never | -- | None |
'on' | Every test | Always | High (large files) |
'on-first-retry' | On first retry after failure | Retries only | Minimal |
'retain-on-failure' | Every test, keeps only failures | Failures | Medium |
'retain-on-first-failure' | Every test, keeps only first failure | First failure | Medium |
Viewing traces:
# Open trace viewer locally
npx playwright show-trace test-results/my-test/trace.zip
# Open trace from HTML report (click "Traces" tab in the report)
npx playwright show-report
# Online trace viewer (upload trace.zip)
# https://trace.playwright.devPattern 5: Screenshot and Video Configuration
Use when: Visual evidence of test failures is valuable for debugging or bug reports. Avoid when: Never disable screenshots in CI -- the on-failure setting is cheap.
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
use: {
// Screenshots
screenshot: 'only-on-failure', // capture final state on failure
// Video
video: 'retain-on-failure', // record all, keep only failures
// Video size (optional -- smaller = less disk)
video: {
mode: 'retain-on-failure',
size: { width: 1280, height: 720 },
},
},
});Screenshot options:
| Value | Captures | Disk cost |
|---|---|---|
'off' | Never | None |
'on' | Every test (at end) | High |
'only-on-failure' | Failed tests only | Low |
Video options:
| Value | Records | Keeps | Disk cost |
|---|---|---|---|
'off' | Never | -- | None |
'on' | Every test | All | Very high |
'on-first-retry' | On retry | Retried tests | Low |
'retain-on-failure' | Every test | Failed only | Medium |
Pattern 6: Artifact Organization for CI
Use when: Keeping test artifacts organized and accessible in CI.
Recommended directory structure:
test-results/ # Playwright's default output directory
├── my-test-chromium/
│ ├── trace.zip # Trace file
│ ├── test-failed-1.png # Screenshot
│ └── video.webm # Video recording
├── another-test-firefox/
│ ├── trace.zip
│ └── test-failed-1.png
└── junit.xml # JUnit report (if configured)
playwright-report/ # HTML report directory
├── index.html
└── data/
└── ...
blob-report/ # Blob report for shard merging
└── report-1.zipGitHub Actions artifact upload:
# Upload HTML report (always -- useful even when tests pass)
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 14
# Upload traces and screenshots (only on failure -- saves storage)
- uses: actions/upload-artifact@v4
if: failure()
with:
name: test-traces
path: |
test-results/**/trace.zip
test-results/**/*.png
test-results/**/*.webm
retention-days: 7Pattern 7: Allure Integration
Use when: Your team uses Allure for test reporting across multiple test frameworks. Avoid when: The built-in HTML reporter meets your needs (it usually does).
# Install Allure reporter
npm install -D allure-playwright// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [
['line'],
['allure-playwright', {
detail: true,
outputFolder: 'allure-results',
suiteTitle: true,
}],
],
});# Generate and view Allure report
npx allure generate allure-results -o allure-report --clean
npx allure open allure-report
# Or use Allure CLI
allure serve allure-resultsAdd Allure metadata to tests:
import { test, expect } from '@playwright/test';
import { allure } from 'allure-playwright';
test('checkout flow', async ({ page }) => {
await allure.epic('E-Commerce');
await allure.feature('Checkout');
await allure.story('Credit Card Payment');
await allure.severity('critical');
await page.goto('/checkout');
// ... test implementation
});Decision Guide
| Scenario | Reporter Configuration | Why |
|---|---|---|
| Local development | [['list'], ['html', { open: 'on-failure' }]] | Verbose console + auto-opening report on failure |
| GitHub Actions | [['dot'], ['html'], ['github']] | Minimal logs + report artifact + PR annotations |
| GitLab CI | [['dot'], ['html'], ['junit']] | Minimal logs + report artifact + test tab |
| Azure DevOps / Jenkins | [['dot'], ['html'], ['junit']] | JUnit for native test results integration |
| Sharded CI | [['blob'], ['github']] | Blob for merging; github for PR annotations |
| Team uses Allure | [['line'], ['allure-playwright']] | Cross-framework reporting consistency |
| Custom dashboard | [['json', { outputFile: '...' }]] + custom reporter | JSON for data, custom for notifications |
| Artifact | When to Collect | Retention | Upload Condition |
|---|---|---|---|
| HTML report | Always | 14 days | if: ${{ !cancelled() }} |
Traces (.zip) | On failure | 7 days | if: failure() |
Screenshots (.png) | On failure | 7 days | if: failure() |
Videos (.webm) | On failure | 7 days | if: failure() |
| JUnit XML | Always | 14 days | if: ${{ !cancelled() }} |
| Blob report | Always (sharded) | 1 day | if: ${{ !cancelled() }} |
Anti-Patterns
| Anti-Pattern | Problem | Do This Instead |
|---|---|---|
| No reporter configured | Default list only; no persistent report | Always configure html + one CI reporter |
trace: 'on' in CI | Massive artifacts (50-100 MB per test), slow uploads | Use trace: 'on-first-retry' |
video: 'on' in CI | Enormous storage cost; slows test execution | Use video: 'retain-on-failure' |
| Only uploading artifacts on failure | No report when tests pass; can't verify results | Upload with if: ${{ !cancelled() }} (always) |
| No retention limits on artifacts | CI storage fills up within weeks | Set retention-days: 7-14 |
Using only dot reporter with no HTML | Can't drill into failures after the run | Always pair dot with html in CI |
| JUnit output to stdout | Interferes with console output; hard to parse | Write to file: ['junit', { outputFile: 'results/junit.xml' }] |
Custom reporter that blocks onEnd | Slow Slack/HTTP calls delay pipeline completion | Use Promise.race with a timeout in custom reporters |
Troubleshooting
HTML report is empty or missing tests
Cause: Another reporter is conflicting, or outputFolder was overridden to a non-default path.
Fix: Check your reporter config. The HTML report defaults to playwright-report/:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: [['html', { outputFolder: 'playwright-report', open: 'never' }]],
});Traces are too large for CI artifact upload
Cause: trace: 'on' records every test, even passing ones.
Fix: Switch to 'on-first-retry' and ensure retries > 0 in CI:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
retries: process.env.CI ? 2 : 0,
use: {
trace: 'on-first-retry',
},
});JUnit XML not recognized by CI platform
Cause: Output path doesn't match what the CI task expects, or the file is empty.
Fix: Ensure the path matches your CI configuration:
// playwright.config.ts -- the outputFile path
reporter: [['junit', { outputFile: 'test-results/junit.xml' }]],# GitHub Actions
- uses: dorny/test-reporter@v1
with:
path: test-results/junit.xml
reporter: java-junit
# Azure DevOps
- task: PublishTestResults@2
inputs:
testResultsFiles: 'test-results/junit.xml'
# Jenkins
junit 'test-results/junit.xml'merge-reports produces empty report
Cause: Shards are using html reporter instead of blob. Only blob output can be merged.
Fix: Use blob reporter for sharded runs:
// playwright.config.ts
import { defineConfig } from '@playwright/test';
export default defineConfig({
reporter: process.env.CI
? [['blob'], ['dot']] // blob for merge, dot for console
: [['html', { open: 'on-failure' }]],
});Screenshots not appearing in HTML report
Cause: screenshot: 'off' or screenshots are in test-results/ but not linked to the report.
Fix: Enable screenshots and ensure both directories are available:
use: {
screenshot: 'only-on-failure',
},The HTML report automatically embeds screenshots from test-results/. If you move or delete test-results/, screenshots will be missing from the report.
Related
- ci/ci-github-actions.md -- artifact upload in GitHub Actions
- ci/ci-gitlab.md -- artifact configuration in GitLab
- ci/parallel-and-sharding.md -- blob reporter for sharded runs
- core/configuration.md -- trace, screenshot, video settings
- core/debugging.md -- using traces and screenshots for debugging