Umbraco E2e Testing

Umbraco E2E Testing

End-to-end testing for Umbraco backoffice extensions using Playwright and @umbraco/playwright-testhelpers. This approach tests against a real running Umbraco instance, validating complete user workflows.

Critical: Use Testhelpers for Core Umbraco

Use @umbraco/playwright-testhelpers for core Umbraco operations:

Why use testhelpers for core Umbraco?

• Umbraco uses data-mark instead of data-testid - testhelpers handle this
• Auth token management is complex - testhelpers manage STORAGE_STAGE_PATH
• API setup/teardown requires specific payload formats - builders ensure correctness
• Selectors change between versions - testhelpers abstract these away

// WRONG - Raw Playwright for core Umbraco (brittle)
await page.goto('/umbraco');
await page.fill('[name="email"]', 'admin@example.com');

// CORRECT - Testhelpers for core Umbraco
import { test } from '@umbraco/playwright-testhelpers';

test('my test', async ({ umbracoApi, umbracoUi }) => {
  await umbracoUi.goToBackOffice();
  await umbracoUi.login.enterEmail('admin@example.com');
});

When to Use Raw Playwright

For custom extensions, use umbracoUi.page (raw Playwright) because testhelpers don't know about your custom elements:

test('my custom extension', async ({ umbracoUi }) => {
  // Testhelpers for core navigation
  await umbracoUi.goToBackOffice();
  await umbracoUi.content.goToSection(ConstantHelper.sections.settings);

  // Raw Playwright for YOUR custom elements
  await umbracoUi.page.getByRole('link', { name: 'My Custom Item' }).click();
  await expect(umbracoUi.page.locator('my-custom-workspace')).toBeVisible();
});

When to Use

• Testing complete user workflows
• Testing data persistence
• Testing authentication/authorization
• Acceptance testing before release
• Integration testing with real API responses

Related Skills

• umbraco-testing - Master skill for testing overview
• umbraco-playwright-testhelpers - Full reference for the testhelpers package
• umbraco-test-builders - JsonModels.Builders for test data
• umbraco-mocked-backoffice - Test without real backend (faster)

Documentation

• Playwright: https://playwright.dev/docs/intro
• Reference tests: Umbraco-CMS/tests/Umbraco.Tests.AcceptanceTest

---

Setup

Dependencies

Add to package.json:

{
  "devDependencies": {
    "@playwright/test": "^1.56",
    "@umbraco/playwright-testhelpers": "^17.0.15",
    "@umbraco/json-models-builders": "^2.0.42",
    "dotenv": "^16.3.1"
  },
  "scripts": {
    "test:e2e": "playwright test",
    "test:e2e:ui": "playwright test --ui",
    "test:e2e:debug": "playwright test --debug"
  }
}

Then run:

npm install
npx playwright install chromium

Version Compatibility: Match testhelpers to your Umbraco version:

Configuration

Create playwright.config.ts:

import { defineConfig, devices } from '@playwright/test';
import { fileURLToPath } from 'url';
import { dirname, join } from 'path';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

export const STORAGE_STATE = join(__dirname, 'tests/e2e/.auth/user.json');

// CRITICAL: Testhelpers read auth tokens from this file
process.env.STORAGE_STAGE_PATH = STORAGE_STATE;

export default defineConfig({
  testDir: './tests/e2e',
  timeout: 30 * 1000,
  expect: { timeout: 5000 },
  fullyParallel: false,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: 1,
  reporter: process.env.CI ? 'line' : 'html',
  use: {
    baseURL: process.env.UMBRACO_URL || 'https://localhost:44325',
    trace: 'retain-on-failure',
    ignoreHTTPSErrors: true,
    // CRITICAL: Umbraco uses 'data-mark' not 'data-testid'
    testIdAttribute: 'data-mark',
  },
  projects: [
    {
      name: 'setup',
      testMatch: '**/*.setup.ts',
    },
    {
      name: 'e2e',
      testMatch: '**/*.spec.ts',
      dependencies: ['setup'],
      use: {
        ...devices['Desktop Chrome'],
        ignoreHTTPSErrors: true,
        storageState: STORAGE_STATE,
      },
    },
  ],
});

Critical Settings

Without `testIdAttribute: 'data-mark'`, all `getByTestId()` calls will fail.

Authentication Setup

Create tests/e2e/auth.setup.ts:

import { test as setup } from '@playwright/test';
import { STORAGE_STATE } from '../../playwright.config';
import { ConstantHelper, UiHelpers } from '@umbraco/playwright-testhelpers';

setup('authenticate', async ({ page }) => {
  const umbracoUi = new UiHelpers(page);

  await umbracoUi.goToBackOffice();
  await umbracoUi.login.enterEmail(process.env.UMBRACO_USER_LOGIN!);
  await umbracoUi.login.enterPassword(process.env.UMBRACO_USER_PASSWORD!);
  await umbracoUi.login.clickLoginButton();
  await umbracoUi.login.goToSection(ConstantHelper.sections.settings);
  await page.context().storageState({ path: STORAGE_STATE });
});

Environment Variables

Create .env (add to .gitignore):

UMBRACO_URL=https://localhost:44325
UMBRACO_USER_LOGIN=admin@example.com
UMBRACO_USER_PASSWORD=yourpassword
UMBRACO_DATA_PATH=/path/to/Umbraco.Web.UI/App_Data  # Optional: for data reset

Directory Structure

my-extension/
├── src/
│   └── ...
├── tests/
│   └── e2e/
│       ├── .auth/
│       │   └── user.json       # Auth state (gitignored)
│       ├── auth.setup.ts       # Authentication
│       └── my-extension.spec.ts
├── playwright.config.ts
├── .env                        # Gitignored
├── .env.example
└── package.json

---

Patterns

Test Fixtures

import { test } from '@umbraco/playwright-testhelpers';

test('my test', async ({ umbracoApi, umbracoUi }) => {
  // umbracoApi - API helpers for setup/teardown
  // umbracoUi - UI helpers for backoffice interaction
});

AAA Pattern (Arrange-Act-Assert)

test('can create content', async ({ umbracoApi, umbracoUi }) => {
  // Arrange - Setup via API
  await umbracoApi.documentType.createDefaultDocumentType('TestDocType');

  // Act - Perform user actions via UI
  await umbracoUi.goToBackOffice();
  await umbracoUi.content.goToSection(ConstantHelper.sections.content);
  await umbracoUi.content.clickActionsMenuAtRoot();

  // Assert - Verify results
  expect(await umbracoApi.document.doesNameExist('TestContent')).toBeTruthy();
});

Idempotent Cleanup

test.afterEach(async ({ umbracoApi }) => {
  await umbracoApi.document.ensureNameNotExists(contentName);
  await umbracoApi.documentType.ensureNameNotExists(documentTypeName);
});

API Helpers (umbracoApi)

Document Types:

await umbracoApi.documentType.createDefaultDocumentType('TestDocType');
await umbracoApi.documentType.createDocumentTypeWithPropertyEditor(
  'TestDocType', 'Textstring', dataTypeData.id
);
await umbracoApi.documentType.ensureNameNotExists('TestDocType');

Documents:

await umbracoApi.document.createDefaultDocument('TestContent', docTypeId);
await umbracoApi.document.createDocumentWithTextContent(
  'TestContent', docTypeId, 'value', 'Textstring'
);
await umbracoApi.document.publish(contentId);
await umbracoApi.document.ensureNameNotExists('TestContent');

Data Types:

const dataType = await umbracoApi.dataType.getByName('Textstring');
await umbracoApi.dataType.create('MyType', 'Umbraco.TextBox', 'Umb.PropertyEditorUi.TextBox', []);

Using Builders for Complex Data

For complex test data, use @umbraco/json-models-builders:

import { DocumentTypeBuilder, DocumentBuilder } from '@umbraco/json-models-builders';

test('create complex document type', async ({ umbracoApi }) => {
  // Build a document type with multiple properties
  const docType = new DocumentTypeBuilder()
    .withName('Article')
    .withAlias('article')
    .addGroup()
      .withName('Content')
      .addTextBoxProperty()
        .withAlias('title')
        .withLabel('Title')
        .done()
      .addRichTextProperty()
        .withAlias('body')
        .withLabel('Body')
        .done()
      .done()
    .build();

  await umbracoApi.documentType.create(docType);
});

Why use builders?

• Fluent API makes complex structures readable
• Ensures valid payload structure for Umbraco API
• Handles required fields and defaults
• Type-safe in TypeScript

UI Helpers (umbracoUi)

Navigation:

await umbracoUi.goToBackOffice();
await umbracoUi.content.goToSection(ConstantHelper.sections.content);
await umbracoUi.content.goToContentWithName('My Page');

Testing Custom Trees in Sidebar

When testing custom tree extensions (e.g., in Settings), use this pattern to handle async loading and scrolling:

test('should click custom tree item', async ({ umbracoUi }) => {
  await umbracoUi.goToBackOffice();
  await umbracoUi.content.goToSection(ConstantHelper.sections.settings);

  // 1. Wait for your tree heading (custom trees often at bottom of sidebar)
  await umbracoUi.page.getByRole('heading', { name: 'My Tree' }).waitFor({ timeout: 15000 });

  // 2. Scroll into view (important - sidebar may be long)
  await umbracoUi.page.getByRole('heading', { name: 'My Tree' }).scrollIntoViewIfNeeded();

  // 3. Wait for tree items to load (async from API)
  const item1Link = umbracoUi.page.getByRole('link', { name: 'Item 1' });
  await item1Link.waitFor({ timeout: 15000 });

  // 4. Click the item
  await item1Link.click();

  // Assert workspace loads
  await expect(umbracoUi.page.locator('my-tree-workspace-editor')).toBeVisible({ timeout: 15000 });
});

Why this pattern?

• Custom trees are often at the bottom of the Settings sidebar
• Tree items load asynchronously from your API
• Using getByRole('link', { name: '...' }) is more reliable than generic umb-tree-item selectors
• Built-in trees (Document Types, etc.) also use umb-tree-item, causing selector conflicts

Content Actions:

await umbracoUi.content.clickActionsMenuAtRoot();
await umbracoUi.content.clickCreateActionMenuOption();
await umbracoUi.content.chooseDocumentType('TestDocType');
await umbracoUi.content.enterContentName('My Page');
await umbracoUi.content.enterTextstring('My text value');
await umbracoUi.content.clickSaveButton();

Constants:

import { ConstantHelper } from '@umbraco/playwright-testhelpers';

ConstantHelper.sections.content
ConstantHelper.sections.settings
ConstantHelper.buttons.save
ConstantHelper.buttons.saveAndPublish

---

Examples

Complete Test

import { expect } from '@playwright/test';
import { ConstantHelper, NotificationConstantHelper, test } from '@umbraco/playwright-testhelpers';

const contentName = 'TestContent';
const documentTypeName = 'TestDocType';
const dataTypeName = 'Textstring';
const contentText = 'Test content text';

test.afterEach(async ({ umbracoApi }) => {
  await umbracoApi.document.ensureNameNotExists(contentName);
  await umbracoApi.documentType.ensureNameNotExists(documentTypeName);
});

test('can create content', { tag: '@smoke' }, async ({ umbracoApi, umbracoUi }) => {
  // Arrange
  const dataTypeData = await umbracoApi.dataType.getByName(dataTypeName);
  await umbracoApi.documentType.createDocumentTypeWithPropertyEditor(
    documentTypeName, dataTypeName, dataTypeData.id
  );

  // Act
  await umbracoUi.goToBackOffice();
  await umbracoUi.content.goToSection(ConstantHelper.sections.content);
  await umbracoUi.content.clickActionsMenuAtRoot();
  await umbracoUi.content.clickCreateActionMenuOption();
  await umbracoUi.content.chooseDocumentType(documentTypeName);
  await umbracoUi.content.enterContentName(contentName);
  await umbracoUi.content.enterTextstring(contentText);
  await umbracoUi.content.clickSaveButton();

  // Assert
  await umbracoUi.content.waitForContentToBeCreated();
  expect(await umbracoApi.document.doesNameExist(contentName)).toBeTruthy();
  const contentData = await umbracoApi.document.getByName(contentName);
  expect(contentData.values[0].value).toBe(contentText);
});

test('can publish content', async ({ umbracoApi, umbracoUi }) => {
  // Arrange
  const dataTypeData = await umbracoApi.dataType.getByName(dataTypeName);
  const docTypeId = await umbracoApi.documentType.createDocumentTypeWithPropertyEditor(
    documentTypeName, dataTypeName, dataTypeData.id
  );
  await umbracoApi.document.createDocumentWithTextContent(
    contentName, docTypeId, contentText, dataTypeName
  );

  // Act
  await umbracoUi.goToBackOffice();
  await umbracoUi.content.goToSection(ConstantHelper.sections.content);
  await umbracoUi.content.clickActionsMenuForContent(contentName);
  await umbracoUi.content.clickPublishActionMenuOption();
  await umbracoUi.content.clickConfirmToPublishButton();

  // Assert
  await umbracoUi.content.doesSuccessNotificationHaveText(
    NotificationConstantHelper.success.published
  );
  const contentData = await umbracoApi.document.getByName(contentName);
  expect(contentData.variants[0].state).toBe('Published');
});

Working Example: tree-example

The tree-example demonstrates E2E testing for a custom tree extension:

Location: umbraco-backoffice/examples/tree-example/Client/

# Run E2E tests (requires running Umbraco)
URL=https://localhost:44325 \
UMBRACO_USER_LOGIN=admin@example.com \
UMBRACO_USER_PASSWORD=yourpassword \
npm run test:e2e                # 7 tests

Key files:

• tests/playwright.e2e.config.ts - E2E configuration with auth setup
• tests/auth.setup.ts - Authentication using testhelpers
• tests/tree-e2e.spec.ts - Tests for custom tree in Settings sidebar

Working Example: notes-wiki (Full-Stack with Data Reset)

The notes-wiki demonstrates E2E testing with persistent data and CRUD operations:

Location: umbraco-backoffice/examples/notes-wiki/Client/

# Run E2E tests (with data reset)
URL=https://localhost:44325 \
UMBRACO_USER_LOGIN=admin@example.com \
UMBRACO_USER_PASSWORD=yourpassword \
UMBRACO_DATA_PATH=/path/to/App_Data \
npm run test:e2e                # 16 tests

Key files:

• tests/playwright.e2e.config.ts - Config with globalSetup for data reset
• tests/global-setup.ts - Resets data to seed state before tests
• tests/test-seed-data.json - Known test data (notes, folders)
• tests/notes-wiki-e2e.spec.ts - CRUD and navigation tests

What it demonstrates:

• Testing a custom section using goToSection('notes')
• Resetting file-based data before each test run
• Testing tree navigation, folders, and workspaces
• Entity actions via "View actions" button (more reliable than right-click)
• Dashboard and workspace view testing

---

Testing Extensions with Persistent Data

When your extension persists data (JSON files, database, etc.), tests need predictable starting state.

Global Setup Pattern

Add globalSetup to reset data before tests:

playwright.e2e.config.ts:

export default defineConfig({
  // ... other config
  globalSetup: './global-setup.ts',
});

global-setup.ts:

import { FullConfig } from '@playwright/test';
import * as fs from 'fs';
import * as path from 'path';

async function globalSetup(config: FullConfig) {
  const dataPath = process.env.UMBRACO_DATA_PATH;
  if (!dataPath) {
    console.warn('⚠️  UMBRACO_DATA_PATH not set. Skipping data reset.');
    return;
  }

  const targetFile = path.join(dataPath, 'MyExtension/data.json');
  const seedFile = path.join(__dirname, 'test-seed-data.json');

  // Ensure directory exists
  fs.mkdirSync(path.dirname(targetFile), { recursive: true });

  // Copy seed data to target
  fs.copyFileSync(seedFile, targetFile);
  console.log('🌱 Reset data to seed state');
}

export default globalSetup;

test-seed-data.json:

{
  "items": [
    { "id": "test-1", "name": "Test Item 1" },
    { "id": "test-2", "name": "Test Item 2" }
  ]
}

Environment Variable

Add UMBRACO_DATA_PATH to locate your Umbraco's App_Data folder:

UMBRACO_DATA_PATH=/path/to/Umbraco.Web.UI/App_Data npm run test:e2e

---

Testing Custom Sections

Custom sections work with testhelpers' goToSection() method - pass the section pathname:

// Section pathname - matches what you defined in section/constants.ts
const MY_SECTION = 'my-section';

// Helper to navigate to custom section using testhelpers
async function goToMySection(umbracoUi: any) {
  await umbracoUi.goToBackOffice();
  await umbracoUi.content.goToSection(MY_SECTION);
  await umbracoUi.page.waitForTimeout(500);
}

test('should navigate to custom section', async ({ umbracoUi }) => {
  await goToMySection(umbracoUi);

  // Assert - your dashboard or tree should be visible
  await expect(umbracoUi.page.getByText('Welcome')).toBeVisible({ timeout: 15000 });
});

// To verify the section exists in the section bar:
test('should display my section', async ({ umbracoUi }) => {
  await umbracoUi.goToBackOffice();
  await expect(umbracoUi.page.getByRole('tab', { name: 'My Section' })).toBeVisible({ timeout: 15000 });
});

---

Context Menu (Entity Actions) Testing

Testing entity actions on tree items. Uses umbracoUi.page since testhelpers don't cover custom entity actions.

Important: Entity actions in Umbraco are rendered as buttons inside the dropdown menu, not as menuitem roles directly. The most reliable approach is to use the "View actions" button rather than right-click:

test('should show delete action via actions button', async ({ umbracoUi }) => {
  await goToMySection(umbracoUi);

  // Wait for tree item
  const itemLink = umbracoUi.page.getByRole('link', { name: 'My Item' });
  await itemLink.waitFor({ timeout: 15000 });

  // Hover to reveal action buttons
  await itemLink.hover();

  // Click the "View actions" button to open dropdown
  const actionsButton = umbracoUi.page.getByRole('button', { name: "View actions for 'My Item'" });
  await actionsButton.click();

  // Wait for dropdown and check for actions (actions are BUTTONS, not menuitems!)
  await umbracoUi.page.waitForTimeout(500);
  const deleteButton = umbracoUi.page.getByRole('button', { name: 'Delete' });
  const renameButton = umbracoUi.page.getByRole('button', { name: 'Rename' });

  // Assert - at least one action should be visible
  await expect(deleteButton.or(renameButton)).toBeVisible({ timeout: 5000 });
});

test('should delete item via actions menu', async ({ umbracoUi }) => {
  await goToMySection(umbracoUi);

  const itemLink = umbracoUi.page.getByRole('link', { name: 'Item to Delete' });
  await itemLink.waitFor({ timeout: 15000 });

  // Hover and open actions menu
  await itemLink.hover();
  await umbracoUi.page.getByRole('button', { name: "View actions for 'Item to Delete'" }).click();

  // Click delete button
  await umbracoUi.page.getByRole('button', { name: 'Delete' }).click();

  // Confirm deletion (if modal appears)
  const confirmButton = umbracoUi.page.getByRole('button', { name: /Confirm|Delete/i });
  if (await confirmButton.isVisible({ timeout: 2000 }).catch(() => false)) {
    await confirmButton.click();
  }

  // Assert - item should be gone
  await expect(itemLink).not.toBeVisible({ timeout: 15000 });
});

Alternative: Right-Click Context Menu

Right-click also works but the actions button approach is more reliable:

// Right-click approach (less reliable than actions button)
await itemLink.click({ button: 'right' });
await umbracoUi.page.waitForTimeout(500);
await umbracoUi.page.getByRole('button', { name: 'Delete' }).click();

---

CRUD Testing Patterns

For custom extensions, use umbracoUi.page for UI interactions. For core Umbraco content, prefer umbracoApi helpers for setup/teardown.

Create via Actions Menu (Custom Extension)

test('should create new item', async ({ umbracoUi }) => {
  await goToMySection(umbracoUi);

  // Hover over parent folder and use Create button
  const folderLink = umbracoUi.page.getByRole('link', { name: 'Parent Folder' });
  await folderLink.hover();

  // Scope to specific menu to avoid ambiguity with multiple items
  const folderMenu = umbracoUi.page.getByRole('menu').filter({ hasText: 'Parent Folder' });
  const createButton = folderMenu.getByRole('button', { name: 'Create Note' });
  await createButton.click();

  // Assert - workspace should open with "New" indicator
  await expect(umbracoUi.page.locator('my-workspace-editor')).toBeVisible({ timeout: 15000 });
});

Scoping to Specific Tree Items

When multiple tree items exist with similar elements, scope selectors to avoid ambiguity:

// WRONG - ambiguous when multiple folders have "Create" buttons
const createButton = page.getByRole('button', { name: 'Create' });

// CORRECT - scoped to specific folder's menu
const folderMenu = page.getByRole('menu').filter({ hasText: 'My Folder' });
const createButton = folderMenu.getByRole('button', { name: 'Create' });

Update and Save

test('should update item', async ({ umbracoUi }) => {
  await goToMySection(umbracoUi);

  // Navigate to item
  await umbracoUi.page.getByRole('link', { name: 'Test Item' }).click();
  await umbracoUi.page.locator('my-workspace-editor').waitFor({ timeout: 15000 });

  // Update field
  const titleInput = umbracoUi.page.locator('uui-input#title');
  await titleInput.clear();
  await titleInput.fill('Updated Title');

  // Save
  await umbracoUi.page.getByRole('button', { name: /Save/i }).click();

  // Wait for save to complete
  await umbracoUi.page.waitForTimeout(2000);

  // Assert - header should reflect change
  await expect(umbracoUi.page.getByText('Updated Title')).toBeVisible();
});

---

Running Tests

# Run all E2E tests
npm run test:e2e

# Run with UI mode (visual debugging)
npm run test:e2e:ui

# Run specific test file
npx playwright test tests/e2e/my-extension.spec.ts

# Run with specific tag
npx playwright test --grep "@smoke"

# Run in debug mode
npx playwright test --debug

---

Troubleshooting

getByTestId() not finding elements

Ensure testIdAttribute: 'data-mark' is set in playwright.config.ts.

Authentication fails

• Check .env credentials are correct
• Ensure Umbraco instance is running
• Verify STORAGE_STAGE_PATH is set

Tests timeout

• Increase timeouts in config
• Ensure Umbraco is responsive
• Check for JS errors in browser console

Tests fail in CI

• Ensure Umbraco instance is accessible
• Set environment variables in CI
• Use npx playwright install chromium

---

Alternative: MSW Mode (No Backend Required)

For faster testing without a real Umbraco backend, use the mocked backoffice approach.

Invoke: skill: umbraco-mocked-backoffice

Document Type CRUD - E2E Testing Example

This example demonstrates how to write E2E tests for Umbraco backoffice using Playwright with @umbraco/playwright-testhelpers.

What This Example Shows

This example demonstrates:

• Test Setup - Using @umbraco/playwright-testhelpers fixtures
• API Helpers - Fast test data creation via umbracoApi
• UI Helpers - Page interactions via umbracoUi
• AAA Pattern - Arrange-Act-Assert test structure
• Idempotent Cleanup - Using ensureNameNotExists for reliable teardown
• Authentication Setup - Separate setup project for login

Files Included

Project Structure

document-type-crud/
├── tests/
│   ├── auth.setup.ts           # Login and save auth state
│   └── document-type.spec.ts   # Document type tests
├── playwright/
│   └── .auth/
│       └── user.json           # Saved auth state (generated)
├── playwright.config.ts
├── umbraco.config.ts
├── package.json
└── README.md

Prerequisites

1. Running Umbraco instance - Tests run against a real Umbraco backoffice 2. Admin credentials - Valid username/password for the instance 3. Matching testhelpers version - The @umbraco/playwright-testhelpers version must match your Umbraco version

Environment Variables

Running the Tests

# Install dependencies
npm install

# Install Playwright browsers
npx playwright install chromium

# Run tests (with environment variables)
URL=https://localhost:44325 \
UMBRACO_USER_LOGIN=admin@example.com \
UMBRACO_USER_PASSWORD=yourpassword \
npm test

# Run with UI mode for debugging
npm run test:ui

# Run in debug mode
npm run test:debug

Critical Configuration

Test ID Attribute

Umbraco uses data-mark as its test ID attribute (not the Playwright default data-testid). This must be configured in playwright.config.ts:

export default defineConfig({
  use: {
    // CRITICAL: Umbraco uses 'data-mark' for test IDs
    testIdAttribute: 'data-mark',
    ignoreHTTPSErrors: true,
  },
});

Without this setting, the testhelpers' getByTestId() calls will fail to find elements.

Storage State Path

The testhelpers need to know where the auth state is stored. Set the STORAGE_STAGE_PATH environment variable in your config:

export const STORAGE_STATE = join(__dirname, 'playwright/.auth/user.json');
process.env.STORAGE_STAGE_PATH = STORAGE_STATE;

Test Patterns

Authentication Setup

The auth.setup.ts file uses testhelpers to log in and save the session:

import { test as setup } from '@playwright/test';
import { STORAGE_STATE } from '../playwright.config';
import { ConstantHelper, UiHelpers } from '@umbraco/playwright-testhelpers';

setup('authenticate', async ({ page }) => {
  const umbracoUi = new UiHelpers(page);

  await umbracoUi.goToBackOffice();
  await umbracoUi.login.enterEmail(process.env.UMBRACO_USER_LOGIN);
  await umbracoUi.login.enterPassword(process.env.UMBRACO_USER_PASSWORD);
  await umbracoUi.login.clickLoginButton();
  await umbracoUi.login.goToSection(ConstantHelper.sections.settings);
  await umbracoUi.page.context().storageState({ path: STORAGE_STATE });
});

Test with Fixtures

import { ConstantHelper, test } from '@umbraco/playwright-testhelpers';
import { expect } from '@playwright/test';

const documentTypeName = 'TestDocumentType';

test.beforeEach(async ({ umbracoApi, umbracoUi }) => {
  await umbracoApi.documentType.ensureNameNotExists(documentTypeName);
  await umbracoUi.goToBackOffice();
});

test.afterEach(async ({ umbracoApi }) => {
  await umbracoApi.documentType.ensureNameNotExists(documentTypeName);
});

test('can create a document type', { tag: '@smoke' }, async ({ umbracoApi, umbracoUi }) => {
  // Arrange
  await umbracoUi.documentType.goToSection(ConstantHelper.sections.settings);

  // Act
  await umbracoUi.documentType.clickActionsMenuAtRoot();
  await umbracoUi.documentType.clickCreateActionMenuOption();
  await umbracoUi.documentType.clickCreateDocumentTypeButton();
  await umbracoUi.documentType.enterDocumentTypeName(documentTypeName);
  await umbracoUi.documentType.clickSaveButton();

  // Assert
  await umbracoUi.documentType.waitForDocumentTypeToBeCreated();
  expect(await umbracoApi.documentType.doesNameExist(documentTypeName)).toBeTruthy();
});

Idempotent Cleanup

test.afterEach(async ({ umbracoApi }) => {
  // Won't fail if item doesn't exist
  await umbracoApi.documentType.ensureNameNotExists('TestDocType');
});

Version Compatibility

The @umbraco/playwright-testhelpers package is versioned to match Umbraco:

Important: Use the testhelpers version that matches your Umbraco instance. For pre-release Umbraco versions, use the corresponding beta testhelpers.

Troubleshooting

"Error refreshing access token"

This usually means there's a version mismatch between the testhelpers and your Umbraco instance, or the authentication state is invalid. Try:

1. Delete playwright/.auth/user.json and re-run tests 2. Verify your Umbraco instance is running 3. Check that credentials are correct 4. Ensure STORAGE_STAGE_PATH is set correctly

"Invalid character in header content"

This is a known issue with certain testhelpers versions. Ensure you're using a compatible version for your Umbraco instance.

Tests timeout waiting for selectors

The testhelpers use specific selectors that may change between Umbraco versions. If you see locator timeouts:

1. Check `testIdAttribute` - Must be set to 'data-mark' in playwright.config.ts 2. Check the Umbraco UI has loaded correctly 3. Verify you're using the correct testhelpers version 4. Check the Umbraco acceptance tests in the source for current patterns

"element(s) not found" errors

Most likely the testIdAttribute is not set correctly. Umbraco uses data-mark, not data-testid.

Reference

• Umbraco Acceptance Tests: Umbraco-CMS/tests/Umbraco.Tests.AcceptanceTest
• Testhelpers Repo: https://github.com/umbraco/Umbraco.Playwright.Testhelpers
• Playwright Docs: https://playwright.dev/docs/intro

Skills Referenced