Phoenix Playwright Tests

Phoenix Playwright Test Writing

Write end-to-end tests for Phoenix using Playwright. Tests live in app/tests/ and follow established patterns.

Timeout Policy

• Do not pass timeout args in test code under app/tests.
• Tune timing centrally in app/playwright.config.ts (global timeout, expect.timeout, use.navigationTimeout, and webServer.timeout).

Quick Start

import { expect, test } from "@playwright/test";
import { randomUUID } from "crypto";

test.describe("Feature Name", () => {
  test.beforeEach(async ({ page }) => {
    await page.goto(`/login`);
    await page.getByLabel("Email").fill("admin@localhost");
    await page.getByLabel("Password").fill("admin123");
    await page.getByRole("button", { name: "Log In", exact: true }).click();
    await page.waitForURL("**/projects");
  });

  test("can do something", async ({ page }) => {
    // Test implementation
  });
});

Test Credentials

Selector Patterns (Priority Order)

1. Role selectors (most robust):

   page.getByRole("button", { name: "Save" });
   page.getByRole("link", { name: "Datasets" });
   page.getByRole("tab", { name: /Evaluators/i });
   page.getByRole("menuitem", { name: "Edit" });
   page.getByRole("cell", { name: "my-item" });
   page.getByRole("heading", { name: "Title" });
   page.getByRole("dialog");
   page.getByRole("textbox", { name: "Name" });
   page.getByRole("combobox", { name: /mapping/i });

2. Label selectors:

   page.getByLabel("Email");
   page.getByLabel("Dataset Name");
   page.getByLabel("Description");

3. Text selectors:

   page.getByText("No evaluators added");
   page.getByPlaceholder("Search...");

4. Test IDs (when available):

   page.getByTestId("create-dataset-button");
   // element with state — select the stable id, filter on the data attribute
   page.locator('[data-testid="llm-evaluator-form-submit-button"][data-mode="create"]');

data-testids are scoped, fully spelled out (...-button, never ...-btn), and constant regardless of state — state is exposed via a sibling data-* attribute (data-mode, data-state, …), so never key a getByTestId off a value that only exists in one mode. If you need a data-testid that doesn't exist yet, add it following references/test-ids.md in the phoenix-frontend skill (pattern: <scope>-<subject>-<role>).

5. CSS locators (last resort):

   page.locator('button:has-text("Save")');

Common UI Patterns

Dropdown Menus

// Click button to open dropdown
await page.getByRole("button", { name: "New Dataset" }).click();
// Select menu item
await page.getByRole("menuitem", { name: "New Dataset" }).click();

Nested Menus (Submenus)

// Open menu, hover over submenu trigger, click submenu item
await page.getByRole("button", { name: "Add evaluator" }).click();
await page
  .getByRole("menuitem", { name: "Use LLM evaluator template" })
  .hover();
await page.getByRole("menuitem", { name: /correctness/i }).click();

// IMPORTANT: Always use getByRole("menuitem") for submenu items, not getByText()
// Playwright's auto-waiting handles the submenu appearance timing
// ❌ BAD - flaky in CI:
// await page.getByText("ExactMatch").first().click();
// ✅ GOOD - reliable:
// await page.getByRole("menuitem", { name: /ExactMatch/i }).click();

Dialogs/Modals

// Wait for dialog
await expect(page.getByRole("dialog")).toBeVisible();
// Fill form in dialog
await page.getByLabel("Name").fill("test-name");
// Submit
await page.getByRole("button", { name: "Create" }).click();
// Wait for close
await expect(page.getByRole("dialog")).not.toBeVisible();

Tables with Row Actions

// Find row by cell content
const row = page.getByRole("row").filter({
  has: page.getByRole("cell", { name: "item-name" }),
});
// Click action button in row (usually last button)
await row.getByRole("button").last().click();
// Select action from menu
await page.getByRole("menuitem", { name: "Edit" }).click();

Tabs

await page.getByRole("tab", { name: /Evaluators/i }).click();
await page.waitForURL("**/evaluators");
await expect(page.getByRole("tab", { name: /Evaluators/i })).toHaveAttribute(
  "aria-selected",
  "true",
);

Form Inputs in Sections

// When multiple textboxes exist, scope to section
const systemSection = page.locator('button:has-text("System")');
const systemTextbox = systemSection
  .locator("..")
  .locator("..")
  .getByRole("textbox");
await systemTextbox.fill("content");

Serial Tests (Shared State)

Use test.describe.serial when tests depend on each other:

test.describe.serial("Workflow", () => {
  const itemName = `item-${randomUUID()}`;

  test("step 1: create item", async ({ page }) => {
    // Creates itemName
  });

  test("step 2: edit item", async ({ page }) => {
    // Uses itemName from previous test
  });

  test("step 3: verify edits", async ({ page }) => {
    // Verifies itemName was edited
  });
});

Assertions

// Visibility
await expect(element).toBeVisible();
await expect(element).not.toBeVisible();

// Text content
await expect(element).toHaveText("expected");
await expect(element).toContainText("partial");

// Attributes
await expect(element).toHaveAttribute("aria-selected", "true");

// Input values
await expect(input).toHaveValue("expected value");

// URL
await page.waitForURL("**/datasets/**/examples");

Navigation Patterns

// Direct navigation
await page.goto("/datasets");
await page.waitForURL("**/datasets");

// Click navigation
await page.getByRole("link", { name: "Datasets" }).click();
await page.waitForURL("**/datasets");

// Extract ID from URL
const url = page.url();
const match = url.match(/datasets\/([^/]+)/);
const datasetId = match ? match[1] : "";

// Navigate with query params
await page.goto(`/playground?datasetId=${datasetId}`);

Running Tests

Before running Playwright tests, build the app so E2E runs against the latest frontend changes:

pnpm run build

# Run specific test file
pnpm exec playwright test tests/server-evaluators.spec.ts --project=chromium

# Run with UI mode
pnpm exec playwright test --ui

# Run specific test by name
pnpm exec playwright test -g "can create"

# Debug mode
pnpm exec playwright test --debug

Avoiding Interactive Report Server

By default, Playwright serves an HTML report after tests finish and waits for Ctrl+C, which can cause command timeouts. Use these options to avoid this:

# Use list reporter (no interactive server)
pnpm exec playwright test tests/example.spec.ts --project=chromium --reporter=list

# Use dot reporter for minimal output
pnpm exec playwright test tests/example.spec.ts --project=chromium --reporter=dot

# Set CI mode to disable interactive features
CI=1 pnpm exec playwright test tests/example.spec.ts --project=chromium

Recommended for automation: Always use --reporter=list or CI=1 when running tests programmatically to ensure the command exits cleanly after tests complete.

Phoenix-Specific Pages

UI Exploration with agent-browser

When selectors are unclear, use agent-browser to explore the Phoenix UI. For detailed agent-browser usage, invoke the /agent-browser skill.

Quick Reference for Phoenix

# Open Phoenix page (dev server runs on port 6006)
agent-browser open "http://localhost:6006/datasets"

# Get interactive snapshot with element refs
agent-browser snapshot -i

# Click using refs from snapshot
agent-browser click @e5

# Fill form fields
agent-browser fill @e2 "test value"

# Get element text
agent-browser get text @e1

Discovering Selectors Workflow

1. Open the page: agent-browser open "http://localhost:6006/datasets" 2. Get snapshot: agent-browser snapshot -i 3. Find element refs in output (e.g., @e1 [button] "New Dataset") 4. Interact: agent-browser click @e1 5. Re-snapshot after navigation/DOM changes: agent-browser snapshot -i

Translating to Playwright

File Naming

• Feature tests: {feature-name}.spec.ts
• Access control: {role}-access.spec.ts
• Rate limiting: {feature}.rate-limit.spec.ts (runs last)

Common Gotchas

1. Dialog not closing: Wait for a deterministic post-action signal (e.g., dialog hidden + success row visible) 2. Multiple elements: Use .first(), .last(), or .nth(n) 3. Dynamic content: Use regex in name: { name: /pattern/i } 4. Flaky waits: Prefer waitForURL over waitForTimeout 5. Menu not appearing: Wait for specific menu state/element visibility

Debugging Flaky Tests

Critical Lessons Learned

1. Don't assume parallelism is the problem

• Phoenix tests run with 7 parallel workers without issues
• The app handles concurrent logins, database operations, and session management properly
• If tests fail with parallelism, it's usually a test timing issue, not infrastructure
• Playwright's browser context isolation is robust - each worker gets isolated cookies/sessions

2. waitForTimeout is almost always wrong

• page.waitForTimeout() is the #1 cause of flakiness in Phoenix tests
• Arbitrary timeouts race against rendering and network speed
• Always replace with state-based waits:

     // ❌ BAD - flaky, races against rendering
     await page.waitForTimeout(500);
     await element.click();

     // ✅ GOOD - waits for actual state
     await element.waitFor({ state: "visible" });
     await element.click();

3. Test the actual failure before fixing

• Run tests with parallelism enabled to see what actually fails
• Check error messages - they often point to the real issue
• Don't optimize prematurely (e.g., caching auth state) if it's not the problem

4. Phoenix test infrastructure is solid

• In-memory SQLite works fine with parallel tests
• No need for per-worker databases
• No need for auth state caching
• Tests use randomUUID() for data isolation - this works well

Debugging Workflow

When tests are flaky:

1. Run with parallelism multiple times to catch intermittent failures:

   for i in 1 2 3 4 5; do
     pnpm exec playwright test --project=chromium --reporter=dot
   done

2. Look for `waitForTimeout` usage - replace with proper waits:

   grep -r "waitForTimeout" app/tests/

3. Check for race conditions in element interactions:

• Wait for element visibility before interacting
• Wait for network idle when needed: page.waitForLoadState("networkidle")
• Use waitForURL after navigation actions

4. Verify selectors are stable:

• Avoid CSS selectors that depend on DOM structure
• Use role/label selectors that match ARIA attributes
• Test selectors don't break when UI updates

5. Run with trace on failure to see what happened:

   pnpm exec playwright test --trace on-first-retry

Common Flaky Patterns and Fixes

Test Stability Best Practices

1. Use proper waits:

   // Wait for element state
   await element.waitFor({ state: "visible" | "hidden" | "attached" })

   // Wait for network
   await page.waitForLoadState("networkidle" | "domcontentloaded" | "load")

   // Wait for URL change
   await page.waitForURL("**/expected-path")

2. Use unique test data:

   const uniqueName = `test-${randomUUID()}`;

3. Prefer role selectors - they're less brittle:

   page.getByRole("button", { name: "Save" }) // ✅ Good
   page.locator('button.save-btn') // ❌ Brittle

4. Don't fight animations - wait for them:

   await expect(dialog).not.toBeVisible();

5. Verify URL changes after navigation:

   await page.waitForURL("**/datasets");

Phoenix Playwright Test Examples

Complete examples from the Phoenix test suite.

Example 1: Basic CRUD Test (Prompt Management)

import { expect, test } from "@playwright/test";
import { randomUUID } from "crypto";

test.describe("Prompt Management", () => {
  test.beforeEach(async ({ page }) => {
    page.goto(`/login`);
    await page.getByLabel("Email").fill("admin@localhost");
    await page.getByLabel("Password").fill("admin123");
    await page.getByRole("button", { name: "Log In", exact: true }).click();
    await page.waitForURL("**/projects");
  });

  test("can create a prompt", async ({ page }) => {
    await page.goto("/prompts");
    await page.waitForURL("**/prompts");
    await page.getByRole("link", { name: "New Prompt" }).click();
    await page.waitForURL("**/playground");
    await page
      .getByText("You are a chatbot")
      .fill("You are a helpful assistant");
    await page.getByRole("button", { name: "Save Prompt" }).click();
    await page.getByPlaceholder("Select or enter new prompt").click();
    const promptName = `chatbot-${randomUUID()}`;
    await page.getByPlaceholder("Select or enter new prompt").fill(promptName);
    await page.getByLabel("Prompt Description").click();
    await page.getByLabel("Prompt Description").fill("very kind chatbot");
    await page.getByRole("button", { name: "Create Prompt" }).click();
    await page.getByRole("button", { name: "View Prompt" }).click();

    await expect(page.getByRole("heading", { name: promptName })).toBeVisible();
    await expect(
      page.getByText("You are a helpful assistant").first()
    ).toBeVisible();
  });
});

Example 2: User Management (Admin Actions)

import { expect, test } from "@playwright/test";
import { randomUUID } from "crypto";

test.beforeEach(async ({ page }) => {
  page.goto(`/login`);
  await page.getByLabel("Email").fill("admin@localhost");
  await page.getByLabel("Password").fill("admin123");
  await page.getByRole("button", { name: "Log In", exact: true }).click();
  await page.waitForURL("**/projects");
});

test("can create a user", async ({ page }) => {
  await page.goto("/settings/general");
  await page.waitForURL("**/settings/general");
  await page.getByRole("button", { name: "Add User" }).click();

  const email = `member-${randomUUID()}@localhost.com`;
  await page.getByLabel("Email").fill(email);
  await page.getByLabel("Username").fill(email);
  await page.getByLabel("Password", { exact: true }).fill("member123");
  await page.getByLabel("Confirm Password").fill("member123");
  await page.getByRole("dialog").getByLabel("member", { exact: true }).click();
  await page
    .getByRole("dialog")
    .getByRole("option", { name: "member" })
    .click();
  await page
    .getByRole("dialog")
    .getByRole("button", { name: "Add User" })
    .click();

  await expect(page.getByRole("cell", { name: email })).toBeVisible();
});

Example 3: Serial Tests with Shared State (Evaluators)

import { expect, test } from "@playwright/test";
import { randomUUID } from "crypto";

test.describe.serial("Server Evaluators", () => {
  const datasetName = `test-dataset-${randomUUID()}`;
  const customEvaluatorName = `custom-eval-${randomUUID().slice(0, 8)}`;
  const updatedDescription = "Updated description for testing";

  test.beforeEach(async ({ page }) => {
    await page.goto(`/login`);
    await page.getByLabel("Email").fill("admin@localhost");
    await page.getByLabel("Password").fill("admin123");
    await page.getByRole("button", { name: "Log In", exact: true }).click();
    await page.waitForURL("**/projects");
  });

  test("can create a dataset", async ({ page }) => {
    await page.goto("/datasets");
    await page.waitForURL("**/datasets");

    await page.getByRole("button", { name: "New Dataset" }).click();
    await page.getByRole("menuitem", { name: "New Dataset" }).click();

    await page.getByLabel("Dataset Name").clear();
    await page.getByLabel("Dataset Name").fill(datasetName);
    await page.getByLabel("Description").fill("Test dataset for evaluators");

    await page.getByRole("button", { name: "Create Dataset" }).click();
    await expect(page.getByRole("dialog")).not.toBeVisible({ timeout: 10000 });

    await expect(page.getByRole("link", { name: datasetName })).toBeVisible({
      timeout: 10000,
    });
  });

  test("can create a custom LLM evaluator", async ({ page }) => {
    await page.goto("/datasets");
    await page.getByRole("link", { name: datasetName }).click();
    await page.waitForURL("**/datasets/**/examples");
    await page.getByRole("tab", { name: /Evaluators/i }).click();
    await page.waitForURL("**/evaluators");

    await page.getByRole("button", { name: "Add evaluator" }).click();
    await page
      .getByRole("menuitem", { name: "Create new LLM evaluator" })
      .click();

    await expect(
      page.getByRole("heading", { name: "Create Evaluator" })
    ).toBeVisible();

    await page
      .getByRole("textbox", { name: "Name" })
      .first()
      .fill(customEvaluatorName);
    await page
      .getByRole("textbox", { name: /Description/i })
      .fill("Initial description for custom evaluator");

    const systemSection = page.locator(
      'button:has-text("System"):not([role="menuitem"])'
    );
    const systemTextbox = systemSection
      .locator("..")
      .locator("..")
      .getByRole("textbox");
    await systemTextbox.fill("You are an evaluator.");

    const userSection = page.locator(
      'button:has-text("User"):not([role="menuitem"])'
    );
    const userTextbox = userSection
      .locator("..")
      .locator("..")
      .getByRole("textbox");
    await userTextbox.fill("Evaluate: {{output}}");

    await page.getByRole("button", { name: "Create" }).click();
    await expect(page.getByRole("dialog")).not.toBeVisible({ timeout: 10000 });

    await expect(
      page.getByRole("cell", { name: customEvaluatorName })
    ).toBeVisible();
  });

  test("can edit an LLM evaluator", async ({ page }) => {
    await page.goto("/datasets");
    await page.getByRole("link", { name: datasetName }).click();
    await page.waitForURL("**/datasets/**/examples");
    await page.getByRole("tab", { name: /Evaluators/i }).click();
    await page.waitForURL("**/evaluators");

    const evaluatorRow = page.getByRole("row").filter({
      has: page.getByRole("cell", { name: customEvaluatorName }),
    });
    await evaluatorRow.getByRole("button").last().click();
    await page.getByRole("menuitem", { name: "Edit" }).click();

    await expect(
      page.getByRole("heading", { name: "Edit Evaluator" })
    ).toBeVisible();

    const descriptionInput = page.getByRole("textbox", {
      name: /Description/i,
    });
    await descriptionInput.clear();
    await descriptionInput.fill(updatedDescription);

    await page.getByRole("button", { name: "Update" }).click();
    await expect(page.getByRole("dialog")).not.toBeVisible({ timeout: 10000 });

    await expect(
      page.getByRole("cell", { name: customEvaluatorName })
    ).toBeVisible();
  });

  test("can verify edits were saved", async ({ page }) => {
    await page.goto("/datasets");
    await page.getByRole("link", { name: datasetName }).click();
    await page.waitForURL("**/datasets/**/examples");
    await page.getByRole("tab", { name: /Evaluators/i }).click();
    await page.waitForURL("**/evaluators");

    const evaluatorRow = page.getByRole("row").filter({
      has: page.getByRole("cell", { name: customEvaluatorName }),
    });
    await evaluatorRow.getByRole("button").last().click();
    await page.getByRole("menuitem", { name: "Edit" }).click();

    const descriptionInput = page.getByRole("textbox", {
      name: /Description/i,
    });
    await expect(descriptionInput).toHaveValue(updatedDescription);

    await page.getByRole("button", { name: "Cancel" }).click();
  });
});

Example 4: Role-Based Access Control

import { expect, test } from "@playwright/test";

test.describe("Viewer Access", () => {
  test.beforeEach(async ({ page }) => {
    page.goto(`/login`);
    await page.getByLabel("Email").fill("viewer@localhost.com");
    await page.getByLabel("Password").fill("viewer123");
    await page.getByRole("button", { name: "Log In", exact: true }).click();
    await page.waitForURL("**/projects");
  });

  test("viewer cannot access settings", async ({ page }) => {
    await page.goto("/settings/general");
    // Viewer should be redirected or see access denied
    await expect(page.getByText("Access Denied")).toBeVisible();
  });
});

Example 5: Testing with Playground Integration

test("evaluators are visible in playground when dataset is selected", async ({
  page,
}) => {
  await page.goto("/datasets");
  await page.getByRole("link", { name: datasetName }).click();
  await page.waitForURL("**/datasets/**/examples");

  const datasetUrl = page.url();
  const datasetIdMatch = datasetUrl.match(/datasets\/([^/]+)/);
  const datasetIdForUrl = datasetIdMatch ? datasetIdMatch[1] : "";

  await page.goto(`/playground?datasetId=${datasetIdForUrl}`);
  await page.waitForURL("**/playground**");

  await expect(
    page.getByRole("button", { name: new RegExp(datasetName) })
  ).toBeVisible();

  await expect(page.getByRole("heading", { name: "Experiment" })).toBeVisible();

  const experimentSection = page.locator("text=Experiment").locator("..");
  await expect(
    experimentSection.getByRole("button", { name: "Evaluators" })
  ).toBeVisible();
});

Phoenix UI Component Reference

Reference for common Phoenix UI components and how to interact with them in Playwright tests.

Navigation Sidebar

The main navigation sidebar contains:

• Projects
• Datasets & Experiments
• Playground
• Evaluators
• Prompts
• APIs
• Settings
• Documentation
• Support
• Profile

// Navigate via sidebar
await page.getByRole("link", { name: "Datasets & Experiments" }).click();
await page.getByRole("link", { name: "Settings" }).click();

Breadcrumbs

Located at top of pages, shows navigation path:

await page.getByRole("link", { name: "Datasets" }).click(); // In breadcrumbs

Tables

Phoenix uses data tables with sortable columns and row actions.

Table Structure

<table>
  <thead>
    <tr>
      <th>name</th>
      <th>description</th>
      <th>actions</th>
      <!-- Often empty header for action column -->
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>item-name</td>
      <td>description</td>
      <td><button>...</button></td>
      <!-- Action menu -->
    </tr>
  </tbody>
</table>

Interacting with Tables

// Click on cell content (usually a link)
await page.getByRole("link", { name: "item-name" }).click();

// Find specific row
const row = page.getByRole("row").filter({
  has: page.getByRole("cell", { name: "item-name" }),
});

// Click action menu in row
await row.getByRole("button").last().click();

// Verify cell exists
await expect(page.getByRole("cell", { name: "item-name" })).toBeVisible();

Tabs

Used on detail pages (e.g., Dataset detail has Experiments, Examples, Evaluators, Versions tabs).

// Click tab
await page.getByRole("tab", { name: /Evaluators/i }).click();

// Verify tab is selected
await expect(page.getByRole("tab", { name: /Evaluators/i })).toHaveAttribute(
  "aria-selected",
  "true"
);

// Get content from tab panel
await page.getByRole("tabpanel").getByText("content");

Dialogs/Modals

Used for forms, confirmations, and complex interactions.

// Wait for dialog to appear
await expect(page.getByRole("dialog")).toBeVisible();

// Get dialog heading
await expect(
  page.getByRole("heading", { name: "Create Evaluator" })
).toBeVisible();

// Interact with dialog content
await page.getByRole("dialog").getByLabel("Name").fill("value");

// Close dialog
await page.getByRole("button", { name: "Cancel" }).click();
// or
await page.getByRole("button", { name: "Create" }).click();

// Wait for dialog to close
await expect(page.getByRole("dialog")).not.toBeVisible({ timeout: 10000 });

Dropdown Menus

Two types: button dropdowns and menu triggers.

Button Dropdown

// Click button to open dropdown
await page.getByRole("button", { name: "New Dataset" }).click();

// Select option
await page.getByRole("menuitem", { name: "New Dataset" }).click();

Action Menu (Three Dots)

// Usually the last button in a row
await row.getByRole("button").last().click();

// Select action
await page.getByRole("menuitem", { name: "Edit" }).click();
await page.getByRole("menuitem", { name: "Delete" }).click();

Nested Submenus

// Open parent menu
await page.getByRole("button", { name: "Add evaluator" }).click();

// Hover/click to open submenu
await page
  .getByRole("menuitem", { name: "Use LLM evaluator template" })
  .click();

// Select from submenu
await page.getByRole("menuitem", { name: /correctness/i }).click();

Form Elements

Text Inputs

// By label
await page.getByLabel("Name").fill("value");
await page.getByLabel("Description").fill("value");

// By role
await page.getByRole("textbox", { name: "Name" }).fill("value");

// By placeholder
await page.getByPlaceholder("Enter name").fill("value");

// Clear and fill
await page.getByLabel("Name").clear();
await page.getByLabel("Name").fill("new value");

Select/Combobox

// Click to open
await page.getByRole("combobox", { name: /mapping/i }).click();

// Type to filter and select
await page.getByRole("combobox", { name: /mapping/i }).fill("option");

// Select from dropdown
await page.getByRole("option", { name: "option-name" }).click();

Checkboxes and Switches

// Toggle switch
await page.getByRole("switch", { name: "Include explanation" }).click();

// Checkbox
await page.getByRole("checkbox", { name: "Option" }).check();
await page.getByRole("checkbox", { name: "Option" }).uncheck();

Radio Buttons

await page.getByRole("radio", { name: "Mustache" }).click();
await page.getByRole("radio", { name: "F-String" }).click();

Buttons

// By name
await page.getByRole("button", { name: "Save" }).click();
await page.getByRole("button", { name: "Create" }).click();
await page.getByRole("button", { name: "Update" }).click();
await page.getByRole("button", { name: "Cancel" }).click();

// Exact match
await page.getByRole("button", { name: "Log In", exact: true }).click();

// With icon (may need different selector)
await page.getByRole("button", { name: "Add evaluator" }).click();

Search Boxes

await page
  .getByRole("searchbox", { name: "Search datasets by name" })
  .fill("query");
await page.getByRole("searchbox", { name: /Search/i }).fill("query");

Slideover Panels

Full-screen modals that slide in from the side (used for editing evaluators).

// These behave like dialogs
await expect(page.getByRole("dialog")).toBeVisible();
await page.getByRole("heading", { name: "Edit Evaluator" }).toBeVisible();

Expandable Sections (Disclosure)

Collapsible sections in forms.

// Click to expand/collapse
await page
  .getByRole("button", { name: "System Role for the chat message" })
  .click();

// Check if expanded
await expect(
  page.getByRole("button", { name: "System Role for the chat message" })
).toHaveAttribute("aria-expanded", "true");

Code Editor Areas (CodeMirror)

Phoenix uses CodeMirror for code/prompt editing. These are typically wrapped in textbox roles.

// Find the textbox within a section
const systemSection = page.locator('button:has-text("System")');
const editor = systemSection.locator("..").locator("..").getByRole("textbox");
await editor.fill("content");

Empty States

Tables and lists show empty state messages when no data exists.

await expect(
  page.getByText("No evaluators added to this dataset")
).toBeVisible();
await expect(page.getByText("No data")).toBeVisible();

Loading States

// Wait for loading to complete
await expect(page.getByText("Loading...")).not.toBeVisible();

// Or wait for specific content
await expect(page.getByRole("table")).toBeVisible();

Error States

// Error alerts
await expect(page.getByRole("alert")).toContainText("Error message");

// Error page
await expect(
  page.getByRole("heading", { name: "Something went wrong" })
).toBeVisible();

Tooltips

// Hover to show tooltip
await page.getByRole("button", { name: "Info" }).hover();
await expect(page.getByRole("tooltip")).toContainText("Help text");