Cypress To Scout Migration

Cypress to Scout Migration

Overview

Migrate Cypress tests to Scout by first validating each test through triage gates, then rewriting using Scout patterns. Never migrate Cypress tests directly — validate first, then rewrite following Scout best practices.

Required sub-skills

• REQUIRED: scout-create-scaffold (generate Scout directory structure)
• REQUIRED: scout-ui-testing (page objects, browser auth, parallel UI)
• REQUIRED: scout-api-testing (apiClient/auth, apiServices patterns)
• REQUIRED: scout-best-practices-reviewer (review migrated tests)

Core principle

Exercise behavior in the least flaky automation layer first: UNIT > API > UI

A Cypress E2E test should only become a Scout E2E test if it genuinely tests a user workflow that cannot be verified at a lower layer.

Tools

• Scaffold a spec file: bash scripts/scaffold_scout_spec.sh --name <name> --domain <path> --plugin-test-dir <path> [--type parallel|sequential]
• Check selector validity: bash scripts/extract_selectors.sh <cypress-test-file> --app-src <path-to-plugin-source>

All paths relative to this skill's directory.

Step 0: Load solution-specific skill [mandatory]

Before starting triage or migration, check the References section for a solution-specific extension skill that matches the Cypress test's location. If one exists, read it immediately — it overrides general conventions with solution-specific paths, packages, roles, API services, and templates. All subsequent phases must follow the solution-specific conventions when they conflict with this skill.

Phase 1: Triage (before touching any code) [medium freedom]

For each Cypress test, pass all five gates before migrating.

Gate 0: Is the feature still valid?

1. Check if the feature under test still exists in the codebase 2. Run scripts/extract_selectors.sh <test-file> to verify selectors still exist 3. Check if the feature was removed, redesigned, or moved behind a feature flag

Gate 1: Is it already covered?

Search for existing coverage in:

• Scout tests in test/scout/
• API integration tests
• Unit tests co-located with source code
• Other Cypress tests covering same behavior

Don't rely on test names — check what the test actually asserts.

If covered at a lower layer → delete Cypress, no migration needed.

If covered in Scout → delete Cypress. Scout now runs on serverless (MKI), so Cypress tests are no longer needed for serverless (MKI) coverage.

Gate 2: Is it at the right layer?

If the test belongs at API/unit layer → write coverage there instead.

Gate 3: Does it add value?

Delete without migrating if the test:

• Only verifies a page loads without errors
• Only checks that a button or element exists
• Tests trivial behavior already covered by type safety
• Has been skipped for 3+ months with no progress
• Tests deprecated or soon-to-be-removed functionality

Gate 4: Flakiness risk assessment

Two checks — current status and source code risk scan.

4a: Current status

If the test is currently skipped (.skip, @skipInServerless, etc.) or chronically flaky: 1. Determine if flakiness is a test problem or an app bug 2. App bug → fix the app first, then write the Scout test 3. Cypress-specific issue (timing, selectors) → migrate with proper patterns

If the solution has a flaky-test-doctor skill, use it for deeper root cause analysis.

4b: Source code risk scan

Even if the Cypress test passes reliably today, its source code may contain patterns that would produce a flaky Scout test. Scan the Cypress file and its imported tasks/screens against the pattern catalog in references/flakiness-risk-patterns.md.

For tests with 3+ critical/high-risk patterns, strongly consider writing the Scout test from scratch using the feature spec rather than porting the Cypress logic.

PR strategy

One Cypress spec file = one PR. Each migrated spec file must be submitted as its own pull request:

• Keeps reviews focused and manageable
• Isolates risk — a problem in one migration doesn't block others
• Makes it easy to revert a single migration if issues surface

Every PR must pass the [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner) before merging. Run the new Scout test through the flaky test suite runner to verify stability.

Phase 2: Migration

Tests that pass all triage gates proceed here. Do not port Cypress code 1:1. Rewrite using Scout patterns.

Step 1: Determine test type [high freedom]

• UI test if it verifies user flows, page rendering, or interactive behavior
• API test if the Cypress test primarily validates data via API assertions

Step 2: Set up Scout scaffold [low freedom — use script]

If the plugin doesn't have a Scout test directory yet, read the scout-create-scaffold skill.

Generate the spec file boilerplate:

bash scripts/scaffold_scout_spec.sh --name <spec_name> --domain <domain_path> \
  --plugin-test-dir <plugin>/test/scout/ui --type parallel

Use the solution-specific skill's page object and API service templates as starting points (if available).

Step 3: Map Cypress patterns to Scout [medium freedom]

Step 3b: Data cleanup audit [low freedom — must be thorough]

Critical: Cypress runs each spec in a clean environment, so many Cypress tests never clean up after themselves. Scout shares the environment across specs — leftover data will break other tests. Do not trust the Cypress test's cleanup.

1. Read the Cypress test and its tasks/setup — identify every resource created:

• Saved objects (rules, cases, dashboards, saved queries)
• ES indices or documents
• Fleet agents, policies, integrations
• User preferences, UI settings, localStorage state
• API keys or credentials

2. Add explicit cleanup in the Scout test (afterAll / afterEach)

3. Add defensive cleanup in `beforeAll` — handles leftover data from a previous failed run

4. Verify cleanup works — run the test twice in a row locally. Second run fails → cleanup is incomplete.

Step 4: Write the Scout test [high freedom]

Read the scout-ui-testing or scout-api-testing skill for implementation details.

Key rules:

• Tags are required — Scout validates UI test tags at runtime
• One suite per file — single top-level test.describe() or spaceTest.describe()
• UI actions in page objects, assertions in specs
• API-based setup/teardown via apiServices / kbnClient
• Use `test.step()` for multi-step flows to reuse browser context
• Parallelize when possible — use spaceTest + scoutSpace for worker-isolated spaces
• Test fixture for per-test isolated setup; Worker fixture for shared setup
• Page objects encapsulate selectors and actions; assertions stay in specs
• EUI wrappers — use Scout's EuiComboBoxWrapper, EuiDataGridWrapper, etc.
• All created data must be cleaned up — see Step 3b

Step 5: Review and verify [low freedom — mandatory checklist]

1. Run the scout-best-practices-reviewer skill against the new test 2. Make sure the test fails — intentionally break the feature and confirm the test catches it 3. Run locally: node scripts/scout.js run-tests --stateful --testFiles <path> 4. Update manifests: node scripts/scout.js update-test-config-manifests 5. Open a PR with only this spec file's migration (one spec per PR) 6. Run the [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner) — do not merge until stable

Phase 3: Cleanup

After the Scout test is verified:

1. Delete the Cypress test file 2. Remove orphaned code (tasks, screens, objects only used by the deleted test) 3. Check for orphaned imports (grep to verify no other usages) 4. Remove from all relevant config/index files 5. Update manifests: node scripts/scout.js update-test-config-manifests

Common mistakes

• Trusting the Cypress test's cleanup — Scout shares the env, so you must add cleanup
• Porting Cypress code line-by-line instead of rewriting with Scout patterns
• Migrating tests that belong at API/unit layer
• Forgetting triage gates and migrating flaky/invalid/duplicate tests
• Skipping the Gate 4b risk scan — Cypress patterns that seem harmless produce flaky Scout tests
• Using page.waitForTimeout() — forbidden, same as cy.wait(ms)
• Using page.waitForLoadState('networkidle') — anti-pattern, actively removed from Scout tests; wait for specific elements instead
• Using short custom timeouts on waitFor() (e.g., 3s) — causes CI flakiness; use the default (10s)
• Adding explicit waits before clear(), fill(), click() — these auto-wait; the extra wait is redundant
• Specifying { state: 'visible' } on waitFor() — it's the default, omit it
• Missing Scout tags (validated at runtime)
• Using esArchiver for system indices (use kbnClient)
• Not parallelizing tests that could run in parallel
• Putting assertions in page objects (keep in specs)
• Skipping the "make sure your test fails" verification
• Batching multiple spec migrations into a single PR
• Merging without running the Flaky Test Runner
• Using fill() on Kibana query bars — QueryStringInput submits React props, not DOM value; use pressSequentially() (see best practices)
• Asserting .euiTableRow count as 0 — EuiBasicTable always renders an empty-state row; assert on the message text instead (see best practices)

Phase 4: Skill improvement

After every migration, review what you learned and suggest updating this skill if any of these apply:

• New Kibana/EUI component interaction pattern — a component required a non-obvious Playwright approach (e.g., pressSequentially for query bars, dispatchEvent for unstable popovers, CSS :has() for tooltip anchors)
• New flakiness pattern — a Cypress pattern caused flakiness in Scout that isn't already in references/flakiness-risk-patterns.md
• New API service or page object — reusable infrastructure warrants documenting in the solution-specific skill
• New role or auth method — a convenience login method was added (e.g., loginAsT1Analyst)
• Lint rule workaround — a Playwright lint rule required a non-obvious alternative (e.g., dispatchEvent instead of force: true, toContainText([...]) instead of nth())
• Ownership or architecture insight — learned where UI code lives, which plugin owns what, or how data flows

Prompt the user: _"During this migration I learned [X]. Want me to add it to the skill so future migrations benefit?"_

References

Open only what you need:

• Key differences between Cypress and Scout (auth, CI, MKI, tags, execution model): references/cypress-vs-scout-differences.md
• Best practices for writing Scout tests (fixtures, page objects, parallelism): references/migration-best-practices.md
• Flakiness risk patterns to scan for during Gate 4b (hard-coded waits, missing cleanup, force clicks, etc.): references/flakiness-risk-patterns.md
• Complete before/after migration example with annotated decisions (Timeline creation): references/example-migration.md

Solution-specific extensions (Step 0)

Load the matching skill when migrating tests from that solution:

• Security Solution (tests in x-pack/solutions/security/): x-pack/solutions/security/plugins/security_solution/.agents/skills/cypress-to-scout-migration/SKILL.md

Cypress vs Scout Differences

Table of Contents

• Folder Structure
• Authentication
• CI Execution & Parallelization
• MKI Pipelines
• Local Execution
• Test Labels/Tags
• Test Patterns
• Serverless Tiers & Feature Flags
• Key Behavioral Differences

Folder Structure

Authentication

Scout uses SAML for both ESS and Serverless, making auth consistent across environments.

CI Test Execution & Parallelization

Critical implication: In Scout, data created by one test may affect others. Use spaceTest + scoutSpace for isolation, and clean up in afterAll. Most Cypress tests have no cleanup logic because the environment is reset per spec. When migrating, you must independently audit what the test creates and add explicit cleanup.

MKI Pipelines

Scout now runs on MKI via the Appex QA Serverless Scout pipeline. Cypress tests with @serverless tags no longer need to be kept solely for MKI coverage — Scout can replace them.

Local Test Execution

Test Labels/Tags

Test Patterns

Serverless Tiers & Feature Flags

Key Behavioral Differences

1. Spec isolation: Cypress creates a clean environment per spec file. Scout shares the environment — you must manage state explicitly.

2. Auto-waiting: Cypress retries commands with built-in timeout. Playwright/Scout also auto-waits on locator actions but uses different mechanisms (expect.poll(), locator assertions).

3. Async model: Cypress chains commands in a queue. Scout/Playwright uses async/await — more predictable control flow.

4. Browser context: In Scout, each test() block spins up a new browser context. Use test.step() for multi-step flows to reuse context and improve execution time.

5. Setup/teardown: Cypress often uses UI for setup. Scout strongly prefers API-based setup via apiServices/kbnClient.

Example Migration: Timeline Creation

Real migration of investigations/timelines/creation.cy.ts to Scout. Demonstrates triage decisions, pattern mapping, page object design, API service design, and cleanup strategy.

Triage summary

Before: Cypress (creation.cy.ts, abbreviated)

import { ROWS } from '../../../screens/timelines';
import { deleteTimelines, createTimelineTemplate } from '../../../tasks/api_calls/timelines';
import { login } from '../../../tasks/login';
import { addNameToTimelineAndSave, executeTimelineKQL, closeTimeline } from '../../../tasks/timeline';

describe('Timelines', { tags: ['@ess', '@serverless'] }, () => {
  beforeEach(() => {
    deleteTimelines();                                  // API cleanup — no afterEach
  });

  it('should show the different timeline states', () => {
    login();
    visitWithTimeRange(TIMELINES_URL);
    openTimelineUsingToggle();

    cy.get(TIMELINE_STATUS).invoke('text').should('match', /^Unsaved/);

    addNameToTimelineAndSave('Test');
    cy.get(TIMELINE_STATUS).should('not.exist');

    cy.get(LOADING_INDICATOR).should('be.visible');     // Sync on background save
    cy.get(LOADING_INDICATOR).should('not.exist');

    executeTimelineKQL('agent.name : *');
    cy.get(TIMELINE_STATUS).invoke('text').should('match', /^Unsaved changes/);
  });

  it('should save timelines as new', () => {
    login();
    visitWithTimeRange(TIMELINES_URL);
    cy.get(ROWS).should('have.length', '0');            // EuiBasicTable quirk — see below

    openTimelineUsingToggle();
    addNameToTimelineAndSave('First');

    cy.get(LOADING_INDICATOR).should('be.visible');
    cy.get(LOADING_INDICATOR).should('not.exist');

    addNameToTimelineAndSaveAsNew('Second');
    closeTimeline();

    cy.get(ROWS).should('have.length', '2');
    cy.get(ROWS).first().invoke('text').should('match', /Second/);  // .first()/.last() — see below
    cy.get(ROWS).last().invoke('text').should('match', /First/);
  });
});

Key problems in the Cypress source

After: Scout (split into two files — one role per file)

The migration splits tests by role: CRUD tests in timeline_creation.spec.ts (platform engineer) and read-only tests in timeline_read_only.spec.ts (T1 analyst). Each file is self-contained with its own setup/teardown and login in beforeEach.

timeline_creation.spec.ts (CRUD role)

import { spaceTest, tags } from '@kbn/scout-security';
import { expect } from '@kbn/scout-security/ui';

spaceTest.describe(
  'Timeline creation',
  { tag: [...tags.stateful.classic, ...tags.serverless.security.complete] },
  () => {
    spaceTest.beforeEach(async ({ browserAuth, apiServices, pageObjects }) => {
      await apiServices.timeline.deleteAll();
      await browserAuth.loginAsPlatformEngineer();
      await pageObjects.timelinePage.navigateToTimelines();
    });

    spaceTest.afterAll(async ({ apiServices }) => {
      await apiServices.timeline.deleteAll();
    });

    spaceTest('should show the different timeline states', async ({ pageObjects }) => {
      const { timelinePage } = pageObjects;

      await timelinePage.open();

      await spaceTest.step('Verify unsaved state', async () => {
        await expect(timelinePage.saveStatus).toHaveText(/^Unsaved/);
      });

      await spaceTest.step('Save and verify saved state', async () => {
        await timelinePage.saveWithName('Test');
        await expect(timelinePage.saveStatus).toBeHidden();
      });

      await spaceTest.step('Modify query and verify unsaved changes', async () => {
        await timelinePage.executeKQL('agent.name : *');
        await expect(timelinePage.saveStatus).toHaveText(/^Unsaved changes/);
      });
    });

    spaceTest('should save timelines as new', async ({ pageObjects }) => {
      const { timelinePage } = pageObjects;

      await spaceTest.step('Verify empty state', async () => {
        await expect(timelinePage.timelinesTable).toContainText(
          '0 timelines match the search criteria'
        );
      });

      await spaceTest.step('Create, save, and save as new', async () => {
        await timelinePage.open();
        await timelinePage.saveWithName('First');
        await expect(timelinePage.saveStatus).toBeHidden();
        await timelinePage.saveAsNew('Second');
      });

      await spaceTest.step('Verify both timelines in list', async () => {
        await timelinePage.close();
        await expect(timelinePage.timelineRows).toHaveCount(2);
        await expect(timelinePage.timelineRows).toContainText(['Second', 'First']);
      });
    });
  }
);

timeline_read_only.spec.ts (read-only role)

import { spaceTest, tags } from '@kbn/scout-security';
import { expect } from '@kbn/scout-security/ui';

spaceTest.describe(
  'Timeline read-only',
  { tag: [...tags.stateful.classic, ...tags.serverless.security.complete] },
  () => {
    spaceTest.beforeEach(async ({ browserAuth, apiServices, pageObjects }) => {
      await apiServices.timeline.deleteAll();
      await browserAuth.loginAsT1Analyst();
      await pageObjects.timelinePage.navigateToTimelines();
    });

    spaceTest.afterAll(async ({ apiServices }) => {
      await apiServices.timeline.deleteAll();
    });

    spaceTest(
      'should not be able to create/update timeline with only read privileges',
      async ({ pageObjects }) => {
        const { timelinePage } = pageObjects;

        await timelinePage.open();
        await timelinePage.createNew();

        await expect(timelinePage.panel).toBeVisible();
        await expect(timelinePage.saveButton).toBeDisabled();

        await spaceTest.step('Hover save button and verify read-only tooltip', async () => {
          await timelinePage.hoverSaveButton();
          await expect(timelinePage.saveTooltip).toContainText(
            'you do not have the required permissions to save timelines'
          );
        });
      }
    );
  }
);

Key decisions annotated

Page object: TimelinePage (abbreviated)

export class TimelinePage {
  // All locators as readonly constructor properties — centralized, auditable
  readonly panel: Locator;
  readonly saveStatus: Locator;
  readonly saveButton: Locator;
  readonly kqlTextarea: Locator;
  readonly saveButtonTooltipAnchor: Locator;

  constructor(private readonly page: ScoutPage) {
    this.panel = this.page.testSubj.locator('timeline-modal-header-panel');
    // saveStatus scoped to panel — avoids strict mode violation (appears in header AND bottom bar)
    this.saveStatus = this.panel.locator('[data-test-subj="timeline-save-status"]');
    this.kqlTextarea = this.page.testSubj
      .locator('timeline-search-or-filter-search-container')
      .locator('textarea');
    // CSS :has() for parent selection — EUI wraps disabled buttons in a tooltip anchor <span>
    this.saveButtonTooltipAnchor = this.page.locator(
      'span:has([data-test-subj="timeline-modal-save-timeline"])'
    );
  }

  async executeKQL(query: string) {
    await this.kqlTextarea.click();
    await this.kqlTextarea.clear();
    // pressSequentially — QueryStringInput submits React props on Enter, not DOM value.
    // fill() sets DOM value synchronously but React props update asynchronously.
    await this.kqlTextarea.pressSequentially(query);
    await this.kqlTextarea.press('Enter');
  }

  async hoverSaveButton() {
    // EUI wraps disabled buttons in a tooltip anchor that intercepts pointer events.
    await this.saveButtonTooltipAnchor.hover();
  }
}

Patterns demonstrated

API service: TimelineApiService (abbreviated)

export const getTimelineApiService = ({
  kbnClient, log, scoutSpace,
}: {
  kbnClient: KbnClient;
  log: ScoutLogger;
  scoutSpace?: ScoutParallelWorkerFixtures['scoutSpace'];  // Space-aware for parallel tests
}): TimelineApiService => {
  const basePath = scoutSpace?.id ? `/s/${scoutSpace.id}` : '';

  return {
    createTimeline: async (input = {}) => { /* POST to /api/timeline */ },
    createTimelineTemplate: async (input = {}) => { /* POST with timelineType: 'template' */ },
    deleteAll: async () => {
      // Must fetch and delete both 'default' and 'template' types separately
      const [defaultIds, templateIds] = await Promise.all([
        fetchAllSavedObjectIds('default'),
        fetchAllSavedObjectIds('template'),
      ]);
      // ...
    },
  };
};

Key design decisions

Flakiness Risk Patterns for Migration

Table of Contents

• Critical — Will cause flakiness in Scout
• High — Likely to cause issues
• Medium — Address during rewrite
• How to use this during triage

Scan Cypress source code for these patterns before migration. Each indicates a risk area that needs specific handling in the Scout rewrite. Check the test file and its imported tasks/screens/objects.

Critical — Will cause flakiness in Scout

Hard-coded waits

• Look for: cy.wait(number) (e.g., cy.wait(500), cy.wait(2000))
• Why: The underlying timing issue the wait masks must be addressed. page.waitForTimeout() is forbidden in Scout.
• Scout approach: Playwright auto-waiting, expect.poll(), or locator assertions with built-in retry.

Missing cleanup / shared state

• Look for: Tests with no afterEach/after cleanup; esArchiverLoad without unload; API resources created but never deleted; global mutable state.
• Why: Cypress runs each spec in a clean browser. Scout shares the environment across specs in a worker — leftover data causes cascading failures.
• Scout approach: Explicit cleanup in afterAll/afterEach, defensive cleanup in beforeAll, unique identifiers per worker (scoutSpace.id).

Force interactions

• Look for: { force: true } on .click(), .type(), .check(), .select()
• Why: Playwright strict mode rejects interactions with hidden/disabled elements. Force-clicks mask real UI issues (element behind overlay, not yet visible, disabled).
• Scout approach: Wait for the element to be actionable. If behind an overlay, close the overlay first.
• Exception: If the Cypress { force: true } exists because an app bug causes continuous DOM re-rendering (e.g., a useEffect loop triggering table re-fetches), use dispatchEvent('click') to bypass actionability checks without triggering the playwright/no-force-option lint rule. Document the app bug location and consider filing a fix. See migration-best-practices.md → "dispatchEvent for app-level DOM instability".

esArchiver for system indices

• Look for: cy.task('esArchiverLoad', ...) targeting system index names (.kibana, .alerts, .fleet, etc.)
• Why: Forbidden in Scout. System indices are managed by Kibana and must be created via APIs.
• Scout approach: Use kbnClient or apiServices to create saved objects and configuration.

High — Likely to cause issues

cy.intercept() + cy.wait('@alias') as sync points

• Look for: cy.intercept('GET|POST', '/api/...').as('alias') followed by cy.wait('@alias')
• Why: Playwright doesn't have Cypress-style request interception for synchronization. Direct ports using page.waitForResponse() are fragile.
• Scout approach: Wait for UI state instead: expect(locator).toBeVisible(), expect.poll(), or data-loading indicators.

recurse() or retry loops

• Look for: recurse(), cy.waitUntil(), manual retry loops, .should() chains wrapping re-tried actions
• Why: Indicates unstable UI or race condition in the app. The instability carries over to Scout.
• Scout approach: Use expect.poll() or expect(locator).toPass(). Fix the underlying instability if it's an app bug.

Index-based selectors

• Look for: .eq(0), .first(), .last(), :nth-child() without data-test-subj
• Why: Fragile when tests run in parallel with dynamic data. Element order may differ between runs.
• Scout approach: Use data-test-subj attributes. If elements are dynamic, filter by text content or unique attributes.

beforeEach with UI navigation for setup

• Look for: beforeEach(() => { cy.visit(URL); ... }) with repeated page navigation or UI-driven data creation
• Why: Slow and flaky in Scout. Each navigation costs time and introduces timing risks.
• Scout approach: API-based setup in beforeAll (worker fixture). Navigate once, use test.step() for multi-step flows.

.within() on re-rendering containers

• Look for: .within(() => { ... }) targeting tables, lists, or containers with loading states
• Why: .within() captures a DOM snapshot that goes stale on re-render. Playwright locator chaining avoids this, but the re-rendering itself signals timing-sensitive UI.
• Scout approach: Use Playwright locator chaining: page.testSubj.locator('container').locator('child'). Locators auto-retry.

Medium — Address during rewrite

cy.request() in test body

• Look for: cy.request({ method: 'POST', url: '/api/...', ... }) inside it() blocks (not just before/beforeEach)
• Why: In-test API calls should be in apiServices. Mixing UI and API in the test body makes tests harder to maintain and debug.
• Scout approach: Extract to apiServices or kbnClient calls in setup/teardown.

localStorage / sessionStorage manipulation

• Look for: cy.window().then(win => win.localStorage...), storage key references
• Why: Scout's spaceTest uses isolated Kibana spaces. Storage keys may differ. Async persistence can race with page reloads.
• Scout approach: Use API-based state management. If storage is needed, verify persistence with polling.

cy.task() for server-side operations

• Look for: cy.task('esArchiverLoad'), cy.task('createSignalsIndex'), custom task plugins
• Why: No equivalent in Scout. Tasks are server-side Node.js functions.
• Scout approach: Replace with kbnClient.request(), esClient, or apiServices.

Conditional test logic based on environment

• Look for: @skipInServerless, @skipInEss, if (isServerless), Cypress.env('IS_SERVERLESS')
• Why: Scout handles environment targeting with tags, not runtime conditionals. Conditional logic suggests the test may not be portable as-is.
• Scout approach: Use Scout tags for environment filtering. Split into separate tests if behavior diverges.

Deeply nested .should() chains

• Look for: .should('be.visible').and('contain', 'text').and('have.attr', 'href', '/path')
• Why: Multiple assertions on a single element work differently in Playwright. Each assertion needs its own expect().
• Scout approach: Separate expect() calls: await expect(locator).toBeVisible(), await expect(locator).toContainText('text').

cy.clock() / cy.tick() for time manipulation

• Look for: cy.clock(), cy.tick(), fake timers
• Why: Playwright has its own clock API (page.clock) with different semantics. Direct port is error-prone.
• Scout approach: Use page.clock.install() / page.clock.fastForward(), or redesign to avoid fake timers.

How to use this during triage

1. Read the Cypress test file and all imported helpers (tasks, screens, objects) 2. Check each pattern against the source code 3. Record which patterns are present and their risk level 4. For critical/high patterns: note specific lines and planned Scout remediation 5. For tests with 3+ critical/high patterns: consider writing from scratch rather than porting 6. Include the risk scan findings in the triage summary before proceeding to migration

Migration Best Practices

Table of Contents

• Testing Layer Priority
• Waits and Assertions
• Page Objects
• API-Based Setup/Teardown
• test.step() for Execution Time
• Parallelization
• Fixtures
• Package Organization
• EUI Wrappers

Testing Layer Priority

Only use Scout UI tests for behavior that genuinely requires a browser.

Waits and Assertions

// Forbidden
await page.waitForTimeout(2000);
await page.waitForLoadState('networkidle');  // Anti-pattern — actively removed from Scout tests

// Locator assertions auto-retry
await expect(page.testSubj.locator('myElement')).toBeVisible();

// Poll for async conditions
await expect.poll(async () => {
  return await page.testSubj.locator('alertRow').count();
}).toBeGreaterThan(0);

waitFor() defaults

• { state: 'visible' } is the default — omit it: await element.waitFor() not await element.waitFor({ state: 'visible' })
• Don't use short custom timeouts (e.g., 3s) — they cause CI flakiness. Use the default (10s) unless there is a strong, documented reason.

Built-in auto-waiting

Many Playwright actions auto-wait before executing. Do not add explicit waits before these:

• click(), fill(), clear(), press(), type(), check(), selectOption()
• waitFor() is only needed when you want to assert readiness without performing an action

Page Objects

Extract all locators as `readonly` properties initialized in the constructor — never create locators inline in methods. This keeps selectors centralized and makes them easy to audit or update.

class DashboardPage {
  readonly riskScoreTable: Locator;
  readonly enableEntityStoreButton: Locator;

  constructor(private readonly page: ScoutPage) {
    this.riskScoreTable = this.page.testSubj.locator('entity-analytics-risk-score');
    this.enableEntityStoreButton = this.page.testSubj.locator('enable-entity-store-btn');
  }

  async goto() {
    await this.page.gotoApp('securitySolution:entity_analytics');
  }

  async enableEntityStore() {
    await this.enableEntityStoreButton.click();
  }
}

Split large pages into smaller page objects or component objects. Assertions stay in specs, not page objects.

If a needed data-test-subj doesn't exist, add it to the source component.

Locator preferences

In order of preference:

1. `page.testSubj.locator('...')` — data-test-subj attributes, most stable 2. `getByRole('row')`, `getByRole('button', { name: '...' })` — ARIA roles, semantic and resilient to class changes 3. CSS `:has()` for parent selection — page.locator('span:has([data-test-subj="..."])') over locator('xpath=..') 4. Scoped locators — parent.locator('[data-test-subj="child"]') to avoid strict mode violations

Avoid:

• EUI CSS class selectors (.euiTableRow, .euiToolTipAnchor) — these are internal and can change between EUI versions
• XPath (xpath=..) — less readable, prefer CSS :has() for parent selection
• Unscoped locators when the same data-test-subj appears in multiple DOM locations

API-Based Setup/Teardown

spaceTest.beforeAll(async ({ apiServices }) => {
  await apiServices.ruleService.createRule(ruleConfig);
});

spaceTest.afterAll(async ({ apiServices, scoutSpace }) => {
  await apiServices.ruleService.deleteAllRules();
  await scoutSpace.savedObjects.cleanStandardList();
});

Do not use esArchiver to manipulate system indices — use kbnClient.

test.step() for Execution Time

Each test() block creates a new browser context. Use test.step() for multi-step flows to reuse context:

spaceTest('full workflow', async ({ pageObjects }) => {
  await spaceTest.step('create entity', async () => {
    await pageObjects.entityStore.createEntity(entityConfig);
  });

  await spaceTest.step('verify entity appears', async () => {
    await expect(pageObjects.dashboard.entityRow(entityConfig.name)).toBeVisible();
  });
});

Parallelization

Parallel test runs are encouraged but have trade-offs:

• Test suites must be Space-isolated (spaceTest + scoutSpace)
• Kibana archive ingestion must be done within the test suite file, not in the global setup hook
• Kibana / ES may be slower because multiple workers ingest and interact with the UI concurrently

Use spaceTest + scoutSpace — each worker gets its own Kibana space.

• Pre-ingest shared ES data in parallel_tests/global.setup.ts via globalSetupHook()
• Clean up space-scoped mutations in afterAll
• Place parallel specs in test/scout*/ui/parallel_tests/
• Place sequential specs in test/scout*/ui/tests/

File size and role separation

• Keep spec files focused and small: aim for 4–5 short test scenarios or 2–3 long scenarios per file. This is critical for parallel execution, where the test runner balances work at the spec-file level — oversized specs create bottlenecks.
• Keep one role per file to simulate a realistic user flow. If tests use different auth roles (e.g., CRUD vs read-only), split them into separate spec files with the appropriate login in each file's beforeEach.

Fixtures

Test fixture — each test gets a fresh, isolated instance:

export const test = baseTest.extend<MyFixtures>({
  myFixture: async ({}, use) => {
    const resource = await createResource();
    await use(resource);
    await cleanupResource(resource);
  },
});

Worker fixture — shared across tests within the same worker:

export const test = baseTest.extend<{}, MyWorkerFixtures>({
  sharedService: [async ({}, use) => {
    const service = await initService();
    await use(service);
  }, { scope: 'worker' }],
});

Package Organization

Put shared code in @kbn/scout, security-specific code in @kbn/scout-security.

EUI Wrappers

Scout provides wrappers for stable EUI interactions — import from @kbn/scout: EuiComboBoxWrapper, EuiDataGridWrapper, EuiSelectableWrapper, EuiCheckBoxWrapper, EuiFieldTextWrapper, EuiCodeBlockWrapper, EuiSuperSelectWrapper, EuiToastWrapper

Kibana Component Interaction Patterns

Patterns learned from real migrations. These apply across all Kibana plugins, not just Security.

Kibana query bar (QueryStringInput)

The unified SearchBar's QueryStringInput submits this.props.query (the React prop) on Enter — not the DOM textarea value. Playwright's fill() sets the DOM value synchronously, but React's props update asynchronously. If press('Enter') fires before props sync, the component submits the stale (old) query and the change never takes effect.

// Broken — fill() races with React prop sync
await textarea.fill('host.name: *');
await textarea.press('Enter'); // submits stale props.query (empty string)

// Working — pressSequentially types character-by-character, giving React time
await textarea.click();
await textarea.clear();
await textarea.pressSequentially('host.name: *');
await textarea.press('Enter');

This applies to any QueryStringInput in Kibana (Timeline, Discover, rule builders, etc.).

EuiBasicTable empty-state row

EuiBasicTable always renders a <tr class="euiTableRow"> for its "no items found" message. You cannot assert .euiTableRow count as 0 — the empty-state row is always present.

// Broken — always finds at least 1 row (the empty-state row)
await expect(table.locator('.euiTableRow')).toHaveCount(0);

// Working — assert the empty-state message text
await expect(table).toContainText('0 timelines match the search criteria');

When the table has actual data rows, the empty-state row is not rendered, so row counts > 0 work normally.

EUI disabled button tooltip

EUI wraps disabled buttons in a tooltip anchor <span> that intercepts pointer events. To trigger the tooltip on hover, target the wrapper element, not the button.

// Broken — hover never reaches the disabled button
await saveButton.hover();

// Working — hover the tooltip anchor wrapper using CSS :has()
const tooltipAnchor = page.locator('span:has([data-test-subj="save-button"])');
await tooltipAnchor.hover();
await expect(tooltip).toBeVisible();

Scoping locators to avoid strict mode violations

Some elements (e.g., save-status badges, action buttons) appear in multiple DOM locations. Scope locators to the relevant container to avoid Playwright strict mode violations.

// Risky — may match elements in the bottom bar AND the header panel
readonly saveStatus = this.page.testSubj.locator('timeline-save-status');

// Safe — scoped to the header panel
readonly saveStatus = this.panel.locator('[data-test-subj="timeline-save-status"]');

dispatchEvent for app-level DOM instability

When an application bug causes continuous DOM re-rendering (e.g., a useEffect loop triggering table refetch()), elements inside affected containers get detached before Playwright's actionability checks complete. Use dispatchEvent('click') instead of force: true — it bypasses actionability checks without triggering the playwright/no-force-option lint rule.

// EUI's collapsed actions popover re-renders continuously due to
// app bug in StatefulOpenTimeline: useEffect on noteIds triggers refetch().
// See: open_timeline/index.tsx lines ~406-419
await this.createFromTemplateButton.dispatchEvent('click');

Always document the app bug and the affected source location. This is distinct from porting Cypress { force: true } blindly.

Avoid .first(), .last(), .nth() — use specific locators

The playwright/no-nth-methods lint rule forbids positional methods. Alternatives:

// Forbidden — positional indexing
await actionsButton.first().click();
await rows.nth(0).toContainText('Second');

// For waitFor() — remove .first(), waitFor doesn't enforce strict mode
await actionsButton.waitFor({ state: 'visible' });

// For click() — ensure the locator matches a single element
// (e.g., scope to a specific table tab where only one row exists)
await actionsButton.click();

// For ordered assertions — toContainText accepts an array
await expect(rows).toContainText(['Second', 'First']);

// For filtering — use filter() instead of nth()
await rows.filter({ hasText: 'Security Timeline' }).click();

toContainText with an array checks that each element in the locator list contains the corresponding text in order — same ordering guarantee as nth() without positional indexing.