Scout Ui Testing

Scout UI Testing

Pick the right test mode

• Sequential UI: <module-root>/test/scout*/ui/tests/**/*.spec.ts.
• Parallel UI: <module-root>/test/scout*/ui/parallel_tests/**/*.spec.ts and (recommended) use spaceTest + scoutSpace (one Kibana space per worker). If you run with workers > 1 but keep using test, you won't get space isolation.
• Use the Scout package that matches the module root:
• src/platform/** or x-pack/platform/** -> @kbn/scout
• x-pack/solutions/observability/** -> @kbn/scout-oblt
• x-pack/solutions/search/** -> @kbn/scout-search
• x-pack/solutions/security/** -> @kbn/scout-security

Imports

• Test framework + tags: import { tags } from '@kbn/scout'; (or the module's Scout package)
• Test fixture: import { test } from '../fixtures'; (or import { test } from '@kbn/scout'; when not extending)
• Assertions: import { expect } from '@kbn/scout/ui'; (or @kbn/scout-oblt/ui, etc.) — not from the main entry
• expect is not exported from the main @kbn/scout entry. Use the /ui subpath for UI tests.

Non-negotiable conventions

• Tags are required: Scout validates UI test tags at runtime. Ensure each test has at least one supported tag (typically by tagging the top-level test.describe(...) / spaceTest.describe(...), e.g. tags.deploymentAgnostic, tags.stateful.classic, or tags.performance).
• No `@` in test titles: Playwright treats @word in test/describe titles as tags. Do not use @ followed by word characters in titles (e.g., @timestamp, @elastic). This causes Scout tag validation to fail with "Unsupported tag(s) found". Rephrase the title instead (e.g., use timestamp field instead of @timestamp).
• Prefer one suite per file: keep a single top-level test.describe(...) (sequential) or spaceTest.describe(...) (parallel) and avoid nested describe blocks where possible.
• UI actions live in page objects; assertions stay in the spec.
• Use APIs for setup/teardown: prefer apiServices/kbnClient/esArchiver in hooks over clicking through the UI.

Auth (UI)

• Use browserAuth — available methods: loginAsAdmin(), loginAsPrivilegedUser(), loginAsViewer(), loginAs(role), loginWithCustomRole(role).
• Prefer least privilege: use loginAsViewer() or loginWithCustomRole() over loginAsAdmin().
• Avoid loginAsAdmin() unless the test is explicitly about admin-only behavior.

Page objects (UI)

• Prefer page.testSubj.locator(...), role/label locators; avoid brittle CSS.
• Keep selectors + interactions inside the page object class. Do not use `expect` assertions in page objects — use waitForSelector for waiting on elements. Assertions belong in test specs only.
• Keep route mocks out of page objects — page objects are for UI interactions only. Put page.route() mocks in a dedicated fixtures/mocks.ts file as standalone functions that accept page as a parameter. See cloud_security_posture/test/scout_cspm_agentless/ui/fixtures/mocks.ts for the reference pattern.
• Don't make API calls from page objects (use apiServices/kbnClient in hooks instead).
• Register plugin page objects by extending the pageObjects fixture in test/scout*/ui/fixtures/index.ts.
• Use `readonly` class fields for static locators — assign them in the constructor, not as getter methods. Use methods only for parameterized locators/actions. See DashboardApp in kbn-scout for the reference pattern.
• Scout provides EUI component wrappers for stable interactions with common EUI widgets: EuiComboBoxWrapper, EuiDataGridWrapper, EuiSelectableWrapper, EuiCheckBoxWrapper, EuiFieldTextWrapper, EuiCodeBlockWrapper, EuiSuperSelectWrapper, EuiToastWrapper. Import them from @kbn/scout and use them as class members in page objects.
• Avoid `.first()`, `.nth()`, `.last()` — the playwright/no-nth-methods lint rule flags these. Instead, use data-test-subj attributes or other targeted selectors. If the component lacks a data-test-subj, add one rather than disabling the rule.
• Do not disable eslint rules — avoid eslint-disable comments in test files. Fix the underlying issue (e.g., use targeted selectors instead of positional ones, add data-test-subj to the components) rather than suppressing the lint rule.

Parallel UI specifics (spaceTest)

• Use spaceTest so you can access scoutSpace for worker-isolated saved objects + UI settings.
• Pre-ingest shared ES data in parallel_tests/global.setup.ts via globalSetupHook(...).
• Only worker fixtures are available there (no page, browserAuth, pageObjects).
• Cleanup space-scoped mutations in afterAll (scoutSpace.savedObjects.cleanStandardList(), unset UI settings you set).

Extending fixtures

Most modules extend the base test (or spaceTest) in test/scout*/ui/fixtures/index.ts to add custom page objects and auth helpers:

import { test as baseTest } from '@kbn/scout'; // or the module's Scout package
import type { ScoutTestFixtures, ScoutWorkerFixtures, ScoutPage } from '@kbn/scout';

class MyPluginPage {
  constructor(private readonly page: ScoutPage) {}
  async goto() { await this.page.gotoApp('myPlugin'); }
}

interface ExtendedFixtures extends ScoutTestFixtures {
  pageObjects: ScoutTestFixtures['pageObjects'] & { myPlugin: MyPluginPage };
}

export const test = baseTest.extend<ExtendedFixtures, ScoutWorkerFixtures>({
  pageObjects: async ({ pageObjects, page }, use) => {
    await use({ ...pageObjects, myPlugin: new MyPluginPage(page) });
  },
});

Tests then import from local fixtures: import { test } from '../fixtures';

Multi-step flows with test.step()

Use test.step(...) to group related actions within a single test. Steps appear in Playwright's trace viewer and HTML report, making failures easier to debug without splitting into many small tests:

test('creates and verifies a dashboard', async ({ pageObjects, page }) => {
  await test.step('create dashboard', async () => {
    await pageObjects.dashboard.create('My Dashboard');
  });
  await test.step('verify dashboard appears in list', async () => {
    await expect(page.testSubj.locator('dashboardTitle')).toHaveText('My Dashboard');
  });
});

Waiting + flake control

• Don’t use page.waitForTimeout. Wait on a page-ready signal (loading indicator hidden, container visible, expect.poll on element counts).
• If selectors aren’t stable, add data-test-subj (Scout uses it as the testIdAttribute).
• Some locators are restricted by @kbn/eslint/scout_no_locators (e.g. globalLoadingIndicator). Don’t use them in tests or page objects for app loading state management; rely on Playwright auto-waiting and page-ready signals instead.

A11y checks (optional, high value)

• Use page.checkA11y() at a few stable checkpoints (landing pages, modals/flyouts).
• Prefer include scoped checks; assert violations is empty.

Run / debug quickly

• Use either --config or --testFiles (they are mutually exclusive).
• Run by config: node scripts/scout.js run-tests --arch stateful --domain classic --config <module-root>/test/scout*/ui/playwright.config.ts (or .../ui/parallel.playwright.config.ts for parallel UI)
• Run by file/dir (Scout derives the right playwright.config.ts vs parallel.playwright.config.ts): node scripts/scout.js run-tests --arch stateful --domain classic --testFiles <module-root>/test/scout*/ui/tests/my.spec.ts
• For faster iteration, start servers once in another terminal: node scripts/scout.js start-server --arch stateful --domain classic [--serverConfigSet <configSet>], then run Playwright directly: npx playwright test --config <...> --project local --grep <tag> --headed.
• run-tests auto-detects custom config sets from .../test/scout_<name>/... paths.
• start-server has no Playwright config to inspect, so pass --serverConfigSet <name> when your tests require a custom config set.
• Debug: SCOUT_LOG_LEVEL=debug, or npx playwright test --config <...> --project local --ui

CI enablement

• Scout tests run in CI only for modules listed under plugins.enabled / packages.enabled in .buildkite/scout_ci_config.yml.
• node scripts/scout.js generate registers the module under enabled so the new configs run in CI.

References

Open only what you need:

• Browser authentication helpers and patterns: references/scout-browser-auth.md
• Parallel UI (spaceTest + scoutSpace) isolation + global setup rules: references/scout-ui-parallelism.md
• API services patterns (setup/teardown helpers shared with UI): ../scout-api-testing/references/scout-api-services.md

Scout Browser Authentication

Use the browserAuth fixture to authenticate UI tests without manual UI logins.

Common methods

• loginAsAdmin()
• loginAsPrivilegedUser()
• loginAsViewer()
• loginAs(role: string)
• loginWithCustomRole(role: KibanaRole)

These methods are async and must be awaited.

Basic usage

import { tags } from '@kbn/scout'; // or the module's Scout package (e.g. @kbn/scout-search)
import { expect } from '@kbn/scout/ui'; // or '@kbn/scout-search/ui', etc.
import { test } from '../fixtures';

test.describe('my suite', { tag: tags.deploymentAgnostic }, () => {
  test.beforeEach(async ({ browserAuth }) => {
    await browserAuth.loginAsViewer();
  });

  test('does something', async ({ page }) => {
    await expect(page.testSubj.locator('someElement')).toBeVisible();
  });
});

Custom roles

Use loginWithCustomRole() for one-off permission sets. If a custom role is used across multiple tests, extend the browserAuth fixture in the plugin's test/scout*/ui/fixtures and add a helper like loginAsMyRole().

Parallel UI Tests (spaceTest + scoutSpace)

Use this when working under .../test/scout*/ui/parallel_tests/ or a parallel.playwright.config.ts.

Key rules

• Parallelism is file-level. Tests within one file still run sequentially.
• Use spaceTest (not test) so you can access scoutSpace.
• Tags are required: every spaceTest.describe(...) must include { tag: ... }.
• Each worker gets a dedicated Kibana space; the fixture creates and deletes it automatically.
• Pre-ingest shared Elasticsearch data before workers start (global setup hook). Avoid ingesting or cleaning shared indices inside individual parallel tests.

Minimal config + layout

• ui/parallel.playwright.config.ts: testDir: './parallel_tests', workers: 2..3, optional runGlobalSetup: true.
• ui/parallel_tests/global.setup.ts: define globalSetupHook(...) (runs once total).

Minimal pattern

import { spaceTest, tags } from '@kbn/scout'; // or the module's Scout package
import { expect } from '@kbn/scout/ui'; // or '@kbn/scout-oblt/ui', etc.

spaceTest.describe('my feature', { tag: tags.deploymentAgnostic }, () => {
  spaceTest.beforeAll(async ({ scoutSpace }) => {
    // Worker-scoped setup in the isolated space.
    await scoutSpace.savedObjects.cleanStandardList();
  });

  spaceTest.afterAll(async ({ scoutSpace }) => {
    await scoutSpace.savedObjects.cleanStandardList();
  });

  spaceTest('does something', async ({ browserAuth, pageObjects, page }) => {
    await browserAuth.loginAsViewer();
    await pageObjects.somePage.goto();
    await expect(page.testSubj.locator('someElement')).toBeVisible();
  });
});

Global setup hook gotchas

• Global setup runs once total (executed by the first worker); other workers wait for it to finish.
• Only worker-scoped fixtures are available (for example esArchiver, apiServices, kbnClient, esClient, log).
• No page, browserAuth, or pageObjects in global.setup.ts.