Scout Api Testing

Scout API Testing

Core rules (API)

• API specs live in <module-root>/test/scout*/api/{tests,parallel_tests}/**/*.spec.ts (examples: test/scout/api/..., test/scout_uiam_local/api/...).
• Use the Scout package that matches the module root:
• src/platform/** or x-pack/platform/** -> @kbn/scout
• x-pack/solutions/observability/** -> @kbn/scout-oblt
• x-pack/solutions/search/** -> @kbn/scout-search
• x-pack/solutions/security/** -> @kbn/scout-security
• Prefer a single top-level apiTest.describe(...) per file and avoid nested describe blocks; multiple top-level describes are supported, but files get hard to read quickly.
• Tags: add { tag: ... } on the suite (or individual tests) so CI/discovery can select the right test target. For solution modules, prefer explicit targets (e.g. [...tags.stateful.classic, ...tags.serverless.observability.complete] in Observability); reserve tags.deploymentAgnostic mainly for platform specs that truly need every deployment-agnostic target (see scout-migrate-from-ftr). Unlike UI tests, API tests don’t currently validate tags at runtime.
• No `@` in test titles: Playwright treats @word in test/describe titles as tags. Do not use @ followed by word characters in titles (e.g., @timestamp, @elastic). Rephrase the title instead (e.g., use timestamp field instead of @timestamp).
• If the module provides Scout fixtures, import apiTest from <module-root>/test/scout*/api/fixtures to get module-specific extensions. Importing directly from the module’s Scout package is also fine when you don’t need extensions.
• Browser fixtures are disabled for apiTest (no page, browserAuth, pageObjects).

Imports

• Test framework + tags: import { apiTest, tags } from '@kbn/scout'; (or the module's Scout package, e.g. @kbn/scout-oblt)
• Assertions: import { expect } from '@kbn/scout/api'; (or @kbn/scout-oblt/api, etc.) — not from the main entry
• Types: import type { RoleApiCredentials } from '@kbn/scout';
• expect is not exported from the main @kbn/scout entry. Use the /api subpath for API tests.

Auth: pick based on endpoint

• api/* endpoints: use API keys via requestAuth (getApiKey, getApiKeyForCustomRole).
• internal/* endpoints: use cookies via samlAuth.asInteractiveUser(...).

Recommended test shape

1. Prepare environment (optional): apiServices/kbnClient/esArchiver in beforeAll. 2. Authenticate (least privilege): generate credentials in beforeAll and reuse. 3. Request: call the endpoint with apiClient and the right headers. 4. Assert: verify statusCode and response body; verify side effects via apiServices/kbnClient when needed.

Important: apiServices/kbnClient run with elevated privileges. Don’t use them to validate the endpoint under test (use apiClient + scoped auth).

Header reminders:

• State-changing requests usually need kbn-xsrf.
• Prefer sending x-elastic-internal-origin: kibana for Kibana APIs.
• Include elastic-api-version for versioned public APIs (e.g. '2023-10-31') or internal APIs (e.g. '1').

Assertions

• apiClient methods (get, post, put, delete, patch, head) return { statusCode, body, headers }.
• Use the custom matchers from @kbn/scout/api:
• expect(response).toHaveStatusCode(200)
• expect(response).toHaveStatusText('OK')
• expect(response).toHaveHeaders({ 'content-type': 'application/json' })
• Standard matchers (toBe, toStrictEqual, toMatchObject, etc.) and asymmetric matchers (expect.objectContaining(...), expect.any(String)) are also available.

API services

• Put reusable server-side helpers behind apiServices (no UI interactions). Use it for setup/teardown and verifying side effects, not for RBAC validation.
• Module-local service: create it under <module-root>/test/scout*/api/services/<service>_api_service.ts (or similar). Register it by extending the module's apiServices fixture in <module-root>/test/scout*/api/fixtures/index.ts (prefer { scope: 'worker' } when the helper doesn't need per-test state).
• Shared service (reused across modules): consider contributing it to the Scout packages under src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/.

Extending fixtures

When tests need custom auth helpers or API services, extend apiTest in the module's fixtures/index.ts:

import { apiTest as base } from '@kbn/scout'; // or the module's Scout package
import type { RequestAuthFixture } from '@kbn/scout';

interface MyApiFixtures {
  requestAuth: RequestAuthFixture & { getMyPluginApiKey: () => Promise<RoleApiCredentials> };
}

export const apiTest = base.extend<MyApiFixtures>({
  requestAuth: async ({ requestAuth }, use) => {
    const getMyPluginApiKey = async () =>
      requestAuth.getApiKeyForCustomRole({
        kibana: [{ base: [], feature: { myPlugin: ['all'] }, spaces: ['*'] }],
      });
    await use({ ...requestAuth, getMyPluginApiKey });
  },
});

Tests then import apiTest from the local fixtures: import { apiTest } from '../fixtures';

Parallelism

• Treat Scout API tests as sequential by default. Parallel API runs require manual isolation (spaces, indices, saved objects) and are uncommon.

Run / debug quickly

• Use either --config or --testFiles (they are mutually exclusive).
• Run by config: node scripts/scout.js run-tests --arch stateful --domain classic --config <module-root>/test/scout*/api/playwright.config.ts (or .../api/parallel.playwright.config.ts for parallel API runs)
• Run by file/dir (Scout derives the right playwright.config.ts vs parallel.playwright.config.ts): node scripts/scout.js run-tests --arch stateful --domain classic --testFiles <module-root>/test/scout*/api/tests/my.spec.ts
• For faster iteration, start servers once in another terminal: node scripts/scout.js start-server --arch stateful --domain classic [--serverConfigSet <configSet>], then run Playwright directly: npx playwright test --config <...> --project local --grep <tag>.
• run-tests auto-detects custom config sets from .../test/scout_<name>/... paths.
• start-server has no Playwright config to inspect, so pass --serverConfigSet <name> when your tests require a custom config set.
• Debug: SCOUT_LOG_LEVEL=debug

CI enablement

• Scout tests run in CI only for modules listed under plugins.enabled / packages.enabled in .buildkite/scout_ci_config.yml.
• node scripts/scout.js generate registers the module under enabled so the new configs run in CI.

References

Open only what you need:

• requestAuth vs samlAuth, headers, and least-privilege auth tips: references/scout-api-auth.md
• Creating and registering apiServices helpers (kbnClient + retries + logging): references/scout-api-services.md

Scout API Test Authentication (requestAuth vs samlAuth)

Use this when writing API tests with apiTest/apiClient, especially when validating RBAC.

Pick an auth method

• api/* endpoints: use API keys via requestAuth.
• internal/* endpoints: use cookies via samlAuth.asInteractiveUser(...).

Both methods return headers you spread into apiClient requests.

API key auth (requestAuth)

• requestAuth.getApiKey(roleName)
• requestAuth.getApiKeyForCustomRole(roleDescriptor)

import type { RoleApiCredentials } from '@kbn/scout'; // or the module's Scout package (e.g. @kbn/scout-oblt)
import { apiTest, tags } from '@kbn/scout'; // or the module's Scout package
import { expect } from '@kbn/scout/api'; // or '@kbn/scout-oblt/api', etc.

const COMMON_HEADERS = {
  'kbn-xsrf': 'scout',
  'x-elastic-internal-origin': 'kibana',
  'elastic-api-version': '2023-10-31', // include for versioned public APIs
};

apiTest.describe('GET /api/my_plugin/foo', { tag: tags.deploymentAgnostic }, () => {
  let viewer: RoleApiCredentials;

  apiTest.beforeAll(async ({ requestAuth }) => {
    viewer = await requestAuth.getApiKey('viewer');
  });

  apiTest('works', async ({ apiClient }) => {
    const response = await apiClient.get('api/my_plugin/foo', {
      headers: { ...COMMON_HEADERS, ...viewer.apiKeyHeader },
      responseType: 'json',
    });

    expect(response).toHaveStatusCode(200);
    expect(response.body).toStrictEqual(expect.objectContaining({ id: expect.any(String) }));
  });
});

Cookie auth (samlAuth) for internal endpoints

import { apiTest, tags } from '@kbn/scout'; // or the module's Scout package
import { expect } from '@kbn/scout/api'; // or '@kbn/scout-oblt/api', etc.

const INTERNAL_HEADERS = {
  'kbn-xsrf': 'scout',
  'x-elastic-internal-origin': 'kibana',
};

apiTest.describe('GET /internal/my_plugin/foo', { tag: tags.deploymentAgnostic }, () => {
  apiTest('calls internal endpoint', async ({ apiClient, samlAuth }) => {
    const { cookieHeader } = await samlAuth.asInteractiveUser('viewer');

    const response = await apiClient.get('internal/my_plugin/foo', {
      headers: { ...INTERNAL_HEADERS, ...cookieHeader },
      responseType: 'json',
    });

    expect(response).toHaveStatusCode(200);
  });
});

API assertions (@kbn/scout/api)

Import expect from @kbn/scout/api (or @kbn/scout-<solution>/api). It provides custom matchers on top of standard ones:

• expect(response).toHaveStatusCode(200) — assert HTTP status code.
• expect(response).toHaveStatusText('OK') — assert HTTP status text.
• expect(response).toHaveHeaders({ 'content-type': 'application/json' }) — assert response headers.
• Standard matchers like toBe, toStrictEqual, toBeDefined, toMatchObject are also available.
• Asymmetric matchers: expect.objectContaining(...), expect.any(String), expect.toBeGreaterThan(0), etc.

apiClient methods (get, post, put, delete, patch, head) return { statusCode, body, headers }.

Tips

• Generate credentials in beforeAll if reused across tests.
• Prefer custom roles for permission-boundary tests instead of admin.

Scout API Services

API services provide server-side helpers through the apiServices fixture. Keep API services strictly server-side (no UI interactions). Import helper utilities (like measurePerformanceAsync) from the Scout package used by the module (@kbn/scout or the relevant solution package).

Create a new API service (summary)

1. Add a new service file under the API fixtures directory. 2. Add a types.ts file for request/response types when the API is non-trivial. 3. Export a helper function that accepts log and kbnClient and uses kbnClient.request with retries (and ignoreErrors when needed). 4. Wrap calls with measurePerformanceAsync for consistent logging. 5. Register the service in the API fixtures index so it appears under apiServices.<name>.

Minimal sketch

import type { KbnClient, ScoutLogger } from '@kbn/scout'; // or the module's Scout package (e.g. @kbn/scout-security)
import { measurePerformanceAsync } from '@kbn/scout'; // or the module's Scout package

export interface MyApiService {
  enable: () => Promise<void>;
}

export const getMyApiService = ({
  log,
  kbnClient,
}: {
  log: ScoutLogger;
  kbnClient: KbnClient;
}): MyApiService => {
  return {
    enable: async () => {
      await measurePerformanceAsync(log, 'myService.enable', async () => {
        await kbnClient.request({
          method: 'POST',
          path: '/api/my/endpoint',
          retries: 3,
        });
      });
    },
  };
};

Register it in the fixture so tests can call:

await apiServices.myService.enable();

When adding a module-local API service, extend the apiServices fixture (prefer worker scope) and merge in the new service:

apiServices: async ({ apiServices, kbnClient, log }, use) => {
  const extended = {
    ...apiServices,
    myService: getMyApiService({ kbnClient, log }),
  };

  await use(extended);
}