Api Testing

<objective> A response that adds a nullable field or quietly drops one slips past toHaveProperty spot-checks and silently breaks the frontend in production. Schema-as-contract tests catch that drift in CI, not prod. This skill produces REST and GraphQL API tests that assert response shape, status codes, headers, auth boundaries, and timing — against a real test environment, not a mocked stand-in. </objective>

Discovery Questions

Check .agents/qa-project-context.md first — if it exists, use it and skip anything already answered there. Then:

1. REST, GraphQL, or both? REST-only suites use standard HTTP assertions. GraphQL needs query/mutation builders and benefits from an introspection-diff snapshot. 2. Auth mechanism? JWT, API key, OAuth 2.0, or session cookies — each needs a different fixture strategy. 3. OpenAPI/Swagger spec available? If yes, auto-generate Zod schemas as contracts (orval, openapi-zod-client) and consider spec-driven fuzzing with Schemathesis.

---

Core Principles

1. Test contracts, not implementations. Assert on response shape, status codes, and headers — not on internal logic or database state. 2. Schema validation catches drift before it breaks consumers. A failing schema test means you caught a breaking change before your frontend did. 3. Auth flows are tests too — don't just hardcode tokens. Test login, refresh, expiration, and permission boundaries. 4. Response time is a testable assertion. Performance regressions caught in CI are cheaper than production incidents.

---

Exploratory vs Automated: Tooling

API exploration (debugging, manual probing, OpenAPI playground) and automated API testing are different jobs. Use the right tool for each:

Skip Postman/Insomnia for new projects unless your team already has investment there — file-based tools (Bruno, Hurl) are easier to review in PRs and survive when collections drift.

Playwright API Testing

APIRequestContext supports standalone API tests without launching a browser and shares cookie/storage state with browser contexts. Use it for:

• Standalone API tests — request.get/post/... with status, header, and body assertions.
• Combined browser + API tests — seed data via API, assert it appears in the UI, then clean up via API.
• Authenticated fixtures — log in once in a fixture, hand a pre-authenticated APIRequestContext to tests, and dispose it on teardown. Never hardcode tokens.

See references/playwright-setup.md for the playwright.config.ts, standalone tests, combined browser+API test, and the authenticated API fixture.

---

Schema Validation

Validate response shape against a schema rather than spot-checking individual fields with toHaveProperty. Two common approaches:

• Zod 4 — define a schema, safeParse the response, and assert result.success. Log result.error.issues on failure for a precise diff. Use the Zod 4 native string formats: z.email(), z.uuid(), z.iso.datetime() — the chained z.string().email() forms are deprecated and slated for removal.
• AJV with JSON Schema — when you already have JSON Schema (e.g. from an OpenAPI spec), compile and validate with ajv + ajv-formats.

Schema-as-contract: have both the API and the tests import the same schema file. If the response shape changes, consumer tests fail immediately. With an OpenAPI spec, auto-generate the schema (orval or openapi-zod-client). For spec-first teams, add Schemathesis as a CI job to fuzz the live API against the spec and catch undocumented shapes and edge-case 500s.

See references/schema-validation.md for the Zod 4, AJV, schema-as-contract, and Schemathesis implementations.

---

Test Patterns

Cover each endpoint with a happy-path test plus at least one error-path test. The common patterns:

• CRUD lifecycle — a describe.serial block that creates, reads, updates, deletes, then verifies the 404. Carries the resource id across steps.
• Auth flows — login success, invalid credentials (401), expired token (401), token refresh, and permission boundary (403). Treat auth as its own describe block.
• Error responses — 400 (malformed body), 422 (validation with field details), 429 (rate limit + retry-after). Don't ship happy-path-only suites.
• Response headers — assert content-type, cache-control, and rate-limit headers directly (not behind a conditional that may never fire). See the pattern below.
• Pagination — first-page metadata, out-of-bounds empty page, and rejection of invalid page size.
• File upload/download — multipart upload and content-disposition header verification.
• GraphQL — a small gql helper, then query / mutation / invalid-query (errors array) cases, plus an introspection-diff snapshot to catch silently-removed fields.
• Webhooks — spin up a throwaway HTTP server, register a webhook, trigger the event, and assert delivery.

See references/test-patterns.md for the full runnable implementations of every pattern above plus performance assertions.

Response Headers

Headers carry the contract: cache directives, rate-limit info, content type, CORS policy. Assert them with response.headers() and index by lowercase name; don't gate the assertion behind an if (rateLimited) that may not fire.

test('GET /api/users sets expected response headers', async ({ request }) => {
  const response = await request.get('/api/users');
  const headers = response.headers();

  expect(headers).toBeDefined();
  expect(headers['content-type']).toContain('application/json');
  expect(headers['cache-control']).toBeDefined();   // "no-store" | "max-age=60" | ...
});

For the rate-limit and retry-after variants, see references/test-patterns.md (Response Header Validation).

---

Performance Assertions

Response time and payload size are testable assertions — assert that a hot endpoint responds within a budget (e.g. 500ms), that payloads stay under a size ceiling, and that the API survives a burst of concurrent requests without 5xx. See references/test-patterns.md (Performance Assertions section) for the code.

---

Anti-Patterns

1. Hardcoded auth tokens

Tokens expire, rotate, and differ across environments. Use a login fixture that acquires tokens dynamically.

2. Testing against production

API tests create, modify, and delete data. Run against a dedicated test environment or local instance.

3. Not validating error responses

Happy-path-only suites miss the most common production issues. Test 400, 401, 403, 404, and 500 responses for every endpoint.

4. Asserting headers only conditionally

Headers carry cache directives, rate limit info, content type, and CORS policy. Assert them directly on every relevant response — a check buried inside if (rateLimited) may never run and proves nothing.

5. No cleanup after test data creation

Tests that create resources without deleting them pollute the database. Use afterEach/afterAll hooks or fixture teardown.

6. Treating API tests as unit tests

Don't mock the database — API tests verify the contract from the consumer's perspective. Mock only genuine third parties you don't own (payment gateways, external SaaS).

7. Ignoring idempotency

PUT and DELETE should be idempotent. Test that calling them twice produces the same result.

---

Done When

• Every target endpoint has at least a happy-path test and at least one error-path test (4xx or 5xx response validated).
• Auth flow tested as its own describe block: successful login, invalid credentials, expired token, and permission boundary (403).
• Schema validation assertions on response shape using Zod 4 or AJV — not just toHaveProperty spot-checks.
• Header assertions exist for at least content-type and any cache/rate-limit headers the API sets, asserted unconditionally.
• Contract tests in place for any endpoint consumed by a different team or service (shared schema file; for consumer-driven verification use contract-testing).
• Genuine third-party calls (payment gateways, external SaaS) are mocked or virtualized; the API and its database run for real.
• CI job for the suite exits 0 (green) against the test environment.

Reference Files (in references/)

• playwright-setup.md — playwright.config.ts, standalone API tests, combined browser+API tests, and the authenticated APIRequestContext fixture.
• schema-validation.md — Zod 4 and AJV/JSON-Schema response validation, the schema-as-contract pattern, and Schemathesis spec-driven fuzzing.
• test-patterns.md — Runnable CRUD lifecycle, auth flows, error responses, response headers, pagination, file upload/download, GraphQL (+ introspection diff), webhook, and performance tests.

Related Skills

• contract-testing — Consumer-driven contract verification with Pact/broker; go there when a separate team consumes your API and you need guaranteed compatibility, not just a shared schema.
• playwright-automation — Browser-based E2E testing, Page Object Model, and combined browser + API patterns.
• ci-cd-integration — Running API test suites in CI pipelines, parallelization, and environment management.
• test-strategy — Deciding what to test at the API layer vs. unit vs. E2E.

Playwright API Testing — Setup & Fixtures

APIRequestContext supports standalone API tests without launching a browser and shares cookie/storage state with browser contexts. The Exploratory vs Automated tooling table and the decision prose live in SKILL.md; this file holds the runnable config, standalone tests, combined browser+API tests, and the authenticated fixture.

Configuration and Standalone Tests

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './api-tests',
  use: {
    baseURL: process.env.API_BASE_URL ?? 'http://localhost:3000',
    extraHTTPHeaders: { 'Accept': 'application/json' },
  },
});

import { test, expect } from '@playwright/test';

test.describe('Users API', () => {
  test('GET /api/users returns a list', async ({ request }) => {
    const response = await request.get('/api/users');
    expect(response.status()).toBe(200);
    expect(response.headers()['content-type']).toContain('application/json');

    const body = await response.json();
    expect(body.users).toBeInstanceOf(Array);
    expect(body.users[0]).toHaveProperty('id');
    expect(body.users[0]).toHaveProperty('email');
  });

  test('GET /api/users/:id returns 404 for missing user', async ({ request }) => {
    const response = await request.get('/api/users/non-existent-id');
    expect(response.status()).toBe(404);
  });
});

Combined Browser + API Tests

test('project created via API appears in dashboard', async ({ request, page }) => {
  const createRes = await request.post('/api/projects', {
    data: { name: 'API-Created Project', description: 'Seeded via API' },
  });
  expect(createRes.ok()).toBeTruthy();
  const project = await createRes.json();

  await page.goto('/dashboard');
  await expect(page.getByText('API-Created Project')).toBeVisible();

  await request.delete(`/api/projects/${project.id}`); // cleanup
});

Authenticated API Fixture

// fixtures/api.fixture.ts
import { test as base, expect, APIRequestContext } from '@playwright/test';

export const test = base.extend<{ authedApi: APIRequestContext }>({
  authedApi: async ({ playwright }, use) => {
    const api = await playwright.request.newContext({
      baseURL: process.env.API_BASE_URL ?? 'http://localhost:3000',
      extraHTTPHeaders: { 'Accept': 'application/json' },
    });
    const loginRes = await api.post('/api/auth/login', {
      data: { email: process.env.TEST_USER_EMAIL!, password: process.env.TEST_USER_PASSWORD! },
    });
    expect(loginRes.ok()).toBeTruthy();
    await use(api);
    await api.dispose();
  },
});
export { expect };

Schema Validation — Zod, AJV, Contract

Runnable schema-validation code. The decision prose (when to validate schema, the schema-as-contract idea) lives in SKILL.md; this file holds the implementations.

Zod (Zod 4 native form)

Zod 4 vs Zod 3. Zod 4 shipped major API changes. Three to know: (1) String formats moved to top-level functions — z.email(), z.uuid(), z.iso.datetime() replace the chained z.string().email(), z.string().uuid(), z.string().datetime(). The chained forms still work but emit deprecation warnings and are slated for removal in the next major; write the new form. (2) z.coerce syntax changed and the error format is different — if your codebase mixes Zod 3 and 4 packages, error-format consumers silently break, so pin the version per package. (3) z.uuid() is now strict per RFC 9562/4122; use z.guid() for a permissive "UUID-like" check. For OpenAPI → Zod codegen, `orval` and `openapi-zod-client` are the maintained round-trip tools.

import { z } from 'zod';
import { test, expect } from '@playwright/test';

const UserSchema = z.object({
  id: z.uuid(),
  email: z.email(),
  name: z.string().min(1),
  role: z.enum(['admin', 'member', 'viewer']),
  createdAt: z.iso.datetime(),
});

const UsersListSchema = z.object({
  users: z.array(UserSchema),
  total: z.number().int().nonnegative(),
  page: z.number().int().positive(),
  pageSize: z.number().int().positive(),
});

test('GET /api/users matches schema', async ({ request }) => {
  const response = await request.get('/api/users');
  const result = UsersListSchema.safeParse(await response.json());
  if (!result.success) console.error('Schema errors:', result.error.issues);
  expect(result.success).toBe(true);
});

AJV with JSON Schema

import Ajv from 'ajv';
import addFormats from 'ajv-formats';

const ajv = new Ajv({ allErrors: true });
addFormats(ajv);

const userSchema = {
  type: 'object',
  required: ['id', 'email', 'name', 'role'],
  properties: {
    id: { type: 'string', format: 'uuid' },
    email: { type: 'string', format: 'email' },
    name: { type: 'string', minLength: 1 },
    role: { type: 'string', enum: ['admin', 'member', 'viewer'] },
  },
  additionalProperties: false,
};

test('GET /api/users/:id conforms to JSON Schema', async ({ request }) => {
  const body = await (await request.get('/api/users/some-valid-id')).json();
  expect(ajv.compile(userSchema)(body)).toBe(true);
});

Schema-as-Contract Pattern

Both API and tests import the same schema file. If the response shape changes, consumer tests fail immediately. With an OpenAPI spec, auto-generate the schema with orval or openapi-zod-client (the maintained round-trip tools) so the contract stays in sync with the spec.

// shared/schemas/user.schema.ts  (imported by both API and tests)
import { z } from 'zod';
export const UserResponseSchema = z.object({
  id: z.uuid(),
  email: z.email(),
  name: z.string(),
  role: z.enum(['admin', 'member', 'viewer']),
  createdAt: z.iso.datetime(),
});
export type UserResponse = z.infer<typeof UserResponseSchema>;

Spec-Driven & Property-Based Validation

When you have an OpenAPI spec, you can go further than hand-written schemas. Schemathesis (Python, built on Hypothesis) generates thousands of test cases from the spec and catches 500s on edge-case input, undocumented response shapes, and validation bypasses — zero per-endpoint maintenance. Point it at the live API and the spec:

schemathesis run http://localhost:3000/openapi.json --checks all

Run it as a CI job for spec-first teams; it complements (does not replace) the targeted happy/error-path tests in test-patterns.md.

API Test Patterns & Performance Assertions

Runnable test implementations for the common API testing patterns. The decision prose, anti-patterns, and "done when" criteria live in SKILL.md; this file holds the code.

CRUD Lifecycle Test

import { test, expect } from '../fixtures/api.fixture';

test.describe.serial('Projects CRUD lifecycle', () => {
  let projectId: string;

  test('CREATE', async ({ authedApi }) => {
    const res = await authedApi.post('/api/projects', {
      data: { name: 'Lifecycle Project', description: 'CRUD test' },
    });
    expect(res.status()).toBe(201);
    const body = await res.json();
    expect(body).toHaveProperty('id');
    projectId = body.id;
  });

  test('READ', async ({ authedApi }) => {
    const res = await authedApi.get(`/api/projects/${projectId}`);
    expect(res.status()).toBe(200);
    expect((await res.json()).name).toBe('Lifecycle Project');
  });

  test('UPDATE', async ({ authedApi }) => {
    const res = await authedApi.patch(`/api/projects/${projectId}`, {
      data: { name: 'Updated Name' },
    });
    expect(res.status()).toBe(200);
    expect((await res.json()).name).toBe('Updated Name');
  });

  test('DELETE', async ({ authedApi }) => {
    expect((await authedApi.delete(`/api/projects/${projectId}`)).status()).toBe(204);
  });

  test('VERIFY DELETED', async ({ authedApi }) => {
    expect((await authedApi.get(`/api/projects/${projectId}`)).status()).toBe(404);
  });
});

Auth Flow Testing

test.describe('Authentication flows', () => {
  test('successful login returns tokens', async ({ request }) => {
    const res = await request.post('/api/auth/login', {
      data: { email: 'user@example.com', password: 'correct-password' },
    });
    expect(res.status()).toBe(200);
    const body = await res.json();
    expect(body).toHaveProperty('accessToken');
    expect(body).toHaveProperty('refreshToken');
  });

  test('invalid credentials return 401', async ({ request }) => {
    const res = await request.post('/api/auth/login', {
      data: { email: 'user@example.com', password: 'wrong' },
    });
    expect(res.status()).toBe(401);
  });

  test('expired token returns 401', async ({ request }) => {
    const res = await request.get('/api/users/me', {
      headers: { Authorization: 'Bearer expired-token-here' },
    });
    expect(res.status()).toBe(401);
  });

  test('token refresh provides new access token', async ({ request }) => {
    const { refreshToken } = await (await request.post('/api/auth/login', {
      data: { email: 'user@example.com', password: 'correct-password' },
    })).json();
    const refreshRes = await request.post('/api/auth/refresh', { data: { refreshToken } });
    expect(refreshRes.status()).toBe(200);
    expect((await refreshRes.json())).toHaveProperty('accessToken');
  });

  test('insufficient permissions return 403', async ({ request }) => {
    const { accessToken } = await (await request.post('/api/auth/login', {
      data: { email: 'viewer@example.com', password: 'viewer-password' },
    })).json();
    const res = await request.delete('/api/admin/users/some-id', {
      headers: { Authorization: `Bearer ${accessToken}` },
    });
    expect(res.status()).toBe(403);
  });
});

Error Response Validation

test.describe('Error responses', () => {
  test('400 - malformed request body', async ({ request }) => {
    const res = await request.post('/api/projects', { data: { name: '' } });
    expect(res.status()).toBe(400);
    const body = await res.json();
    expect(body.details).toEqual(
      expect.arrayContaining([expect.objectContaining({ field: 'name' })]),
    );
  });

  test('422 - validation error with field details', async ({ request }) => {
    const res = await request.post('/api/users', { data: { email: 'not-an-email', name: 'Test' } });
    expect(res.status()).toBe(422);
    expect((await res.json()).details).toEqual(
      expect.arrayContaining([expect.objectContaining({ field: 'email' })]),
    );
  });

  test('429 - rate limiting returns retry-after header', async ({ request }) => {
    const responses = await Promise.all(
      Array.from({ length: 20 }, () => request.get('/api/status')),
    );
    const rateLimited = responses.find(r => r.status() === 429);
    if (rateLimited) {
      expect(rateLimited.headers()['retry-after']).toBeDefined();
    }
  });
});

Response Header Validation

Headers carry the contract too: content-type, cache directives, and rate-limit info. Assert them directly — don't gate the assertion behind an if that may never fire.

test('GET /api/users sets expected response headers', async ({ request }) => {
  const response = await request.get('/api/users');
  const headers = response.headers();

  expect(headers).toBeDefined();
  expect(headers['content-type']).toContain('application/json');
  expect(headers['cache-control']).toBeDefined();      // e.g. "no-store" or "max-age=60"
});

test('rate-limited endpoint exposes limit headers', async ({ request }) => {
  const response = await request.get('/api/status');
  const headers = response.headers();

  // Assert the headers exist on every response, not only on a 429.
  expect(headers['x-ratelimit-limit']).toBeDefined();
  expect(headers['x-ratelimit-remaining']).toBeDefined();
});

test('429 returns a retry-after header', async ({ request }) => {
  const responses = await Promise.all(
    Array.from({ length: 50 }, () => request.get('/api/status')),
  );
  const limited = responses.find((r) => r.status() === 429);
  expect(limited, 'expected at least one 429 from the burst').toBeDefined();
  expect(limited!.headers()['retry-after']).toBeDefined();
});

Pagination Testing

test('first page returns correct metadata', async ({ request }) => {
  const body = await (await request.get('/api/projects?page=1&pageSize=10')).json();
  expect(body.page).toBe(1);
  expect(body.items.length).toBeLessThanOrEqual(10);
  expect(body.total).toBeGreaterThanOrEqual(body.items.length);
});

test('out of bounds page returns empty items', async ({ request }) => {
  const body = await (await request.get('/api/projects?page=99999&pageSize=10')).json();
  expect(body.items).toHaveLength(0);
});

test('invalid page size is rejected', async ({ request }) => {
  expect((await request.get('/api/projects?page=1&pageSize=0')).status()).toBe(400);
});

File Upload/Download via API

test('upload via multipart form', async ({ request }) => {
  const res = await request.post('/api/files/upload', {
    multipart: {
      file: { name: 'sample.csv', mimeType: 'text/csv', buffer: Buffer.from('id,name\n1,Test') },
    },
  });
  expect(res.status()).toBe(201);
  expect((await res.json()).fileName).toBe('sample.csv');
});

test('download and verify headers', async ({ request }) => {
  const res = await request.get('/api/files/some-file-id/download');
  expect(res.headers()['content-disposition']).toContain('attachment');
});

GraphQL Testing

test.describe('GraphQL API', () => {
  const gql = (request: any, query: string, variables?: Record<string, unknown>) =>
    request.post('/graphql', { data: { query, variables } });

  test('query - fetches user by ID', async ({ request }) => {
    const body = await (await gql(request, `
      query GetUser($id: ID!) { user(id: $id) { id email name } }
    `, { id: 'user-1' })).json();

    expect(body.errors).toBeUndefined();
    expect(body.data.user).toMatchObject({ id: 'user-1', email: expect.any(String) });
  });

  test('mutation - creates a project', async ({ request }) => {
    const body = await (await gql(request, `
      mutation CreateProject($input: CreateProjectInput!) {
        createProject(input: $input) { id name }
      }
    `, { input: { name: 'GQL Project' } })).json();

    expect(body.errors).toBeUndefined();
    expect(body.data.createProject.name).toBe('GQL Project');
  });

  test('invalid query returns errors array', async ({ request }) => {
    const body = await (await gql(request, `query { nonExistentField }`)).json();
    expect(body.errors).toBeDefined();
    expect(body.errors[0]).toHaveProperty('message');
  });

});

GraphQL Introspection-Diff Snapshot

The highest-value GraphQL API test: assert that fields and resolvers didn't disappear without a migration note. Snapshot the introspected type map and fail when it shrinks unexpectedly — a removed field is a breaking change for every consumer.

test('schema has not lost public fields', async ({ request }) => {
  const introspection = `
    { __schema { types { name fields { name } } } }`;
  const body = await (await gql(request, introspection)).json();
  const userType = body.data.__schema.types.find((t: any) => t.name === 'User');
  const fields = userType.fields.map((f: any) => f.name).sort();

  // Snapshot serializes to a committed file; a removed field fails the diff.
  expect(fields).toMatchSnapshot('user-type-fields.json');
});

Webhook Testing

import http from 'http';

test.describe('Webhook delivery', () => {
  let server: http.Server;
  let payloads: any[] = [];
  let webhookUrl: string;

  test.beforeAll(async () => {
    server = http.createServer((req, res) => {
      let body = '';
      req.on('data', (c) => (body += c));
      req.on('end', () => { payloads.push(JSON.parse(body)); res.writeHead(200).end(); });
    });
    await new Promise<void>((r) => server.listen(0, r));
    webhookUrl = `http://localhost:${(server.address() as any).port}`;
  });
  test.afterAll(() => server?.close());

  test('receives event on project creation', async ({ request }) => {
    const { id: hookId } = await (await request.post('/api/webhooks', {
      data: { url: webhookUrl, events: ['project.created'] },
    })).json();
    await request.post('/api/projects', { data: { name: 'Webhook Project' } });
    await new Promise((r) => setTimeout(r, 2000));
    expect(payloads.at(-1).event).toBe('project.created');
    await request.delete(`/api/webhooks/${hookId}`);
  });
});

Performance Assertions

test('GET /api/users responds within 500ms', async ({ request }) => {
  const start = Date.now();
  const res = await request.get('/api/users');
  expect(res.ok()).toBeTruthy();
  expect(Date.now() - start).toBeLessThan(500);
});

test('response payload stays under 1MB', async ({ request }) => {
  const body = await (await request.get('/api/users')).body();
  expect(body.length / 1024).toBeLessThan(1024);
});

test('handles 50 concurrent requests without errors', async ({ request }) => {
  const results = await Promise.all(
    Array.from({ length: 50 }, () => request.get('/api/status').then(r => r.status())),
  );
  expect(results.every(s => s >= 200 && s < 500)).toBe(true);
});