
Test Data Management
- 255 installs
- 55 repo stars
- Updated June 10, 2026
- petrkindlmann/qa-skills
Test Data Management is a QA skill that defines factory, fixture, seeding, and anonymization patterns so automated tests can run on isolated deterministic data.
About
Test Data Management is a QA skill for factories, fixtures, synthetic data, database seeding, anonymization, and cleanup. It helps engineers escape flaky shared fixtures and unsafe production copies by giving each test isolated, deterministic data. Reach for it when setting up Fishery or FactoryBot, masking PII for staging, or designing parallel-safe teardown. It explicitly defers DB migration testing and environment provisioning to sibling skills.
- Fishery, FactoryBot, and Factory Boy patterns with faker
- GDPR-minded anonymization pipeline and compliance checklist
- Idempotent seed scripts and per-test vs per-suite strategies
- Cleanup via rollback, truncate, or API teardown for E2E
Test Data Management by the numbers
- 255 all-time installs (skills.sh)
- +51 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #759 of 2,153 Testing & QA skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/petrkindlmann/qa-skills --skill test-data-managementAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 255 |
|---|---|
| repo stars | ★ 55 |
| Last updated | June 10, 2026 |
| Repository | petrkindlmann/qa-skills ↗ |
How do you create realistic test data that stays isolated, parallel-safe, and free of production PII?
Designs factories, fixtures, anonymization, seeds, and cleanup for isolated deterministic test data.
Who is it for?
QA and backend engineers designing factories, seeds, or anonymization for unit, integration, and E2E suites.
Skip if: Database schema migration validation or full test-environment provisioning without a data strategy focus.
When should I use this skill?
The user mentions test data, fixtures, factories, seed data, synthetic data, or data anonymization for tests.
What you get
A test data plan with factory/fixture choices, seed and cleanup strategy, and anonymization steps where needed.
Files
<objective> Create, maintain, and clean up test data that is deterministic, isolated, realistic, and safe. Good test data is the foundation of reliable tests -- without it, tests are either flaky (shared mutable state), unrealistic (hardcoded nonsense values), or dangerous (production PII in test environments). This skill delivers factories, fixtures, idempotent seeds, anonymization pipelines, and cleanup strategies that survive parallel execution. </objective>
---
Quick Route
| Situation | Go to |
|---|---|
| Need fresh entity data with per-test overrides | Factory Patterns → references/factories.md |
| Mocking an API response or golden file | Fixture Strategies → references/factories.md |
| Copying production data anywhere non-prod | Data Anonymization |
| Populating a test DB / reference data idempotently | Database Seeding → references/seeding-and-synthetic.md |
| Cleaning up after tests / parallel isolation | Cleanup Strategies → references/seeding-and-synthetic.md |
| Generating edge cases and boundary values | Synthetic Data → references/seeding-and-synthetic.md |
---
Discovery Questions
Before designing a test data strategy, understand the current state. Check .agents/qa-project-context.md first -- if it exists, use it as the foundation and skip questions already answered there.
Current Data Practices
- How is test data created today? (manually, scripts, copy of production, none)
- Do tests share data or does each test create its own?
- How is test data cleaned up? (truncate, rollback, manual, never)
- Are there seed scripts? Are they idempotent?
Privacy and Compliance
- Does the product handle PII? (names, emails, addresses, phone numbers, SSNs)
- Are there GDPR, HIPAA, PCI-DSS, or other data protection requirements?
- Is production data ever used in test environments?
Scale and Complexity
- How large are the test datasets? (dozens of records, thousands, millions)
- How complex are the data relationships? (simple CRUD, deep nested hierarchies, polymorphic)
- Are there cross-service data dependencies? (microservices sharing data)
---
Core Principles
1. Each Test Owns Its Data
Tests that rely on pre-existing shared data are fragile. When Test A modifies shared data, Test B breaks. Every test should create exactly the data it needs, verify against that data, and clean up after itself. This enables parallel execution and eliminates ordering dependencies.
2. Factories Over Fixtures for Dynamic Data
Static fixtures (JSON/YAML files) are appropriate for reference data that does not change (country codes, currency lists). For entity data that tests create and manipulate (users, orders, products), use factory functions that generate fresh instances with sensible defaults and allow per-test overrides.
3. Anonymize Production Data Before Use
Production databases contain the most realistic data, but they also contain real user information. Never copy production data to test environments without anonymization. Replace PII with synthetic equivalents while preserving data distributions and relationships.
4. Deterministic Data Enables Reproducible Tests
Tests should produce the same results regardless of when or where they run. Avoid Math.random(), Date.now(), or auto-increment IDs in assertions. Use seeded random generators (faker.seed(n)), fixed timestamps, factory sequences, and -- when an ID must be a UUID you assert on -- a seeded faker.string.uuid() so it stays stable across runs.
5. Minimize Data, Maximize Signal
Create only the data each test needs. A test for user search does not need a complete user profile with billing address, payment method, and order history. Over-specified test data obscures the intent of the test and increases maintenance burden.
---
Factory Patterns
Factories are functions that produce test data with sensible defaults, allowing individual tests to override only what matters for their scenario. Fishery (2.4.0) is the default for TypeScript, FactoryBot (6.6.0) for Ruby, factory-boy (3.3.3) for Python; all pair with faker (v10.4.0) for realistic field values.
See references/factories.md for the full Fishery (with associations and deterministic UUIDs), FactoryBot (User + Product out_of_stock/discounted traits), Factory Boy (class Params + Trait), and Playwright fixture implementations. The shape every factory follows:
- Defaults + overrides —
Factory.defineproduces sensible defaults; tests pass overrides for the one field they care about (userFactory.build({ role: 'admin' })). - Sequences for unique fields —
sequence(Fishery),sequence(:email)(FactoryBot),factory.Sequence(Factory Boy) — never hardcode IDs or emails. - Traits for variants — name common states (
:admin,:inactive,out_of_stock,discounted) instead of spawning a fixture file per combination. - Associations — one factory builds another (an order builds its user), keeping referential structure without manual wiring.
When to Use Factories vs Fixtures
| Scenario | Factories | Static Fixtures |
|---|---|---|
| Entity data that tests create/modify | Yes | No |
| Reference data (countries, currencies, configs) | No | Yes |
| Data with many variations per test | Yes | No -- file explosion |
| Data with complex relationships | Yes -- associations | No -- hard to maintain |
| API response mocks | No | Yes -- JSON fixtures |
| Snapshot/golden file comparisons | No | Yes |
Decision rule: If the data has a lifecycle (created, modified, deleted during tests), use a factory. If the data is read-only reference material, use a fixture file.
---
Fixture Strategies
Three fixture shapes, all in references/factories.md:
- Static fixtures (JSON/YAML) — best for API response mocks (
page.route+route.fulfill), config data, and golden file comparisons. - Dynamic fixtures (Playwright) —
test.extendcreates data via API before the test and deletes it afterawait use(...). The standard per-test setup/teardown. - Fixture composition — combine factory-built data (
userFactory.build(),orderFactory.buildList(3)) inside a singletest.extendthat seeds and cleans up in one step.
---
Data Anonymization
When production data is needed for realistic testing, anonymize it before use.
PII Masking Rules
| Data Type | Anonymization Method | Example |
|---|---|---|
| Faker email with original domain pattern | jane.doe@acme.com -> user-7291@test.example.com | |
| Full name | Faker name | Jane Doe -> Alice Johnson |
| Phone number | Faker phone, preserve format | +1-555-123-4567 -> +1-555-987-6543 |
| Address | Faker address, preserve country/region | 123 Main St, NYC -> 456 Oak Ave, NYC |
| SSN/National ID | Test pattern | 123-45-6789 -> 000-00-0001 |
| Credit card | Test card numbers | 4111-... -> 4242-4242-4242-4242 |
| Date of birth | Shift by fixed offset | 1990-03-15 -> 1987-07-22 |
The anonymization pipeline -- seeded Faker for determinism, an in-memory lookup table, parent-records-first ordering, and a wrapping transaction -- is in references/seeding-and-synthetic.md (Anonymization with Faker.js, Referential Integrity During Anonymization). Anonymizing a user's email must also update that email everywhere it is referenced (orders, comments, audit logs); process parents first, children second, using the same lookup, all inside one transaction.
GDPR Compliance Checklist
- [ ] No real PII exists in any non-production environment
- [ ] Anonymization is irreversible (no lookup table mapping back to originals is stored)
- [ ] Anonymization preserves data distributions (age ranges, geographic spread) for realistic testing
- [ ] Anonymized data cannot be re-identified through combination of quasi-identifiers
- [ ] Data retention policies apply to test environments (auto-delete after N days)
- [ ] The anonymization pipeline runs automatically, not manually (eliminates human error)
---
Database Seeding
Idempotent Seed Scripts
Seed scripts must be safe to run multiple times without duplicating data. Use upsert -- INSERT ... ON CONFLICT (natural_key) DO UPDATE SET ... -- keyed on a stable natural key, not the primary key. A DELETE-then-INSERT "reset" is not idempotent: it breaks foreign keys and reassigns serial IDs. See references/seeding-and-synthetic.md (Idempotent Seed Scripts) for the full INSERT ... ON CONFLICT (code) DO UPDATE countries/currencies example and the reasoning.
Database Branching (DB-as-a-Service)
If your prod DB lives on Neon, Supabase, or PlanetScale, branching can give a PR its own database instead of seeding from scratch -- but the providers differ on whether the branch carries data:
- Neon Branching — copy-on-write Postgres branches in seconds, with data; ideal for ephemeral preview envs. The strongest "PR gets a real DB copy" story.
- Supabase Branching —
supabase branches create pr-123clones schema and (optionally, from a backup) data; preview env points at the branch URL. - PlanetScale Branching — MySQL branches are schema-only by default (no data), so you still seed the branch. Note: PlanetScale removed its free Hobby tier (April 2024); MySQL now starts at ~$39/mo, Postgres ~$5/mo.
Pair with the Preview Environments pattern in test-environments.
Avoid: Snaplet (hosted) — shut down 31 Aug 2024; the team joined Supabase. @snaplet/seedlives on as supabase-community/seed (community-maintained, last meaningful release v0.98.0,July 2024, no feature work since). For new projects, prefer the DB-branching providers above
plus factory-generated seeds.
Per-Test vs Per-Suite Data
| Strategy | When to Use | Pros | Cons |
|---|---|---|---|
| Per-test setup/teardown | Tests that modify data | Full isolation, parallel-safe | Slower, more setup code |
| Per-suite seed | Read-only reference data | Fast, simple | Cannot be modified by tests |
| Per-worker seed | Playwright parallel workers | Balances speed and isolation | Requires worker-scoped fixtures |
| Global seed | Environment bootstrap | Runs once, sets up baseline | Must be idempotent, shared state risk |
For the worker-scoped fixture (test.extend with { scope: 'worker' }) that powers per-worker seeding, see references/factories.md (Worker-Scoped Seeding).
Cleanup Strategies
| Strategy | When to use | Speed |
|---|---|---|
| Transaction rollback | Unit/integration tests with direct DB access | Fastest |
Truncation (TRUNCATE ... CASCADE) | Resetting tables between suites | Medium |
| API-based cleanup | E2E tests with no direct DB access | Slowest |
Transaction rollback cannot clean up E2E tests -- the app opens its own DB connections, so a test-side transaction can't undo the app's writes; use API-based cleanup (delete in reverse creation order) there. All three implementations are in references/seeding-and-synthetic.md (Cleanup Strategies).
---
Synthetic Data Generation
Factories should make it easy to generate edge cases and boundary values without hand-writing them per test. The reusable arrays and helpers -- edgeCaseStrings (empty, whitespace, very long, XSS, SQL injection, null/control chars, RTL override), edgeCaseDates, and boundaryValues(min, max) driving a test.each -- are in references/seeding-and-synthetic.md (Synthetic Data Generation).
---
Anti-Patterns
Shared Mutable Test Data
Multiple tests reading and writing the same database rows. Test A creates a user, Test B modifies it, Test C asserts on the original state and fails. Fix by having each test create its own data through factories.
Production Data Without Anonymization
Copying the production database to staging for "realistic testing." This violates GDPR, risks data breaches in less-secured environments, and creates compliance liability. Always anonymize before use, or generate synthetic data that matches production distributions.
Non-Deterministic Data
Using Math.random() or Date.now() in test data creation without seeding. Tests pass on Monday and fail on Tuesday because the random name generated happens to exceed a field length limit. Use seeded Faker instances and fixed timestamps.
No Cleanup Strategy
Tests that create data and never clean it up. The test database grows until it affects performance, or stale data causes false positives in other tests. Every data creation must have a corresponding cleanup.
Fixture File Explosion
Creating a separate JSON fixture file for every test variation. Instead of user-admin.json, user-inactive.json, user-admin-inactive.json, use a factory with traits. Fixtures should be reserved for static reference data and API response mocks.
Over-Specified Test Data
Creating a complete user object with 30 fields when the test only cares about role. This obscures intent and makes tests brittle. Factories with sensible defaults solve this: override only what the test cares about.
Hard-Coded IDs
Using userId: '1' in tests. This couples tests to database state and breaks when running in parallel (ID collision) or against a database with existing data. Use factory sequences or seeded UUIDs (see Core Principle 4).
---
Verification
Prove the data layer is deterministic, isolated, and PII-free, smallest check first:
1. Seeds are idempotent — run the seed twice back-to-back and diff the row counts: psql -c "SELECT count(*) FROM countries" && <seed> && psql -c "SELECT count(*) FROM countries" returns the same number both times and exits 0. A growing count means a missing ON CONFLICT. 2. No shared mutable state — run the suite under parallelism and randomized order: npx playwright test --workers=4 (or pytest -n auto -p randomly) stays green. A failure that only appears here is an ordering or shared-data dependency. 3. Determinism holds — run the same data-generating test twice; with faker.seed(n) set, generated names/IDs/UUIDs match across runs. If they drift, an unseeded Faker call or Date.now()/crypto.randomUUID() leaked in. 4. No real PII — grep -rE '@(gmail|outlook|yahoo)\.com|[0-9]{3}-[0-9]{2}-[0-9]{4}' tests/ fixtures/ returns nothing (real-looking emails and SSNs). Anything it finds is an anonymization gap. 5. Cleanup returns to baseline — snapshot row counts before the suite, run it, snapshot again: the test DB is back to baseline with no orphaned records.
---
Done When
- Every entity type the suite creates has a factory or fixture (no inline ad-hoc object literals in tests for shared entities -- grep the test dir for hand-built fixtures and confirm none remain).
- Test data is isolated per test -- the suite passes with parallelism on (
--workers=N/pytest -n auto) and under randomized order (--shuffle/-p randomly), proving no shared mutable state or ordering dependency. - Seed scripts are idempotent -- running the seed twice in a row produces the same row count and exits 0; the CI job runs them with no manual intervention.
- No real PII used in test fixtures -- all sensitive data anonymized or synthetic (grep for production domains / real-looking SSNs returns nothing).
- Data cleanup verified -- row counts in the test DB return to baseline after the suite (no orphaned records accumulate across runs).
---
Reference Files (in references/)
- factories.md — Full Fishery (associations, deterministic UUIDs), FactoryBot (User + Product traits), Factory Boy (
Params/Trait), static/dynamic/composed Playwright fixtures, and the worker-scoped seeding fixture. - seeding-and-synthetic.md — Idempotent
ON CONFLICTseed script, Faker.js anonymization + referential-integrity pipeline, cleanup strategies (rollback / truncate / API), and synthetic edge-case + boundary-value generators.
Related Skills
- unit-testing -- Unit tests are the primary consumer of factory-generated data; this skill provides the data layer.
- api-testing -- API tests use both factories (for request bodies) and fixtures (for mocked responses).
- playwright-automation -- E2E tests need test data seeded via API or fixtures before browser interaction.
- test-reliability -- Deterministic test data eliminates a major source of test flakiness.
- test-environments -- Owns environment provisioning and database-branching strategy (Neon, Supabase, PlanetScale) for preview envs; this skill owns the data that fills them.
- database-testing -- Migration testing, data-integrity assertions, and Testcontainers for the database layer specifically -- go there to test the DB, come here to populate it.
- ci-cd-integration -- Database seeding and cleanup must be integrated into CI pipeline stages.
Factory & Fixture Patterns
Full implementations for Fishery (TypeScript), FactoryBot (Ruby), Factory Boy (Python), and Playwright fixtures. Cited from SKILL.md (Factory Patterns and Fixture Strategies).
Factories are functions that produce test data with sensible defaults, allowing individual tests to override only what matters for their scenario.
Fishery (TypeScript)
Fishery (2.4.0) is the recommended factory library for TypeScript projects. It provides type safety, traits, sequences, associations, and transient parameters.
npm install --save-dev fishery @faker-js/fakerFaker v10 (latest v10.4.0) needs modern Node. v10 is ESM-only but still loads from
CommonJS via Node's require(esm) support on Node 20.19+ / 22.13+ / 24+. Pin@faker-js/faker@^9 only if you must support older Node or a bundler that lacksrequire(esm). On supported Node,require('@faker-js/faker')works in v10 — no migration
needed.
// tests/factories/user.factory.ts
import { Factory } from 'fishery';
import { faker } from '@faker-js/faker';
interface User {
id: string;
email: string;
name: string;
role: 'admin' | 'member' | 'viewer';
organizationId: string;
createdAt: Date;
isActive: boolean;
}
export const userFactory = Factory.define<User>(({ sequence, params }) => ({
id: `user-${sequence}`,
email: `user-${sequence}@test.example.com`,
name: faker.person.fullName(),
role: params.role ?? 'member',
organizationId: params.organizationId ?? `org-${sequence}`,
createdAt: new Date('2025-01-15T10:00:00Z'),
isActive: true,
}));
// Trait variants
const adminUser = userFactory.params({ role: 'admin' });
const orgMembers = userFactory.params({ organizationId: 'org-shared' });Using in Tests
import { userFactory } from '../factories/user.factory';
const user = userFactory.build(); // Sensible defaults
const admin = userFactory.build({ role: 'admin' }); // Override specific fields
const users = userFactory.buildList(5); // Build multiple
const orgMembers = userFactory.buildList(3, { organizationId: 'org-1' }); // With associationsAssociations Between Factories
// tests/factories/order.factory.ts
import { Factory } from 'fishery';
import { userFactory } from './user.factory';
interface Order {
id: string;
userId: string;
items: Array<{ productId: string; quantity: number; unitPrice: number }>;
totalCents: number;
status: 'pending' | 'paid' | 'shipped' | 'delivered' | 'cancelled';
}
export const orderFactory = Factory.define<Order>(({ sequence }) => {
const items = [{ productId: `prod-${sequence}`, quantity: 2, unitPrice: 1999 }];
return {
id: `order-${sequence}`,
userId: userFactory.build().id,
items,
totalCents: items.reduce((sum, i) => sum + i.quantity * i.unitPrice, 0),
status: 'pending',
};
});Deterministic IDs for Snapshot Assertions
Sequences (user-${sequence}) already produce stable IDs. When a field genuinely needs a UUID that must stay stable across runs (e.g. snapshot/golden comparisons), seed Faker first so faker.string.uuid() is reproducible:
import { faker } from '@faker-js/faker';
faker.seed(42); // Same UUID sequence on every run
const stableId = faker.string.uuid(); // deterministic under the seedNever use a raw crypto.randomUUID() in data you assert on — it changes every run and breaks snapshots.
FactoryBot (Ruby)
FactoryBot (6.6.0, thoughtbot). The Product example below shows the out_of_stock and discounted traits with a price field:
# spec/factories/products.rb
FactoryBot.define do
factory :product do
sequence(:name) { |n| "Product #{n}" }
price { Faker::Commerce.price(range: 1.0..500.0) }
category { Faker::Commerce.department }
stock { 50 }
trait :out_of_stock do stock { 0 } end
trait :discounted do price { 9.99 } end
end
end
# Usage: create(:product), create(:product, :out_of_stock), create(:product, :discounted)A User factory with :admin / :inactive traits and an association:
# spec/factories/users.rb
FactoryBot.define do
factory :user do
sequence(:email) { |n| "user-#{n}@test.example.com" }
name { Faker::Name.name }
role { :member }
organization
trait :admin do role { :admin } end
trait :inactive do is_active { false } end
end
end
# Usage: create(:user), create(:user, :admin), create_list(:user, 3, :inactive)Factory Boy (Python)
factory-boy (3.3.3). Use class Params with factory.Trait for variant flags:
# tests/factories.py
import factory
from myapp.models import User
class UserFactory(factory.django.DjangoModelFactory):
class Meta:
model = User
email = factory.Sequence(lambda n: f"user-{n}@test.example.com")
username = factory.Sequence(lambda n: f"user{n}")
name = factory.Faker("name")
role = "member"
is_active = True
class Params:
admin = factory.Trait(role="admin")
inactive = factory.Trait(is_active=False)
# Usage: UserFactory(), UserFactory(admin=True), UserFactory.create_batch(3, inactive=True)
# Multiple users with different is_active values:
active_users = UserFactory.create_batch(2)
inactive_users = UserFactory.create_batch(2, inactive=True)Fixture Strategies
Static Fixtures (JSON/YAML)
Best for API response mocks, configuration data, and golden file comparisons.
// Using JSON fixtures in Playwright tests
import productsResponse from '../fixtures/data/api-responses/products.json';
test('displays products from API', async ({ page }) => {
await page.route('**/api/products*', async (route) => {
await route.fulfill({ status: 200, contentType: 'application/json', body: JSON.stringify(productsResponse) });
});
await page.goto('/products');
await expect(page.getByText('Widget')).toBeVisible();
});Dynamic Fixtures (Playwright)
Use Playwright fixtures to create and clean up data per test:
// e2e/fixtures/data.fixture.ts
import { test as base, expect } from '@playwright/test';
import { userFactory } from '../factories/user.factory';
export const test = base.extend<{ testOrder: { id: string; userId: string } }>({
testOrder: async ({ request }, use) => {
// Unique userId from the factory sequence -- never `Date.now()` (Core Principle 4)
const response = await request.post('/api/test/orders', {
data: { userId: userFactory.build().id, items: [{ productId: 'prod-1', quantity: 1 }] },
});
expect(response.ok()).toBeTruthy();
const order = await response.json();
await use(order);
await request.delete(`/api/test/orders/${order.id}`);
},
});Fixture Composition
Compose fixtures from smaller, reusable pieces by combining factory-generated data with Playwright fixtures:
// e2e/fixtures/composed.fixture.ts
import { test as base } from '@playwright/test';
import { userFactory } from '../factories/user.factory';
import { orderFactory } from '../factories/order.factory';
export const test = base.extend<{ seedData: { user: { id: string }; orders: Array<{ id: string }> } }>({
seedData: async ({ request }, use) => {
const resp = await request.post('/api/test/seed', {
data: { user: userFactory.build(), orders: orderFactory.buildList(3) },
});
const seedData = await resp.json();
await use(seedData);
await request.post('/api/test/cleanup', { data: { userId: seedData.user.id } });
},
});Worker-Scoped Seeding (Playwright parallel workers)
A worker-scoped fixture seeds baseline reference data once per parallel worker instead of once per test — balancing speed and isolation. Use it for read-only data that every test in a worker shares; keep mutable data per-test.
// e2e/fixtures/worker-seed.fixture.ts
import { test as base } from '@playwright/test';
export const test = base.extend<{}, { workerSeed: { orgId: string } }>({
workerSeed: [async ({}, use, workerInfo) => {
const orgId = `org-w${workerInfo.parallelIndex}`;
// seed baseline rows scoped to this worker's org so workers never collide
await seedOrg(orgId);
await use({ orgId });
await cleanupOrg(orgId);
}, { scope: 'worker' }],
});Seeding, Anonymization & Synthetic Data
Full code for idempotent seeding, the anonymization pipeline, cleanup strategies, and synthetic data generation. Cited from SKILL.md (Database Seeding, Data Anonymization, Synthetic Data).
Idempotent Seed Scripts
Seed scripts must be safe to run multiple times without duplicating data. Use upsert (INSERT ... ON CONFLICT ... DO UPDATE) keyed on a stable natural key — not the primary key — so a re-run updates the existing row instead of inserting a duplicate.
-- seeds/reference_data.sql -- countries and currencies (reference data)
INSERT INTO countries (code, name, currency) VALUES
('US', 'United States', 'USD'),
('GB', 'United Kingdom', 'GBP'),
('JP', 'Japan', 'JPY')
ON CONFLICT (code) DO UPDATE
SET name = EXCLUDED.name,
currency = EXCLUDED.currency;ON CONFLICT (code) DO UPDATE matters because reference data (country codes, currency lists) is seeded into every environment and CI run. Without it, a second run either errors on the unique constraint or — if you wrote DELETE then INSERT to "reset" the table — is not idempotent: the DELETE breaks foreign keys from rows created between runs and reassigns serial IDs, so anything referencing those rows now points at the wrong record. Upsert leaves IDs and foreign keys intact.
Anonymization with Faker.js
When production data is needed for realistic testing, anonymize it before use.
// scripts/anonymize.ts
import { faker } from '@faker-js/faker';
faker.seed(42); // Deterministic output across runs
function anonymizeUser(user: Record<string, unknown>, index: number) {
return {
...user,
email: `user-${index + 1}@test.example.com`,
name: faker.person.fullName(),
phone: faker.phone.number(),
// faker.date.birthdate returns a Date -- serialize to ISO before a DB write
dateOfBirth: faker.date.birthdate({ min: 18, max: 80, mode: 'age' }).toISOString(),
ssn: `000-00-${String(index + 1).padStart(4, '0')}`,
};
}Referential Integrity During Anonymization
Anonymizing a user's email must also update their email in orders, comments, audit logs, and every other table that references it. Build an anonymization pipeline that:
1. Maps original values to anonymized values in a lookup table (in memory, for the duration of the run only). 2. Processes parent records first, then child records using the same lookup, so foreign keys stay consistent. 3. Validates referential integrity after anonymization. 4. Runs in a transaction so partial anonymization cannot occur.
// scripts/anonymize-pipeline.ts -- sketch
const lookup = new Map<string, string>(); // origEmail -> anonEmail, never persisted
await db.transaction(async (tx) => {
// parents first: users
for (const [i, u] of users.entries()) {
const anon = anonymizeUser(u, i);
lookup.set(u.email as string, anon.email as string);
await tx.users.update(u.id, anon);
}
// children: rewrite the FK email column using the same lookup
for (const o of orders) {
await tx.orders.update(o.id, { customerEmail: lookup.get(o.customerEmail) });
}
});The lookup is held in memory for the run and discarded — do not persist a reversible mapping back to originals (that defeats GDPR irreversibility).
Cleanup Strategies
Transaction Rollback (Fastest): Wrap each test in a transaction and roll back after. Works for unit and integration tests with direct DB access. Not usable for E2E tests that hit the app over HTTP — the app opens its own connections, so a test-side transaction can't roll back the app's writes.
let tx: Transaction;
beforeEach(async () => { tx = await db.beginTransaction(); });
afterEach(async () => { await tx.rollback(); });Truncation (Thorough): Delete all data from test tables between suites. Use TRUNCATE TABLE ... CASCADE for efficiency.
API-Based Cleanup (E2E Tests): For E2E tests that cannot access the database directly, register resources for cleanup via a fixture and delete in reverse creation order (children before parents):
export const test = base.extend<{ cleanup: (id: string, type: string) => void }>({
cleanup: async ({ request }, use) => {
const toClean: Array<{ id: string; type: string }> = [];
await use((id, type) => toClean.push({ id, type }));
for (const r of toClean.reverse()) {
await request.delete(`/api/test/${r.type}/${r.id}`);
}
},
});Synthetic Data Generation
Edge Case Distributions
Factories should make it easy to generate edge case data:
// tests/factories/edge-cases.ts
export const edgeCaseStrings = [
'', // Empty string
' leading and trailing ', // Whitespace
'a'.repeat(10_000), // Very long string
'<script>alert("xss")</script>', // XSS attempt
"Robert'); DROP TABLE users;--", // SQL injection
'\u0000\u0001\u0002', // Null/control characters
'\u202Eoverride\u202C', // RTL override
];
export const edgeCaseDates = [
new Date('1970-01-01T00:00:00Z'), // Unix epoch
new Date('2038-01-19T03:14:07Z'), // 32-bit overflow
new Date('2024-02-29T00:00:00Z'), // Leap day
new Date('2025-03-09T02:30:00-05:00'), // During DST transition
];Boundary Value Generation
export function boundaryValues(min: number, max: number): number[] {
return [min - 1, min, min + 1, Math.floor((min + max) / 2), max - 1, max, max + 1];
}
// Usage
test.each(boundaryValues(1, 100).map(v => [v]))(
'validates quantity %i correctly',
(quantity) => {
const result = validateQuantity(quantity);
if (quantity >= 1 && quantity <= 100) {
expect(result.valid).toBe(true);
} else {
expect(result.valid).toBe(false);
}
}
);Related skills
FAQ
Factories or static fixtures?
Use factories for entity data with lifecycle changes; use static JSON/YAML for read-only reference or API mocks.
Can production data be copied to test?
Only after irreversible anonymization that preserves distributions without storing re-identification maps.
How should E2E tests clean up?
Prefer API-based deletion in reverse creation order because app DB connections break transaction rollback.