Node

When to use

Use this skill whenever you are dealing with Node.js code to obtain domain-specific knowledge for building robust, performant, and maintainable Node.js applications.

TypeScript with Type Stripping

When writing TypeScript for Node.js, use type stripping (Node.js 22.6+) instead of build tools like ts-node or tsx. Type stripping runs TypeScript directly by removing type annotations at runtime without transpilation.

Key requirements for type stripping compatibility:

• Use import type for type-only imports
• Use const objects instead of enums
• Avoid namespaces and parameter properties
• Use .ts extensions in imports

Minimal example — a valid type-stripped TypeScript file:

// greet.ts
import type { IncomingMessage } from 'node:http';

const greet = (name: string): string => `Hello, ${name}!`;
console.log(greet('world'));

Run directly with:

node greet.ts

See rules/typescript.md for complete configuration and examples.

Common Workflows

For multi-step processes, follow these high-level sequences before consulting the relevant rule file:

Graceful shutdown: Register signal handlers (SIGTERM/SIGINT) → stop accepting new work → drain in-flight requests → close external connections (DB, cache) → exit with appropriate code. See rules/graceful-shutdown.md.

Error handling: Define a shared error base class → classify errors (operational vs programmer) → add async boundary handlers (process.on('unhandledRejection')) → propagate typed errors through the call stack → log with context before responding or crashing. See rules/error-handling.md.

Diagnosing flaky tests: Isolate the test with --test-only → check for shared state or timer dependencies → inspect async teardown order → add retry logic as a temporary diagnostic step → fix root cause. See rules/flaky-tests.md.

Diagnosing stuck processes/tests (node --test hangs, "process did not exit", CI timeout, open handles): isolate file/test → run with explicit timeout/reporter → inspect handles via why-is-node-running (SIGUSR1) → patch deterministic teardown in resource-creation scope → rerun isolated + full suite until stable. See rules/stuck-processes-and-tests.md.

Profiling a slow path: Reproduce under realistic load → capture a CPU profile with --cpu-prof → identify hot functions → check for stream backpressure or unnecessary serialisation → validate improvement with a benchmark. See rules/profiling.md and rules/performance.md.

High-priority activation checklist (streams + caching)

When the task mentions CSV, ETL, ingestion pipelines, large file processing, backpressure, repeated lookups, or deduplicating concurrent async calls, explicitly apply this checklist:

1. Use await pipeline(...) from node:stream/promises (prefer this over chained .pipe() in guidance/code). 2. Include at least one explicit async function* transform when data is being transformed in-stream. 3. Choose a cache strategy when repeated work appears:

• lru-cache for bounded in-memory reuse in a single process.
• async-cache-dedupe for async request deduplication / stale-while-revalidate behavior.

4. Show where backpressure is handled (implicitly via pipeline() or explicitly via drain).

Integrated example pattern (CSV/ETL)

For CSV/ETL-style prompts, prefer an answer structure like:

• createReadStream(input)
• async function* parser/transform
• optional cached enrichment lookup (async-cache-dedupe or lru-cache)
• await pipeline(...) to a writable destination

Link relevant rules directly in explanations so models can retrieve details:

• rules/streams.md
• rules/caching.md

How to use

Read individual rule files for detailed explanations and code examples:

• rules/error-handling.md - Error handling patterns in Node.js
• rules/async-patterns.md - Async/await and Promise patterns
• rules/streams.md - Working with Node.js streams
• rules/modules.md - ES Modules and CommonJS patterns
• rules/testing.md - Testing strategies for Node.js applications
• rules/flaky-tests.md - Identifying and diagnosing flaky tests with node:test
• rules/stuck-processes-and-tests.md - Diagnosing processes that do not exit and tests that get stuck
• rules/node-modules-exploration.md - Navigating and analyzing node_modules directories
• rules/performance.md - Performance optimization techniques
• rules/caching.md - Caching patterns and libraries
• rules/profiling.md - Profiling and benchmarking tools
• rules/logging.md - Logging and debugging patterns
• rules/environment.md - Environment configuration and secrets management
• rules/graceful-shutdown.md - Graceful shutdown and signal handling
• rules/typescript.md - TypeScript configuration and type stripping in Node.js

Async Patterns in Node.js

Always Prefer async/await

Use async/await over raw Promises for readability:

// GOOD
async function processItems(items: Item[]): Promise<Result[]> {
  const results: Result[] = [];
  for (const item of items) {
    const result = await processItem(item);
    results.push(result);
  }
  return results;
}

// AVOID - callback-style Promise chains
function processItems(items: Item[]): Promise<Result[]> {
  return Promise.resolve([])
    .then((results) => {
      return items.reduce((chain, item) => {
        return chain.then((r) => processItem(item).then((res) => [...r, res]));
      }, Promise.resolve(results));
    });
}

Parallel Execution with Promise.all

Use Promise.all for independent operations:

async function fetchAllData(ids: string[]): Promise<Data[]> {
  const promises = ids.map((id) => fetchData(id));
  return Promise.all(promises);
}

Controlled Concurrency

Limit concurrent operations to prevent resource exhaustion and extreme memory usage. Use p-limit or p-map:

import pLimit from 'p-limit';

const limit = pLimit(5); // Max 5 concurrent operations

const results = await Promise.all(
  items.map((item) => limit(() => processItem(item)))
);

Or use p-map for cleaner syntax:

import pMap from 'p-map';

const results = await pMap(items, processItem, { concurrency: 5 });

Promise.allSettled for Fault Tolerance

Use Promise.allSettled when some failures are acceptable:

async function fetchMultiple(urls: string[]): Promise<Map<string, string | Error>> {
  const results = await Promise.allSettled(
    urls.map((url) => fetch(url).then((r) => r.text()))
  );

  const map = new Map<string, string | Error>();
  urls.forEach((url, i) => {
    const result = results[i];
    map.set(
      url,
      result.status === 'fulfilled' ? result.value : result.reason
    );
  });

  return map;
}

Avoid Async in Constructors

Constructors cannot be async. Use factory functions instead:

// BAD - constructor cannot await
class Database {
  constructor() {
    // Cannot use await here
  }
}

// GOOD - factory function
class Database {
  private constructor(private connection: Connection) {}

  static async create(config: Config): Promise<Database> {
    const connection = await connect(config);
    return new Database(connection);
  }
}

// Usage
const db = await Database.create(config);

AbortController for Cancellation

Use AbortController to cancel long-running operations:

async function fetchWithTimeout(
  url: string,
  timeoutMs: number
): Promise<Response> {
  const controller = new AbortController();
  const timeoutId = setTimeout(() => controller.abort(), timeoutMs);

  try {
    return await fetch(url, { signal: controller.signal });
  } finally {
    clearTimeout(timeoutId);
  }
}

Caching in Node.js

High-signal triggers

If prompts mention repeated async lookups, duplicate concurrent requests, CSV enrichment, ETL transforms, N+1 remote calls, or cache hot keys, select a cache strategy explicitly and justify it.

Cache selection quick guide

• Use `lru-cache` for process-local, bounded in-memory reuse where deduplicating concurrent requests is not the main concern.
• Use `async-cache-dedupe` when multiple concurrent calls can request the same key and you want one in-flight request per key.
• In stream/ETL scenarios, prefer async-cache-dedupe for enrichment calls inside an async function* transform.

Memoization with mnemoist

Use mnemoist for synchronous memoization:

import { LRUCache } from 'mnemonist';

const cache = new LRUCache<string, User>(1000);

function getUser(id: string): User | undefined {
  if (cache.has(id)) {
    return cache.get(id);
  }
  const user = fetchUserSync(id);
  cache.set(id, user);
  return user;
}

Async Caching with async-cache-dedupe

Use async-cache-dedupe for async operations with request deduplication:

import { createCache } from 'async-cache-dedupe';

const cache = createCache({
  ttl: 60, // seconds
  stale: 5, // serve stale while revalidating
  storage: { type: 'memory' },
});

cache.define('getUser', async (id: string) => {
  return await db.users.findById(id);
});

cache.define('getPost', {
  ttl: 300,
  stale: 30,
}, async (id: string) => {
  return await db.posts.findById(id);
});

// Usage - concurrent calls are deduplicated
const user = await cache.getUser('123');
const post = await cache.getPost('456');

Request Deduplication

async-cache-dedupe automatically deduplicates concurrent requests:

// These three concurrent calls result in only ONE database query
const [user1, user2, user3] = await Promise.all([
  cache.getUser('123'),
  cache.getUser('123'),
  cache.getUser('123'),
]);

Stream/ETL enrichment example

Use deduplicated async cache inside an async function* transform when rows repeatedly reference the same key:

import { createCache } from 'async-cache-dedupe';

const cache = createCache({ ttl: 120, stale: 10, storage: { type: 'memory' } });

cache.define('getPlan', async (planId: string) => {
  return await db.plans.findById(planId);
});

async function* enrichRows(source: AsyncIterable<{ userId: string, planId: string }>) {
  for await (const row of source) {
    const plan = await cache.getPlan(row.planId); // one in-flight call per planId
    yield { ...row, planName: plan.name };
  }
}

Redis Storage

For distributed caching across multiple instances:

import { createCache } from 'async-cache-dedupe';
import Redis from 'ioredis';

const redis = new Redis();

const cache = createCache({
  ttl: 60,
  storage: {
    type: 'redis',
    options: { client: redis },
  },
});

LRU Cache

Use lru-cache for bounded in-memory caching:

import { LRUCache } from 'lru-cache';

const cache = new LRUCache<string, User>({
  max: 500,           // Maximum items
  ttl: 1000 * 60 * 5, // 5 minutes
  updateAgeOnGet: true,
});

cache.set('user:123', user);
const cached = cache.get('user:123');

Cache Invalidation Patterns

Time-Based Expiration

const cache = createCache({
  ttl: 60,    // Fresh for 60 seconds
  stale: 30,  // Serve stale for 30 more seconds while revalidating
});

Manual Invalidation

// Invalidate single entry
await cache.invalidate('getUser', '123');

// Invalidate all entries for a function
await cache.clear('getUser');

// Clear entire cache
await cache.clear();

Reference-Based Invalidation

const cache = createCache({
  ttl: 60,
  storage: { type: 'memory' },
});

cache.define('getUser', {
  references: (args, key, result) => [`user:${result.id}`],
}, async (id: string) => {
  return await db.users.findById(id);
});

cache.define('getUserPosts', {
  references: (args, key, result) => [`user:${args[0]}`],
}, async (userId: string) => {
  return await db.posts.findByUserId(userId);
});

// Invalidate all cache entries referencing this user
await cache.invalidateAll(`user:123`);

When to Cache

• Database query results
• External API responses
• Computed values that are expensive to calculate
• Configuration that rarely changes

When NOT to Cache

• User-specific sensitive data (without proper isolation)
• Rapidly changing data
• Data that must always be consistent
• Large objects that would exhaust memory

Environment Configuration in Node.js

Loading Environment Files

Use Node.js built-in --env-file flag to load environment variables:

# Load from .env file
node --env-file=.env app.ts

# Load multiple env files (later files override earlier ones)
node --env-file=.env --env-file=.env.local app.ts

Programmatic API

Load environment files programmatically with process.loadEnvFile():

import { loadEnvFile } from 'node:process';

// Load .env from current directory
loadEnvFile();

// Load specific file
loadEnvFile('.env.local');

Environment Variables Validation

Using env-schema with TypeBox

Use env-schema with TypeBox for type-safe environment validation:

import { envSchema } from 'env-schema';
import { Type, Static } from '@sinclair/typebox';

const schema = Type.Object({
  PORT: Type.Number({ default: 3000 }),
  DATABASE_URL: Type.String(),
  API_KEY: Type.String({ minLength: 1 }),
  LOG_LEVEL: Type.Union([
    Type.Literal('debug'),
    Type.Literal('info'),
    Type.Literal('warn'),
    Type.Literal('error'),
  ], { default: 'info' }),
});

type Env = Static<typeof schema>;

export const env = envSchema<Env>({ schema });

Using Zod

Alternatively, use Zod for validation:

import { z } from 'zod';

const EnvSchema = z.object({
  PORT: z.coerce.number().default(3000),
  DATABASE_URL: z.string().url(),
  API_KEY: z.string().min(1),
  LOG_LEVEL: z.enum(['debug', 'info', 'warn', 'error']).default('info'),
});

type Env = z.infer<typeof EnvSchema>;

function loadEnv(): Env {
  const result = EnvSchema.safeParse(process.env);

  if (!result.success) {
    console.error('Invalid environment variables:');
    console.error(result.error.format());
    process.exit(1);
  }

  return result.data;
}

export const env = loadEnv();

Avoid NODE_ENV

NODE_ENV is an antipattern. It conflates multiple concerns into a single variable:

• Environment detection (development vs production vs staging)
• Behavior toggling (verbose logging, debug features)
• Optimization flags (minification, caching)
• Security settings (strict validation, HTTPS)

This leads to problems:

// BAD - NODE_ENV conflates concerns
if (process.env.NODE_ENV === 'development') {
  enableDebugLogging();    // logging concern
  disableRateLimiting();   // security concern
  useMockDatabase();       // infrastructure concern
}

Instead, use explicit environment variables for each concern:

// GOOD - explicit variables for each concern
const config = {
  logging: {
    level: process.env.LOG_LEVEL || 'info',
    pretty: process.env.LOG_PRETTY === 'true',
  },
  security: {
    rateLimitEnabled: process.env.RATE_LIMIT_ENABLED !== 'false',
    httpsOnly: process.env.HTTPS_ONLY === 'true',
  },
  database: {
    url: process.env.DATABASE_URL,
  },
};

This approach:

• Makes configuration explicit and discoverable
• Allows fine-grained control per environment
• Avoids hidden behavior changes
• Makes testing easier (toggle individual features)

Configuration Object Pattern

Create a typed configuration object:

interface Config {
  server: {
    port: number;
    host: string;
  };
  database: {
    url: string;
    poolSize: number;
  };
  auth: {
    jwtSecret: string;
    jwtExpiresIn: string;
  };
  features: {
    enableMetrics: boolean;
    enableTracing: boolean;
  };
}

function createConfig(): Config {
  return {
    server: {
      port: parseInt(process.env.PORT || '3000', 10),
      host: process.env.HOST || '0.0.0.0',
    },
    database: {
      url: requireEnv('DATABASE_URL'),
      poolSize: parseInt(process.env.DB_POOL_SIZE || '10', 10),
    },
    auth: {
      jwtSecret: requireEnv('JWT_SECRET'),
      jwtExpiresIn: process.env.JWT_EXPIRES_IN || '1h',
    },
    features: {
      enableMetrics: process.env.ENABLE_METRICS === 'true',
      enableTracing: process.env.ENABLE_TRACING === 'true',
    },
  };
}

function requireEnv(name: string): string {
  const value = process.env[name];
  if (!value) {
    throw new Error(`Missing required environment variable: ${name}`);
  }
  return value;
}

export const config = createConfig();

.env File Structure

Organize .env files properly:

# .env.example - committed to git, documents all variables
PORT=3000
DATABASE_URL=postgresql://user:pass@localhost:5432/db
API_KEY=your-api-key-here

# .env - local development, NOT committed
PORT=3000
DATABASE_URL=postgresql://dev:dev@localhost:5432/myapp
API_KEY=sk-dev-key-123

# .env.test - test environment
DATABASE_URL=postgresql://test:test@localhost:5432/myapp_test

Secrets in Production

Never commit secrets to version control. Use a secrets management service appropriate for your infrastructure:

Cloud Provider Services:

• AWS Secrets Manager
• Google Cloud Secret Manager
• Azure Key Vault

Infrastructure Tools:

• HashiCorp Vault
• Doppler
• Infisical

Container Orchestration:

• Kubernetes Secrets
• Docker Swarm Secrets

CI/CD Platforms:

• GitHub Actions Secrets
• GitLab CI/CD Variables
• CircleCI Contexts

These services inject secrets as environment variables at runtime, keeping them out of your codebase and version history.

Feature Flags

Implement feature flags via environment:

const features = {
  newDashboard: process.env.FEATURE_NEW_DASHBOARD === 'true',
  betaApi: process.env.FEATURE_BETA_API === 'true',
  darkMode: process.env.FEATURE_DARK_MODE === 'true',
};

export function isFeatureEnabled(feature: keyof typeof features): boolean {
  return features[feature] ?? false;
}

Error Handling in Node.js

Custom Errors with @fastify/create-error

Use @fastify/create-error for creating custom errors with codes:

import createError from '@fastify/create-error';

const NotFoundError = createError('NOT_FOUND', '%s not found', 404);
const ValidationError = createError('VALIDATION_ERROR', '%s', 400);
const DatabaseError = createError('DATABASE_ERROR', 'Database operation failed: %s', 500);

// Usage
throw new NotFoundError('User');
throw new ValidationError('Email is required');

Minimal Error Code Implementation

If you prefer no dependencies, use this minimal pattern:

interface AppErrorOptions {
  code: string;
  statusCode?: number;
  cause?: Error;
}

function createAppError(message: string, options: AppErrorOptions): Error {
  const error = new Error(message, { cause: options.cause });
  (error as any).code = options.code;
  (error as any).statusCode = options.statusCode ?? 500;
  Error.captureStackTrace(error, createAppError);
  return error;
}

// Factory functions for common errors
function notFound(resource: string): Error {
  return createAppError(`${resource} not found`, { code: 'NOT_FOUND', statusCode: 404 });
}

function validationError(message: string): Error {
  return createAppError(message, { code: 'VALIDATION_ERROR', statusCode: 400 });
}

function databaseError(message: string, cause?: Error): Error {
  return createAppError(message, { code: 'DATABASE_ERROR', statusCode: 500, cause });
}

// Usage
throw notFound('User');
throw validationError('Email is required');

Checking Error Codes

Check errors by code, not by class:

function isAppError(error: unknown): error is Error & { code: string; statusCode: number } {
  return error instanceof Error && 'code' in error && 'statusCode' in error;
}

try {
  await fetchUser(id);
} catch (error) {
  if (isAppError(error) && error.code === 'NOT_FOUND') {
    return null;
  }
  throw error;
}

Async Error Handling

Always use try-catch with async/await and propagate errors properly:

async function fetchUser(id: string): Promise<User> {
  try {
    const user = await db.users.findById(id);
    if (!user) {
      throw notFound('User');
    }
    return user;
  } catch (error) {
    if (isAppError(error)) {
      throw error;
    }
    throw databaseError('Failed to fetch user', error as Error);
  }
}

Unhandled Rejections and Exceptions

Do not handle unhandledRejection and uncaughtException manually. Use close-with-grace which handles these automatically and triggers graceful shutdown.

See graceful-shutdown.md for proper shutdown handling.

Fastify Error Handling

Fastify has built-in error handling:

import Fastify from 'fastify';

const app = Fastify({ logger: true });

app.setErrorHandler((error, request, reply) => {
  const statusCode = (error as any).statusCode ?? 500;
  const code = (error as any).code ?? 'INTERNAL_ERROR';

  request.log.error(error);

  reply.status(statusCode).send({
    error: {
      code,
      message: error.message,
    },
  });
});

Never Swallow Errors

Never use empty catch blocks that hide errors:

// BAD - error is swallowed
try {
  await riskyOperation();
} catch (error) {
  // Do nothing
}

// GOOD - handle or re-throw
try {
  await riskyOperation();
} catch (error) {
  logger.error({ err: error }, 'Operation failed');
  throw error;
}

Error Cause Chain

Use the cause option to preserve error chains:

try {
  await externalService.call();
} catch (error) {
  throw new Error('Service call failed', { cause: error });
}

Identifying and Diagnosing Flaky Tests

Flaky tests are tests that pass or fail intermittently without code changes. They erode trust in the test suite and waste debugging time. This guide helps identify root causes and fix them.

Identifying Which Test/File is Timing Out

When tests timeout, use these techniques to identify the culprit:

1. Use --test-reporter for Detailed Output

# Show each test as it runs (tap format shows test file and name)
node --test --test-reporter=tap

# Use spec reporter for hierarchical view
node --test --test-reporter=spec

# Run with verbose output to see which test hangs
node --test --test-reporter=spec 2>&1 | tee test-output.log

2. Run Tests with Timeout Tracking

# Set a global timeout and see which test exceeds it
node --test --test-timeout=5000

# The error message will include the test name and file:
# Error: test timed out after 5000ms
#     at /path/to/test.ts:42:5

3. Run Individual Test Files

# Isolate by running files one at a time
for f in src/**/*.test.ts; do
  echo "Running: $f"
  timeout 30s node --test "$f" || echo "TIMEOUT or FAIL: $f"
done

4. Add Diagnostic Logging to Test Hooks

import { describe, it, before, after, beforeEach, afterEach } from 'node:test';

describe('MyTests', () => {
  before(() => console.log('[BEFORE] MyTests starting'));
  after(() => console.log('[AFTER] MyTests complete'));
  beforeEach((t) => console.log(`[BEFORE EACH] Starting: ${t.name}`));
  afterEach((t) => console.log(`[AFTER EACH] Finished: ${t.name}`));

  it('test 1', () => { /* ... */ });
  it('test 2', () => { /* ... */ });
});

5. Check for Hanging Async Operations

# Use --inspect to debug hanging tests
node --inspect --test src/hanging.test.ts

# Then connect Chrome DevTools to chrome://inspect
# Check the "Async" call stack to see what's pending

6. Use wtfnode to Find Open Handles

import { describe, it, after } from 'node:test';
import wtfnode from 'wtfnode';

describe('Debug hanging tests', () => {
  after(() => {
    // Dump what's keeping Node.js alive
    wtfnode.dump();
  });

  it('might hang', async () => {
    // Your test
  });
});

Common Causes of Flaky Tests

1. Timing and Race Conditions

Symptom: Test passes locally but fails in CI, or fails randomly.

// BAD - Race condition with setTimeout
it('should process after delay', async (t) => {
  let processed = false;
  processAsync(() => { processed = true; });

  await new Promise(resolve => setTimeout(resolve, 100));
  t.assert.equal(processed, true); // May fail if processing takes > 100ms
});

// GOOD - Wait for the actual condition
it('should process after delay', async (t) => {
  const result = await processAsync();
  t.assert.equal(result.processed, true);
});

2. Uncontrolled Time Dependencies

Symptom: Tests fail around midnight, month boundaries, or in different timezones.

// BAD - Depends on current time
it('should format today', (t) => {
  const result = formatDate(new Date());
  t.assert.equal(result, '2024-01-15'); // Fails tomorrow
});

// GOOD - Use fixed dates or mock time
it('should format date', (t) => {
  const fixedDate = new Date('2024-01-15T12:00:00Z');
  const result = formatDate(fixedDate);
  t.assert.equal(result, '2024-01-15');
});

// GOOD - Mock Date with node:test
it('should format today', (t) => {
  t.mock.timers.enable({ apis: ['Date'] });
  t.mock.timers.setTime(new Date('2024-01-15T12:00:00Z').getTime());

  const result = formatDate(new Date());
  t.assert.equal(result, '2024-01-15');
});

3. Port Conflicts

Symptom: "EADDRINUSE" errors, tests fail when run in parallel.

// BAD - Hardcoded port
it('should start server', async (t) => {
  const server = await startServer({ port: 3000 }); // Conflicts with other tests
  // ...
});

// GOOD - Use dynamic port (port 0)
it('should start server', async (t) => {
  const server = await startServer({ port: 0 });
  const address = server.address();
  const port = address.port; // OS assigns available port
  // ...
});

4. Shared State Between Tests

Symptom: Tests pass individually but fail when run together.

// BAD - Module-level state persists between tests
let cache = new Map();

it('test 1', (t) => {
  cache.set('key', 'value1');
  t.assert.equal(cache.get('key'), 'value1');
});

it('test 2', (t) => {
  t.assert.equal(cache.get('key'), undefined); // FAILS - still has 'value1'
});

// GOOD - Reset state in beforeEach or use test-scoped state
describe('cache tests', () => {
  let cache;

  beforeEach(() => {
    cache = new Map();
  });

  it('test 1', (t) => {
    cache.set('key', 'value1');
    t.assert.equal(cache.get('key'), 'value1');
  });

  it('test 2', (t) => {
    t.assert.equal(cache.get('key'), undefined); // PASSES
  });
});

5. Test Order Dependencies

Symptom: Tests pass with --test but fail with --test --parallel.

// BAD - Test 2 depends on side effect from Test 1
it('test 1: create user', async (t) => {
  await db.insert({ id: 1, name: 'John' });
  t.assert.ok(true);
});

it('test 2: find user', async (t) => {
  const user = await db.findById(1); // Fails if test 1 didn't run first
  t.assert.equal(user.name, 'John');
});

// GOOD - Each test sets up its own data
it('test 2: find user', async (t) => {
  await db.insert({ id: 1, name: 'John' }); // Setup within test
  const user = await db.findById(1);
  t.assert.equal(user.name, 'John');
});

6. Unhandled Promise Rejections

Symptom: Test appears to pass but process exits with error, or random failures.

// BAD - Fire-and-forget async operation
it('should send notification', async (t) => {
  sendNotification(user); // Not awaited - may reject after test ends
  t.assert.ok(true);
});

// GOOD - Await all async operations
it('should send notification', async (t) => {
  await sendNotification(user);
  t.assert.ok(true);
});

7. Resource Cleanup Failures

Symptom: Tests fail with "too many open files" or connections exhausted.

// BAD - Resources not cleaned up
it('should read file', async (t) => {
  const handle = await fs.open('test.txt');
  const content = await handle.read();
  t.assert.ok(content);
  // handle never closed!
});

// GOOD - Always clean up resources
it('should read file', async (t) => {
  const handle = await fs.open('test.txt');
  t.after(() => handle.close()); // Cleanup registered

  const content = await handle.read();
  t.assert.ok(content);
});

Debugging Strategies

1. Run Tests in Isolation

# Run single test file
node --test src/user.test.ts

# Run single test by name
node --test --test-name-pattern="should create user" src/user.test.ts

2. Increase Concurrency to Expose Race Conditions

# Run with high concurrency to surface race conditions
node --test --test-concurrency=10

# Or run the same test multiple times
for i in {1..50}; do node --test src/flaky.test.ts || echo "Failed on run $i"; done

3. Use Test Retry to Identify Flaky Tests

// Temporarily add retry to identify flaky test
it('potentially flaky test', { retry: 3 }, async (t) => {
  // If this needs retries to pass, it's flaky
});

4. Add Diagnostic Logging

it('flaky test', async (t) => {
  console.log('Test started at:', Date.now());
  console.log('Environment:', process.env.NODE_ENV);

  const result = await operation();
  console.log('Result:', JSON.stringify(result));

  t.assert.ok(result);
});

5. Check for Async Leaks

import { describe, it, after } from 'node:test';

describe('async leak detection', () => {
  const activeHandles = new Set();

  after(() => {
    if (activeHandles.size > 0) {
      console.error('Leaked handles:', [...activeHandles]);
    }
  });

  it('should not leak', async (t) => {
    const timer = setTimeout(() => {}, 10000);
    activeHandles.add(timer);

    // Do test work...

    clearTimeout(timer);
    activeHandles.delete(timer);
  });
});

Prevention Best Practices

1. Use Deterministic IDs

// BAD - Random IDs make debugging hard
const id = crypto.randomUUID();

// GOOD - Predictable IDs in tests
const id = `test-user-${t.name}`;

2. Mock External Services

it('should fetch user', async (t) => {
  // Mock fetch to avoid network flakiness
  t.mock.method(globalThis, 'fetch', async () => ({
    ok: true,
    json: async () => ({ id: '1', name: 'John' }),
  }));

  const user = await fetchUser('1');
  t.assert.equal(user.name, 'John');
});

3. Use Explicit Waits Instead of Timeouts

// BAD - Arbitrary timeout
await new Promise(r => setTimeout(r, 1000));

// GOOD - Wait for specific condition
await waitFor(() => element.isVisible());

// Helper function
async function waitFor(condition, timeout = 5000) {
  const start = Date.now();
  while (Date.now() - start < timeout) {
    if (await condition()) return;
    await new Promise(r => setTimeout(r, 50));
  }
  throw new Error('Condition not met within timeout');
}

4. Ensure Test Isolation with Transactions

describe('database tests', () => {
  beforeEach(async () => {
    await db.query('BEGIN');
  });

  afterEach(async () => {
    await db.query('ROLLBACK');
  });

  it('should insert record', async (t) => {
    await db.insert({ name: 'test' });
    const records = await db.findAll();
    t.assert.equal(records.length, 1);
  });
});

CI-Specific Flakiness

1. Resource Constraints

CI environments often have less CPU/memory. Add appropriate timeouts:

it('heavy computation', { timeout: 30000 }, async (t) => {
  // Longer timeout for CI
  const result = await heavyOperation();
  t.assert.ok(result);
});

2. Parallel Test Execution

Ensure tests don't conflict when run in parallel:

# In CI, run with controlled concurrency
node --test --test-concurrency=2

3. Network Reliability

Mock external APIs in tests to avoid network-related flakiness:

// Always mock external HTTP calls in unit tests
t.mock.method(globalThis, 'fetch', async (url) => {
  if (url.includes('api.external.com')) {
    return { ok: true, json: async () => mockData };
  }
  throw new Error(`Unmocked URL: ${url}`);
});

Graceful Shutdown in Node.js

Use close-with-grace

Always use close-with-grace for handling graceful shutdowns:

import closeWithGrace from 'close-with-grace';

closeWithGrace({ delay: 10000 }, async ({ signal, err }) => {
  if (err) {
    console.error('Error triggered shutdown:', err);
  }
  console.log(`Received ${signal}, shutting down...`);

  await server.close();
  await db.end();
});

HTTP Server Shutdown

Close HTTP servers gracefully with close-with-grace:

import { createServer } from 'node:http';
import closeWithGrace from 'close-with-grace';

const server = createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'application/json' });
  res.end(JSON.stringify({ status: 'ok' }));
});

server.listen(3000, () => {
  console.log('Server listening on port 3000');
});

closeWithGrace({ delay: 10000 }, async ({ signal, err }) => {
  if (err) {
    console.error('Shutdown error:', err);
  }
  console.log(`${signal} received, closing server...`);

  await new Promise<void>((resolve, reject) => {
    server.close((err) => (err ? reject(err) : resolve()));
  });

  console.log('Server closed');
});

Fastify Integration

Fastify has built-in close-with-grace support:

import Fastify from 'fastify';

const app = Fastify({
  logger: true,
});

// Fastify automatically handles graceful shutdown
// Just register your cleanup in onClose hooks

app.addHook('onClose', async () => {
  await db.end();
  await cache.quit();
});

await app.listen({ port: 3000 });

Multiple Resources Cleanup

Clean up multiple resources in order:

import closeWithGrace from 'close-with-grace';
import { createServer } from 'node:http';

const server = createServer(handler);
const db = await connectDatabase();
const redis = await connectRedis();

server.listen(3000);

closeWithGrace({ delay: 15000 }, async ({ signal, err }) => {
  if (err) {
    console.error('Error:', err);
  }
  console.log(`${signal} received`);

  // Close in reverse order of initialization
  await new Promise<void>((resolve) => server.close(() => resolve()));
  console.log('HTTP server closed');

  await redis.quit();
  console.log('Redis connection closed');

  await db.end();
  console.log('Database connection closed');
});

Health Checks

Implement health checks that respect shutdown state:

import closeWithGrace from 'close-with-grace';

let isShuttingDown = false;

function healthHandler(req: Request, res: Response) {
  if (isShuttingDown) {
    return res.status(503).json({ status: 'shutting_down' });
  }
  return res.json({ status: 'healthy' });
}

function readinessHandler(req: Request, res: Response) {
  if (isShuttingDown) {
    return res.status(503).json({ ready: false });
  }
  return res.json({ ready: true });
}

closeWithGrace({ delay: 10000 }, async ({ signal }) => {
  isShuttingDown = true;
  console.log(`${signal} received, marked as shutting down`);

  // Wait for load balancer to stop sending traffic
  await new Promise((r) => setTimeout(r, 5000));

  await cleanup();
});

Custom Delay for Kubernetes

Use appropriate delays for container orchestration:

import closeWithGrace from 'close-with-grace';

// Kubernetes sends SIGTERM, then waits terminationGracePeriodSeconds (default 30s)
// Set delay slightly lower to ensure clean exit
closeWithGrace({ delay: 25000 }, async ({ signal }) => {
  console.log(`${signal} received`);

  // Mark as not ready immediately
  isShuttingDown = true;

  // Wait for in-flight requests (k8s stops sending new traffic after SIGTERM)
  await new Promise((r) => setTimeout(r, 5000));

  // Close resources
  await server.close();
  await db.end();
});

Manual Signal Handling (when close-with-grace is not available)

If you cannot use close-with-grace, handle signals manually:

const signals: NodeJS.Signals[] = ['SIGTERM', 'SIGINT'];
let isShuttingDown = false;

async function shutdown(signal: string): Promise<void> {
  if (isShuttingDown) return;
  isShuttingDown = true;

  console.log(`${signal} received, shutting down...`);

  const timeout = setTimeout(() => {
    console.error('Shutdown timeout, forcing exit');
    process.exit(1);
  }, 10000);

  try {
    await cleanup();
    clearTimeout(timeout);
    process.exit(0);
  } catch (error) {
    console.error('Shutdown error:', error);
    clearTimeout(timeout);
    process.exit(1);
  }
}

for (const signal of signals) {
  process.on(signal, () => shutdown(signal));
}

Logging in Node.js

Use Pino

Use pino for fast, structured JSON logging:

import pino from 'pino';

const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
});

logger.info({ userId: user.id }, 'User created');
logger.error({ err, orderId: order.id }, 'Failed to process payment');

Log Levels

Use appropriate log levels:

// DEBUG - detailed information for debugging
logger.debug({ itemId: item.id, step: 'validation' }, 'Processing item');

// INFO - general operational information
logger.info({ userId: user.id }, 'User created');

// WARN - unexpected but handled situations
logger.warn({ currentRate: 95, limit: 100 }, 'Rate limit approaching');

// ERROR - errors that need attention
logger.error({ err, orderId: order.id }, 'Failed to process payment');

Transports

Pino uses transports to process logs outside the main thread. Transports handle formatting, filtering, and sending logs to external services.

Pretty Printing for Development

Use pino-pretty for human-readable output during development:

node app.ts | pino-pretty

Or configure programmatically:

import pino from 'pino';

const logger = pino({
  transport: {
    target: 'pino-pretty',
    options: {
      colorize: true,
    },
  },
});

Multiple Transports

Send logs to multiple destinations:

import pino from 'pino';

const logger = pino({
  transport: {
    targets: [
      {
        target: 'pino-pretty',
        options: { colorize: true },
        level: 'info',
      },
      {
        target: 'pino/file',
        options: { destination: '/var/log/app.log' },
        level: 'error',
      },
    ],
  },
});

Available Transports

• pino-pretty - Human-readable formatting
• pino-elasticsearch - Send to Elasticsearch
• pino-loki - Send to Grafana Loki
• pino-datadog - Send to Datadog

Child Loggers

Create child loggers with bound context:

const requestLogger = logger.child({
  requestId: req.id,
  userId: req.user?.id,
});

requestLogger.info('Processing request');
requestLogger.info({ itemId }, 'Item processed');

Fastify Integration

Fastify has built-in pino integration:

import Fastify from 'fastify';

const app = Fastify({
  logger: {
    level: 'info',
    transport: {
      target: 'pino-pretty',
    },
  },
});

app.get('/', async (request) => {
  request.log.info('Handling request');
  return { status: 'ok' };
});

Redaction

Use pino's built-in redaction for sensitive fields:

const logger = pino({
  redact: ['password', 'token', 'apiKey', 'req.headers.authorization'],
});

// Sensitive values are replaced with [Redacted]
logger.info({ password: 'secret123' }, 'User login');
// Output: {"password":"[Redacted]","msg":"User login"...}

Debug Module

The debug module is useful for library and module authors to emit tracing information. It is not meant for application logging:

import createDebug from 'debug';

const debug = createDebug('mymodule:connection');

debug('Connecting to %s:%d', host, port);
debug('Query executed in %dms', duration);

Enable debug output with the DEBUG environment variable:

DEBUG=mymodule:* node app.ts

Use debug for internal module diagnostics; use pino for application logs.

Node.js Built-in Debugging

Use util.debuglog for module tracing without external dependencies:

import { debuglog } from 'node:util';

const debug = debuglog('mymodule');

debug('Starting operation %s', operationId);
debug('Connection established to %s', host);

Enable with the NODE_DEBUG environment variable:

NODE_DEBUG=mymodule node app.ts
NODE_DEBUG=mymodule,http,net node app.ts

This also works with Node.js internals (NODE_DEBUG=http,net,tls) for debugging core module behavior.

Avoid Logging Sensitive Data

Never log credentials, tokens, or personal data:

// BAD - logging sensitive data
logger.info({ email, password }, 'User login');

// GOOD - log only safe identifiers
logger.info({ email }, 'User login');

Node.js Modules

Prefer ES Modules

Use ES Modules (ESM) for new projects:

// package.json
{
  "type": "module"
}

// Named exports (preferred)
export function processData(data: Data): Result {
  // ...
}

export const CONFIG = {
  timeout: 5000,
};

// Named imports
import { processData, CONFIG } from './utils.js';

File Extensions in ESM

Always include file extensions in ESM imports:

// GOOD - explicit extension
import { helper } from './helper.js';
import config from './config.json' with { type: 'json' };

// BAD - missing extension (works in bundlers but not native ESM)
import { helper } from './helper';

Barrel Exports

Use index files to simplify imports:

// src/utils/index.ts
export { formatDate, parseDate } from './date.js';
export { formatCurrency } from './currency.js';
export { validateEmail } from './validation.js';

// Consumer
import { formatDate, formatCurrency } from './utils/index.js';

Default vs Named Exports

Prefer named exports for better refactoring and tree-shaking:

// GOOD - named exports
export function createServer(config: Config): Server {
  // ...
}

export function createClient(config: Config): Client {
  // ...
}

// AVOID - default exports
export default function createServer(config: Config): Server {
  // ...
}

Dynamic Imports

Use dynamic imports for code splitting and conditional loading:

async function loadPlugin(name: string): Promise<Plugin> {
  const module = await import(`./plugins/${name}.js`);
  return module.default;
}

// Conditional loading
const { default: heavy } = await import('./heavy-module.js');

__dirname and __filename in ESM

Use import.meta.dirname and import.meta.filename (Node.js 20.11+):

import { join } from 'node:path';

const configPath = join(import.meta.dirname, 'config.json');
const currentFile = import.meta.filename;

Exploring node_modules

When to Explore node_modules

Explore node_modules when you need to:

• Find specific packages and their versions
• Analyze dependencies and dependency trees
• Examine package contents
• Investigate dependency conflicts
• Understand how a package works internally

Core Techniques

Finding Package Versions

# Check actual installed version
cat node_modules/fastify/package.json | grep '"version"'

# For scoped packages
cat node_modules/@fastify/cors/package.json | grep '"version"'

# List all versions with npm
npm ls fastify

Navigating Directory Structure

# List package contents
ls node_modules/fastify/

# Find TypeScript definitions
ls node_modules/fastify/*.d.ts
ls node_modules/@types/node/

# Check main entry point
cat node_modules/fastify/package.json | grep '"main"\|"exports"'

Understanding Package Manager Differences

npm/yarn (node_modules hoisting):

node_modules/
  fastify/
  pino/          # hoisted from fastify's dependencies
  @fastify/cors/

pnpm (content-addressable storage):

node_modules/
  .pnpm/
    fastify@4.0.0/
      node_modules/
        fastify/
        pino/    # symlinked, not hoisted
  fastify -> .pnpm/fastify@4.0.0/node_modules/fastify

Finding Package READMEs

CRITICAL: Never use `find`, `grep`, or `rg` for locating READMEs. Follow this sequence:

1. Direct Read attempts (try in order):

   node_modules/[package-name]/README.md
   node_modules/[package-name]/readme.md
   node_modules/[package-name]/README

For scoped packages: node_modules/@scope/package-name/README.md

2. If not found, list directory contents:

   ls node_modules/[package-name]/

Look for README files in output, then read the exact filename.

3. Alternative locations:

   node_modules/[package-name]/docs/README.md

Or check readme field in package.json.

Analyzing Dependency Trees

# See why a package is installed
npm why lodash

# Full dependency tree
npm ls --all

# Only production dependencies
npm ls --prod

# Find duplicates
npm ls --all 2>&1 | grep -E "^\s+.*@[0-9]" | sort | uniq -d

Investigating Conflicts

Peer Dependency Issues

# Check peer dependencies
cat node_modules/some-plugin/package.json | grep -A 10 '"peerDependencies"'

# See what's actually installed vs. what's expected
npm ls react

Duplicate Packages

When the same package appears multiple times:

# Find all instances of a package
find node_modules -name "package.json" -path "*/lodash/*" 2>/dev/null

# Check for version mismatches
npm ls lodash

Examining Package Internals

Entry Points

# Check exports field (modern)
node -e "console.log(JSON.stringify(require('./node_modules/fastify/package.json').exports, null, 2))"

# Check main field (legacy)
cat node_modules/fastify/package.json | grep '"main"'

TypeScript Definitions

# Find type definitions
ls node_modules/fastify/*.d.ts
cat node_modules/fastify/package.json | grep '"types"\|"typings"'

# For DefinitelyTyped packages
ls node_modules/@types/

Source Files

# Examine source structure
ls node_modules/fastify/lib/
head -50 node_modules/fastify/lib/server.js

Debugging Module Resolution

# See how Node.js resolves a module
node -e "console.log(require.resolve('fastify'))"

# With full resolution paths
node --print "require.resolve.paths('fastify')"

Performance in Node.js

Avoid Blocking the Event Loop

Never perform CPU-intensive operations synchronously:

// BAD - blocks event loop
function hashPasswordSync(password: string): string {
  return crypto.pbkdf2Sync(password, salt, 100000, 64, 'sha512').toString('hex');
}

// GOOD - async operation
function hashPassword(password: string): Promise<string> {
  return new Promise((resolve, reject) => {
    crypto.pbkdf2(password, salt, 100000, 64, 'sha512', (err, key) => {
      if (err) reject(err);
      else resolve(key.toString('hex'));
    });
  });
}

Worker Threads with Piscina

Use piscina for CPU-intensive tasks:

// worker.ts
export default function heavyComputation(data: { input: string }): string {
  // CPU-intensive work here
  return result;
}

// main.ts
import Piscina from 'piscina';

const piscina = new Piscina({
  filename: new URL('./worker.ts', import.meta.url).href,
});

const result = await piscina.run({ input: 'data' });

Piscina handles worker pool management, task queuing, and load balancing automatically.

Connection Pooling

Always use connection pools for databases:

import { Pool } from 'pg';

const pool = new Pool({
  max: 20,
  idleTimeoutMillis: 30000,
  connectionTimeoutMillis: 2000,
});

async function query<T>(sql: string, params: unknown[]): Promise<T[]> {
  const client = await pool.connect();
  try {
    const result = await client.query(sql, params);
    return result.rows;
  } finally {
    client.release();
  }
}

Avoid Memory Leaks

Common memory leak patterns to avoid:

// BAD - unbounded cache
const cache = new Map();
function addToCache(key: string, value: unknown) {
  cache.set(key, value); // Never cleaned up
}

// GOOD - LRU cache with max size
import { LRUCache } from 'lru-cache';

const cache = new LRUCache<string, unknown>({
  max: 500,
  ttl: 1000 * 60 * 5,
});

// BAD - listener leak
function subscribe(emitter: EventEmitter) {
  emitter.on('event', handler); // Never removed
}

// GOOD - cleanup listeners
function subscribe(emitter: EventEmitter): () => void {
  emitter.on('event', handler);
  return () => emitter.off('event', handler);
}

Lazy Loading

Load modules only when needed:

let heavyModule: HeavyModule | null = null;

async function getHeavyModule(): Promise<HeavyModule> {
  if (!heavyModule) {
    const { HeavyModule } = await import('./heavy-module.js');
    heavyModule = new HeavyModule();
  }
  return heavyModule;
}

Related

• caching.md - Caching patterns and libraries
• profiling.md - Profiling and benchmarking tools

Profiling in Node.js

Flame Graphs with @platformatic/flame

Use @platformatic/flame for CPU profiling with flame graph visualization:

npx @platformatic/flame app.ts

This starts your application with profiling enabled and generates an interactive flame graph.

Markdown Output for AI Analysis

flame can output markdown reports suitable for AI-assisted performance analysis:

npx @platformatic/flame --output markdown app.ts

This enables a fully agentic workflow where you can: 1. Profile your application 2. Get markdown output describing hotspots 3. Feed the report to an AI assistant for optimization suggestions

Programmatic Usage

import { profile } from '@platformatic/flame';

const stop = await profile({
  outputFile: 'profile.html',
});

// Run your workload
await runBenchmark();

await stop();

Load Testing Tools

autocannon

Use autocannon for HTTP benchmarking:

# Basic benchmark
npx autocannon http://localhost:3000

# With options
npx autocannon -c 100 -d 30 -p 10 http://localhost:3000

# POST request with body
npx autocannon -m POST -H "Content-Type: application/json" -b '{"name":"test"}' http://localhost:3000/users

Options:

• -c - Number of concurrent connections (default: 10)
• -d - Duration in seconds (default: 10)
• -p - Number of pipelined requests (default: 1)
• -m - HTTP method
• -b - Request body

Programmatic autocannon

import autocannon from 'autocannon';

const result = await autocannon({
  url: 'http://localhost:3000',
  connections: 100,
  duration: 30,
  pipelining: 10,
});

console.log(autocannon.printResult(result));

wrk

wrk is a high-performance HTTP benchmarking tool:

# Basic benchmark
wrk -t12 -c400 -d30s http://localhost:3000

# With Lua script for custom requests
wrk -t12 -c400 -d30s -s post.lua http://localhost:3000

Options:

• -t - Number of threads
• -c - Number of connections
• -d - Duration
• -s - Lua script for custom logic

k6

k6 is ideal for complex load testing scenarios:

// load-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  vus: 100,
  duration: '30s',
};

export default function () {
  const res = http.get('http://localhost:3000');
  check(res, {
    'status is 200': (r) => r.status === 200,
    'response time < 200ms': (r) => r.timings.duration < 200,
  });
  sleep(1);
}

k6 run load-test.js

Built-in Node.js Profiling

CPU Profiling

# Generate V8 profiling log
node --prof app.js

# Process the log
node --prof-process isolate-*.log > profile.txt

Heap Snapshots

# Start with inspector
node --inspect app.js

# Then use Chrome DevTools (chrome://inspect) to:
# - Take heap snapshots
# - Record allocation timelines
# - Find memory leaks

Diagnostic Reports

# Generate report on signal
node --report-on-signal app.js
kill -SIGUSR2 <pid>

# Generate report on uncaught exception
node --report-on-fatalerror app.js

Profiling Workflow

1. Establish baseline - Run autocannon to get initial metrics 2. Profile - Use @platformatic/flame to identify hotspots 3. Optimize - Fix the identified bottlenecks 4. Verify - Run autocannon again to measure improvement 5. Repeat - Continue until performance goals are met

Tool Comparison

Node.js Streams

High-signal triggers

If the prompt mentions CSV, ETL, ingestion, large files, transform streams, backpressure, or line-by-line processing, prioritize pipeline() + explicit async-generator transforms.

Use pipeline for Stream Composition

Always use pipeline instead of .pipe() for proper error handling:

import { pipeline } from 'node:stream/promises';
import { createReadStream, createWriteStream } from 'node:fs';
import { createGzip } from 'node:zlib';

async function compressFile(input: string, output: string): Promise<void> {
  await pipeline(
    createReadStream(input),
    createGzip(),
    createWriteStream(output)
  );
}

Async Generators in Pipeline

Use async generators for transformation:

import { pipeline } from 'node:stream/promises';
import { createReadStream, createWriteStream } from 'node:fs';

async function* toUpperCase(source: AsyncIterable<Buffer>): AsyncGenerator<string> {
  for await (const chunk of source) {
    yield chunk.toString().toUpperCase();
  }
}

async function processFile(input: string, output: string): Promise<void> {
  await pipeline(
    createReadStream(input),
    toUpperCase,
    createWriteStream(output)
  );
}

CSV/ETL pattern: pipeline + async transform + deduplicated enrichment

For ingestion-style tasks, show an explicit async function* transform and integrate deduped async lookups:

import { pipeline } from 'node:stream/promises';
import { createReadStream, createWriteStream } from 'node:fs';
import { createCache } from 'async-cache-dedupe';

const cache = createCache({
  ttl: 60,
  stale: 5,
  storage: { type: 'memory' },
});

cache.define('lookupPlan', async (planId: string) => {
  return await fetch(`https://billing.internal/plans/${planId}`).then(async (res) => await res.json());
});

async function* enrichCsvRows(source: AsyncIterable<Buffer>): AsyncGenerator<string> {
  let tail = '';

  for await (const chunk of source) {
    tail += chunk.toString('utf8');
    const lines = tail.split('\n');
    tail = lines.pop() ?? '';

    for (const line of lines) {
      if (line.trim().length === 0) continue;
      const [userId, planId] = line.split(',');
      const plan = await cache.lookupPlan(planId); // concurrent requests dedupe by key
      yield `${userId},${planId},${plan.tier}\n`;
    }
  }

  if (tail.trim().length > 0) {
    const [userId, planId] = tail.split(',');
    const plan = await cache.lookupPlan(planId);
    yield `${userId},${planId},${plan.tier}\n`;
  }
}

await pipeline(
  createReadStream('users.csv'),
  enrichCsvRows,
  createWriteStream('users-enriched.csv')
);

Multiple Transformations

Chain multiple async generators:

import { pipeline } from 'node:stream/promises';

async function* parseLines(source: AsyncIterable<Buffer>): AsyncGenerator<string> {
  let buffer = '';
  for await (const chunk of source) {
    buffer += chunk.toString();
    const lines = buffer.split('\n');
    buffer = lines.pop() ?? '';
    for (const line of lines) {
      yield line;
    }
  }
  if (buffer) yield buffer;
}

async function* filterNonEmpty(source: AsyncIterable<string>): AsyncGenerator<string> {
  for await (const line of source) {
    if (line.trim()) {
      yield line + '\n';
    }
  }
}

await pipeline(
  createReadStream('input.txt'),
  parseLines,
  filterNonEmpty,
  createWriteStream('output.txt')
);

Async Iterators with Streams

Use async iterators for consuming streams:

import { createReadStream } from 'node:fs';
import { createInterface } from 'node:readline';

async function processLines(filePath: string): Promise<void> {
  const fileStream = createReadStream(filePath);
  const rl = createInterface({
    input: fileStream,
    crlfDelay: Infinity,
  });

  for await (const line of rl) {
    await processLine(line);
  }
}

Readable.from for Creating Streams

Create readable streams from iterables:

import { Readable } from 'node:stream';

async function* generateData(): AsyncGenerator<string> {
  for (let i = 0; i < 100; i++) {
    yield JSON.stringify({ id: i, timestamp: Date.now() }) + '\n';
  }
}

const stream = Readable.from(generateData());

Backpressure Handling

Respect backpressure signals using once from events:

import { Writable } from 'node:stream';
import { once } from 'node:events';

async function writeData(
  writable: Writable,
  data: string[]
): Promise<void> {
  for (const chunk of data) {
    const canContinue = writable.write(chunk);
    if (!canContinue) {
      await once(writable, 'drain');
    }
  }
}

Stream Consumers (Node.js 18+)

Use stream consumers for common operations:

import { text, json, buffer } from 'node:stream/consumers';
import { Readable } from 'node:stream';

async function readStreamAsJson<T>(stream: Readable): Promise<T> {
  return json(stream) as Promise<T>;
}

async function readStreamAsText(stream: Readable): Promise<string> {
  return text(stream);
}

Diagnosing Stuck Node.js Processes and Hanging Tests

Activation triggers (apply this rule immediately)

Use this rule when prompts/logs include any of:

• "node --test hangs" / "test run never exits"
• "CI timed out" after tests appear done
• "process is still running" / "did not exit cleanly"
• "open handles" / "active handles"
• "passes alone, hangs in full suite"

Non-negotiable checklist (all 5 required)

1. Isolate first: reproduce with one file, then one test name. 2. Fail fast: run with explicit timeout + reporter so hangs produce location context. 3. Capture handles: run why-is-node-running and record the reported handle owner. 4. Patch teardown in the same scope that created the resource (t.after(...), await close/terminate/disconnect). 5. Verify stability: repeat isolated test many times, then run full suite.

Do not mark the issue fixed unless step 5 passes.

Command path (run in order)

# 1) Reproduce full command with fail-fast settings
node --test --test-reporter=spec --test-timeout=15000

# 2) Isolate file, then test name
node --test --test-timeout=15000 path/to/file.test.ts
node --test --test-timeout=15000 --test-name-pattern="should close resources" path/to/file.test.ts

# 3) Dump active handles when hung
node --import why-is-node-running/include --test path/to/file.test.ts
# in another shell, send SIGUSR1 to printed PID
kill -SIGUSR1 <pid>

# 4) Stress rerun isolated repro (Linux: timeout, macOS: gtimeout)
TIMEOUT_BIN="$(command -v timeout || command -v gtimeout)"
for i in {1..30}; do
  "$TIMEOUT_BIN" 30s node --test path/to/file.test.ts || { echo "failed on run $i"; break; }
done

why-is-node-running install note

• ESM / modern Node: npm i -D why-is-node-running
• older CommonJS projects: npm i -D why-is-node-running@v2

Use it for diagnostics only; gate noisy output behind an environment flag in CI.

High-probability root causes

• HTTP server started but never closed (server.close() not awaited)
• setInterval left running (or timer not unref()ed when appropriate)
• DB/Redis/Kafka clients not disconnected
• worker threads, child processes, or message channels left alive
• file watchers / readline interfaces still open
• unresolved promises from fire-and-forget async code
• teardown hooks that throw before cleanup completes

Integrated fix example (CI hangs after tests appear complete)

Symptom: CI times out; local full-suite run hangs after most tests pass.

Step sequence

1. Reproduce with:

   node --test --test-reporter=spec --test-timeout=15000

2. Isolate to one test using --test-name-pattern. 3. Run with --import why-is-node-running/include and send SIGUSR1. 4. Patch teardown where resource is created. 5. Verify isolated 30x, then full suite.

Bad (common leak)

it('serves requests', async () => {
  const server = await startServer({ port: 0 });
  const id = setInterval(() => {}, 1000);
  // test body...
  // ❌ no deterministic teardown
});

Good (deterministic teardown)

import { once } from 'node:events';

it('serves requests', async (t) => {
  const server = await startServer({ port: 0 });
  const id = setInterval(() => {}, 1000);

  t.after(async () => {
    clearInterval(id);
    server.close();
    await once(server, 'close');
  });

  // test body...
});

Definition of done (must all pass)

• [ ] Isolated repro passes repeatedly (recommended: 30 runs) with no hangs
• [ ] Full node --test run exits with code 0
• [ ] CI completes with no timeout
• [ ] why-is-node-running shows no unexpected leftover handles

Related rules

• flaky-tests.md
• testing.md
• graceful-shutdown.md

Testing in Node.js

Node.js Built-in Test Runner

Use the built-in test runner (Node.js 22+):

import { describe, it, before, after } from 'node:test';

describe('UserService', () => {
  let service: UserService;

  before(() => {
    service = new UserService();
  });

  it('should create a user', async (t) => {
    const user = await service.create({ name: 'John' });
    t.assert.equal(user.name, 'John');
    t.assert.ok(user.id);
  });

  it('should throw on invalid input', async (t) => {
    await t.assert.rejects(
      () => service.create({ name: '' }),
      { message: 'Name is required' }
    );
  });
});

Mocking with Test Context

Use the test context t.mock for mocking:

import { describe, it } from 'node:test';

describe('EmailService', () => {
  it('should send email via provider', async (t) => {
    const sendMock = t.mock.fn(async () => ({ success: true }));
    const provider = { send: sendMock };
    const service = new EmailService(provider);

    await service.sendWelcome('user@example.com');

    t.assert.equal(sendMock.mock.calls.length, 1);
    t.assert.deepEqual(sendMock.mock.calls[0].arguments, [
      'user@example.com',
      'Welcome!',
    ]);
  });
});

Mocking Methods

import { describe, it } from 'node:test';

describe('UserController', () => {
  it('should fetch user from API', async (t) => {
    t.mock.method(globalThis, 'fetch', async () => ({
      ok: true,
      json: async () => ({ id: '1', name: 'John' }),
    }));

    const user = await fetchUser('1');
    t.assert.equal(user.name, 'John');
  });
});

Test Organization

Structure tests alongside source files:

src/
  user/
    user.service.ts
    user.service.test.ts
    user.repository.ts
    user.repository.test.ts

Snapshot Testing

Use snapshots for complex outputs:

import { describe, it } from 'node:test';

describe('ReportGenerator', () => {
  it('should generate expected report', async (t) => {
    const report = await generateReport(sampleData);
    t.assert.snapshot(report);
  });
});

Test Hooks for Setup/Teardown

Use lifecycle hooks properly:

import { describe, it, before, after, beforeEach, afterEach } from 'node:test';

describe('Database tests', () => {
  let db: Database;

  before(async () => {
    db = await Database.connect(testConfig);
  });

  after(async () => {
    await db.disconnect();
  });

  beforeEach(async () => {
    await db.beginTransaction();
  });

  afterEach(async () => {
    await db.rollback();
  });

  it('should insert record', async (t) => {
    await db.insert({ name: 'test' });
    const records = await db.findAll();
    t.assert.equal(records.length, 1);
  });
});

Isolation and Independence

Tests must be independent and not share state:

// BAD - shared mutable state
let counter = 0;

it('test 1', (t) => {
  counter++;
  t.assert.equal(counter, 1);
});

it('test 2', (t) => {
  counter++;
  t.assert.equal(counter, 2); // Depends on test 1
});

// GOOD - isolated state
it('test 1', (t) => {
  let counter = 0;
  counter++;
  t.assert.equal(counter, 1);
});

it('test 2', (t) => {
  let counter = 0;
  counter++;
  t.assert.equal(counter, 1);
});

EventEmitter Timing in Tests

When testing EventEmitter behavior, always register listeners before triggering the action that emits events.

If you call emit() before on(), once(), or events.once(...) is attached, the event is lost and the test may hang or fail intermittently.

import { EventEmitter, once } from 'node:events';

it('waits for ready event', async (t) => {
  const emitter = new EventEmitter();

  // GOOD: subscribe first
  const readyPromise = once(emitter, 'ready');

  startWorkThatEmitsReady(emitter);

  const [payload] = await readyPromise;
  t.assert.equal(payload.status, 'ok');
});

This ordering is critical for reliable tests and avoiding flaky race conditions.

Running Tests

Run tests with the built-in runner:

# Run all tests
node --test

# Run specific file
node --test src/user/user.service.test.ts

# With TypeScript (Node.js 22.6+)
node --test src/**/*.test.ts

# With coverage
node --test --experimental-test-coverage

# Watch mode
node --test --watch

TypeScript in Node.js

Use Type Stripping

Node.js 22.6+ supports running TypeScript files directly by stripping types at runtime. In Node.js 23.6+ and 24+, type stripping is enabled by default.

node.js 20.x and 22.x

Enable with the experimental flag:

node --experimental-strip-types app.ts

Node.js 22.19+, 23.6+ and 24+

TypeScript files run directly without flags:

node app.ts

Type Stripping Requirements

Type stripping works by removing type annotations without transforming code. Your TypeScript must follow these rules:

Use Type-Only Imports

Always use type keyword for type imports:

// GOOD - type-only import
import type { User, Config } from './types.ts';
import { createUser } from './user.ts';

// GOOD - inline type imports
import { createUser, type User } from './user.ts';

// BAD - may fail with type stripping
import { User, createUser } from './user.ts';

No Enums

Enums require code transformation. Use const objects instead:

// BAD - enums don't work with type stripping
enum Status {
  Active = 'active',
  Inactive = 'inactive',
}

// GOOD - const object with type
const Status = {
  Active: 'active',
  Inactive: 'inactive',
} as const;

type Status = (typeof Status)[keyof typeof Status];

No Namespaces

Namespaces require transformation:

// BAD - namespaces don't work
namespace Utils {
  export function format(s: string): string {
    return s.trim();
  }
}

// GOOD - use modules
export function format(s: string): string {
  return s.trim();
}

No Constructor Parameter Properties

Parameter properties require transformation:

// BAD - parameter properties don't work
class User {
  constructor(public name: string, private age: number) {}
}

// GOOD - explicit property declaration
class User {
  name: string;
  private age: number;

  constructor(name: string, age: number) {
    this.name = name;
    this.age = age;
  }
}

No Legacy Decorators

Use the TC39 decorator syntax (stage 3):

// BAD - legacy/experimental decorators
@Injectable()
class Service {}

// Decorators support depends on Node.js version
// Check compatibility before using

File Extensions

Use .ts extensions in imports. TypeScript will rewrite them to .js during build:

// GOOD - .ts extension (works with type stripping, rewritten during build)
import { helper } from './helper.ts';
import type { Config } from './types.ts';

// JSON imports
import config from './config.json' with { type: 'json' };

tsconfig.json for Development

Configure TypeScript for development with type stripping:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "noEmit": true,
    "resolveJsonModule": true,
    "isolatedModules": true,
    "verbatimModuleSyntax": true,
    "allowImportingTsExtensions": true,
    "lib": ["ES2022"],
    "types": ["node"]
  },
  "include": ["src/**/*.ts", "test/**/*.ts"],
  "exclude": ["node_modules"]
}

Key options:

• noEmit: No compilation, Node.js runs TypeScript directly
• allowImportingTsExtensions: Allow .ts imports
• verbatimModuleSyntax: Enforces type-only imports
• isolatedModules: Ensures compatibility with type stripping

tsconfig.build.json for Publishing

Create a separate config for building distributable packages:

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "declaration": true,
    "declarationMap": true,
    "sourceMap": true,
    "outDir": "dist",
    "rootDir": "src",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "verbatimModuleSyntax": true,
    "allowImportingTsExtensions": true,
    "rewriteRelativeImportExtensions": true,
    "lib": ["ES2022"],
    "types": ["node"]
  },
  "include": ["src/**/*.ts"],
  "exclude": ["node_modules", "test"]
}

Key build options:

• rewriteRelativeImportExtensions: Rewrites .ts imports to .js in output
• declaration: Generates .d.ts type declaration files
• declarationMap: Generates source maps for declarations
• outDir: Output directory for compiled files

Running Tests

Use the built-in test runner with TypeScript:

# Node.js 22.19+/23.6+/24+
node --test test/*.test.ts

Building for Distribution

Use tsc with the build config for publishing packages:

# Build with tsc
tsc -p tsconfig.build.json

package.json Configuration

Configure package.json for ESM with TypeScript:

{
  "type": "module",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "exports": {
    ".": {
      "types": "./dist/index.d.ts",
      "import": "./dist/index.js"
    }
  },
  "files": ["dist", "README.md", "LICENSE"],
  "scripts": {
    "build": "tsc -p tsconfig.build.json",
    "clean": "rm -rf dist",
    "prepublishOnly": "npm run clean && npm run build",
    "test": "node --test test/*.test.ts",
    "typecheck": "tsc --noEmit"
  },
  "engines": {
    "node": ">=22.6.0"
  }
}

Development Workflow

Run type checking separately since type stripping doesn't validate types:

# Check types without emitting
tsc --noEmit

# Watch mode for development
tsc --noEmit --watch