Fullstack Dev

Full-Stack Development Practices

MANDATORY WORKFLOW — Follow These Steps In Order

When this skill is triggered, you MUST follow this workflow before writing any code.

Step 0: Gather Requirements

Before scaffolding anything, ask the user to clarify (or infer from context):

1. Stack: Language/framework for backend and frontend (e.g., Express + React, Django + Vue, Go + HTMX) 2. Service type: API-only, full-stack monolith, or microservice? 3. Database: SQL (PostgreSQL, SQLite, MySQL) or NoSQL (MongoDB, Redis)? 4. Integration: REST, GraphQL, tRPC, or gRPC? 5. Real-time: Needed? If yes — SSE, WebSocket, or polling? 6. Auth: Needed? If yes — JWT, session, OAuth, or third-party (Clerk, Auth.js)?

If the user has already specified these in their request, skip asking and proceed.

Step 1: Architectural Decisions

Based on requirements, make and state these decisions before coding:

Briefly explain each choice (1 sentence per decision).

Step 2: Scaffold with Checklist

Use the appropriate checklist below. Ensure ALL checked items are implemented — do not skip any.

Step 3: Implement Following Patterns

Write code following the patterns in this document. Reference specific sections as you implement each part.

Step 4: Test & Verify

After implementation, run these checks before claiming completion:

1. Build check: Ensure both backend and frontend compile without errors

   # Backend
   cd server && npm run build
   # Frontend
   cd client && npm run build

2. Start & smoke test: Start the server, verify key endpoints return expected responses

   # Start server, then test
   curl http://localhost:3000/health
   curl http://localhost:3000/api/<resource>

3. Integration check: Verify frontend can connect to backend (CORS, API base URL, auth flow) 4. Real-time check (if applicable): Open two browser tabs, verify changes sync

If any check fails, fix the issue before proceeding.

Step 5: Handoff Summary

Provide a brief summary to the user:

• What was built: List of implemented features and endpoints
• How to run: Exact commands to start backend and frontend
• What's missing / next steps: Any deferred items, known limitations, or recommended improvements
• Key files: List the most important files the user should know about

---

Scope

USE this skill when:

• Building a full-stack application (backend + frontend)
• Scaffolding a new backend service or API
• Designing service layers and module boundaries
• Implementing database access, caching, or background jobs
• Writing error handling, logging, or configuration management
• Reviewing backend code for architectural issues
• Hardening for production
• Setting up API clients, auth flows, file uploads, or real-time features

NOT for:

• Pure frontend/UI concerns (use your frontend framework's docs)
• Pure database schema design without backend context

---

Quick Start — New Backend Service Checklist

• [ ] Project scaffolded with feature-first structure
• [ ] Configuration centralized, env vars validated at startup (fail fast)
• [ ] Typed error hierarchy defined (not generic Error)
• [ ] Global error handler middleware
• [ ] Structured JSON logging with request ID propagation
• [ ] Database: migrations set up, connection pooling configured
• [ ] Input validation on all endpoints (Zod / Pydantic / Go validator)
• [ ] Authentication middleware in place
• [ ] Health check endpoints (/health, /ready)
• [ ] Graceful shutdown handling (SIGTERM)
• [ ] CORS configured (explicit origins, not *)
• [ ] Security headers (helmet or equivalent)
• [ ] .env.example committed (no real secrets)

Quick Start — Frontend-Backend Integration Checklist

• [ ] API client configured (typed fetch wrapper, React Query, tRPC, or OpenAPI generated)
• [ ] Base URL from environment variable (not hardcoded)
• [ ] Auth token attached to requests automatically (interceptor / middleware)
• [ ] Error handling — API errors mapped to user-facing messages
• [ ] Loading states handled (skeleton/spinner, not blank screen)
• [ ] Type safety across the boundary (shared types, OpenAPI, or tRPC)
• [ ] CORS configured with explicit origins (not * in production)
• [ ] Refresh token flow implemented (httpOnly cookie + transparent retry on 401)

---

Quick Navigation

---

Core Principles (7 Iron Rules)

1. ✅ Organize by FEATURE, not by technical layer
2. ✅ Controllers never contain business logic
3. ✅ Services never import HTTP request/response types
4. ✅ All config from env vars, validated at startup, fail fast
5. ✅ Every error is typed, logged, and returns consistent format
6. ✅ All input validated at the boundary — trust nothing from client
7. ✅ Structured JSON logging with request ID — not console.log

---

1. Project Structure & Layering (CRITICAL)

Feature-First Organization

✅ Feature-first                    ❌ Layer-first
src/                                src/
  orders/                             controllers/
    order.controller.ts                 order.controller.ts
    order.service.ts                    user.controller.ts
    order.repository.ts               services/
    order.dto.ts                        order.service.ts
    order.test.ts                       user.service.ts
  users/                              repositories/
    user.controller.ts                  ...
    user.service.ts
  shared/
    database/
    middleware/

Three-Layer Architecture

Controller (HTTP) → Service (Business Logic) → Repository (Data Access)

Dependency Injection (All Languages)

TypeScript:

class OrderService {
  constructor(
    private readonly orderRepo: OrderRepository,    // ✅ injected interface
    private readonly emailService: EmailService,
  ) {}
}

Python:

class OrderService:
    def __init__(self, order_repo: OrderRepository, email_service: EmailService):
        self.order_repo = order_repo                 # ✅ injected
        self.email_service = email_service

Go:

type OrderService struct {
    orderRepo    OrderRepository                      // ✅ interface
    emailService EmailService
}

func NewOrderService(repo OrderRepository, email EmailService) *OrderService {
    return &OrderService{orderRepo: repo, emailService: email}
}

---

2. Configuration & Environment (CRITICAL)

Centralized, Typed, Fail-Fast

TypeScript:

const config = {
  port: parseInt(process.env.PORT || '3000', 10),
  database: { url: requiredEnv('DATABASE_URL'), poolSize: intEnv('DB_POOL_SIZE', 10) },
  auth: { jwtSecret: requiredEnv('JWT_SECRET'), expiresIn: process.env.JWT_EXPIRES_IN || '1h' },
} as const;

function requiredEnv(name: string): string {
  const value = process.env[name];
  if (!value) throw new Error(`Missing required env var: ${name}`);  // fail fast
  return value;
}

Python:

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    database_url: str                        # required — app won't start without it
    jwt_secret: str                          # required
    port: int = 3000                         # optional with default
    db_pool_size: int = 10
    class Config:
        env_file = ".env"

settings = Settings()                        # fails fast if DATABASE_URL missing

Rules

✅ All config via environment variables (Twelve-Factor)
✅ Validate required vars at startup — fail fast
✅ Type-cast at config layer, not at usage sites
✅ Commit .env.example with dummy values

❌ Never hardcode secrets, URLs, or credentials
❌ Never commit .env files
❌ Never scatter process.env / os.environ throughout code

---

3. Error Handling & Resilience (HIGH)

Typed Error Hierarchy

// Base (TypeScript)
class AppError extends Error {
  constructor(
    message: string,
    public readonly code: string,
    public readonly statusCode: number,
    public readonly isOperational: boolean = true,
  ) { super(message); }
}
class NotFoundError extends AppError {
  constructor(resource: string, id: string) {
    super(`${resource} not found: ${id}`, 'NOT_FOUND', 404);
  }
}
class ValidationError extends AppError {
  constructor(public readonly errors: FieldError[]) {
    super('Validation failed', 'VALIDATION_ERROR', 422);
  }
}

# Base (Python)
class AppError(Exception):
    def __init__(self, message: str, code: str, status_code: int):
        self.message, self.code, self.status_code = message, code, status_code

class NotFoundError(AppError):
    def __init__(self, resource: str, id: str):
        super().__init__(f"{resource} not found: {id}", "NOT_FOUND", 404)

Global Error Handler

// TypeScript (Express)
app.use((err, req, res, next) => {
  if (err instanceof AppError && err.isOperational) {
    return res.status(err.statusCode).json({
      title: err.code, status: err.statusCode,
      detail: err.message, request_id: req.id,
    });
  }
  logger.error('Unexpected error', { error: err.message, stack: err.stack, request_id: req.id });
  res.status(500).json({ title: 'Internal Error', status: 500, request_id: req.id });
});

Rules

✅ Typed, domain-specific error classes
✅ Global error handler catches everything
✅ Operational errors → structured response
✅ Programming errors → log + generic 500
✅ Retry transient failures with exponential backoff

❌ Never catch and ignore errors silently
❌ Never return stack traces to client
❌ Never throw generic Error('something')

---

4. Database Access Patterns (HIGH)

Migrations Always

# TypeScript (Prisma)           # Python (Alembic)              # Go (golang-migrate)
npx prisma migrate dev          alembic revision --autogenerate  migrate -source file://migrations
npx prisma migrate deploy       alembic upgrade head             migrate -database $DB up

✅ Schema changes via migrations, never manual SQL
✅ Migrations must be reversible
✅ Review migration SQL before production
❌ Never modify production schema manually

N+1 Prevention

// ❌ N+1: 1 query + N queries
const orders = await db.order.findMany();
for (const o of orders) { o.items = await db.item.findMany({ where: { orderId: o.id } }); }

// ✅ Single JOIN query
const orders = await db.order.findMany({ include: { items: true } });

Transactions for Multi-Step Writes

await db.$transaction(async (tx) => {
  const order = await tx.order.create({ data: orderData });
  await tx.inventory.decrement({ productId, quantity });
  await tx.payment.create({ orderId: order.id, amount });
});

Connection Pooling

Pool size = (CPU cores × 2) + spindle_count (start with 10-20). Always set connection timeout. Use PgBouncer for serverless.

---

5. API Client Patterns (MEDIUM)

The "glue layer" between frontend and backend. Choose the approach that fits your team and stack.

Option A: Typed Fetch Wrapper (Simple, No Dependencies)

// lib/api-client.ts
const BASE_URL = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:3001';

class ApiError extends Error {
  constructor(public status: number, public body: any) {
    super(body?.detail || body?.message || `API error ${status}`);
  }
}

async function api<T>(path: string, options: RequestInit = {}): Promise<T> {
  const token = getAuthToken();  // from cookie / memory / context

  const res = await fetch(`${BASE_URL}${path}`, {
    ...options,
    headers: {
      'Content-Type': 'application/json',
      ...(token ? { Authorization: `Bearer ${token}` } : {}),
      ...options.headers,
    },
  });

  if (!res.ok) {
    const body = await res.json().catch(() => null);
    throw new ApiError(res.status, body);
  }

  if (res.status === 204) return undefined as T;
  return res.json();
}

export const apiClient = {
  get: <T>(path: string) => api<T>(path),
  post: <T>(path: string, data: unknown) => api<T>(path, { method: 'POST', body: JSON.stringify(data) }),
  put: <T>(path: string, data: unknown) => api<T>(path, { method: 'PUT', body: JSON.stringify(data) }),
  patch: <T>(path: string, data: unknown) => api<T>(path, { method: 'PATCH', body: JSON.stringify(data) }),
  delete: <T>(path: string) => api<T>(path, { method: 'DELETE' }),
};

Option B: React Query + Typed Client (Recommended for React)

// hooks/use-orders.ts
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { apiClient } from '@/lib/api-client';

interface Order { id: string; total: number; status: string; }
interface CreateOrderInput { items: { productId: string; quantity: number }[] }

export function useOrders() {
  return useQuery({
    queryKey: ['orders'],
    queryFn: () => apiClient.get<{ data: Order[] }>('/api/orders'),
    staleTime: 1000 * 60,  // 1 min
  });
}

export function useCreateOrder() {
  const queryClient = useQueryClient();
  return useMutation({
    mutationFn: (data: CreateOrderInput) =>
      apiClient.post<{ data: Order }>('/api/orders', data),
    onSuccess: () => {
      queryClient.invalidateQueries({ queryKey: ['orders'] });
    },
  });
}

// Usage in component:
function OrdersPage() {
  const { data, isLoading, error } = useOrders();
  const createOrder = useCreateOrder();
  if (isLoading) return <Skeleton />;
  if (error) return <ErrorBanner error={error} />;
  // ...
}

Option C: tRPC (Same Team Owns Both Sides)

// server: trpc/router.ts
export const appRouter = router({
  orders: router({
    list: publicProcedure.query(async () => {
      return db.order.findMany({ include: { items: true } });
    }),
    create: protectedProcedure
      .input(z.object({ items: z.array(orderItemSchema) }))
      .mutation(async ({ input, ctx }) => {
        return orderService.create(ctx.user.id, input);
      }),
  }),
});
export type AppRouter = typeof appRouter;

// client: automatic type safety, no code generation
const { data } = trpc.orders.list.useQuery();
const createOrder = trpc.orders.create.useMutation();

Option D: OpenAPI Generated Client (Public / Multi-Consumer APIs)

npx openapi-typescript-codegen \
  --input http://localhost:3001/api/openapi.json \
  --output src/generated/api \
  --client axios

Decision: Which API Client?

---

6. Authentication & Middleware (HIGH)

Full reference: references/auth-flow.md — JWT bearer flow, automatic token refresh, Next.js server-side auth, RBAC pattern, backend middleware order.

Standard Middleware Order

Request → 1.RequestID → 2.Logging → 3.CORS → 4.RateLimit → 5.BodyParse
       → 6.Auth → 7.Authz → 8.Validation → 9.Handler → 10.ErrorHandler → Response

JWT Rules

✅ Short expiry access token (15min) + refresh token (server-stored)
✅ Minimal claims: userId, roles (not entire user object)
✅ Rotate signing keys periodically

❌ Never store tokens in localStorage (XSS risk)
❌ Never pass tokens in URL query params

RBAC Pattern

function authorize(...roles: Role[]) {
  return (req, res, next) => {
    if (!req.user) throw new UnauthorizedError();
    if (!roles.some(r => req.user.roles.includes(r))) throw new ForbiddenError();
    next();
  };
}
router.delete('/users/:id', authenticate, authorize('admin'), deleteUser);

Auth Token Automatic Refresh

// lib/api-client.ts — transparent refresh on 401
async function apiWithRefresh<T>(path: string, options: RequestInit = {}): Promise<T> {
  try {
    return await api<T>(path, options);
  } catch (err) {
    if (err instanceof ApiError && err.status === 401) {
      const refreshed = await api<{ accessToken: string }>('/api/auth/refresh', {
        method: 'POST',
        credentials: 'include',  // send httpOnly cookie
      });
      setAuthToken(refreshed.accessToken);
      return api<T>(path, options);  // retry
    }
    throw err;
  }
}

---

7. Logging & Observability (MEDIUM-HIGH)

Structured JSON Logging

// ✅ Structured — parseable, filterable, alertable
logger.info('Order created', {
  orderId: order.id, userId: user.id, total: order.total,
  items: order.items.length, duration_ms: Date.now() - startTime,
});
// Output: {"level":"info","msg":"Order created","orderId":"ord_123",...}

// ❌ Unstructured — useless at scale
console.log(`Order created for user ${user.id} with total ${order.total}`);

Log Levels

Rules

✅ Request ID in every log entry (propagated via middleware)
✅ Log at layer boundaries (request in, response out, external call)
❌ Never log passwords, tokens, PII, or secrets
❌ Never use console.log in production code

---

8. Background Jobs & Async (MEDIUM)

Rules

✅ All jobs must be IDEMPOTENT (same job running twice = same result)
✅ Failed jobs → retry (max 3) → dead letter queue → alert
✅ Workers run as SEPARATE processes (not threads in API server)

❌ Never put long-running tasks in request handlers
❌ Never assume job runs exactly once

Idempotent Job Pattern

async function processPayment(data: { orderId: string }) {
  const order = await orderRepo.findById(data.orderId);
  if (order.paymentStatus === 'completed') return;  // already processed
  await paymentGateway.charge(order);
  await orderRepo.updatePaymentStatus(order.id, 'completed');
}

---

9. Caching Patterns (MEDIUM)

Cache-Aside (Lazy Loading)

async function getUser(id: string): Promise<User> {
  const cached = await redis.get(`user:${id}`);
  if (cached) return JSON.parse(cached);

  const user = await userRepo.findById(id);
  if (!user) throw new NotFoundError('User', id);

  await redis.set(`user:${id}`, JSON.stringify(user), 'EX', 900);  // 15min TTL
  return user;
}

Rules

✅ ALWAYS set TTL — never cache without expiry
✅ Invalidate on write (delete cache key after update)
✅ Use cache for reads, never for authoritative state

❌ Never cache without TTL (stale data is worse than slow data)

---

10. File Upload Patterns (MEDIUM)

Option A: Presigned URL (Recommended for Large Files)

Client → GET /api/uploads/presign?filename=photo.jpg&type=image/jpeg
Server → { uploadUrl: "https://s3.../presigned", fileKey: "uploads/abc123.jpg" }
Client → PUT uploadUrl (direct to S3, bypasses your server)
Client → POST /api/photos { fileKey: "uploads/abc123.jpg" }  (save reference)

Backend:

app.get('/api/uploads/presign', authenticate, async (req, res) => {
  const { filename, type } = req.query;
  const key = `uploads/${crypto.randomUUID()}-${filename}`;
  const url = await s3.getSignedUrl('putObject', {
    Bucket: process.env.S3_BUCKET, Key: key,
    ContentType: type, Expires: 300,  // 5 min
  });
  res.json({ uploadUrl: url, fileKey: key });
});

Frontend:

async function uploadFile(file: File) {
  const { uploadUrl, fileKey } = await apiClient.get<PresignResponse>(
    `/api/uploads/presign?filename=${file.name}&type=${file.type}`
  );
  await fetch(uploadUrl, { method: 'PUT', body: file, headers: { 'Content-Type': file.type } });
  return apiClient.post('/api/photos', { fileKey });
}

Option B: Multipart (Small Files < 10MB)

// Frontend
const formData = new FormData();
formData.append('file', file);
formData.append('description', 'Profile photo');
const res = await fetch('/api/upload', { method: 'POST', body: formData });
// Note: do NOT set Content-Type header — browser sets boundary automatically

Decision

---

11. Real-Time Patterns (MEDIUM)

Option A: Server-Sent Events (SSE) — One-Way Server → Client

Best for: notifications, live feeds, streaming AI responses.

Backend (Express):

app.get('/api/events', authenticate, (req, res) => {
  res.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    Connection: 'keep-alive',
  });
  const send = (event: string, data: unknown) => {
    res.write(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
  };
  const unsubscribe = eventBus.subscribe(req.user.id, (event) => {
    send(event.type, event.payload);
  });
  req.on('close', () => unsubscribe());
});

Frontend:

function useServerEvents(userId: string) {
  useEffect(() => {
    const source = new EventSource(`/api/events?userId=${userId}`);
    source.addEventListener('notification', (e) => {
      showToast(JSON.parse(e.data).message);
    });
    source.onerror = () => { source.close(); setTimeout(() => /* reconnect */, 3000); };
    return () => source.close();
  }, [userId]);
}

Option B: WebSocket — Bidirectional

Best for: chat, collaborative editing, gaming.

Backend (ws library):

import { WebSocketServer } from 'ws';
const wss = new WebSocketServer({ server: httpServer, path: '/ws' });
wss.on('connection', (ws, req) => {
  const userId = authenticateWs(req);
  if (!userId) { ws.close(4001, 'Unauthorized'); return; }
  ws.on('message', (raw) => handleMessage(userId, JSON.parse(raw.toString())));
  ws.on('close', () => cleanupUser(userId));
  const interval = setInterval(() => ws.ping(), 30000);
  ws.on('pong', () => { /* alive */ });
  ws.on('close', () => clearInterval(interval));
});

Frontend:

function useWebSocket(url: string) {
  const [ws, setWs] = useState<WebSocket | null>(null);
  useEffect(() => {
    const socket = new WebSocket(url);
    socket.onopen = () => setWs(socket);
    socket.onclose = () => setTimeout(() => /* reconnect */, 3000);
    return () => socket.close();
  }, [url]);
  const send = useCallback((data: unknown) => ws?.send(JSON.stringify(data)), [ws]);
  return { ws, send };
}

Option C: Polling (Simplest, No Infrastructure)

function useOrderStatus(orderId: string) {
  return useQuery({
    queryKey: ['order-status', orderId],
    queryFn: () => apiClient.get<Order>(`/api/orders/${orderId}`),
    refetchInterval: (query) => {
      if (query.state.data?.status === 'completed') return false;
      return 5000;
    },
  });
}

Decision

---

12. Cross-Boundary Error Handling (MEDIUM)

API Error → User-Facing Message

// lib/error-handler.ts
export function getErrorMessage(error: unknown): string {
  if (error instanceof ApiError) {
    switch (error.status) {
      case 401: return 'Please log in to continue.';
      case 403: return 'You don\'t have permission to do this.';
      case 404: return 'The item you\'re looking for doesn\'t exist.';
      case 409: return 'This conflicts with an existing item.';
      case 422:
        const fields = error.body?.errors;
        if (fields?.length) return fields.map((f: any) => f.message).join('. ');
        return 'Please check your input.';
      case 429: return 'Too many requests. Please wait a moment.';
      default: return 'Something went wrong. Please try again.';
    }
  }
  if (error instanceof TypeError && error.message === 'Failed to fetch') {
    return 'Cannot connect to server. Check your internet connection.';
  }
  return 'An unexpected error occurred.';
}

React Query Global Error Handler

const queryClient = new QueryClient({
  defaultOptions: {
    mutations: { onError: (error) => toast.error(getErrorMessage(error)) },
    queries: {
      retry: (failureCount, error) => {
        if (error instanceof ApiError && error.status < 500) return false;
        return failureCount < 3;
      },
    },
  },
});

Rules

✅ Map every API error code to a human-readable message
✅ Show field-level validation errors next to form inputs
✅ Auto-retry on 5xx (max 3, with backoff), never on 4xx
✅ Redirect to login on 401 (after refresh attempt fails)
✅ Show "offline" banner when fetch fails with TypeError

❌ Never show raw API error messages to users ("NullPointerException")
❌ Never silently swallow errors (show toast or log)
❌ Never retry 4xx errors (client is wrong, retrying won't help)

Integration Decision Tree

Same team owns frontend + backend?
│
├─ YES, both TypeScript
│   └─ tRPC (end-to-end type safety, zero codegen)
│
├─ YES, different languages
│   └─ OpenAPI spec → generated client (type safety via codegen)
│
├─ NO, public API
│   └─ REST + OpenAPI → generated SDKs for consumers
│
└─ Complex data needs, multiple frontends
    └─ GraphQL + codegen (flexible queries per client)

Real-time needed?
│
├─ Server → Client only (notifications, feeds, AI streaming)
│   └─ SSE (simplest, auto-reconnect, works through proxies)
│
├─ Bidirectional (chat, collaboration)
│   └─ WebSocket (need heartbeat + reconnection logic)
│
└─ Simple status polling (< 10 clients)
    └─ React Query refetchInterval (no infrastructure needed)

---

13. Production Hardening (MEDIUM)

Health Checks

app.get('/health', (req, res) => res.json({ status: 'ok' }));           // liveness
app.get('/ready', async (req, res) => {                                   // readiness
  const checks = {
    database: await checkDb(), redis: await checkRedis(), 
  };
  const ok = Object.values(checks).every(c => c.status === 'ok');
  res.status(ok ? 200 : 503).json({ status: ok ? 'ok' : 'degraded', checks });
});

Graceful Shutdown

process.on('SIGTERM', async () => {
  logger.info('SIGTERM received');
  server.close();              // stop new connections
  await drainConnections();    // finish in-flight
  await closeDatabase();
  process.exit(0);
});

Security Checklist

✅ CORS: explicit origins (never '*' in production)
✅ Security headers (helmet / equivalent)
✅ Rate limiting on public endpoints
✅ Input validation on ALL endpoints (trust nothing)
✅ HTTPS enforced
❌ Never expose internal errors to clients

---

Anti-Patterns

---

Common Issues

Issue 1: "Where does this business rule go?"

Rule: If it involves HTTP (request parsing, status codes, headers) → controller. If it involves business decisions (pricing, permissions, rules) → service. If it touches the database → repository.

Issue 2: "Service is getting too big"

Symptom: One service file > 500 lines with 20+ methods.

Fix: Split by sub-domain. OrderService → OrderCreationService + OrderFulfillmentService + OrderQueryService. Each focused on one workflow.

Issue 3: "Tests are slow because they hit the database"

Fix: Unit tests mock the repository layer (fast). Integration tests use test containers or transaction rollback (real DB, still fast). Never mock the service layer in integration tests.

---

Reference Documents

This skill includes deep-dive references for specialized topics. Read the relevant reference when you need detailed guidance.

API Design Guidelines

Framework-agnostic API design guide for backend and full-stack engineers. 50+ rules across 10 categories, prioritized by impact. Covers REST, GraphQL, and gRPC.

Scope

USE this skill when:

• Designing a new API or adding endpoints
• Reviewing API pull requests
• Choosing between REST / GraphQL / gRPC
• Writing OpenAPI specifications
• Migrating or versioning an existing API

NOT for:

• Framework-specific implementation details (use your framework's own skill/docs)
• Frontend data fetching patterns (use React Query / SWR docs)
• Authentication implementation details (use your auth library's docs)
• Database schema design (→ database-schema-design)

Context Required

Before applying this skill, gather:

---

Quick Start Checklist

New API endpoint? Run through this before writing code:

• [ ] Resource named as plural noun (/orders, not /getOrders)
• [ ] URL in kebab-case, body fields in camelCase
• [ ] Correct HTTP method (GET=read, POST=create, PUT=replace, PATCH=partial, DELETE=remove)
• [ ] Correct status code (201 Created, 422 Validation, 404 Not Found…)
• [ ] Error response follows RFC 9457 envelope
• [ ] Pagination on all list endpoints (default 20, max 100)
• [ ] Authentication required (Bearer token, not query param)
• [ ] Request ID in response header (X-Request-Id)
• [ ] Rate limit headers included
• [ ] Endpoint documented in OpenAPI spec

---

Quick Navigation

---

1. Resource Modeling (CRITICAL)

Core Rules

✅ /users                         — plural noun
✅ /users/{id}/orders              — 1 level nesting
✅ /reviews?orderId={oid}          — flatten deep nesting with query params

❌ /getUsers                       — verb in URL
❌ /user                           — singular
❌ /users/{uid}/orders/{oid}/items/{iid}/reviews  — 3+ levels deep

Max nesting: 2 levels. Beyond that, promote to top-level resource with filters.

Domain Alignment

Resources map to domain concepts, not database tables:

✅ /checkout-sessions       (domain aggregate)
✅ /shipping-labels          (domain concept)

❌ /tbl_order_header          (database table leak)
❌ /join_user_role            (internal schema leak)

---

2. URL & Naming (CRITICAL)

Python exception: If your entire stack is Python/snake_case, you MAY use snake_case in JSON — but be consistent across all endpoints.

✅ GET /users          ❌ GET /users/
✅ GET /reports/annual  ❌ GET /reports/annual.json
✅ POST /users          ❌ POST /users/create

---

3. HTTP Methods & Status Codes (CRITICAL)

Method Semantics

Status Code Quick Reference

Success:

Client Errors:

Server Errors: 500 (unexpected), 502 (upstream fail), 503 (overloaded), 504 (upstream timeout)

---

4. Error Handling (HIGH)

Standard Error Envelope (RFC 9457)

Every error response uses this format:

{
  "type": "https://api.example.com/errors/insufficient-funds",
  "title": "Insufficient Funds",
  "status": 422,
  "detail": "Account balance $10.00 is less than withdrawal $50.00.",
  "instance": "/transactions/txn_abc123",
  "request_id": "req_7f3a8b2c",
  "errors": [
    { "field": "amount", "message": "Exceeds balance", "code": "INSUFFICIENT_BALANCE" }
  ]
}

Multi-Language Implementation

TypeScript (Express):

class AppError extends Error {
  constructor(
    public readonly title: string,
    public readonly status: number,
    public readonly detail: string,
    public readonly code: string,
  ) { super(detail); }
}

// Middleware
app.use((err, req, res, next) => {
  if (err instanceof AppError) {
    return res.status(err.status).json({
      type: `https://api.example.com/errors/${err.code}`,
      title: err.title, status: err.status,
      detail: err.detail, request_id: req.id,
    });
  }
  res.status(500).json({ title: 'Internal Error', status: 500, request_id: req.id });
});

Python (FastAPI):

from fastapi import Request
from fastapi.responses import JSONResponse

class AppError(Exception):
    def __init__(self, title: str, status: int, detail: str, code: str):
        self.title, self.status, self.detail, self.code = title, status, detail, code

@app.exception_handler(AppError)
async def app_error_handler(request: Request, exc: AppError):
    return JSONResponse(status_code=exc.status, content={
        "type": f"https://api.example.com/errors/{exc.code}",
        "title": exc.title, "status": exc.status,
        "detail": exc.detail, "request_id": request.state.request_id,
    })

Iron Rules

✅ Return RFC 9457 error envelope for ALL errors
✅ Include request_id in every error response
✅ Return per-field validation errors in `errors` array

❌ Never expose stack traces in production
❌ Never return 200 for errors
❌ Never swallow errors silently

---

5. Authentication & Authorization (HIGH)

✅ Authorization: Bearer eyJhbGci...      (header)
❌ GET /users?token=eyJhbGci...            (URL — appears in logs)

✅ 401 → "Who are you?"  (missing/invalid credentials)
✅ 403 → "You can't do this"  (authenticated, no permission)
✅ 404 → Hide resource existence  (use instead of 403 when needed)

Rate Limit Headers (always include):

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 42
X-RateLimit-Reset: 1625097600
Retry-After: 30

---

6. Pagination & Filtering (HIGH)

Cursor vs Offset

Cursor pagination response:

{
  "data": [...],
  "pagination": { "next_cursor": "eyJpZCI6MTIwfQ", "has_more": true }
}

Offset pagination response:

{
  "data": [...],
  "pagination": { "page": 3, "per_page": 20, "total": 256, "total_pages": 13 }
}

Always enforce: Default 20 items, max 100 items.

Standard Filter Patterns

GET /orders?status=shipped&created_after=2025-01-01&sort=-created_at&fields=id,status

---

7. Versioning (MEDIUM-HIGH)

Non-breaking changes (no version bump): New optional response fields, new endpoints, new optional params.

Breaking changes (new version required): Removing/renaming fields, changing types, stricter validation, removing endpoints.

Deprecation headers:

Sunset: Sat, 01 Mar 2026 00:00:00 GMT
Deprecation: true
Link: <https://api.example.com/v2/users>; rel="successor-version"

---

8. Request / Response Design (MEDIUM)

Consistent Envelope

{
  "data": { "id": "ord_123", "status": "pending", "total": 99.50 },
  "meta": { "request_id": "req_abc123", "timestamp": "2025-06-15T10:30:00Z" }
}

Key Rules

---

9. Documentation — OpenAPI (MEDIUM)

Design-first workflow:

1. Write OpenAPI 3.1 spec
2. Review spec with stakeholders
3. Generate server stubs + client SDKs
4. Implement handlers
5. Validate responses against spec in CI

Every endpoint documents: summary, all parameters, request body + examples, all response codes + schemas, auth requirements.

---

10. API Style Decision Tree

What kind of API?
│
├─ Browser + mobile clients, flexible queries
│   └─ GraphQL
│       Rules: DataLoader (no N+1), depth limit ≤7, Relay pagination
│
├─ Standard CRUD, public consumers, caching important
│   └─ REST (this guide)
│       Rules: Resources, HTTP methods, status codes, OpenAPI
│
├─ Service-to-service, high throughput, strong typing
│   └─ gRPC
│       Rules: Protobuf schemas, streaming for large data, deadlines
│
├─ Full-stack TypeScript, same team owns client + server
│   └─ tRPC
│       Rules: Shared types, no code generation needed
│
└─ Real-time bidirectional
    └─ WebSocket / SSE
        Rules: Heartbeat, reconnection, message ordering

---

Anti-Patterns Checklist

---

Common Issues

Issue 1: "Should this be a new resource or a sub-resource?"

Symptom: URL path keeps growing (/users/{id}/orders/{id}/items/{id}/reviews)

Rule: If the child entity makes sense on its own, promote it. If it only exists within the parent context, keep it nested (max 2 levels).

/reviews?orderId=123      ✅  (reviews exist independently)
/orders/{id}/items         ✅  (items belong to orders, 1 level)

Issue 2: "PUT or PATCH?"

Symptom: Team can't agree on update semantics.

Rule:

• PUT = client sends complete resource (missing fields → set to default/null)
• PATCH = client sends only changed fields (missing fields → unchanged)
• When unsure → PATCH (safer, less surprising)

Issue 3: "400 or 422?"

Symptom: Inconsistent validation error codes.

Rule:

• 400 = can't parse request at all (malformed JSON, wrong content-type)
• 422 = parsed OK, but values fail validation (invalid email, negative quantity)

Authentication Flow Patterns

Complete auth flow across frontend and backend. Covers JWT bearer flow, automatic token refresh, Next.js server-side auth, RBAC, and backend middleware order.

---

JWT Bearer Flow (Most Common)

1. Login
   Client → POST /api/auth/login { email, password }
   Server → { accessToken (15min), refreshToken (7d, httpOnly cookie) }

2. Authenticated Requests
   Client → GET /api/orders  Authorization: Bearer <accessToken>
   Server → validates JWT → returns data

3. Token Refresh (transparent)
   Client → 401 received → POST /api/auth/refresh (cookie auto-sent)
   Server → new accessToken
   Client → retry original request with new token

4. Logout
   Client → POST /api/auth/logout
   Server → invalidate refresh token → clear cookie

---

Frontend: Automatic Token Refresh

// lib/api-client.ts — add to existing fetch wrapper
async function apiWithRefresh<T>(path: string, options: RequestInit = {}): Promise<T> {
  try {
    return await api<T>(path, options);
  } catch (err) {
    if (err instanceof ApiError && err.status === 401) {
      // Try refresh
      const refreshed = await api<{ accessToken: string }>('/api/auth/refresh', {
        method: 'POST',
        credentials: 'include',  // send httpOnly cookie
      });
      setAuthToken(refreshed.accessToken);
      // Retry original request
      return api<T>(path, options);
    }
    throw err;
  }
}

---

Next.js: Server-Side Auth (App Router)

// middleware.ts — protect routes server-side
import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';

export function middleware(request: NextRequest) {
  const token = request.cookies.get('session')?.value;
  if (!token && request.nextUrl.pathname.startsWith('/dashboard')) {
    return NextResponse.redirect(new URL('/login', request.url));
  }
  return NextResponse.next();
}

// app/dashboard/page.tsx — server component with auth
import { cookies } from 'next/headers';

export default async function Dashboard() {
  const token = (await cookies()).get('session')?.value;
  const user = await fetch(`${process.env.API_URL}/api/me`, {
    headers: { Authorization: `Bearer ${token}` },
  }).then(r => r.json());

  return <DashboardContent user={user} />;
}

---

Backend: Standard Middleware Order

Request → 1.RequestID → 2.Logging → 3.CORS → 4.RateLimit → 5.BodyParse
       → 6.Auth → 7.Authz → 8.Validation → 9.Handler → 10.ErrorHandler → Response

---

Backend: JWT Rules

✅ Short expiry access token (15min) + refresh token (server-stored)
✅ Minimal claims: userId, roles (not entire user object)
✅ Rotate signing keys periodically

❌ Never store tokens in localStorage (XSS risk)
❌ Never pass tokens in URL query params

---

Backend: RBAC Pattern

function authorize(...roles: Role[]) {
  return (req, res, next) => {
    if (!req.user) throw new UnauthorizedError();
    if (!roles.some(r => req.user.roles.includes(r))) throw new ForbiddenError();
    next();
  };
}
router.delete('/users/:id', authenticate, authorize('admin'), deleteUser);

---

Auth Decision Table

---

Iron Rules

✅ Access token: short-lived (15min), in memory
✅ Refresh token: httpOnly cookie (XSS-safe)
✅ Automatic transparent refresh on 401
✅ Redirect to login when refresh fails

❌ Never store tokens in localStorage (XSS risk)
❌ Never send tokens in URL query params (logged)
❌ Never trust client-side auth checks alone (server must validate)

---

Common Issues

Issue 1: "Auth works on page load but breaks on navigation"

Cause: Token stored in component state (lost on unmount).

Fix: Store access token in a persistent location:

• React Context (survives navigation, lost on refresh)
• Cookie (survives refresh)
• React Query cache with staleTime: Infinity for session

Issue 2: "CORS error with auth requests"

Cause: Missing credentials: 'include' on frontend or credentials: true on backend CORS config.

Fix: 1. Frontend: fetch(url, { credentials: 'include' }) 2. Backend: cors({ origin: 'https://your-frontend.com', credentials: true }) 3. Backend: explicit origin (not *) when using credentials

Database Schema Design

ORM-agnostic guide for relational database schema design. Covers data modeling, normalization, indexing, migrations, multi-tenancy, and common application patterns. Primarily PostgreSQL-focused but principles apply to MySQL/MariaDB.

Scope

USE this skill when:

• Designing a schema for a new project or feature
• Deciding between normalization and denormalization
• Choosing which indexes to create
• Planning a zero-downtime migration on a live database
• Implementing multi-tenant data isolation
• Adding audit trails, soft delete, or versioning
• Diagnosing slow queries caused by schema problems

NOT for:

• Choosing which database technology to use (→ technology-selection)
• PostgreSQL-specific query tuning (use PostgreSQL performance docs)
• ORM-specific configuration (→ django-best-practices or your ORM's docs)
• Application-layer caching (→ fullstack-dev-practices)

Context Required

---

Quick Start Checklist

Designing a new schema:

• [ ] Domain entities identified — map 1 entity = 1 table (not 1 class = 1 table)
• [ ] Primary keys: UUID for public IDs, serial/bigserial for internal-only
• [ ] Foreign keys with explicit ON DELETE behavior
• [ ] NOT NULL by default — nullable only when business logic requires it
• [ ] Timestamps: created_at + updated_at on every table
• [ ] Indexes created for every WHERE, JOIN, ORDER BY column
• [ ] No premature denormalization — start normalized, denormalize when measured
• [ ] Naming convention consistent: snake_case, plural table names

---

Quick Navigation

---

Core Principles (7 Rules)

1. ✅ Start normalized (3NF) — denormalize only when you have measured evidence
2. ✅ Every table has a primary key, created_at, updated_at
3. ✅ UUID for public-facing IDs, serial for internal join keys
4. ✅ NOT NULL by default — null is a business decision, not a lazy default
5. ✅ Index every column used in WHERE, JOIN, ORDER BY
6. ✅ Foreign keys enforced in database (not just application code)
7. ✅ Migrations are additive — never drop/rename in production without a multi-step plan

---

1. Data Modeling (CRITICAL)

Table Naming

-- ✅ Plural, snake_case
CREATE TABLE orders (...);
CREATE TABLE order_items (...);
CREATE TABLE user_profiles (...);

-- ❌ Singular, mixed case
CREATE TABLE Order (...);
CREATE TABLE OrderItem (...);
CREATE TABLE tbl_usr_prof (...);    -- cryptic abbreviation

Primary Keys

Recommended default:

CREATE TABLE orders (
    id          bigserial PRIMARY KEY,             -- internal FK target
    public_id   uuid NOT NULL DEFAULT gen_random_uuid() UNIQUE,  -- API-facing
    -- ...
    created_at  timestamptz NOT NULL DEFAULT now(),
    updated_at  timestamptz NOT NULL DEFAULT now()
);

Relationships

-- One-to-Many: user → orders
CREATE TABLE orders (
    id         bigserial PRIMARY KEY,
    user_id    bigint NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    -- ...
);
CREATE INDEX idx_orders_user_id ON orders(user_id);

-- Many-to-Many: orders ↔ products (via junction table)
CREATE TABLE order_items (
    id         bigserial PRIMARY KEY,
    order_id   bigint NOT NULL REFERENCES orders(id) ON DELETE CASCADE,
    product_id bigint NOT NULL REFERENCES products(id) ON DELETE RESTRICT,
    quantity   int NOT NULL CHECK (quantity > 0),
    unit_price numeric(10,2) NOT NULL,
    UNIQUE (order_id, product_id)  -- prevent duplicate line items
);

-- One-to-One: user → profile
CREATE TABLE user_profiles (
    user_id    bigint PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
    bio        text,
    avatar_url text,
    -- ...
);

ON DELETE Behavior

---

2. Normalization vs Denormalization (CRITICAL)

Start Normalized (3NF)

Normal forms in practice:

1NF violation fix:

-- ❌ Tags as comma-separated string
CREATE TABLE posts (id serial, tags text);  -- tags = "go,python"

-- ✅ Separate table (or array/JSONB if simple)
CREATE TABLE post_tags (
    post_id bigint REFERENCES posts(id) ON DELETE CASCADE,
    tag_id  bigint REFERENCES tags(id) ON DELETE CASCADE,
    PRIMARY KEY (post_id, tag_id)
);

-- ✅ Alternative: PostgreSQL array (if tags are just strings, no metadata)
CREATE TABLE posts (id serial, tags text[] NOT NULL DEFAULT '{}');
CREATE INDEX idx_posts_tags ON posts USING GIN(tags);

When to Denormalize

Denormalize ONLY when: 1. You have measured a performance problem (EXPLAIN ANALYZE, not "I think it's slow") 2. The denormalized data is read-heavy (read:write ratio > 100:1) 3. You accept the consistency maintenance cost (triggers, application logic, or materialized views)

Safe denormalization patterns:

-- Pattern 1: Materialized view (computed, refreshable)
CREATE MATERIALIZED VIEW order_summary AS
SELECT o.id, o.user_id, o.total,
       COUNT(oi.id) AS item_count,
       u.email AS user_email
FROM orders o
JOIN order_items oi ON oi.order_id = o.id
JOIN users u ON u.id = o.user_id
GROUP BY o.id, u.email;

REFRESH MATERIALIZED VIEW CONCURRENTLY order_summary;  -- non-blocking

-- Pattern 2: Cached aggregate column (application-maintained)
ALTER TABLE orders ADD COLUMN item_count int NOT NULL DEFAULT 0;
-- Update via trigger or application code on order_item insert/delete

-- Pattern 3: JSONB snapshot (freeze-at-write-time)
-- Store a copy of the product details at the time of purchase
CREATE TABLE order_items (
    id          bigserial PRIMARY KEY,
    order_id    bigint NOT NULL REFERENCES orders(id),
    product_id  bigint REFERENCES products(id),
    quantity    int NOT NULL,
    unit_price  numeric(10,2) NOT NULL,      -- frozen price
    product_snapshot jsonb NOT NULL           -- frozen name, description, image
);

---

3. Indexing Strategy (CRITICAL)

Index Types (PostgreSQL)

Index Decision Rules

Rule 1: Index every column in WHERE clauses
Rule 2: Index every column used in JOIN ON conditions
Rule 3: Index every column in ORDER BY (if queried with LIMIT)
Rule 4: Composite index for multi-column WHERE (leftmost prefix rule)
Rule 5: Partial index when filtering a subset (e.g., only active records)
Rule 6: Covering index (INCLUDE) to avoid table lookup
Rule 7: DON'T index low-cardinality columns alone (e.g., boolean)

Composite Index: Column Order Matters

-- Query: WHERE user_id = ? AND status = ? ORDER BY created_at DESC
-- ✅ Optimal: matches query pattern left-to-right
CREATE INDEX idx_orders_user_status_created
ON orders(user_id, status, created_at DESC);

-- ❌ Wrong order: can't use for this query efficiently
CREATE INDEX idx_orders_created_user_status
ON orders(created_at DESC, user_id, status);

Leftmost prefix rule: Index on (A, B, C) supports queries on (A), (A, B), (A, B, C) but NOT (B), (C), or (B, C).

Partial Index (Index Only What Matters)

-- Only 5% of orders are 'pending', but queried frequently
CREATE INDEX idx_orders_pending
ON orders(created_at DESC)
WHERE status = 'pending';

-- Only active users matter for login
CREATE INDEX idx_users_active_email
ON users(email)
WHERE is_active = true;

Covering Index (Avoid Table Lookup)

-- Query only needs id and status, no need to read the table row
CREATE INDEX idx_orders_user_covering
ON orders(user_id) INCLUDE (status, total);

-- Now this query is index-only:
SELECT status, total FROM orders WHERE user_id = 123;

When NOT to Index

❌ Columns rarely used in WHERE/JOIN/ORDER BY
❌ Tables with < 1,000 rows (sequential scan is faster)
❌ Columns with very low cardinality alone (e.g., boolean is_active)
❌ Write-heavy tables where index maintenance cost > read benefit
❌ Duplicate indexes (check pg_stat_user_indexes for unused indexes)

---

4. Zero-Downtime Migrations (HIGH)

The Golden Rule

NEVER make destructive changes in one step.
Always: ADD → MIGRATE DATA → REMOVE OLD (in separate deploys).

Safe Migration Patterns

Rename a column (3 deploys):

Deploy 1: Add new column
  ALTER TABLE users ADD COLUMN full_name text;
  UPDATE users SET full_name = name;           -- backfill
  -- App writes to BOTH name and full_name

Deploy 2: Switch reads to new column
  -- App reads from full_name, still writes to both

Deploy 3: Drop old column
  ALTER TABLE users DROP COLUMN name;
  -- App only uses full_name

Add a NOT NULL column (2 deploys):

-- Deploy 1: Add nullable column, backfill
ALTER TABLE orders ADD COLUMN currency text;              -- nullable first
UPDATE orders SET currency = 'USD' WHERE currency IS NULL; -- backfill

-- Deploy 2: Add constraint (after all rows backfilled)
ALTER TABLE orders ALTER COLUMN currency SET NOT NULL;
ALTER TABLE orders ALTER COLUMN currency SET DEFAULT 'USD';

Add an index without locking:

-- ✅ CONCURRENTLY: no table lock, can run on live DB
CREATE INDEX CONCURRENTLY idx_orders_status ON orders(status);

-- ❌ Without CONCURRENTLY: locks table for writes during build
CREATE INDEX idx_orders_status ON orders(status);

Migration Safety Checklist

✅ Migration runs in < 30 seconds on production data size
✅ No exclusive table locks (use CONCURRENTLY for indexes)
✅ Rollback plan documented and tested
✅ Backfill runs in batches (not one giant UPDATE)
✅ New column added as nullable first, constraint added later
✅ Old column kept until all code references removed

❌ Never rename/drop columns in one deploy
❌ Never ALTER TYPE on large tables without testing timing
❌ Never run data backfill in a transaction (OOM on large tables)

Batch Backfill Template

-- Backfill in batches of 10,000 (avoids long-running transactions)
DO $$
DECLARE
  batch_size int := 10000;
  affected int;
BEGIN
  LOOP
    UPDATE orders
    SET currency = 'USD'
    WHERE id IN (
      SELECT id FROM orders WHERE currency IS NULL LIMIT batch_size
    );
    GET DIAGNOSTICS affected = ROW_COUNT;
    RAISE NOTICE 'Updated % rows', affected;
    EXIT WHEN affected = 0;
    PERFORM pg_sleep(0.1);  -- brief pause to reduce load
  END LOOP;
END $$;

---

5. Multi-Tenant Design (HIGH)

Three Approaches

Row-Level Tenancy (Most Common)

-- Every table has tenant_id
CREATE TABLE orders (
    id         bigserial PRIMARY KEY,
    tenant_id  bigint NOT NULL REFERENCES tenants(id),
    user_id    bigint NOT NULL REFERENCES users(id),
    total      numeric(10,2) NOT NULL,
    -- ...
);

-- Composite index: tenant first (most queries filter by tenant)
CREATE INDEX idx_orders_tenant_user ON orders(tenant_id, user_id);
CREATE INDEX idx_orders_tenant_status ON orders(tenant_id, status);

-- Row-Level Security (PostgreSQL)
ALTER TABLE orders ENABLE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON orders
  USING (tenant_id = current_setting('app.tenant_id')::bigint);

Application-level enforcement:

// Middleware: set tenant context on every request
app.use((req, res, next) => {
  const tenantId = req.headers['x-tenant-id'];
  if (!tenantId) return res.status(400).json({ error: 'Missing tenant' });
  req.tenantId = tenantId;
  next();
});

// Repository: ALWAYS filter by tenant
async findOrders(tenantId: string, userId: string) {
  return db.order.findMany({
    where: { tenantId, userId },  // ← tenant_id in EVERY query
  });
}

Rules

✅ tenant_id in EVERY table that holds tenant data
✅ tenant_id as FIRST column in every composite index
✅ Application middleware enforces tenant context
✅ Use RLS (PostgreSQL) as defense-in-depth, not sole protection
✅ Test with 2+ tenants to verify isolation

❌ Never allow cross-tenant queries in application code
❌ Never skip tenant_id in WHERE clauses (even in admin tools)

---

6. Common Schema Patterns (MEDIUM)

Soft Delete

ALTER TABLE orders ADD COLUMN deleted_at timestamptz;

-- All queries filter deleted records
CREATE VIEW active_orders AS
SELECT * FROM orders WHERE deleted_at IS NULL;

-- Partial index: only index non-deleted rows
CREATE INDEX idx_orders_active_status
ON orders(status, created_at DESC)
WHERE deleted_at IS NULL;

ORM integration:

// Prisma middleware: auto-filter soft-deleted records
prisma.$use(async (params, next) => {
  if (params.action === 'findMany' || params.action === 'findFirst') {
    params.args.where = { ...params.args.where, deletedAt: null };
  }
  return next(params);
});

Audit Trail

-- Option A: Audit columns on every table
ALTER TABLE orders ADD COLUMN created_by bigint REFERENCES users(id);
ALTER TABLE orders ADD COLUMN updated_by bigint REFERENCES users(id);

-- Option B: Separate audit log table (more detail)
CREATE TABLE audit_log (
    id          bigserial PRIMARY KEY,
    table_name  text NOT NULL,
    record_id   bigint NOT NULL,
    action      text NOT NULL CHECK (action IN ('INSERT', 'UPDATE', 'DELETE')),
    old_data    jsonb,
    new_data    jsonb,
    changed_by  bigint REFERENCES users(id),
    changed_at  timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX idx_audit_table_record ON audit_log(table_name, record_id);
CREATE INDEX idx_audit_changed_at ON audit_log(changed_at DESC);

Enum Columns

-- Option A: PostgreSQL enum type (strict, but ALTER TYPE is painful)
CREATE TYPE order_status AS ENUM ('pending', 'confirmed', 'shipped', 'delivered', 'cancelled');
ALTER TABLE orders ADD COLUMN status order_status NOT NULL DEFAULT 'pending';

-- Option B: Text + CHECK constraint (easier to migrate)
ALTER TABLE orders ADD COLUMN status text NOT NULL DEFAULT 'pending'
  CHECK (status IN ('pending', 'confirmed', 'shipped', 'delivered', 'cancelled'));

-- Option C: Lookup table (most flexible, best for UI-driven lists)
CREATE TABLE order_statuses (
    id    serial PRIMARY KEY,
    name  text UNIQUE NOT NULL,
    label text NOT NULL      -- display name
);

Recommendation: Option B (text + CHECK) for most cases. Option C if statuses are managed by non-developers.

Polymorphic Associations

-- ❌ Anti-pattern: polymorphic FK (no referential integrity)
CREATE TABLE comments (
    id             bigserial PRIMARY KEY,
    commentable_type text,    -- 'Post' or 'Photo'
    commentable_id   bigint,  -- no FK constraint possible!
    body           text
);

-- ✅ Pattern A: Separate FK columns (nullable)
CREATE TABLE comments (
    id       bigserial PRIMARY KEY,
    post_id  bigint REFERENCES posts(id) ON DELETE CASCADE,
    photo_id bigint REFERENCES photos(id) ON DELETE CASCADE,
    body     text NOT NULL,
    CHECK (
      (post_id IS NOT NULL AND photo_id IS NULL) OR
      (post_id IS NULL AND photo_id IS NOT NULL)
    )
);

-- ✅ Pattern B: Separate tables (cleanest, best for different schemas)
CREATE TABLE post_comments (..., post_id bigint REFERENCES posts(id));
CREATE TABLE photo_comments (..., photo_id bigint REFERENCES photos(id));

JSONB Columns (Semi-Structured Data)

-- Good uses: metadata, settings, flexible attributes
CREATE TABLE products (
    id         bigserial PRIMARY KEY,
    name       text NOT NULL,
    price      numeric(10,2) NOT NULL,
    attributes jsonb NOT NULL DEFAULT '{}'  -- color, size, weight...
);

-- Index for JSONB queries
CREATE INDEX idx_products_attrs ON products USING GIN(attributes);

-- Query
SELECT * FROM products WHERE attributes->>'color' = 'red';
SELECT * FROM products WHERE attributes @> '{"size": "XL"}';

✅ Use JSONB for truly flexible/optional data (metadata, settings, preferences)
✅ Index JSONB columns with GIN when queried

❌ Never use JSONB for data that should be columns (email, status, price)
❌ Never use JSONB to avoid schema design (it's not MongoDB-in-Postgres)

---

7. Table Partitioning (MEDIUM)

When to Partition

✅ Table > 100M rows AND growing
✅ Most queries filter on the partition key (date range, tenant)
✅ Old data can be dropped/archived by partition (efficient DELETE)

❌ Table < 10M rows (overhead not worth it)
❌ Queries don't filter on partition key (scans all partitions)

Range Partitioning (Time-Series)

CREATE TABLE events (
    id         bigserial,
    tenant_id  bigint NOT NULL,
    event_type text NOT NULL,
    payload    jsonb,
    created_at timestamptz NOT NULL DEFAULT now()
) PARTITION BY RANGE (created_at);

-- Monthly partitions
CREATE TABLE events_2025_01 PARTITION OF events
  FOR VALUES FROM ('2025-01-01') TO ('2025-02-01');
CREATE TABLE events_2025_02 PARTITION OF events
  FOR VALUES FROM ('2025-02-01') TO ('2025-03-01');

-- Automate partition creation with pg_partman or cron

List Partitioning (Multi-Tenant)

CREATE TABLE orders (
    id        bigserial,
    tenant_id bigint NOT NULL,
    total     numeric(10,2)
) PARTITION BY LIST (tenant_id);

CREATE TABLE orders_tenant_1 PARTITION OF orders FOR VALUES IN (1);
CREATE TABLE orders_tenant_2 PARTITION OF orders FOR VALUES IN (2);

---

Anti-Patterns

---

Common Issues

Issue 1: "Query is slow but I already have an index"

Symptom: EXPLAIN ANALYZE shows Sequential Scan despite existing index.

Causes: 1. Wrong index column order — composite index (A, B) won't help WHERE B = ? 2. Low selectivity — index on boolean column (50% of rows match), planner prefers seq scan 3. Stale statistics — run ANALYZE table_name; 4. Type mismatch — comparing varchar column with integer parameter → no index use

Fix: Check EXPLAIN (ANALYZE, BUFFERS), verify index matches query pattern, run ANALYZE.

Issue 2: "Migration locks the table for minutes"

Symptom: ALTER TABLE blocks all writes during execution.

Cause: Adding NOT NULL constraint, changing column type, or creating index without CONCURRENTLY.

Fix:

-- Add index without lock
CREATE INDEX CONCURRENTLY idx_name ON table(col);

-- Add NOT NULL constraint without lock (Postgres 12+)
ALTER TABLE t ADD CONSTRAINT t_col_nn CHECK (col IS NOT NULL) NOT VALID;
ALTER TABLE t VALIDATE CONSTRAINT t_col_nn;  -- non-blocking validation

Issue 3: "How many indexes is too many?"

Rule of thumb:

• Read-heavy table (reports, product catalog): 5-10 indexes is fine
• Write-heavy table (events, logs): 2-3 indexes max
• Monitor with pg_stat_user_indexes — drop indexes with idx_scan = 0

-- Find unused indexes
SELECT schemaname, relname, indexrelname, idx_scan
FROM pg_stat_user_indexes
WHERE idx_scan = 0 AND indexrelname NOT LIKE '%pkey%'
ORDER BY pg_relation_size(indexrelid) DESC;

Django Best Practices

Production-grade guide for Django 5.x and Django REST Framework. 40+ rules across 8 categories.

Core Principles (7 Rules)

1. ✅ Custom User model BEFORE first migration (can't change later)
2. ✅ One Django app per domain concept (users, orders, payments)
3. ✅ Fat models, thin views — business logic in models/managers, not views
4. ✅ Always use select_related/prefetch_related (prevent N+1)
5. ✅ Settings split by environment (base + dev + prod)
6. ✅ Test with pytest-django + factory_boy (not fixtures)
7. ✅ Never use runserver in production (Gunicorn + Nginx)

---

1. Project Structure (CRITICAL)

App-Per-Domain

myproject/
├── config/                     # Project config
│   ├── __init__.py
│   ├── settings/
│   │   ├── base.py             # Shared settings
│   │   ├── dev.py              # DEBUG=True, SQLite ok
│   │   └── prod.py             # DEBUG=False, Postgres, HTTPS
│   ├── urls.py
│   ├── wsgi.py
│   └── asgi.py
├── apps/
│   ├── users/                  # Custom User model
│   │   ├── models.py
│   │   ├── serializers.py
│   │   ├── views.py
│   │   ├── urls.py
│   │   ├── admin.py
│   │   ├── services.py         # Business logic
│   │   ├── selectors.py        # Complex queries
│   │   └── tests/
│   │       ├── test_models.py
│   │       ├── test_views.py
│   │       └── factories.py
│   ├── orders/
│   └── payments/
├── manage.py
├── requirements/
│   ├── base.txt
│   ├── dev.txt
│   └── prod.txt
└── docker-compose.yml

Rules

✅ One app = one bounded context (users, orders, payments)
✅ Business logic in services.py / selectors.py, not views
✅ Each app has its own urls.py, admin.py, tests/

❌ Never put everything in one app
❌ Never import across app boundaries at the model level (use IDs)
❌ Never put business logic in views or serializers

---

2. Models & Migrations (CRITICAL)

Custom User Model (Day 1!)

# apps/users/models.py
from django.contrib.auth.models import AbstractUser
from django.db import models
import uuid

class User(AbstractUser):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    email = models.EmailField(unique=True)

    USERNAME_FIELD = 'email'
    REQUIRED_FIELDS = ['username']

    class Meta:
        db_table = 'users'

# config/settings/base.py
AUTH_USER_MODEL = 'users.User'

This MUST be done before `migrate`. Cannot change after.

Model Best Practices

class TimeStampedModel(models.Model):
    created_at = models.DateTimeField(auto_now_add=True)
    updated_at = models.DateTimeField(auto_now=True)
    class Meta:
        abstract = True

class Order(TimeStampedModel):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE, related_name='orders')
    status = models.CharField(max_length=20, choices=OrderStatus.choices, default=OrderStatus.PENDING, db_index=True)
    total = models.DecimalField(max_digits=10, decimal_places=2)

    class Meta:
        db_table = 'orders'
        ordering = ['-created_at']
        indexes = [
            models.Index(fields=['user', 'status']),
        ]

    def can_cancel(self) -> bool:
        return self.status in [OrderStatus.PENDING, OrderStatus.CONFIRMED]

    def cancel(self):
        if not self.can_cancel():
            raise ValueError(f"Cannot cancel order in {self.status} status")
        self.status = OrderStatus.CANCELLED
        self.save(update_fields=['status', 'updated_at'])

Migration Rules

✅ Review migration SQL: python manage.py sqlmigrate app_name 0001
✅ Name migrations descriptively: --name add_status_index_to_orders
✅ Separate data migrations from schema migrations
✅ Non-destructive first: add column → backfill → remove old column

❌ Never edit or delete applied migrations
❌ Never use RunPython without reverse function

---

3. Views & Serializers — DRF (HIGH)

Service Layer Pattern

# apps/orders/services.py
from django.db import transaction

class OrderService:
    @staticmethod
    @transaction.atomic
    def create_order(user, items_data: list[dict]) -> Order:
        total = sum(item['price'] * item['quantity'] for item in items_data)
        order = Order.objects.create(user=user, total=total)
        OrderItem.objects.bulk_create([
            OrderItem(order=order, **item) for item in items_data
        ])
        return order

    @staticmethod
    def cancel_order(order_id: str, user) -> Order:
        order = Order.objects.select_for_update().get(id=order_id, user=user)
        order.cancel()
        return order

Serializers

class OrderSerializer(serializers.ModelSerializer):
    items = OrderItemSerializer(many=True, read_only=True)
    class Meta:
        model = Order
        fields = ['id', 'status', 'total', 'items', 'created_at']
        read_only_fields = ['id', 'total', 'created_at']

class CreateOrderSerializer(serializers.Serializer):
    """Input-only serializer — separate from output."""
    items = serializers.ListField(
        child=serializers.DictField(), min_length=1, max_length=50,
    )
    def validate_items(self, items):
        for item in items:
            if item.get('quantity', 0) < 1:
                raise serializers.ValidationError("Quantity must be at least 1")
        return items

Views (Thin!)

@api_view(['POST'])
@permission_classes([IsAuthenticated])
def create_order(request):
    serializer = CreateOrderSerializer(data=request.data)
    serializer.is_valid(raise_exception=True)
    order = OrderService.create_order(request.user, serializer.validated_data['items'])
    return Response({'data': OrderSerializer(order).data}, status=status.HTTP_201_CREATED)

Rules

✅ Separate input serializers from output serializers
✅ Views only: validate → call service → serialize → respond
✅ Use @transaction.atomic for multi-model writes

❌ Never put business logic in views or serializers
❌ Never use ModelSerializer for write operations (too implicit)

---

4. Authentication (HIGH)

JWT Config (djangorestframework-simplejwt)

SIMPLE_JWT = {
    'ACCESS_TOKEN_LIFETIME': timedelta(minutes=15),
    'REFRESH_TOKEN_LIFETIME': timedelta(days=7),
    'ROTATE_REFRESH_TOKENS': True,
    'BLACKLIST_AFTER_ROTATION': True,
}

---

5. Performance Optimization (HIGH)

N+1 Query Prevention

# ❌ N+1: 1 query for orders + N queries for users
orders = Order.objects.all()
for o in orders:
    print(o.user.email)     # hits DB each iteration

# ✅ select_related (FK/OneToOne — JOIN)
orders = Order.objects.select_related('user').all()

# ✅ prefetch_related (ManyToMany/reverse FK — 2 queries)
orders = Order.objects.prefetch_related('items').all()

# ✅ Combined
orders = Order.objects.select_related('user').prefetch_related('items').all()

Query Optimization Toolkit

# Only fetch needed columns
User.objects.values('id', 'email')
User.objects.values_list('email', flat=True)

# Annotate instead of Python loops
from django.db.models import Count, Sum
Order.objects.annotate(item_count=Count('items'), revenue=Sum('items__price'))

# Bulk operations
OrderItem.objects.bulk_create([...])
Order.objects.filter(status='pending').update(status='cancelled')

# Database indexes
class Meta:
    indexes = [
        models.Index(fields=['user', 'status']),
        models.Index(fields=['-created_at']),
        models.Index(fields=['email'], condition=Q(is_active=True)),
    ]

# Pagination
from rest_framework.pagination import CursorPagination
class OrderPagination(CursorPagination):
    page_size = 20
    ordering = '-created_at'

Caching

from django.core.cache import cache

def get_product(product_id: str):
    cache_key = f'product:{product_id}'
    product = cache.get(cache_key)
    if product is None:
        product = Product.objects.get(id=product_id)
        cache.set(cache_key, product, timeout=300)
    return product

---

6. Testing (MEDIUM-HIGH)

pytest-django + factory_boy

# conftest.py
@pytest.fixture
def api_client():
    return APIClient()

@pytest.fixture
def authenticated_client(api_client, user_factory):
    user = user_factory()
    api_client.force_authenticate(user=user)
    return api_client

# factories.py
class UserFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = User
    email = factory.Sequence(lambda n: f'user{n}@example.com')
    username = factory.Sequence(lambda n: f'user{n}')

class OrderFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = 'orders.Order'
    user = factory.SubFactory(UserFactory)
    total = factory.Faker('pydecimal', left_digits=3, right_digits=2, positive=True)

# test_views.py
@pytest.mark.django_db
class TestListOrders:
    def test_returns_user_orders(self, authenticated_client):
        OrderFactory.create_batch(3, user=authenticated_client.handler._force_user)
        response = authenticated_client.get('/api/orders/')
        assert response.status_code == 200
        assert len(response.data['data']) == 3

    def test_requires_authentication(self, api_client):
        response = api_client.get('/api/orders/')
        assert response.status_code == 401

---

7. Admin Customization (MEDIUM)

class OrderItemInline(admin.TabularInline):
    model = OrderItem
    extra = 0
    readonly_fields = ['price']

@admin.register(Order)
class OrderAdmin(admin.ModelAdmin):
    list_display = ['id', 'user', 'status', 'total', 'created_at']
    list_filter = ['status', 'created_at']
    search_fields = ['user__email', 'id']
    readonly_fields = ['id', 'created_at', 'updated_at']
    inlines = [OrderItemInline]
    date_hierarchy = 'created_at'

    def get_queryset(self, request):
        return super().get_queryset(request).select_related('user')

---

8. Production Deployment (MEDIUM)

Security Settings

# settings/prod.py
DEBUG = False
ALLOWED_HOSTS = ['example.com', 'www.example.com']
CSRF_TRUSTED_ORIGINS = ['https://example.com']
SECURE_SSL_REDIRECT = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
SECURE_HSTS_SECONDS = 31536000

Deployment Stack

Nginx → Gunicorn → Django
         ↕
      PostgreSQL + Redis (cache)
         ↕
      Celery (background tasks)

gunicorn config.wsgi:application \
  --bind 0.0.0.0:8000 \
  --workers 4 \
  --timeout 120 \
  --access-logfile -

WhiteNoise for Static Files

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',  # right after Security
    ...
]
STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

Rules

✅ Gunicorn + Nginx (or Cloud Run / Railway)
✅ PostgreSQL (not SQLite)
✅ python manage.py check --deploy
✅ Sentry for error tracking

❌ Never use runserver in production
❌ Never use DEBUG=True in production
❌ Never use SQLite in production

---

Anti-Patterns

---

Common Issues

Issue 1: "Can't change User model after first migration"

Fix: If starting fresh: delete all migrations + DB, set custom User, re-migrate. If data exists: complex migration (use django-allauth or incremental field migration).

Issue 2: "Serializer is too slow on large querysets"

Fix: Missing select_related / prefetch_related → N+1 queries.

queryset = Order.objects.select_related('user').prefetch_related('items')

Issue 3: "Circular import between apps"

Fix: Use string references: models.ForeignKey('orders.Order', ...) instead of importing the model class. For services, import inside the function.

Environment & CORS Management

Patterns for managing environment variables, API URLs, and CORS configuration across frontend and backend stacks.

---

Standard Environment Pattern

# .env.local (gitignored, for local dev)
NEXT_PUBLIC_API_URL=http://localhost:3001
NEXT_PUBLIC_WS_URL=ws://localhost:3001

# Staging (set in Vercel/CI)
NEXT_PUBLIC_API_URL=https://api-staging.example.com

# Production (set in Vercel/CI)
NEXT_PUBLIC_API_URL=https://api.example.com

---

Environment Variable Rules

✅ API base URL from environment variable — NEVER hardcoded
✅ Prefix client-side vars with NEXT_PUBLIC_ (Next.js) or VITE_ (Vite)
✅ Backend URL = server-only env var (for SSR calls, not exposed to browser)
✅ CORS on backend: explicit list of allowed origins per environment

❌ Never use localhost URLs in production builds
❌ Never expose backend-only secrets with NEXT_PUBLIC_ prefix
❌ Never commit .env.local (commit .env.example with placeholders)

---

CORS Configuration

// Backend: environment-aware CORS
const ALLOWED_ORIGINS = {
  development: ['http://localhost:3000', 'http://localhost:5173'],
  staging: ['https://staging.example.com'],
  production: ['https://example.com', 'https://www.example.com'],
};

app.use(cors({
  origin: ALLOWED_ORIGINS[process.env.NODE_ENV || 'development'],
  credentials: true,  // needed for cookies (auth)
  methods: ['GET', 'POST', 'PUT', 'PATCH', 'DELETE'],
}));

---

Common Issues

Issue 1: "CORS error in browser but works in Postman"

Cause: CORS is a browser security feature. Postman/curl skip it.

Fix: 1. Backend must return Access-Control-Allow-Origin: https://your-frontend.com 2. For cookies/auth: credentials: true on both sides 3. Check that preflight OPTIONS request returns correct headers

Issue 2: "Environment variable undefined in browser"

Cause: Missing NEXT_PUBLIC_ or VITE_ prefix for client-side access.

Fix: Client-side vars MUST have the framework prefix. Rebuild after adding new env vars (they are embedded at build time).

Issue 3: "Works locally, fails in staging"

Cause: Different origins, missing CORS config for staging domain.

Fix: Add staging origin to ALLOWED_ORIGINS, verify env vars are set in deployment platform.

Release & Acceptance Checklist

6-gate release checklist for backend and full-stack applications. Prevents "it works on my machine" and "we forgot to check X" failures.

Iron Law: NO RELEASE WITHOUT ALL GATES PASSING.

---

Release Gates Overview

Feature Complete
    ↓
Gate 1: Functional Acceptance        → Does it do what it should?
    ↓
Gate 2: Non-Functional Acceptance    → Is it fast, reliable, observable?
    ↓
Gate 3: Security Review              → Is it safe?
    ↓
Gate 4: Deployment Readiness         → Can we deploy and rollback safely?
    ↓
Gate 5: Release Execution            → Deploy with canary + monitoring
    ↓
Gate 6: Post-Release Validation      → Did it actually work in production?

---

Gate 1: Functional Acceptance

Question: Does it do what the requirements say?

• [ ] All acceptance criteria from ticket/PRD have passing tests
• [ ] Happy path works end-to-end
• [ ] Edge cases tested (empty inputs, max lengths, Unicode)
• [ ] Error cases tested (invalid input, not found, timeout)
• [ ] Data integrity verified (CRUD cycle produces correct state)
• [ ] Backward compatibility confirmed (existing clients not broken)
• [ ] API contract matches OpenAPI spec
• [ ] Idempotency verified (retries don't create duplicates)

Evidence Template

---

Gate 2: Non-Functional Acceptance

Question: Is it fast, reliable, and observable?

Performance

• [ ] Response time within budget (p95 < ___ms) — measured, not assumed
• [ ] No N+1 queries (checked with query logging)
• [ ] New queries use indexes (EXPLAIN ANALYZE)
• [ ] Pagination works on large datasets
• [ ] Caching effective (hit rate > 80%)
• [ ] Connection pool healthy under load

Reliability

• [ ] Graceful degradation when dependencies fail (circuit breaker)
• [ ] Retry logic works for transient failures
• [ ] All external calls have timeouts
• [ ] Rate limiting returns 429 correctly
• [ ] Health check endpoints verified (/health, /ready)

Observability

• [ ] Structured logging with request ID (not console.log)
• [ ] Metrics exposed (request count, latency, error rate)
• [ ] Alerts configured (error spike, latency spike)
• [ ] Request tracing works end-to-end
• [ ] Dashboard updated for new feature

Evidence

---

Gate 3: Security Review

Question: Does this introduce vulnerabilities?

Input & Output

• [ ] All input validated server-side (never trust client)
• [ ] SQL injection prevented (parameterized queries only)
• [ ] XSS prevented (output encoding)
• [ ] File upload validated (type, size, name sanitized)
• [ ] Rate limiting on sensitive endpoints (login, reset, APIs)

Auth & Data

• [ ] Protected endpoints require valid credentials
• [ ] Users can only access their own resources
• [ ] Admin routes require admin role
• [ ] Tokens expire (short-lived access + refresh)
• [ ] Passwords hashed (bcrypt/argon2, not MD5/SHA)
• [ ] Sensitive data not logged (passwords, tokens, PII)
• [ ] Secrets in env vars (not hardcoded)
• [ ] Error messages don't leak internals

Dependencies

• [ ] No known vulnerabilities (npm audit / pip audit / govulncheck)
• [ ] Dependencies pinned in lockfile
• [ ] Unused dependencies removed

---

Gate 4: Deployment Readiness

Question: Can we deploy safely and roll back if needed?

Code

• [ ] All tests pass in CI (not "it passed locally")
• [ ] Linter clean, build succeeds
• [ ] Code reviewed and approved
• [ ] No unresolved TODO/FIXME/HACK

Database

• [ ] Migration tested on staging with production-like data
• [ ] Down migration works (tested!)
• [ ] Migration is non-destructive (additive only)
• [ ] Migration timing estimated on production data size
• [ ] Backfill plan documented (if needed)

Configuration

• [ ] New env vars documented in .env.example
• [ ] Env vars set in staging and verified
• [ ] Env vars set in production
• [ ] Feature flags configured (if applicable)

Rollback Plan Template

## Rollback Plan: [Feature]

### When to rollback
- Error rate > 1% sustained 5 minutes
- p99 latency > 3000ms sustained 10 minutes
- Critical business function broken

### Steps
1. Revert deploy: [command]
2. Rollback migration (if applied): [command]
3. Invalidate cache: [command]
4. Notify team: #incidents channel
5. Verify rollback: [verification steps]

### Estimated time: [X minutes]
### Data recovery: [procedure if data was modified]

---

Gate 5: Release Execution

Deployment Sequence

1. 📢 ANNOUNCE in release channel

2. 🗄️ DATABASE — Apply migration
   - Run migration
   - Verify completion
   - Check data integrity

3. 🚀 DEPLOY — Roll out code
   - Canary first (10% traffic)
   - Monitor 5 minutes
   - If OK → 50% → monitor → 100%
   - If NOT OK → STOP immediately

4. 🔍 SMOKE TEST
   - Health check → 200
   - Login works
   - Core operation works
   - No error spikes

5. ✅ ANNOUNCE "Release complete. Monitoring 30 min."

Canary Decision Table

---

Gate 6: Post-Release Validation

Immediate (0-30 min)

• [ ] Health checks green on all instances
• [ ] Error rate within normal range
• [ ] Latency normal (p95, p99)
• [ ] Core user journey manually tested
• [ ] Logs clean — no unexpected errors
• [ ] Alerts silent

Short-term (1-24 hours)

• [ ] No customer complaints
• [ ] Business metrics stable (conversion, revenue, signups)
• [ ] Memory/CPU stable (no creeping usage)
• [ ] Queue backlogs clear
• [ ] Database performance stable

Post-Release Report Template

## Release Report: [Feature]
- Deployed: [timestamp] by @[engineer]
- Duration: [minutes]

| Check | Status | Notes |
|-------|--------|-------|
| Health checks | ✅ | All healthy |
| Error rate | ✅ | 0.03% (baseline: 0.05%) |
| p95 latency | ✅ | 310ms (baseline: 300ms) |
| Core flow | ✅ | Order creation verified |

Issues found: None / [details]
Rollback used: No / Yes: [reason]

---

Release Readiness Score

Score each gate 0-2: (0 = not checked, 1 = partially, 2 = fully verified with evidence)

Decision:

• 12/12 → Ship it ✅
• 10-11 → Ship with documented exceptions + owner assigned
• < 10 → Do NOT release. Fix gaps first.

---

Common Rationalizations

Technology Selection Framework

Structured decision framework for backend and full-stack technology choices. Prevents analysis paralysis while ensuring rigorous evaluation.

Iron Law: NO TECHNOLOGY CHOICE WITHOUT EXPLICIT TRADE-OFF ANALYSIS.

"I like it" and "it's trending" are not engineering arguments.

---

Phase 1: Requirements Before Technology

Non-Functional Requirements (Quantify!)

Team Constraints

• Team size and seniority level
• What the team already knows well
• Can you hire for this stack? (check job market)
• Timeline pressure (days vs months to production)
• Budget for licenses, infrastructure, training

---

Phase 2: Evaluation Matrix

Score each option 1-5 on weighted criteria:

Rules:

• Any option scoring 1 on a 5× criterion → automatically disqualified
• Options within 10% of each other → choose what team knows best
• Options within 15% → run a time-boxed PoC (2-5 days max)

---

Phase 3: Decision Trees

Backend Language / Framework

What type of project?
│
├─ REST/GraphQL API, rapid development
│   ├─ Team knows TypeScript → Node.js
│   │   ├─ Full-featured, enterprise patterns → NestJS
│   │   ├─ Lightweight, flexible → Fastify / Hono / Express
│   │   └─ Full-stack with React → Next.js API routes
│   ├─ Team knows Python
│   │   ├─ High-perf async API → FastAPI
│   │   ├─ Full-stack, admin-heavy → Django
│   │   └─ Lightweight → Flask / Litestar
│   └─ Team knows Java/Kotlin
│       ├─ Enterprise, large team → Spring Boot
│       └─ Lightweight, fast startup → Quarkus / Ktor
│
├─ High concurrency, systems-level
│   ├─ Microservices, network → Go
│   ├─ Extreme perf, safety → Rust (Axum / Actix)
│   └─ Fault tolerance → Elixir (Phoenix)
│
├─ Real-time (WebSocket, streaming)
│   ├─ Node.js ecosystem → Socket.io / ws
│   ├─ Scalable pub/sub → Elixir Phoenix
│   └─ Low-latency → Go / Rust
│
└─ ML / data-intensive
    └─ Python (FastAPI + ML libs)

Database

What data model?
│
├─ Structured, relational, ACID
│   ├─ General purpose → PostgreSQL ← DEFAULT CHOICE
│   ├─ Read-heavy, MySQL ecosystem → MySQL / MariaDB
│   └─ Embedded / serverless edge → SQLite / Turso / D1
│
├─ Semi-structured, flexible schema
│   ├─ Document-oriented → MongoDB
│   ├─ Serverless document → DynamoDB / Firestore
│   └─ Search-heavy → Elasticsearch / OpenSearch
│
├─ Key-value / cache
│   ├─ In-memory + data structures → Redis / Valkey
│   └─ Planet-scale KV → DynamoDB / Cassandra
│
├─ Time-series → TimescaleDB / ClickHouse / InfluxDB
├─ Graph → Neo4j / Apache AGE (Postgres extension)
└─ Vector (AI embeddings) → pgvector / Pinecone / Qdrant

Default: Start with PostgreSQL. It handles 80% of use cases.

Caching Strategy

Message Queue / Event Streaming

Hosting / Deployment

---

Phase 4: Decision Documentation

ADR (Architecture Decision Record) Template

# ADR-{NNN}: {Title}

## Status: Proposed | Accepted | Deprecated | Superseded by ADR-{NNN}

## Context
What problem are we solving? What forces are at play?

## Decision
What did we choose and why?

## Evaluation
| Criterion | Weight | Chosen | Runner-up |
|-----------|--------|--------|-----------|

## Consequences
- Positive: ...
- Negative: ...
- Risks: ...

## Alternatives Rejected
- Option B: rejected because...
- Option C: rejected because...

---

Common Stack Templates

A: Startup / MVP (Speed)

B: SaaS / Business App (Balance)

C: High-Performance (Scale)

D: AI / ML Application

---

Anti-Patterns

---

Common Issues

Issue 1: "Team can't agree on a framework"

Fix: Time-box to 3 days. Fill the evaluation matrix. If scores within 10%, pick what the majority knows. Document in ADR. Move on.

Issue 2: "We picked X but it doesn't fit"

Fix: Sunk cost fallacy check. If < 2 weeks invested, switch now. If > 2 weeks, document pain points and plan phased migration.

Issue 3: "Do we need microservices?"

Fix: Almost certainly no. Start with a well-structured monolith. Extract to services only when: (a) different scaling needs, (b) different team ownership, (c) different deployment cadence.

Backend Testing Strategy

Comprehensive testing guide for backend and full-stack applications. Covers the full testing pyramid with deep focus on API integration tests, database testing, contract testing, and performance testing.

Quick Start Checklist

• [ ] Test runner configured (Jest/Vitest, Pytest, Go test)
• [ ] Test database ready (Docker container or in-memory)
• [ ] Database isolation per test (transaction rollback or truncation)
• [ ] Test factories for common entities (user, order, product)
• [ ] Auth helper to generate tokens for tests
• [ ] CI pipeline runs tests with real database service
• [ ] Coverage threshold enforced (≥ 80%)

---

The Testing Pyramid

         ╱╲        E2E (few, slow) — full flows across services
        ╱  ╲
       ╱────╲       Integration (moderate) — API + DB + external
      ╱      ╲
     ╱────────╲      Unit (many, fast) — pure business logic
    ╱__________╲

---

1. API Integration Testing (CRITICAL)

What to Test for Every Endpoint

TypeScript (Jest + Supertest)

describe('POST /api/orders', () => {
  let token: string;
  let product: Product;

  beforeAll(async () => {
    await resetDatabase();
    const user = await createTestUser({ role: 'customer' });
    token = await getAuthToken(user);
    product = await createTestProduct({ price: 29.99, stock: 10 });
  });

  it('creates order → 201 + correct DB state', async () => {
    const res = await request(app)
      .post('/api/orders')
      .set('Authorization', `Bearer ${token}`)
      .send({ items: [{ productId: product.id, quantity: 2 }] });

    expect(res.status).toBe(201);
    expect(res.body.data.total).toBe(59.98);

    const updated = await db.product.findUnique({ where: { id: product.id } });
    expect(updated!.stock).toBe(8);
  });

  it('rejects without auth → 401', async () => {
    const res = await request(app).post('/api/orders').send({ items: [] });
    expect(res.status).toBe(401);
  });

  it('rejects empty items → 422', async () => {
    const res = await request(app)
      .post('/api/orders')
      .set('Authorization', `Bearer ${token}`)
      .send({ items: [] });
    expect(res.status).toBe(422);
    expect(res.body.errors[0].field).toBe('items');
  });
});

Python (Pytest + FastAPI TestClient)

@pytest.fixture
def client(db_session):
    def override_get_db():
        yield db_session
    app.dependency_overrides[get_db] = override_get_db
    yield TestClient(app)
    app.dependency_overrides.clear()

def test_create_order_success(client, auth_headers, test_product):
    response = client.post("/api/orders", json={
        "items": [{"product_id": test_product.id, "quantity": 2}]
    }, headers=auth_headers)
    assert response.status_code == 201
    assert response.json()["data"]["total"] == 59.98

def test_create_order_no_auth(client):
    response = client.post("/api/orders", json={"items": []})
    assert response.status_code == 401

def test_create_order_empty_items(client, auth_headers):
    response = client.post("/api/orders", json={"items": []}, headers=auth_headers)
    assert response.status_code == 422

---

2. Database Testing (HIGH)

Test Isolation Strategies

Transaction rollback (recommended default):

let tx: Transaction;
beforeEach(async () => { tx = await db.beginTransaction(); });
afterEach(async () => { await tx.rollback(); });

Docker test containers (CI):

# docker-compose.test.yml
services:
  test-db:
    image: postgres:16-alpine
    tmpfs: /var/lib/postgresql/data   # RAM disk for speed
    environment:
      POSTGRES_DB: myapp_test

Test Factories (Not Raw SQL)

// factories/user.factory.ts
import { faker } from '@faker-js/faker';

export function buildUser(overrides: Partial<User> = {}): CreateUserDTO {
  return {
    email: faker.internet.email(),
    firstName: faker.person.firstName(),
    role: 'customer',
    ...overrides,
  };
}
export async function createUser(overrides = {}) {
  return db.user.create({ data: buildUser(overrides) });
}

# factories/user_factory.py
import factory
from faker import Faker

class UserFactory(factory.Factory):
    class Meta:
        model = User
    email = factory.LazyAttribute(lambda _: Faker().email())
    first_name = factory.LazyAttribute(lambda _: Faker().first_name())
    role = "customer"

---

3. External Service Testing (HIGH)

HTTP-Level Mocking (Not Function Mocking)

TypeScript (nock):

import nock from 'nock';

it('processes payment successfully', async () => {
  nock('https://api.stripe.com')
    .post('/v1/charges')
    .reply(200, { id: 'ch_123', status: 'succeeded', amount: 5000 });

  const result = await paymentService.charge({ amount: 50.00, currency: 'usd' });
  expect(result.status).toBe('succeeded');
});

it('handles payment timeout', async () => {
  nock('https://api.stripe.com').post('/v1/charges').delay(10000).reply(200);
  await expect(paymentService.charge({ amount: 50, currency: 'usd' }))
    .rejects.toThrow('timeout');
});

Python (responses):

import responses

@responses.activate
def test_payment_success():
    responses.post("https://api.stripe.com/v1/charges",
                   json={"id": "ch_123", "status": "succeeded"}, status=200)
    result = payment_service.charge(amount=50.00, currency="usd")
    assert result.status == "succeeded"

Test Containers for Infrastructure

import { PostgreSqlContainer } from '@testcontainers/postgresql';
import { RedisContainer } from '@testcontainers/redis';

beforeAll(async () => {
  const pg = await new PostgreSqlContainer('postgres:16').start();
  process.env.DATABASE_URL = pg.getConnectionUri();
  await runMigrations();
}, 60000);

---

4. Contract Testing (MEDIUM-HIGH)

Consumer-Driven Contracts (Pact)

Consumer (OrderService calls UserService):

it('can fetch user by ID', async () => {
  await pact.addInteraction()
    .given('user usr_123 exists')
    .uponReceiving('GET /users/usr_123')
    .withRequest('GET', '/api/users/usr_123')
    .willRespondWith(200, (b) => {
      b.jsonBody({ data: { id: MatchersV3.string(), email: MatchersV3.email() } });
    })
    .executeTest(async (mockserver) => {
      const user = await new UserClient(mockserver.url).getUser('usr_123');
      expect(user.id).toBeDefined();
    });
});

Provider verifies in CI:

await new Verifier({
  providerBaseUrl: 'http://localhost:3001',
  pactBrokerUrl: process.env.PACT_BROKER_URL,
  provider: 'UserService',
}).verifyProvider();

---

5. Performance Testing (MEDIUM)

k6 Load Test

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 20 },    // ramp up
    { duration: '1m',  target: 100 },   // sustain
    { duration: '30s', target: 0 },     // ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<500', 'p(99)<1000'],
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  const res = http.get(`${__ENV.BASE_URL}/api/orders`);
  check(res, { 'status 200': (r) => r.status === 200 });
  sleep(1);
}

Performance Budgets

When to Run

---

Test File Organization

tests/
  unit/                      # Pure logic, mocked dependencies
    order.service.test.ts
  integration/               # API + real DB
    orders.api.test.ts
    auth.api.test.ts
  contracts/                 # Consumer-driven contracts
    user-service.consumer.pact.ts
  performance/               # Load tests
    load-test.js
  fixtures/
    factories/               # Test data factories
      user.factory.ts
    seeds/
      test-data.ts
  helpers/
    setup.ts                 # Global test config
    auth.helper.ts           # Token generation
    db.helper.ts             # DB cleanup

---

Anti-Patterns

---

Common Issues

Issue 1: "Tests pass alone but fail together"

Cause: Shared database state between tests. Missing cleanup.

Fix:

beforeEach(async () => { await db.raw('TRUNCATE orders, users CASCADE'); });
// OR use transaction rollback per test

Issue 2: "Jest did not exit one second after test run"

Cause: Unclosed database connections or HTTP servers.

Fix:

afterAll(async () => {
  await db.destroy();
  await server.close();
});

Issue 3: "Async callback was not invoked within timeout"

Cause: Missing async/await or unhandled promise.

Fix:

// ❌ Promise not awaited
it('should work', () => { request(app).get('/users'); });

// ✅ Properly awaited
it('should work', async () => { await request(app).get('/users'); });

Issue 4: "Integration tests too slow in CI"

Fix: 1. Use tmpfs for PostgreSQL data dir (RAM disk) 2. Run migrations once in beforeAll, truncate in beforeEach 3. Parallelize test suites with --maxWorkers 4. Skip performance tests on feature branches (only main)