Api Design Patterns

API Design Patterns

Design robust, scalable APIs using proven patterns for REST, GraphQL, and gRPC with proper versioning, authentication, and error handling.

Quick Reference

API Style Selection:

• REST: Resource-based CRUD, simple clients, HTTP-native caching
• GraphQL: Client-driven queries, complex data graphs, real-time subscriptions
• gRPC: High-performance RPC, microservices, strong typing, streaming

Critical Patterns:

• Versioning: URI (/v1/users), header (Accept: application/vnd.api+json;version=1), content negotiation
• Pagination: Offset (simple), cursor (stable), keyset (performant)
• Auth: OAuth2 (delegated), JWT (stateless), API keys (service-to-service)
• Rate limiting: Token bucket, fixed window, sliding window
• Idempotency: Idempotency keys, conditional requests, safe retry

See references/ for deep dives: rest-patterns.md, graphql-patterns.md, grpc-patterns.md, versioning-strategies.md, authentication.md

Core Principles

Universal API Design Standards

Apply these principles across all API styles:

1. Consistency Over Cleverness

• Follow established conventions for your API style
• Use predictable naming patterns (snake_case or camelCase, pick one)
• Maintain consistent error response formats
• Version breaking changes, never surprise clients

2. Design for Evolution

• Plan for versioning from day one
• Use optional fields with sensible defaults
• Deprecate gracefully with sunset dates
• Document breaking vs non-breaking changes

3. Security by Default

• Require authentication unless explicitly public
• Use HTTPS/TLS for all production endpoints
• Implement rate limiting and throttling
• Validate and sanitize all inputs
• Return minimal error details to clients

4. Developer Experience First

• Provide comprehensive documentation (OpenAPI, GraphQL schema)
• Return meaningful error messages with actionable guidance
• Use standard HTTP status codes correctly
• Include request IDs for debugging
• Offer SDKs and code generators

API Style Decision Tree

When to Choose REST

✅ Use REST when:

• Building CRUD-focused resource APIs
• Clients need HTTP caching (ETags, Cache-Control)
• Wide platform compatibility required (browsers, mobile, IoT)
• Simple, stateless client-server model fits
• Team familiar with HTTP/REST conventions

❌ Avoid REST when:

• Complex data fetching with nested relationships (N+1 queries)
• Real-time updates are primary use case
• Need strong typing and code generation
• High-performance RPC between microservices

Example Use Cases: Public APIs, mobile backends, traditional web services

When to Choose GraphQL

✅ Use GraphQL when:

• Clients need flexible, client-driven queries
• Complex data graphs with nested relationships
• Multiple client types with different data needs
• Real-time subscriptions required
• Strong typing and schema validation needed

❌ Avoid GraphQL when:

• Simple CRUD operations dominate
• HTTP caching is critical (GraphQL uses POST)
• File uploads are primary feature (requires extensions)
• Team lacks GraphQL expertise
• Performance optimization is complex (N+1 problem)

Example Use Cases: Client-facing APIs, dashboards, mobile apps with varied UIs

When to Choose gRPC

✅ Use gRPC when:

• Microservice-to-microservice communication
• High performance and low latency critical
• Bidirectional streaming needed
• Strong typing with Protocol Buffers
• Polyglot environments (language interop)

❌ Avoid gRPC when:

• Browser clients (limited support, needs grpc-web)
• HTTP/JSON required for compatibility
• Human-readable payloads preferred
• Simple request/response patterns

Example Use Cases: Internal microservices, streaming data, service mesh

REST API Patterns

Resource Naming

✅ Good: Plural nouns, hierarchical

GET    /users              # List users
GET    /users/123          # Get user
POST   /users              # Create user
PUT    /users/123          # Update user (full)
PATCH  /users/123          # Update user (partial)
DELETE /users/123          # Delete user
GET    /users/123/orders   # User's orders (sub-resource)

❌ Bad: Verbs, mixed conventions

GET    /getUsers           # Don't use verbs
POST   /user/create        # Don't use verbs
GET    /Users/123          # Don't capitalize
GET    /user/123           # Don't mix singular/plural

HTTP Status Codes

Success Codes:

• 200 OK: Successful GET, PUT, PATCH, DELETE with body
• 201 Created: Successful POST, return Location header
• 202 Accepted: Async operation started
• 204 No Content: Successful DELETE, no body

Client Error Codes:

• 400 Bad Request: Invalid input, validation error
• 401 Unauthorized: Missing or invalid authentication
• 403 Forbidden: Authenticated but insufficient permissions
• 404 Not Found: Resource doesn't exist
• 409 Conflict: State conflict (duplicate, version mismatch)
• 422 Unprocessable Entity: Semantic validation error
• 429 Too Many Requests: Rate limit exceeded

Server Error Codes:

• 500 Internal Server Error: Unexpected error
• 502 Bad Gateway: Upstream service error
• 503 Service Unavailable: Temporary outage
• 504 Gateway Timeout: Upstream timeout

Error Response Format

✅ Consistent error structure

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "Invalid request parameters",
    "details": [
      {
        "field": "email",
        "message": "Invalid email format",
        "code": "INVALID_FORMAT"
      }
    ],
    "request_id": "req_abc123",
    "documentation_url": "https://api.example.com/docs/errors/validation"
  }
}

Pagination Patterns

Offset Pagination (simple, familiar):

GET /users?limit=20&offset=40

✅ Use for: Small datasets, admin interfaces ❌ Avoid for: Large datasets (skips become expensive), real-time data

Cursor Pagination (stable, efficient):

GET /users?limit=20&cursor=eyJpZCI6MTIzfQ
Response: { "data": [...], "next_cursor": "eyJpZCI6MTQzfQ" }

✅ Use for: Infinite scroll, real-time feeds, large datasets ❌ Avoid for: Random access, page numbers

Keyset Pagination (performant):

GET /users?limit=20&after_id=123

✅ Use for: Ordered data, database index friendly ❌ Avoid for: Complex sorting, multiple sort keys

See references/rest-patterns.md for filtering, sorting, field selection, HATEOAS

GraphQL Patterns

Schema Design

✅ Good: Clear types, nullable by default

type User {
  id: ID!                    # Non-null ID
  email: String!             # Required field
  name: String               # Optional (nullable by default)
  createdAt: DateTime!
  orders: [Order!]!          # Non-null array of non-null orders
}

type Query {
  user(id: ID!): User
  users(first: Int, after: String): UserConnection!
}

type Mutation {
  createUser(input: CreateUserInput!): CreateUserPayload!
}

input CreateUserInput {
  email: String!
  name: String
}

type CreateUserPayload {
  user: User
  userEdge: UserEdge
  errors: [UserError!]
}

Resolver Patterns

Avoid N+1 Queries with DataLoader:

import DataLoader from 'dataloader';

const userLoader = new DataLoader(async (userIds: string[]) => {
  const users = await db.users.findMany({ where: { id: { in: userIds } } });
  return userIds.map(id => users.find(u => u.id === id));
});

// Resolver batches queries automatically
const resolvers = {
  Order: {
    user: (order) => userLoader.load(order.userId)
  }
};

Query Complexity Analysis

Prevent expensive queries:

import { createComplexityLimitRule } from 'graphql-validation-complexity';

const server = new ApolloServer({
  schema,
  validationRules: [
    createComplexityLimitRule(1000, {
      onCost: (cost) => console.log('Query cost:', cost),
    }),
  ],
});

See references/graphql-patterns.md for subscriptions, relay cursor connections, error handling

gRPC Patterns

Service Definition

syntax = "proto3";

package users.v1;

service UserService {
  rpc GetUser (GetUserRequest) returns (User) {}
  rpc ListUsers (ListUsersRequest) returns (ListUsersResponse) {}
  rpc CreateUser (CreateUserRequest) returns (User) {}
  rpc StreamUsers (StreamUsersRequest) returns (stream User) {}
  rpc BidiChat (stream ChatMessage) returns (stream ChatMessage) {}
}

message User {
  string id = 1;
  string email = 2;
  string name = 3;
  google.protobuf.Timestamp created_at = 4;
}

message GetUserRequest {
  string id = 1;
}

message ListUsersRequest {
  int32 page_size = 1;
  string page_token = 2;
}

message ListUsersResponse {
  repeated User users = 1;
  string next_page_token = 2;
}

Error Handling

import (
    "google.golang.org/grpc/codes"
    "google.golang.org/grpc/status"
)

func (s *server) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.User, error) {
    if req.Id == "" {
        return nil, status.Error(codes.InvalidArgument, "user ID is required")
    }

    user, err := s.db.GetUser(ctx, req.Id)
    if err != nil {
        if errors.Is(err, sql.ErrNoRows) {
            return nil, status.Error(codes.NotFound, "user not found")
        }
        return nil, status.Error(codes.Internal, "database error")
    }

    return user, nil
}

See references/grpc-patterns.md for streaming, interceptors, metadata, health checks

Versioning Strategies

URI Versioning (Simple, Explicit)

✅ Most common, easy to understand

GET /v1/users/123
GET /v2/users/123

Pros: Clear, easy to route, browser-friendly Cons: Couples version to URL, duplicates routes

Header Versioning (Clean URLs)

GET /users/123
Accept: application/vnd.myapi.v2+json

Pros: Clean URLs, version separate from resource Cons: Less visible, harder to test manually

Content Negotiation (Granular)

GET /users/123
Accept: application/vnd.myapi.user.v2+json

Pros: Resource-level versioning, backward compatible Cons: Complex, harder to implement

Version Deprecation Process

{
  "version": "1.0",
  "deprecated": true,
  "sunset_date": "2025-12-31",
  "migration_guide": "https://docs.api.com/v1-to-v2",
  "replacement_version": "2.0"
}

Include deprecation warnings:

HTTP/1.1 200 OK
Deprecation: true
Sunset: Sat, 31 Dec 2025 23:59:59 GMT
Link: <https://docs.api.com/v1-to-v2>; rel="deprecation"

See references/versioning-strategies.md for detailed migration patterns

Authentication & Authorization

OAuth 2.0 (Delegated Access)

Use for: Third-party access, user consent, token refresh

Authorization Code Flow (most secure for web/mobile):

1. Client redirects to /authorize
2. User authenticates, grants permissions
3. Auth server redirects to callback with code
4. Client exchanges code for access token
5. Client uses access token for API requests

# Request token
POST /oauth/token
Content-Type: application/x-www-form-urlencoded

grant_type=authorization_code
&code=AUTH_CODE
&redirect_uri=https://client.com/callback
&client_id=CLIENT_ID
&client_secret=CLIENT_SECRET

# Response
{
  "access_token": "eyJhbGc...",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "tGzv3JOkF0XG5Qx2TlKWIA",
  "scope": "read write"
}

# Use token
GET /v1/users/me
Authorization: Bearer eyJhbGc...

JWT (Stateless Auth)

Use for: Microservices, stateless API auth, short-lived tokens

✅ Good: Minimal claims, short expiry

{
  "sub": "user_123",
  "iat": 1516239022,
  "exp": 1516242622,
  "scope": "read:users write:orders"
}

Validation:

import jwt from 'jsonwebtoken';

const token = req.headers.authorization?.split(' ')[1];
const payload = jwt.verify(token, process.env.JWT_SECRET);
req.userId = payload.sub;

API Keys (Service-to-Service)

Use for: Server-to-server, CLI tools, webhooks

GET /v1/users
X-API-Key: sk_live_abc123...

# Or query parameter (less secure)
GET /v1/users?api_key=sk_live_abc123

Key Practices:

• Prefix keys with environment (sk_live_, sk_test_)
• Hash keys before storage (bcrypt, scrypt)
• Allow key rotation without downtime
• Support multiple keys per user
• Rate limit per key

See references/authentication.md for API key rotation, scopes, RBAC

Rate Limiting

Token Bucket (Burst-Friendly)

Bucket: 100 tokens, refill 10/second
Request costs 1 token
Allows bursts up to bucket size

Headers:

HTTP/1.1 200 OK
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 73
X-RateLimit-Reset: 1640995200

429 Response:

HTTP/1.1 429 Too Many Requests
Retry-After: 60
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1640995200

{
  "error": {
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "Rate limit exceeded. Try again in 60 seconds.",
    "limit": 100,
    "reset_at": "2025-01-01T00:00:00Z"
  }
}

Sliding Window (Fair Distribution)

Counts requests in rolling time window. More accurate than fixed window.

Per-User vs Per-IP

• Per-User: Authenticated requests, fair quotas
• Per-IP: Unauthenticated requests, prevent abuse
• Combined: Both limits, take stricter

Idempotency

Idempotent Methods (HTTP Spec)

Naturally Idempotent: GET, PUT, DELETE, HEAD, OPTIONS Not Idempotent: POST, PATCH

Idempotency Keys

Make POST requests idempotent:

POST /v1/payments
Idempotency-Key: uuid-or-client-generated-key
Content-Type: application/json

{
  "amount": 1000,
  "currency": "USD",
  "customer": "cust_123"
}

Server behavior: 1. First request: Process and store result with key 2. Duplicate request (same key): Return stored result (200 or 201) 3. Different request (same key): Return 409 Conflict

Implementation:

const idempotencyKey = req.headers['idempotency-key'];
if (idempotencyKey) {
  const cached = await redis.get(`idempotency:${idempotencyKey}`);
  if (cached) {
    return res.status(cached.status).json(cached.body);
  }
}

const result = await processPayment(req.body);
await redis.setex(`idempotency:${idempotencyKey}`, 86400, {
  status: 201,
  body: result
});

Conditional Requests

Use ETags for safe updates:

# Get resource with ETag
GET /v1/users/123
Response: ETag: "abc123"

# Update only if unchanged
PUT /v1/users/123
If-Match: "abc123"

# 412 Precondition Failed if ETag changed

Caching Strategies

HTTP Caching Headers

# Public, cacheable for 1 hour
Cache-Control: public, max-age=3600

# Private (user-specific), revalidate
Cache-Control: private, must-revalidate, max-age=0

# No caching
Cache-Control: no-store, no-cache, must-revalidate

ETag Validation

# Server returns ETag
GET /v1/users/123
Response:
  ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
  Cache-Control: max-age=3600

# Client conditional request
GET /v1/users/123
If-None-Match: "33a64df551425fcc55e4d42a148795d9f25f89d4"

# 304 Not Modified if unchanged (saves bandwidth)
HTTP/1.1 304 Not Modified

Last-Modified

GET /v1/users/123
Response:
  Last-Modified: Wed, 21 Oct 2025 07:28:00 GMT

# Conditional request
GET /v1/users/123
If-Modified-Since: Wed, 21 Oct 2025 07:28:00 GMT

# 304 Not Modified if not modified

Webhooks

Event Delivery

POST https://client.com/webhooks/payments
Content-Type: application/json
X-Webhook-Signature: sha256=abc123...
X-Webhook-Id: evt_abc123
X-Webhook-Timestamp: 1640995200

{
  "id": "evt_abc123",
  "type": "payment.succeeded",
  "created": 1640995200,
  "data": {
    "object": {
      "id": "pay_123",
      "amount": 1000,
      "status": "succeeded"
    }
  }
}

Signature Verification

import crypto from 'crypto';

function verifyWebhookSignature(
  payload: string,
  signature: string,
  secret: string
): boolean {
  const expectedSignature = crypto
    .createHmac('sha256', secret)
    .update(payload)
    .digest('hex');
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(`sha256=${expectedSignature}`)
  );
}

Retry Strategy

• Exponential backoff: 1s, 2s, 4s, 8s, 16s, 32s, 64s
• Timeout: 5-30 seconds per attempt
• Max attempts: 3-7 attempts
• Dead letter queue: Store failed events
• Manual retry: UI for re-sending failed events

API Documentation

OpenAPI/Swagger (REST)

openapi: 3.0.0
info:
  title: User API
  version: 1.0.0
paths:
  /users/{id}:
    get:
      summary: Get user by ID
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/User'
        '404':
          description: User not found
components:
  schemas:
    User:
      type: object
      required: [id, email]
      properties:
        id:
          type: string
        email:
          type: string
          format: email
        name:
          type: string

GraphQL Schema (Self-Documenting)

GraphQL introspection provides automatic documentation. Use descriptions:

"""
Represents a user account in the system.
Created via the createUser mutation.
"""
type User {
  """Unique identifier for the user"""
  id: ID!

  """Email address, must be unique"""
  email: String!

  """Optional display name"""
  name: String
}

API Documentation Best Practices

1. Interactive examples: Provide working code samples 2. Authentication guide: Step-by-step auth setup 3. Error catalog: Document all error codes with examples 4. Rate limits: Clearly state limits and headers 5. Changelog: Track breaking and non-breaking changes 6. Migration guides: Version upgrade instructions 7. SDKs: Provide client libraries for popular languages

Anti-Patterns

❌ Over-fetching (REST): Returning entire objects when fields are unused ✅ Solution: Support field selection (?fields=id,name,email)

❌ Under-fetching (REST): Requiring multiple requests for related data ✅ Solution: Support expansion (?expand=orders,profile) or use GraphQL

❌ Chatty APIs: Too many round-trips for common operations ✅ Solution: Batch endpoints, compound documents, or GraphQL

❌ Ignoring HTTP semantics: Using GET for mutations, wrong status codes ✅ Solution: Follow HTTP spec, use correct methods and status codes

❌ Exposing internal structure: URLs/schemas mirror database ✅ Solution: Design resource-oriented APIs independent of storage

❌ Missing versioning: Breaking changes without version increments ✅ Solution: Version from day one, never break existing versions

❌ Poor error messages: Generic "An error occurred" ✅ Solution: Specific, actionable error messages with codes

❌ No rate limiting: APIs vulnerable to abuse ✅ Solution: Implement rate limiting from the start

Testing Strategies

Contract Testing

// Pact contract test
import { PactV3 } from '@pact-foundation/pact';

const provider = new PactV3({
  consumer: 'FrontendApp',
  provider: 'UserAPI'
});

it('gets a user by ID', () => {
  provider
    .given('user 123 exists')
    .uponReceiving('a request for user 123')
    .withRequest({
      method: 'GET',
      path: '/users/123'
    })
    .willRespondWith({
      status: 200,
      body: { id: '123', email: 'user@example.com' }
    });
});

Load Testing

// k6 load test
import http from 'k6/http';
import { check } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 20 },
    { duration: '1m', target: 20 },
    { duration: '10s', target: 0 }
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'], // 95% under 500ms
    http_req_failed: ['rate<0.01']    // <1% errors
  }
};

export default function () {
  const res = http.get('https://api.example.com/users');
  check(res, {
    'status is 200': (r) => r.status === 200,
    'response time < 500ms': (r) => r.timings.duration < 500
  });
}

Related Skills

• graphql: Deep GraphQL schema design, resolvers, Apollo Server
• typescript: Type-safe API clients and servers
• nodejs-backend: Express/Fastify REST API implementation
• django: Django REST Framework patterns
• fastapi: FastAPI Python REST/GraphQL APIs
• flask: Flask-RESTful patterns

References

• rest-patterns.md: Deep REST coverage (HATEOAS, filtering, field selection)
• graphql-patterns.md: GraphQL subscriptions, relay cursor connections, federation
• grpc-patterns.md: Streaming patterns, interceptors, service mesh integration
• versioning-strategies.md: Detailed versioning approaches and migration patterns
• authentication.md: OAuth flows, JWT best practices, API key rotation, RBAC

Additional Resources

• REST API Design Rulebook - O'Reilly REST guide
• GraphQL Best Practices - Official GraphQL guide
• gRPC Best Practices - Official gRPC guide
• RFC 7807: Problem Details for HTTP APIs - Standard error format
• OpenAPI Specification - REST documentation standard

API Authentication & Authorization - Deep Dive

Comprehensive authentication and authorization patterns for APIs including OAuth 2.0, JWT, API keys, RBAC, and security best practices.

Authentication vs Authorization

Authentication: Who are you? (Identity verification) Authorization: What can you do? (Permission checking)

Authentication → Who is making the request?
Authorization → Is this user allowed to perform this action?

OAuth 2.0

Grant Types

Authorization Code Flow (Most Secure)

Use for: Web applications, mobile apps with backend

1. User clicks "Login" on client
2. Client redirects to /authorize
3. User authenticates and grants permissions
4. Auth server redirects to callback with authorization code
5. Client exchanges code for access token (server-to-server)
6. Client uses access token for API requests

Step-by-step:

# 1. Client redirects to authorization endpoint
https://auth.example.com/oauth/authorize?
  response_type=code&
  client_id=CLIENT_ID&
  redirect_uri=https://client.com/callback&
  scope=read:users write:orders&
  state=random_csrf_token

# 2. User authenticates and approves

# 3. Auth server redirects to callback
https://client.com/callback?
  code=AUTH_CODE&
  state=random_csrf_token

# 4. Client exchanges code for token (server-side)
POST https://auth.example.com/oauth/token
Content-Type: application/x-www-form-urlencoded

grant_type=authorization_code&
code=AUTH_CODE&
redirect_uri=https://client.com/callback&
client_id=CLIENT_ID&
client_secret=CLIENT_SECRET

# 5. Response with access token
{
  "access_token": "eyJhbGc...",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "tGzv3JOkF0XG5Qx2TlKWIA",
  "scope": "read:users write:orders"
}

# 6. Use access token
GET https://api.example.com/users
Authorization: Bearer eyJhbGc...

Authorization Code Flow with PKCE

Use for: Mobile apps, SPAs (no client secret)

PKCE (Proof Key for Code Exchange) prevents authorization code interception.

1. Client generates code_verifier (random string)
2. Client creates code_challenge = SHA256(code_verifier)
3. Client includes code_challenge in /authorize request
4. Auth server stores code_challenge with authorization code
5. Client includes code_verifier in token exchange
6. Auth server verifies SHA256(code_verifier) == code_challenge

Implementation:

import crypto from 'crypto';

// Generate code verifier (43-128 characters)
const codeVerifier = crypto.randomBytes(32).toString('base64url');

// Generate code challenge
const codeChallenge = crypto
  .createHash('sha256')
  .update(codeVerifier)
  .digest('base64url');

// Step 1: Redirect to authorization
const authUrl = new URL('https://auth.example.com/oauth/authorize');
authUrl.searchParams.set('response_type', 'code');
authUrl.searchParams.set('client_id', 'CLIENT_ID');
authUrl.searchParams.set('redirect_uri', 'https://client.com/callback');
authUrl.searchParams.set('code_challenge', codeChallenge);
authUrl.searchParams.set('code_challenge_method', 'S256');
authUrl.searchParams.set('scope', 'read:users');
authUrl.searchParams.set('state', crypto.randomBytes(16).toString('hex'));

// Step 2: Exchange code for token
const response = await fetch('https://auth.example.com/oauth/token', {
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  body: new URLSearchParams({
    grant_type: 'authorization_code',
    code: authCode,
    redirect_uri: 'https://client.com/callback',
    client_id: 'CLIENT_ID',
    code_verifier: codeVerifier, // Send original verifier
  }),
});

Client Credentials Flow

Use for: Server-to-server, microservices, background jobs

POST https://auth.example.com/oauth/token
Content-Type: application/x-www-form-urlencoded

grant_type=client_credentials&
client_id=CLIENT_ID&
client_secret=CLIENT_SECRET&
scope=read:users

# Response
{
  "access_token": "eyJhbGc...",
  "token_type": "Bearer",
  "expires_in": 3600,
  "scope": "read:users"
}

No user context - acts as the application itself.

Implicit Flow (Deprecated)

Do not use: Replaced by Authorization Code + PKCE

# Returns token directly in URL fragment (insecure)
https://client.com/callback#access_token=TOKEN&...

❌ Security issues:

• Token exposed in URL (browser history, referrer)
• No refresh token
• No client authentication

Resource Owner Password Credentials (Avoid)

Use only for: Trusted first-party apps (migration scenarios)

POST https://auth.example.com/oauth/token

grant_type=password&
username=user@example.com&
password=secretpassword&
client_id=CLIENT_ID&
client_secret=CLIENT_SECRET

❌ Avoid because:

• Client handles user password (security risk)
• Doesn't support MFA
• No consent screen
• Use Authorization Code instead

Token Types

Access Token

Purpose: Short-lived token for API access

{
  "access_token": "eyJhbGc...",
  "token_type": "Bearer",
  "expires_in": 3600
}

Characteristics:

• Short expiration (15 min - 1 hour)
• Contains permissions (scopes)
• Can be opaque or JWT
• Sent in Authorization header

Refresh Token

Purpose: Long-lived token to get new access tokens

POST https://auth.example.com/oauth/token

grant_type=refresh_token&
refresh_token=tGzv3JOkF0XG5Qx2TlKWIA&
client_id=CLIENT_ID&
client_secret=CLIENT_SECRET

# Response: New access token
{
  "access_token": "new_access_token",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "new_refresh_token"
}

Characteristics:

• Long expiration (days to months)
• Stored securely (encrypted database)
• Can be revoked
• May rotate on use (refresh token rotation)

Scopes

Define permission boundaries:

read:users         - Read user data
write:users        - Create/update users
delete:users       - Delete users
admin:users        - Full user management

read:orders        - Read orders
write:orders       - Create/update orders

Request specific scopes:

GET /oauth/authorize?scope=read:users write:orders

Check scopes in API:

function requireScope(requiredScope: string) {
  return (req, res, next) => {
    const tokenScopes = req.token.scope.split(' ');

    if (!tokenScopes.includes(requiredScope)) {
      return res.status(403).json({
        error: 'insufficient_scope',
        message: `Requires scope: ${requiredScope}`,
      });
    }

    next();
  };
}

app.get('/users', requireScope('read:users'), async (req, res) => {
  // Handler
});

Implementation (Node.js)

Auth server using oauth2-server:

import OAuth2Server from 'oauth2-server';

const oauth = new OAuth2Server({
  model: {
    // Get client by ID
    getClient: async (clientId, clientSecret) => {
      const client = await db.clients.findUnique({ where: { clientId } });

      if (!client || client.clientSecret !== clientSecret) {
        return null;
      }

      return {
        id: client.id,
        redirectUris: client.redirectUris,
        grants: client.grants,
      };
    },

    // Save authorization code
    saveAuthorizationCode: async (code, client, user) => {
      return db.authorizationCodes.create({
        data: {
          code: code.authorizationCode,
          expiresAt: code.expiresAt,
          redirectUri: code.redirectUri,
          clientId: client.id,
          userId: user.id,
        },
      });
    },

    // Get authorization code
    getAuthorizationCode: async (code) => {
      const authCode = await db.authorizationCodes.findUnique({
        where: { code },
        include: { client: true, user: true },
      });

      return {
        code: authCode.code,
        expiresAt: authCode.expiresAt,
        redirectUri: authCode.redirectUri,
        client: authCode.client,
        user: authCode.user,
      };
    },

    // Revoke authorization code
    revokeAuthorizationCode: async (code) => {
      await db.authorizationCodes.delete({ where: { code: code.code } });
      return true;
    },

    // Save access token
    saveToken: async (token, client, user) => {
      return db.accessTokens.create({
        data: {
          accessToken: token.accessToken,
          accessTokenExpiresAt: token.accessTokenExpiresAt,
          refreshToken: token.refreshToken,
          refreshTokenExpiresAt: token.refreshTokenExpiresAt,
          clientId: client.id,
          userId: user.id,
        },
      });
    },

    // Get access token
    getAccessToken: async (accessToken) => {
      const token = await db.accessTokens.findUnique({
        where: { accessToken },
        include: { client: true, user: true },
      });

      return {
        accessToken: token.accessToken,
        accessTokenExpiresAt: token.accessTokenExpiresAt,
        client: token.client,
        user: token.user,
      };
    },

    // Get refresh token
    getRefreshToken: async (refreshToken) => {
      const token = await db.accessTokens.findUnique({
        where: { refreshToken },
        include: { client: true, user: true },
      });

      return {
        refreshToken: token.refreshToken,
        refreshTokenExpiresAt: token.refreshTokenExpiresAt,
        client: token.client,
        user: token.user,
      };
    },

    // Revoke refresh token
    revokeToken: async (token) => {
      await db.accessTokens.delete({ where: { refreshToken: token.refreshToken } });
      return true;
    },
  },
});

// Endpoints
app.post('/oauth/authorize', async (req, res) => {
  const request = new OAuth2Server.Request(req);
  const response = new OAuth2Server.Response(res);

  try {
    const code = await oauth.authorize(request, response);
    res.redirect(`${code.redirectUri}?code=${code.authorizationCode}&state=${req.query.state}`);
  } catch (err) {
    res.status(err.code || 500).json({ error: err.name, message: err.message });
  }
});

app.post('/oauth/token', async (req, res) => {
  const request = new OAuth2Server.Request(req);
  const response = new OAuth2Server.Response(res);

  try {
    const token = await oauth.token(request, response);
    res.json(token);
  } catch (err) {
    res.status(err.code || 500).json({ error: err.name, message: err.message });
  }
});

JWT (JSON Web Tokens)

Structure

HEADER.PAYLOAD.SIGNATURE

Header:

{
  "alg": "HS256",
  "typ": "JWT"
}

Payload (claims):

{
  "sub": "user_123",
  "iat": 1516239022,
  "exp": 1516242622,
  "iss": "https://auth.example.com",
  "aud": "https://api.example.com",
  "scope": "read:users write:orders"
}

Signature:

HMACSHA256(
  base64UrlEncode(header) + "." + base64UrlEncode(payload),
  secret
)

Standard Claims

• iss (Issuer): Who issued the token
• sub (Subject): User ID
• aud (Audience): Who the token is for
• exp (Expiration): Unix timestamp
• nbf (Not Before): Unix timestamp
• iat (Issued At): Unix timestamp
• jti (JWT ID): Unique token ID

Custom Claims

{
  "sub": "user_123",
  "email": "user@example.com",
  "role": "admin",
  "permissions": ["read:users", "write:users"],
  "org_id": "org_456"
}

Creating JWTs

import jwt from 'jsonwebtoken';

const payload = {
  sub: 'user_123',
  email: 'user@example.com',
  role: 'admin',
};

const token = jwt.sign(payload, process.env.JWT_SECRET, {
  expiresIn: '1h',
  issuer: 'https://auth.example.com',
  audience: 'https://api.example.com',
});

Verifying JWTs

import jwt from 'jsonwebtoken';

function verifyToken(token: string) {
  try {
    const payload = jwt.verify(token, process.env.JWT_SECRET, {
      issuer: 'https://auth.example.com',
      audience: 'https://api.example.com',
    });

    return payload;
  } catch (error) {
    if (error.name === 'TokenExpiredError') {
      throw new Error('Token expired');
    } else if (error.name === 'JsonWebTokenError') {
      throw new Error('Invalid token');
    }
    throw error;
  }
}

// Middleware
app.use((req, res, next) => {
  const authHeader = req.headers.authorization;

  if (!authHeader || !authHeader.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'Missing or invalid authorization header' });
  }

  const token = authHeader.split(' ')[1];

  try {
    req.user = verifyToken(token);
    next();
  } catch (error) {
    return res.status(401).json({ error: error.message });
  }
});

Asymmetric Keys (RS256)

More secure: Private key signs, public key verifies

import fs from 'fs';

const privateKey = fs.readFileSync('private.key');
const publicKey = fs.readFileSync('public.key');

// Sign with private key
const token = jwt.sign(payload, privateKey, {
  algorithm: 'RS256',
  expiresIn: '1h',
});

// Verify with public key
const payload = jwt.verify(token, publicKey, {
  algorithms: ['RS256'],
});

Benefits:

• API servers only need public key (can't create tokens)
• Key rotation easier (distribute public keys)
• More secure (private key never leaves auth server)

JWT Best Practices

✅ Use short expiration: 15 minutes to 1 hour ✅ Use asymmetric keys (RS256): More secure than symmetric (HS256) ✅ Include minimal claims: Tokens sent with every request ✅ Validate all claims: iss, aud, exp, nbf ✅ Use JTI for revocation: Track token IDs in database ✅ Store refresh tokens: Don't extend JWT expiration

❌ Don't store sensitive data: JWTs are not encrypted (only signed) ❌ Don't use long expiration: Hard to revoke ❌ Don't skip validation: Always verify signature and claims ❌ Don't trust client-provided JWTs: Always verify signature

API Keys

Types

Service-to-Service:

sk_live_abc123...
sk_test_xyz789...

User API Keys:

pk_live_user123_abc...
pk_test_user123_xyz...

Key Format

Prefix (identifies environment and type):

• sk_live_: Live secret key
• sk_test_: Test secret key
• pk_live_: Live publishable key
• pk_test_: Test publishable key

Body (random, URL-safe):

import crypto from 'crypto';

function generateAPIKey(prefix: string): string {
  const randomBytes = crypto.randomBytes(32);
  const key = randomBytes.toString('base64url');
  return `${prefix}${key}`;
}

const liveKey = generateAPIKey('sk_live_');
// Example output: sk_live_[random_32_byte_base64url_string]

Storage

Hash keys before storage:

import bcrypt from 'bcrypt';

async function createAPIKey(userId: string): Promise<string> {
  const key = generateAPIKey('sk_live_');
  const hashedKey = await bcrypt.hash(key, 10);

  await db.apiKeys.create({
    data: {
      userId,
      keyHash: hashedKey,
      keyPrefix: key.substring(0, 15), // Store prefix for identification
      createdAt: new Date(),
    },
  });

  // Return unhashed key ONCE (user must store it)
  return key;
}

async function validateAPIKey(key: string): Promise<User | null> {
  const prefix = key.substring(0, 15);

  const apiKey = await db.apiKeys.findFirst({
    where: { keyPrefix: prefix },
    include: { user: true },
  });

  if (!apiKey) {
    return null;
  }

  const isValid = await bcrypt.compare(key, apiKey.keyHash);

  if (!isValid) {
    return null;
  }

  return apiKey.user;
}

Key Rotation

Support multiple active keys:

async function rotateAPIKey(oldKey: string): Promise<string> {
  const user = await validateAPIKey(oldKey);

  if (!user) {
    throw new Error('Invalid API key');
  }

  // Create new key
  const newKey = await createAPIKey(user.id);

  // Mark old key as deprecated (don't delete immediately)
  await db.apiKeys.update({
    where: { keyPrefix: oldKey.substring(0, 15) },
    data: {
      deprecated: true,
      deprecatedAt: new Date(),
      expiresAt: new Date(Date.now() + 30 * 24 * 60 * 60 * 1000), // 30 days grace period
    },
  });

  return newKey;
}

Key Scopes

interface APIKey {
  id: string;
  userId: string;
  keyHash: string;
  scopes: string[];  // ["read:users", "write:orders"]
  expiresAt: Date | null;
}

function requireAPIKeyScope(scope: string) {
  return async (req, res, next) => {
    const apiKey = req.apiKey; // Set by auth middleware

    if (!apiKey.scopes.includes(scope)) {
      return res.status(403).json({
        error: 'insufficient_scope',
        message: `This API key does not have the ${scope} scope`,
      });
    }

    next();
  };
}

app.get('/users', requireAPIKeyScope('read:users'), async (req, res) => {
  // Handler
});

Usage

Query parameter (less secure, convenient for testing):

GET /v1/users?api_key=sk_live_abc123

Header (recommended):

GET /v1/users
Authorization: Bearer sk_live_abc123

Or custom header:

GET /v1/users
X-API-Key: sk_live_abc123

Role-Based Access Control (RBAC)

Roles and Permissions

interface Role {
  id: string;
  name: string;  // "admin", "editor", "viewer"
  permissions: Permission[];
}

interface Permission {
  id: string;
  resource: string;  // "users", "orders", "products"
  action: string;    // "read", "write", "delete"
}

// Example roles
const roles = {
  admin: {
    name: 'admin',
    permissions: [
      { resource: '*', action: '*' },  // All permissions
    ],
  },
  editor: {
    name: 'editor',
    permissions: [
      { resource: 'users', action: 'read' },
      { resource: 'users', action: 'write' },
      { resource: 'posts', action: 'read' },
      { resource: 'posts', action: 'write' },
      { resource: 'posts', action: 'delete' },
    ],
  },
  viewer: {
    name: 'viewer',
    permissions: [
      { resource: 'users', action: 'read' },
      { resource: 'posts', action: 'read' },
    ],
  },
};

Permission Checking

function hasPermission(
  user: User,
  resource: string,
  action: string
): boolean {
  const role = roles[user.role];

  return role.permissions.some(
    (perm) =>
      (perm.resource === '*' || perm.resource === resource) &&
      (perm.action === '*' || perm.action === action)
  );
}

function requirePermission(resource: string, action: string) {
  return (req, res, next) => {
    if (!hasPermission(req.user, resource, action)) {
      return res.status(403).json({
        error: 'forbidden',
        message: `Requires ${action} permission on ${resource}`,
      });
    }

    next();
  };
}

app.delete('/users/:id', requirePermission('users', 'delete'), async (req, res) => {
  // Handler
});

Attribute-Based Access Control (ABAC)

More granular: Check user attributes, resource attributes, context

interface Policy {
  resource: string;
  action: string;
  condition: (context: AccessContext) => boolean;
}

interface AccessContext {
  user: User;
  resource: any;
  environment: {
    ip: string;
    time: Date;
  };
}

const policies: Policy[] = [
  {
    resource: 'users',
    action: 'delete',
    condition: (ctx) =>
      ctx.user.role === 'admin' ||
      (ctx.user.role === 'editor' && ctx.resource.createdBy === ctx.user.id),
  },
  {
    resource: 'posts',
    action: 'write',
    condition: (ctx) =>
      ctx.user.role === 'admin' ||
      ctx.user.role === 'editor' ||
      (ctx.user.role === 'author' && ctx.resource.authorId === ctx.user.id),
  },
];

function checkAccess(context: AccessContext, resource: string, action: string): boolean {
  const policy = policies.find(
    (p) => p.resource === resource && p.action === action
  );

  if (!policy) {
    return false;
  }

  return policy.condition(context);
}

Security Best Practices

HTTPS/TLS Only

// Redirect HTTP to HTTPS
app.use((req, res, next) => {
  if (!req.secure && process.env.NODE_ENV === 'production') {
    return res.redirect(`https://${req.hostname}${req.url}`);
  }
  next();
});

// Strict-Transport-Security header
app.use((req, res, next) => {
  res.setHeader('Strict-Transport-Security', 'max-age=31536000; includeSubDomains');
  next();
});

Rate Limiting Per User

import rateLimit from 'express-rate-limit';

const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // Limit each user to 100 requests per windowMs
  keyGenerator: (req) => req.user?.id || req.ip,
  handler: (req, res) => {
    res.status(429).json({
      error: 'too_many_requests',
      message: 'Rate limit exceeded. Try again later.',
    });
  },
});

app.use('/api/', limiter);

Token Blacklisting

// Blacklist JWT on logout
app.post('/logout', async (req, res) => {
  const token = req.headers.authorization?.split(' ')[1];
  const payload = jwt.decode(token);

  // Store token ID in blacklist until expiration
  await redis.setex(
    `blacklist:${payload.jti}`,
    payload.exp - Math.floor(Date.now() / 1000),
    '1'
  );

  res.json({ message: 'Logged out successfully' });
});

// Check blacklist
app.use(async (req, res, next) => {
  const token = req.headers.authorization?.split(' ')[1];
  const payload = jwt.decode(token);

  const isBlacklisted = await redis.get(`blacklist:${payload.jti}`);

  if (isBlacklisted) {
    return res.status(401).json({ error: 'Token has been revoked' });
  }

  next();
});

IP Whitelisting

const allowedIPs = ['192.168.1.1', '10.0.0.0/8'];

function ipWhitelist(req, res, next) {
  const clientIP = req.ip || req.connection.remoteAddress;

  if (!isIPAllowed(clientIP, allowedIPs)) {
    return res.status(403).json({
      error: 'forbidden',
      message: 'Access denied from this IP address',
    });
  }

  next();
}

app.use('/admin', ipWhitelist);

CORS Configuration

import cors from 'cors';

app.use(cors({
  origin: (origin, callback) => {
    const allowedOrigins = [
      'https://app.example.com',
      'https://dashboard.example.com',
    ];

    if (!origin || allowedOrigins.includes(origin)) {
      callback(null, true);
    } else {
      callback(new Error('Not allowed by CORS'));
    }
  },
  credentials: true,
  maxAge: 86400,
}));

Best Practices Summary

✅ Use OAuth 2.0 for third-party access: Industry standard, secure delegation ✅ Use JWT for stateless auth: Microservices, mobile apps ✅ Use API keys for service-to-service: Simple, revocable ✅ Always use HTTPS: Encrypt all traffic ✅ Hash API keys before storage: bcrypt or scrypt ✅ Use short-lived tokens: 15 min - 1 hour for access tokens ✅ Implement refresh tokens: Avoid long-lived access tokens ✅ Use asymmetric JWT signing: RS256, not HS256 ✅ Validate all JWT claims: iss, aud, exp, nbf ✅ Implement rate limiting: Per user, per IP ✅ Use PKCE for mobile/SPA: Prevents code interception ✅ Support token revocation: Blacklist or database check ✅ Implement RBAC or ABAC: Fine-grained permissions

❌ Don't use Implicit Flow: Use Authorization Code + PKCE ❌ Don't use Resource Owner Password: Use Authorization Code ❌ Don't store passwords in JWT: JWTs are not encrypted ❌ Don't use long-lived JWTs: Hard to revoke ❌ Don't skip signature verification: Always verify JWTs ❌ Don't expose tokens in URLs: Use headers ❌ Don't reuse API keys: Rotate compromised keys ❌ Don't skip HTTPS: Production must use TLS

Additional Resources

• OAuth 2.0 RFC 6749
• OAuth 2.0 Security Best Practices
• JWT RFC 7519
• PKCE RFC 7636
• OAuth 2.0 Playground
• JWT.io Debugger

GraphQL Patterns - Deep Dive

Advanced GraphQL schema design, resolver optimization, subscriptions, federation, and production best practices.

Schema Design Best Practices

Type System Fundamentals

Scalar Types:

type User {
  id: ID!              # Unique identifier
  email: String!       # Required string
  age: Int             # Optional integer
  balance: Float       # Optional float
  isActive: Boolean!   # Required boolean
}

Custom Scalars:

scalar DateTime
scalar URL
scalar EmailAddress
scalar JSON

type Post {
  id: ID!
  publishedAt: DateTime!
  website: URL
  content: JSON
}

Implementation (GraphQL Scalars library):

import { DateTimeResolver, URLResolver, EmailAddressResolver, JSONResolver } from 'graphql-scalars';

const resolvers = {
  DateTime: DateTimeResolver,
  URL: URLResolver,
  EmailAddress: EmailAddressResolver,
  JSON: JSONResolver,
};

Object Types and Interfaces

Interface (shared fields):

interface Node {
  id: ID!
  createdAt: DateTime!
  updatedAt: DateTime!
}

type User implements Node {
  id: ID!
  createdAt: DateTime!
  updatedAt: DateTime!
  email: String!
  name: String
}

type Post implements Node {
  id: ID!
  createdAt: DateTime!
  updatedAt: DateTime!
  title: String!
  content: String!
  author: User!
}

Query interface implementations:

query {
  node(id: "123") {
    id
    ... on User {
      email
      name
    }
    ... on Post {
      title
      author { name }
    }
  }
}

Union Types

union SearchResult = User | Post | Comment

type Query {
  search(query: String!): [SearchResult!]!
}

Query with fragments:

query {
  search(query: "graphql") {
    ... on User {
      id
      name
      email
    }
    ... on Post {
      id
      title
      content
    }
    ... on Comment {
      id
      text
      author { name }
    }
  }
}

Enums

enum UserRole {
  ADMIN
  MODERATOR
  USER
  GUEST
}

enum OrderStatus {
  PENDING
  PROCESSING
  SHIPPED
  DELIVERED
  CANCELLED
}

type User {
  id: ID!
  role: UserRole!
  orders: [Order!]!
}

type Order {
  id: ID!
  status: OrderStatus!
}

Input Types

input CreateUserInput {
  email: String!
  name: String
  role: UserRole = USER  # Default value
}

input UpdateUserInput {
  email: String
  name: String
  role: UserRole
}

input UserFilterInput {
  role: UserRole
  isActive: Boolean
  createdAfter: DateTime
}

type Mutation {
  createUser(input: CreateUserInput!): User!
  updateUser(id: ID!, input: UpdateUserInput!): User!
}

type Query {
  users(filter: UserFilterInput): [User!]!
}

Nullable vs Non-Null Fields

Design philosophy: Nullable by default, non-null where guaranteed

✅ Good: Defensive nullability

type User {
  id: ID!                 # Always present
  email: String!          # Required, validated
  name: String            # Optional (nullable)
  profile: Profile        # May not exist
  posts: [Post!]!         # Array never null, posts never null
  # Can return empty array []
}

❌ Bad: Over-promising with non-null

type User {
  id: ID!
  email: String!
  lastLoginAt: DateTime!  # What if never logged in?
  favoritePost: Post!     # What if no favorite?
  # Breaking change if needs to be nullable later
}

Nullability rules:

• String: Nullable string
• String!: Non-null string
• [String]: Nullable array of nullable strings
• [String!]: Nullable array of non-null strings
• [String!]!: Non-null array of non-null strings
• [String]!: Non-null array of nullable strings

Resolver Patterns

Basic Resolvers

const resolvers = {
  Query: {
    user: async (_parent, { id }, context) => {
      return context.db.users.findUnique({ where: { id } });
    },
    users: async (_parent, { filter }, context) => {
      return context.db.users.findMany({ where: filter });
    },
  },

  User: {
    // Field resolver (computed field)
    fullName: (user) => `${user.firstName} ${user.lastName}`,

    // Async field resolver (database fetch)
    posts: async (user, _args, context) => {
      return context.db.posts.findMany({ where: { authorId: user.id } });
    },
  },

  Mutation: {
    createUser: async (_parent, { input }, context) => {
      return context.db.users.create({ data: input });
    },
  },
};

DataLoader (N+1 Solution)

Problem: N+1 query pattern

// BAD: Triggers separate query for each user's posts
const resolvers = {
  User: {
    posts: async (user, _args, context) => {
      // Called once PER user in result set
      return context.db.posts.findMany({ where: { authorId: user.id } });
    },
  },
};

// Query for 100 users = 1 query + 100 queries for posts = 101 queries!

Solution: DataLoader batches requests

import DataLoader from 'dataloader';

// Create loader in context (per-request)
const createLoaders = (db) => ({
  postsLoader: new DataLoader(async (userIds: string[]) => {
    // Single query for all users
    const posts = await db.posts.findMany({
      where: { authorId: { in: userIds } },
    });

    // Group by userId
    const postsByUser = userIds.map(userId =>
      posts.filter(post => post.authorId === userId)
    );

    return postsByUser;
  }),

  userLoader: new DataLoader(async (userIds: string[]) => {
    const users = await db.users.findMany({
      where: { id: { in: userIds } },
    });

    // Maintain order matching userIds
    return userIds.map(id => users.find(user => user.id === id));
  }),
});

// Context setup
const context = ({ req }) => ({
  db: prisma,
  loaders: createLoaders(prisma),
  userId: req.userId,
});

// Resolver using DataLoader
const resolvers = {
  User: {
    posts: (user, _args, context) => {
      return context.loaders.postsLoader.load(user.id);
    },
  },
  Post: {
    author: (post, _args, context) => {
      return context.loaders.userLoader.load(post.authorId);
    },
  },
};

// Query for 100 users = 1 query + 1 batched query for posts = 2 queries!

Resolver Chain and Parent

const resolvers = {
  Query: {
    user: async (_parent, { id }, context) => {
      // Returns user object passed to User resolvers
      return context.db.users.findUnique({ where: { id } });
    },
  },

  User: {
    // parent is the user object from Query.user
    fullName: (parent) => `${parent.firstName} ${parent.lastName}`,

    // Can access parent fields
    posts: async (parent, _args, context) => {
      return context.db.posts.findMany({
        where: { authorId: parent.id },
      });
    },

    // Nested resolver chain
    profile: async (parent, _args, context) => {
      // Returns profile object passed to Profile resolvers
      return context.db.profiles.findUnique({
        where: { userId: parent.id },
      });
    },
  },

  Profile: {
    // parent is the profile object from User.profile
    avatarUrl: (parent) => {
      return parent.avatar
        ? `https://cdn.example.com/${parent.avatar}`
        : 'https://cdn.example.com/default-avatar.png';
    },
  },
};

Error Handling

import { GraphQLError } from 'graphql';

const resolvers = {
  Query: {
    user: async (_parent, { id }, context) => {
      const user = await context.db.users.findUnique({ where: { id } });

      if (!user) {
        throw new GraphQLError('User not found', {
          extensions: {
            code: 'NOT_FOUND',
            argumentName: 'id',
          },
        });
      }

      return user;
    },
  },

  Mutation: {
    createUser: async (_parent, { input }, context) => {
      // Validation error
      if (!input.email.includes('@')) {
        throw new GraphQLError('Invalid email format', {
          extensions: {
            code: 'INVALID_INPUT',
            field: 'email',
          },
        });
      }

      try {
        return await context.db.users.create({ data: input });
      } catch (error) {
        // Database unique constraint
        if (error.code === 'P2002') {
          throw new GraphQLError('Email already exists', {
            extensions: {
              code: 'DUPLICATE_EMAIL',
              field: 'email',
            },
          });
        }

        // Unexpected error
        throw new GraphQLError('Failed to create user', {
          extensions: {
            code: 'INTERNAL_ERROR',
          },
        });
      }
    },
  },
};

Error response:

{
  "errors": [
    {
      "message": "User not found",
      "locations": [{ "line": 2, "column": 3 }],
      "path": ["user"],
      "extensions": {
        "code": "NOT_FOUND",
        "argumentName": "id"
      }
    }
  ],
  "data": {
    "user": null
  }
}

Pagination Patterns

Offset Pagination (Simple)

type Query {
  users(limit: Int = 10, offset: Int = 0): UserConnection!
}

type UserConnection {
  nodes: [User!]!
  totalCount: Int!
  pageInfo: PageInfo!
}

type PageInfo {
  hasNextPage: Boolean!
  hasPreviousPage: Boolean!
}

Resolver:

const resolvers = {
  Query: {
    users: async (_parent, { limit, offset }, context) => {
      const [nodes, totalCount] = await Promise.all([
        context.db.users.findMany({ take: limit, skip: offset }),
        context.db.users.count(),
      ]);

      return {
        nodes,
        totalCount,
        pageInfo: {
          hasNextPage: offset + limit < totalCount,
          hasPreviousPage: offset > 0,
        },
      };
    },
  },
};

Cursor Pagination (Relay Connection)

Schema:

type Query {
  users(first: Int, after: String, last: Int, before: String): UserConnection!
}

type UserConnection {
  edges: [UserEdge!]!
  nodes: [User!]!
  pageInfo: PageInfo!
  totalCount: Int!
}

type UserEdge {
  cursor: String!
  node: User!
}

type PageInfo {
  hasNextPage: Boolean!
  hasPreviousPage: Boolean!
  startCursor: String
  endCursor: String
}

Resolver (using graphql-relay):

import { connectionFromArraySlice, cursorToOffset } from 'graphql-relay';

const resolvers = {
  Query: {
    users: async (_parent, args, context) => {
      const { first, after, last, before } = args;

      // Decode cursors to offsets
      const afterOffset = after ? cursorToOffset(after) + 1 : 0;
      const beforeOffset = before ? cursorToOffset(before) : undefined;

      // Calculate limit and offset
      const limit = first || last || 10;
      const offset = afterOffset;

      // Fetch data
      const [users, totalCount] = await Promise.all([
        context.db.users.findMany({
          take: limit + 1, // Fetch one extra to check hasNextPage
          skip: offset,
        }),
        context.db.users.count(),
      ]);

      // Build connection
      const hasNextPage = users.length > limit;
      const nodes = hasNextPage ? users.slice(0, -1) : users;

      return connectionFromArraySlice(nodes, args, {
        sliceStart: offset,
        arrayLength: totalCount,
      });
    },
  },
};

Query:

query {
  users(first: 10, after: "cursor123") {
    edges {
      cursor
      node {
        id
        name
      }
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

Mutations

Input Object Pattern

✅ Good: Single input object

input CreatePostInput {
  title: String!
  content: String!
  tags: [String!]
  publishedAt: DateTime
}

type Mutation {
  createPost(input: CreatePostInput!): CreatePostPayload!
}

❌ Bad: Multiple arguments

type Mutation {
  createPost(
    title: String!
    content: String!
    tags: [String!]
    publishedAt: DateTime
  ): Post!
}

Payload Object Pattern

Include user errors and edge for optimistic updates:

type CreatePostPayload {
  post: Post
  postEdge: PostEdge
  errors: [UserError!]
  clientMutationId: String
}

type UserError {
  message: String!
  field: String
  code: String!
}

type PostEdge {
  cursor: String!
  node: Post!
}

type Mutation {
  createPost(input: CreatePostInput!): CreatePostPayload!
}

Resolver:

const resolvers = {
  Mutation: {
    createPost: async (_parent, { input }, context) => {
      // Validation
      const errors = [];
      if (input.title.length < 3) {
        errors.push({
          message: 'Title must be at least 3 characters',
          field: 'title',
          code: 'TITLE_TOO_SHORT',
        });
      }

      if (errors.length > 0) {
        return { post: null, postEdge: null, errors };
      }

      // Create post
      const post = await context.db.posts.create({
        data: {
          ...input,
          authorId: context.userId,
        },
      });

      return {
        post,
        postEdge: {
          cursor: encodeCursor(post.id),
          node: post,
        },
        errors: [],
      };
    },
  },
};

Optimistic Updates (Client)

const [createPost] = useMutation(CREATE_POST, {
  optimisticResponse: {
    createPost: {
      __typename: 'CreatePostPayload',
      post: {
        __typename: 'Post',
        id: 'temp-id',
        title: variables.input.title,
        content: variables.input.content,
        createdAt: new Date().toISOString(),
      },
      errors: [],
    },
  },
  update: (cache, { data }) => {
    // Update cache with new post
    const existing = cache.readQuery({ query: GET_POSTS });
    cache.writeQuery({
      query: GET_POSTS,
      data: {
        posts: {
          ...existing.posts,
          edges: [
            data.createPost.postEdge,
            ...existing.posts.edges,
          ],
        },
      },
    });
  },
});

Subscriptions (Real-Time)

Schema

type Subscription {
  postAdded: Post!
  postUpdated(id: ID!): Post!
  commentAdded(postId: ID!): Comment!
  userStatusChanged(userId: ID!): UserStatus!
}

type UserStatus {
  userId: ID!
  isOnline: Boolean!
  lastSeen: DateTime
}

Resolver (with PubSub)

import { PubSub } from 'graphql-subscriptions';

const pubsub = new PubSub();

const resolvers = {
  Subscription: {
    postAdded: {
      subscribe: () => pubsub.asyncIterator(['POST_ADDED']),
    },

    postUpdated: {
      subscribe: (_parent, { id }) => {
        return pubsub.asyncIterator([`POST_UPDATED_${id}`]);
      },
    },

    commentAdded: {
      subscribe: (_parent, { postId }, context) => {
        // Auth check
        if (!context.userId) {
          throw new GraphQLError('Unauthorized');
        }
        return pubsub.asyncIterator([`COMMENT_ADDED_${postId}`]);
      },
      // Optional resolve function
      resolve: (payload) => payload.comment,
    },
  },

  Mutation: {
    createPost: async (_parent, { input }, context) => {
      const post = await context.db.posts.create({ data: input });

      // Trigger subscription
      pubsub.publish('POST_ADDED', { postAdded: post });

      return { post };
    },

    updatePost: async (_parent, { id, input }, context) => {
      const post = await context.db.posts.update({
        where: { id },
        data: input,
      });

      pubsub.publish(`POST_UPDATED_${id}`, { postUpdated: post });

      return { post };
    },
  },
};

Redis PubSub (Production)

import { RedisPubSub } from 'graphql-redis-subscriptions';
import Redis from 'ioredis';

const options = {
  host: process.env.REDIS_HOST,
  port: process.env.REDIS_PORT,
  retryStrategy: (times) => Math.min(times * 50, 2000),
};

const pubsub = new RedisPubSub({
  publisher: new Redis(options),
  subscriber: new Redis(options),
});

WebSocket Setup (Apollo Server)

import { ApolloServer } from '@apollo/server';
import { expressMiddleware } from '@apollo/server/express4';
import { ApolloServerPluginDrainHttpServer } from '@apollo/server/plugin/drainHttpServer';
import { createServer } from 'http';
import { WebSocketServer } from 'ws';
import { useServer } from 'graphql-ws/lib/use/ws';
import { makeExecutableSchema } from '@graphql-tools/schema';
import express from 'express';

const schema = makeExecutableSchema({ typeDefs, resolvers });
const app = express();
const httpServer = createServer(app);

// WebSocket server
const wsServer = new WebSocketServer({
  server: httpServer,
  path: '/graphql',
});

const serverCleanup = useServer({ schema }, wsServer);

// Apollo Server
const server = new ApolloServer({
  schema,
  plugins: [
    ApolloServerPluginDrainHttpServer({ httpServer }),
    {
      async serverWillStart() {
        return {
          async drainServer() {
            await serverCleanup.dispose();
          },
        };
      },
    },
  ],
});

await server.start();

app.use('/graphql', express.json(), expressMiddleware(server));

httpServer.listen(4000);

Client Subscription (Apollo Client)

import { useSubscription } from '@apollo/client';

const POST_ADDED = gql`
  subscription OnPostAdded {
    postAdded {
      id
      title
      author { name }
    }
  }
`;

function RecentPosts() {
  const { data, loading } = useSubscription(POST_ADDED, {
    onData: ({ client, data }) => {
      // Update cache
      client.cache.modify({
        fields: {
          posts: (existing) => ({
            ...existing,
            edges: [
              { node: data.postAdded, cursor: '' },
              ...existing.edges,
            ],
          }),
        },
      });
    },
  });

  return <div>New post: {data?.postAdded.title}</div>;
}

Directives

Built-in Directives

query GetUser($includeEmail: Boolean!, $skipProfile: Boolean!) {
  user(id: "123") {
    id
    name
    email @include(if: $includeEmail)
    profile @skip(if: $skipProfile) {
      bio
    }
  }
}

Custom Directives

Schema:

directive @auth(requires: UserRole!) on FIELD_DEFINITION
directive @deprecated(reason: String) on FIELD_DEFINITION
directive @length(min: Int, max: Int) on INPUT_FIELD_DEFINITION

type Query {
  users: [User!]! @auth(requires: ADMIN)
  me: User!
}

type User {
  id: ID!
  email: String! @deprecated(reason: "Use contactEmail instead")
  contactEmail: String!
}

input CreateUserInput {
  name: String! @length(min: 3, max: 50)
  email: String!
}

Implementation (using graphql-tools):

import { mapSchema, getDirective, MapperKind } from '@graphql-tools/utils';

function authDirective(schema, directiveName) {
  return mapSchema(schema, {
    [MapperKind.OBJECT_FIELD]: (fieldConfig) => {
      const authDirective = getDirective(schema, fieldConfig, directiveName)?.[0];

      if (authDirective) {
        const { requires } = authDirective;
        const { resolve = defaultFieldResolver } = fieldConfig;

        fieldConfig.resolve = async (source, args, context, info) => {
          if (!context.user || context.user.role !== requires) {
            throw new GraphQLError('Unauthorized', {
              extensions: { code: 'FORBIDDEN' },
            });
          }

          return resolve(source, args, context, info);
        };
      }

      return fieldConfig;
    },
  });
}

let schema = makeExecutableSchema({ typeDefs, resolvers });
schema = authDirective(schema, 'auth');

Performance Optimization

Query Complexity Analysis

import { createComplexityLimitRule } from 'graphql-validation-complexity';

const server = new ApolloServer({
  schema,
  validationRules: [
    createComplexityLimitRule(1000, {
      scalarCost: 1,
      objectCost: 10,
      listFactor: 10,
      introspectionListFactor: 10,
      onCost: (cost) => {
        console.log('Query cost:', cost);
      },
    }),
  ],
});

Custom cost per field:

const typeDefs = gql`
  type Query {
    users: [User!]! @cost(complexity: 100)
    expensiveAnalytics: Analytics! @cost(complexity: 500)
  }
`;

Query Depth Limiting

import depthLimit from 'graphql-depth-limit';

const server = new ApolloServer({
  schema,
  validationRules: [depthLimit(10)],
});

Persisted Queries

Benefits: Reduce payload size, prevent arbitrary queries in production

import { ApolloServer } from '@apollo/server';

const server = new ApolloServer({
  schema,
  persistedQueries: {
    cache: new Map(), // Use Redis in production
  },
  allowBatchedHttpRequests: false,
  introspection: process.env.NODE_ENV !== 'production',
});

Client sends hash:

POST /graphql
{
  "extensions": {
    "persistedQuery": {
      "version": 1,
      "sha256Hash": "abc123..."
    }
  }
}

Response Caching

HTTP caching:

import responseCachePlugin from '@apollo/server-plugin-response-cache';

const server = new ApolloServer({
  schema,
  plugins: [
    responseCachePlugin({
      sessionId: (context) => context.user?.id || null,
    }),
  ],
});

Cache hints:

type Query {
  user(id: ID!): User @cacheControl(maxAge: 60, scope: PRIVATE)
  publicPosts: [Post!]! @cacheControl(maxAge: 300, scope: PUBLIC)
}

Schema Stitching and Federation

Apollo Federation

Service 1 (Users):

type User @key(fields: "id") {
  id: ID!
  email: String!
  name: String
}

extend type Query {
  user(id: ID!): User
}

Service 2 (Posts):

type Post @key(fields: "id") {
  id: ID!
  title: String!
  author: User!
}

extend type User @key(fields: "id") {
  id: ID! @external
  posts: [Post!]!
}

extend type Query {
  posts: [Post!]!
}

Gateway:

import { ApolloGateway, IntrospectAndCompose } from '@apollo/gateway';
import { ApolloServer } from '@apollo/server';

const gateway = new ApolloGateway({
  supergraphSdl: new IntrospectAndCompose({
    subgraphs: [
      { name: 'users', url: 'http://localhost:4001/graphql' },
      { name: 'posts', url: 'http://localhost:4002/graphql' },
    ],
  }),
});

const server = new ApolloServer({ gateway });

Testing

Unit Testing Resolvers

import { resolvers } from './resolvers';

describe('User Resolvers', () => {
  it('fetches user by ID', async () => {
    const mockDb = {
      users: {
        findUnique: jest.fn().mockResolvedValue({
          id: '123',
          email: 'test@example.com',
        }),
      },
    };

    const result = await resolvers.Query.user(
      {},
      { id: '123' },
      { db: mockDb }
    );

    expect(mockDb.users.findUnique).toHaveBeenCalledWith({
      where: { id: '123' },
    });
    expect(result).toEqual({
      id: '123',
      email: 'test@example.com',
    });
  });
});

Integration Testing

import { ApolloServer } from '@apollo/server';

const server = new ApolloServer({ typeDefs, resolvers });

it('creates a user', async () => {
  const response = await server.executeOperation({
    query: `
      mutation CreateUser($input: CreateUserInput!) {
        createUser(input: $input) {
          user {
            id
            email
          }
          errors {
            message
          }
        }
      }
    `,
    variables: {
      input: {
        email: 'test@example.com',
        name: 'Test User',
      },
    },
  });

  expect(response.body.kind).toBe('single');
  expect(response.body.singleResult.errors).toBeUndefined();
  expect(response.body.singleResult.data?.createUser.user).toHaveProperty('id');
});

Best Practices Summary

✅ Nullable by default: Only use non-null (!) when guaranteed ✅ Use DataLoader: Batch queries to prevent N+1 ✅ Pagination: Use cursor-based for large lists ✅ Input objects: Group mutation arguments ✅ Payload objects: Return errors with data ✅ Custom scalars: Use DateTime, Email, URL, JSON ✅ Interfaces: Share common fields across types ✅ Query complexity: Limit expensive queries ✅ Persisted queries: Reduce payload, improve security ✅ Error handling: Return specific error codes and fields

❌ Avoid over-fetching: Let clients request exact fields ❌ Don't expose internal IDs: Use opaque IDs or UUIDs ❌ Don't ignore N+1: Always use DataLoader for relationships ❌ Don't make everything non-null: Breaks schema evolution ❌ Don't use query strings for mutations: Use input objects ❌ Don't skip authorization: Check permissions in resolvers

Additional Resources

• GraphQL Official Documentation
• Apollo Server Documentation
• GraphQL Best Practices
• Relay Cursor Connections Specification
• Apollo Federation
• DataLoader Documentation

gRPC Patterns - Deep Dive

Comprehensive gRPC service design, streaming patterns, error handling, interceptors, and production deployment strategies.

Protocol Buffers (Protobuf)

Basic Message Definition

syntax = "proto3";

package users.v1;

// Import well-known types
import "google/protobuf/timestamp.proto";
import "google/protobuf/empty.proto";

// User message
message User {
  string id = 1;
  string email = 2;
  string name = 3;
  UserRole role = 4;
  google.protobuf.Timestamp created_at = 5;
  google.protobuf.Timestamp updated_at = 6;
}

// Enum for user roles
enum UserRole {
  USER_ROLE_UNSPECIFIED = 0;  // Required first value
  USER_ROLE_USER = 1;
  USER_ROLE_ADMIN = 2;
  USER_ROLE_MODERATOR = 3;
}

Field Numbering Best Practices

✅ Good: Strategic numbering

message User {
  // 1-15: Single-byte encoding (most common fields)
  string id = 1;
  string email = 2;
  string name = 3;

  // 16-2047: Two-byte encoding (less common fields)
  string bio = 16;
  string website = 17;

  // 19000-19999: Reserved range (do not use)
  // 20000+: Multi-byte encoding (rare fields)
}

❌ Bad: Random numbering

message User {
  string id = 100;      // Wastes encoding space
  string email = 3;
  string name = 15000;  // Very inefficient
}

Nested Messages

message User {
  string id = 1;
  string email = 2;
  Profile profile = 3;

  message Profile {
    string bio = 1;
    string avatar_url = 2;
    Address address = 3;

    message Address {
      string street = 1;
      string city = 2;
      string country = 3;
      string postal_code = 4;
    }
  }
}

Repeated Fields (Arrays)

message User {
  string id = 1;
  repeated string tags = 2;           // Array of strings
  repeated Role roles = 3;            // Array of enums
  repeated Address addresses = 4;     // Array of messages
}

Maps

message User {
  string id = 1;
  map<string, string> metadata = 2;        // String map
  map<string, int32> settings = 3;         // Mixed types
  map<string, Address> addresses = 4;      // Complex values
}

Oneofs (Union Types)

message SearchRequest {
  string query = 1;

  oneof filter {
    UserFilter user_filter = 2;
    PostFilter post_filter = 3;
    CommentFilter comment_filter = 4;
  }
}

message UserFilter {
  UserRole role = 1;
  bool is_active = 2;
}

Reserved Fields

message User {
  reserved 4, 5, 6;                    // Reserved field numbers
  reserved "old_field", "deprecated";  // Reserved field names

  string id = 1;
  string email = 2;
  string name = 3;
  // Fields 4-6 cannot be reused
  string new_field = 7;
}

Service Definition

Unary RPC (Request/Response)

service UserService {
  rpc GetUser(GetUserRequest) returns (User) {}
  rpc CreateUser(CreateUserRequest) returns (User) {}
  rpc UpdateUser(UpdateUserRequest) returns (User) {}
  rpc DeleteUser(DeleteUserRequest) returns (google.protobuf.Empty) {}
}

message GetUserRequest {
  string id = 1;
}

message CreateUserRequest {
  string email = 1;
  string name = 2;
  UserRole role = 3;
}

message UpdateUserRequest {
  string id = 1;
  optional string email = 2;
  optional string name = 3;
  optional UserRole role = 4;
}

message DeleteUserRequest {
  string id = 1;
}

Server Streaming RPC

Server sends multiple messages in response to single client request:

service UserService {
  // Stream all users
  rpc ListUsers(ListUsersRequest) returns (stream User) {}

  // Stream user events
  rpc WatchUser(WatchUserRequest) returns (stream UserEvent) {}
}

message ListUsersRequest {
  int32 page_size = 1;
  string page_token = 2;
  UserRole role = 3;
}

message WatchUserRequest {
  string user_id = 1;
}

message UserEvent {
  string event_id = 1;
  EventType type = 2;
  User user = 3;
  google.protobuf.Timestamp timestamp = 4;
}

enum EventType {
  EVENT_TYPE_UNSPECIFIED = 0;
  EVENT_TYPE_CREATED = 1;
  EVENT_TYPE_UPDATED = 2;
  EVENT_TYPE_DELETED = 3;
}

Client Streaming RPC

Client sends multiple messages, server responds once:

service UserService {
  // Bulk create users
  rpc BulkCreateUsers(stream CreateUserRequest) returns (BulkCreateUsersResponse) {}

  // Upload user data
  rpc UploadUserData(stream UserDataChunk) returns (UploadResponse) {}
}

message BulkCreateUsersResponse {
  int32 created_count = 1;
  repeated User users = 2;
  repeated Error errors = 3;
}

message UserDataChunk {
  bytes data = 1;
  int32 chunk_number = 2;
}

message UploadResponse {
  int64 bytes_received = 1;
  string file_id = 2;
}

Bidirectional Streaming RPC

Both client and server send multiple messages:

service ChatService {
  rpc Chat(stream ChatMessage) returns (stream ChatMessage) {}
}

message ChatMessage {
  string id = 1;
  string user_id = 2;
  string text = 3;
  google.protobuf.Timestamp sent_at = 4;
}

Server Implementation

Go Server

package main

import (
    "context"
    "log"
    "net"

    "google.golang.org/grpc"
    "google.golang.org/grpc/codes"
    "google.golang.org/grpc/status"
    pb "myapp/proto/users/v1"
)

type server struct {
    pb.UnimplementedUserServiceServer
    db *Database
}

// Unary RPC
func (s *server) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.User, error) {
    if req.Id == "" {
        return nil, status.Error(codes.InvalidArgument, "user ID is required")
    }

    user, err := s.db.GetUser(ctx, req.Id)
    if err != nil {
        if errors.Is(err, sql.ErrNoRows) {
            return nil, status.Error(codes.NotFound, "user not found")
        }
        return nil, status.Error(codes.Internal, "database error")
    }

    return &pb.User{
        Id:    user.ID,
        Email: user.Email,
        Name:  user.Name,
        Role:  pb.UserRole(user.Role),
    }, nil
}

// Server streaming RPC
func (s *server) ListUsers(req *pb.ListUsersRequest, stream pb.UserService_ListUsersServer) error {
    users, err := s.db.ListUsers(stream.Context(), req)
    if err != nil {
        return status.Error(codes.Internal, "failed to fetch users")
    }

    for _, user := range users {
        if err := stream.Send(&pb.User{
            Id:    user.ID,
            Email: user.Email,
            Name:  user.Name,
        }); err != nil {
            return status.Error(codes.Internal, "failed to send user")
        }
    }

    return nil
}

// Client streaming RPC
func (s *server) BulkCreateUsers(stream pb.UserService_BulkCreateUsersServer) error {
    var users []*pb.User
    var errors []*pb.Error

    for {
        req, err := stream.Recv()
        if err == io.EOF {
            // Client finished sending
            return stream.SendAndClose(&pb.BulkCreateUsersResponse{
                CreatedCount: int32(len(users)),
                Users:        users,
                Errors:       errors,
            })
        }
        if err != nil {
            return status.Error(codes.Internal, "failed to receive request")
        }

        user, err := s.db.CreateUser(stream.Context(), req)
        if err != nil {
            errors = append(errors, &pb.Error{
                Message: err.Error(),
                Field:   "email",
            })
            continue
        }

        users = append(users, user)
    }
}

// Bidirectional streaming RPC
func (s *server) Chat(stream pb.ChatService_ChatServer) error {
    for {
        msg, err := stream.Recv()
        if err == io.EOF {
            return nil
        }
        if err != nil {
            return status.Error(codes.Internal, "failed to receive message")
        }

        // Process message
        response := &pb.ChatMessage{
            Id:     generateID(),
            UserId: "bot",
            Text:   fmt.Sprintf("Echo: %s", msg.Text),
            SentAt: timestamppb.Now(),
        }

        if err := stream.Send(response); err != nil {
            return status.Error(codes.Internal, "failed to send message")
        }
    }
}

func main() {
    lis, err := net.Listen("tcp", ":50051")
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }

    s := grpc.NewServer()
    pb.RegisterUserServiceServer(s, &server{db: newDatabase()})

    log.Printf("server listening at %v", lis.Addr())
    if err := s.Serve(lis); err != nil {
        log.Fatalf("failed to serve: %v", err)
    }
}

Node.js/TypeScript Server

import * as grpc from '@grpc/grpc-js';
import * as protoLoader from '@grpc/proto-loader';
import { UserServiceHandlers } from './proto/users/v1/user_service';

const packageDefinition = protoLoader.loadSync('proto/users/v1/user.proto', {
  keepCase: true,
  longs: String,
  enums: String,
  defaults: true,
  oneofs: true,
});

const userProto = grpc.loadPackageDefinition(packageDefinition).users.v1;

const server = new grpc.Server();

const userService: UserServiceHandlers = {
  // Unary RPC
  getUser: async (call, callback) => {
    const { id } = call.request;

    if (!id) {
      return callback({
        code: grpc.status.INVALID_ARGUMENT,
        message: 'User ID is required',
      });
    }

    try {
      const user = await db.users.findUnique({ where: { id } });

      if (!user) {
        return callback({
          code: grpc.status.NOT_FOUND,
          message: 'User not found',
        });
      }

      callback(null, {
        id: user.id,
        email: user.email,
        name: user.name,
        role: user.role,
      });
    } catch (error) {
      callback({
        code: grpc.status.INTERNAL,
        message: 'Database error',
      });
    }
  },

  // Server streaming RPC
  listUsers: async (call) => {
    const users = await db.users.findMany();

    for (const user of users) {
      call.write({
        id: user.id,
        email: user.email,
        name: user.name,
      });
    }

    call.end();
  },

  // Client streaming RPC
  bulkCreateUsers: async (call, callback) => {
    const users: any[] = [];
    const errors: any[] = [];

    call.on('data', async (request) => {
      try {
        const user = await db.users.create({ data: request });
        users.push(user);
      } catch (error) {
        errors.push({ message: error.message, field: 'email' });
      }
    });

    call.on('end', () => {
      callback(null, {
        created_count: users.length,
        users,
        errors,
      });
    });

    call.on('error', (error) => {
      callback({
        code: grpc.status.INTERNAL,
        message: error.message,
      });
    });
  },

  // Bidirectional streaming RPC
  chat: (call) => {
    call.on('data', (message) => {
      // Echo message back
      call.write({
        id: generateId(),
        user_id: 'bot',
        text: `Echo: ${message.text}`,
        sent_at: new Date(),
      });
    });

    call.on('end', () => {
      call.end();
    });
  },
};

server.addService(userProto.UserService.service, userService);

server.bindAsync(
  '0.0.0.0:50051',
  grpc.ServerCredentials.createInsecure(),
  (err, port) => {
    if (err) {
      console.error('Failed to bind server:', err);
      return;
    }
    console.log(`Server running on port ${port}`);
    server.start();
  }
);

Client Implementation

Go Client

package main

import (
    "context"
    "log"
    "time"

    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials/insecure"
    pb "myapp/proto/users/v1"
)

func main() {
    conn, err := grpc.Dial("localhost:50051", grpc.WithTransportCredentials(insecure.NewCredentials()))
    if err != nil {
        log.Fatalf("did not connect: %v", err)
    }
    defer conn.Close()

    client := pb.NewUserServiceClient(conn)
    ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
    defer cancel()

    // Unary call
    user, err := client.GetUser(ctx, &pb.GetUserRequest{Id: "123"})
    if err != nil {
        log.Fatalf("could not get user: %v", err)
    }
    log.Printf("User: %v", user)

    // Server streaming call
    stream, err := client.ListUsers(ctx, &pb.ListUsersRequest{PageSize: 10})
    if err != nil {
        log.Fatalf("could not list users: %v", err)
    }

    for {
        user, err := stream.Recv()
        if err == io.EOF {
            break
        }
        if err != nil {
            log.Fatalf("error receiving: %v", err)
        }
        log.Printf("User: %v", user)
    }

    // Client streaming call
    bulkStream, err := client.BulkCreateUsers(ctx)
    if err != nil {
        log.Fatalf("could not create bulk stream: %v", err)
    }

    users := []*pb.CreateUserRequest{
        {Email: "alice@example.com", Name: "Alice"},
        {Email: "bob@example.com", Name: "Bob"},
    }

    for _, req := range users {
        if err := bulkStream.Send(req); err != nil {
            log.Fatalf("failed to send: %v", err)
        }
    }

    response, err := bulkStream.CloseAndRecv()
    if err != nil {
        log.Fatalf("failed to receive response: %v", err)
    }
    log.Printf("Created %d users", response.CreatedCount)
}

TypeScript Client

import * as grpc from '@grpc/grpc-js';
import * as protoLoader from '@grpc/proto-loader';

const packageDefinition = protoLoader.loadSync('proto/users/v1/user.proto');
const userProto = grpc.loadPackageDefinition(packageDefinition).users.v1;

const client = new userProto.UserService(
  'localhost:50051',
  grpc.credentials.createInsecure()
);

// Unary call
client.getUser({ id: '123' }, (error, response) => {
  if (error) {
    console.error('Error:', error);
    return;
  }
  console.log('User:', response);
});

// Promise wrapper for unary calls
function getUserAsync(id: string): Promise<any> {
  return new Promise((resolve, reject) => {
    client.getUser({ id }, (error, response) => {
      if (error) reject(error);
      else resolve(response);
    });
  });
}

// Server streaming call
const stream = client.listUsers({ page_size: 10 });

stream.on('data', (user) => {
  console.log('User:', user);
});

stream.on('end', () => {
  console.log('Stream ended');
});

stream.on('error', (error) => {
  console.error('Stream error:', error);
});

// Client streaming call
const bulkStream = client.bulkCreateUsers((error, response) => {
  if (error) {
    console.error('Error:', error);
    return;
  }
  console.log(`Created ${response.created_count} users`);
});

bulkStream.write({ email: 'alice@example.com', name: 'Alice' });
bulkStream.write({ email: 'bob@example.com', name: 'Bob' });
bulkStream.end();

Error Handling

gRPC Status Codes

import "google.golang.org/grpc/codes"

// codes.OK                 - Success
// codes.Canceled           - Operation canceled
// codes.Unknown            - Unknown error
// codes.InvalidArgument    - Invalid client input
// codes.DeadlineExceeded   - Timeout
// codes.NotFound           - Resource not found
// codes.AlreadyExists      - Resource already exists
// codes.PermissionDenied   - No permission
// codes.ResourceExhausted  - Rate limit, quota
// codes.FailedPrecondition - System state invalid
// codes.Aborted            - Concurrency conflict
// codes.OutOfRange         - Out of valid range
// codes.Unimplemented      - Not implemented
// codes.Internal           - Internal server error
// codes.Unavailable        - Service unavailable
// codes.DataLoss           - Data corruption
// codes.Unauthenticated    - Invalid credentials

Rich Error Details

import (
    "google.golang.org/genproto/googleapis/rpc/errdetails"
    "google.golang.org/grpc/status"
)

func (s *server) CreateUser(ctx context.Context, req *pb.CreateUserRequest) (*pb.User, error) {
    // Validation errors
    if req.Email == "" || !strings.Contains(req.Email, "@") {
        st := status.New(codes.InvalidArgument, "invalid email")

        br := &errdetails.BadRequest{
            FieldViolations: []*errdetails.BadRequest_FieldViolation{
                {
                    Field:       "email",
                    Description: "email must be valid format",
                },
            },
        }

        st, _ = st.WithDetails(br)
        return nil, st.Err()
    }

    // Quota/rate limit
    if !s.checkQuota(ctx) {
        st := status.New(codes.ResourceExhausted, "quota exceeded")

        qi := &errdetails.QuotaFailure{
            Violations: []*errdetails.QuotaFailure_Violation{
                {
                    Subject:     "user:" + getUserID(ctx),
                    Description: "API quota exceeded. Try again in 60 seconds",
                },
            },
        }

        st, _ = st.WithDetails(qi)
        return nil, st.Err()
    }

    return user, nil
}

Client error handling:

user, err := client.GetUser(ctx, req)
if err != nil {
    st := status.Convert(err)
    log.Printf("Error code: %s", st.Code())
    log.Printf("Error message: %s", st.Message())

    for _, detail := range st.Details() {
        switch t := detail.(type) {
        case *errdetails.BadRequest:
            for _, violation := range t.FieldViolations {
                log.Printf("Field %s: %s", violation.Field, violation.Description)
            }
        case *errdetails.QuotaFailure:
            for _, violation := range t.Violations {
                log.Printf("Quota: %s", violation.Description)
            }
        }
    }
}

Interceptors (Middleware)

Server Interceptor (Go)

import (
    "context"
    "log"
    "time"

    "google.golang.org/grpc"
)

// Unary interceptor
func loggingInterceptor(
    ctx context.Context,
    req interface{},
    info *grpc.UnaryServerInfo,
    handler grpc.UnaryHandler,
) (interface{}, error) {
    start := time.Now()

    // Call handler
    resp, err := handler(ctx, req)

    duration := time.Since(start)
    log.Printf("Method: %s, Duration: %v, Error: %v", info.FullMethod, duration, err)

    return resp, err
}

// Stream interceptor
func streamLoggingInterceptor(
    srv interface{},
    ss grpc.ServerStream,
    info *grpc.StreamServerInfo,
    handler grpc.StreamHandler,
) error {
    start := time.Now()

    err := handler(srv, ss)

    duration := time.Since(start)
    log.Printf("Stream: %s, Duration: %v, Error: %v", info.FullMethod, duration, err)

    return err
}

// Register interceptors
s := grpc.NewServer(
    grpc.UnaryInterceptor(loggingInterceptor),
    grpc.StreamInterceptor(streamLoggingInterceptor),
)

Authentication Interceptor

func authInterceptor(
    ctx context.Context,
    req interface{},
    info *grpc.UnaryServerInfo,
    handler grpc.UnaryHandler,
) (interface{}, error) {
    // Extract metadata
    md, ok := metadata.FromIncomingContext(ctx)
    if !ok {
        return nil, status.Error(codes.Unauthenticated, "missing metadata")
    }

    // Check authorization header
    tokens := md["authorization"]
    if len(tokens) == 0 {
        return nil, status.Error(codes.Unauthenticated, "missing token")
    }

    token := tokens[0]
    userID, err := validateToken(token)
    if err != nil {
        return nil, status.Error(codes.Unauthenticated, "invalid token")
    }

    // Add user to context
    ctx = context.WithValue(ctx, "userID", userID)

    return handler(ctx, req)
}

Client Interceptor (Go)

func clientLoggingInterceptor(
    ctx context.Context,
    method string,
    req, reply interface{},
    cc *grpc.ClientConn,
    invoker grpc.UnaryInvoker,
    opts ...grpc.CallOption,
) error {
    start := time.Now()

    err := invoker(ctx, method, req, reply, cc, opts...)

    log.Printf("Method: %s, Duration: %v", method, time.Since(start))

    return err
}

// Use interceptor
conn, err := grpc.Dial(
    "localhost:50051",
    grpc.WithUnaryInterceptor(clientLoggingInterceptor),
)

Metadata (Headers)

Server: Read Metadata

import "google.golang.org/grpc/metadata"

func (s *server) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.User, error) {
    md, ok := metadata.FromIncomingContext(ctx)
    if !ok {
        return nil, status.Error(codes.InvalidArgument, "missing metadata")
    }

    // Get header values
    tokens := md["authorization"]
    userAgent := md["user-agent"]

    log.Printf("Authorization: %v", tokens)
    log.Printf("User-Agent: %v", userAgent)

    return user, nil
}

Server: Send Metadata

func (s *server) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.User, error) {
    // Send header
    header := metadata.Pairs("x-request-id", generateID())
    grpc.SendHeader(ctx, header)

    // Send trailer
    trailer := metadata.Pairs("x-response-time", "123ms")
    grpc.SetTrailer(ctx, trailer)

    return user, nil
}

Client: Send Metadata

func main() {
    ctx := context.Background()

    // Add metadata to context
    md := metadata.New(map[string]string{
        "authorization": "Bearer token123",
        "x-request-id":  generateID(),
    })
    ctx = metadata.NewOutgoingContext(ctx, md)

    // Make call with metadata
    user, err := client.GetUser(ctx, &pb.GetUserRequest{Id: "123"})
}

Client: Receive Metadata

var header, trailer metadata.MD

user, err := client.GetUser(
    ctx,
    req,
    grpc.Header(&header),
    grpc.Trailer(&trailer),
)

if err == nil {
    log.Printf("Header: %v", header)
    log.Printf("Trailer: %v", trailer)
}

Performance Optimization

Connection Pooling

// Client-side connection pool
var (
    conn     *grpc.ClientConn
    connOnce sync.Once
)

func getConnection() *grpc.ClientConn {
    connOnce.Do(func() {
        var err error
        conn, err = grpc.Dial(
            "localhost:50051",
            grpc.WithTransportCredentials(insecure.NewCredentials()),
            grpc.WithDefaultCallOptions(
                grpc.MaxCallRecvMsgSize(10*1024*1024), // 10MB
                grpc.MaxCallSendMsgSize(10*1024*1024),
            ),
        )
        if err != nil {
            log.Fatalf("Failed to dial: %v", err)
        }
    })
    return conn
}

Keep-Alive Settings

// Server keep-alive
s := grpc.NewServer(
    grpc.KeepaliveParams(keepalive.ServerParameters{
        Time:    10 * time.Second, // Ping every 10s if no activity
        Timeout: 3 * time.Second,  // Wait 3s for pong
    }),
    grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
        MinTime:             5 * time.Second, // Min time between pings
        PermitWithoutStream: true,            // Allow pings when no streams
    }),
)

// Client keep-alive
conn, err := grpc.Dial(
    "localhost:50051",
    grpc.WithKeepaliveParams(keepalive.ClientParameters{
        Time:                10 * time.Second,
        Timeout:             3 * time.Second,
        PermitWithoutStream: true,
    }),
)

Compression

// Enable gzip compression
conn, err := grpc.Dial(
    "localhost:50051",
    grpc.WithDefaultCallOptions(grpc.UseCompressor("gzip")),
)

// Per-call compression
user, err := client.GetUser(
    ctx,
    req,
    grpc.UseCompressor("gzip"),
)

TLS/SSL Security

Server TLS

import "google.golang.org/grpc/credentials"

creds, err := credentials.NewServerTLSFromFile("server.crt", "server.key")
if err != nil {
    log.Fatalf("Failed to load TLS: %v", err)
}

s := grpc.NewServer(grpc.Creds(creds))

Client TLS

creds, err := credentials.NewClientTLSFromFile("ca.crt", "")
if err != nil {
    log.Fatalf("Failed to load TLS: %v", err)
}

conn, err := grpc.Dial(
    "localhost:50051",
    grpc.WithTransportCredentials(creds),
)

Mutual TLS (mTLS)

// Server
cert, err := tls.LoadX509KeyPair("server.crt", "server.key")
certPool := x509.NewCertPool()
ca, _ := ioutil.ReadFile("ca.crt")
certPool.AppendCertsFromPEM(ca)

creds := credentials.NewTLS(&tls.Config{
    ClientAuth:   tls.RequireAndVerifyClientCert,
    Certificates: []tls.Certificate{cert},
    ClientCAs:    certPool,
})

s := grpc.NewServer(grpc.Creds(creds))

Health Checking

syntax = "proto3";

package grpc.health.v1;

service Health {
  rpc Check(HealthCheckRequest) returns (HealthCheckResponse);
  rpc Watch(HealthCheckRequest) returns (stream HealthCheckResponse);
}

message HealthCheckRequest {
  string service = 1;
}

message HealthCheckResponse {
  enum ServingStatus {
    UNKNOWN = 0;
    SERVING = 1;
    NOT_SERVING = 2;
    SERVICE_UNKNOWN = 3;
  }
  ServingStatus status = 1;
}

Implementation:

import "google.golang.org/grpc/health/grpc_health_v1"

healthServer := health.NewServer()
healthServer.SetServingStatus("users.v1.UserService", grpc_health_v1.HealthCheckResponse_SERVING)

grpc_health_v1.RegisterHealthServer(s, healthServer)

Testing

Unit Testing

import (
    "testing"

    "google.golang.org/grpc/codes"
    "google.golang.org/grpc/status"
)

func TestGetUser(t *testing.T) {
    mockDB := &MockDatabase{
        users: map[string]*User{
            "123": {ID: "123", Email: "test@example.com"},
        },
    }

    srv := &server{db: mockDB}

    user, err := srv.GetUser(context.Background(), &pb.GetUserRequest{Id: "123"})

    if err != nil {
        t.Fatalf("unexpected error: %v", err)
    }

    if user.Email != "test@example.com" {
        t.Errorf("expected test@example.com, got %s", user.Email)
    }

    // Test not found
    _, err = srv.GetUser(context.Background(), &pb.GetUserRequest{Id: "999"})
    if status.Code(err) != codes.NotFound {
        t.Errorf("expected NotFound, got %v", status.Code(err))
    }
}

Integration Testing

func TestUserService(t *testing.T) {
    // Start test server
    lis := bufconn.Listen(1024 * 1024)
    s := grpc.NewServer()
    pb.RegisterUserServiceServer(s, &server{db: testDB})

    go func() {
        if err := s.Serve(lis); err != nil {
            log.Fatalf("Server exited with error: %v", err)
        }
    }()
    defer s.Stop()

    // Create test client
    conn, err := grpc.DialContext(
        context.Background(),
        "bufnet",
        grpc.WithContextDialer(func(context.Context, string) (net.Conn, error) {
            return lis.Dial()
        }),
        grpc.WithInsecure(),
    )
    if err != nil {
        t.Fatalf("Failed to dial: %v", err)
    }
    defer conn.Close()

    client := pb.NewUserServiceClient(conn)

    // Test GetUser
    user, err := client.GetUser(context.Background(), &pb.GetUserRequest{Id: "123"})
    if err != nil {
        t.Fatalf("GetUser failed: %v", err)
    }

    if user.Id != "123" {
        t.Errorf("expected ID 123, got %s", user.Id)
    }
}

Best Practices Summary

✅ Use proto3: Modern syntax, better performance ✅ Version services: Use package versioning (users.v1) ✅ Reserve field numbers: Protect against breaking changes ✅ Use well-known types: Timestamp, Duration, Empty, Any ✅ Implement health checks: For load balancers, Kubernetes ✅ Enable TLS: Encrypt traffic in production ✅ Add interceptors: Logging, auth, metrics ✅ Use keep-alive: Maintain long-lived connections ✅ Stream large datasets: Avoid large unary responses ✅ Handle errors properly: Use correct status codes with details

❌ Don't reuse field numbers: Reserved fields protect from bugs ❌ Don't use HTTP/JSON for gRPC: Use binary Protobuf ❌ Don't ignore deadlines: Always set request timeouts ❌ Don't skip error details: Provide actionable error info ❌ Don't run without TLS: Production must use encryption ❌ Don't forget connection pooling: Reuse connections

Additional Resources

• gRPC Official Documentation
• Protocol Buffers Guide
• gRPC Go Examples
• gRPC Best Practices
• gRPC Error Handling
• Awesome gRPC

REST API Patterns - Deep Dive

Comprehensive REST API design patterns covering advanced resource modeling, filtering, field selection, HATEOAS, and optimization techniques.

Resource Modeling

Single vs Collection Resources

Collection Resources (plural nouns):

GET    /users              → List all users
POST   /users              → Create new user

Single Resources (with ID):

GET    /users/123          → Get specific user
PUT    /users/123          → Replace user
PATCH  /users/123          → Update user fields
DELETE /users/123          → Delete user

Sub-Resources (Nested Relationships)

✅ Good: Clear hierarchy, logical nesting

GET    /users/123/orders          → User's orders
POST   /users/123/orders          → Create order for user
GET    /users/123/orders/456      → Specific order for user
DELETE /users/123/orders/456      → Cancel user's order

❌ Bad: Excessive nesting

GET /organizations/1/departments/2/teams/3/members/4/tasks/5

✅ Better: Shallow hierarchy, use query params

GET /tasks/5
GET /tasks?member_id=4&team_id=3

Non-CRUD Actions

When operations don't map to CRUD:

Option 1: Treat as sub-resource

POST /orders/123/cancel           → Cancel order
POST /users/123/activate          → Activate user
POST /invoices/456/send           → Send invoice

Option 2: Use controller-style endpoints (less RESTful but pragmatic)

POST /search                      → Complex search
POST /bulk-operations             → Batch operations

Option 3: Use status field updates

PATCH /orders/123
{ "status": "cancelled", "reason": "Customer request" }

HTTP Methods Deep Dive

GET (Safe, Idempotent, Cacheable)

Characteristics:

• No side effects (safe)
• Multiple identical requests = same result (idempotent)
• Should be cached
• No request body

GET /users?status=active&role=admin HTTP/1.1
Host: api.example.com
Accept: application/json
Authorization: Bearer token123

Response:

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: public, max-age=300
ETag: "abc123"

{
  "data": [
    { "id": "1", "name": "Alice", "role": "admin" }
  ],
  "meta": {
    "total": 1,
    "page": 1,
    "per_page": 20
  }
}

POST (Not Safe, Not Idempotent)

Use for:

• Creating resources
• Operations with side effects
• Searches with complex body
• Bulk operations

POST /users HTTP/1.1
Host: api.example.com
Content-Type: application/json

{
  "email": "alice@example.com",
  "name": "Alice",
  "role": "admin"
}

Response:

HTTP/1.1 201 Created
Location: /users/123
Content-Type: application/json

{
  "id": "123",
  "email": "alice@example.com",
  "name": "Alice",
  "role": "admin",
  "created_at": "2025-01-01T00:00:00Z"
}

PUT (Idempotent, Full Replace)

Characteristics:

• Replaces entire resource
• Must include all fields
• Idempotent (same request multiple times = same result)
• Creates if doesn't exist (optional)

PUT /users/123 HTTP/1.1
Content-Type: application/json
If-Match: "abc123"

{
  "email": "alice@example.com",
  "name": "Alice Smith",
  "role": "admin",
  "department": "engineering"
}

PATCH (Idempotent, Partial Update)

Use for: Updating specific fields without replacing entire resource

PATCH /users/123 HTTP/1.1
Content-Type: application/json

{
  "name": "Alice Smith"
}

JSON Patch (RFC 6902) - more expressive:

PATCH /users/123 HTTP/1.1
Content-Type: application/json-patch+json

[
  { "op": "replace", "path": "/name", "value": "Alice Smith" },
  { "op": "add", "path": "/tags/-", "value": "premium" },
  { "op": "remove", "path": "/temporary_flag" }
]

DELETE (Idempotent)

DELETE /users/123 HTTP/1.1

Response options:

# Option 1: No content
HTTP/1.1 204 No Content

# Option 2: Return deleted resource
HTTP/1.1 200 OK
Content-Type: application/json
{ "id": "123", "deleted_at": "2025-01-01T00:00:00Z" }

# Option 3: Already deleted (still success)
HTTP/1.1 204 No Content

HEAD (Metadata Only)

Same as GET but no response body:

HEAD /users/123 HTTP/1.1

HTTP/1.1 200 OK
Content-Length: 256
Last-Modified: Wed, 21 Oct 2025 07:28:00 GMT
ETag: "abc123"

OPTIONS (CORS, API Discovery)

OPTIONS /users HTTP/1.1

HTTP/1.1 204 No Content
Allow: GET, POST, HEAD, OPTIONS
Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS
Access-Control-Allow-Headers: Authorization, Content-Type

Query Parameters

Filtering

# Single filter
GET /users?status=active

# Multiple filters (AND logic)
GET /users?status=active&role=admin&department=engineering

# Range filters
GET /users?created_after=2025-01-01&created_before=2025-12-31

# IN filters
GET /users?id=1,2,3,4
GET /users?status=active,pending

# Pattern matching (use carefully, can be expensive)
GET /users?name_like=alice
GET /users?email_ends_with=@example.com

Advanced filtering with query operators:

GET /users?age[gte]=18&age[lte]=65
GET /products?price[gt]=100&price[lt]=1000
GET /posts?published[eq]=true

Sorting

# Single field
GET /users?sort=created_at

# Descending
GET /users?sort=-created_at

# Multiple fields
GET /users?sort=last_name,first_name
GET /users?sort=-created_at,name

Alternative formats:

GET /users?order_by=created_at&order=desc
GET /users?sort[created_at]=desc&sort[name]=asc

Field Selection (Sparse Fieldsets)

Reduce payload size by requesting only needed fields:

# Select specific fields
GET /users?fields=id,name,email

# Exclude fields
GET /users?fields_exclude=internal_notes,password_hash

# Nested field selection
GET /users?fields=id,name,profile(avatar,bio)

Response:

{
  "data": [
    {
      "id": "123",
      "name": "Alice",
      "email": "alice@example.com"
    }
  ]
}

Expansion (Include Related Resources)

Avoid N+1 queries by including related data:

# Basic expansion
GET /orders/123?expand=customer

# Multiple expansions
GET /orders/123?expand=customer,items

# Nested expansion
GET /orders/123?expand=customer,items.product

# Selective nested fields
GET /orders/123?expand=customer(name,email),items(quantity,price)

Response:

{
  "id": "123",
  "total": 1500,
  "customer": {
    "id": "456",
    "name": "Alice",
    "email": "alice@example.com"
  },
  "items": [
    {
      "id": "789",
      "quantity": 2,
      "price": 750,
      "product": {
        "id": "101",
        "name": "Widget",
        "sku": "WDG-001"
      }
    }
  ]
}

Pagination Patterns

Offset Pagination

Simple and familiar:

GET /users?limit=20&offset=0   # Page 1
GET /users?limit=20&offset=20  # Page 2
GET /users?limit=20&offset=40  # Page 3

Response format:

{
  "data": [...],
  "meta": {
    "total": 1543,
    "limit": 20,
    "offset": 40,
    "page": 3,
    "total_pages": 78
  },
  "links": {
    "first": "/users?limit=20&offset=0",
    "prev": "/users?limit=20&offset=20",
    "next": "/users?limit=20&offset=60",
    "last": "/users?limit=20&offset=1540"
  }
}

Pros:

• Easy to implement
• Supports random access (jump to page 10)
• Familiar to users

Cons:

• Performance degrades with high offsets (database skips rows)
• Inconsistent results if data changes (items shift between pages)
• Not suitable for real-time feeds

Cursor Pagination

Efficient and stable:

GET /users?limit=20                        # First page
GET /users?limit=20&cursor=eyJpZCI6MjB9   # Next page

Response format:

{
  "data": [...],
  "meta": {
    "limit": 20,
    "has_more": true
  },
  "cursors": {
    "before": "eyJpZCI6MX0",
    "after": "eyJpZCI6MjB9"
  },
  "links": {
    "next": "/users?limit=20&cursor=eyJpZCI6MjB9",
    "prev": "/users?limit=20&cursor=eyJpZCI6MX0&direction=prev"
  }
}

Cursor encoding (base64 JSON):

// Encode cursor
const cursor = Buffer.from(JSON.stringify({ id: 20 })).toString('base64url');

// Decode cursor
const decoded = JSON.parse(Buffer.from(cursor, 'base64url').toString());

Pros:

• Consistent results even if data changes
• Efficient for large datasets
• No offset performance penalty

Cons:

• No random access (can't jump to page 10)
• More complex to implement
• Cursor reveals internal structure (encrypt if sensitive)

Keyset Pagination

Database-optimized:

GET /users?limit=20&after_id=123&order=id

SQL implementation:

-- First page
SELECT * FROM users ORDER BY id ASC LIMIT 20;

-- Next page (after_id from last result)
SELECT * FROM users WHERE id > 123 ORDER BY id ASC LIMIT 20;

Pros:

• Most performant (uses database index)
• Simple implementation
• Stable results

Cons:

• Requires ordered, unique field
• No backward pagination (easily)
• Complex with multi-field sorting

Page Number Pagination

User-friendly:

GET /users?page=1&per_page=20
GET /users?page=2&per_page=20

Response:

{
  "data": [...],
  "meta": {
    "current_page": 2,
    "per_page": 20,
    "total": 1543,
    "total_pages": 78,
    "from": 21,
    "to": 40
  },
  "links": {
    "first": "/users?page=1&per_page=20",
    "prev": "/users?page=1&per_page=20",
    "next": "/users?page=3&per_page=20",
    "last": "/users?page=78&per_page=20"
  }
}

Same pros/cons as offset pagination (it's offset in disguise: offset = (page - 1) * per_page).

HATEOAS (Hypermedia)

Hypermedia As The Engine Of Application State: Include links to related actions and resources.

Basic HATEOAS

{
  "id": "123",
  "name": "Alice",
  "email": "alice@example.com",
  "links": {
    "self": "/users/123",
    "orders": "/users/123/orders",
    "edit": "/users/123",
    "delete": "/users/123"
  }
}

HAL (Hypertext Application Language)

{
  "_links": {
    "self": { "href": "/orders/123" },
    "customer": { "href": "/customers/456" },
    "payment": { "href": "/payments/789" }
  },
  "id": "123",
  "total": 1500,
  "status": "shipped",
  "_embedded": {
    "customer": {
      "_links": { "self": { "href": "/customers/456" } },
      "id": "456",
      "name": "Alice"
    }
  }
}

JSON:API

{
  "data": {
    "type": "orders",
    "id": "123",
    "attributes": {
      "total": 1500,
      "status": "shipped"
    },
    "relationships": {
      "customer": {
        "links": {
          "self": "/orders/123/relationships/customer",
          "related": "/orders/123/customer"
        },
        "data": { "type": "customers", "id": "456" }
      }
    },
    "links": {
      "self": "/orders/123"
    }
  },
  "included": [
    {
      "type": "customers",
      "id": "456",
      "attributes": {
        "name": "Alice",
        "email": "alice@example.com"
      }
    }
  ]
}

Batch Operations

Batch Create

POST /users/batch HTTP/1.1
Content-Type: application/json

{
  "items": [
    { "email": "alice@example.com", "name": "Alice" },
    { "email": "bob@example.com", "name": "Bob" }
  ]
}

Response:

{
  "results": [
    {
      "status": 201,
      "id": "123",
      "email": "alice@example.com"
    },
    {
      "status": 201,
      "id": "124",
      "email": "bob@example.com"
    }
  ],
  "summary": {
    "total": 2,
    "succeeded": 2,
    "failed": 0
  }
}

Batch Update

PATCH /users/batch HTTP/1.1
Content-Type: application/json

{
  "updates": [
    { "id": "123", "status": "active" },
    { "id": "124", "status": "inactive" }
  ]
}

Batch Get

GET /users?id=123,124,125 HTTP/1.1

# Or POST for large lists
POST /users/batch/get HTTP/1.1
{ "ids": ["123", "124", "125", ...] }

Async Operations

Long-Running Operations

Pattern: Return 202 Accepted with status URL:

POST /reports/generate HTTP/1.1
{ "type": "annual", "year": 2024 }

HTTP/1.1 202 Accepted
Location: /operations/op_abc123
Content-Type: application/json

{
  "operation_id": "op_abc123",
  "status": "pending",
  "status_url": "/operations/op_abc123",
  "estimated_completion": "2025-01-01T00:05:00Z"
}

Status endpoint:

GET /operations/op_abc123

# While processing
HTTP/1.1 200 OK
{
  "id": "op_abc123",
  "status": "processing",
  "progress": 45,
  "message": "Generating report..."
}

# When complete
HTTP/1.1 303 See Other
Location: /reports/rep_xyz789
{
  "id": "op_abc123",
  "status": "completed",
  "result_url": "/reports/rep_xyz789"
}

Compression

Request compression (rare, large request bodies):

POST /data/import HTTP/1.1
Content-Encoding: gzip
Content-Type: application/json

Response compression (common):

GET /users HTTP/1.1
Accept-Encoding: gzip, deflate, br

HTTP/1.1 200 OK
Content-Encoding: gzip
Content-Type: application/json

Enable compression for responses >1KB. Use Brotli (br) for best compression.

Content Negotiation

# Request JSON
GET /users/123
Accept: application/json

# Request XML
GET /users/123
Accept: application/xml

# Request specific version
GET /users/123
Accept: application/vnd.myapi.v2+json

# Multiple acceptable types (quality values)
GET /users/123
Accept: application/json; q=1.0, application/xml; q=0.8

Conditional Requests

ETags (Strong Validation)

# Get with ETag
GET /users/123
Response: ETag: "abc123"

# Update only if unchanged
PUT /users/123
If-Match: "abc123"
{ "name": "Alice Smith" }

# Success if ETag matches
HTTP/1.1 200 OK

# Failure if ETag changed (concurrent update)
HTTP/1.1 412 Precondition Failed
{
  "error": "Resource was modified by another request",
  "current_etag": "def456"
}

Last-Modified (Weak Validation)

GET /users/123
Response: Last-Modified: Wed, 21 Oct 2025 07:28:00 GMT

PUT /users/123
If-Unmodified-Since: Wed, 21 Oct 2025 07:28:00 GMT

Performance Optimization

HTTP/2 and HTTP/3

• Multiplexing: Multiple requests over single connection
• Server Push: Proactively send resources (use carefully)
• Header compression: HPACK reduces overhead

Enable HTTP/2 in production:

listen 443 ssl http2;

Connection Pooling

Reuse TCP connections:

import http from 'http';

const agent = new http.Agent({
  keepAlive: true,
  maxSockets: 50,
  maxFreeSockets: 10,
  timeout: 60000
});

fetch('https://api.example.com/users', { agent });

Response Streaming

Stream large responses:

app.get('/export', (req, res) => {
  res.setHeader('Content-Type', 'application/json');
  res.write('[');

  const stream = db.users.stream();
  let first = true;

  stream.on('data', (user) => {
    if (!first) res.write(',');
    res.write(JSON.stringify(user));
    first = false;
  });

  stream.on('end', () => {
    res.write(']');
    res.end();
  });
});

Security Headers

HTTP/1.1 200 OK
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Content-Security-Policy: default-src 'self'
X-XSS-Protection: 1; mode=block

CORS (Cross-Origin Resource Sharing)

Preflight request (OPTIONS):

OPTIONS /users HTTP/1.1
Origin: https://example.com
Access-Control-Request-Method: POST
Access-Control-Request-Headers: Content-Type, Authorization

HTTP/1.1 204 No Content
Access-Control-Allow-Origin: https://example.com
Access-Control-Allow-Methods: GET, POST, PUT, DELETE
Access-Control-Allow-Headers: Content-Type, Authorization
Access-Control-Max-Age: 86400

Actual request:

POST /users HTTP/1.1
Origin: https://example.com

HTTP/1.1 201 Created
Access-Control-Allow-Origin: https://example.com
Access-Control-Expose-Headers: Location, X-Request-Id

REST API Testing

Integration Tests

import request from 'supertest';
import { app } from './app';

describe('User API', () => {
  it('creates a user', async () => {
    const response = await request(app)
      .post('/users')
      .send({ email: 'test@example.com', name: 'Test' })
      .expect(201)
      .expect('Content-Type', /json/);

    expect(response.body).toHaveProperty('id');
    expect(response.headers.location).toBe(`/users/${response.body.id}`);
  });

  it('returns 404 for non-existent user', async () => {
    await request(app)
      .get('/users/999')
      .expect(404);
  });

  it('validates email format', async () => {
    const response = await request(app)
      .post('/users')
      .send({ email: 'invalid', name: 'Test' })
      .expect(400);

    expect(response.body.error.details).toContainEqual(
      expect.objectContaining({ field: 'email' })
    );
  });
});

API Schema Validation

import Ajv from 'ajv';
import { openApiSchema } from './openapi.json';

const ajv = new Ajv();
const validate = ajv.compile(openApiSchema.components.schemas.User);

it('response matches OpenAPI schema', async () => {
  const response = await request(app).get('/users/123');
  const valid = validate(response.body);
  expect(valid).toBe(true);
});

Best Practices Summary

✅ Use plural nouns for collections: /users not /user ✅ Use HTTP methods correctly: GET (read), POST (create), PUT/PATCH (update), DELETE (delete) ✅ Return appropriate status codes: 200, 201, 400, 404, 500, etc. ✅ Version your API: /v1/users or header-based ✅ Support pagination: Offset, cursor, or keyset ✅ Include HATEOAS links: Help clients discover actions ✅ Use ETags for caching: Conditional requests (If-Match, If-None-Match) ✅ Compress responses: gzip, Brotli for >1KB ✅ Implement rate limiting: Protect against abuse ✅ Document with OpenAPI: Interactive, machine-readable docs ✅ Test thoroughly: Unit, integration, contract, load tests

❌ Avoid verbs in URLs: /getUser should be GET /users/{id} ❌ Don't ignore HTTP semantics: Use correct methods and status codes ❌ Don't over-nest resources: Keep hierarchy shallow (2-3 levels max) ❌ Don't return entire objects: Support field selection for efficiency ❌ Don't break existing versions: Version breaking changes ❌ Don't expose internal structure: Abstract implementation details ❌ Don't skip error details: Provide actionable error messages

Additional Resources

• RFC 7231: HTTP/1.1 Semantics
• RFC 6902: JSON Patch
• RFC 5988: Web Linking
• JSON:API Specification
• HAL Specification
• OpenAPI 3.1.0 Specification