Performance Optimizer

Performance Optimizer

Overview

Provides a systematic approach to application performance optimization across the full stack. Use when diagnosing slow page loads, high API latency, database bottlenecks, or scaling issues. Not a substitute for application-specific profiling -- always measure before optimizing.

Quick Reference

Performance Budgets

Optimization Phases

Caching Layers

Common Mistakes

Delegation

When working on performance optimization, delegate to:

• frontend-builder -- React-specific performance patterns
• application-security -- Rate limiting and DDoS protection
• ci-cd-architecture -- Build pipeline optimization

References

• Profiling and Measurement -- Chrome DevTools, React Profiler, Node.js/Python profiling, database EXPLAIN
• Database Optimization -- Strategic indexes, N+1 fixes, query optimization, connection pooling
• Caching Strategies -- Redis patterns, HTTP cache headers, CDN configuration
• Frontend Performance -- Bundle analysis, code splitting, resource hints, third-party scripts, mobile performance, React patterns
• Backend Performance -- Serverless optimization, streaming responses, conditional requests, background queues, rate limiting
• Monitoring and Alerting -- APM tools, custom monitoring, dashboards, alert thresholds

Backend Performance

Async Background Processing

// Synchronous (slow response)
app.post('/send-email', async (req, res) => {
  await sendEmail(req.body); // 3 seconds
  res.json({ success: true });
});

// Queue job (fast response)
import { Queue, Worker } from 'bullmq';

const emailQueue = new Queue('emails', {
  connection: { host: 'localhost', port: 6379 },
});

app.post('/send-email', async (req, res) => {
  await emailQueue.add('send', req.body);
  res.json({ success: true, message: 'Email queued' });
});

// Process jobs in background worker
const worker = new Worker(
  'emails',
  async (job) => {
    await sendEmail(job.data);
  },
  { connection: { host: 'localhost', port: 6379 } },
);

API Response Optimization

// 1. Compression
import compression from 'compression';
app.use(compression()); // Gzip responses

// 2. Pagination
app.get('/api/posts', async (req, res) => {
  const page = parseInt(req.query.page) || 1;
  const limit = parseInt(req.query.limit) || 20;

  const posts = await db.posts.findAll({
    offset: (page - 1) * limit,
    limit: limit,
  });

  res.json({
    data: posts,
    pagination: {
      page,
      limit,
      total: await db.posts.count(),
    },
  });
});

// 3. Field filtering (GraphQL-style)
app.get('/api/users/:id', async (req, res) => {
  const fields = req.query.fields?.split(',') || ['id', 'name', 'email'];

  const user = await db.users.findById(req.params.id, {
    attributes: fields,
  });

  res.json(user);
});

Partial Responses (Sparse Fieldsets)

// JSON:API sparse fieldsets
app.get('/api/users/:id', async (req, res) => {
  const fields = req.query['fields[user]']?.split(',');
  const defaultFields = ['id', 'name', 'email'];

  const user = await db.users.findById(req.params.id, {
    attributes: fields || defaultFields,
  });

  res.json({ data: user });
});

// GraphQL-style field selection via query param
// GET /api/posts/123?fields=id,title,author.name
app.get('/api/posts/:id', async (req, res) => {
  const fields = req.query.fields?.split(',') || ['*'];
  const post = await db.posts.findById(req.params.id, {
    select: Object.fromEntries(fields.map((f: string) => [f, true])),
  });

  res.json(post);
});

Conditional Requests (ETag / If-None-Match)

import { createHash } from 'crypto';

function generateETag(data: unknown): string {
  return createHash('md5').update(JSON.stringify(data)).digest('hex');
}

app.get('/api/products/:id', async (req, res) => {
  const product = await db.products.findById(req.params.id);
  const etag = `"${generateETag(product)}"`;

  res.setHeader('ETag', etag);

  if (req.headers['if-none-match'] === etag) {
    return res.status(304).end();
  }

  res.json(product);
});

Streaming Responses

import { Readable } from 'stream';

app.get('/api/export/users', async (req, res) => {
  res.setHeader('Content-Type', 'application/json');
  res.setHeader('Transfer-Encoding', 'chunked');

  res.write('[');
  let first = true;

  const cursor = db.users.find().cursor();
  for await (const user of cursor) {
    if (!first) res.write(',');
    res.write(JSON.stringify(user));
    first = false;
  }

  res.write(']');
  res.end();
});

// Node.js Web Streams API (fetch-compatible)
app.get('/api/stream', (req, res) => {
  const stream = new ReadableStream({
    async pull(controller) {
      const chunk = await getNextChunk();
      if (chunk) {
        controller.enqueue(new TextEncoder().encode(chunk));
      } else {
        controller.close();
      }
    },
  });

  return new Response(stream, {
    headers: { 'Content-Type': 'text/event-stream' },
  });
});

Serverless Optimization

Cold Start Mitigation

# AWS Lambda — provisioned concurrency keeps instances warm
Resources:
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs20.x
      MemorySize: 512
      Timeout: 30
      ProvisionedConcurrencyConfig:
        ProvisionedConcurrentExecutions: 5

// Keep-warm pattern: scheduled invocation every 5 minutes
// AWS EventBridge rule
// { "schedule": "rate(5 minutes)", "target": "my-lambda-arn" }

// Handler detects warm-up events and returns early
export const handler = async (event: any) => {
  if (event.source === 'aws.events') {
    return { statusCode: 200, body: 'warm' };
  }
  return processRequest(event);
};

Memory/timeout budgeting:
  128 MB  — Simple transforms, redirects
  256 MB  — API handlers, light DB queries
  512 MB  — Image processing, moderate computation
  1024 MB — Heavy computation, ML inference

  Timeout guidelines:
  API Gateway limit: 29s max
  Direct invoke: up to 15 min
  Set timeout to 2x expected P95 duration

Connection Pooling for Serverless

// RDS Proxy — managed connection pooling for AWS Lambda
// Configure in AWS console, then connect via proxy endpoint
import { Pool } from 'pg';

const pool = new Pool({
  host: process.env.RDS_PROXY_ENDPOINT,
  max: 1, // Single connection per Lambda instance
  ssl: { rejectUnauthorized: false },
});

// Reuse connection across invocations (module-level)
let client: any;

export const handler = async () => {
  if (!client) {
    client = await pool.connect();
  }
  const result = await client.query('SELECT * FROM users LIMIT 10');
  return { statusCode: 200, body: JSON.stringify(result.rows) };
};

Serverless connection pooling options:
  AWS RDS Proxy     — Managed, supports PostgreSQL/MySQL
  PgBouncer         — Self-hosted, PostgreSQL only
  Neon              — Serverless Postgres with built-in pooling
  PlanetScale       — Serverless MySQL with HTTP-based queries

Rate Limiting

import rateLimit from 'express-rate-limit';

const apiLimiter = rateLimit({
  windowMs: 15 * 60 * 1000,
  max: 100,
  message: 'Too many requests, please try again later',
});

app.use('/api/', apiLimiter);

const authLimiter = rateLimit({
  windowMs: 60 * 60 * 1000,
  max: 5,
  skipSuccessfulRequests: true,
});

app.post('/api/auth/login', authLimiter, loginHandler);

Caching Strategies

Multi-Layer Caching

Browser Cache (HTTP headers)
  ↓
CDN Cache (Cloudflare, CloudFront)
  ↓
Application Cache (Redis, Memcached)
  ↓
Database Query Cache
  ↓
Database

Redis Caching

import Redis from 'ioredis';

const redis = new Redis({
  maxRetriesPerRequest: 3,
  enableReadyCheck: true,
});

async function getUser(id: string): Promise<User> {
  const cacheKey = `user:${id}`;

  // 1. Check cache
  const cached = await redis.get(cacheKey);
  if (cached) {
    return JSON.parse(cached);
  }

  // 2. Cache miss - fetch from database
  const user = await db.users.findById(id);

  // 3. Store in cache (expire in 1 hour)
  await redis.setex(cacheKey, 3600, JSON.stringify(user));

  return user;
}

// Cache invalidation
async function updateUser(id: string, data: Partial<User>) {
  await db.users.update(id, data);
  await redis.del(`user:${id}`); // Invalidate cache
}

HTTP Cache Headers

// Express middleware
app.use((req, res, next) => {
  // Static assets: cache for 1 year
  if (req.url.match(/\.(js|css|png|jpg|jpeg|gif|svg|woff|woff2)$/)) {
    res.setHeader('Cache-Control', 'public, max-age=31536000, immutable');
  }

  // HTML: no cache (always revalidate)
  if (req.url.endsWith('.html') || req.url === '/') {
    res.setHeader('Cache-Control', 'no-cache, must-revalidate');
  }

  // API responses: cache for 5 minutes
  if (req.url.startsWith('/api/')) {
    res.setHeader('Cache-Control', 'public, max-age=300');
    res.setHeader('ETag', generateETag(req.url));
  }

  next();
});

CDN Configuration

Static Assets to CDN:
  - Images: /images/**
  - JavaScript: /js/**
  - CSS: /css/**
  - Fonts: /fonts/**

CDN Settings:
  - Cache duration: 1 year (with versioned URLs)
  - Gzip/Brotli compression: enabled
  - Image optimization: WebP conversion
  - Purge on deploy: yes (via API)

Recommended CDNs:
  - Cloudflare (free tier excellent)
  - CloudFront (AWS integration)
  - Fastly (enterprise, very fast)

Database Optimization

Strategic Indexes

-- Before: Table scan (slow)
SELECT * FROM users WHERE email = 'user@example.com';
-- Execution time: 2000ms on 1M rows

-- After: Index scan (fast)
CREATE INDEX idx_users_email ON users(email);
SELECT * FROM users WHERE email = 'user@example.com';
-- Execution time: 5ms

-- Composite index for multi-column queries
CREATE INDEX idx_posts_user_date ON posts(user_id, created_at DESC);
SELECT * FROM posts WHERE user_id = 123 ORDER BY created_at DESC;

-- Partial index for filtered queries
CREATE INDEX idx_active_users ON users(created_at) WHERE is_active = true;

Eliminate N+1 Queries

// N+1 query problem (101 database queries)
const users = await User.findAll(); // 1 query
for (const user of users) {
  user.posts = await Post.findAll({ where: { userId: user.id } }); // N queries
}

// Eager loading (2 queries)
const users = await User.findAll({
  include: [{ model: Post }],
});

// DataLoader (batching + caching)
const userLoader = new DataLoader(async (userIds) => {
  const users = await User.findAll({ where: { id: userIds } });
  return userIds.map((id) => users.find((u) => u.id === id));
});

Query Optimization

-- Avoid SELECT *
SELECT id, name, email FROM users WHERE id = 1;

-- Use LIMIT
SELECT * FROM posts ORDER BY created_at DESC LIMIT 20;

-- Avoid functions in WHERE clause (can't use index)
-- Bad:  SELECT * FROM users WHERE LOWER(email) = 'user@example.com';
-- Good: SELECT * FROM users WHERE email = 'user@example.com';
-- Store email as lowercase, or use generated column + index

Connection Pooling

import { Pool } from 'pg';

const pool = new Pool({
  max: 20, // Maximum connections
  min: 5, // Minimum connections
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 2000, // Error if can't connect in 2s
});

// Always release connections
const client = await pool.connect();
try {
  const result = await client.query('SELECT * FROM users');
  return result.rows;
} finally {
  client.release();
}

Frontend Performance

Code Splitting and Lazy Loading

import { lazy, Suspense } from 'react'

// Lazy load routes
const Dashboard = lazy(() => import('./Dashboard'))
const AdminPanel = lazy(() => import('./AdminPanel'))

function App() {
  return (
    <Suspense fallback={<LoadingSpinner />}>
      <Routes>
        <Route path="/dashboard" element={<Dashboard />} />
        <Route path="/admin" element={<AdminPanel />} />
      </Routes>
    </Suspense>
  )
}

// Next.js dynamic imports
import dynamic from 'next/dynamic'

const HeavyComponent = dynamic(() => import('./HeavyComponent'), {
  loading: () => <LoadingSpinner />,
  ssr: false // Skip SSR for this component
})

Image Optimization

// Next.js Image component (automatic optimization)
import Image from 'next/image'

<Image
  src="/photo.jpg"
  width={800}
  height={600}
  alt="Description"
  loading="lazy"        // Lazy load off-screen images
  placeholder="blur"    // Blur placeholder while loading
  quality={75}          // 75% quality (good balance)
/>

// WebP format with fallback
<picture>
  <source srcset="image.webp" type="image/webp" />
  <source srcset="image.jpg" type="image/jpeg" />
  <img src="image.jpg" alt="Description" loading="lazy" />
</picture>

// Responsive images
<img
  srcset="
    small.jpg 480w,
    medium.jpg 768w,
    large.jpg 1200w
  "
  sizes="(max-width: 480px) 480px, (max-width: 768px) 768px, 1200px"
  src="medium.jpg"
  alt="Description"
/>

Bundle Analysis

# webpack-bundle-analyzer (webpack/Next.js)
npm install --save-dev webpack-bundle-analyzer
ANALYZE=true npm run build

# source-map-explorer (works with any bundler)
npm install --save-dev source-map-explorer
npx source-map-explorer dist/assets/*.js

// next.config.ts — enable bundle analyzer
import withBundleAnalyzer from '@next/bundle-analyzer';

export default withBundleAnalyzer({
  enabled: process.env.ANALYZE === 'true',
})({
  // next config
});

// Verify tree-shaking: named imports only
import { debounce } from 'lodash-es'; // tree-shakeable
import debounce from 'lodash/debounce'; // direct import fallback

// Verify with sideEffects in package.json
// { "sideEffects": false } — marks all modules as side-effect-free
// { "sideEffects": ["*.css"] } — except CSS files

Bundle Size Reduction

# Remove unused dependencies
npx depcheck

# Check import cost before adding a dependency
npx bundlephobia-cli react-datepicker

Third-Party Script Management

<!-- Defer non-critical scripts (download parallel, execute after parse) -->
<script src="https://analytics.example.com/tracker.js" defer></script>

<!-- Async for independent scripts (download parallel, execute immediately) -->
<script src="https://cdn.example.com/widget.js" async></script>

// Dynamic import for non-critical libraries
async function initAnalytics() {
  const { init } = await import('./analytics');
  init();
}

if (typeof requestIdleCallback !== 'undefined') {
  requestIdleCallback(() => initAnalytics());
} else {
  setTimeout(() => initAnalytics(), 2000);
}

// Track third-party script impact with PerformanceObserver
const observer = new PerformanceObserver((list) => {
  for (const entry of list.getEntries()) {
    if (entry.initiatorType === 'script') {
      console.log(`${entry.name}: ${entry.duration.toFixed(0)}ms`);
    }
  }
});
observer.observe({ type: 'resource', buffered: true });

Network Waterfall Optimization

<!-- dns-prefetch: resolve DNS for third-party origins early -->
<link rel="dns-prefetch" href="https://api.example.com" />

<!-- preconnect: DNS + TCP + TLS handshake (use for critical origins) -->
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://cdn.example.com" crossorigin />

<!-- preload: fetch critical resources early (fonts, hero images, key scripts) -->
<link
  rel="preload"
  href="/fonts/inter.woff2"
  as="font"
  type="font/woff2"
  crossorigin
/>
<link rel="preload" href="/hero.webp" as="image" />

<!-- prefetch: low-priority fetch for likely next navigation -->
<link rel="prefetch" href="/dashboard" />

<!-- fetchpriority: signal resource importance to the browser -->
<img src="/hero.webp" fetchpriority="high" alt="Hero" />
<img
  src="/below-fold.webp"
  fetchpriority="low"
  loading="lazy"
  alt="Secondary"
/>

<script src="/critical.js" fetchpriority="high"></script>
<script src="/analytics.js" fetchpriority="low" defer></script>

<!-- Connection coalescing: serve multiple subdomains from one HTTP/2 connection -->
<!-- Use same certificate covering *.example.com -->
<!-- Combine: api.example.com, cdn.example.com, static.example.com -->
<!-- Browser reuses single TCP connection for all, reducing handshake overhead -->

Mobile Performance

// Viewport-aware loading: only load resources when they enter the viewport
const observer = new IntersectionObserver(
  (entries) => {
    for (const entry of entries) {
      if (entry.isIntersecting) {
        const img = entry.target as HTMLImageElement;
        img.src = img.dataset.src!;
        observer.unobserve(img);
      }
    }
  },
  { rootMargin: '200px' },
);

document
  .querySelectorAll('img[data-src]')
  .forEach((img) => observer.observe(img));

// Adaptive loading based on connection quality
const connection = (navigator as any).connection;

function getImageQuality(): 'low' | 'medium' | 'high' {
  if (!connection) return 'high';
  if (connection.saveData) return 'low';
  if (
    connection.effectiveType === '2g' ||
    connection.effectiveType === 'slow-2g'
  )
    return 'low';
  if (connection.effectiveType === '3g') return 'medium';
  return 'high';
}

DevTools mobile testing profiles:
  Slow 3G:  Download 500 Kbps, Upload 500 Kbps, Latency 400ms
  Fast 3G:  Download 1.5 Mbps, Upload 750 Kbps, Latency 100ms
  4G:       Download 4 Mbps, Upload 3 Mbps, Latency 20ms

Steps: DevTools → Network tab → Throttling dropdown → select profile
Also enable: DevTools → Performance tab → CPU throttling (4x/6x slowdown)

/* Touch interaction responsiveness: remove 300ms tap delay */
html {
  touch-action: manipulation;
}

/* Reduce layout shift from tap highlights */
* {
  -webkit-tap-highlight-color: transparent;
}

React Performance

// 1. Memoize expensive calculations
import { useMemo } from 'react'

function DataTable({ data }) {
  const sortedData = useMemo(
    () => data.sort((a, b) => a.name.localeCompare(b.name)),
    [data]
  )

  return <Table data={sortedData} />
}

// 2. Memoize components
import { memo } from 'react'

const ExpensiveComponent = memo(function ExpensiveComponent({ data }) {
  return <div>{/* expensive rendering */}</div>
})

// 3. useCallback for stable function references
import { useCallback } from 'react'

function Parent() {
  const handleClick = useCallback(() => {
    console.log('Clicked')
  }, [])

  return <ExpensiveChild onClick={handleClick} />
}

// 4. Virtualize long lists
import { FixedSizeList } from 'react-window'

<FixedSizeList
  height={600}
  itemCount={10000}
  itemSize={50}
>
  {({ index, style }) => (
    <div style={style}>Row {index}</div>
  )}
</FixedSizeList>

Monitoring and Alerting

APM Tools

Custom Monitoring

// Track response times
app.use((req, res, next) => {
  const start = Date.now();

  res.on('finish', () => {
    const duration = Date.now() - start;

    // Log to monitoring service
    metrics.recordResponseTime(req.path, duration);

    // Alert on slow requests
    if (duration > 1000) {
      logger.warn(`Slow request: ${req.path} took ${duration}ms`);
    }
  });

  next();
});

// Track database query times
db.on('query', (query, duration) => {
  if (duration > 100) {
    logger.warn(`Slow query: ${query} took ${duration}ms`);
  }
});

Performance Dashboards

Key Metrics to Track:
  - Response time (P50, P95, P99)
  - Throughput (requests/second)
  - Error rate (%)
  - Database query times
  - Cache hit ratio
  - Memory usage
  - CPU usage

Alerting Thresholds:
  - P95 response time > 1s
  - Error rate > 1%
  - Cache hit ratio < 80%
  - Memory usage > 80%

Profiling and Measurement

Goal: Identify actual bottlenecks, not perceived ones.

Frontend Profiling

Chrome DevTools

// 1. Performance tab → Record → Reload page
// 2. Analyze:
//    - Main thread activity
//    - Network waterfall
//    - JavaScript execution time
//    - Rendering time

// 3. Lighthouse audit
// Run: chrome://lighthouse or `npm i -g lighthouse`
lighthouse https://yoursite.com --view

React DevTools Profiler

import { Profiler } from 'react';

function onRenderCallback(id, phase, actualDuration) {
  console.log(`${id} (${phase}) took ${actualDuration}ms`);
}

<Profiler id="ExpensiveComponent" onRender={onRenderCallback}>
  <ExpensiveComponent />
</Profiler>;

Backend Profiling

Node.js

# Generate CPU profile
node --prof app.js

# Process profile
node --prof-process isolate-0x*.log > processed.txt

# Flame graphs (better visualization)
npm i -g 0x
0x app.js

Python

import cProfile
import pstats

# Profile function
cProfile.run('slow_function()', 'output.prof')

# Analyze
p = pstats.Stats('output.prof')
p.sort_stats('cumulative').print_stats(20)

Database Profiling

PostgreSQL

-- Enable query logging
ALTER DATABASE yourdb SET log_min_duration_statement = 100; -- Log queries >100ms

-- Analyze query
EXPLAIN (ANALYZE, BUFFERS)
SELECT * FROM users WHERE email = 'test@example.com';

-- Find slow queries
SELECT query, mean_exec_time, calls
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 20;

MongoDB

// Enable profiling
db.setProfilingLevel(1, { slowms: 100 });

// View slow queries
db.system.profile.find({ millis: { $gt: 100 } }).sort({ ts: -1 });

// Explain query
db.collection.find({ email: 'test@example.com' }).explain('executionStats');