Performance Optimizer

Performance Optimizer

Make applications fast, scalable, and cost-efficient.

Core Principle

Measure first, optimize second. Don't guess at bottlenecks—profile, measure, then fix the slowest parts.

Performance Budget

Web Vitals (Target Metrics)

Core Web Vitals:
  Largest Contentful Paint (LCP): < 2.5s # Main content visible
  First Input Delay (FID): < 100ms # Interaction responsiveness
  Cumulative Layout Shift (CLS): < 0.1 # Visual stability

Additional Metrics:
  First Contentful Paint (FCP): < 1.8s # First content rendered
  Time to Interactive (TTI): < 3.8s # Fully interactive
  Total Blocking Time (TBT): < 200ms # Main thread blocked
  Speed Index: < 3.4s # Visual progress

Backend Metrics:
  API Response Time (P95): < 500ms
  Database Query Time (P95): < 100ms
  Server Response Time (TTFB): < 600ms

---

Phase 1: Profiling & Measurement

Goal: Identify actual bottlenecks, not perceived ones

Frontend Profiling

Chrome DevTools:

// 1. Performance tab → Record → Reload page
// 2. Analyze:
//    - Main thread activity
//    - Network waterfall
//    - JavaScript execution time
//    - Rendering time

// 3. Lighthouse audit
// Run: chrome://lighthouse or `npm i -g lighthouse`
lighthouse https://yoursite.com --view

React DevTools Profiler:

// Wrap component to profile
import { Profiler } from 'react'

function onRenderCallback(id, phase, actualDuration) {
  console.log(`${id} (${phase}) took ${actualDuration}ms`)
}

;<Profiler id="ExpensiveComponent" onRender={onRenderCallback}>
  <ExpensiveComponent />
</Profiler>

Backend Profiling

Node.js Profiling:

# Generate CPU profile
node --prof app.js

# Process profile
node --prof-process isolate-0x*.log > processed.txt

# Flame graphs (better visualization)
npm i -g 0x
0x app.js

Python Profiling:

import cProfile
import pstats

# Profile function
cProfile.run('slow_function()', 'output.prof')

# Analyze
p = pstats.Stats('output.prof')
p.sort_stats('cumulative').print_stats(20)

Database Profiling

PostgreSQL:

-- Enable query logging
ALTER DATABASE yourdb SET log_min_duration_statement = 100; -- Log queries >100ms

-- Analyze query
EXPLAIN (ANALYZE, BUFFERS)
SELECT * FROM users WHERE email = 'test@example.com';

-- Find slow queries
SELECT query, mean_exec_time, calls
FROM pg_stat_statements
ORDER BY mean_exec_time DESC
LIMIT 20;

MongoDB:

// Enable profiling
db.setProfilingLevel(1, { slowms: 100 })

// View slow queries
db.system.profile.find({ millis: { $gt: 100 } }).sort({ ts: -1 })

// Explain query
db.collection.find({ email: 'test@example.com' }).explain('executionStats')

---

Phase 2: Database Optimization

Add Strategic Indexes

-- Before: Table scan (slow)
SELECT * FROM users WHERE email = 'user@example.com';
-- Execution time: 2000ms on 1M rows

-- After: Index scan (fast)
CREATE INDEX idx_users_email ON users(email);
SELECT * FROM users WHERE email = 'user@example.com';
-- Execution time: 5ms

-- Composite index for multi-column queries
CREATE INDEX idx_posts_user_date ON posts(user_id, created_at DESC);
SELECT * FROM posts WHERE user_id = 123 ORDER BY created_at DESC;

-- Partial index for filtered queries
CREATE INDEX idx_active_users ON users(created_at) WHERE is_active = true;

Eliminate N+1 Queries

// ❌ Bad: N+1 query problem (101 database queries)
const users = await User.findAll() // 1 query
for (const user of users) {
  user.posts = await Post.findAll({ where: { userId: user.id } }) // N queries
}

// ✅ Good: Eager loading (2 queries)
const users = await User.findAll({
  include: [{ model: Post }]
})

// ✅ Better: DataLoader (batching + caching)
const userLoader = new DataLoader(async userIds => {
  const users = await User.findAll({ where: { id: userIds } })
  return userIds.map(id => users.find(u => u.id === id))
})

Query Optimization

-- Avoid SELECT *
-- ❌ Bad
SELECT * FROM users WHERE id = 1;

-- ✅ Good
SELECT id, name, email FROM users WHERE id = 1;

-- Use LIMIT
-- ❌ Bad
SELECT * FROM posts ORDER BY created_at DESC;

-- ✅ Good
SELECT * FROM posts ORDER BY created_at DESC LIMIT 20;

-- Avoid functions in WHERE clause
-- ❌ Bad (can't use index)
SELECT * FROM users WHERE LOWER(email) = 'user@example.com';

-- ✅ Good (can use index)
SELECT * FROM users WHERE email = 'user@example.com';
-- Store email as lowercase, or use generated column + index

Connection Pooling

// PostgreSQL connection pool
import { Pool } from 'pg'

const pool = new Pool({
  max: 20, // Maximum connections
  min: 5, // Minimum connections
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 2000 // Error if can't connect in 2s
})

// Always release connections
const client = await pool.connect()
try {
  const result = await client.query('SELECT * FROM users')
  return result.rows
} finally {
  client.release()
}

---

Phase 3: Caching Strategy

Multi-Layer Caching

Browser Cache (HTTP headers)
  ↓
CDN Cache (Cloudflare, CloudFront)
  ↓
Application Cache (Redis, Memcached)
  ↓
Database Query Cache
  ↓
Database

Redis Caching

import Redis from 'ioredis'

const redis = new Redis({
  maxRetriesPerRequest: 3,
  enableReadyCheck: true
})

async function getUser(id: string): Promise<User> {
  const cacheKey = `user:${id}`

  // 1. Check cache
  const cached = await redis.get(cacheKey)
  if (cached) {
    return JSON.parse(cached)
  }

  // 2. Cache miss - fetch from database
  const user = await db.users.findById(id)

  // 3. Store in cache (expire in 1 hour)
  await redis.setex(cacheKey, 3600, JSON.stringify(user))

  return user
}

// Cache invalidation
async function updateUser(id: string, data: Partial<User>) {
  await db.users.update(id, data)
  await redis.del(`user:${id}`) // Invalidate cache
}

HTTP Caching Headers

// Express middleware
app.use((req, res, next) => {
  // Static assets: cache for 1 year
  if (req.url.match(/\.(js|css|png|jpg|jpeg|gif|svg|woff|woff2)$/)) {
    res.setHeader('Cache-Control', 'public, max-age=31536000, immutable')
  }

  // HTML: no cache (always revalidate)
  if (req.url.endsWith('.html') || req.url === '/') {
    res.setHeader('Cache-Control', 'no-cache, must-revalidate')
  }

  // API responses: cache for 5 minutes
  if (req.url.startsWith('/api/')) {
    res.setHeader('Cache-Control', 'public, max-age=300')
    res.setHeader('ETag', generateETag(req.url))
  }

  next()
})

CDN Configuration

Static Assets to CDN:
  - Images: /images/**
  - JavaScript: /js/**
  - CSS: /css/**
  - Fonts: /fonts/**

CDN Settings:
  - Cache duration: 1 year (with versioned URLs)
  - Gzip/Brotli compression: enabled
  - Image optimization: WebP conversion
  - Purge on deploy: yes (via API)

Recommended CDNs:
  - Cloudflare (free tier excellent)
  - CloudFront (AWS integration)
  - Fastly (enterprise, very fast)

---

Phase 4: Frontend Optimization

Code Splitting & Lazy Loading

// React lazy loading
import { lazy, Suspense } from 'react'

// ❌ Bad: Load everything upfront
import Dashboard from './Dashboard'
import AdminPanel from './AdminPanel'

// ✅ Good: Lazy load routes
const Dashboard = lazy(() => import('./Dashboard'))
const AdminPanel = lazy(() => import('./AdminPanel'))

function App() {
  return (
    <Suspense fallback={<LoadingSpinner />}>
      <Routes>
        <Route path="/dashboard" element={<Dashboard />} />
        <Route path="/admin" element={<AdminPanel />} />
      </Routes>
    </Suspense>
  )
}

// Next.js dynamic imports
import dynamic from 'next/dynamic'

const HeavyComponent = dynamic(() => import('./HeavyComponent'), {
  loading: () => <LoadingSpinner />,
  ssr: false // Skip SSR for this component
})

Image Optimization

// Next.js Image component (automatic optimization)
import Image from 'next/image'

<Image
  src="/photo.jpg"
  width={800}
  height={600}
  alt="Description"
  loading="lazy"        // Lazy load off-screen images
  placeholder="blur"    // Blur placeholder while loading
  quality={75}          // 75% quality (good balance)
/>

// WebP format with fallback
<picture>
  <source srcset="image.webp" type="image/webp" />
  <source srcset="image.jpg" type="image/jpeg" />
  <img src="image.jpg" alt="Description" loading="lazy" />
</picture>

// Responsive images
<img
  srcset="
    small.jpg 480w,
    medium.jpg 768w,
    large.jpg 1200w
  "
  sizes="(max-width: 480px) 480px, (max-width: 768px) 768px, 1200px"
  src="medium.jpg"
  alt="Description"
/>

Bundle Size Optimization

# Analyze bundle
npm run build -- --analyze

# Reduce bundle size:
# 1. Remove unused dependencies
npm uninstall unused-package

# 2. Use tree-shaking compatible imports
# ❌ Bad
import _ from 'lodash'
# ✅ Good
import debounce from 'lodash/debounce'

# 3. Dynamic imports for large libraries
const moment = await import('moment')

# 4. Minification (automatic in production builds)
# Vite/Next.js handle this automatically

React Performance

// 1. Memoize expensive calculations
import { useMemo } from 'react'

function DataTable({ data }) {
  const sortedData = useMemo(
    () => data.sort((a, b) => a.name.localeCompare(b.name)),
    [data]
  )

  return <Table data={sortedData} />
}

// 2. Memoize components
import { memo } from 'react'

const ExpensiveComponent = memo(function ExpensiveComponent({ data }) {
  // Only re-renders if data changes
  return <div>{/* expensive rendering */}</div>
})

// 3. useCallback for stable function references
import { useCallback } from 'react'

function Parent() {
  const handleClick = useCallback(() => {
    console.log('Clicked')
  }, [])

  return <ExpensiveChild onClick={handleClick} />
}

// 4. Virtualize long lists
import { FixedSizeList } from 'react-window'

<FixedSizeList
  height={600}
  itemCount={10000}
  itemSize={50}
>
  {({ index, style }) => (
    <div style={style}>Row {index}</div>
  )}
</FixedSizeList>

---

Phase 5: Backend Optimization

Async Background Processing

// ❌ Bad: Synchronous (slow response)
app.post('/send-email', async (req, res) => {
  await sendEmail(req.body) // 3 seconds
  res.json({ success: true })
})

// ✅ Good: Queue job (fast response)
import Bull from 'bull'

const emailQueue = new Bull('emails', 'redis://localhost:6379')

app.post('/send-email', async (req, res) => {
  await emailQueue.add('send', req.body)
  res.json({ success: true, message: 'Email queued' })
})

// Process jobs in background worker
emailQueue.process('send', async job => {
  await sendEmail(job.data)
})

API Response Optimization

// 1. Compression
import compression from 'compression'
app.use(compression()) // Gzip responses

// 2. Pagination
app.get('/api/posts', async (req, res) => {
  const page = parseInt(req.query.page) || 1
  const limit = parseInt(req.query.limit) || 20

  const posts = await db.posts.findAll({
    offset: (page - 1) * limit,
    limit: limit
  })

  res.json({
    data: posts,
    pagination: {
      page,
      limit,
      total: await db.posts.count()
    }
  })
})

// 3. Field filtering (GraphQL-style)
app.get('/api/users/:id', async (req, res) => {
  const fields = req.query.fields?.split(',') || ['id', 'name', 'email']

  const user = await db.users.findById(req.params.id, {
    attributes: fields
  })

  res.json(user)
})

Rate Limiting

import rateLimit from 'express-rate-limit'

// General API rate limit
const apiLimiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // 100 requests per window
  message: 'Too many requests, please try again later'
})

app.use('/api/', apiLimiter)

// Stricter limit for expensive endpoints
const authLimiter = rateLimit({
  windowMs: 60 * 60 * 1000, // 1 hour
  max: 5, // 5 requests per hour
  skipSuccessfulRequests: true
})

app.post('/api/auth/login', authLimiter, loginHandler)

---

Phase 6: Monitoring & Alerting

Application Performance Monitoring (APM)

Tools:

• Sentry: Error tracking + performance
• New Relic: Full-stack APM
• Datadog: Infrastructure + APM
• Vercel Analytics: Next.js optimized

Custom Monitoring:

// Track response times
app.use((req, res, next) => {
  const start = Date.now()

  res.on('finish', () => {
    const duration = Date.now() - start

    // Log to monitoring service
    metrics.recordResponseTime(req.path, duration)

    // Alert on slow requests
    if (duration > 1000) {
      logger.warn(`Slow request: ${req.path} took ${duration}ms`)
    }
  })

  next()
})

// Track database query times
db.on('query', (query, duration) => {
  if (duration > 100) {
    logger.warn(`Slow query: ${query} took ${duration}ms`)
  }
})

Performance Dashboards

Key Metrics to Track:
  - Response time (P50, P95, P99)
  - Throughput (requests/second)
  - Error rate (%)
  - Database query times
  - Cache hit ratio
  - Memory usage
  - CPU usage

Alerting Thresholds:
  - P95 response time > 1s
  - Error rate > 1%
  - Cache hit ratio < 80%
  - Memory usage > 80%

---

Optimization Checklist

Frontend ✅

• [ ] Lighthouse score > 90
• [ ] LCP < 2.5s
• [ ] FID < 100ms
• [ ] CLS < 0.1
• [ ] Bundle size < 200KB (initial)
• [ ] Images optimized (WebP, lazy loading)
• [ ] Code splitting implemented
• [ ] Critical CSS inlined

Backend ✅

• [ ] P95 response time < 500ms
• [ ] Database queries indexed
• [ ] N+1 queries eliminated
• [ ] Connection pooling enabled
• [ ] Background jobs async
• [ ] Rate limiting configured
• [ ] API responses compressed

Database ✅

• [ ] Slow query log enabled
• [ ] All queries < 100ms (P95)
• [ ] Indexes on foreign keys
• [ ] Indexes on WHERE/ORDER BY columns
• [ ] Query explain plans reviewed
• [ ] Connection pool sized correctly

Caching ✅

• [ ] Redis/Memcached configured
• [ ] CDN for static assets
• [ ] HTTP cache headers set
• [ ] Cache hit ratio > 80%
• [ ] Cache invalidation strategy

Infrastructure ✅

• [ ] Auto-scaling configured
• [ ] Load balancer healthy
• [ ] Monitoring/alerting active
• [ ] Logs centralized
• [ ] Backups automated

---

Related Resources

Related Skills:

• deployment-advisor - For infrastructure optimization
• frontend-builder - For React performance patterns
• api-designer - For API optimization

Related Patterns:

• META/DECISION-FRAMEWORK.md - Scaling decisions
• STANDARDS/architecture-patterns/caching-patterns.md - Caching strategies (when created)

Related Playbooks:

• PLAYBOOKS/optimize-database-performance.md - DB optimization steps (when created)
• PLAYBOOKS/frontend-performance-audit.md - Frontend audit procedure (when created)

Performance Optimizer - Quick Start

Version: 1.0.0 Category: Infrastructure & DevOps Difficulty: Advanced

What This Skill Does

Guides systematic performance optimization through profiling, database tuning, caching, frontend optimization, and monitoring to achieve sub-second response times and excellent user experience.

When to Use

Use this skill when you need to:

• Investigate slow page loads or API responses
• Optimize database queries and eliminate N+1 problems
• Implement caching strategies (Redis, CDN)
• Improve Core Web Vitals for SEO
• Scale applications to handle more traffic
• Reduce infrastructure costs through efficiency

Quick Start

Fastest path to better performance:

1. Measure first (don't guess!)

• Run Lighthouse audit: lighthouse https://yoursite.com
• Profile with Chrome DevTools Performance tab
• Check backend: Enable slow query logging
• Identify actual bottlenecks

2. Quick wins (15-30 minutes each)

• Add database indexes on WHERE/ORDER BY columns
• Enable compression (gzip/brotli)
• Add CDN for static assets (Cloudflare free)
• Lazy load images with loading="lazy"
• Enable Redis caching for API responses

3. Frontend optimization (Phase 4)

• Code splitting: Lazy load routes
• Image optimization: WebP format, Next.js Image
• Bundle reduction: Remove unused dependencies
• React memoization: useMemo, memo, useCallback

4. Backend optimization (Phase 5)

• Fix N+1 queries with eager loading
• Move slow tasks to background queue (Bull/BullMQ)
• Add connection pooling
• Implement rate limiting

5. Monitor continuously (Phase 6)

• Set up APM (Sentry, New Relic, Datadog)
• Track P95 response times
• Alert on slow requests (>1s)
• Monitor cache hit ratio

Time to improvements: Quick wins in 1-2 hours, full optimization 1-2 weeks

File Structure

performance-optimizer/
├── SKILL.md           # Main skill instructions (start here)
└── README.md          # This file

Prerequisites

Knowledge:

• Profiling tools (Chrome DevTools, Node --prof)
• Database query optimization
• HTTP caching and CDN concepts

Tools:

• Lighthouse CLI or Chrome DevTools
• Profiling tools (Chrome DevTools, Node profiler)
• APM tool (Sentry, New Relic, Datadog)
• Redis for caching (optional but recommended)

Related Skills:

• frontend-builder for React performance patterns
• deployment-advisor for infrastructure optimization

Success Criteria

You've successfully used this skill when:

• ✅ Core Web Vitals: LCP < 2.5s, FID < 100ms, CLS < 0.1
• ✅ Lighthouse score > 90
• ✅ P95 API response time < 500ms
• ✅ Database queries indexed (P95 < 100ms)
• ✅ N+1 queries eliminated
• ✅ Caching implemented (cache hit ratio > 80%)
• ✅ Bundle size < 200KB (initial load)
• ✅ CDN configured for static assets
• ✅ Monitoring/alerting active
• ✅ Performance budget defined and met

Common Workflows

Workflow 1: Fix Slow Page Loads

1. Run Lighthouse audit → identify issues 2. Use performance-optimizer Phase 4 (Frontend) 3. Optimize images (WebP, lazy loading) 4. Code split heavy components 5. Add CDN for static assets 6. Verify: LCP < 2.5s

Workflow 2: Optimize Slow API

1. Profile backend (Node --prof or similar) 2. Enable slow query logging 3. Use performance-optimizer Phase 2 (Database) 4. Add indexes, fix N+1 queries 5. Implement Redis caching (Phase 3) 6. Verify: P95 < 500ms

Workflow 3: Scale for Growth

1. Profile current performance 2. Optimize database (indexes, connection pooling) 3. Add multi-layer caching (Redis + CDN) 4. Move slow tasks to background queue 5. Set up auto-scaling 6. Monitor with APM tool

Key Concepts

Performance Budget:

• LCP: Largest Contentful Paint < 2.5s
• FID: First Input Delay < 100ms
• CLS: Cumulative Layout Shift < 0.1
• API P95: 95th percentile response time < 500ms

Profiling Tools:

• Chrome DevTools: Performance tab, Lighthouse, Network
• React DevTools: Profiler for component render times
• Node.js: --prof flag, flame graphs with 0x
• Database: EXPLAIN ANALYZE, slow query logs

Optimization Phases:

1. Profiling: Identify bottlenecks 2. Database: Indexes, query optimization, N+1 fixes 3. Caching: Redis, CDN, HTTP headers 4. Frontend: Code splitting, image optimization, bundle size 5. Backend: Async processing, compression, rate limiting 6. Monitoring: APM, alerts, dashboards

Caching Layers:

• Browser cache (HTTP headers)
• CDN cache (Cloudflare, CloudFront)
• Application cache (Redis, Memcached)
• Database query cache

Troubleshooting

Skill not activating?

• Try explicitly requesting: "Use the performance-optimizer skill to..."
• Mention keywords: "performance", "slow", "optimization", "caching"

Don't know where to start?

• Run Lighthouse audit first
• Profile with Chrome DevTools Performance tab
• Check backend with slow query logging
• Start with Phase 1 (Profiling) to identify real bottlenecks

LCP (Largest Contentful Paint) too slow?

• Optimize images (WebP, lazy loading, Next.js Image)
• Reduce JavaScript execution time (code splitting)
• Preload critical resources
• Use CDN for faster asset delivery
• Server-side render above-the-fold content

API responses too slow?

• Profile to find bottleneck (database, external API, computation)
• Add database indexes on WHERE/ORDER BY columns
• Fix N+1 queries with eager loading
• Implement Redis caching for frequent queries
• Move heavy computations to background queue

Database queries slow?

• Run EXPLAIN ANALYZE on slow queries
• Add indexes (but not too many—slows writes)
• Avoid SELECT \*, fetch only needed columns
• Use LIMIT for large result sets
• Avoid functions in WHERE clause (breaks indexes)
• Check query execution plan

N+1 query problem?

• Use eager loading (include/join in ORM)
• Implement DataLoader for batching
• Check ORM logs for query count
• Profile with database slow query log

Bundle size too large?

• Analyze with npm run build -- --analyze
• Remove unused dependencies
• Use tree-shaking imports (import debounce from 'lodash/debounce')
• Dynamic imports for large libraries
• Code split routes

React app re-rendering too much?

• Use React DevTools Profiler
• Add memo to expensive components
• Use useMemo for expensive calculations
• Use useCallback for stable function references
• Check dependency arrays in useEffect, useMemo, useCallback

Cache not helping?

• Check cache hit ratio (should be > 80%)
• Ensure cache keys are correct
• Set appropriate TTL (Time To Live)
• Implement cache warming for critical data
• Monitor cache eviction rate

How to prioritize optimizations?

• Fix issues impacting most users first
• Target Core Web Vitals for SEO
• Optimize P95 response times (not just average)
• Address slowest queries first (80/20 rule)
• Quick wins: indexes, compression, CDN

Version History

• 1.0.0 (2025-10-21): Initial release, comprehensive web application performance optimization

License

Part of ai-dev-standards repository.