
Load Balancing Patterns
- 46 installs
- 426 repo stars
- Updated December 11, 2025
- ancoleman/ai-design-components
load-balancing-patterns is a Claude Code skill that helps select and configure L4/L7, cloud-managed, self-managed, or Kubernetes load balancers with health checks and session management.
About
This skill helps select and configure load balancing solutions for distributing traffic across servers or regions. It covers Layer 4 versus Layer 7 balancing, load-balancing algorithms, health check strategies, and session persistence. Developers use it when implementing high availability, routing by URL or geography, deploying to Kubernetes, or choosing between cloud-managed load balancers (AWS ALB/NLB, GCP, Azure) and self-managed ones (NGINX, HAProxy, Envoy).
- Select and configure load balancing solutions (L4/L7, cloud, self-managed, Kubernetes)
- Covers algorithms, health checks, and session persistence
- Includes AWS/GCP/Azure LBs plus NGINX, HAProxy, and Envoy
Load Balancing Patterns by the numbers
- 46 all-time installs (skills.sh)
- Ranked #733 of 1,039 Cloud & Infrastructure skills by installs in the Skillselion catalog
- Data as of Aug 5, 2026 (Skillselion catalog sync)
load-balancing-patterns capabilities & compatibility
- Capabilities
- load balancer selection · health checks · session persistence · multi region failover
- Works with
- aws · gcp · azure · kubernetes
- Use cases
- devops
- Pricing
- Free
What load-balancing-patterns says it does
Distribute traffic across infrastructure using the appropriate load balancing approach, from simple round-robin to global multi-region failover.
npx skills add https://github.com/ancoleman/ai-design-components --skill load-balancing-patternsAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 46 |
|---|---|
| repo stars | ★ 426 |
| Last updated | December 11, 2025 |
| Repository | ancoleman/ai-design-components ↗ |
What it does
Selecting and configuring L4/L7 load balancers with health checks and failover across servers or regions.
Who is it for?
Choosing and configuring load balancers for HA, failover, and multi-region routing.
Skip if: Single-server apps with no scaling or availability requirements.
When should I use this skill?
You are distributing traffic across servers or regions and need the right load balancing solution with health checks.
What you get
An appropriate load balancer configured with correct algorithm, health checks, and session persistence.
- Load balancer selection decision
- NGINX or HAProxy configuration
- Health check strategy
By the numbers
- 6 load balancing algorithms tabulated
- 3-failure-down / 2-success-up health check hysteresis example
Files
Load Balancing Patterns
Distribute traffic across infrastructure using the appropriate load balancing approach, from simple round-robin to global multi-region failover.
When to Use This Skill
Use load-balancing-patterns when:
- Distributing traffic across multiple application servers
- Implementing high availability and failover
- Routing traffic based on URLs, headers, or geographic location
- Managing session persistence across stateless backends
- Deploying applications to Kubernetes clusters
- Configuring global traffic management across regions
- Implementing zero-downtime deployments (blue-green, canary)
- Selecting between cloud-managed and self-managed load balancers
Core Load Balancing Concepts
Layer 4 vs Layer 7
Layer 4 (L4) - Transport Layer:
- Routes based on IP address and port (TCP/UDP packets)
- No application data inspection, lower latency, higher throughput
- Protocol agnostic, preserves client IP addresses
- Use for: Database connections, video streaming, gaming, financial transactions, non-HTTP protocols
Layer 7 (L7) - Application Layer:
- Routes based on HTTP URLs, headers, cookies, request body
- Full application data visibility, SSL/TLS termination, caching, WAF integration
- Content-based routing capabilities
- Use for: Web applications, REST APIs, microservices, GraphQL endpoints, complex routing logic
For detailed comparison including performance benchmarks and hybrid approaches, see references/l4-vs-l7-comparison.md.
Load Balancing Algorithms
| Algorithm | Distribution Method | Use Case |
|---|---|---|
| Round Robin | Sequential | Stateless, similar servers |
| Weighted Round Robin | Capacity-based | Different server specs |
| Least Connections | Fewest active connections | Long-lived connections |
| Least Response Time | Fastest server | Performance-sensitive |
| IP Hash | Client IP-based | Session persistence |
| Resource-Based | CPU/memory metrics | Varying workloads |
Health Check Types
Shallow (Liveness): Is the process alive?
- Endpoint:
/health/liveor/live - Returns: 200 if process running
- Use for: Process monitoring, container health
Deep (Readiness): Can the service handle requests?
- Endpoint:
/health/readyor/ready - Validates: Database, cache, external API connectivity
- Use for: Load balancer routing decisions
Health Check Hysteresis: Different thresholds for marking up vs down to prevent flapping
- Example: 3 failures to mark down, 2 successes to mark up
For complete health check implementation patterns, see references/health-check-strategies.md.
Cloud Load Balancers
AWS Load Balancing
Application Load Balancer (ALB) - Layer 7:
- Use for: HTTP/HTTPS applications, microservices, WebSocket
- Features: Path/host/header routing, AWS WAF integration, Lambda targets
- Choose when: Content-based routing needed
Network Load Balancer (NLB) - Layer 4:
- Use for: Ultra-low latency (<1ms), TCP/UDP, static IPs, millions RPS
- Features: Preserves source IP, TLS termination
- Choose when: Non-HTTP protocols, performance critical
Global Accelerator - Layer 4 Global:
- Use for: Multi-region applications, global users, DDoS protection
- Features: Anycast IPs, automatic regional failover
GCP Load Balancing
Application LB (L7): Global HTTPS LB, Cloud CDN integration, Cloud Armor (WAF/DDoS) Network LB (L4): Regional TCP/UDP, pass-through balancing, session affinity Cloud Load Balancing: Single anycast IP, global distribution, backend buckets
Azure Load Balancing
Application Gateway (L7): WAF integration, URL-based routing, SSL termination, autoscaling Load Balancer (L4): Basic and Standard SKUs, health probes, HA ports Traffic Manager (Global): DNS-based routing (priority, weighted, performance, geographic)
For complete cloud provider configurations and Terraform examples, see references/cloud-load-balancers.md.
Self-Managed Load Balancers
NGINX
Best for: General-purpose HTTP/HTTPS load balancing, web application stacks
Capabilities:
- HTTP reverse proxy with multiple algorithms
- TCP/UDP stream load balancing
- SSL/TLS termination
- Passive health checks (open source), active health checks (NGINX Plus)
- Cookie-based sticky sessions (NGINX Plus)
Basic configuration:
upstream backend {
least_conn;
server backend1.example.com:8080 weight=3;
server backend2.example.com:8080 weight=2;
keepalive 32;
}
server {
listen 80;
location / {
proxy_pass http://backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}For complete NGINX patterns and advanced configurations, see references/nginx-patterns.md.
HAProxy
Best for: Maximum performance, database load balancing, resource efficiency
Capabilities:
- Highest raw throughput, lowest memory footprint
- 10+ load balancing algorithms
- Sophisticated health checks (HTTP, TCP, Redis, MySQL, etc.)
- Cookie or IP-based persistence
Basic configuration:
frontend http_front
bind *:80
default_backend web_servers
backend web_servers
balance roundrobin
option httpchk GET /health
server web1 192.168.1.101:8080 check
server web2 192.168.1.102:8080 checkFor complete HAProxy patterns, see references/haproxy-patterns.md.
Envoy
Best for: Microservices, Kubernetes, service mesh integration
Capabilities:
- Cloud-native design with dynamic configuration (xDS APIs)
- Circuit breakers, retries, timeouts
- Advanced health checks (TCP, HTTP, gRPC)
- Excellent observability
For complete Envoy patterns, see references/envoy-patterns.md.
Traefik
Best for: Docker/Kubernetes environments, dynamic configuration, ease of use
Capabilities:
- Automatic service discovery
- Native Kubernetes integration
- Built-in Let's Encrypt support
- Middleware system (auth, rate limiting)
For complete Traefik patterns, see references/traefik-patterns.md.
Kubernetes Ingress Controllers
Selection Guide
| Controller | Best For | Strengths |
|---|---|---|
| NGINX Ingress (F5) | General purpose | Stability, wide adoption, mature features |
| Traefik | Dynamic environments | Easy configuration, service discovery |
| HAProxy Ingress | High performance | Advanced L7 routing, reliability |
| Envoy (Contour/Gateway) | Service mesh | Rich L7 features, extensibility |
| Kong | API-heavy apps | JWT auth, rate limiting, plugins |
| Cloud Provider | Single-cloud | Native cloud integration |
Basic Ingress Example
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
ingressClassName: nginx
tls:
- hosts:
- app.example.com
secretName: app-tls
rules:
- host: app.example.com
http:
paths:
- path: /api
pathType: Prefix
backend:
service:
name: api-service
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: web-service
port:
number: 80For complete Kubernetes ingress examples and Gateway API patterns, see references/kubernetes-ingress.md.
Session Persistence
Sticky Sessions (Use Sparingly)
Cookie-Based: Load balancer sets cookie to track server affinity
- Accurate routing, works with NAT/proxies
- HTTP only, adds cookie overhead
IP Hash: Hash client IP to select backend server
- No cookie required, works for non-HTTP
- Poor distribution with NAT/proxies
Drawbacks: Uneven load distribution, session lost on server failure, complicates scaling
Shared Session Store (Recommended)
Architecture: Stateless application servers + centralized session storage (Redis, Memcached)
Benefits:
- No sticky sessions needed
- True load balancing
- Server failures don't lose sessions
- Horizontal scaling trivial
Client-Side Tokens (Best for APIs)
JWT (JSON Web Tokens): Server generates signed token, client stores and sends with requests
Benefits:
- Fully stateless servers
- Perfect load balancing
- No session storage needed
For complete session management patterns and code examples, see references/session-persistence.md.
Global Load Balancing
GeoDNS Routing
Route users to nearest server based on geographic location:
- DNS returns different IPs based on client location
- Reduces latency, supports compliance and regional content
- Implementation: AWS Route 53, GCP Cloud DNS, Azure Traffic Manager
Multi-Region Failover
Primary/secondary region configuration:
- Health checks determine primary region health
- Automatic DNS failover to secondary
- Transparent to clients
CDN Integration
Combine load balancing with CDN:
- GeoDNS routes to closest CDN PoP
- CDN caches content globally
- Origin load balancing for cache misses
For complete global load balancing examples with Terraform, see references/global-load-balancing.md.
Decision Frameworks
L4 vs L7 Selection
Choose L4 when:
- Protocol is TCP/UDP (not HTTP)
- Ultra-low latency critical (<1ms)
- High throughput required (millions RPS)
- Client source IP preservation needed
Choose L7 when:
- Protocol is HTTP/HTTPS
- Content-based routing needed (URL, headers)
- SSL termination required
- WAF integration needed
- Microservices architecture
Cloud vs Self-Managed
Choose Cloud-Managed when:
- Single cloud deployment
- Auto-scaling required
- Team lacks load balancer expertise
- Managed service preferred
Choose Self-Managed when:
- Multi-cloud or hybrid deployment
- Advanced routing requirements
- Cost optimization important
- Full control needed
- Vendor lock-in avoidance
Self-Managed Selection
- NGINX: General-purpose, web stacks, HTTP/3 support
- HAProxy: Maximum performance, database LB, lowest resource usage
- Envoy: Microservices, service mesh, dynamic configuration
- Traefik: Docker/Kubernetes, automatic discovery, easy configuration
Configuration Examples
Complete working examples available in examples/ directory:
Cloud Providers:
examples/aws/alb-terraform.tf- AWS ALB with path-based routingexamples/aws/nlb-terraform.tf- AWS NLB for TCP load balancing
Self-Managed:
examples/nginx/http-load-balancing.conf- NGINX HTTP reverse proxyexamples/haproxy/http-lb.cfg- HAProxy configurationexamples/envoy/basic-lb.yaml- Envoy cluster configurationexamples/traefik/kubernetes-ingress.yaml- Traefik IngressRoute
Kubernetes:
examples/kubernetes/nginx-ingress.yaml- NGINX Ingress with TLSexamples/kubernetes/traefik-ingress.yaml- Traefik IngressRouteexamples/kubernetes/gateway-api.yaml- Gateway API configuration
Monitoring and Observability
Key Metrics
Throughput: Requests per second, bytes transferred, connection rate Latency: Request duration (p50, p95, p99), backend response time, SSL handshake time Errors: HTTP error rates (4xx, 5xx), backend connection failures, health check failures Resource Utilization: CPU, memory, active connections, connection queue depth Health: Healthy/unhealthy backend count, health check success rate
Load Balancer Logs
Enable access logs for request/response details, client IPs, response times, error tracking
- AWS ALB: Store in S3, analyze with Athena
- NGINX: Custom log format, ship to centralized logging
- HAProxy: Syslog integration, structured logging
Troubleshooting
Uneven Load Distribution
Symptoms: One server receives disproportionate traffic Causes: Sticky sessions with few clients, IP hash with NAT concentration, long-lived connections Solutions: Switch to least connections, disable sticky sessions, implement connection draining
Health Check Flapping
Symptoms: Servers rapidly transition between healthy/unhealthy Causes: Health check timeout too short, threshold too low, network instability Solutions: Increase interval and timeout, implement hysteresis, use deep health checks
Session Loss After Failover
Symptoms: Users logged out when server fails Causes: Sticky sessions without replication, in-memory sessions Solutions: Implement shared session store (Redis), use client-side tokens (JWT)
Integration Points
Related Skills:
infrastructure-as-code- Deploy load balancers via Terraform/Pulumikubernetes-operations- Ingress controllers for K8s traffic managementnetwork-architecture- Network design and topology for load balancingdeploying-applications- Blue-green and canary deployments via load balancersobservability- Load balancer metrics, access logs, distributed tracingsecurity-hardening- WAF integration, rate limiting, DDoS protectionservice-mesh- Envoy as both ingress and service mesh proxyimplementing-tls- TLS termination and certificate management
Quick Reference
Selection Matrix
| Use Case | Recommended Solution |
|---|---|
| HTTP web app (AWS) | ALB |
| Non-HTTP protocol (AWS) | NLB |
| Kubernetes HTTP ingress | NGINX Ingress or Traefik |
| Maximum performance | HAProxy |
| Service mesh | Envoy |
| Docker Swarm | Traefik |
| Multi-cloud portable | NGINX or HAProxy |
| Global distribution | CloudFlare, AWS Global Accelerator |
Algorithm Selection
| Traffic Pattern | Algorithm |
|---|---|
| Stateless, similar servers | Round Robin |
| Stateless, different capacity | Weighted Round Robin |
| Long-lived connections | Least Connections |
| Performance-sensitive | Least Response Time |
| Session persistence needed | IP Hash or Cookie |
| Varying server load | Resource-Based |
Health Check Configuration
| Service Type | Check Type | Interval | Timeout |
|---|---|---|---|
| Web app | HTTP /health | 10s | 3s |
| API | HTTP /health/ready | 10s | 5s |
| Database | TCP connect | 5s | 2s |
| Critical service | HTTP deep check | 5s | 3s |
| Background worker | HTTP /live | 30s | 5s |
Summary
Load balancing is essential for distributing traffic, ensuring high availability, and enabling horizontal scaling. Choose L4 for raw performance and non-HTTP protocols, L7 for intelligent content-based routing. Prefer cloud-managed load balancers for simplicity and auto-scaling, self-managed for multi-cloud portability and advanced features. Implement proper health checks with hysteresis, avoid sticky sessions when possible, and monitor key metrics continuously.
For deployment patterns, see examples in examples/aws/, examples/nginx/, examples/kubernetes/, and other provider directories.
# AWS Application Load Balancer (ALB) - Layer 7 HTTP Load Balancing
#
# This example demonstrates:
# - ALB with HTTPS listener
# - Path-based routing to multiple target groups
# - Health checks with custom endpoints
# - Cookie-based sticky sessions
# - SSL/TLS termination
#
# Dependencies:
# - terraform >= 1.0
# - AWS provider configured
# - VPC and subnets created
# - ACM certificate for SSL
# Application Load Balancer
resource "aws_lb" "application" {
name = "app-alb"
internal = false
load_balancer_type = "application"
security_groups = [aws_security_group.alb.id]
subnets = aws_subnet.public[*].id
enable_deletion_protection = true
enable_http2 = true
enable_cross_zone_load_balancing = true
access_logs {
bucket = aws_s3_bucket.lb_logs.id
prefix = "alb"
enabled = true
}
tags = {
Name = "app-alb"
Environment = "production"
}
}
# Target Group: Web Application
resource "aws_lb_target_group" "web" {
name = "web-targets"
port = 80
protocol = "HTTP"
vpc_id = aws_vpc.main.id
health_check {
enabled = true
healthy_threshold = 2
unhealthy_threshold = 3
timeout = 5
interval = 30
path = "/health"
matcher = "200"
protocol = "HTTP"
}
deregistration_delay = 30
stickiness {
type = "lb_cookie"
cookie_duration = 86400 # 24 hours
enabled = true
}
tags = {
Name = "web-targets"
}
}
# Target Group: API
resource "aws_lb_target_group" "api" {
name = "api-targets"
port = 8080
protocol = "HTTP"
vpc_id = aws_vpc.main.id
health_check {
enabled = true
healthy_threshold = 2
unhealthy_threshold = 3
timeout = 5
interval = 15 # More frequent for API
path = "/api/health"
matcher = "200,204"
protocol = "HTTP"
}
deregistration_delay = 10 # Faster for stateless API
# Slow start for gradual traffic ramp-up
slow_start = 30
tags = {
Name = "api-targets"
}
}
# HTTPS Listener (primary)
resource "aws_lb_listener" "https" {
load_balancer_arn = aws_lb.application.arn
port = "443"
protocol = "HTTPS"
ssl_policy = "ELBSecurityPolicy-TLS-1-2-2017-01"
certificate_arn = aws_acm_certificate.main.arn
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.web.arn
}
}
# HTTP Listener (redirect to HTTPS)
resource "aws_lb_listener" "http" {
load_balancer_arn = aws_lb.application.arn
port = "80"
protocol = "HTTP"
default_action {
type = "redirect"
redirect {
port = "443"
protocol = "HTTPS"
status_code = "HTTP_301"
}
}
}
# Listener Rule: API path-based routing
resource "aws_lb_listener_rule" "api" {
listener_arn = aws_lb_listener.https.arn
priority = 100
action {
type = "forward"
target_group_arn = aws_lb_target_group.api.arn
}
condition {
path_pattern {
values = ["/api/*"]
}
}
}
# Listener Rule: Admin host-based routing
resource "aws_lb_listener_rule" "admin" {
listener_arn = aws_lb_listener.https.arn
priority = 90
action {
type = "forward"
target_group_arn = aws_lb_target_group.admin.arn
}
condition {
host_header {
values = ["admin.example.com"]
}
}
}
# Listener Rule: Header-based routing (API versioning)
resource "aws_lb_listener_rule" "api_v2" {
listener_arn = aws_lb_listener.https.arn
priority = 80
action {
type = "forward"
target_group_arn = aws_lb_target_group.api_v2.arn
}
condition {
path_pattern {
values = ["/api/*"]
}
}
condition {
http_header {
http_header_name = "X-API-Version"
values = ["v2"]
}
}
}
# Security Group for ALB
resource "aws_security_group" "alb" {
name = "alb-sg"
description = "Security group for Application Load Balancer"
vpc_id = aws_vpc.main.id
ingress {
description = "HTTPS from internet"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
ingress {
description = "HTTP from internet (redirect to HTTPS)"
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
egress {
description = "Allow all outbound"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "alb-sg"
}
}
# Outputs
output "alb_dns_name" {
description = "DNS name of the Application Load Balancer"
value = aws_lb.application.dns_name
}
output "alb_zone_id" {
description = "Route 53 zone ID for ALB"
value = aws_lb.application.zone_id
}
output "alb_arn" {
description = "ARN of the Application Load Balancer"
value = aws_lb.application.arn
}
# AWS Network Load Balancer (NLB) Configuration with Terraform
# This configuration demonstrates production-ready NLB setup with health checks and cross-zone load balancing
# Terraform and provider configuration
terraform {
required_version = ">= 1.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
# Variables for configuration
variable "vpc_id" {
description = "VPC ID where the NLB will be deployed"
type = string
}
variable "subnet_ids" {
description = "List of subnet IDs for the NLB (must be in different AZs)"
type = list(string)
}
variable "environment" {
description = "Environment name (e.g., production, staging)"
type = string
default = "production"
}
# Data source to get VPC information
data "aws_vpc" "main" {
id = var.vpc_id
}
#---------------------------------------------------------------------
# Network Load Balancer
#---------------------------------------------------------------------
resource "aws_lb" "main" {
name = "${var.environment}-app-nlb"
internal = false # Set to true for internal NLB
load_balancer_type = "network"
# Enable cross-zone load balancing for even distribution
enable_cross_zone_load_balancing = true
# Enable deletion protection for production
enable_deletion_protection = var.environment == "production" ? true : false
# Subnet mappings (must span multiple AZs for high availability)
# NLB requires at least 2 subnets in different availability zones
dynamic "subnet_mapping" {
for_each = var.subnet_ids
content {
subnet_id = subnet_mapping.value
# Optional: Assign static IP per AZ
# allocation_id = aws_eip.nlb[subnet_mapping.key].id
}
}
# Enable access logs to S3 (optional but recommended)
# access_logs {
# bucket = aws_s3_bucket.nlb_logs.id
# prefix = "nlb"
# enabled = true
# }
tags = {
Name = "${var.environment}-app-nlb"
Environment = var.environment
ManagedBy = "Terraform"
Service = "LoadBalancing"
}
}
#---------------------------------------------------------------------
# Target Group: TCP (for general TCP traffic)
#---------------------------------------------------------------------
resource "aws_lb_target_group" "tcp" {
name = "${var.environment}-tcp-tg"
port = 8080
protocol = "TCP"
vpc_id = var.vpc_id
# Target type: instance | ip | alb | lambda
target_type = "instance"
# Deregistration delay (time to drain connections)
deregistration_delay = 30
# Health check configuration
health_check {
enabled = true
interval = 30 # Check every 30 seconds
port = "traffic-port" # Use same port as target
protocol = "TCP" # TCP health check
healthy_threshold = 3 # 3 successful checks = healthy
unhealthy_threshold = 3 # 3 failed checks = unhealthy
}
# Connection termination on deregistration
connection_termination = true
# Preserve client IP address
preserve_client_ip = true
tags = {
Name = "${var.environment}-tcp-target-group"
Environment = var.environment
Protocol = "TCP"
}
}
#---------------------------------------------------------------------
# Target Group: TLS (for encrypted TCP traffic)
#---------------------------------------------------------------------
resource "aws_lb_target_group" "tls" {
name = "${var.environment}-tls-tg"
port = 443
protocol = "TLS"
vpc_id = var.vpc_id
target_type = "instance"
# TLS-specific settings
# Proxy protocol v2 adds connection information to the TCP stream
proxy_protocol_v2 = false
health_check {
enabled = true
interval = 30
port = 443
protocol = "TCP" # Can also use "HTTPS" for application-layer health checks
healthy_threshold = 3
unhealthy_threshold = 3
# Optional: HTTP/HTTPS health check parameters
# protocol = "HTTPS"
# path = "/health"
# matcher = "200-299"
}
tags = {
Name = "${var.environment}-tls-target-group"
Environment = var.environment
Protocol = "TLS"
}
}
#---------------------------------------------------------------------
# Target Group: UDP (for UDP traffic like DNS, QUIC)
#---------------------------------------------------------------------
resource "aws_lb_target_group" "udp" {
name = "${var.environment}-udp-tg"
port = 53
protocol = "UDP"
vpc_id = var.vpc_id
target_type = "instance"
health_check {
enabled = true
interval = 30
port = 53
protocol = "TCP" # UDP health checks use TCP or HTTP(S)
healthy_threshold = 3
unhealthy_threshold = 3
}
tags = {
Name = "${var.environment}-udp-target-group"
Environment = var.environment
Protocol = "UDP"
}
}
#---------------------------------------------------------------------
# Listener: TCP on port 80
#---------------------------------------------------------------------
resource "aws_lb_listener" "tcp" {
load_balancer_arn = aws_lb.main.arn
port = 80
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.tcp.arn
}
tags = {
Name = "${var.environment}-tcp-listener"
}
}
#---------------------------------------------------------------------
# Listener: TLS on port 443 with certificate
#---------------------------------------------------------------------
resource "aws_lb_listener" "tls" {
load_balancer_arn = aws_lb.main.arn
port = 443
protocol = "TLS"
certificate_arn = aws_acm_certificate.main.arn
# TLS security policy
# Options: ELBSecurityPolicy-TLS13-1-2-2021-06, ELBSecurityPolicy-TLS-1-2-2017-01, etc.
ssl_policy = "ELBSecurityPolicy-TLS13-1-2-2021-06"
# ALPN policy for HTTP/2, HTTP/1.1
alpn_policy = "HTTP2Preferred"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.tls.arn
}
tags = {
Name = "${var.environment}-tls-listener"
}
}
#---------------------------------------------------------------------
# Listener: UDP on port 53 (DNS example)
#---------------------------------------------------------------------
resource "aws_lb_listener" "udp" {
load_balancer_arn = aws_lb.main.arn
port = 53
protocol = "UDP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.udp.arn
}
tags = {
Name = "${var.environment}-udp-listener"
}
}
#---------------------------------------------------------------------
# Target Group Attachments (register EC2 instances)
#---------------------------------------------------------------------
# Example: Attach existing EC2 instances to target groups
# You would typically use auto-scaling groups instead
resource "aws_lb_target_group_attachment" "tcp_instance_1" {
target_group_arn = aws_lb_target_group.tcp.arn
target_id = aws_instance.app_server_1.id
port = 8080
}
resource "aws_lb_target_group_attachment" "tcp_instance_2" {
target_group_arn = aws_lb_target_group.tcp.arn
target_id = aws_instance.app_server_2.id
port = 8080
}
resource "aws_lb_target_group_attachment" "tls_instance_1" {
target_group_arn = aws_lb_target_group.tls.arn
target_id = aws_instance.app_server_1.id
port = 443
}
resource "aws_lb_target_group_attachment" "tls_instance_2" {
target_group_arn = aws_lb_target_group.tls.arn
target_id = aws_instance.app_server_2.id
port = 443
}
#---------------------------------------------------------------------
# ACM Certificate (for TLS listener)
#---------------------------------------------------------------------
resource "aws_acm_certificate" "main" {
domain_name = "example.com"
validation_method = "DNS"
subject_alternative_names = [
"*.example.com",
]
lifecycle {
create_before_destroy = true
}
tags = {
Name = "${var.environment}-certificate"
Environment = var.environment
}
}
#---------------------------------------------------------------------
# Security Group for NLB targets (EC2 instances)
#---------------------------------------------------------------------
resource "aws_security_group" "nlb_targets" {
name = "${var.environment}-nlb-targets-sg"
description = "Security group for NLB target instances"
vpc_id = var.vpc_id
# Allow TCP traffic on port 8080 from anywhere
ingress {
description = "HTTP from NLB"
from_port = 8080
to_port = 8080
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
# Allow TLS traffic on port 443
ingress {
description = "HTTPS from NLB"
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
}
# Allow UDP traffic on port 53
ingress {
description = "DNS from NLB"
from_port = 53
to_port = 53
protocol = "udp"
cidr_blocks = ["0.0.0.0/0"]
}
# Allow all outbound traffic
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "${var.environment}-nlb-targets-sg"
Environment = var.environment
}
}
#---------------------------------------------------------------------
# Example EC2 Instances (target servers)
#---------------------------------------------------------------------
# These are placeholder instances - in production, use Auto Scaling Groups
data "aws_ami" "amazon_linux_2" {
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["amzn2-ami-hvm-*-x86_64-gp2"]
}
}
resource "aws_instance" "app_server_1" {
ami = data.aws_ami.amazon_linux_2.id
instance_type = "t3.medium"
subnet_id = var.subnet_ids[0]
vpc_security_group_ids = [aws_security_group.nlb_targets.id]
user_data = <<-EOF
#!/bin/bash
yum update -y
yum install -y httpd
systemctl start httpd
systemctl enable httpd
echo "Server 1" > /var/www/html/index.html
EOF
tags = {
Name = "${var.environment}-app-server-1"
Environment = var.environment
}
}
resource "aws_instance" "app_server_2" {
ami = data.aws_ami.amazon_linux_2.id
instance_type = "t3.medium"
subnet_id = var.subnet_ids[1]
vpc_security_group_ids = [aws_security_group.nlb_targets.id]
user_data = <<-EOF
#!/bin/bash
yum update -y
yum install -y httpd
systemctl start httpd
systemctl enable httpd
echo "Server 2" > /var/www/html/index.html
EOF
tags = {
Name = "${var.environment}-app-server-2"
Environment = var.environment
}
}
#---------------------------------------------------------------------
# Outputs
#---------------------------------------------------------------------
output "nlb_dns_name" {
description = "DNS name of the Network Load Balancer"
value = aws_lb.main.dns_name
}
output "nlb_arn" {
description = "ARN of the Network Load Balancer"
value = aws_lb.main.arn
}
output "nlb_zone_id" {
description = "Zone ID of the Network Load Balancer"
value = aws_lb.main.zone_id
}
output "tcp_target_group_arn" {
description = "ARN of the TCP target group"
value = aws_lb_target_group.tcp.arn
}
output "tls_target_group_arn" {
description = "ARN of the TLS target group"
value = aws_lb_target_group.tls.arn
}
output "udp_target_group_arn" {
description = "ARN of the UDP target group"
value = aws_lb_target_group.udp.arn
}
# Envoy Proxy Load Balancer Configuration
# This configuration demonstrates core load balancing patterns with health checking and circuit breaking
static_resources:
listeners:
# HTTP listener on port 8080
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 8080
filter_chains:
- filters:
# HTTP Connection Manager - handles HTTP/1.1 and HTTP/2
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: backend
domains: ["*"]
routes:
# Route all traffic to the service cluster
- match:
prefix: "/"
route:
cluster: service_cluster
# Timeout for the entire request (including retries)
timeout: 15s
# Retry policy for transient failures
retry_policy:
retry_on: "5xx,reset,connect-failure,refused-stream"
num_retries: 3
per_try_timeout: 5s
http_filters:
# Router filter - required for routing
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
clusters:
# Backend service cluster with multiple endpoints
- name: service_cluster
type: STRICT_DNS
# DNS refresh rate
dns_lookup_family: V4_ONLY
# Load balancing policy: ROUND_ROBIN
# Alternatives: LEAST_REQUEST, RANDOM, RING_HASH, MAGLEV
lb_policy: ROUND_ROBIN
# Least Request LB settings (uncomment to use)
# lb_policy: LEAST_REQUEST
# least_request_lb_config:
# choice_count: 2 # P2C (Power of Two Choices)
# Connection pool settings
circuit_breakers:
thresholds:
- priority: DEFAULT
# Maximum connections to all hosts in the cluster
max_connections: 1000
# Maximum pending requests (queued when all connections are in use)
max_pending_requests: 1000
# Maximum concurrent requests
max_requests: 1000
# Maximum active retries
max_retries: 3
# Outlier detection (circuit breaking based on consecutive failures)
outlier_detection:
# Number of consecutive 5xx errors before ejection
consecutive_5xx: 5
# Time interval for checking ejection
interval: 10s
# Minimum percentage of hosts that must be ejected
base_ejection_time: 30s
# Maximum percentage of hosts that can be ejected
max_ejection_percent: 50
# Number of consecutive gateway failures before ejection
consecutive_gateway_failure: 5
# Success rate-based outlier detection
enforcing_consecutive_5xx: 100
enforcing_consecutive_gateway_failure: 100
# Health check configuration
health_checks:
- timeout: 1s
interval: 5s
# Number of healthy checks before marking healthy
healthy_threshold: 2
# Number of unhealthy checks before marking unhealthy
unhealthy_threshold: 3
http_health_check:
path: "/health"
# Expected HTTP status codes for healthy response
expected_statuses:
- start: 200
end: 299
# Upstream connection settings
connect_timeout: 5s
# Backend endpoints (servers)
load_assignment:
cluster_name: service_cluster
endpoints:
# Locality-aware endpoint group (zone A)
- lb_endpoints:
# Backend server 1
- endpoint:
address:
socket_address:
address: backend-1.example.com
port_value: 8000
health_check_config:
port_value: 8000
# Optional: per-endpoint load balancing weight
load_balancing_weight: 1
# Backend server 2
- endpoint:
address:
socket_address:
address: backend-2.example.com
port_value: 8000
health_check_config:
port_value: 8000
load_balancing_weight: 1
# Backend server 3
- endpoint:
address:
socket_address:
address: backend-3.example.com
port_value: 8000
health_check_config:
port_value: 8000
load_balancing_weight: 2 # This server gets 2x traffic
# Locality information for zone-aware routing
locality:
zone: "us-east-1a"
# Admin interface for monitoring and management
admin:
address:
socket_address:
address: 127.0.0.1
port_value: 9901
# HAProxy HTTP Load Balancer Configuration
# Version: 2.8+
# This configuration demonstrates production-ready HTTP/HTTPS load balancing with health checks
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# Logging configuration
log /dev/log local0
log /dev/log local1 notice
# Process management
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon
# Default SSL/TLS settings
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
# Performance tuning
tune.ssl.default-dh-param 2048
maxconn 4096
#---------------------------------------------------------------------
# Default settings for all proxies
#---------------------------------------------------------------------
defaults
log global
mode http
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
# Timeouts
timeout connect 5s
timeout client 50s
timeout server 50s
# Error handling
retries 3
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
#---------------------------------------------------------------------
# Statistics page (password: changeme)
#---------------------------------------------------------------------
listen stats
bind *:8404
stats enable
stats uri /stats
stats refresh 30s
stats auth admin:changeme
stats admin if TRUE
#---------------------------------------------------------------------
# Frontend: HTTP traffic (port 80)
#---------------------------------------------------------------------
frontend http_front
bind *:80
# ACL rules for request routing
# Match requests with /api prefix
acl is_api path_beg /api
# Match requests for specific host
acl is_admin_host hdr(host) -i admin.example.com
# Match requests based on source IP
acl is_internal_ip src 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
# Routing decisions based on ACLs
use_backend api_backend if is_api
use_backend admin_backend if is_admin_host
use_backend internal_backend if is_internal_ip
# Default backend
default_backend web_backend
# Security headers
http-response set-header X-Frame-Options SAMEORIGIN
http-response set-header X-Content-Type-Options nosniff
http-response set-header X-XSS-Protection "1; mode=block"
#---------------------------------------------------------------------
# Frontend: HTTPS traffic (port 443)
#---------------------------------------------------------------------
frontend https_front
bind *:443 ssl crt /etc/haproxy/certs/example.com.pem alpn h2,http/1.1
# Redirect HTTP to HTTPS
http-request redirect scheme https unless { ssl_fc }
# ACL rules (same as HTTP frontend)
acl is_api path_beg /api
acl is_admin_host hdr(host) -i admin.example.com
acl is_internal_ip src 10.0.0.0/8
# Routing
use_backend api_backend if is_api
use_backend admin_backend if is_admin_host
use_backend internal_backend if is_internal_ip
default_backend web_backend
# HSTS header (6 months)
http-response set-header Strict-Transport-Security "max-age=15768000; includeSubDomains"
#---------------------------------------------------------------------
# Backend: Web servers (round-robin with session persistence)
#---------------------------------------------------------------------
backend web_backend
# Load balancing algorithm: round-robin
# Alternatives: leastconn, source, uri, random
balance roundrobin
# Session persistence using cookies
cookie SERVERID insert indirect nocache httponly secure
# Health check: HTTP GET to /health
option httpchk GET /health HTTP/1.1\r\nHost:\ www.example.com
http-check expect status 200
# Server definitions
# server <name> <address>:<port> [options]
server web1 10.0.1.10:8080 check cookie web1 weight 100 maxconn 500
server web2 10.0.1.11:8080 check cookie web2 weight 100 maxconn 500
server web3 10.0.1.12:8080 check cookie web3 weight 100 maxconn 500
# Backup server (only used when all others are down)
server web4 10.0.1.13:8080 check backup
#---------------------------------------------------------------------
# Backend: API servers (least connections)
#---------------------------------------------------------------------
backend api_backend
# Use least connections algorithm for API traffic
balance leastconn
# Stick table for rate limiting (per source IP)
stick-table type ip size 100k expire 30s store conn_rate(10s)
# Track connection rate
tcp-request connection track-sc0 src
# Deny if more than 100 connections in 10 seconds
tcp-request connection reject if { sc_conn_rate(0) gt 100 }
# Health check: TCP check with HTTP validation
option httpchk GET /api/health HTTP/1.1\r\nHost:\ api.example.com
http-check expect string "healthy"
# Compression for API responses
compression algo gzip
compression type application/json text/plain text/html
# Server definitions with inter-check intervals
server api1 10.0.2.10:8080 check inter 2000 rise 2 fall 3
server api2 10.0.2.11:8080 check inter 2000 rise 2 fall 3
server api3 10.0.2.12:8080 check inter 2000 rise 2 fall 3
server api4 10.0.2.13:8080 check inter 2000 rise 2 fall 3
#---------------------------------------------------------------------
# Backend: Admin servers (source IP hash for session persistence)
#---------------------------------------------------------------------
backend admin_backend
# Source IP hash - same client always goes to same server
balance source
# TCP health check on port 8080
option tcp-check
tcp-check connect port 8080
# Longer timeout for admin operations
timeout server 120s
# Server definitions
server admin1 10.0.3.10:8080 check
server admin2 10.0.3.11:8080 check
#---------------------------------------------------------------------
# Backend: Internal services (URI hash for cache efficiency)
#---------------------------------------------------------------------
backend internal_backend
# URI hash - same URI always goes to same server (cache locality)
balance uri
# Health check: HTTP with custom header
option httpchk GET /internal/health HTTP/1.1\r\nHost:\ internal.example.com\r\nX-Health-Check:\ true
http-check expect status 200-299
# Server definitions with weight adjustment
server internal1 10.0.4.10:8080 check weight 50
server internal2 10.0.4.11:8080 check weight 100
server internal3 10.0.4.12:8080 check weight 150
#---------------------------------------------------------------------
# Additional backend examples
#---------------------------------------------------------------------
# Backend with custom health check intervals
backend custom_health_backend
balance roundrobin
# Custom health check parameters:
# inter: interval between checks (default: 2000ms)
# rise: number of successful checks before marking server up (default: 2)
# fall: number of failed checks before marking server down (default: 3)
# downinter: interval when server is down (faster detection)
server srv1 10.0.5.10:8080 check inter 5000 rise 2 fall 3 downinter 1000
server srv2 10.0.5.11:8080 check inter 5000 rise 2 fall 3 downinter 1000
# Backend with connection limits and queuing
backend limited_backend
balance roundrobin
# Global backend connection limit
fullconn 1000
# Server with maxconn and maxqueue limits
server lim1 10.0.6.10:8080 check maxconn 250 maxqueue 50
server lim2 10.0.6.11:8080 check maxconn 250 maxqueue 50
# Kubernetes Gateway API Configuration
# Gateway API is the successor to Ingress, providing more expressive traffic routing
# Requires Gateway API CRDs installed: kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml
---
# GatewayClass: Defines the controller implementation
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: traefik-gateway-class
spec:
controllerName: traefik.io/gateway-controller
description: "Traefik-based Gateway implementation"
---
# Gateway: Entry point for traffic (replaces LoadBalancer/NodePort services)
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: production-gateway
namespace: default
labels:
environment: production
spec:
gatewayClassName: traefik-gateway-class
# Listeners define the ports and protocols
listeners:
# HTTP listener (port 80)
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: All # Allow routes from all namespaces
# HTTPS listener (port 443)
- name: https
protocol: HTTPS
port: 443
allowedRoutes:
namespaces:
from: All
tls:
mode: Terminate
certificateRefs:
- kind: Secret
name: example-com-tls
namespace: default
# Additional HTTPS listener with SNI
- name: https-api
protocol: HTTPS
port: 443
hostname: api.example.com
allowedRoutes:
namespaces:
from: Same
tls:
mode: Terminate
certificateRefs:
- kind: Secret
name: api-example-com-tls
---
# HTTPRoute: Main application routing with load balancing
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: app-route
namespace: default
labels:
app: web-application
spec:
# Attach to the Gateway
parentRefs:
- name: production-gateway
namespace: default
sectionName: https
# Hostname matching
hostnames:
- "example.com"
- "www.example.com"
# Routing rules
rules:
# Rule 1: Homepage with round-robin load balancing
- matches:
- path:
type: PathPrefix
value: /
headers:
- name: X-Version
value: stable
backendRefs:
# Multiple backends for load balancing
- name: web-frontend-v1
port: 80
weight: 100 # All traffic to v1
filters:
# Add response header
- type: ResponseHeaderModifier
responseHeaderModifier:
add:
- name: X-Backend-Version
value: v1
# Rule 2: API path with weighted traffic splitting (canary)
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
# Weighted load balancing: 90% to v1, 10% to v2
- name: api-backend-v1
port: 8080
weight: 90
- name: api-backend-v2
port: 8080
weight: 10
filters:
# Strip /api prefix before forwarding
- type: URLRewrite
urlRewrite:
path:
type: ReplacePrefixMatch
replacePrefixMatch: /
# Rule 3: Static content with header-based routing
- matches:
- path:
type: PathPrefix
value: /static
headers:
- name: X-Cache-Control
type: Exact
value: no-cache
backendRefs:
- name: static-backend-nocache
port: 80
weight: 100
# Rule 4: Exact path match for health check
- matches:
- path:
type: Exact
value: /health
backendRefs:
- name: health-check-backend
port: 8080
---
# HTTPRoute: API subdomain with advanced routing
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: api-route
namespace: default
spec:
parentRefs:
- name: production-gateway
sectionName: https-api
hostnames:
- "api.example.com"
rules:
# Rule 1: Versioned API routing (v1)
- matches:
- path:
type: PathPrefix
value: /v1
backendRefs:
- name: api-backend-v1
port: 8080
filters:
# Add API version header
- type: RequestHeaderModifier
requestHeaderModifier:
add:
- name: X-API-Version
value: "1"
remove:
- Authorization # Example: remove sensitive headers
# Rule 2: Versioned API routing (v2)
- matches:
- path:
type: PathPrefix
value: /v2
backendRefs:
- name: api-backend-v2
port: 8080
filters:
- type: RequestHeaderModifier
requestHeaderModifier:
add:
- name: X-API-Version
value: "2"
# Rule 3: Query parameter-based routing
- matches:
- path:
type: PathPrefix
value: /search
queryParams:
- name: beta
value: "true"
backendRefs:
- name: api-backend-v2-beta
port: 8080
# Rule 4: Default route (no version specified)
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: api-backend-v1
port: 8080
weight: 80
- name: api-backend-v2
port: 8080
weight: 20
---
# HTTPRoute: Traffic mirroring for testing
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: mirrored-route
namespace: default
spec:
parentRefs:
- name: production-gateway
hostnames:
- "test.example.com"
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
# Primary backend
- name: api-backend-v1
port: 8080
weight: 100
# Mirror backend (responses ignored)
- name: api-backend-v2
port: 8080
weight: 0 # Weight 0 means mirror only
filters:
- type: RequestMirror
requestMirror:
backendRef:
name: api-backend-v2
port: 8080
---
# HTTPRoute: Redirect rules
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: redirect-route
namespace: default
spec:
parentRefs:
- name: production-gateway
sectionName: http # HTTP listener
hostnames:
- "example.com"
rules:
# Redirect HTTP to HTTPS
- matches:
- path:
type: PathPrefix
value: /
filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
---
# HTTPRoute: Header-based routing for A/B testing
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: ab-test-route
namespace: default
spec:
parentRefs:
- name: production-gateway
hostnames:
- "app.example.com"
rules:
# Rule 1: Beta users (based on cookie)
- matches:
- path:
type: PathPrefix
value: /
headers:
- name: Cookie
type: RegularExpression
value: ".*beta=true.*"
backendRefs:
- name: app-backend-beta
port: 80
# Rule 2: Premium users (based on header)
- matches:
- path:
type: PathPrefix
value: /
headers:
- name: X-User-Tier
value: premium
backendRefs:
- name: app-backend-premium
port: 80
# Rule 3: Default users
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: app-backend-standard
port: 80
---
# HTTPRoute: Method-based routing
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: method-route
namespace: default
spec:
parentRefs:
- name: production-gateway
hostnames:
- "api.example.com"
rules:
# Route GET requests to read-only backend
- matches:
- path:
type: PathPrefix
value: /data
method: GET
backendRefs:
- name: api-backend-readonly
port: 8080
# Route POST/PUT/DELETE to read-write backend
- matches:
- path:
type: PathPrefix
value: /data
method: POST
backendRefs:
- name: api-backend-readwrite
port: 8080
- matches:
- path:
type: PathPrefix
value: /data
method: PUT
backendRefs:
- name: api-backend-readwrite
port: 8080
- matches:
- path:
type: PathPrefix
value: /data
method: DELETE
backendRefs:
- name: api-backend-readwrite
port: 8080
---
# ReferenceGrant: Allow cross-namespace references
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
name: allow-gateway-to-backend
namespace: backend-namespace
spec:
from:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: default
to:
- group: ""
kind: Service
---
# Backend Services (Kubernetes Services)
---
apiVersion: v1
kind: Service
metadata:
name: web-frontend-v1
namespace: default
spec:
selector:
app: web-frontend
version: v1
ports:
- port: 80
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-backend-v1
namespace: default
spec:
selector:
app: api-backend
version: v1
ports:
- port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-backend-v2
namespace: default
spec:
selector:
app: api-backend
version: v2
ports:
- port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-backend-v2-beta
namespace: default
spec:
selector:
app: api-backend
version: v2-beta
ports:
- port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: static-backend-nocache
namespace: default
spec:
selector:
app: static-assets
cache: disabled
ports:
- port: 80
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: health-check-backend
namespace: default
spec:
selector:
app: health-check
ports:
- port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: app-backend-beta
namespace: default
spec:
selector:
app: web-app
version: beta
ports:
- port: 80
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: app-backend-premium
namespace: default
spec:
selector:
app: web-app
tier: premium
ports:
- port: 80
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: app-backend-standard
namespace: default
spec:
selector:
app: web-app
tier: standard
ports:
- port: 80
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-backend-readonly
namespace: default
spec:
selector:
app: api-backend
mode: readonly
ports:
- port: 8080
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: api-backend-readwrite
namespace: default
spec:
selector:
app: api-backend
mode: readwrite
ports:
- port: 8080
targetPort: 8080
---
# TLS Secret for HTTPS
apiVersion: v1
kind: Secret
metadata:
name: example-com-tls
namespace: default
type: kubernetes.io/tls
data:
tls.crt: LS0tLS1CRUdJTi... # Base64 encoded certificate
tls.key: LS0tLS1CRUdJTi... # Base64 encoded private key
---
apiVersion: v1
kind: Secret
metadata:
name: api-example-com-tls
namespace: default
type: kubernetes.io/tls
data:
tls.crt: LS0tLS1CRUdJTi...
tls.key: LS0tLS1CRUdJTi...
# Kubernetes NGINX Ingress Controller Example
#
# This example demonstrates:
# - Basic Ingress resource with path-based routing
# - TLS/SSL termination
# - Sticky sessions (cookie-based affinity)
# - Custom annotations for NGINX configuration
# - Multiple services with different routing rules
#
# Prerequisites:
# - Kubernetes cluster
# - NGINX Ingress Controller installed
# - cert-manager for TLS certificate management (optional)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
namespace: default
annotations:
# Use NGINX Ingress Controller
kubernetes.io/ingress.class: nginx
# SSL redirect
nginx.ingress.kubernetes.io/ssl-redirect: "true"
# Rewrite target (strip path prefix)
# nginx.ingress.kubernetes.io/rewrite-target: /
# CORS configuration
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, PUT, DELETE, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-origin: "https://example.com"
# Rate limiting
nginx.ingress.kubernetes.io/limit-rps: "100"
nginx.ingress.kubernetes.io/limit-burst-multiplier: "5"
# Timeouts
nginx.ingress.kubernetes.io/proxy-connect-timeout: "5"
nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
# Body size limit
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
# Session affinity (sticky sessions)
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/affinity-mode: "persistent"
nginx.ingress.kubernetes.io/session-cookie-name: "app_session"
nginx.ingress.kubernetes.io/session-cookie-max-age: "3600"
nginx.ingress.kubernetes.io/session-cookie-path: "/"
nginx.ingress.kubernetes.io/session-cookie-samesite: "Strict"
# Load balancing algorithm
nginx.ingress.kubernetes.io/load-balance: "least_conn"
# cert-manager certificate
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
ingressClassName: nginx
# TLS configuration
tls:
- hosts:
- app.example.com
- api.example.com
secretName: app-tls-cert
# Routing rules
rules:
# Main application domain
- host: app.example.com
http:
paths:
# API routes
- path: /api/v1
pathType: Prefix
backend:
service:
name: api-service-v1
port:
number: 80
- path: /api/v2
pathType: Prefix
backend:
service:
name: api-service-v2
port:
number: 80
# Static assets
- path: /static
pathType: Prefix
backend:
service:
name: static-service
port:
number: 80
# WebSocket endpoint
- path: /ws
pathType: Prefix
backend:
service:
name: websocket-service
port:
number: 8080
# Default: Web application
- path: /
pathType: Prefix
backend:
service:
name: web-service
port:
number: 80
# API subdomain
- host: api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-service-v1
port:
number: 80
---
# Service: Web Application
apiVersion: v1
kind: Service
metadata:
name: web-service
namespace: default
spec:
type: ClusterIP
selector:
app: web
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
---
# Service: API v1
apiVersion: v1
kind: Service
metadata:
name: api-service-v1
namespace: default
spec:
type: ClusterIP
selector:
app: api
version: v1
ports:
- port: 80
targetPort: 8080
protocol: TCP
name: http
---
# Deployment: Web Application (with health checks)
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-deployment
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: myapp/web:latest
ports:
- containerPort: 8080
name: http
# Liveness probe
livenessProbe:
httpGet:
path: /health/live
port: 8080
httpHeaders:
- name: X-Health-Check
value: kubernetes
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
# Readiness probe
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 3
successThreshold: 1
failureThreshold: 3
# Startup probe (for slow-starting apps)
startupProbe:
httpGet:
path: /health/startup
port: 8080
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 3
failureThreshold: 30 # Allow up to 5 minutes for startup
# Resource limits
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
# Environment variables
env:
- name: PORT
value: "8080"
- name: LOG_LEVEL
value: "info"
---
# Alternative: Ingress with multiple services and weighted routing
# (Requires NGINX Ingress Controller with canary support)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress-canary
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "10" # 10% traffic to canary
spec:
ingressClassName: nginx
rules:
- host: app.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-service-canary
port:
number: 80
# Traefik IngressRoute Configuration for Kubernetes
# Demonstrates advanced load balancing with Traefik CRDs including middleware and traffic management
---
# IngressRoute: HTTP to HTTPS redirect
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: http-redirect
namespace: default
labels:
app: web-application
component: ingress
spec:
entryPoints:
- web # HTTP entry point (port 80)
routes:
# Catch-all route to redirect HTTP to HTTPS
- match: HostRegexp(`{host:.+}`)
kind: Rule
services:
- name: noop@internal
kind: TraefikService
middlewares:
- name: redirect-to-https
---
# IngressRoute: Main HTTPS traffic with load balancing
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: web-app-https
namespace: default
labels:
app: web-application
component: ingress
annotations:
# Optional: External DNS annotation
external-dns.alpha.kubernetes.io/hostname: example.com
spec:
entryPoints:
- websecure # HTTPS entry point (port 443)
routes:
# Route 1: Homepage and static content
- match: Host(`example.com`) && PathPrefix(`/`)
kind: Rule
services:
# Primary backend service
- name: web-frontend
port: 80
weight: 10 # Traffic weight for load distribution
# Health check configuration
healthCheck:
path: /health
intervalSeconds: 10
timeoutSeconds: 3
# Load balancing strategy: wrr (Weighted Round Robin)
strategy: RoundRobin
middlewares:
- name: security-headers
- name: compression
- name: rate-limit
# Route 2: API traffic with different middleware
- match: Host(`example.com`) && PathPrefix(`/api`)
kind: Rule
priority: 100 # Higher priority routes are evaluated first
services:
# Multiple backends for API with weighted load balancing
- name: api-backend-v1
port: 8080
weight: 80 # 80% of traffic
- name: api-backend-v2
port: 8080
weight: 20 # 20% of traffic (canary deployment)
middlewares:
- name: api-auth
- name: api-rate-limit
- name: cors-headers
- name: strip-api-prefix
# Route 3: WebSocket traffic
- match: Host(`example.com`) && PathPrefix(`/ws`)
kind: Rule
services:
- name: websocket-backend
port: 8080
middlewares:
- name: websocket-headers
# TLS configuration
tls:
secretName: example-com-tls
# Optional: TLS options reference
options:
name: default
---
# IngressRoute: Multiple host routing
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: multi-host-routing
namespace: default
spec:
entryPoints:
- websecure
routes:
# Admin subdomain
- match: Host(`admin.example.com`)
kind: Rule
services:
- name: admin-backend
port: 80
middlewares:
- name: admin-auth
- name: ip-whitelist
# API subdomain
- match: Host(`api.example.com`)
kind: Rule
services:
- name: api-backend-v1
port: 8080
middlewares:
- name: api-rate-limit
tls:
secretName: wildcard-example-com-tls
---
# Middleware: HTTPS redirect
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: redirect-to-https
namespace: default
spec:
redirectScheme:
scheme: https
permanent: true
port: "443"
---
# Middleware: Security headers
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: security-headers
namespace: default
spec:
headers:
# Security headers
sslRedirect: true
stsSeconds: 31536000 # HSTS: 1 year
stsIncludeSubdomains: true
stsPreload: true
forceSTSHeader: true
frameDeny: true
contentTypeNosniff: true
browserXssFilter: true
customResponseHeaders:
X-Content-Type-Options: "nosniff"
X-Frame-Options: "SAMEORIGIN"
X-XSS-Protection: "1; mode=block"
Referrer-Policy: "strict-origin-when-cross-origin"
---
# Middleware: Compression
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: compression
namespace: default
spec:
compress:
excludedContentTypes:
- "text/event-stream"
- "application/grpc"
---
# Middleware: Rate limiting (per IP)
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: rate-limit
namespace: default
spec:
rateLimit:
average: 100 # Average requests per second
period: 1m
burst: 200 # Maximum burst size
---
# Middleware: API-specific rate limiting
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: api-rate-limit
namespace: default
spec:
rateLimit:
average: 50
period: 1m
burst: 100
---
# Middleware: CORS headers
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: cors-headers
namespace: default
spec:
headers:
accessControlAllowMethods:
- "GET"
- "POST"
- "PUT"
- "DELETE"
- "OPTIONS"
accessControlAllowOriginList:
- "https://example.com"
- "https://app.example.com"
accessControlAllowHeaders:
- "Content-Type"
- "Authorization"
- "X-Requested-With"
accessControlExposeHeaders:
- "Content-Length"
- "Content-Range"
accessControlAllowCredentials: true
accessControlMaxAge: 3600
---
# Middleware: Strip API prefix
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: strip-api-prefix
namespace: default
spec:
stripPrefix:
prefixes:
- "/api"
forceSlash: false
---
# Middleware: Basic authentication
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: api-auth
namespace: default
spec:
basicAuth:
secret: api-auth-credentials # Reference to Kubernetes Secret
---
# Middleware: Admin authentication
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: admin-auth
namespace: default
spec:
basicAuth:
secret: admin-auth-credentials
---
# Middleware: IP whitelist for admin access
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: ip-whitelist
namespace: default
spec:
ipWhiteList:
sourceRange:
- "10.0.0.0/8" # Internal network
- "192.168.0.0/16" # Private network
- "172.16.0.0/12" # Private network
---
# Middleware: WebSocket headers
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: websocket-headers
namespace: default
spec:
headers:
customRequestHeaders:
X-Forwarded-Proto: "https"
X-Forwarded-For: ""
---
# TLS Options: Custom TLS configuration
apiVersion: traefik.io/v1alpha1
kind: TLSOption
metadata:
name: default
namespace: default
spec:
minVersion: VersionTLS12
maxVersion: VersionTLS13
cipherSuites:
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
curvePreferences:
- CurveP521
- CurveP384
sniStrict: true
---
# Service: Web frontend deployment
apiVersion: v1
kind: Service
metadata:
name: web-frontend
namespace: default
labels:
app: web-frontend
spec:
type: ClusterIP
selector:
app: web-frontend
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
---
# Service: API backend v1
apiVersion: v1
kind: Service
metadata:
name: api-backend-v1
namespace: default
labels:
app: api-backend
version: v1
spec:
type: ClusterIP
selector:
app: api-backend
version: v1
ports:
- name: http
port: 8080
targetPort: 8080
protocol: TCP
---
# Service: API backend v2 (canary)
apiVersion: v1
kind: Service
metadata:
name: api-backend-v2
namespace: default
labels:
app: api-backend
version: v2
spec:
type: ClusterIP
selector:
app: api-backend
version: v2
ports:
- name: http
port: 8080
targetPort: 8080
protocol: TCP
---
# Service: WebSocket backend
apiVersion: v1
kind: Service
metadata:
name: websocket-backend
namespace: default
labels:
app: websocket-backend
spec:
type: ClusterIP
selector:
app: websocket-backend
ports:
- name: http
port: 8080
targetPort: 8080
protocol: TCP
---
# Service: Admin backend
apiVersion: v1
kind: Service
metadata:
name: admin-backend
namespace: default
labels:
app: admin-backend
spec:
type: ClusterIP
selector:
app: admin-backend
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
---
# Secret: TLS certificate
apiVersion: v1
kind: Secret
metadata:
name: example-com-tls
namespace: default
type: kubernetes.io/tls
data:
# Base64-encoded certificate and key
tls.crt: LS0tLS1CRUdJTi... # Your certificate here
tls.key: LS0tLS1CRUdJTi... # Your private key here
---
# Secret: API authentication credentials
apiVersion: v1
kind: Secret
metadata:
name: api-auth-credentials
namespace: default
type: Opaque
stringData:
# Format: username:password (hashed with htpasswd)
users: |
api_user:$apr1$xyz...
# NGINX HTTP Load Balancing Configuration
#
# This example demonstrates:
# - Multiple upstream server pools
# - Different load balancing algorithms
# - Health checks (passive in open source, active in NGINX Plus)
# - Sticky sessions
# - Connection keepalive
# - SSL/TLS termination
#
# Place in: /etc/nginx/conf.d/load-balancer.conf
# Upstream: Web servers (round-robin)
upstream web_backend {
# Load balancing method: round-robin (default)
# Alternatives: least_conn, ip_hash, hash $variable [consistent]
server web1.example.com:8080 weight=3 max_fails=3 fail_timeout=30s;
server web2.example.com:8080 weight=2 max_fails=3 fail_timeout=30s;
server web3.example.com:8080 weight=1 backup; # Backup server
# Keepalive connections to upstream
keepalive 32;
keepalive_requests 100;
keepalive_timeout 60s;
}
# Upstream: API servers (least connections)
upstream api_backend {
least_conn; # Route to server with fewest active connections
server api1.example.com:8080 max_fails=2 fail_timeout=10s;
server api2.example.com:8080 max_fails=2 fail_timeout=10s;
server api3.example.com:8080 max_fails=2 fail_timeout=10s;
keepalive 64;
}
# Upstream: Static file servers (IP hash for cache affinity)
upstream static_backend {
ip_hash; # Same client IP always routed to same server
server static1.example.com:8080;
server static2.example.com:8080;
server static3.example.com:8080;
}
# Server: HTTP (redirect to HTTPS)
server {
listen 80;
listen [::]:80;
server_name example.com www.example.com;
# Redirect all HTTP to HTTPS
return 301 https://$server_name$request_uri;
}
# Server: HTTPS
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name example.com www.example.com;
# SSL/TLS Configuration
ssl_certificate /etc/nginx/ssl/example.com.crt;
ssl_certificate_key /etc/nginx/ssl/example.com.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
# Security headers
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
# Logging
access_log /var/log/nginx/lb_access.log combined;
error_log /var/log/nginx/lb_error.log warn;
# Location: API (to API backend)
location /api/ {
proxy_pass http://api_backend;
# Headers for backend
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
# Timeouts
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
# Keepalive to upstream
proxy_http_version 1.1;
proxy_set_header Connection "";
# Buffering
proxy_buffering on;
proxy_buffer_size 4k;
proxy_buffers 8 4k;
proxy_busy_buffers_size 8k;
# Error handling
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503;
proxy_next_upstream_tries 2;
proxy_next_upstream_timeout 10s;
}
# Location: Static files (to static backend)
location /static/ {
proxy_pass http://static_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Caching at load balancer
proxy_cache static_cache;
proxy_cache_valid 200 1h;
proxy_cache_valid 404 1m;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
add_header X-Cache-Status $upstream_cache_status;
# Long timeouts for large files
proxy_read_timeout 300s;
}
# Location: Default (to web backend)
location / {
proxy_pass http://web_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 5s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
proxy_http_version 1.1;
proxy_set_header Connection "";
# Error handling
proxy_next_upstream error timeout http_502 http_503 http_504;
}
# Health check endpoint (for monitoring)
location /health {
access_log off;
return 200 "healthy\n";
add_header Content-Type text/plain;
}
}
# Cache configuration (for static backend)
proxy_cache_path /var/cache/nginx/static
levels=1:2
keys_zone=static_cache:10m
max_size=1g
inactive=60m
use_temp_path=off;
# Rate limiting zone
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
# Alternative server with rate limiting
server {
listen 443 ssl http2;
server_name api.example.com;
ssl_certificate /etc/nginx/ssl/api.example.com.crt;
ssl_certificate_key /etc/nginx/ssl/api.example.com.key;
location / {
# Rate limiting
limit_req zone=api_limit burst=20 nodelay;
limit_req_status 429;
proxy_pass http://api_backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
# Standard Kubernetes Ingress with Traefik-specific annotations
# This demonstrates using the standard Ingress resource with Traefik annotations
# for load balancing, rather than Traefik's CRDs
---
# Ingress: Main application with path-based routing
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
namespace: default
labels:
app: web-application
annotations:
# Ingress class to use Traefik
kubernetes.io/ingress.class: traefik
# TLS certificate resolver
cert-manager.io/cluster-issuer: letsencrypt-prod
# Traefik-specific: Entry points
traefik.ingress.kubernetes.io/router.entrypoints: web,websecure
# Traefik-specific: Redirect HTTP to HTTPS
traefik.ingress.kubernetes.io/redirect-scheme: https
traefik.ingress.kubernetes.io/redirect-permanent: "true"
# Traefik-specific: Middleware chain
traefik.ingress.kubernetes.io/router.middlewares: default-security-headers@kubernetescrd,default-compression@kubernetescrd,default-rate-limit@kubernetescrd
# Traefik-specific: TLS options
traefik.ingress.kubernetes.io/router.tls: "true"
traefik.ingress.kubernetes.io/router.tls.options: default@kubernetescrd
spec:
# TLS configuration
tls:
- hosts:
- example.com
- www.example.com
secretName: example-com-tls
# Routing rules
rules:
# Rule 1: Main domain
- host: example.com
http:
paths:
# Path 1: Homepage
- path: /
pathType: Prefix
backend:
service:
name: web-frontend
port:
number: 80
# Path 2: API endpoints
- path: /api
pathType: Prefix
backend:
service:
name: api-backend
port:
number: 8080
# Path 3: Static assets
- path: /static
pathType: Prefix
backend:
service:
name: static-assets
port:
number: 80
# Rule 2: WWW subdomain (mirror of main)
- host: www.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-frontend
port:
number: 80
---
# Ingress: API subdomain with weighted load balancing
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api-subdomain-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: traefik
# Weighted load balancing between v1 and v2 (canary deployment)
# Note: This requires TraefikService CRD for proper weighted routing
# Standard Ingress doesn't support weights, but we can use multiple backends
traefik.ingress.kubernetes.io/service.weighted: "true"
# Rate limiting for API
traefik.ingress.kubernetes.io/router.middlewares: default-api-rate-limit@kubernetescrd,default-cors-headers@kubernetescrd
spec:
tls:
- hosts:
- api.example.com
secretName: api-example-com-tls
rules:
- host: api.example.com
http:
paths:
# All API traffic
- path: /
pathType: Prefix
backend:
service:
name: api-backend-v1
port:
number: 8080
---
# Ingress: Admin interface with IP whitelisting
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: admin-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: traefik
# IP whitelist middleware
traefik.ingress.kubernetes.io/router.middlewares: default-ip-whitelist@kubernetescrd,default-admin-auth@kubernetescrd
# Priority (higher number = higher priority)
traefik.ingress.kubernetes.io/router.priority: "100"
spec:
tls:
- hosts:
- admin.example.com
secretName: admin-example-com-tls
rules:
- host: admin.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: admin-backend
port:
number: 8080
---
# Ingress: WebSocket support
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: websocket-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: traefik
# WebSocket-specific headers
traefik.ingress.kubernetes.io/router.middlewares: default-websocket-headers@kubernetescrd
# Sticky sessions for WebSocket connections
traefik.ingress.kubernetes.io/affinity: "true"
traefik.ingress.kubernetes.io/session-cookie-name: "ws-sticky"
spec:
tls:
- hosts:
- ws.example.com
secretName: ws-example-com-tls
rules:
- host: ws.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: websocket-backend
port:
number: 8080
---
# Ingress: Regex path matching
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: regex-path-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: traefik
# Enable regex path matching
traefik.ingress.kubernetes.io/router.pathmatcher: PathRegexp
spec:
rules:
- host: example.com
http:
paths:
# Match versioned API paths: /api/v1/*, /api/v2/*, etc.
- path: /api/v[0-9]+
pathType: ImplementationSpecific
backend:
service:
name: versioned-api-backend
port:
number: 8080
---
# Ingress: Custom error pages
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: error-pages-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: traefik
# Custom error page middleware
traefik.ingress.kubernetes.io/router.middlewares: default-error-pages@kubernetescrd
spec:
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-frontend
port:
number: 80
---
# TraefikService: Weighted load balancing (requires Traefik CRD)
# This is used to implement proper canary deployments with weight distribution
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
name: api-weighted-service
namespace: default
spec:
weighted:
services:
- name: api-backend-v1
port: 8080
weight: 80 # 80% traffic to v1
- name: api-backend-v2
port: 8080
weight: 20 # 20% traffic to v2 (canary)
---
# TraefikService: Mirroring traffic for testing
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:
name: api-mirrored-service
namespace: default
spec:
mirroring:
name: api-backend-v1
port: 8080
# Mirror 10% of traffic to v2 for testing (responses are ignored)
mirrors:
- name: api-backend-v2
port: 8080
percent: 10
---
# Middleware: Security headers (defined earlier in traefik-ingress.yaml)
# Referenced here for clarity
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: security-headers
namespace: default
spec:
headers:
sslRedirect: true
stsSeconds: 31536000
stsIncludeSubdomains: true
stsPreload: true
frameDeny: true
contentTypeNosniff: true
browserXssFilter: true
---
# Middleware: Compression
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: compression
namespace: default
spec:
compress:
excludedContentTypes:
- "text/event-stream"
---
# Middleware: Rate limiting
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: rate-limit
namespace: default
spec:
rateLimit:
average: 100
period: 1m
burst: 200
---
# Middleware: API-specific rate limiting
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: api-rate-limit
namespace: default
spec:
rateLimit:
average: 50
period: 1m
burst: 100
---
# Middleware: CORS headers
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: cors-headers
namespace: default
spec:
headers:
accessControlAllowMethods:
- "GET"
- "POST"
- "PUT"
- "DELETE"
- "OPTIONS"
accessControlAllowOriginList:
- "https://example.com"
- "https://app.example.com"
accessControlAllowCredentials: true
---
# Middleware: IP whitelist
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: ip-whitelist
namespace: default
spec:
ipWhiteList:
sourceRange:
- "10.0.0.0/8"
- "192.168.0.0/16"
---
# Middleware: Admin authentication
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: admin-auth
namespace: default
spec:
basicAuth:
secret: admin-auth-credentials
---
# Middleware: WebSocket headers
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: websocket-headers
namespace: default
spec:
headers:
customRequestHeaders:
X-Forwarded-Proto: "https"
---
# Middleware: Error pages
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: error-pages
namespace: default
spec:
errors:
status:
- "400-599"
service:
name: error-page-service
port: 80
query: "/{status}.html"
---
# Service: Error pages
apiVersion: v1
kind: Service
metadata:
name: error-page-service
namespace: default
spec:
selector:
app: error-pages
ports:
- port: 80
targetPort: 80
---
# Service: Web frontend
apiVersion: v1
kind: Service
metadata:
name: web-frontend
namespace: default
spec:
selector:
app: web-frontend
ports:
- port: 80
targetPort: 8080
---
# Service: API backend v1
apiVersion: v1
kind: Service
metadata:
name: api-backend-v1
namespace: default
labels:
version: v1
spec:
selector:
app: api-backend
version: v1
ports:
- port: 8080
targetPort: 8080
---
# Service: API backend v2
apiVersion: v1
kind: Service
metadata:
name: api-backend-v2
namespace: default
labels:
version: v2
spec:
selector:
app: api-backend
version: v2
ports:
- port: 8080
targetPort: 8080
---
# Service: Static assets
apiVersion: v1
kind: Service
metadata:
name: static-assets
namespace: default
spec:
selector:
app: static-assets
ports:
- port: 80
targetPort: 8080
---
# Service: Admin backend
apiVersion: v1
kind: Service
metadata:
name: admin-backend
namespace: default
spec:
selector:
app: admin-backend
ports:
- port: 8080
targetPort: 8080
---
# Service: WebSocket backend
apiVersion: v1
kind: Service
metadata:
name: websocket-backend
namespace: default
spec:
selector:
app: websocket-backend
ports:
- port: 8080
targetPort: 8080
---
# Service: Versioned API backend
apiVersion: v1
kind: Service
metadata:
name: versioned-api-backend
namespace: default
spec:
selector:
app: versioned-api-backend
ports:
- port: 8080
targetPort: 8080
skill: "load-balancing-patterns"
version: "1.0"
domain: "infrastructure"
base_outputs:
# Files ALWAYS produced by this skill
- path: "docs/load-balancing-design.md"
must_contain: ["## Load Balancer Selection", "## Health Check Configuration", "## Routing Rules"]
description: "Load balancing architecture documentation including selection rationale, routing rules, and health check strategy"
conditional_outputs:
maturity:
starter:
- path: "nginx.conf"
must_contain: ["upstream", "server", "location"]
description: "Basic NGINX reverse proxy configuration with simple round-robin load balancing"
- path: "docker-compose.yml"
must_contain: ["nginx", "services"]
description: "Docker Compose setup with NGINX load balancer and backend services"
intermediate:
- path: "terraform/load-balancer.tf"
must_contain: ["aws_lb|google_compute_", "health_check"]
description: "Cloud load balancer infrastructure with Terraform (ALB/NLB for AWS, Load Balancer for GCP/Azure)"
- path: "nginx/load-balancer.conf"
must_contain: ["upstream", "least_conn|ip_hash", "proxy_pass"]
description: "Advanced NGINX configuration with multiple algorithms and health checks"
- path: "haproxy/haproxy.cfg"
must_contain: ["frontend", "backend", "balance"]
description: "HAProxy configuration for high-performance load balancing"
advanced:
- path: "kubernetes/ingress.yaml"
must_contain: ["kind: Ingress", "ingressClassName", "rules"]
description: "Kubernetes Ingress resource with path-based routing and TLS"
- path: "kubernetes/ingress-controller.yaml"
must_contain: ["kind: Deployment", "nginx-ingress|traefik|haproxy"]
description: "Ingress controller deployment configuration"
- path: "terraform/global-load-balancer.tf"
must_contain: ["health_check", "backend|target_group"]
description: "Multi-region global load balancer with geographic routing"
- path: "envoy/envoy.yaml"
must_contain: ["clusters", "listeners", "routes"]
description: "Envoy proxy configuration for service mesh integration"
infrastructure:
kubernetes:
- path: "kubernetes/ingress.yaml"
must_contain: ["kind: Ingress", "networking.k8s.io/v1"]
description: "Kubernetes Ingress resource for cluster traffic management"
- path: "kubernetes/service.yaml"
must_contain: ["kind: Service", "type: ClusterIP|LoadBalancer"]
description: "Kubernetes Service definitions for backend pods"
- path: "manifests/ingress-controller.yaml"
must_contain: ["nginx-ingress|traefik|contour"]
description: "Ingress controller installation manifest"
docker_compose:
- path: "docker-compose.yml"
must_contain: ["nginx|haproxy|traefik", "depends_on"]
description: "Docker Compose configuration with load balancer and services"
- path: "nginx/nginx.conf"
must_contain: ["upstream", "proxy_pass"]
description: "NGINX configuration for Docker Compose deployment"
vm_based:
- path: "nginx/load-balancer.conf"
must_contain: ["upstream", "server.*weight"]
description: "NGINX load balancer configuration for VM-based infrastructure"
- path: "haproxy/haproxy.cfg"
must_contain: ["frontend", "backend", "server"]
description: "HAProxy configuration for VM deployments"
- path: "systemd/nginx.service"
description: "Systemd service file for NGINX load balancer"
cloud_provider:
aws:
- path: "terraform/alb.tf"
must_contain: ["aws_lb.*application", "aws_lb_target_group", "aws_lb_listener"]
description: "AWS Application Load Balancer (Layer 7) Terraform configuration"
- path: "terraform/nlb.tf"
must_contain: ["aws_lb.*network", "aws_lb_target_group"]
description: "AWS Network Load Balancer (Layer 4) Terraform configuration"
- path: "terraform/target-groups.tf"
must_contain: ["aws_lb_target_group", "health_check"]
description: "Target group definitions with health check configuration"
gcp:
- path: "terraform/load-balancer.tf"
must_contain: ["google_compute_.*load_balancer|google_compute_backend_service", "health_check"]
description: "GCP Load Balancer configuration (Application or Network LB)"
- path: "terraform/backend-service.tf"
must_contain: ["google_compute_backend_service", "google_compute_health_check"]
description: "GCP backend service with health checks"
azure:
- path: "terraform/load-balancer.tf"
must_contain: ["azurerm_lb|azurerm_application_gateway", "backend_address_pool"]
description: "Azure Load Balancer or Application Gateway configuration"
- path: "terraform/health-probe.tf"
must_contain: ["azurerm_lb_probe"]
description: "Azure health probe configuration"
iac_tool:
terraform:
- path: "terraform/main.tf"
must_contain: ["provider", "terraform"]
description: "Terraform main configuration file"
- path: "terraform/load-balancer.tf"
must_contain: ["resource.*lb|load_balancer", "health_check"]
description: "Load balancer resource definitions"
- path: "terraform/variables.tf"
must_contain: ["variable"]
description: "Input variables for load balancer configuration"
- path: "terraform/outputs.tf"
must_contain: ["output.*dns_name|ip_address"]
description: "Load balancer DNS name or IP address outputs"
pulumi:
- path: "index.ts"
must_contain: ["import.*pulumi", "LoadBalancer|ApplicationGateway"]
description: "Pulumi program for load balancer deployment"
- path: "Pulumi.yaml"
must_contain: ["name:", "runtime:"]
description: "Pulumi project configuration"
ansible:
- path: "playbooks/deploy-load-balancer.yml"
must_contain: ["hosts:", "tasks:", "nginx|haproxy"]
description: "Ansible playbook to deploy and configure load balancer"
- path: "templates/nginx.conf.j2"
must_contain: ["upstream", "{{.*}}"]
description: "Jinja2 template for NGINX configuration"
scaffolding:
- path: "monitoring/"
reason: "Directory for Prometheus exporters, Grafana dashboards, and health check scripts"
- path: "certs/"
reason: "Directory for SSL/TLS certificates (should remain empty in version control)"
- path: "tests/"
reason: "Directory for load balancer configuration tests and validation scripts"
metadata:
primary_blueprints: ["api-first", "k8s", "infrastructure"]
contributes_to:
- "Load balancer configuration files"
- "Traffic distribution and routing rules"
- "Health check endpoints and monitoring"
- "SSL/TLS termination setup"
- "Session persistence configuration"
- "Infrastructure as code for load balancers"
- "Kubernetes Ingress resources"
- "Global traffic management"
Cloud Load Balancers
Complete configurations for AWS, GCP, and Azure managed load balancing services.
Table of Contents
- AWS Load Balancers
- Application Load Balancer (ALB)
- Network Load Balancer (NLB)
- Global Accelerator
- GCP Load Balancing
- Application Load Balancer
- Network Load Balancer
- Cloud Load Balancing
- Azure Load Balancing
- Application Gateway
- Load Balancer
- Traffic Manager
- Cost Comparison
- Selection Guide
- Multi-Cloud Considerations
AWS Load Balancers
Application Load Balancer (ALB)
Layer 7 HTTP/HTTPS load balancer with advanced routing capabilities.
Use cases:
- Web applications
- Microservices
- Container-based applications
- Lambda functions
Key features:
- Path-based routing (
/api/*→ API servers) - Host-based routing (
api.example.com→ API servers) - HTTP header routing
- Query string parameter routing
- WebSocket support
- HTTP/2 and gRPC support
- AWS WAF integration
- Cognito authentication
Terraform example: See examples/aws/alb-terraform.tf
Network Load Balancer (NLB)
Layer 4 TCP/UDP load balancer for ultra-low latency and high throughput.
Use cases:
- Non-HTTP protocols
- Static IP addresses required
- Extreme performance requirements (millions RPS)
- Client IP preservation critical
Key features:
- Static IP per availability zone
- Elastic IP support
- Preserves source client IP
- TLS termination
- Cross-zone load balancing
- PrivateLink support
Terraform example: See examples/aws/nlb-terraform.tf
Global Accelerator
Global Layer 4 load balancer using AWS global network.
Use cases:
- Multi-region applications
- Global user base
- DDoS protection
- Automatic regional failover
Key features:
- Two static anycast IP addresses
- AWS Shield Standard DDoS protection
- Health checks per endpoint
- Traffic dials (gradual migration)
- Integration with ALB, NLB, EC2, EIP
GCP Load Balancing
Application Load Balancer
Global Layer 7 load balancer.
Types:
- External Application Load Balancer (global)
- Internal Application Load Balancer (regional)
- Regional External Application Load Balancer
Key features:
- URL map-based routing
- Cloud CDN integration
- Cloud Armor (WAF and DDoS)
- SSL policies and certificates
- Backend services with health checks
Network Load Balancer
Regional Layer 4 load balancer.
Types:
- External passthrough Network Load Balancer
- Internal passthrough Network Load Balancer
Key features:
- TCP/UDP load balancing
- Preserves client IP
- Regional or zonal backends
- Session affinity
Cloud Load Balancing
Global load balancing service with single anycast IP.
Key features:
- Single global anycast IP
- Cross-region load balancing
- Automatic multi-region failover
- Backend buckets (Cloud Storage)
Azure Load Balancing
Application Gateway
Layer 7 web traffic load balancer.
Key features:
- WAF integration (Web Application Firewall)
- URL-based routing
- Multi-site hosting
- SSL termination and end-to-end SSL
- Autoscaling
- Zone redundancy
- Rewrite HTTP headers and URL
Load Balancer
Layer 4 network load balancer.
SKUs:
- Basic: Simple load balancing, free tier
- Standard: Production-ready with zone redundancy
Key features:
- TCP/UDP load balancing
- Health probes (TCP, HTTP, HTTPS)
- Outbound rules
- HA ports
- Multiple frontends
- Zone redundant
Traffic Manager
DNS-based global load balancer.
Routing methods:
- Priority: Failover routing
- Weighted: Distribute across endpoints
- Performance: Lowest latency endpoint
- Geographic: Based on DNS location
- MultiValue: Return multiple healthy endpoints
- Subnet: Based on client subnet
Cost Comparison
| Provider | L4 Solution | L7 Solution | Pricing Model |
|---|---|---|---|
| AWS | NLB | ALB | Per hour + LCU (capacity units) |
| GCP | Network LB | Application LB | Per hour + forwarding rules |
| Azure | Load Balancer | Application Gateway | Per hour + data processed |
Selection Guide
Choose AWS ALB when:
- HTTP/HTTPS applications
- Need AWS WAF integration
- Lambda as backend targets
- Path-based microservices routing
Choose AWS NLB when:
- Non-HTTP protocols
- Static IPs required
- Ultra-low latency critical
- High throughput needs
Choose GCP Application LB when:
- Global HTTP(S) load balancing
- Cloud CDN integration needed
- Cloud Armor for DDoS/WAF
Choose Azure Application Gateway when:
- Azure-native web applications
- WAF required
- URL rewriting needed
Multi-Cloud Considerations
For multi-cloud deployments, consider:
- Self-managed load balancers (NGINX, HAProxy)
- Cloud-agnostic tools (Traefik, Envoy)
- DNS-based global load balancing
- Avoid cloud-specific features that lock you in
Complete configuration examples available in examples/aws/, examples/gcp/, and examples/azure/ directories.
Envoy Proxy Load Balancing Patterns
Cloud-native proxy for microservices and service mesh architectures.
Table of Contents
- Basic HTTP Load Balancing
- Load Balancing Policies
- Health Checks
- HTTP Health Check
- TCP Health Check
- gRPC Health Check
- Circuit Breakers
- Retry and Timeout Policies
- Advanced Routing
- Path-Based Routing
- Header-Based Routing
- TLS Configuration
- Dynamic Configuration (xDS)
Basic HTTP Load Balancing
static_resources:
listeners:
- name: main_listener
address:
socket_address:
address: 0.0.0.0
port_value: 80
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
http_filters:
- name: envoy.filters.http.router
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
route_config:
name: local_route
virtual_hosts:
- name: backend
domains: ["*"]
routes:
- match:
prefix: "/"
route:
cluster: backend_cluster
clusters:
- name: backend_cluster
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: backend_cluster
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: backend1.example.com
port_value: 8080
- endpoint:
address:
socket_address:
address: backend2.example.com
port_value: 8080Load Balancing Policies
Round Robin:
lb_policy: ROUND_ROBINLeast Request:
lb_policy: LEAST_REQUEST
least_request_lb_config:
choice_count: 2Random:
lb_policy: RANDOMRing Hash (consistent hashing):
lb_policy: RING_HASH
ring_hash_lb_config:
minimum_ring_size: 1024Health Checks
HTTP Health Check
clusters:
- name: backend_cluster
health_checks:
- timeout: 1s
interval: 5s
unhealthy_threshold: 3
healthy_threshold: 2
http_health_check:
path: /health
expected_statuses:
- start: 200
end: 299TCP Health Check
health_checks:
- timeout: 1s
interval: 5s
unhealthy_threshold: 3
healthy_threshold: 2
tcp_health_check: {}gRPC Health Check
health_checks:
- timeout: 1s
interval: 5s
grpc_health_check:
service_name: "myservice"
authority: "grpc.example.com"Circuit Breakers
clusters:
- name: backend_cluster
circuit_breakers:
thresholds:
- priority: DEFAULT
max_connections: 1000
max_pending_requests: 100
max_requests: 1000
max_retries: 3
retry_budget:
budget_percent:
value: 25.0
min_retry_concurrency: 10Retry and Timeout Policies
routes:
- match:
prefix: "/api"
route:
cluster: backend_cluster
timeout: 15s
retry_policy:
retry_on: "5xx"
num_retries: 3
per_try_timeout: 5sAdvanced Routing
Path-Based Routing
virtual_hosts:
- name: backend
domains: ["*"]
routes:
- match:
prefix: "/api"
route:
cluster: api_cluster
- match:
prefix: "/static"
route:
cluster: static_cluster
- match:
prefix: "/"
route:
cluster: web_clusterHeader-Based Routing
routes:
- match:
prefix: "/"
headers:
- name: "X-API-Version"
exact_match: "v2"
route:
cluster: api_v2_clusterTLS Configuration
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
# ... config ...
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
- certificate_chain:
filename: "/etc/envoy/certs/cert.pem"
private_key:
filename: "/etc/envoy/certs/key.pem"Dynamic Configuration (xDS)
Envoy supports dynamic configuration via xDS APIs:
- EDS: Endpoint Discovery Service
- CDS: Cluster Discovery Service
- RDS: Route Discovery Service
- LDS: Listener Discovery Service
Used by service meshes like Istio and Consul Connect.
Complete examples in examples/envoy/ directory.
Global Load Balancing
DNS-based global traffic management for multi-region deployments.
Table of Contents
- GeoDNS Routing
- AWS Route 53 Geolocation Routing
- Latency-Based Routing
- Multi-Region Failover
- AWS Global Accelerator
- CDN Integration
- Summary
GeoDNS Routing
Route users to nearest server based on geographic location.
AWS Route 53 Geolocation Routing
resource "aws_route53_record" "geo_us" {
zone_id = aws_route53_zone.main.zone_id
name = "app.example.com"
type = "A"
geolocation_routing_policy {
continent = "NA"
}
set_identifier = "US-East"
alias {
name = aws_lb.us_east.dns_name
zone_id = aws_lb.us_east.zone_id
evaluate_target_health = true
}
}
resource "aws_route53_record" "geo_eu" {
zone_id = aws_route53_zone.main.zone_id
name = "app.example.com"
type = "A"
geolocation_routing_policy {
continent = "EU"
}
set_identifier = "EU-West"
alias {
name = aws_lb.eu_west.dns_name
zone_id = aws_lb.eu_west.zone_id
evaluate_target_health = true
}
}
# Default fallback
resource "aws_route53_record" "geo_default" {
zone_id = aws_route53_zone.main.zone_id
name = "app.example.com"
type = "A"
geolocation_routing_policy {
continent = "*"
}
set_identifier = "Default"
alias {
name = aws_lb.us_east.dns_name
zone_id = aws_lb.us_east.zone_id
evaluate_target_health = true
}
}Latency-Based Routing
DNS returns IP of lowest-latency endpoint.
resource "aws_route53_record" "latency_us" {
zone_id = aws_route53_zone.main.zone_id
name = "app.example.com"
type = "A"
latency_routing_policy {
region = "us-east-1"
}
set_identifier = "US-East"
alias {
name = aws_lb.us_east.dns_name
zone_id = aws_lb.us_east.zone_id
evaluate_target_health = true
}
}Multi-Region Failover
Primary/secondary configuration with health checks.
resource "aws_route53_record" "primary" {
zone_id = aws_route53_zone.main.zone_id
name = "app.example.com"
type = "A"
failover_routing_policy {
type = "PRIMARY"
}
set_identifier = "Primary"
alias {
name = aws_lb.us_east.dns_name
zone_id = aws_lb.us_east.zone_id
evaluate_target_health = true
}
health_check_id = aws_route53_health_check.primary.id
}
resource "aws_route53_record" "secondary" {
zone_id = aws_route53_zone.main.zone_id
name = "app.example.com"
type = "A"
failover_routing_policy {
type = "SECONDARY"
}
set_identifier = "Secondary"
alias {
name = aws_lb.eu_west.dns_name
zone_id = aws_lb.eu_west.zone_id
evaluate_target_health = true
}
}
resource "aws_route53_health_check" "primary" {
fqdn = aws_lb.us_east.dns_name
port = 443
type = "HTTPS"
resource_path = "/health"
failure_threshold = 3
request_interval = 30
}AWS Global Accelerator
Global Layer 4 load balancer using AWS backbone.
resource "aws_globalaccelerator_accelerator" "main" {
name = "app-accelerator"
ip_address_type = "IPV4"
enabled = true
}
resource "aws_globalaccelerator_listener" "main" {
accelerator_arn = aws_globalaccelerator_accelerator.main.id
protocol = "TCP"
port_ranges {
from_port = 443
to_port = 443
}
}
resource "aws_globalaccelerator_endpoint_group" "us_east" {
listener_arn = aws_globalaccelerator_listener.main.id
endpoint_group_region = "us-east-1"
endpoint_configuration {
endpoint_id = aws_lb.us_east.arn
weight = 100
}
health_check_interval_seconds = 30
health_check_path = "/health"
health_check_protocol = "HTTPS"
threshold_count = 3
traffic_dial_percentage = 100
}CDN Integration
Combine load balancing with CDN for global content delivery.
CloudFlare:
- Global anycast network
- DDoS protection
- Edge caching
- Load balancing across origins
AWS CloudFront + Route 53:
- GeoDNS routes to CloudFront
- CloudFront caches at edge locations
- Origin load balancing with ALB/NLB
Summary
Use GeoDNS for geographic routing, latency-based routing for performance, and failover routing for high availability. Combine with CDN for optimal global performance. Monitor DNS propagation times and TTLs carefully.
HAProxy Load Balancing Patterns
Complete guide to HAProxy configuration for high-performance load balancing.
Table of Contents
- Basic HTTP Load Balancing
- Load Balancing Algorithms
- Health Checks
- HTTP Health Check
- TCP Health Check
- Advanced Health Checks
- Sticky Sessions
- Cookie-Based
- Source IP
- Application Cookie
- SSL/TLS Termination
- ACL-Based Routing
- TCP Mode (Layer 4)
- Advanced Features
- Rate Limiting
- Connection Draining
- Statistics Dashboard
Basic HTTP Load Balancing
global
log /dev/log local0
maxconn 4096
user haproxy
group haproxy
daemon
defaults
log global
mode http
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
retries 3
timeout connect 5000
timeout client 50000
timeout server 50000
frontend http_front
bind *:80
default_backend web_servers
backend web_servers
balance roundrobin
option httpchk GET /health
server web1 192.168.1.101:8080 check
server web2 192.168.1.102:8080 checkLoad Balancing Algorithms
HAProxy supports 10+ algorithms:
Round Robin:
backend web_servers
balance roundrobin
server web1 192.168.1.101:8080
server web2 192.168.1.102:8080Least Connections:
backend web_servers
balance leastconn
server web1 192.168.1.101:8080
server web2 192.168.1.102:8080Source IP Hash:
backend web_servers
balance source
hash-type consistent
server web1 192.168.1.101:8080
server web2 192.168.1.102:8080URI Hash:
backend web_servers
balance uri
hash-type consistent
server web1 192.168.1.101:8080
server web2 192.168.1.102:8080Health Checks
HTTP Health Check
backend web_servers
option httpchk GET /health HTTP/1.1\r\nHost:\ example.com
http-check expect status 200
http-check expect rstring "healthy"
server web1 192.168.1.101:8080 check inter 5s fall 3 rise 2Parameters:
check: Enable health checksinter 5s: Check every 5 secondsfall 3: Mark down after 3 failuresrise 2: Mark up after 2 successes
TCP Health Check
backend mysql_servers
mode tcp
option tcp-check
tcp-check connect port 3306
server mysql1 192.168.1.101:3306 check inter 5sAdvanced Health Checks
Redis:
backend redis_servers
mode tcp
option tcp-check
tcp-check send PING\r\n
tcp-check expect string +PONG
server redis1 192.168.1.101:6379 checkMySQL:
backend mysql_servers
mode tcp
option mysql-check user haproxy
server mysql1 192.168.1.101:3306 checkSticky Sessions
Cookie-Based
backend web_servers
balance roundrobin
cookie SERVERID insert indirect nocache
server web1 192.168.1.101:8080 check cookie web1
server web2 192.168.1.102:8080 check cookie web2Source IP
backend web_servers
balance source
hash-type consistent
server web1 192.168.1.101:8080 check
server web2 192.168.1.102:8080 checkApplication Cookie
backend web_servers
stick-table type string len 32 size 100k expire 30m
stick on cookie(JSESSIONID)
server web1 192.168.1.101:8080 check
server web2 192.168.1.102:8080 checkSSL/TLS Termination
frontend https_front
bind *:443 ssl crt /etc/haproxy/certs/example.com.pem
redirect scheme https code 301 if !{ ssl_fc }
default_backend web_servers
backend web_servers
balance roundrobin
server web1 192.168.1.101:8080 checkACL-Based Routing
frontend http_front
bind *:80
acl is_api path_beg /api
acl is_static path_beg /static
acl is_admin hdr(host) -i admin.example.com
use_backend api_servers if is_api
use_backend static_servers if is_static
use_backend admin_servers if is_admin
default_backend web_serversTCP Mode (Layer 4)
frontend mysql_front
mode tcp
bind *:3306
default_backend mysql_servers
backend mysql_servers
mode tcp
balance leastconn
option tcp-check
server mysql1 192.168.1.101:3306 check
server mysql2 192.168.1.102:3306 checkAdvanced Features
Rate Limiting
frontend http_front
stick-table type ip size 100k expire 30s store http_req_rate(10s)
http-request track-sc0 src
http-request deny if { sc_http_req_rate(0) gt 100 }Connection Draining
backend web_servers
server web1 192.168.1.101:8080 check weight 100
server web2 192.168.1.102:8080 check weight 100
# To drain web1: set weight to 0, wait for connections to finishStatistics Dashboard
frontend stats
bind *:8404
stats enable
stats uri /stats
stats refresh 30s
stats auth admin:passwordComplete configuration examples available in examples/haproxy/ directory.
Kubernetes Ingress Controllers
Complete guide to Kubernetes ingress for HTTP load balancing.
Table of Contents
- NGINX Ingress Controller
- Installation
- Basic Ingress
- Advanced Features
- Traefik Ingress
- Installation
- IngressRoute (CRD)
- HAProxy Ingress
- Gateway API (Next Generation)
- Health Checks
NGINX Ingress Controller
Installation
helm repo add nginx-stable https://helm.nginx.com/stable
helm install nginx-ingress nginx-stable/nginx-ingress \
--namespace ingress-nginx \
--create-namespaceBasic Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
ingressClassName: nginx
rules:
- host: app.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-service
port:
number: 80Advanced Features
Sticky Sessions:
annotations:
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-name: "app_session"Rate Limiting:
annotations:
nginx.ingress.kubernetes.io/limit-rps: "100"Traefik Ingress
Installation
helm install traefik traefik/traefik \
--namespace traefik \
--create-namespaceIngressRoute (CRD)
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: app-route
spec:
entryPoints:
- websecure
routes:
- match: Host(`app.example.com`)
kind: Rule
services:
- name: app-service
port: 80
tls:
certResolver: letsencryptHAProxy Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
haproxy.org/load-balance: "leastconn"
haproxy.org/cookie-persistence: "app-cookie"
spec:
ingressClassName: haproxy
rules:
- host: app.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: app-service
port:
number: 80Gateway API (Next Generation)
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: app-gateway
spec:
gatewayClassName: envoy
listeners:
- name: https
protocol: HTTPS
port: 443
tls:
certificateRefs:
- name: app-tls
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: app-route
spec:
parentRefs:
- name: app-gateway
hostnames:
- "app.example.com"
rules:
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: api-service
port: 80Health Checks
All ingress controllers rely on Kubernetes Service and Pod health:
apiVersion: v1
kind: Pod
spec:
containers:
- name: app
livenessProbe:
httpGet:
path: /health/live
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /health/ready
port: 8080
initialDelaySeconds: 5
periodSeconds: 5Complete examples in examples/kubernetes/ directory.
NGINX Load Balancing Patterns
Complete guide to NGINX and NGINX Plus load balancing configurations.
Table of Contents
- Basic HTTP Load Balancing
- Upstream Configuration
- Load Balancing Algorithms
- Server Parameters
- Health Checks
- Passive Health Checks (Open Source)
- Active Health Checks (NGINX Plus)
- Sticky Sessions (NGINX Plus)
- Cookie-Based
- Learn from Application Cookie
- TCP/UDP Stream Load Balancing
- SSL/TLS Termination
- Advanced Patterns
- Slow Start (NGINX Plus)
- Connection Limits
- Error Handling
- Complete Configuration Example
Basic HTTP Load Balancing
Upstream Configuration
upstream backend {
# Algorithm: round-robin (default)
# Alternatives: least_conn, ip_hash, hash, random
server backend1.example.com:8080;
server backend2.example.com:8080;
server backend3.example.com:8080;
# Keepalive connections
keepalive 32;
}
server {
listen 80;
location / {
proxy_pass http://backend;
}
}Load Balancing Algorithms
Round Robin (default):
upstream backend {
server backend1:8080;
server backend2:8080;
}Least Connections:
upstream backend {
least_conn;
server backend1:8080;
server backend2:8080;
}IP Hash (sticky sessions):
upstream backend {
ip_hash;
server backend1:8080;
server backend2:8080;
}Hash (custom key):
upstream backend {
hash $request_uri consistent;
server backend1:8080;
server backend2:8080;
}Random:
upstream backend {
random two least_conn;
server backend1:8080;
server backend2:8080;
}Server Parameters
upstream backend {
server backend1:8080 weight=3 max_fails=3 fail_timeout=30s;
server backend2:8080 weight=2 max_fails=3 fail_timeout=30s;
server backend3:8080 backup;
server backend4:8080 down;
}Parameters:
weight=n: Server weight (default: 1)max_fails=n: Failed attempts before marking server downfail_timeout=time: Time server marked down after max_failsbackup: Backup server (used when primaries unavailable)down: Permanently mark server as unavailable
Health Checks
Passive Health Checks (Open Source)
upstream backend {
server backend1:8080 max_fails=3 fail_timeout=30s;
server backend2:8080 max_fails=3 fail_timeout=30s;
}Active Health Checks (NGINX Plus)
upstream backend {
zone backend 64k;
server backend1:8080;
server backend2:8080;
}
match server_ok {
status 200-399;
header Content-Type = "application/json";
body ~ "\"status\":\"healthy\"";
}
server {
location / {
proxy_pass http://backend;
health_check interval=5s fails=3 passes=2 uri=/health match=server_ok;
}
}Sticky Sessions (NGINX Plus)
Cookie-Based
upstream backend {
zone backend 64k;
server backend1:8080;
server backend2:8080;
sticky cookie srv_id expires=1h domain=.example.com path=/;
}Learn from Application Cookie
upstream backend {
zone backend 64k;
server backend1:8080;
server backend2:8080;
sticky learn
create=$upstream_cookie_JSESSIONID
lookup=$cookie_JSESSIONID
zone=client_sessions:1m;
}TCP/UDP Stream Load Balancing
stream {
upstream mysql_backend {
least_conn;
server mysql1:3306 max_fails=3 fail_timeout=30s;
server mysql2:3306 max_fails=3 fail_timeout=30s;
}
server {
listen 3306;
proxy_pass mysql_backend;
proxy_timeout 5s;
proxy_connect_timeout 1s;
}
}SSL/TLS Termination
upstream backend {
server backend1:8080;
server backend2:8080;
keepalive 32;
}
server {
listen 443 ssl http2;
server_name example.com;
ssl_certificate /etc/nginx/ssl/example.com.crt;
ssl_certificate_key /etc/nginx/ssl/example.com.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
proxy_pass http://backend;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
}Advanced Patterns
Slow Start (NGINX Plus)
Gradually increase traffic to newly added or recovered servers:
upstream backend {
zone backend 64k;
server backend1:8080 slow_start=30s;
server backend2:8080 slow_start=30s;
}Connection Limits
upstream backend {
server backend1:8080 max_conns=100;
server backend2:8080 max_conns=100;
queue 50 timeout=30s;
}Error Handling
location / {
proxy_pass http://backend;
proxy_next_upstream error timeout http_500 http_502 http_503;
proxy_next_upstream_tries 2;
proxy_next_upstream_timeout 10s;
}Complete Configuration Example
See examples/nginx/http-load-balancing.conf for a production-ready configuration with:
- Multiple upstream pools
- SSL termination
- Health checks
- Caching
- Rate limiting
- Security headers
Traefik Load Balancing Patterns
Cloud-native edge router with automatic service discovery.
Table of Contents
- Docker Provider
- File Provider
- Kubernetes Ingress
- Load Balancing Methods
- Middleware
- Rate Limiting
- Circuit Breaker
Docker Provider
# docker-compose.yml
version: '3.8'
services:
traefik:
image: traefik:v3.0
command:
- --api.insecure=true
- --providers.docker=true
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
ports:
- "80:80"
- "443:443"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
app:
image: myapp:latest
labels:
- "traefik.enable=true"
- "traefik.http.routers.app.rule=Host(`app.example.com`)"
- "traefik.http.services.app.loadbalancer.server.port=8080"
- "traefik.http.services.app.loadbalancer.healthcheck.path=/health"
- "traefik.http.services.app.loadbalancer.healthcheck.interval=10s"
deploy:
replicas: 3File Provider
# traefik.yml
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
providers:
file:
filename: /etc/traefik/dynamic.yml
# dynamic.yml
http:
routers:
app-router:
rule: "Host(`app.example.com`)"
service: app-service
entryPoints:
- websecure
tls:
certResolver: letsencrypt
services:
app-service:
loadBalancer:
servers:
- url: "http://backend1:8080"
- url: "http://backend2:8080"
healthCheck:
path: /health
interval: 10s
timeout: 3sKubernetes Ingress
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: app-route
spec:
entryPoints:
- websecure
routes:
- match: Host(`app.example.com`)
kind: Rule
services:
- name: app-service
port: 80
sticky:
cookie:
name: app_session
httpOnly: true
tls:
certResolver: letsencryptLoad Balancing Methods
Weighted Round Robin:
services:
app-service:
loadBalancer:
servers:
- url: "http://backend1:8080"
- url: "http://backend2:8080"
weight: 2Sticky Sessions:
services:
app-service:
loadBalancer:
sticky:
cookie:
name: app_session
httpOnly: true
secure: trueMiddleware
Rate Limiting
http:
middlewares:
api-ratelimit:
rateLimit:
average: 100
burst: 50
period: 1s
routers:
api-router:
rule: "Host(`api.example.com`)"
middlewares:
- api-ratelimit
service: api-serviceCircuit Breaker
http:
middlewares:
api-circuit-breaker:
circuitBreaker:
expression: "NetworkErrorRatio() > 0.30"Complete examples in examples/traefik/ directory.
Related skills
FAQ
What is the difference between L4 and L7 load balancing?
L4 routes on IP and port with lower latency and no app-data inspection; L7 routes on HTTP URLs, headers, and cookies with SSL termination and content-based routing.
Which self-managed load balancer should I use?
NGINX for general HTTP/HTTPS stacks, HAProxy for maximum performance and database balancing, and Envoy for microservices and service mesh integration.