Infra Engineer

Infrastructure Engineering Skill

Comprehensive guide for modern infrastructure engineering covering DevOps practices, multi-cloud platforms (AWS, Azure, GCP, Cloudflare), FinOps cost optimization, and DevSecOps security practices.

When to Use This Skill

Use this skill when:

• DevOps: Setting up CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins), implementing GitOps workflows (ArgoCD, Flux)
• AWS: Deploying to EC2, Lambda, ECS, EKS, managing S3, RDS, using CloudFormation/CDK
• Azure: Working with Azure VMs, App Service, AKS, Azure Functions, Storage Accounts
• GCP: Managing Compute Engine, GKE, Cloud Run, Cloud Storage, App Engine
• Cloudflare: Deploying Workers, R2 storage, D1 databases, Pages applications
• Kubernetes: Managing clusters, deployments, services, ingress, Helm charts, operators
• Docker: Containerizing applications, multi-stage builds, Docker Compose, registries
• FinOps: Analyzing cloud costs, optimizing spend, reserved instances, spot instances, rightsizing
• DevSecOps: Security scanning (SAST/DAST), vulnerability management, secrets management, compliance
• IaC: Terraform, CloudFormation, Pulumi, configuration management
• Monitoring: Setting up observability, logging, metrics, alerting, distributed tracing

Platform Selection Guide

When to Use AWS

Best For:

• General-purpose cloud computing at scale
• Mature ecosystem with 200+ services
• Enterprise workloads with compliance requirements
• Hybrid cloud with AWS Outposts
• Extensive third-party integrations
• Advanced networking and security controls

Key Services:

• EC2 (virtual machines, flexible compute)
• Lambda (serverless functions, event-driven)
• ECS/EKS (container orchestration)
• S3 (object storage, industry standard)
• RDS (managed relational databases)
• DynamoDB (NoSQL, global tables)
• CloudFormation/CDK (infrastructure as code)
• IAM (identity and access management)
• VPC (virtual private cloud networking)

Cost Profile: Pay-as-you-go, reserved instances (up to 72% discount), savings plans, spot instances (up to 90% discount)

When to Use Azure

Best For:

• Microsoft-centric organizations (.NET, Active Directory)
• Hybrid cloud scenarios (Azure Arc, Stack)
• Enterprise agreements with Microsoft
• Windows Server and SQL Server workloads
• Integration with Microsoft 365 and Dynamics
• Strong compliance certifications (90+ certifications)

Key Services:

• Virtual Machines (Windows/Linux compute)
• App Service (PaaS for web apps)
• AKS (managed Kubernetes)
• Azure Functions (serverless compute)
• Storage Accounts (Blob, File, Queue, Table)
• SQL Database (managed SQL Server)
• Active Directory (identity management)
• ARM Templates/Bicep (infrastructure as code)

Cost Profile: Pay-as-you-go, reserved instances, Azure Hybrid Benefit for Windows/SQL Server licenses

When to Use Cloudflare

Best For:

• Edge-first applications with global distribution
• Ultra-low latency requirements (<50ms)
• Static sites with serverless functions
• Zero egress cost scenarios (R2 storage)
• WebSocket/real-time applications (Durable Objects)
• AI/ML at the edge (Workers AI)

Key Products:

• Workers (serverless functions)
• R2 (object storage, S3-compatible)
• D1 (SQLite database with global replication)
• KV (key-value store)
• Pages (static hosting + functions)
• Durable Objects (stateful compute)
• Browser Rendering (headless browser automation)

Cost Profile: Pay-per-request, generous free tier, zero egress fees

When to Use Kubernetes

Best For:

• Container orchestration at scale
• Microservices architectures with 10+ services
• Multi-cloud and hybrid deployments
• Self-healing and auto-scaling workloads
• Complex deployment strategies (blue/green, canary)
• Service mesh architectures (Istio, Linkerd)
• Stateful applications with operators

Key Features:

• Declarative configuration (YAML manifests)
• Automated rollouts and rollbacks
• Service discovery and load balancing
• Self-healing (restarts failed containers)
• Horizontal pod autoscaling
• Secret and configuration management
• Storage orchestration
• Batch job execution

Managed Options: EKS (AWS), AKS (Azure), GKE (GCP), managed k8s providers

Cost Profile: Cluster management fees + node costs (optimize with spot instances, cluster autoscaling)

When to Use Docker

Best For:

• Local development consistency
• Microservices architectures
• Multi-language stack applications
• Traditional VPS/VM deployments
• Foundation for Kubernetes workloads
• CI/CD build environments
• Database containerization (dev/test)

Key Capabilities:

• Application isolation and portability
• Multi-stage builds for optimization
• Docker Compose for multi-container apps
• Volume management for data persistence
• Network configuration and service discovery
• Cross-platform compatibility (amd64, arm64)
• BuildKit for improved build performance

Cost Profile: Infrastructure cost only (compute + storage), no orchestration overhead

When to Use Google Cloud

Best For:

• Enterprise-scale applications
• Data analytics and ML pipelines (BigQuery, Vertex AI)
• Hybrid/multi-cloud deployments
• Kubernetes at scale (GKE)
• Managed databases (Cloud SQL, Firestore, Spanner)
• Complex IAM and compliance requirements

Key Services:

• Compute Engine (VMs)
• GKE (managed Kubernetes)
• Cloud Run (containerized serverless)
• App Engine (PaaS)
• Cloud Storage (object storage)
• Cloud SQL (managed databases)

Cost Profile: Varied pricing, sustained use discounts, committed use contracts

Quick Start

AWS Lambda Function

# Install AWS CLI
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip && sudo ./aws/install

# Configure credentials
aws configure

# Create Lambda function with SAM
sam init --runtime python3.11
sam build && sam deploy --guided

See: references/aws-lambda.md

AWS EKS Kubernetes Cluster

# Install eksctl
brew install eksctl  # or curl download

# Create cluster
eksctl create cluster \
  --name my-cluster \
  --region us-west-2 \
  --nodegroup-name standard-workers \
  --node-type t3.medium \
  --nodes 3 \
  --nodes-min 1 \
  --nodes-max 4

See: references/kubernetes-basics.md

Azure Deployment

# Install Azure CLI
curl -L https://aka.ms/InstallAzureCli | bash

# Login and create resources
az login
az group create --name myResourceGroup --location eastus
az webapp create --resource-group myResourceGroup \
  --name myapp --runtime "NODE:18-lts"

See: references/azure-basics.md

Cloudflare Workers

# Install Wrangler CLI
npm install -g wrangler

# Create and deploy Worker
wrangler init my-worker
cd my-worker
wrangler deploy

See: references/cloudflare-workers-basics.md

Kubernetes Deployment

# Create deployment
kubectl create deployment nginx --image=nginx:latest
kubectl expose deployment nginx --port=80 --type=LoadBalancer

# Apply from manifest
kubectl apply -f deployment.yaml

# Check status
kubectl get pods,services,deployments

See: references/kubernetes-basics.md

Docker Container

# Create Dockerfile
cat > Dockerfile <<EOF
FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --production
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]
EOF

# Build and run
docker build -t myapp .
docker run -p 3000:3000 myapp

See: references/docker-basics.md

Reference Navigation

AWS (Amazon Web Services)

• aws-overview.md - AWS fundamentals, account setup, IAM basics
• aws-ec2.md - EC2 instances, AMIs, security groups, auto-scaling
• aws-lambda.md - Serverless functions, SAM, event sources, layers
• aws-ecs-eks.md - Container orchestration, ECS vs EKS, Fargate
• aws-s3-rds.md - S3 storage, RDS databases, backup strategies
• aws-cloudformation.md - Infrastructure as code, CDK, best practices
• aws-networking.md - VPC, subnets, security groups, load balancers

Azure (Microsoft Azure)

• azure-basics.md - Azure fundamentals, subscriptions, resource groups
• azure-compute.md - VMs, App Service, AKS, Azure Functions
• azure-storage.md - Storage Accounts, Blob, Files, managed disks

Cloudflare Platform

• cloudflare-platform.md - Edge computing overview, key components
• cloudflare-workers-basics.md - Getting started, handler types, basic patterns
• cloudflare-workers-advanced.md - Advanced patterns, performance, optimization
• cloudflare-workers-apis.md - Runtime APIs, bindings, integrations
• cloudflare-r2-storage.md - R2 object storage, S3 compatibility, best practices
• cloudflare-d1-kv.md - D1 SQLite database, KV store, use cases
• browser-rendering.md - Puppeteer/Playwright automation on Cloudflare

Kubernetes & Container Orchestration

• kubernetes-basics.md - Core concepts, pods, deployments, services
• kubernetes-advanced.md - StatefulSets, operators, custom resources
• kubernetes-networking.md - Ingress, service mesh, network policies
• helm-charts.md - Package management, charts, repositories

Docker Containerization

• docker-basics.md - Core concepts, Dockerfile, images, containers
• docker-compose.md - Multi-container apps, networking, volumes
• docker-security.md - Image scanning, secrets, best practices

Google Cloud Platform

• gcloud-platform.md - GCP overview, gcloud CLI, authentication
• gcloud-services.md - Compute Engine, GKE, Cloud Run, App Engine

CI/CD & GitOps

• cicd-github-actions.md - GitHub Actions workflows, runners, secrets
• cicd-gitlab.md - GitLab CI/CD pipelines, artifacts, caching
• gitops-argocd.md - ArgoCD setup, app of apps pattern, sync policies
• gitops-flux.md - Flux controllers, GitOps toolkit, multi-tenancy

FinOps (Cost Optimization)

• finops-basics.md - Cost optimization principles, FinOps lifecycle
• finops-aws.md - AWS cost optimization, RI, savings plans, spot
• finops-azure.md - Azure cost management, reservations, hybrid benefit
• finops-gcp.md - GCP cost optimization, committed use, sustained use
• finops-tools.md - Cost analysis tools, Kubecost, CloudHealth, Infracost

DevSecOps (Security)

• devsecops-basics.md - Security best practices, shift-left security
• devsecops-scanning.md - SAST, DAST, SCA, container scanning
• secrets-management.md - Vault, AWS Secrets Manager, sealed secrets
• compliance.md - SOC2, HIPAA, PCI-DSS, audit logging

Infrastructure as Code

• terraform-basics.md - Terraform fundamentals, providers, state
• terraform-advanced.md - Modules, workspaces, remote state
• cloudformation-basics.md - CloudFormation templates, stacks, change sets

Utilities & Scripts

• scripts/cloudflare-deploy.py - Automate Cloudflare Worker deployments
• scripts/docker-optimize.py - Analyze and optimize Dockerfiles
• scripts/cost-analyzer.py - Cloud cost analysis and reporting
• scripts/security-scanner.py - Automated security scanning

Common Workflows

Multi-Cloud Architecture

# Edge Layer: Cloudflare Workers (global routing, caching)
# Compute Layer: AWS ECS/Lambda or Azure App Service (application logic)
# Data Layer: AWS RDS or Azure SQL (persistent storage)
# CDN/Storage: Cloudflare R2 or AWS S3 (static assets)

Benefits:
- Best-of-breed services per layer
- Geographic redundancy
- Cost optimization across providers

AWS ECS Deployment with CI/CD

# GitHub Actions workflow
name: Deploy to ECS
on: push
jobs:
  deploy:
    - Build Docker image
    - Push to ECR
    - Update ECS task definition
    - Deploy to ECS service
    - Wait for deployment stabilization

Kubernetes GitOps with ArgoCD

# Git repository structure
/apps
  /production
    - deployment.yaml
    - service.yaml
    - ingress.yaml
  /staging
    - deployment.yaml

# ArgoCD syncs cluster state from Git
# Changes: Git commit → ArgoCD detects → Auto-sync to cluster

Multi-Stage Docker Build

# Build stage
FROM node:20-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Production stage
FROM node:20-alpine
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
USER node
CMD ["node", "dist/server.js"]

FinOps Cost Optimization Workflow

# 1. Discovery: Identify untagged resources
# 2. Analysis: Right-size instances (CPU/memory utilization)
# 3. Optimization:
#    - Convert to reserved instances (predictable workloads)
#    - Use spot instances (fault-tolerant workloads)
#    - Schedule start/stop (dev environments)
# 4. Monitoring: Set budget alerts, track savings
# 5. Governance: Enforce tagging policies

DevSecOps Security Pipeline

# 1. Code Commit
# 2. SAST Scan: SonarQube, Semgrep (static code analysis)
# 3. Dependency Check: Snyk, Trivy (vulnerability scanning)
# 4. Build: Docker image
# 5. Container Scan: Trivy, Grype (image vulnerabilities)
# 6. DAST Scan: OWASP ZAP (runtime security testing)
# 7. Deploy: Only if all scans pass
# 8. Runtime Protection: Falco, AWS GuardDuty

Terraform Infrastructure Deployment

# 1. Write: Define infrastructure in .tf files
# 2. Init: terraform init (download providers)
# 3. Plan: terraform plan (preview changes)
# 4. Apply: terraform apply (create/update resources)
# 5. State: Store state in S3 with DynamoDB locking
# 6. Modules: Reuse common patterns across environments

Best Practices

DevOps

• CI/CD: Automate testing and deployment, use feature flags for progressive rollouts
• GitOps: Declarative infrastructure, Git as single source of truth, automated sync
• Monitoring: Implement observability (logs, metrics, traces), set up alerting
• Incident Management: Runbooks, postmortems, blameless culture
• Automation: Infrastructure as code, configuration management, self-service platforms

Security (DevSecOps)

• Shift Left: Security scanning early in pipeline (SAST, dependency checks)
• Secrets Management: Use Vault, AWS Secrets Manager, or sealed secrets (never in code/Git)
• Container Security: Run as non-root, minimal base images, regular scanning
• Network Security: Zero-trust architecture, service mesh, network policies
• Access Control: Least privilege IAM, MFA, temporary credentials
• Compliance: Audit logging, encryption at rest/transit, regular security reviews
• Runtime Protection: Security monitoring, intrusion detection, automated response

Cost Optimization (FinOps)

• Tagging: Enforce resource tagging for cost allocation and tracking
• Rightsizing: Analyze utilization, downsize over-provisioned resources
• Reserved Capacity: Purchase RI/savings plans for predictable workloads (up to 72% discount)
• Spot/Preemptible: Use for fault-tolerant workloads (up to 90% discount)
• Scheduling: Auto-stop dev/test environments during off-hours
• Storage Optimization: Lifecycle policies, archive to cheaper tiers, delete orphaned resources
• Monitoring: Budget alerts, cost anomaly detection, chargeback/showback
• Governance: Approval workflows for expensive resources, quota management

Kubernetes

• Resource Management: Set requests/limits, use horizontal pod autoscaling
• High Availability: Multi-zone clusters, pod disruption budgets, anti-affinity rules
• Security: RBAC, pod security policies, network policies, admission controllers
• Observability: Prometheus metrics, distributed tracing, centralized logging
• GitOps: ArgoCD/Flux for declarative deployments, automatic drift correction

Performance

• Compute: Auto-scaling, load balancing, multi-region for low latency
• Caching: CDN, in-memory caching (Redis/Memcached), edge computing
• Storage: Choose appropriate tier (SSD vs HDD), enable caching, CDN for static assets
• Containers: Multi-stage builds, minimal images, layer caching
• Databases: Connection pooling, read replicas, query optimization, indexing

Development

• Local Development: Docker Compose for consistent environments, dev containers
• Testing: Unit, integration, end-to-end tests in CI/CD pipeline
• Infrastructure as Code: Terraform/CloudFormation for repeatability
• Documentation: Architecture diagrams, runbooks, API documentation
• Version Control: Git for code and infrastructure, semantic versioning

Decision Matrix

Resources

Cloud Providers

• AWS Docs: https://docs.aws.amazon.com
• Azure Docs: https://docs.microsoft.com/azure
• GCP Docs: https://cloud.google.com/docs
• Cloudflare Docs: https://developers.cloudflare.com

Container & Orchestration

• Docker Docs: https://docs.docker.com
• Kubernetes Docs: https://kubernetes.io/docs
• Helm: https://helm.sh/docs

CI/CD & GitOps

• GitHub Actions: https://docs.github.com/actions
• GitLab CI: https://docs.gitlab.com/ee/ci/
• ArgoCD: https://argo-cd.readthedocs.io
• Flux: https://fluxcd.io/docs

Infrastructure as Code

• Terraform: https://developer.hashicorp.com/terraform
• AWS CDK: https://docs.aws.amazon.com/cdk
• Pulumi: https://www.pulumi.com/docs

Security & Compliance

• OWASP: https://owasp.org
• CIS Benchmarks: https://www.cisecurity.org/cis-benchmarks
• HashiCorp Vault: https://developer.hashicorp.com/vault

FinOps & Cost Optimization

• FinOps Foundation: https://www.finops.org
• AWS Cost Optimization: https://aws.amazon.com/pricing/cost-optimization
• Kubecost: https://www.kubecost.com

Implementation Checklist

AWS Lambda Deployment

• [ ] Install AWS CLI and SAM CLI
• [ ] Configure AWS credentials (access key, secret key)
• [ ] Create Lambda function with SAM template
• [ ] Configure IAM role and policies
• [ ] Test locally with sam local invoke
• [ ] Deploy with sam deploy
• [ ] Set up CloudWatch monitoring and alarms

AWS EKS Kubernetes Cluster

• [ ] Install kubectl, eksctl, aws-cli
• [ ] Configure AWS credentials
• [ ] Create EKS cluster with eksctl
• [ ] Configure kubectl context
• [ ] Install cluster autoscaler
• [ ] Set up Helm for package management
• [ ] Deploy applications with kubectl/Helm
• [ ] Configure ingress controller (ALB/NGINX)

Azure Deployment

• [ ] Install Azure CLI
• [ ] Login with az login
• [ ] Create resource group
• [ ] Deploy App Service or AKS
• [ ] Configure continuous deployment
• [ ] Set up monitoring with Application Insights

Kubernetes on Any Cloud

• [ ] Install kubectl and helm
• [ ] Connect to cluster (update kubeconfig)
• [ ] Create namespaces for environments
• [ ] Apply RBAC policies
• [ ] Deploy applications (deployments, services)
• [ ] Configure ingress for external access
• [ ] Set up monitoring (Prometheus, Grafana)
• [ ] Implement GitOps with ArgoCD/Flux

CI/CD Pipeline (GitHub Actions)

• [ ] Create .github/workflows/deploy.yml
• [ ] Configure secrets (cloud credentials, API keys)
• [ ] Add build and test jobs
• [ ] Add container build and push to registry
• [ ] Add deployment job to cloud platform
• [ ] Set up branch protection rules
• [ ] Enable status checks and notifications

FinOps Cost Optimization

• [ ] Implement resource tagging strategy
• [ ] Enable cost allocation tags
• [ ] Set up budget alerts
• [ ] Analyze resource utilization (CloudWatch, Azure Monitor)
• [ ] Identify rightsizing opportunities
• [ ] Purchase reserved instances for predictable workloads
• [ ] Configure auto-scaling and scheduling
• [ ] Regular cost reviews and optimization

DevSecOps Security

• [ ] Add SAST scanning to CI/CD (SonarQube, Semgrep)
• [ ] Add dependency scanning (Snyk, Trivy)
• [ ] Implement container image scanning
• [ ] Set up secrets management (Vault, cloud provider)
• [ ] Configure security groups and network policies
• [ ] Enable audit logging
• [ ] Implement security monitoring and alerting
• [ ] Regular vulnerability assessments

Cloudflare Workers

• [ ] Install Wrangler CLI
• [ ] Create Worker project
• [ ] Configure wrangler.toml (bindings, routes)
• [ ] Test locally with wrangler dev
• [ ] Deploy with wrangler deploy

Docker

• [ ] Write Dockerfile with multi-stage builds
• [ ] Create .dockerignore file
• [ ] Test build locally
• [ ] Push to registry (ECR, ACR, GCR, Docker Hub)
• [ ] Deploy to target platform

AWS Overview

Amazon Web Services (AWS) - comprehensive cloud computing platform with 200+ services across compute, storage, databases, networking, security, and more.

Account Setup

Initial Configuration

# Install AWS CLI v2
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# Verify installation
aws --version

# Configure credentials
aws configure
# AWS Access Key ID: <your-access-key>
# AWS Secret Access Key: <your-secret-key>
# Default region: us-east-1
# Default output format: json

Multiple Profiles

# Configure named profile
aws configure --profile production

# Use profile
aws s3 ls --profile production
export AWS_PROFILE=production

IAM (Identity and Access Management)

Core Concepts

• Users: Individual identities with long-term credentials
• Groups: Collections of users with shared permissions
• Roles: Assumed by services or users for temporary credentials
• Policies: JSON documents defining permissions

Best Practices

• Enable MFA for all users
• Use IAM roles for EC2 instances (not access keys)
• Follow least privilege principle
• Rotate credentials regularly
• Use AWS Organizations for multi-account management

Example Policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}

Core Services

Compute

• EC2: Virtual servers
• Lambda: Serverless functions
• ECS: Container orchestration (Docker)
• EKS: Managed Kubernetes
• Fargate: Serverless containers
• Lightsail: Simplified VPS

Storage

• S3: Object storage (industry standard)
• EBS: Block storage for EC2
• EFS: Managed NFS file system
• Glacier: Archive storage

Database

• RDS: Managed relational databases (PostgreSQL, MySQL, MariaDB, Oracle, SQL Server)
• Aurora: MySQL/PostgreSQL-compatible with better performance
• DynamoDB: NoSQL key-value and document database
• ElastiCache: In-memory caching (Redis, Memcached)
• DocumentDB: MongoDB-compatible

Networking

• VPC: Virtual Private Cloud (isolated network)
• CloudFront: CDN
• Route 53: DNS service
• ELB: Load balancers (ALB, NLB, CLB)
• API Gateway: Managed API service

Regions and Availability Zones

Global Infrastructure

• Regions: Geographic locations (us-east-1, eu-west-1, ap-southeast-1)
• Availability Zones: Isolated data centers within region (us-east-1a, us-east-1b)
• Edge Locations: CloudFront CDN points of presence

High Availability Pattern

Architecture:
  Region: us-east-1
  AZ-1 (us-east-1a):
    - EC2 instances
    - RDS primary
  AZ-2 (us-east-1b):
    - EC2 instances
    - RDS standby
  Load Balancer: Distributes traffic across AZs

Cost Management

Free Tier

• 750 hours/month EC2 t2.micro or t3.micro (12 months)
• 5GB S3 storage
• 25GB DynamoDB storage
• 1 million Lambda requests/month

Cost Optimization

• Use Reserved Instances (up to 72% discount)
• Use Savings Plans (flexible commitment-based discount)
• Use Spot Instances (up to 90% discount)
• Enable Cost Explorer and Budget Alerts
• Right-size instances based on CloudWatch metrics
• Use S3 Intelligent-Tiering or Lifecycle policies

Common CLI Commands

# EC2
aws ec2 describe-instances
aws ec2 start-instances --instance-ids i-1234567890abcdef0
aws ec2 stop-instances --instance-ids i-1234567890abcdef0

# S3
aws s3 ls
aws s3 cp file.txt s3://my-bucket/
aws s3 sync ./local-dir s3://my-bucket/remote-dir

# Lambda
aws lambda list-functions
aws lambda invoke --function-name my-function output.json

# CloudFormation
aws cloudformation create-stack --stack-name my-stack --template-body file://template.yaml
aws cloudformation describe-stacks --stack-name my-stack

Security Best Practices

1. Enable CloudTrail for audit logging 2. Use AWS Config for compliance monitoring 3. Enable GuardDuty for threat detection 4. Encrypt data at rest (S3, EBS, RDS) and in transit 5. Use Security Groups as virtual firewalls 6. Enable VPC Flow Logs for network monitoring 7. Use AWS Secrets Manager for credentials 8. Regular security assessments with AWS Inspector

Resources

• AWS Documentation: https://docs.aws.amazon.com
• AWS CLI Reference: https://awscli.amazonaws.com/v2/documentation/api/latest/index.html
• AWS Well-Architected Framework: https://aws.amazon.com/architecture/well-architected
• AWS Training: https://aws.amazon.com/training

Cloudflare Browser Rendering

Headless browser automation with Puppeteer/Playwright on Cloudflare Workers.

Setup

wrangler.toml:

name = "browser-worker"
main = "src/index.ts"
compatibility_date = "2024-01-01"

browser = { binding = "MYBROWSER" }

Basic Screenshot Worker

import puppeteer from '@cloudflare/puppeteer';

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const browser = await puppeteer.launch(env.MYBROWSER);
    const page = await browser.newPage();

    await page.goto('https://example.com', { waitUntil: 'networkidle2' });
    const screenshot = await page.screenshot({ type: 'png' });

    await browser.close();

    return new Response(screenshot, {
      headers: { 'Content-Type': 'image/png' }
    });
  }
};

Session Reuse (Cost Optimization)

// Disconnect instead of close
await browser.disconnect();

// Retrieve and reconnect
const sessions = await puppeteer.sessions(env.MYBROWSER);
const freeSession = sessions.find(s => !s.connectionId);

if (freeSession) {
  const browser = await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
}

PDF Generation

const browser = await puppeteer.launch(env.MYBROWSER);
const page = await browser.newPage();

await page.setContent(`
  <!DOCTYPE html>
  <html>
    <head>
      <style>
        body { font-family: Arial; padding: 50px; }
        h1 { color: #2c3e50; }
      </style>
    </head>
    <body>
      <h1>Certificate</h1>
      <p>Awarded to: <strong>John Doe</strong></p>
    </body>
  </html>
`);

const pdf = await page.pdf({
  format: 'A4',
  printBackground: true,
  margin: { top: '1cm', right: '1cm', bottom: '1cm', left: '1cm' }
});

await browser.close();

return new Response(pdf, {
  headers: { 'Content-Type': 'application/pdf' }
});

Durable Objects for Persistent Sessions

export class Browser {
  state: DurableObjectState;
  browser: any;
  lastUsed: number;

  constructor(state: DurableObjectState, env: Env) {
    this.state = state;
    this.lastUsed = Date.now();
  }

  async fetch(request: Request, env: Env) {
    if (!this.browser) {
      this.browser = await puppeteer.launch(env.MYBROWSER);
    }

    this.lastUsed = Date.now();
    await this.state.storage.setAlarm(Date.now() + 10000);

    const page = await this.browser.newPage();
    const url = new URL(request.url).searchParams.get('url');
    await page.goto(url);
    const screenshot = await page.screenshot();
    await page.close();

    return new Response(screenshot, {
      headers: { 'Content-Type': 'image/png' }
    });
  }

  async alarm() {
    if (Date.now() - this.lastUsed > 60000) {
      await this.browser?.close();
      this.browser = null;
    } else {
      await this.state.storage.setAlarm(Date.now() + 10000);
    }
  }
}

AI-Powered Web Scraper

import { Ai } from '@cloudflare/ai';

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const browser = await puppeteer.launch(env.MYBROWSER);
    const page = await browser.newPage();
    await page.goto('https://news.ycombinator.com');
    const content = await page.content();
    await browser.close();

    const ai = new Ai(env.AI);
    const response = await ai.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: 'Extract top 5 article titles and URLs as JSON'
        },
        { role: 'user', content: content }
      ]
    });

    return Response.json(response);
  }
};

Crawler with Queues

export default {
  async queue(batch: MessageBatch<any>, env: Env): Promise<void> {
    const browser = await puppeteer.launch(env.MYBROWSER);

    for (const message of batch.messages) {
      const page = await browser.newPage();
      await page.goto(message.body.url);

      const links = await page.evaluate(() => {
        return Array.from(document.querySelectorAll('a')).map(a => a.href);
      });

      for (const link of links) {
        await env.QUEUE.send({ url: link });
      }

      await page.close();
      message.ack();
    }

    await browser.close();
  }
};

Configuration

Timeout

await page.goto(url, {
  timeout: 60000,  // 60 seconds max
  waitUntil: 'networkidle2'
});

await page.waitForSelector('.content', { timeout: 45000 });

Viewport

await page.setViewport({ width: 1920, height: 1080 });

Screenshot Options

const screenshot = await page.screenshot({
  type: 'png',       // 'png' | 'jpeg' | 'webp'
  quality: 90,       // JPEG/WebP only
  fullPage: true,    // Full scrollable page
  clip: {            // Crop
    x: 0, y: 0,
    width: 800,
    height: 600
  }
});

Limits & Pricing

Free Plan

• 10 minutes/day
• 3 concurrent browsers
• 3 new browsers/minute

Paid Plan

• 10 hours/month included
• 30 concurrent browsers
• 30 new browsers/minute
• $0.09/hour overage
• $2.00/concurrent browser overage

Cost Optimization

1. Use disconnect() instead of close() 2. Enable Keep-Alive (10 min max) 3. Pool tabs with browser contexts 4. Cache auth state with KV 5. Implement Durable Objects cleanup

Best Practices

Session Management

• Always use disconnect() for reuse
• Implement session pooling
• Track session IDs and states

Performance

• Cache content in KV
• Use browser contexts vs multiple browsers
• Choose appropriate waitUntil strategy
• Set realistic timeouts

Error Handling

• Handle timeout errors gracefully
• Check session availability before connecting
• Validate responses before caching

Security

• Validate user-provided URLs
• Implement authentication
• Sanitize extracted content
• Set appropriate CORS headers

Troubleshooting

Timeout Errors:

await page.goto(url, {
  timeout: 60000,
  waitUntil: 'domcontentloaded'  // Faster than networkidle2
});

Memory Issues:

await page.close();  // Close pages
await browser.disconnect();  // Reuse session

Font Rendering: Use supported fonts (Noto Sans, Roboto, etc.) or inject custom:

<link href="https://fonts.googleapis.com/css2?family=Poppins" rel="stylesheet">

Key Methods

Puppeteer

• puppeteer.launch(binding) - Start browser
• puppeteer.connect(binding, sessionId) - Reconnect
• puppeteer.sessions(binding) - List sessions
• browser.newPage() - Create page
• browser.disconnect() - Disconnect (keep alive)
• browser.close() - Close (terminate)
• page.goto(url, options) - Navigate
• page.screenshot(options) - Capture
• page.pdf(options) - Generate PDF
• page.content() - Get HTML
• page.evaluate(fn) - Execute JS

Resources

• Docs: https://developers.cloudflare.com/browser-rendering/
• Puppeteer: https://pptr.dev/
• Examples: https://developers.cloudflare.com/workers/examples/

GitHub Actions CI/CD

Automation platform for building, testing, and deploying code directly from GitHub repositories.

Core Concepts

Workflow Components

• Workflow: Automated process defined in YAML (.github/workflows/)
• Event: Trigger that starts workflow (push, pull_request, schedule)
• Job: Set of steps that execute on same runner
• Step: Individual task (action or shell command)
• Runner: Server that runs workflows (GitHub-hosted or self-hosted)

Basic Workflow

name: CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run tests
        run: npm test
      
      - name: Run linter
        run: npm run lint

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Build Docker image
        run: docker build -t myapp:${{ github.sha }} .
      
      - name: Push to registry
        run: |
          echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
          docker push myapp:${{ github.sha }}

Triggers (Events)

Push Events

on:
  push:
    branches:
      - main
      - 'releases/**'
    paths:
      - 'src/**'
      - '!src/docs/**'
    tags:
      - 'v*'

Pull Request Events

on:
  pull_request:
    types: [opened, synchronize, reopened]
    branches: [main]

Scheduled Events (Cron)

on:
  schedule:
    # Every day at 2am UTC
    - cron: '0 2 * * *'

Manual Trigger

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to deploy to'
        required: true
        default: 'staging'
        type: choice
        options:
          - staging
          - production

Secrets Management

Setting Secrets

1. Repository Settings → Secrets and variables → Actions 2. Add repository secret or environment secret

Using Secrets

steps:
  - name: Deploy to AWS
    env:
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    run: aws s3 sync ./build s3://my-bucket

Environment Variables

Repository/Workflow Level

env:
  NODE_ENV: production
  API_URL: https://api.example.com

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      DEPLOY_ENV: staging
    steps:
      - name: Print variables
        run: |
          echo "NODE_ENV: $NODE_ENV"
          echo "DEPLOY_ENV: $DEPLOY_ENV"

GitHub Default Variables

steps:
  - name: Print GitHub variables
    run: |
      echo "Repository: ${{ github.repository }}"
      echo "Commit SHA: ${{ github.sha }}"
      echo "Branch: ${{ github.ref_name }}"
      echo "Actor: ${{ github.actor }}"
      echo "Event: ${{ github.event_name }}"

Matrix Builds

Test across multiple versions/platforms:

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest, macos-latest]
        node-version: [16, 18, 20]
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm test

Caching

Speed up workflows by caching dependencies:

steps:
  - uses: actions/checkout@v3
  
  # Cache npm dependencies
  - uses: actions/cache@v3
    with:
      path: ~/.npm
      key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
      restore-keys: |
        ${{ runner.os }}-node-
  
  - run: npm ci

Artifacts

Share data between jobs or download build outputs:

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - run: npm run build
      
      - name: Upload artifact
        uses: actions/upload-artifact@v3
        with:
          name: build-files
          path: dist/
          retention-days: 7

  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Download artifact
        uses: actions/download-artifact@v3
        with:
          name: build-files
          path: dist/

Docker Build and Push

Basic Docker Workflow

jobs:
  docker:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2
      
      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      
      - name: Build and push
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: |
            myapp:latest
            myapp:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Multi-Platform Builds

- name: Build multi-platform image
  uses: docker/build-push-action@v4
  with:
    platforms: linux/amd64,linux/arm64
    push: true
    tags: myapp:latest

Deployment Examples

Deploy to AWS ECS

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: us-east-1
      
      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
      
      - name: Build and push image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: myapp
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:${{ github.sha }} .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:${{ github.sha }}
      
      - name: Deploy to ECS
        uses: aws-actions/amazon-ecs-deploy-task-definition@v1
        with:
          task-definition: task-definition.json
          service: myapp-service
          cluster: production
          wait-for-service-stability: true

Deploy to Kubernetes

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Set up kubectl
        uses: azure/setup-kubectl@v3
      
      - name: Configure kubeconfig
        run: |
          echo "${{ secrets.KUBECONFIG }}" | base64 -d > kubeconfig
          export KUBECONFIG=kubeconfig
      
      - name: Deploy to cluster
        run: |
          kubectl set image deployment/myapp \
            myapp=myapp:${{ github.sha }} \
            --record
          kubectl rollout status deployment/myapp

Deploy to Cloudflare Workers

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - uses: actions/setup-node@v3
        with:
          node-version: '18'
      
      - run: npm ci
      
      - name: Deploy to Cloudflare Workers
        uses: cloudflare/wrangler-action@v3
        with:
          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          command: deploy

Reusable Workflows

Define Reusable Workflow

# .github/workflows/reusable-deploy.yml
name: Reusable Deploy
on:
  workflow_call:
    inputs:
      environment:
        required: true
        type: string
    secrets:
      deploy-token:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - run: echo "Deploying to ${{ inputs.environment }}"

Use Reusable Workflow

jobs:
  deploy-staging:
    uses: ./.github/workflows/reusable-deploy.yml
    with:
      environment: staging
    secrets:
      deploy-token: ${{ secrets.STAGING_TOKEN }}

Conditional Execution

steps:
  - name: Deploy to production
    if: github.ref == 'refs/heads/main' && github.event_name == 'push'
    run: ./deploy-production.sh
  
  - name: Run only on PR
    if: github.event_name == 'pull_request'
    run: ./pr-checks.sh
  
  - name: Run if previous step failed
    if: failure()
    run: ./cleanup.sh

Self-Hosted Runners

Set Up Self-Hosted Runner

# Download runner
mkdir actions-runner && cd actions-runner
curl -o actions-runner-linux-x64-2.311.0.tar.gz -L \
  https://github.com/actions/runner/releases/download/v2.311.0/actions-runner-linux-x64-2.311.0.tar.gz
tar xzf actions-runner-linux-x64-2.311.0.tar.gz

# Configure
./config.sh --url https://github.com/myorg/myrepo --token TOKEN

# Run
./run.sh

Use Self-Hosted Runner

jobs:
  build:
    runs-on: self-hosted
    steps:
      - uses: actions/checkout@v3
      - run: ./build.sh

Best Practices

1. Use Latest Actions: Keep actions up to date with Dependabot 2. Pin Action Versions: Use specific SHA or version (not @main) 3. Minimize Secrets: Use OIDC for cloud providers when possible 4. Cache Dependencies: Speed up builds with caching 5. Fail Fast: Use continue-on-error: false for critical steps 6. Parallel Jobs: Run independent jobs in parallel 7. Limit Workflows: Use path filters to avoid unnecessary runs 8. Monitor Usage: Track Actions minutes and storage

Resources

• GitHub Actions Documentation: https://docs.github.com/actions
• Actions Marketplace: https://github.com/marketplace?type=actions
• Workflow Syntax: https://docs.github.com/actions/reference/workflow-syntax-for-github-actions
• Security Hardening: https://docs.github.com/actions/security-guides

Cloudflare D1 & KV

D1 (SQLite Database)

Setup

# Create database
wrangler d1 create my-database

# Add to wrangler.toml
[[d1_databases]]
binding = "DB"
database_name = "my-database"
database_id = "YOUR_DATABASE_ID"

# Apply schema
wrangler d1 execute my-database --file=./schema.sql

Usage

// Query
const result = await env.DB.prepare(
  "SELECT * FROM users WHERE id = ?"
).bind(userId).first();

// Insert
await env.DB.prepare(
  "INSERT INTO users (name, email) VALUES (?, ?)"
).bind("Alice", "alice@example.com").run();

// Batch (atomic)
await env.DB.batch([
  env.DB.prepare("UPDATE accounts SET balance = balance - 100 WHERE id = ?").bind(user1),
  env.DB.prepare("UPDATE accounts SET balance = balance + 100 WHERE id = ?").bind(user2)
]);

// All results
const { results } = await env.DB.prepare("SELECT * FROM users").all();

Features

• Global read replication (low-latency reads)
• Single-writer consistency
• Standard SQLite syntax
• 25GB database size limit
• ACID transactions with batch

KV (Key-Value Store)

Setup

# Create namespace
wrangler kv:namespace create MY_KV

# Add to wrangler.toml
[[kv_namespaces]]
binding = "KV"
id = "YOUR_NAMESPACE_ID"

Usage

// Put with TTL
await env.KV.put("session:token", JSON.stringify(data), {
  expirationTtl: 3600,
  metadata: { userId: "123" }
});

// Get
const value = await env.KV.get("session:token");
const json = await env.KV.get("session:token", "json");
const buffer = await env.KV.get("session:token", "arrayBuffer");
const stream = await env.KV.get("session:token", "stream");

// Get with metadata
const { value, metadata } = await env.KV.getWithMetadata("session:token");

// Delete
await env.KV.delete("session:token");

// List
const list = await env.KV.list({ prefix: "user:" });

Features

• Sub-millisecond reads (edge-cached)
• Eventual consistency (~60 seconds globally)
• 25MB value size limit
• Automatic expiration (TTL)

Use Cases

D1

• Relational data
• Complex queries with JOINs
• ACID transactions
• User accounts, orders, inventory

KV

• Cache
• Sessions
• Feature flags
• Rate limiting
• Real-time counters

Decision Matrix

Resources

• D1: https://developers.cloudflare.com/d1/
• KV: https://developers.cloudflare.com/kv/

Cloudflare Platform Overview

Cloudflare Developer Platform: comprehensive edge computing ecosystem for full-stack applications on global network across 300+ cities.

Core Concepts

Edge Computing Model

Global Network:

• Code runs on servers in 300+ cities globally
• Requests execute from nearest location
• Ultra-low latency (<50ms typical)
• Automatic failover and redundancy

V8 Isolates:

• Lightweight execution environments (faster than containers)
• Millisecond cold starts
• Zero infrastructure management
• Automatic scaling
• Pay-per-request pricing

Key Components

Workers - Serverless functions on edge

• HTTP/scheduled/queue/email handlers
• JavaScript/TypeScript/Python/Rust support
• Max 50ms CPU (free), 30s (paid)
• 128MB memory limit

D1 - SQLite database with global read replication

• Standard SQLite syntax
• Single-writer consistency
• Global read replication
• 25GB database size limit
• Batch operations for transactions

KV - Distributed key-value store

• Sub-millisecond reads (edge-cached)
• Eventual consistency (~60s globally)
• 25MB value size limit
• Automatic TTL expiration
• Best for: cache, sessions, feature flags

R2 - Object storage (S3-compatible)

• Zero egress fees (huge cost advantage)
• Unlimited storage
• 5TB object size limit
• S3-compatible API
• Multipart upload support

Durable Objects - Stateful compute with WebSockets

• Single-instance coordination (strong consistency)
• Persistent storage (1GB limit paid)
• WebSocket support
• Automatic hibernation

Queues - Message queue system

• At-least-once delivery
• Automatic retries (exponential backoff)
• Dead-letter queue support
• Batch processing

Pages - Static site hosting + serverless functions

• Git integration (auto-deploy)
• Directory-based routing
• Framework support (Next.js, Remix, Astro, SvelteKit)
• Built-in preview deployments

Workers AI - Run AI models on edge

• LLMs (Llama 3, Mistral, Gemma, Qwen)
• Image generation (Stable Diffusion, DALL-E)
• Embeddings (BGE, GTE)
• Speech recognition (Whisper)
• No GPU management required

Browser Rendering - Headless browser automation

• Puppeteer/Playwright support
• Screenshots, PDFs, web scraping
• Session reuse for cost optimization
• MCP server support for AI agents

Architecture Patterns

Full-Stack Application

┌─────────────────────────────────────────┐
│    Cloudflare Pages (Frontend)          │
│    Next.js / Remix / Astro               │
└──────────────────┬──────────────────────┘
                   │
┌──────────────────▼──────────────────────┐
│    Workers (API Layer)                   │
│    - Routing                             │
│    - Authentication                      │
│    - Business logic                      │
└─┬──────┬──────┬──────┬──────┬───────────┘
  │      │      │      │      │
  ▼      ▼      ▼      ▼      ▼
┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌────────────┐
│ D1 │ │ KV │ │ R2 │ │ DO │ │ Workers AI │
└────┘ └────┘ └────┘ └────┘ └────────────┘

Polyglot Storage Pattern

export default {
  async fetch(request: Request, env: Env) {
    // KV: Fast cache
    const cached = await env.KV.get(key);
    if (cached) return new Response(cached);

    // D1: Structured data
    const user = await env.DB.prepare(
      "SELECT * FROM users WHERE id = ?"
    ).bind(userId).first();

    // R2: Media files
    const avatar = await env.R2_BUCKET.get(`avatars/${user.id}.jpg`);

    // Durable Objects: Real-time
    const chat = env.CHAT_ROOM.get(env.CHAT_ROOM.idFromName(roomId));

    // Queue: Async processing
    await env.EMAIL_QUEUE.send({ to: user.email, template: 'welcome' });

    return new Response(JSON.stringify({ user }));
  }
};

Wrangler CLI Essentials

Installation

npm install -g wrangler
wrangler login
wrangler init my-worker

Core Commands

# Development
wrangler dev                    # Local dev server
wrangler dev --remote          # Dev on real edge

# Deployment
wrangler deploy                # Deploy to production
wrangler deploy --dry-run      # Preview changes

# Logs
wrangler tail                  # Real-time logs
wrangler tail --format pretty  # Formatted logs

# Versions
wrangler deployments list      # List deployments
wrangler rollback [version]    # Rollback

# Secrets
wrangler secret put SECRET_NAME
wrangler secret list

Resource Management

# D1
wrangler d1 create my-db
wrangler d1 execute my-db --file=schema.sql

# KV
wrangler kv:namespace create MY_KV
wrangler kv:key put --binding=MY_KV "key" "value"

# R2
wrangler r2 bucket create my-bucket
wrangler r2 object put my-bucket/file.txt --file=./file.txt

Configuration (wrangler.toml)

name = "my-worker"
main = "src/index.ts"
compatibility_date = "2024-01-01"

# Environment variables
[vars]
ENVIRONMENT = "production"

# D1 Database
[[d1_databases]]
binding = "DB"
database_name = "my-database"
database_id = "YOUR_DATABASE_ID"

# KV Namespace
[[kv_namespaces]]
binding = "KV"
id = "YOUR_NAMESPACE_ID"

# R2 Bucket
[[r2_buckets]]
binding = "R2_BUCKET"
bucket_name = "my-bucket"

# Durable Objects
[[durable_objects.bindings]]
name = "COUNTER"
class_name = "Counter"
script_name = "my-worker"

# Queues
[[queues.producers]]
binding = "MY_QUEUE"
queue = "my-queue"

# Workers AI
[ai]
binding = "AI"

# Cron triggers
[triggers]
crons = ["0 0 * * *"]

Best Practices

Performance

• Keep Workers lightweight (<1MB bundled)
• Use bindings over fetch (faster than HTTP)
• Leverage KV and Cache API for frequently accessed data
• Use D1 batch for multiple queries
• Stream large responses

Security

• Use wrangler secret for API keys
• Separate production/staging/development environments
• Validate user input
• Implement rate limiting (KV or Durable Objects)
• Configure proper CORS headers

Cost Optimization

• R2 for large files (zero egress fees vs S3)
• KV for caching (reduce D1/R2 requests)
• Request deduplication with caching
• Efficient D1 queries (proper indexing)
• Monitor usage via Cloudflare Analytics

Decision Matrix

Resources

• Docs: https://developers.cloudflare.com
• Wrangler: https://developers.cloudflare.com/workers/wrangler/
• Discord: https://discord.cloudflare.com
• Examples: https://developers.cloudflare.com/workers/examples/
• Status: https://www.cloudflarestatus.com

Cloudflare R2 Storage

S3-compatible object storage with zero egress fees.

Quick Start

Create Bucket

wrangler r2 bucket create my-bucket
wrangler r2 bucket create my-bucket --location=wnam

Locations: wnam, enam, weur, eeur, apac

Upload Object

wrangler r2 object put my-bucket/file.txt --file=./local-file.txt

Workers Binding

wrangler.toml:

[[r2_buckets]]
binding = "MY_BUCKET"
bucket_name = "my-bucket"

Worker:

// Put
await env.MY_BUCKET.put('user-uploads/photo.jpg', imageData, {
  httpMetadata: {
    contentType: 'image/jpeg',
    cacheControl: 'public, max-age=31536000'
  },
  customMetadata: {
    uploadedBy: userId,
    uploadDate: new Date().toISOString()
  }
});

// Get
const object = await env.MY_BUCKET.get('large-file.mp4');
if (!object) {
  return new Response('Not found', { status: 404 });
}

return new Response(object.body, {
  headers: {
    'Content-Type': object.httpMetadata.contentType,
    'ETag': object.etag
  }
});

// List
const listed = await env.MY_BUCKET.list({
  prefix: 'user-uploads/',
  limit: 100
});

// Delete
await env.MY_BUCKET.delete('old-file.txt');

// Head (check existence)
const object = await env.MY_BUCKET.head('file.txt');
if (object) {
  console.log('Size:', object.size);
}

S3 API Integration

AWS CLI

# Configure
aws configure
# Access Key ID: <your-key-id>
# Secret Access Key: <your-secret>
# Region: auto

# Operations
aws s3api list-buckets --endpoint-url https://<accountid>.r2.cloudflarestorage.com

aws s3 cp file.txt s3://my-bucket/ --endpoint-url https://<accountid>.r2.cloudflarestorage.com

# Presigned URL
aws s3 presign s3://my-bucket/file.txt --endpoint-url https://<accountid>.r2.cloudflarestorage.com --expires-in 3600

JavaScript (AWS SDK v3)

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  region: "auto",
  endpoint: `https://${accountId}.r2.cloudflarestorage.com`,
  credentials: {
    accessKeyId: process.env.R2_ACCESS_KEY_ID,
    secretAccessKey: process.env.R2_SECRET_ACCESS_KEY
  }
});

await s3.send(new PutObjectCommand({
  Bucket: "my-bucket",
  Key: "file.txt",
  Body: fileContents
}));

Python (Boto3)

import boto3

s3 = boto3.client(
    service_name='s3',
    endpoint_url=f'https://{account_id}.r2.cloudflarestorage.com',
    aws_access_key_id=access_key_id,
    aws_secret_access_key=secret_access_key,
    region_name='auto'
)

s3.upload_fileobj(file_obj, 'my-bucket', 'file.txt')
s3.download_file('my-bucket', 'file.txt', './local-file.txt')

Multipart Uploads

For files >100MB:

const multipart = await env.MY_BUCKET.createMultipartUpload('large-file.mp4');

// Upload parts (5MiB - 5GiB each, max 10,000 parts)
const part1 = await multipart.uploadPart(1, chunk1);
const part2 = await multipart.uploadPart(2, chunk2);

// Complete
const object = await multipart.complete([part1, part2]);

Rclone (Large Files)

rclone config  # Configure Cloudflare R2

# Upload with optimization
rclone copy large-video.mp4 r2:my-bucket/ \
  --s3-upload-cutoff=100M \
  --s3-chunk-size=100M

Public Buckets

Enable Public Access

1. Dashboard → R2 → Bucket → Settings → Public Access 2. Add custom domain (recommended) or use r2.dev

r2.dev (rate-limited):

https://pub-<hash>.r2.dev/file.txt

Custom domain (production): Cloudflare handles DNS/TLS automatically

CORS Configuration

wrangler r2 bucket cors put my-bucket --rules '[
  {
    "AllowedOrigins": ["https://example.com"],
    "AllowedMethods": ["GET", "PUT", "POST"],
    "AllowedHeaders": ["*"],
    "ExposeHeaders": ["ETag"],
    "MaxAgeSeconds": 3600
  }
]'

Lifecycle Rules

wrangler r2 bucket lifecycle put my-bucket --rules '[
  {
    "action": {"type": "AbortIncompleteMultipartUpload"},
    "filter": {},
    "abortIncompleteMultipartUploadDays": 7
  },
  {
    "action": {"type": "Transition", "storageClass": "InfrequentAccess"},
    "filter": {"prefix": "archives/"},
    "daysFromCreation": 90
  }
]'

Event Notifications

wrangler r2 bucket notification create my-bucket \
  --queue=my-queue \
  --event-type=object-create

Supported events: object-create, object-delete

Data Migration

Sippy (Incremental)

wrangler r2 bucket sippy enable my-bucket \
  --provider=aws \
  --bucket=source-bucket \
  --region=us-east-1 \
  --access-key-id=$AWS_KEY \
  --secret-access-key=$AWS_SECRET

Objects migrate on first request.

Super Slurper (Bulk)

Use dashboard for one-time complete migration from AWS, GCS, Azure.

Best Practices

Performance

• Use Cloudflare Cache with custom domains
• Multipart uploads for files >100MB
• Rclone for batch operations
• Location hints match user geography

Security

• Never commit Access Keys
• Use environment variables
• Bucket-scoped tokens for least privilege
• Presigned URLs for temporary access
• Enable Cloudflare Access for protection

Cost Optimization

• Infrequent Access storage for archives (30+ days)
• Lifecycle rules to auto-transition/delete
• Larger multipart chunks = fewer Class A operations
• Monitor usage via dashboard

Naming

• Bucket names: lowercase, hyphens, 3-63 chars
• Avoid sequential prefixes (use hashed for performance)
• No dots in bucket names if using custom domains with TLS

Limits

• Buckets per account: 1,000
• Object size: 5TB max
• Lifecycle rules: 1,000 per bucket
• Event notification rules: 100 per bucket
• r2.dev rate limit: 1,000 req/min (use custom domains)

Troubleshooting

401 Unauthorized:

• Verify Access Keys
• Check endpoint URL includes account ID
• Ensure region is "auto"

403 Forbidden:

• Check bucket permissions
• Verify CORS configuration
• Confirm bucket exists

Presigned URLs not working:

• Verify CORS configuration
• Check URL expiry time
• Ensure origin matches CORS rules

Resources

• Docs: https://developers.cloudflare.com/r2/
• Wrangler: https://developers.cloudflare.com/r2/reference/wrangler-commands/
• S3 Compatibility: https://developers.cloudflare.com/r2/api/s3/api/
• Workers API: https://developers.cloudflare.com/r2/api/workers/

Cloudflare Workers Advanced Patterns

Advanced techniques for optimization, performance, and complex workflows.

Session Reuse and Connection Pooling

Durable Objects for Persistent Sessions

export class Browser {
  state: DurableObjectState;
  browser: any;
  lastUsed: number;

  constructor(state: DurableObjectState, env: Env) {
    this.state = state;
    this.lastUsed = Date.now();
  }

  async fetch(request: Request, env: Env) {
    if (!this.browser) {
      this.browser = await puppeteer.launch(env.MYBROWSER);
    }

    this.lastUsed = Date.now();
    await this.state.storage.setAlarm(Date.now() + 10000);

    const page = await this.browser.newPage();
    await page.goto(new URL(request.url).searchParams.get('url'));
    const screenshot = await page.screenshot();
    await page.close();

    return new Response(screenshot);
  }

  async alarm() {
    if (Date.now() - this.lastUsed > 60000) {
      await this.browser?.close();
      this.browser = null;
    } else {
      await this.state.storage.setAlarm(Date.now() + 10000);
    }
  }
}

Multi-Tier Caching Strategy

const CACHE_TTL = 3600;

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    const cache = caches.default;
    const cacheKey = new Request(request.url);

    // 1. Check edge cache
    let response = await cache.match(cacheKey);
    if (response) return response;

    // 2. Check KV cache
    const kvCached = await env.MY_KV.get(request.url);
    if (kvCached) {
      response = new Response(kvCached);
      ctx.waitUntil(cache.put(cacheKey, response.clone()));
      return response;
    }

    // 3. Fetch from origin
    response = await fetch(request);

    // 4. Store in both caches
    ctx.waitUntil(Promise.all([
      cache.put(cacheKey, response.clone()),
      env.MY_KV.put(request.url, await response.clone().text(), {
        expirationTtl: CACHE_TTL
      })
    ]));

    return response;
  }
};

WebSocket with Durable Objects

export class ChatRoom {
  state: DurableObjectState;
  sessions: Set<WebSocket>;

  constructor(state: DurableObjectState) {
    this.state = state;
    this.sessions = new Set();
  }

  async fetch(request: Request) {
    const pair = new WebSocketPair();
    const [client, server] = Object.values(pair);

    this.state.acceptWebSocket(server);
    this.sessions.add(server);

    return new Response(null, { status: 101, webSocket: client });
  }

  async webSocketMessage(ws: WebSocket, message: string) {
    // Broadcast to all connected clients
    for (const session of this.sessions) {
      session.send(message);
    }
  }

  async webSocketClose(ws: WebSocket) {
    this.sessions.delete(ws);
  }
}

Queue-Based Crawler

export default {
  async queue(batch: MessageBatch<any>, env: Env): Promise<void> {
    const browser = await puppeteer.launch(env.MYBROWSER);

    for (const message of batch.messages) {
      const page = await browser.newPage();
      await page.goto(message.body.url);

      // Extract links
      const links = await page.evaluate(() => {
        return Array.from(document.querySelectorAll('a'))
          .map(a => a.href);
      });

      // Queue new links
      for (const link of links) {
        await env.QUEUE.send({ url: link });
      }

      await page.close();
      message.ack();
    }

    await browser.close();
  }
};

Authentication Pattern

import { sign, verify } from 'hono/jwt';

async function authenticate(request: Request, env: Env): Promise<any> {
  const authHeader = request.headers.get('Authorization');

  if (!authHeader?.startsWith('Bearer ')) {
    throw new Error('Missing token');
  }

  const token = authHeader.substring(7);
  const payload = await verify(token, env.JWT_SECRET);

  return payload;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    try {
      const user = await authenticate(request, env);
      return new Response(`Hello ${user.name}`);
    } catch (error) {
      return new Response('Unauthorized', { status: 401 });
    }
  }
};

Code Splitting

// Lazy load large dependencies
export default {
  async fetch(request: Request): Promise<Response> {
    const url = new URL(request.url);

    if (url.pathname === '/heavy') {
      const { processHeavy } = await import('./heavy');
      return processHeavy(request);
    }

    return new Response('OK');
  }
};

Batch Operations with D1

// Efficient bulk inserts
const statements = users.map(user =>
  env.DB.prepare('INSERT INTO users (name, email) VALUES (?, ?)')
    .bind(user.name, user.email)
);

await env.DB.batch(statements);

Stream Processing

const { readable, writable } = new TransformStream({
  transform(chunk, controller) {
    // Process chunk
    controller.enqueue(chunk);
  }
});

response.body.pipeTo(writable);
return new Response(readable);

AI-Powered Web Scraper

import { Ai } from '@cloudflare/ai';

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    // Render page
    const browser = await puppeteer.launch(env.MYBROWSER);
    const page = await browser.newPage();
    await page.goto('https://news.ycombinator.com');
    const content = await page.content();
    await browser.close();

    // Extract with AI
    const ai = new Ai(env.AI);
    const response = await ai.run('@cf/meta/llama-3-8b-instruct', {
      messages: [
        {
          role: 'system',
          content: 'Extract top 5 article titles and URLs as JSON array'
        },
        { role: 'user', content: content }
      ]
    });

    return Response.json(response);
  }
};

Performance Optimization

Bundle Size

• Keep Workers <1MB bundled
• Remove unused dependencies
• Use code splitting
• Check with: wrangler deploy --dry-run --outdir=dist

Cold Starts

• Minimize initialization code
• Use bindings over fetch
• Avoid large imports at top level

Memory Management

• Close pages when done: await page.close()
• Disconnect browsers: await browser.disconnect()
• Implement cleanup alarms in Durable Objects

Request Optimization

• Use server-side filtering with --filter
• Batch operations with D1 .batch()
• Stream large responses
• Implement proper caching

Monitoring & Debugging

# Real-time logs
wrangler tail --format pretty

# Filter by status
wrangler tail --status error

# Check deployments
wrangler deployments list

# Rollback
wrangler rollback [version-id]

Production Checklist

• [ ] Multi-stage error handling implemented
• [ ] Rate limiting configured
• [ ] Caching strategy in place
• [ ] Secrets managed with wrangler secret
• [ ] Health checks implemented
• [ ] Monitoring alerts configured
• [ ] Session reuse for browser rendering
• [ ] Resource cleanup (pages, browsers)
• [ ] Proper timeout configurations
• [ ] CI/CD pipeline set up

Resources

• Advanced Patterns: https://developers.cloudflare.com/workers/examples/
• Durable Objects: https://developers.cloudflare.com/workers/runtime-apis/durable-objects/
• Performance: https://developers.cloudflare.com/workers/platform/limits/

Cloudflare Workers Runtime APIs

Key runtime APIs for Workers development.

Fetch API

// Subrequest
const response = await fetch('https://api.example.com/data', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ key: 'value' }),
  cf: {
    cacheTtl: 3600,
    cacheEverything: true
  }
});

const data = await response.json();

Headers API

// Read headers
const userAgent = request.headers.get('User-Agent');

// Cloudflare-specific
const country = request.cf?.country;
const colo = request.cf?.colo;
const clientIP = request.headers.get('CF-Connecting-IP');

// Set headers
const headers = new Headers();
headers.set('Content-Type', 'application/json');
headers.append('X-Custom-Header', 'value');

HTMLRewriter

export default {
  async fetch(request: Request): Promise<Response> {
    const response = await fetch(request);

    return new HTMLRewriter()
      .on('title', {
        element(element) {
          element.setInnerContent('New Title');
        }
      })
      .on('a[href]', {
        element(element) {
          const href = element.getAttribute('href');
          element.setAttribute('href', href.replace('http://', 'https://'));
        }
      })
      .transform(response);
  }
};

WebSockets

export default {
  async fetch(request: Request): Promise<Response> {
    const upgradeHeader = request.headers.get('Upgrade');
    if (upgradeHeader !== 'websocket') {
      return new Response('Expected WebSocket', { status: 426 });
    }

    const pair = new WebSocketPair();
    const [client, server] = Object.values(pair);

    server.accept();

    server.addEventListener('message', (event) => {
      server.send(`Echo: ${event.data}`);
    });

    return new Response(null, {
      status: 101,
      webSocket: client
    });
  }
};

Streams API

const { readable, writable } = new TransformStream();

const writer = writable.getWriter();
writer.write(new TextEncoder().encode('chunk 1'));
writer.write(new TextEncoder().encode('chunk 2'));
writer.close();

return new Response(readable, {
  headers: { 'Content-Type': 'text/plain' }
});

Web Crypto API

// Generate hash
const data = new TextEncoder().encode('message');
const hashBuffer = await crypto.subtle.digest('SHA-256', data);
const hashArray = Array.from(new Uint8Array(hashBuffer));
const hashHex = hashArray.map(b => b.toString(16).padStart(2, '0')).join('');

// HMAC signature
const key = await crypto.subtle.importKey(
  'raw',
  new TextEncoder().encode('secret'),
  { name: 'HMAC', hash: 'SHA-256' },
  false,
  ['sign', 'verify']
);

const signature = await crypto.subtle.sign('HMAC', key, data);
const valid = await crypto.subtle.verify('HMAC', key, signature, data);

// Random values
const randomBytes = crypto.getRandomValues(new Uint8Array(32));
const uuid = crypto.randomUUID();

Encoding APIs

// TextEncoder
const encoder = new TextEncoder();
const bytes = encoder.encode('Hello');

// TextDecoder
const decoder = new TextDecoder();
const text = decoder.decode(bytes);

// Base64
const base64 = btoa('Hello');
const decoded = atob(base64);

URL API

const url = new URL(request.url);
const hostname = url.hostname;
const pathname = url.pathname;
const search = url.search;

// Query parameters
const name = url.searchParams.get('name');
url.searchParams.set('page', '2');
url.searchParams.delete('old');

FormData API

// Parse form data
const formData = await request.formData();
const name = formData.get('name');
const file = formData.get('file');

// Create form data
const form = new FormData();
form.append('name', 'value');
form.append('file', blob, 'filename.txt');

Response Types

// Text
return new Response('Hello');

// JSON
return Response.json({ message: 'Hello' });

// Stream
return new Response(readable);

// Redirect
return Response.redirect('https://example.com', 302);

// Error
return new Response('Not Found', { status: 404 });

Request Cloning

// Clone for multiple reads
const clone = request.clone();
const body1 = await request.json();
const body2 = await clone.json();

AbortController

const controller = new AbortController();
const { signal } = controller;

setTimeout(() => controller.abort(), 5000);

try {
  const response = await fetch('https://slow-api.com', { signal });
} catch (error) {
  if (error.name === 'AbortError') {
    console.log('Request timed out');
  }
}

Scheduling APIs

// setTimeout
const timeoutId = setTimeout(() => {
  console.log('Delayed');
}, 1000);

// setInterval
const intervalId = setInterval(() => {
  console.log('Repeated');
}, 1000);

// Clear
clearTimeout(timeoutId);
clearInterval(intervalId);

Console API

console.log('Info message');
console.error('Error message');
console.warn('Warning message');
console.debug('Debug message');

// Structured logging
console.log(JSON.stringify({
  level: 'info',
  message: 'Request processed',
  url: request.url,
  timestamp: new Date().toISOString()
}));

Performance API

const start = performance.now();
await processRequest();
const duration = performance.now() - start;
console.log(`Processed in ${duration}ms`);

Bindings Reference

KV Operations

await env.KV.put(key, value, { expirationTtl: 3600, metadata: { userId: '123' } });
const value = await env.KV.get(key, 'json');
const { value, metadata } = await env.KV.getWithMetadata(key);
await env.KV.delete(key);
const list = await env.KV.list({ prefix: 'user:' });

D1 Operations

const result = await env.DB.prepare('SELECT * FROM users WHERE id = ?').bind(userId).first();
const { results } = await env.DB.prepare('SELECT * FROM users').all();
await env.DB.prepare('INSERT INTO users (name) VALUES (?)').bind(name).run();
await env.DB.batch([stmt1, stmt2, stmt3]);

R2 Operations

await env.R2.put(key, value, { httpMetadata: { contentType: 'image/jpeg' } });
const object = await env.R2.get(key);
await env.R2.delete(key);
const list = await env.R2.list({ prefix: 'uploads/' });
const multipart = await env.R2.createMultipartUpload(key);

Queue Operations

await env.QUEUE.send({ type: 'email', to: 'user@example.com' });
await env.QUEUE.sendBatch([{ body: msg1 }, { body: msg2 }]);

Workers AI

const response = await env.AI.run('@cf/meta/llama-3-8b-instruct', {
  messages: [{ role: 'user', content: 'What is edge computing?' }]
});

Resources

• Runtime APIs: https://developers.cloudflare.com/workers/runtime-apis/
• Web Standards: https://developers.cloudflare.com/workers/runtime-apis/web-standards/
• Bindings: https://developers.cloudflare.com/workers/runtime-apis/bindings/

Cloudflare Workers Basics

Getting started with Cloudflare Workers: serverless functions that run on edge network across 300+ cities.

Handler Types

Fetch Handler (HTTP Requests)

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    return new Response('Hello World!');
  }
};

Scheduled Handler (Cron Jobs)

export default {
  async scheduled(event: ScheduledEvent, env: Env, ctx: ExecutionContext): Promise<void> {
    await fetch('https://api.example.com/cleanup');
  }
};

Configure in wrangler.toml:

[triggers]
crons = ["0 0 * * *"]  # Daily at midnight

Queue Handler (Message Processing)

export default {
  async queue(batch: MessageBatch, env: Env, ctx: ExecutionContext): Promise<void> {
    for (const message of batch.messages) {
      await processMessage(message.body);
      message.ack();  // Acknowledge success
    }
  }
};

Email Handler (Email Routing)

export default {
  async email(message: ForwardableEmailMessage, env: Env, ctx: ExecutionContext): Promise<void> {
    await message.forward('destination@example.com');
  }
};

Request/Response Basics

Parsing Request

const url = new URL(request.url);
const method = request.method;
const headers = request.headers;

// Query parameters
const name = url.searchParams.get('name');

// JSON body
const data = await request.json();

// Text body
const text = await request.text();

// Form data
const formData = await request.formData();

Creating Response

// Text response
return new Response('Hello', { status: 200 });

// JSON response
return new Response(JSON.stringify({ message: 'Hello' }), {
  status: 200,
  headers: { 'Content-Type': 'application/json' }
});

// Stream response
return new Response(readable, {
  headers: { 'Content-Type': 'text/plain' }
});

// Redirect
return Response.redirect('https://example.com', 302);

Routing Patterns

URL-Based Routing

export default {
  async fetch(request: Request): Promise<Response> {
    const url = new URL(request.url);

    switch (url.pathname) {
      case '/':
        return new Response('Home');
      case '/about':
        return new Response('About');
      default:
        return new Response('Not Found', { status: 404 });
    }
  }
};

Using Hono Framework (Recommended)

import { Hono } from 'hono';

const app = new Hono();

app.get('/', (c) => c.text('Home'));
app.get('/api/users/:id', async (c) => {
  const id = c.req.param('id');
  const user = await getUser(id);
  return c.json(user);
});

export default app;

Working with Bindings

Environment Variables

# wrangler.toml
[vars]
API_URL = "https://api.example.com"

const apiUrl = env.API_URL;

KV Namespace

// Put with TTL
await env.KV.put('session:token', JSON.stringify(data), {
  expirationTtl: 3600
});

// Get
const data = await env.KV.get('session:token', 'json');

// Delete
await env.KV.delete('session:token');

// List with prefix
const list = await env.KV.list({ prefix: 'user:123:' });

D1 Database

// Query
const result = await env.DB.prepare(
  'SELECT * FROM users WHERE id = ?'
).bind(userId).first();

// Insert
await env.DB.prepare(
  'INSERT INTO users (name, email) VALUES (?, ?)'
).bind('Alice', 'alice@example.com').run();

// Batch (atomic)
await env.DB.batch([
  env.DB.prepare('UPDATE accounts SET balance = balance - 100 WHERE id = ?').bind(1),
  env.DB.prepare('UPDATE accounts SET balance = balance + 100 WHERE id = ?').bind(2)
]);

R2 Bucket

// Put object
await env.R2_BUCKET.put('path/to/file.jpg', fileBuffer, {
  httpMetadata: {
    contentType: 'image/jpeg'
  }
});

// Get object
const object = await env.R2_BUCKET.get('path/to/file.jpg');
if (!object) {
  return new Response('Not found', { status: 404 });
}

// Stream response
return new Response(object.body, {
  headers: {
    'Content-Type': object.httpMetadata?.contentType || 'application/octet-stream'
  }
});

// Delete
await env.R2_BUCKET.delete('path/to/file.jpg');

Context API

waitUntil (Background Tasks)

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    // Run analytics after response sent
    ctx.waitUntil(
      fetch('https://analytics.example.com/log', {
        method: 'POST',
        body: JSON.stringify({ url: request.url })
      })
    );

    return new Response('OK');
  }
};

passThroughOnException

// Continue to origin on error
ctx.passThroughOnException();

// Your code that might throw
const data = await riskyOperation();

Error Handling

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    try {
      const response = await processRequest(request, env);
      return response;
    } catch (error) {
      console.error('Error:', error);

      // Log to external service
      ctx.waitUntil(
        fetch('https://logging.example.com/error', {
          method: 'POST',
          body: JSON.stringify({
            error: error.message,
            url: request.url
          })
        })
      );

      return new Response('Internal Server Error', { status: 500 });
    }
  }
};

CORS

function corsHeaders(origin: string) {
  return {
    'Access-Control-Allow-Origin': origin,
    'Access-Control-Allow-Methods': 'GET, POST, PUT, DELETE, OPTIONS',
    'Access-Control-Allow-Headers': 'Content-Type, Authorization',
    'Access-Control-Max-Age': '86400'
  };
}

export default {
  async fetch(request: Request): Promise<Response> {
    const origin = request.headers.get('Origin') || '*';

    // Handle preflight
    if (request.method === 'OPTIONS') {
      return new Response(null, { headers: corsHeaders(origin) });
    }

    // Handle request
    const response = await handleRequest(request);
    const headers = new Headers(response.headers);
    Object.entries(corsHeaders(origin)).forEach(([key, value]) => {
      headers.set(key, value);
    });

    return new Response(response.body, {
      status: response.status,
      headers
    });
  }
};

Cache API

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    const cache = caches.default;
    const cacheKey = new Request(request.url);

    // Check cache
    let response = await cache.match(cacheKey);
    if (response) return response;

    // Fetch from origin
    response = await fetch(request);

    // Cache response
    ctx.waitUntil(cache.put(cacheKey, response.clone()));

    return response;
  }
};

Secrets Management

# Add secret
wrangler secret put API_KEY
# Enter value when prompted

# Use in Worker
const apiKey = env.API_KEY;

Local Development

# Start local dev server
wrangler dev

# Test with remote edge
wrangler dev --remote

# Custom port
wrangler dev --port 8080

# Access at http://localhost:8787

Deployment

# Deploy to production
wrangler deploy

# Deploy to specific environment
wrangler deploy --env staging

# Preview deployment
wrangler deploy --dry-run

Common Patterns

API Gateway

import { Hono } from 'hono';

const app = new Hono();

app.get('/api/users', async (c) => {
  const users = await c.env.DB.prepare('SELECT * FROM users').all();
  return c.json(users.results);
});

app.post('/api/users', async (c) => {
  const { name, email } = await c.req.json();
  await c.env.DB.prepare(
    'INSERT INTO users (name, email) VALUES (?, ?)'
  ).bind(name, email).run();
  return c.json({ success: true }, 201);
});

export default app;

Rate Limiting

async function rateLimit(ip: string, env: Env): Promise<boolean> {
  const key = `ratelimit:${ip}`;
  const limit = 100;
  const window = 60;

  const current = await env.KV.get(key);
  const count = current ? parseInt(current) : 0;

  if (count >= limit) return false;

  await env.KV.put(key, (count + 1).toString(), {
    expirationTtl: window
  });

  return true;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const ip = request.headers.get('CF-Connecting-IP') || 'unknown';

    if (!await rateLimit(ip, env)) {
      return new Response('Rate limit exceeded', { status: 429 });
    }

    return new Response('OK');
  }
};

Resources

• Docs: https://developers.cloudflare.com/workers/
• Examples: https://developers.cloudflare.com/workers/examples/
• Runtime APIs: https://developers.cloudflare.com/workers/runtime-apis/

DevSecOps Basics

Security integrated into DevOps practices - "shift left" approach bringing security early in the development lifecycle.

Core Principles

Shift-Left Security

• Security testing early in development (not at the end)
• Automated security scanning in CI/CD pipeline
• Developers empowered with security tools and knowledge
• Fast feedback on security issues

Security as Code

• Infrastructure security in version control
• Automated compliance checks
• Security policies as code
• Immutable infrastructure

Continuous Monitoring

• Real-time security monitoring
• Automated threat detection
• Incident response automation
• Security metrics and dashboards

Security Scanning Types

1. SAST (Static Application Security Testing)

Analyzes source code for vulnerabilities without executing it.

Tools:

• SonarQube: Code quality and security
• Semgrep: Pattern-based code scanning
• Checkmarx: Enterprise SAST
• CodeQL: GitHub's code analysis engine

Example: SonarQube in CI/CD

# GitHub Actions
- name: SonarQube Scan
  uses: SonarSource/sonarqube-scan-action@master
  env:
    SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
    SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}

2. DAST (Dynamic Application Security Testing)

Tests running applications for vulnerabilities (black-box testing).

Tools:

• OWASP ZAP: Open-source web app scanner
• Burp Suite: Security testing platform
• Acunetix: Automated web vulnerability scanner

Example: OWASP ZAP

# Run ZAP baseline scan
docker run -v $(pwd):/zap/wrk/:rw \
  -t owasp/zap2docker-stable zap-baseline.py \
  -t https://example.com \
  -r zap-report.html

3. SCA (Software Composition Analysis)

Identifies vulnerabilities in dependencies and open-source libraries.

Tools:

• Snyk: Vulnerability scanning for dependencies
• Trivy: Container and dependency scanner
• Dependabot: GitHub's automated dependency updates
• WhiteSource: Open-source security management

Example: Snyk in CI/CD

# GitHub Actions
- name: Run Snyk to check for vulnerabilities
  uses: snyk/actions/node@master
  env:
    SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
  with:
    args: --severity-threshold=high

4. Container Scanning

Scans container images for vulnerabilities and misconfigurations.

Tools:

• Trivy: Comprehensive container scanner
• Grype: Vulnerability scanner for container images
• Clair: Static analysis for container vulnerabilities
• AWS ECR Scanning: Built-in scanning for ECR images

Example: Trivy

# Scan Docker image
trivy image nginx:latest

# Scan image in CI/CD and fail on HIGH/CRITICAL
trivy image --exit-code 1 --severity HIGH,CRITICAL myapp:latest

# Scan Kubernetes manifests
trivy config ./k8s/

Secrets Management

Never Commit Secrets to Git

Scan for leaked secrets:

# TruffleHog - find secrets in git history
docker run -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest \
  git file:///pwd --json

# GitGuardian - monitor for exposed secrets
# Integrate with GitHub/GitLab for real-time alerts

Secrets Management Solutions

HashiCorp Vault

# Start Vault server
vault server -dev

# Store secret
vault kv put secret/database password=supersecret

# Retrieve secret
vault kv get secret/database

# Use in app via API or SDK
curl -H "X-Vault-Token: $VAULT_TOKEN" \
  http://127.0.0.1:8200/v1/secret/data/database

AWS Secrets Manager

# Create secret
aws secretsmanager create-secret \
  --name prod/db/password \
  --secret-string "supersecret"

# Retrieve secret
aws secretsmanager get-secret-value \
  --secret-id prod/db/password \
  --query SecretString --output text

# Automatic rotation enabled
aws secretsmanager rotate-secret \
  --secret-id prod/db/password \
  --rotation-lambda-arn arn:aws:lambda:...

Kubernetes Sealed Secrets

# Install sealed-secrets controller
kubectl apply -f https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.24.0/controller.yaml

# Seal a secret
echo -n supersecret | kubectl create secret generic db-secret \
  --dry-run=client --from-file=password=/dev/stdin -o yaml | \
  kubeseal -o yaml > sealed-secret.yaml

# Commit sealed-secret.yaml to Git (encrypted)
kubectl apply -f sealed-secret.yaml

Security in CI/CD Pipeline

Complete Security Pipeline

# GitHub Actions example
name: Security Pipeline
on: [push, pull_request]

jobs:
  security:
    runs-on: ubuntu-latest
    steps:
      # 1. Code checkout
      - uses: actions/checkout@v3
      
      # 2. Secret scanning
      - name: TruffleHog Scan
        uses: trufflesecurity/trufflehog@main
        with:
          path: ./
      
      # 3. SAST - Static code analysis
      - name: SonarQube Scan
        uses: SonarSource/sonarqube-scan-action@master
        env:
          SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
      
      # 4. SCA - Dependency check
      - name: Snyk Dependency Scan
        uses: snyk/actions/node@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
      
      # 5. Build Docker image
      - name: Build Image
        run: docker build -t myapp:${{ github.sha }} .
      
      # 6. Container scanning
      - name: Trivy Container Scan
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: myapp:${{ github.sha }}
          severity: 'HIGH,CRITICAL'
          exit-code: '1'
      
      # 7. Infrastructure as Code scanning
      - name: Trivy IaC Scan
        run: trivy config ./terraform/
      
      # 8. Deploy (only if all scans pass)
      - name: Deploy
        if: success()
        run: kubectl apply -f k8s/

Compliance and Governance

Compliance Frameworks

• SOC 2: Security, availability, processing integrity, confidentiality, privacy
• HIPAA: Healthcare data protection
• PCI-DSS: Payment card data security
• GDPR: EU data protection regulation
• ISO 27001: Information security management

Compliance Automation

AWS Config

# Enable AWS Config
aws configservice put-configuration-recorder \
  --configuration-recorder name=default,roleARN=arn:aws:iam::...

# Add compliance rules
aws configservice put-config-rule \
  --config-rule file://encrypted-volumes.json

# Check compliance
aws configservice get-compliance-details-by-config-rule \
  --config-rule-name encrypted-volumes

Policy as Code (OPA - Open Policy Agent)

# Deny deployments without resource limits
package kubernetes.admission

deny[msg] {
  input.request.kind.kind == "Pod"
  container := input.request.object.spec.containers[_]
  not container.resources.limits
  msg := sprintf("Container %v must have resource limits", [container.name])
}

Network Security

Zero Trust Architecture

• No implicit trust based on network location
• Verify every access request
• Least privilege access
• Micro-segmentation

Kubernetes Network Policies

# Deny all ingress traffic by default
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
spec:
  podSelector: {}
  policyTypes:
  - Ingress

---
# Allow specific traffic
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: frontend
    ports:
    - protocol: TCP
      port: 8080

Service Mesh (Istio/Linkerd)

• mTLS between services
• Traffic encryption
• Fine-grained access control
• Distributed tracing for security events

Runtime Security

Container Runtime Protection

Falco (CNCF Project)

# Detect suspicious activity
rules:
  - rule: Unauthorized Process
    desc: Detect unexpected process in container
    condition: container and not proc.name in (allowed_processes)
    output: "Unauthorized process started (proc=%proc.name container=%container.id)"
    priority: WARNING

AWS GuardDuty

• Threat detection for AWS accounts
• ML-based anomaly detection
• Integration with AWS Security Hub

Security Metrics

Key Metrics

• Mean Time to Remediate (MTTR): Time from vulnerability discovery to fix
• Vulnerability Density: Vulnerabilities per 1000 lines of code
• Security Test Coverage: % of code covered by security tests
• False Positive Rate: % of security alerts that are false positives
• Compliance Score: % of resources meeting compliance requirements

Best Practices

1. Automate Security: Integrate scanning in CI/CD, fail builds on critical issues 2. Least Privilege: Minimum permissions for users, services, containers 3. Defense in Depth: Multiple security layers (network, application, data) 4. Encrypt Everything: Data at rest and in transit 5. Audit Logging: Comprehensive logging for security events 6. Regular Updates: Patch OS, dependencies, containers regularly 7. Security Training: Educate developers on secure coding 8. Incident Response Plan: Documented process for security incidents

Resources

• OWASP Top 10: https://owasp.org/www-project-top-ten
• CIS Benchmarks: https://www.cisecurity.org/cis-benchmarks
• NIST Cybersecurity Framework: https://www.nist.gov/cyberframework
• DevSecOps Manifesto: https://www.devsecops.org
• Cloud Security Alliance: https://cloudsecurityalliance.org

Docker Basics

Core concepts and workflows for Docker containerization.

Core Concepts

Containers: Lightweight, isolated processes bundling apps with dependencies. Ephemeral by default.

Images: Read-only blueprints for containers. Layered filesystem for reusability.

Volumes: Persistent storage surviving container deletion.

Networks: Enable container communication.

Dockerfile Best Practices

Essential Instructions

FROM node:20-alpine              # Base image (use specific versions)
WORKDIR /app                     # Working directory
COPY package*.json ./            # Copy dependency files first
RUN npm install --production     # Execute build commands
COPY . .                         # Copy application code
ENV NODE_ENV=production          # Environment variables
EXPOSE 3000                      # Document exposed ports
USER node                        # Run as non-root (security)
CMD ["node", "server.js"]        # Default command

Multi-Stage Builds (Production)

# Stage 1: Build
FROM node:20-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build

# Stage 2: Production
FROM node:20-alpine AS production
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]

Benefits: Smaller images, improved security, no build tools in production.

.dockerignore

node_modules
.git
.env
*.log
.DS_Store
README.md
docker-compose.yml
dist
coverage

Building Images

# Build with tag
docker build -t myapp:1.0 .

# Build targeting specific stage
docker build -t myapp:dev --target build .

# Build for multiple platforms
docker buildx build --platform linux/amd64,linux/arm64 -t myapp:1.0 .

# View layers
docker image history myapp:1.0

Running Containers

# Basic run
docker run myapp:1.0

# Background (detached)
docker run -d --name myapp myapp:1.0

# Port mapping (host:container)
docker run -p 8080:3000 myapp:1.0

# Environment variables
docker run -e NODE_ENV=production myapp:1.0

# Volume mount (named volume)
docker run -v mydata:/app/data myapp:1.0

# Bind mount (development)
docker run -v $(pwd)/src:/app/src myapp:1.0

# Resource limits
docker run --memory 512m --cpus 0.5 myapp:1.0

# Interactive terminal
docker run -it myapp:1.0 /bin/sh

Container Management

# List containers
docker ps
docker ps -a

# Logs
docker logs myapp
docker logs -f myapp          # Follow
docker logs --tail 100 myapp  # Last 100 lines

# Execute command
docker exec myapp ls /app
docker exec -it myapp /bin/sh  # Interactive shell

# Stop/start
docker stop myapp
docker start myapp

# Remove
docker rm myapp
docker rm -f myapp  # Force remove running

# Inspect
docker inspect myapp

# Monitor resources
docker stats myapp

# Copy files
docker cp myapp:/app/logs ./logs

Volume Management

# Create volume
docker volume create mydata

# List volumes
docker volume ls

# Remove volume
docker volume rm mydata

# Remove unused volumes
docker volume prune

Network Management

# Create network
docker network create my-network

# List networks
docker network ls

# Connect container
docker network connect my-network myapp

# Disconnect
docker network disconnect my-network myapp

Language-Specific Dockerfiles

Node.js

FROM node:20-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

FROM node:20-alpine
WORKDIR /app
COPY --from=build /app/dist ./dist
COPY --from=build /app/node_modules ./node_modules
USER node
CMD ["node", "dist/server.js"]

Python

FROM python:3.11-slim AS build
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

FROM python:3.11-slim
WORKDIR /app
COPY --from=build /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY . .
RUN adduser --disabled-password appuser
USER appuser
CMD ["python", "app.py"]

Go

FROM golang:1.21-alpine AS build
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o main .

FROM scratch
COPY --from=build /app/main /main
CMD ["/main"]

Security Hardening

# Use specific versions
FROM node:20.11.0-alpine3.19

# Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001

# Set ownership
COPY --chown=nodejs:nodejs . .

# Switch to non-root
USER nodejs

Troubleshooting

Container exits immediately

docker logs myapp
docker run -it myapp /bin/sh
docker run -it --entrypoint /bin/sh myapp

Cannot connect

docker ps
docker port myapp
docker network inspect bridge
docker inspect myapp | grep IPAddress

Out of disk space

docker system df
docker system prune -a
docker volume prune

Build cache issues

docker build --no-cache -t myapp .
docker builder prune

Best Practices

• Use specific image versions, not latest
• Run as non-root user
• Multi-stage builds to minimize size
• Implement health checks
• Set resource limits
• Keep images under 500MB
• Scan for vulnerabilities: docker scout cves myapp:1.0

Quick Reference

Resources

• Docs: https://docs.docker.com
• Best Practices: https://docs.docker.com/develop/dev-best-practices/
• Dockerfile Reference: https://docs.docker.com/engine/reference/builder/

Docker Compose

Multi-container application orchestration.

Basic Structure

version: '3.8'

services:
  web:
    build: .
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://user:pass@db:5432/app
    depends_on:
      - db
      - redis
    volumes:
      - ./src:/app/src
    networks:
      - app-network
    restart: unless-stopped

  db:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
      POSTGRES_DB: app
    volumes:
      - postgres_data:/var/lib/postgresql/data
    networks:
      - app-network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine
    networks:
      - app-network
    volumes:
      - redis_data:/data

volumes:
  postgres_data:
  redis_data:

networks:
  app-network:
    driver: bridge

Commands

# Start services
docker compose up
docker compose up -d

# Build images before starting
docker compose up --build

# Scale service
docker compose up -d --scale web=3

# Stop services
docker compose down

# Stop and remove volumes
docker compose down --volumes

# Logs
docker compose logs
docker compose logs -f web

# Execute command
docker compose exec web sh
docker compose exec db psql -U user -d app

# List services
docker compose ps

# Restart service
docker compose restart web

# Pull images
docker compose pull

# Validate
docker compose config

Environment-Specific Configs

compose.yml (base):

services:
  web:
    build: .
    ports:
      - "3000:3000"

compose.override.yml (dev, auto-loaded):

services:
  web:
    volumes:
      - ./src:/app/src  # Live reload
    environment:
      - NODE_ENV=development
      - DEBUG=true
    command: npm run dev

compose.prod.yml (production):

services:
  web:
    image: registry.example.com/myapp:1.0
    restart: always
    environment:
      - NODE_ENV=production
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: '0.5'
          memory: 512M

Usage:

# Development (uses compose.yml + compose.override.yml)
docker compose up

# Production
docker compose -f compose.yml -f compose.prod.yml up -d

Health Checks

services:
  web:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 3s
      start_period: 40s
      retries: 3

Resource Limits

services:
  web:
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
        reservations:
          cpus: '0.25'
          memory: 256M

Logging

services:
  web:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

Environment Variables

Using .env file:

# .env
DATABASE_URL=postgresql://user:pass@db:5432/app
API_KEY=secret

services:
  web:
    env_file:
      - .env

Networking

Services on same network communicate via service name:

services:
  web:
    depends_on:
      - db
    environment:
      # Use service name as hostname
      - DATABASE_URL=postgresql://user:pass@db:5432/app

Volume Backup/Restore

# Backup
docker compose run --rm -v app_data:/data -v $(pwd):/backup \
  alpine tar czf /backup/backup.tar.gz /data

# Restore
docker compose run --rm -v app_data:/data -v $(pwd):/backup \
  alpine tar xzf /backup/backup.tar.gz -C /data

Common Stacks

Web + Database + Cache

services:
  web:
    build: .
    depends_on:
      - db
      - redis
  db:
    image: postgres:15-alpine
  redis:
    image: redis:7-alpine

Microservices

services:
  api-gateway:
    build: ./gateway
  user-service:
    build: ./services/users
  order-service:
    build: ./services/orders
  rabbitmq:
    image: rabbitmq:3-management

Best Practices

• Use named volumes for data persistence
• Implement health checks for all services
• Set restart policies for production
• Use environment-specific compose files
• Configure resource limits
• Enable logging with size limits
• Use depends_on for service ordering
• Network isolation with custom networks

Troubleshooting

# View service logs
docker compose logs -f service-name

# Check service status
docker compose ps

# Restart specific service
docker compose restart service-name

# Rebuild service
docker compose up --build service-name

# Remove everything
docker compose down --volumes --rmi all

Resources

• Docs: https://docs.docker.com/compose/
• Compose Specification: https://docs.docker.com/compose/compose-file/
• Best Practices: https://docs.docker.com/compose/production/

FinOps Basics

Financial Operations (FinOps) - cultural practice of bringing financial accountability to cloud spending through collaboration between engineering, finance, and business teams.

FinOps Principles

Core Values

1. Teams collaborate: Cross-functional accountability for cloud spend 2. Everyone owns usage: Engineers make cost-conscious decisions 3. Centralized team: FinOps team enables and drives best practices 4. Reports accessible: Real-time visibility into cloud costs 5. Decisions driven by business value: Cost vs. performance trade-offs 6. Take advantage of variable cost: Leverage cloud pricing models

FinOps Lifecycle

1. Inform Phase

Goal: Understand current cloud spending

Actions:

• Enable cost allocation tags
• Implement showback/chargeback
• Create cost dashboards
• Analyze spending trends
• Benchmark against industry

Tools:

• AWS Cost Explorer
• Azure Cost Management
• GCP Cost Management
• CloudHealth
• Datadog Cloud Cost Management

2. Optimize Phase

Goal: Reduce cloud spend while maintaining performance

Actions:

• Rightsize resources (downsize over-provisioned)
• Purchase reserved capacity
• Use spot/preemptible instances
• Implement auto-scaling
• Clean up unused resources
• Optimize storage tiers

3. Operate Phase

Goal: Continuously monitor and improve

Actions:

• Set budget alerts
• Track KPIs (cost per customer, cost per transaction)
• Regular cost reviews
• Enforce governance policies
• Celebrate wins, share learnings

Cost Optimization Strategies

1. Compute Optimization

Rightsizing

# AWS: Analyze CloudWatch metrics
aws cloudwatch get-metric-statistics \
  --namespace AWS/EC2 \
  --metric-name CPUUtilization \
  --dimensions Name=InstanceId,Value=i-1234567890 \
  --start-time 2024-01-01T00:00:00Z \
  --end-time 2024-01-31T23:59:59Z \
  --period 3600 \
  --statistics Average

# Recommendation: If avg CPU < 20%, downsize instance type

Reserved Instances (AWS/Azure/GCP)

• Commitment: 1 or 3 years
• Discount: Up to 72% vs on-demand
• Best For: Steady-state, predictable workloads
• Types: Standard (fixed instance), Convertible (changeable)

Savings Plans (AWS)

• Commitment: $X/hour for 1 or 3 years
• Discount: Up to 72%
• Flexibility: Applies across instance family, size, region, OS

Spot/Preemptible Instances

• Discount: Up to 90% vs on-demand
• Risk: Can be interrupted with short notice
• Best For: Fault-tolerant, flexible workloads (batch processing, CI/CD)

2. Storage Optimization

S3 Lifecycle Policies (AWS)

Transition objects to cheaper tiers:
- Day 0-30: S3 Standard ($0.023/GB)
- Day 30-90: S3 Intelligent-Tiering (auto-optimization)
- Day 90+: Glacier Flexible Retrieval ($0.0036/GB)
- Delete after 365 days

Storage Cleanup

# Find old EBS snapshots (AWS)
aws ec2 describe-snapshots --owner-ids self \
  --query 'Snapshots[?StartTime<=`2023-01-01`].[SnapshotId,StartTime,VolumeSize]'

# Delete unattached volumes
aws ec2 describe-volumes --filters Name=status,Values=available

3. Auto-Scaling

Horizontal Pod Autoscaler (Kubernetes)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Time-Based Scaling

# Scale down dev environment at night
# AWS Auto Scaling scheduled action
aws autoscaling put-scheduled-update-group-action \
  --auto-scaling-group-name dev-asg \
  --scheduled-action-name scale-down-evening \
  --recurrence "0 18 * * *" \
  --min-size 0 --max-size 0 --desired-capacity 0

4. Container Cost Optimization

Kubernetes Resource Requests/Limits

# Set appropriate requests (guaranteed) and limits (max)
resources:
  requests:
    memory: "256Mi"
    cpu: "250m"
  limits:
    memory: "512Mi"
    cpu: "500m"

Cluster Autoscaler

• Automatically adjusts node count based on pod resource requests
• Removes underutilized nodes
• AWS: Cluster Autoscaler, Karpenter
• GCP: GKE Autopilot

Kubecost

• Real-time cost visibility for Kubernetes
• Cost allocation by namespace, deployment, pod
• Recommendations for rightsizing and efficiency

Tagging Strategy

Mandatory Tags

Cost Center: finance, engineering, marketing
Environment: dev, staging, prod
Project: project-alpha, project-beta
Owner: team-name or email
Expiration: 2024-12-31 (for temporary resources)

AWS Tag Enforcement (IAM Policy)

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Deny",
    "Action": "ec2:RunInstances",
    "Resource": "arn:aws:ec2:*:*:instance/*",
    "Condition": {
      "StringNotLike": {
        "aws:RequestTag/CostCenter": "*"
      }
    }
  }]
}

Budget and Alerts

AWS Budgets

# Create budget with alert
aws budgets create-budget \
  --account-id 123456789012 \
  --budget file://budget.json \
  --notifications-with-subscribers file://notifications.json

Azure Budget Alert

az consumption budget create \
  --budget-name monthly-budget \
  --amount 10000 \
  --time-grain Monthly \
  --resource-group myResourceGroup

GCP Budget Alert

gcloud billing budgets create \
  --billing-account=BILLING_ACCOUNT_ID \
  --display-name="Monthly Budget" \
  --budget-amount=10000 \
  --threshold-rule=percent=80 \
  --threshold-rule=percent=100

FinOps Metrics (KPIs)

Unit Economics

• Cost per User: Total cloud cost / Active users
• Cost per Transaction: Total cloud cost / Transactions
• Cost per Environment: Dev vs staging vs production spend

Efficiency Metrics

• Coverage: % of compute covered by RI/savings plans
• Utilization: % of purchased RI/savings plans used
• Waste: Idle resources, unattached volumes, old snapshots
• Rightsizing Opportunities: Over-provisioned resources

Financial Metrics

• Month-over-Month Growth: Spending trend
• Budget Variance: Actual vs forecasted spend
• Cost Avoidance: Savings from optimization initiatives

FinOps Tools

Native Cloud Tools

• AWS: Cost Explorer, Budgets, Compute Optimizer, Trusted Advisor
• Azure: Cost Management + Billing, Advisor
• GCP: Cost Management, Recommender

Third-Party Tools

• CloudHealth: Multi-cloud cost management
• Kubecost: Kubernetes cost monitoring
• Infracost: Cost estimation for Terraform
• Spot.io: Automated infrastructure optimization
• Datadog Cloud Cost: Integrated monitoring and cost

Best Practices

1. Tag Everything: Consistent tagging for cost allocation 2. Regular Reviews: Weekly/monthly cost reviews with teams 3. Forecast Proactively: Predict spending based on growth 4. Educate Teams: FinOps training for engineers 5. Automate Cleanup: Scripts to delete orphaned resources 6. Right Commitment Level: Balance flexibility vs savings 7. Measure Unit Costs: Track cost per business metric 8. Celebrate Wins: Recognize teams for cost savings

Resources

• FinOps Foundation: https://www.finops.org
• AWS Cost Optimization: https://aws.amazon.com/pricing/cost-optimization
• Azure FinOps: https://learn.microsoft.com/azure/cost-management-billing
• GCP Cost Optimization: https://cloud.google.com/cost-management
• Kubecost: https://www.kubecost.com

Google Cloud Platform with gcloud CLI

Comprehensive guide for gcloud CLI - command-line interface for Google Cloud Platform.

Installation

Linux

curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-linux-x86_64.tar.gz
tar -xf google-cloud-cli-linux-x86_64.tar.gz
./google-cloud-sdk/install.sh
./google-cloud-sdk/bin/gcloud init

Debian/Ubuntu

echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
sudo apt-get update && sudo apt-get install google-cloud-cli

macOS

curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-darwin-arm.tar.gz
tar -xf google-cloud-cli-darwin-arm.tar.gz
./google-cloud-sdk/install.sh

Authentication

User Account

# Login with browser
gcloud auth login

# Login without browser (remote/headless)
gcloud auth login --no-browser

# List accounts
gcloud auth list

# Switch account
gcloud config set account user@example.com

Service Account

# Activate with key file
gcloud auth activate-service-account SA_EMAIL --key-file=key.json

# Create service account
gcloud iam service-accounts create SA_NAME \
  --display-name="Service Account"

# Create key
gcloud iam service-accounts keys create key.json \
  --iam-account=SA_EMAIL

# Grant role
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:SA_EMAIL" \
  --role="roles/compute.admin"

Service Account Impersonation (Recommended)

# Impersonate for single command
gcloud compute instances list \
  --impersonate-service-account=SA_EMAIL

# Set default impersonation
gcloud config set auth/impersonate_service_account SA_EMAIL

# Clear impersonation
gcloud config unset auth/impersonate_service_account

Why impersonation? Short-lived credentials, no key files, centralized management.

Configuration Management

Named Configurations

# Create configuration
gcloud config configurations create dev

# List configurations
gcloud config configurations list

# Activate configuration
gcloud config configurations activate dev

# Set properties
gcloud config set project my-project-dev
gcloud config set compute/region us-central1
gcloud config set compute/zone us-central1-a

# View properties
gcloud config list

# Delete configuration
gcloud config configurations delete dev

Multi-Environment Pattern

# Development
gcloud config configurations create dev
gcloud config set project my-project-dev
gcloud config set account dev@example.com

# Staging
gcloud config configurations create staging
gcloud config set project my-project-staging
gcloud config set auth/impersonate_service_account staging-sa@project.iam.gserviceaccount.com

# Production
gcloud config configurations create prod
gcloud config set project my-project-prod
gcloud config set auth/impersonate_service_account prod-sa@project.iam.gserviceaccount.com

Project Management

# List projects
gcloud projects list

# Create project
gcloud projects create PROJECT_ID --name="Project Name"

# Set active project
gcloud config set project PROJECT_ID

# Get current project
gcloud config get-value project

# Enable API
gcloud services enable compute.googleapis.com
gcloud services enable container.googleapis.com

# List enabled APIs
gcloud services list

Output Formats

# JSON (recommended for scripting)
gcloud compute instances list --format=json

# YAML
gcloud compute instances list --format=yaml

# CSV
gcloud compute instances list --format="csv(name,zone,status)"

# Value (single field)
gcloud config get-value project --format="value()"

# Custom table
gcloud compute instances list \
  --format="table(name,zone,machineType,status)"

Filtering

# Server-side filtering (efficient)
gcloud compute instances list --filter="zone:us-central1-a"
gcloud compute instances list --filter="status=RUNNING"
gcloud compute instances list --filter="name~^web-.*"

# Multiple conditions
gcloud compute instances list \
  --filter="zone:us-central1 AND status=RUNNING"

# Negation
gcloud compute instances list --filter="NOT status=TERMINATED"

CI/CD Integration

GitHub Actions

name: Deploy to GCP

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - id: auth
        uses: google-github-actions/auth@v1
        with:
          credentials_json: ${{ secrets.GCP_SA_KEY }}

      - name: Set up Cloud SDK
        uses: google-github-actions/setup-gcloud@v1

      - name: Deploy
        run: |
          gcloud run deploy my-service \
            --image=gcr.io/${{ secrets.GCP_PROJECT_ID }}/my-image \
            --region=us-central1

GitLab CI

deploy:
  image: google/cloud-sdk:alpine
  script:
    - echo $GCP_SA_KEY | base64 -d > key.json
    - gcloud auth activate-service-account --key-file=key.json
    - gcloud config set project $GCP_PROJECT_ID
    - gcloud app deploy
  only:
    - main

Best Practices

Security

• Never commit credentials
• Use service account impersonation
• Grant minimal IAM permissions
• Rotate keys regularly

Performance

• Use server-side filtering: --filter
• Limit output: --limit=10
• Project only needed fields: --format="value(name)"
• Batch operations with --async

Maintainability

• Use named configurations for environments
• Document commands
• Use environment variables
• Implement error handling and retries

Troubleshooting

# Check authentication
gcloud auth list

# Re-authenticate
gcloud auth login
gcloud auth application-default login

# Check IAM permissions
gcloud projects get-iam-policy PROJECT_ID \
  --flatten="bindings[].members" \
  --filter="bindings.members:user@example.com"

# View configuration
gcloud config list

# Reset configuration
gcloud config configurations delete default
gcloud init

Quick Reference

Global Flags

Resources

• gcloud Reference: https://cloud.google.com/sdk/gcloud/reference
• Installation: https://cloud.google.com/sdk/docs/install
• Authentication: https://cloud.google.com/docs/authentication
• Cheatsheet: https://cloud.google.com/sdk/docs/cheatsheet

Google Cloud Services

Compute Engine (VMs)

# List instances
gcloud compute instances list

# Create instance
gcloud compute instances create my-instance \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --image-family=debian-11 \
  --image-project=debian-cloud \
  --boot-disk-size=10GB

# SSH into instance
gcloud compute ssh my-instance --zone=us-central1-a

# Copy files
gcloud compute scp local-file.txt my-instance:~/remote-file.txt \
  --zone=us-central1-a

# Stop instance
gcloud compute instances stop my-instance --zone=us-central1-a

# Delete instance
gcloud compute instances delete my-instance --zone=us-central1-a

Google Kubernetes Engine (GKE)

# Create cluster
gcloud container clusters create my-cluster \
  --zone=us-central1-a \
  --num-nodes=3 \
  --machine-type=e2-medium

# Get credentials
gcloud container clusters get-credentials my-cluster --zone=us-central1-a

# List clusters
gcloud container clusters list

# Resize cluster
gcloud container clusters resize my-cluster \
  --num-nodes=5 \
  --zone=us-central1-a

# Delete cluster
gcloud container clusters delete my-cluster --zone=us-central1-a

Cloud Run (Serverless Containers)

# Deploy container
gcloud run deploy my-service \
  --image=gcr.io/PROJECT_ID/my-image:tag \
  --platform=managed \
  --region=us-central1 \
  --allow-unauthenticated

# List services
gcloud run services list

# Describe service
gcloud run services describe my-service --region=us-central1

# Delete service
gcloud run services delete my-service --region=us-central1

App Engine

# Deploy application
gcloud app deploy app.yaml

# View application
gcloud app browse

# View logs
gcloud app logs tail

# List versions
gcloud app versions list

# Delete version
gcloud app versions delete VERSION_ID

# Set traffic split
gcloud app services set-traffic SERVICE \
  --splits v1=0.5,v2=0.5

Cloud Storage

# Create bucket
gsutil mb gs://my-bucket-name

# Upload file
gsutil cp local-file.txt gs://my-bucket-name/

# Download file
gsutil cp gs://my-bucket-name/file.txt ./

# List contents
gsutil ls gs://my-bucket-name/

# Sync directory
gsutil rsync -r ./local-dir gs://my-bucket-name/remote-dir

# Set permissions
gsutil iam ch user:user@example.com:objectViewer gs://my-bucket-name

# Delete bucket
gsutil rm -r gs://my-bucket-name

Cloud SQL

# Create instance
gcloud sql instances create my-instance \
  --database-version=POSTGRES_14 \
  --tier=db-f1-micro \
  --region=us-central1

# Create database
gcloud sql databases create my-database \
  --instance=my-instance

# Create user
gcloud sql users create my-user \
  --instance=my-instance \
  --password=PASSWORD

# Connect
gcloud sql connect my-instance --user=my-user

# Delete instance
gcloud sql instances delete my-instance

Cloud Functions

# Deploy function
gcloud functions deploy my-function \
  --runtime=python39 \
  --trigger-http \
  --allow-unauthenticated \
  --entry-point=main

# List functions
gcloud functions list

# Describe function
gcloud functions describe my-function

# Call function
gcloud functions call my-function

# Delete function
gcloud functions delete my-function

BigQuery

# List datasets
bq ls

# Create dataset
bq mk my_dataset

# Load data
bq load --source_format=CSV my_dataset.my_table \
  gs://my-bucket/data.csv \
  schema.json

# Query
bq query --use_legacy_sql=false \
  'SELECT * FROM `my_dataset.my_table` LIMIT 10'

# Delete dataset
bq rm -r -f my_dataset

Cloud Build

# Submit build
gcloud builds submit --tag=gcr.io/PROJECT_ID/my-image

# List builds
gcloud builds list

# Describe build
gcloud builds describe BUILD_ID

# Cancel build
gcloud builds cancel BUILD_ID

Artifact Registry

# Create repository
gcloud artifacts repositories create my-repo \
  --repository-format=docker \
  --location=us-central1

# Configure Docker
gcloud auth configure-docker us-central1-docker.pkg.dev

# Push image
docker tag my-image us-central1-docker.pkg.dev/PROJECT_ID/my-repo/my-image
docker push us-central1-docker.pkg.dev/PROJECT_ID/my-repo/my-image

# List repositories
gcloud artifacts repositories list

Networking

# Create VPC network
gcloud compute networks create my-network \
  --subnet-mode=auto

# Create firewall rule
gcloud compute firewall-rules create allow-http \
  --network=my-network \
  --allow=tcp:80

# List networks
gcloud compute networks list

# List firewall rules
gcloud compute firewall-rules list

IAM

# List IAM policy
gcloud projects get-iam-policy PROJECT_ID

# Add IAM binding
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="user:user@example.com" \
  --role="roles/viewer"

# Remove IAM binding
gcloud projects remove-iam-policy-binding PROJECT_ID \
  --member="user:user@example.com" \
  --role="roles/viewer"

# List service accounts
gcloud iam service-accounts list

Monitoring & Logging

# View logs
gcloud logging read "resource.type=gce_instance" \
  --limit=10 \
  --format=json

# Create log sink
gcloud logging sinks create my-sink \
  storage.googleapis.com/my-bucket \
  --log-filter="resource.type=gce_instance"

# List metrics
gcloud monitoring metrics-descriptors list

Quick Reference

Resources

• Compute Engine: https://cloud.google.com/compute/docs
• GKE: https://cloud.google.com/kubernetes-engine/docs
• Cloud Run: https://cloud.google.com/run/docs
• App Engine: https://cloud.google.com/appengine/docs
• Cloud Storage: https://cloud.google.com/storage/docs