Writing Infrastructure Code

Infrastructure as Code

Provision and manage cloud infrastructure using code-based automation tools. This skill covers tool selection, state management, module design, and operational patterns across Terraform/OpenTofu, Pulumi, and AWS CDK.

When to Use

Use this skill when:

• Provisioning cloud infrastructure (compute, networking, databases, storage)
• Migrating from manual infrastructure to code-based workflows
• Designing reusable infrastructure modules
• Implementing multi-cloud or hybrid-cloud deployments
• Establishing state management and drift detection patterns
• Integrating infrastructure provisioning into CI/CD pipelines
• Evaluating IaC tools (Terraform vs Pulumi vs CDK)

Common requests:

• "Create a Terraform module for VPC provisioning"
• "Set up remote state with locking for team collaboration"
• "Compare Pulumi vs Terraform for our use case"
• "Design composable infrastructure modules"
• "Implement drift detection for existing infrastructure"

Core Concepts

Infrastructure as Code Fundamentals

Key Principles: 1. Declarative vs Imperative - Describe desired state (Terraform) or program infrastructure (Pulumi) 2. Idempotency - Same input produces same output, safe to re-run 3. Version Control - Infrastructure changes tracked in Git 4. State Management - Track actual infrastructure state 5. Module Composition - Reusable, versioned infrastructure components

Benefits:

• Reproducibility (same code = same infrastructure)
• Auditability (Git history shows all changes)
• Collaboration (code reviews for infrastructure changes)
• Automation (CI/CD deploys infrastructure)
• Disaster recovery (rebuild from code)

Tool Selection Framework

Choose IaC tools based on team composition and cloud strategy:

Terraform/OpenTofu - Declarative, HCL-based

• Multi-cloud and hybrid-cloud deployments
• Operations/SRE teams prefer declarative approach
• Largest provider ecosystem (AWS, GCP, Azure, 3000+ providers)
• Mature module registry and community

Pulumi - Imperative, programming language-based

• Developer-centric teams familiar with TypeScript/Python/Go
• Complex logic requires programming constructs (loops, conditionals, functions)
• Native unit testing using familiar test frameworks
• Strong typing and IDE support

AWS CDK - AWS-native, programming language-based

• AWS-only infrastructure
• Tight integration with AWS services
• L1/L2/L3 construct abstractions
• CloudFormation under the hood

Decision Tree:

Multi-cloud required?
├─ YES → Team composition?
│  ├─ Ops/SRE focused → Terraform/OpenTofu
│  └─ Developer focused → Pulumi
└─ NO → AWS only?
   ├─ YES → Language preference?
   │  ├─ HCL/declarative → Terraform
   │  ├─ TypeScript/Python → AWS CDK
   │  └─ YAML/simple → CloudFormation
   └─ NO → GCP/Azure only?
      └─ Terraform or Pulumi

State Management Architecture

Remote state with locking enables team collaboration:

Backend Selection:

State Isolation Strategies:

1. Directory Separation (recommended for most teams)

• Separate directories per environment (prod/, staging/, dev/)
• Complete state file isolation
• No risk of cross-environment contamination

2. Workspaces

• Single codebase, multiple environments
• Shared state backend, environment namespacing
• Risk: accidental cross-environment operations

3. Layered Architecture

• Separate state files for networking, compute, data layers
• Blast radius reduction
• Cross-layer references via remote state data sources

Critical State Management Rules:

• Always use remote state for team environments
• Enable state file encryption at rest
• Enable versioning on state storage
• Use state locking to prevent concurrent modifications
• Never commit state files to Git
• Mark sensitive outputs as sensitive = true

Module Design Patterns

Composable Module Structure:

modules/
├── vpc/              # Network foundation
├── security-group/   # Reusable security group patterns
├── rds/              # Database with backups, encryption
├── ecs-cluster/      # Container orchestration base
├── ecs-service/      # Individual microservice
└── alb/              # Application load balancer

Module Versioning:

• Pin module versions in production (version = "5.1.0")
• Use semantic versioning for internal modules
• Test module updates in non-prod first
• Maintain CHANGELOG for module releases

Module Design Principles:

• Clear input contract (required vs optional variables)
• Documented outputs (what consumers can reference)
• Sane defaults where possible
• Validation rules for inputs
• Examples directory showing usage

When to Create a Module:

• Resource group is reused 3+ times
• Clear boundaries and responsibilities
• Stable interface contract
• Team has module maintenance capacity

When to Keep Monolithic:

• One-off infrastructure
• Rapid prototyping phase
• High coupling between resources
• Small team, simple infrastructure

Quick Reference

Terraform/OpenTofu Commands

# Initialize providers and backend
terraform init

# Plan changes (preview)
terraform plan

# Apply changes
terraform apply

# Destroy infrastructure
terraform destroy

# Format HCL files
terraform fmt

# Validate syntax
terraform validate

# Show state
terraform state list
terraform state show <resource>

# Import existing resources
terraform import <resource.name> <id>

# Workspace management
terraform workspace list
terraform workspace new staging
terraform workspace select prod

Pulumi Commands

# Initialize new project
pulumi new aws-typescript

# Preview changes
pulumi preview

# Apply changes
pulumi up

# Destroy infrastructure
pulumi destroy

# Show stack outputs
pulumi stack output

# Manage stacks
pulumi stack ls
pulumi stack select prod

# Import existing resources
pulumi import <type> <name> <id>

# Export/import state
pulumi stack export > state.json
pulumi stack import < state.json

AWS CDK Commands

# Initialize new app
cdk init app --language typescript

# Synthesize CloudFormation
cdk synth

# Preview changes
cdk diff

# Deploy stack
cdk deploy

# Destroy stack
cdk destroy

# Bootstrap account/region
cdk bootstrap

# List stacks
cdk list

Common Patterns Checklist

Infrastructure Provisioning:

• [ ] Remote state configured with locking
• [ ] State file encryption enabled
• [ ] Provider versions pinned
• [ ] Module versions pinned (production)
• [ ] Variables have descriptions and types
• [ ] Sensitive outputs marked as sensitive
• [ ] Tagging strategy implemented
• [ ] Cost allocation tags applied

Module Development:

• [ ] Clear README with usage examples
• [ ] Required vs optional variables documented
• [ ] Outputs documented with descriptions
• [ ] Validation rules for critical inputs
• [ ] Examples directory with working code
• [ ] Tests for module behavior (Terratest/CDK assertions)
• [ ] CHANGELOG for version tracking
• [ ] Semantic versioning followed

Operational Readiness:

• [ ] Drift detection scheduled
• [ ] CI/CD pipeline for plan/apply
• [ ] State backup strategy
• [ ] Disaster recovery documented
• [ ] Team access controls configured (IAM/RBAC)
• [ ] Cost estimation integrated (Infracost)
• [ ] Security scanning integrated (Checkov/tfsec)
• [ ] Documentation kept current

Detailed Documentation

For comprehensive patterns and implementation details:

Tool-Specific Patterns:

• references/terraform-patterns.md - Terraform/OpenTofu best practices, HCL patterns
• references/pulumi-patterns.md - Pulumi across TypeScript/Python/Go

Architecture and Design:

• references/state-management.md - Remote state, locking, isolation strategies
• references/module-design.md - Composable modules, versioning, registries

Operations:

• references/drift-detection.md - Detecting and remediating infrastructure drift

Working Examples

Practical implementations demonstrating IaC patterns:

Terraform Examples:

• examples/terraform/vpc-module/ - Multi-AZ VPC with public/private subnets
• examples/terraform/ecs-service/ - ECS service with ALB, autoscaling
• examples/terraform/rds-cluster/ - Aurora cluster with backups, encryption
• examples/terraform/state-backend/ - S3 + DynamoDB backend setup

Pulumi Examples:

• examples/pulumi/typescript/vpc/ - TypeScript VPC component
• examples/pulumi/python/ecs-service/ - Python ECS service
• examples/pulumi/go/rds-cluster/ - Go RDS cluster
• examples/pulumi/testing/ - Unit tests for Pulumi programs

AWS CDK Examples:

• examples/cdk/typescript/vpc-stack/ - VPC using L2 constructs
• examples/cdk/typescript/ecs-fargate/ - Fargate service with ALB
• examples/cdk/typescript/pipeline-stack/ - Self-mutating CDK pipeline
• examples/cdk/testing/ - CDK assertions and snapshot tests

Utility Scripts

Automated validation and operational tools:

• scripts/validate-terraform.sh - Terraform fmt, validate, tflint
• scripts/cost-estimate.sh - Infracost wrapper for cost analysis
• scripts/drift-check.sh - Scheduled drift detection
• scripts/security-scan.sh - Checkov/tfsec security scanning
• scripts/state-backup.sh - State file backup automation
• scripts/module-release.sh - Module versioning and publishing

Integration with Other Skills

Deployment Pipeline:

• building-ci-pipelines - Automate terraform plan/apply in CI/CD
• gitops-workflows - GitOps-based infrastructure deployment

Platform Engineering:

• kubernetes-operations - Provision EKS, GKE, AKS clusters
• platform-engineering - Internal developer platform infrastructure

Security:

• secret-management - Provision Vault, External Secrets Operator
• security-hardening - Implement infrastructure security controls
• compliance-frameworks - Policy-as-code for compliance

Operations:

• observability - Provision monitoring infrastructure (Prometheus, Grafana)
• disaster-recovery - Infrastructure rebuild procedures
• cost-optimization - Implement cost controls via IaC

Data Platform:

• data-architecture - Provision data lakes, warehouses
• streaming-data - Provision Kafka, Kinesis infrastructure

Best Practices

Development Workflow: 1. Write infrastructure code in feature branches 2. Run terraform plan / pulumi preview locally 3. Submit pull request with plan output 4. Code review focuses on security, cost, blast radius 5. CI runs automated tests and security scans 6. Apply only after approval and CI passes 7. Monitor for drift post-deployment

State Management:

• Use remote state from day one (never local state for teams)
• Separate state files per environment
• Enable state locking to prevent concurrent modifications
• Version state storage for rollback capability
• Encrypt state at rest (contains sensitive data)
• Regular state backups to separate location

Module Development:

• Start with monolithic code, extract modules when patterns emerge
• Design for reusability but avoid premature abstraction
• Document all inputs and outputs
• Provide working examples in examples/ directory
• Pin provider versions in modules
• Test modules before publishing
• Use semantic versioning for releases

Security:

• Scan IaC for security issues before apply (Checkov, tfsec)
• Never commit secrets to code (use secret references)
• Mark sensitive outputs as sensitive = true
• Implement least-privilege IAM policies
• Enable resource encryption by default
• Use private module registries for internal modules

Cost Management:

• Estimate costs before applying changes (Infracost)
• Tag all resources for cost allocation
• Review cost impact in pull requests
• Set up cost alerts for drift
• Rightsize resources based on usage

Operational Excellence:

• Schedule regular drift detection
• Document disaster recovery procedures
• Maintain runbooks for common operations
• Monitor state file access logs
• Practice infrastructure rebuilds periodically
• Keep provider versions current with testing

Common Pitfalls

State File Issues:

• Manual state editing - Use terraform state commands, not direct edits
• No state locking - Race conditions corrupt state
• Local state for teams - State divergence across team members
• Large state files - Break into multiple state files by layer

Module Design:

• Over-abstraction - Too generic, hard to understand
• Under-abstraction - Copy-paste code everywhere
• No version pinning - Unexpected breaking changes
• No examples - Users don't know how to consume module

Operations:

• No drift detection - Manual changes go unnoticed
• Direct resource modification - Bypassing IaC creates drift
• No rollback plan - Can't recover from failed apply
• Ignoring plan output - Surprises during apply

Security:

• Secrets in code - Hard-coded credentials
• No security scanning - Vulnerabilities in production
• Overly permissive IAM - Excessive privileges
• No state encryption - Sensitive data exposed

Troubleshooting Guide

State Lock Issues:

terraform force-unlock <lock-id>  # Use only if certain no other process running

Import Existing Resources:

terraform import aws_vpc.main vpc-12345678
pulumi import aws:ec2/vpc:Vpc main vpc-12345678

Drift Detection:

terraform plan -detailed-exitcode  # Exit 2 = drift detected
pulumi preview --diff

For detailed drift remediation, see references/drift-detection.md.

State Recovery:

# Terraform: Restore from S3 versioning
aws s3 cp s3://bucket/backup/terraform.tfstate terraform.tfstate

# Pulumi: Restore from checkpoint
pulumi stack export --version <timestamp> | pulumi stack import

Related Skills

For cloud-specific implementations:

• aws-patterns - AWS-specific resource patterns
• gcp-patterns - GCP-specific resource patterns
• azure-patterns - Azure-specific resource patterns

For infrastructure operations:

• kubernetes-operations - Manage Kubernetes clusters provisioned via IaC
• gitops-workflows - GitOps-based infrastructure deployment
• platform-engineering - Internal developer platforms

For security and compliance:

• security-hardening - Infrastructure security controls
• secret-management - Secret injection and rotation
• compliance-frameworks - Policy-as-code for compliance

For deployment automation:

• building-ci-pipelines - CI/CD for infrastructure code
• deploying-applications - Application deployment to provisioned infrastructure

For cost and observability:

• cost-optimization - FinOps practices for infrastructure
• observability - Monitoring infrastructure health

Drift Detection and Remediation

Detecting and remediating infrastructure drift - when actual cloud resources diverge from infrastructure code.

What is Drift?

Drift occurs when cloud resources are modified outside of infrastructure as code:

• Manual changes via cloud console
• Direct API/CLI modifications
• Third-party tools making changes
• Emergency hotfixes bypassing IaC

Drift Detection Methods

Terraform Drift Detection

# Detect drift with exit codes
terraform plan -detailed-exitcode

# Exit codes:
# 0 - No changes (no drift)
# 1 - Error
# 2 - Changes needed (drift detected)

Pulumi Drift Detection

# Preview changes to detect drift
pulumi preview --diff

# Refresh state
pulumi refresh

Automated Drift Detection

# Schedule drift checks (cron example)
0 */6 * * * cd /path/to/infra && ./drift-check.sh >> drift.log 2>&1

Drift Remediation

Option 1: Apply Code (Recommended)

# Bring infrastructure back to desired state
terraform apply

# Or for Pulumi
pulumi up

Option 2: Update Code to Match Reality

# Update state to match current infrastructure
terraform apply -refresh-only

# Review and accept changes
terraform plan

# Or for Pulumi
pulumi refresh --yes

Option 3: Import Manual Changes

# Import manually created resource
terraform import aws_vpc.main vpc-12345678

Prevention Strategies

• Enable CloudTrail/audit logging
• Use IAM policies to restrict manual changes
• Implement policy-as-code (OPA, Sentinel)
• Regular drift detection schedules
• Team training on IaC workflows

Module Design - Composable, Reusable Infrastructure

Comprehensive guide to designing, versioning, and testing infrastructure modules.

Table of Contents

1. Module Fundamentals 2. Module Structure 3. Input Design 4. Output Design 5. Module Composition 6. Versioning Strategy 7. Module Registries 8. Testing Modules 9. Documentation

---

Module Fundamentals

What is a Module?

A module is a reusable package of infrastructure code that encapsulates related resources with a clear interface.

Benefits:

• ✅ DRY (Don't Repeat Yourself)
• ✅ Consistency across environments
• ✅ Testable in isolation
• ✅ Versioned and governed
• ✅ Composable into larger systems

When to Create a Module

Create a Module When:

• Resource group is reused 3+ times
• Clear input/output boundaries exist
• Complexity benefits from abstraction
• Team has capacity to maintain

Keep Monolithic When:

• One-off infrastructure
• Rapid prototyping phase
• High coupling between resources
• Small team, simple infrastructure

---

Module Structure

Terraform Module Structure

modules/vpc/
├── README.md              # Module documentation
├── main.tf                # Primary resource definitions
├── variables.tf           # Input variable declarations
├── outputs.tf             # Output declarations
├── versions.tf            # Provider version constraints
├── CHANGELOG.md           # Version history
├── examples/
│   ├── basic/
│   │   └── main.tf        # Basic usage example
│   └── complete/
│       └── main.tf        # Advanced usage example
└── tests/
    └── vpc_test.go        # Terratest tests

Pulumi Component Structure

components/vpc/
├── README.md              # Component documentation
├── index.ts               # Main component (TypeScript)
├── package.json           # Dependencies
├── tsconfig.json          # TypeScript config
├── CHANGELOG.md           # Version history
├── examples/
│   ├── basic/
│   │   └── index.ts
│   └── complete/
│       └── index.ts
└── tests/
    └── vpc.test.ts        # Unit tests

---

Input Design

Variable Patterns (Terraform)

Required vs Optional:

# variables.tf

# Required: No default
variable "name" {
  type        = string
  description = "Name prefix for all resources"
}

# Optional: Has default
variable "vpc_cidr" {
  type        = string
  description = "VPC CIDR block"
  default     = "10.0.0.0/16"
}

# Optional: Nullable
variable "custom_tags" {
  type        = map(string)
  description = "Custom tags to apply to all resources"
  default     = {}
  nullable    = false  # Prevent null values
}

Complex Types:

# Simple list
variable "availability_zones" {
  type        = list(string)
  description = "List of availability zones"
}

# List of objects
variable "subnets" {
  type = list(object({
    name = string
    cidr = string
    az   = string
    type = string  # "public" or "private"
  }))
  description = "Subnet configuration"
  default     = []
}

# Map of objects
variable "security_groups" {
  type = map(object({
    description = string
    ingress_rules = list(object({
      from_port   = number
      to_port     = number
      protocol    = string
      cidr_blocks = list(string)
    }))
  }))
  description = "Security group configurations"
  default     = {}
}

Validation:

variable "environment" {
  type        = string
  description = "Environment name (prod, staging, dev)"

  validation {
    condition     = contains(["prod", "staging", "dev"], var.environment)
    error_message = "Environment must be prod, staging, or dev."
  }
}

variable "vpc_cidr" {
  type        = string
  description = "VPC CIDR block"

  validation {
    condition     = can(cidrhost(var.vpc_cidr, 0))
    error_message = "Must be a valid IPv4 CIDR block."
  }
}

variable "instance_count" {
  type        = number
  description = "Number of instances to create"

  validation {
    condition     = var.instance_count >= 1 && var.instance_count <= 10
    error_message = "Instance count must be between 1 and 10."
  }
}

Input Design (Pulumi TypeScript)

// vpc.ts
export interface VpcArgs {
  // Required
  name: string;
  availabilityZones: pulumi.Input<string>[];

  // Optional with defaults
  cidrBlock?: pulumi.Input<string>;
  enableNatGateway?: pulumi.Input<boolean>;
  enableVpnGateway?: pulumi.Input<boolean>;

  // Optional without defaults
  tags?: pulumi.Input<{ [key: string]: pulumi.Input<string> }>;
}

export class Vpc extends pulumi.ComponentResource {
  constructor(name: string, args: VpcArgs, opts?: pulumi.ComponentResourceOptions) {
    super("custom:network:Vpc", name, {}, opts);

    // Provide defaults
    const cidr = args.cidrBlock ?? "10.0.0.0/16";
    const enableNat = args.enableNatGateway ?? false;
    const enableVpn = args.enableVpnGateway ?? false;

    // Validate inputs
    if (pulumi.runtime.isDryRun()) {
      const azCount = pulumi.output(args.availabilityZones).apply(azs => azs.length);
      azCount.apply(count => {
        if (count < 2) {
          throw new Error("At least 2 availability zones required");
        }
      });
    }

    // ...
  }
}

---

Output Design

Output Patterns (Terraform)

# outputs.tf

# Simple output
output "vpc_id" {
  description = "VPC ID"
  value       = aws_vpc.main.id
}

# List output
output "private_subnet_ids" {
  description = "List of private subnet IDs"
  value       = aws_subnet.private[*].id
}

# Map output
output "subnet_cidrs" {
  description = "Map of subnet names to CIDR blocks"
  value = {
    for k, subnet in aws_subnet.private :
    k => subnet.cidr_block
  }
}

# Complex output
output "vpc_info" {
  description = "VPC information"
  value = {
    vpc_id              = aws_vpc.main.id
    vpc_cidr            = aws_vpc.main.cidr_block
    public_subnet_ids   = aws_subnet.public[*].id
    private_subnet_ids  = aws_subnet.private[*].id
    nat_gateway_ids     = aws_nat_gateway.main[*].id
  }
}

# Sensitive output
output "database_password" {
  description = "Database master password"
  value       = random_password.db.result
  sensitive   = true  # Hidden in logs
}

# Conditional output
output "bastion_ip" {
  description = "Bastion host public IP (if enabled)"
  value       = var.enable_bastion ? aws_instance.bastion[0].public_ip : null
}

Output Design (Pulumi)

export class Vpc extends pulumi.ComponentResource {
  // Public outputs
  public readonly vpcId: pulumi.Output<string>;
  public readonly publicSubnetIds: pulumi.Output<string>[];
  public readonly privateSubnetIds: pulumi.Output<string>[];

  constructor(name: string, args: VpcArgs, opts?: pulumi.ComponentResourceOptions) {
    super("custom:network:Vpc", name, {}, opts);

    // Create resources...

    // Assign outputs
    this.vpcId = vpc.id;
    this.publicSubnetIds = publicSubnets.map(s => s.id);
    this.privateSubnetIds = privateSubnets.map(s => s.id);

    // Register outputs for stack exports
    this.registerOutputs({
      vpcId: this.vpcId,
      publicSubnetIds: this.publicSubnetIds,
      privateSubnetIds: this.privateSubnetIds,
    });
  }
}

---

Module Composition

Terraform Module Composition

modules/vpc/main.tf:

# VPC Module (Low-level)
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true
  tags                 = merge(var.tags, { Name = var.name })
}

resource "aws_subnet" "private" {
  count             = length(var.availability_zones)
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 4, count.index)
  availability_zone = var.availability_zones[count.index]
  tags              = merge(var.tags, { Name = "${var.name}-private-${count.index}" })
}

# ... more resources

modules/ecs-cluster/main.tf:

# ECS Cluster Module (Mid-level)
# Depends on VPC module

resource "aws_ecs_cluster" "main" {
  name = var.cluster_name

  setting {
    name  = "containerInsights"
    value = "enabled"
  }
}

resource "aws_ecs_cluster_capacity_providers" "main" {
  cluster_name       = aws_ecs_cluster.main.name
  capacity_providers = ["FARGATE", "FARGATE_SPOT"]

  default_capacity_provider_strategy {
    base              = 1
    weight            = 100
    capacity_provider = "FARGATE"
  }
}

environments/prod/main.tf:

# Application Composition (High-level)
# Composes multiple modules

module "vpc" {
  source = "../../modules/vpc"

  name               = "prod"
  vpc_cidr           = "10.0.0.0/16"
  availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]
  enable_nat_gateway = true

  tags = local.common_tags
}

module "ecs_cluster" {
  source = "../../modules/ecs-cluster"

  cluster_name = "prod-cluster"
  vpc_id       = module.vpc.vpc_id
  subnet_ids   = module.vpc.private_subnet_ids

  tags = local.common_tags
}

module "api_service" {
  source = "../../modules/ecs-service"

  service_name  = "api"
  cluster_id    = module.ecs_cluster.cluster_id
  vpc_id        = module.vpc.vpc_id
  subnet_ids    = module.vpc.private_subnet_ids
  desired_count = 3

  tags = local.common_tags
}

Pulumi Component Composition

// infrastructure/index.ts
import { Vpc } from "./components/vpc";
import { EcsCluster } from "./components/ecs-cluster";
import { EcsService } from "./components/ecs-service";

// Create VPC
const vpc = new Vpc("prod", {
  cidrBlock: "10.0.0.0/16",
  availabilityZones: ["us-east-1a", "us-east-1b", "us-east-1c"],
  enableNatGateway: true,
});

// Create ECS Cluster
const cluster = new EcsCluster("prod-cluster", {
  vpcId: vpc.vpcId,
  subnetIds: vpc.privateSubnetIds,
});

// Create ECS Service
const apiService = new EcsService("api", {
  clusterId: cluster.clusterId,
  vpcId: vpc.vpcId,
  subnetIds: vpc.privateSubnetIds,
  desiredCount: 3,
});

export const vpcId = vpc.vpcId;
export const clusterArn = cluster.clusterArn;
export const serviceArn = apiService.serviceArn;

---

Versioning Strategy

Semantic Versioning

Follow Semantic Versioning:

• Major version (1.0.0 → 2.0.0): Breaking changes
• Minor version (1.0.0 → 1.1.0): New features, backward compatible
• Patch version (1.0.0 → 1.0.1): Bug fixes, backward compatible

Example: Breaking vs Non-Breaking Changes

Breaking Change (Major Version):

# v1.0.0
variable "vpc_cidr" {
  type = string
}

# v2.0.0 (BREAKING: changed variable name)
variable "cidr_block" {  # Renamed variable
  type = string
}

Non-Breaking Change (Minor Version):

# v1.0.0
output "vpc_id" {
  value = aws_vpc.main.id
}

# v1.1.0 (NON-BREAKING: new output)
output "vpc_id" {
  value = aws_vpc.main.id
}

output "vpc_cidr" {  # New output
  value = aws_vpc.main.cidr_block
}

Version Pinning

Terraform - Pin Module Versions:

# ❌ BAD: No version constraint (uses latest)
module "vpc" {
  source = "git::https://github.com/company/terraform-modules.git//vpc"
}

# ⚠️ BETTER: Use version tag
module "vpc" {
  source = "git::https://github.com/company/terraform-modules.git//vpc?ref=v2.3.0"
}

# ✅ BEST: Use Terraform Registry with version constraint
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.1.0"  # Exact version for production

  # Or allow patch updates
  # version = "~> 5.1.0"  # Allows 5.1.x
}

Pulumi - Pin Package Versions:

// package.json
{
  "dependencies": {
    "@pulumi/aws": "6.8.0",  // Exact version
    "@company/pulumi-vpc": "^2.3.0"  // Allows 2.x updates
  }
}

CHANGELOG.md

# Changelog

All notable changes to this module will be documented in this file.

## [2.1.0] - 2025-01-15

### Added
- Support for IPv6 dual-stack VPCs
- New output: `ipv6_cidr_block`

### Changed
- Default NAT gateway count changed from 1 to match AZ count

## [2.0.0] - 2024-12-01

### Breaking Changes
- Renamed variable `vpc_cidr` to `cidr_block` for consistency
- Removed deprecated `enable_s3_endpoint` variable (use `vpc_endpoints` instead)

### Added
- Support for multiple VPC endpoints

## [1.2.1] - 2024-10-20

### Fixed
- Fixed NAT gateway route table associations

---

Module Registries

Terraform Registry (Public)

Publishing to Terraform Registry:

1. GitHub repository structure:

terraform-aws-vpc/
├── main.tf
├── variables.tf
├── outputs.tf
├── README.md
├── LICENSE
├── examples/
└── .github/
    └── workflows/
        └── release.yml

2. Tag release:

git tag v1.0.0
git push origin v1.0.0

3. Registry automatically indexes module.

Using Public Modules:

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.1.0"

  name = "my-vpc"
  cidr = "10.0.0.0/16"
}

Private Module Registry

Terraform Cloud/Enterprise:

module "vpc" {
  source  = "app.terraform.io/company/vpc/aws"
  version = "2.3.0"
}

Git-Based Private Registry:

module "vpc" {
  source = "git::ssh://git@github.com/company/terraform-modules.git//vpc?ref=v2.3.0"
}

Pulumi Packages

Publishing to npm (TypeScript):

npm publish

Using Private Package:

// package.json
{
  "dependencies": {
    "@company/pulumi-vpc": "^2.3.0"
  }
}

import { Vpc } from "@company/pulumi-vpc";

const vpc = new Vpc("main", { ... });

---

Testing Modules

Unit Testing with Terratest (Go)

// tests/vpc_test.go
package test

import (
    "testing"

    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestVpcModule(t *testing.T) {
    t.Parallel()

    terraformOptions := &terraform.Options{
        TerraformDir: "../examples/complete",
        Vars: map[string]interface{}{
            "name":               "test",
            "vpc_cidr":           "10.0.0.0/16",
            "availability_zones": []string{"us-east-1a", "us-east-1b"},
        },
        EnvVars: map[string]string{
            "AWS_DEFAULT_REGION": "us-east-1",
        },
    }

    // Clean up resources after test
    defer terraform.Destroy(t, terraformOptions)

    // Deploy infrastructure
    terraform.InitAndApply(t, terraformOptions)

    // Validate outputs
    vpcId := terraform.Output(t, terraformOptions, "vpc_id")
    assert.NotEmpty(t, vpcId)
    assert.Regexp(t, "^vpc-", vpcId)

    privateSubnetIds := terraform.OutputList(t, terraformOptions, "private_subnet_ids")
    assert.Len(t, privateSubnetIds, 2)
}

func TestVpcValidation(t *testing.T) {
    t.Parallel()

    terraformOptions := &terraform.Options{
        TerraformDir: "../",
        Vars: map[string]interface{}{
            "name":     "test",
            "vpc_cidr": "invalid-cidr",  // Invalid CIDR
        },
    }

    // Expect validation to fail
    _, err := terraform.InitAndApplyE(t, terraformOptions)
    assert.Error(t, err)
}

Unit Testing with Pulumi (TypeScript)

// tests/vpc.test.ts
import * as pulumi from "@pulumi/pulumi";
import { Vpc } from "../index";

pulumi.runtime.setMocks({
  newResource: function(args: pulumi.runtime.MockResourceArgs): {id: string, state: any} {
    switch (args.type) {
      case "aws:ec2/vpc:Vpc":
        return { id: "vpc-12345", state: { ...args.inputs, id: "vpc-12345" } };
      case "aws:ec2/subnet:Subnet":
        return { id: "subnet-12345", state: { ...args.inputs, id: "subnet-12345" } };
      default:
        return { id: args.inputs.name + "_id", state: args.inputs };
    }
  },
  call: function(args: pulumi.runtime.MockCallArgs) {
    return args.inputs;
  },
});

describe("VPC Component", () => {
  let vpc: Vpc;

  before(async () => {
    vpc = new Vpc("test", {
      name: "test-vpc",
      availabilityZones: ["us-east-1a", "us-east-1b"],
    });
  });

  it("must have a VPC ID", (done) => {
    pulumi.all([vpc.vpcId]).apply(([vpcId]) => {
      expect(vpcId).to.contain("vpc-");
      done();
    });
  });

  it("must create subnets in each AZ", (done) => {
    pulumi.all([vpc.privateSubnetIds]).apply(([subnetIds]) => {
      expect(subnetIds).to.have.length(2);
      done();
    });
  });
});

Policy Testing (Sentinel/OPA)

Sentinel (Terraform Cloud):

# sentinel.hcl
policy "enforce-vpc-encryption" {
  enforcement_level = "hard-mandatory"
}

policy "require-tags" {
  enforcement_level = "soft-mandatory"
}

# enforce-vpc-encryption.sentinel
import "tfplan/v2" as tfplan

main = rule {
  all tfplan.resource_changes as _, rc {
    rc.type is "aws_vpc" implies
      rc.change.after.enable_dns_support is true
  }
}

---

Documentation

Module README Template

# VPC Module

Terraform module for creating a production-ready AWS VPC.

## Features

- Multi-AZ deployment
- Public and private subnets
- NAT gateways for private subnet internet access
- Optional VPN gateway
- Flow logs to CloudWatch

## Usage

### Basic Example

module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "5.1.0"

name = "my-vpc" cidr = "10.0.0.0/16"

availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"] }

### Complete Example

module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "5.1.0"

name = "production-vpc" cidr = "10.0.0.0/16" availability_zones = ["us-east-1a", "us-east-1b", "us-east-1c"]

enable_nat_gateway = true enable_vpn_gateway = true enable_flow_logs = true

tags = { Environment = "production" ManagedBy = "terraform" } }

## Requirements

| Name | Version |
|------|---------|
| terraform | >= 1.6 |
| aws | >= 5.0 |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|----------|
| name | Name prefix for resources | `string` | n/a | yes |
| cidr | VPC CIDR block | `string` | `"10.0.0.0/16"` | no |
| availability_zones | List of AZs | `list(string)` | n/a | yes |
| enable_nat_gateway | Enable NAT gateways | `bool` | `false` | no |

## Outputs

| Name | Description |
|------|-------------|
| vpc_id | VPC ID |
| private_subnet_ids | List of private subnet IDs |
| public_subnet_ids | List of public subnet IDs |

## License

MIT

---

Best Practices

Module Design:

• ✅ Single responsibility principle
• ✅ Clear input/output contract
• ✅ Validation for critical inputs
• ✅ Sane defaults where appropriate
• ✅ Comprehensive documentation

Versioning:

• ✅ Semantic versioning (MAJOR.MINOR.PATCH)
• ✅ Pin versions in production
• ✅ Maintain CHANGELOG.md
• ✅ Test before releasing

Testing:

• ✅ Unit tests for module logic
• ✅ Integration tests for deployments
• ✅ Policy tests for compliance
• ✅ Automated testing in CI

Documentation:

• ✅ Clear README with examples
• ✅ Document all inputs and outputs
• ✅ Provide basic and complete examples
• ✅ Explain common use cases

---

See SKILL.md for links to testing infrastructure and CI/CD integration topics.

Pulumi Patterns - Multi-Language Infrastructure as Code

Comprehensive guide to Pulumi across TypeScript, Python, and Go with modern programming patterns.

Table of Contents

1. Core Concepts 2. TypeScript Patterns 3. Python Patterns 4. Go Patterns 5. Component Resources 6. Stack References 7. Configuration Management 8. Secrets Management 9. Testing Patterns 10. Automation API

---

Core Concepts

Pulumi vs Terraform

Project Structure

pulumi-project/
├── Pulumi.yaml           # Project metadata
├── Pulumi.dev.yaml       # Dev stack config
├── Pulumi.staging.yaml   # Staging stack config
├── Pulumi.prod.yaml      # Prod stack config
├── index.ts              # Main program (TypeScript)
├── package.json          # Node dependencies
├── tsconfig.json         # TypeScript config
├── src/
│   ├── vpc.ts            # VPC component
│   ├── ecs.ts            # ECS component
│   └── rds.ts            # RDS component
└── tests/
    └── infrastructure.test.ts

---

TypeScript Patterns

Basic Project Setup

# Initialize new project
pulumi new aws-typescript

# Install dependencies
npm install

# Preview changes
pulumi preview

# Deploy
pulumi up

# Destroy
pulumi destroy

Simple Resources

// index.ts
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create VPC
const vpc = new aws.ec2.Vpc("main-vpc", {
  cidrBlock: "10.0.0.0/16",
  enableDnsHostnames: true,
  enableDnsSupport: true,
  tags: {
    Name: "main-vpc",
    ManagedBy: "pulumi",
  },
});

// Create subnet
const subnet = new aws.ec2.Subnet("private-subnet", {
  vpcId: vpc.id,
  cidrBlock: "10.0.1.0/24",
  availabilityZone: "us-east-1a",
  tags: {
    Name: "private-subnet-1a",
  },
});

// Export outputs
export const vpcId = vpc.id;
export const subnetId = subnet.id;

Component Resources (TypeScript)

// src/vpc.ts
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

export interface VpcArgs {
  cidrBlock?: pulumi.Input<string>;
  availabilityZones: pulumi.Input<string>[];
  enableNatGateway?: boolean;
  tags?: pulumi.Input<{ [key: string]: pulumi.Input<string> }>;
}

export class Vpc extends pulumi.ComponentResource {
  public readonly vpcId: pulumi.Output<string>;
  public readonly publicSubnetIds: pulumi.Output<string>[];
  public readonly privateSubnetIds: pulumi.Output<string>[];
  public readonly natGatewayIds?: pulumi.Output<string>[];

  constructor(name: string, args: VpcArgs, opts?: pulumi.ComponentResourceOptions) {
    super("custom:network:Vpc", name, {}, opts);

    const cidr = args.cidrBlock ?? "10.0.0.0/16";

    // Create VPC
    const vpc = new aws.ec2.Vpc(
      `${name}-vpc`,
      {
        cidrBlock: cidr,
        enableDnsHostnames: true,
        enableDnsSupport: true,
        tags: pulumi.output(args.tags).apply(tags => ({
          ...tags,
          Name: `${name}-vpc`,
        })),
      },
      { parent: this }
    );

    this.vpcId = vpc.id;

    // Create Internet Gateway
    const igw = new aws.ec2.InternetGateway(
      `${name}-igw`,
      {
        vpcId: vpc.id,
        tags: { Name: `${name}-igw` },
      },
      { parent: this }
    );

    // Create public subnets
    this.publicSubnetIds = [];
    const publicSubnets: aws.ec2.Subnet[] = [];

    pulumi.output(args.availabilityZones).apply(azs => {
      azs.forEach((az, i) => {
        const subnet = new aws.ec2.Subnet(
          `${name}-public-${i}`,
          {
            vpcId: vpc.id,
            cidrBlock: `10.0.${i}.0/24`,
            availabilityZone: az,
            mapPublicIpOnLaunch: true,
            tags: {
              Name: `${name}-public-${az}`,
              Type: "public",
            },
          },
          { parent: this }
        );
        publicSubnets.push(subnet);
        this.publicSubnetIds.push(subnet.id);
      });
    });

    // Create private subnets
    this.privateSubnetIds = [];
    const privateSubnets: aws.ec2.Subnet[] = [];

    pulumi.output(args.availabilityZones).apply(azs => {
      azs.forEach((az, i) => {
        const subnet = new aws.ec2.Subnet(
          `${name}-private-${i}`,
          {
            vpcId: vpc.id,
            cidrBlock: `10.0.${i + 10}.0/24`,
            availabilityZone: az,
            tags: {
              Name: `${name}-private-${az}`,
              Type: "private",
            },
          },
          { parent: this }
        );
        privateSubnets.push(subnet);
        this.privateSubnetIds.push(subnet.id);
      });
    });

    // Create NAT Gateways (optional)
    if (args.enableNatGateway) {
      this.natGatewayIds = [];

      pulumi.output(args.availabilityZones).apply(azs => {
        azs.forEach((az, i) => {
          const eip = new aws.ec2.Eip(
            `${name}-nat-eip-${i}`,
            {
              domain: "vpc",
              tags: { Name: `${name}-nat-eip-${az}` },
            },
            { parent: this }
          );

          const natGw = new aws.ec2.NatGateway(
            `${name}-nat-${i}`,
            {
              allocationId: eip.id,
              subnetId: publicSubnets[i].id,
              tags: { Name: `${name}-nat-${az}` },
            },
            { parent: this }
          );

          this.natGatewayIds!.push(natGw.id);
        });
      });
    }

    // Register outputs
    this.registerOutputs({
      vpcId: this.vpcId,
      publicSubnetIds: this.publicSubnetIds,
      privateSubnetIds: this.privateSubnetIds,
      natGatewayIds: this.natGatewayIds,
    });
  }
}

Using Component Resources

// index.ts
import { Vpc } from "./src/vpc";

const vpc = new Vpc("production", {
  cidrBlock: "10.0.0.0/16",
  availabilityZones: ["us-east-1a", "us-east-1b", "us-east-1c"],
  enableNatGateway: true,
  tags: {
    Environment: "production",
    ManagedBy: "pulumi",
  },
});

export const vpcId = vpc.vpcId;
export const publicSubnets = vpc.publicSubnetIds;
export const privateSubnets = vpc.privateSubnetIds;

Advanced TypeScript Patterns

Dynamic Resource Creation:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const config = new pulumi.Config();
const subnetCount = config.getNumber("subnetCount") || 3;

// Create subnets dynamically
const subnets = Array.from({ length: subnetCount }, (_, i) => {
  return new aws.ec2.Subnet(`subnet-${i}`, {
    vpcId: vpc.id,
    cidrBlock: `10.0.${i}.0/24`,
    availabilityZone: `us-east-1${String.fromCharCode(97 + i)}`,
  });
});

export const subnetIds = subnets.map(s => s.id);

Conditional Resources:

const config = new pulumi.Config();
const enableBastion = config.getBoolean("enableBastion") ?? false;

let bastionInstance: aws.ec2.Instance | undefined;

if (enableBastion) {
  bastionInstance = new aws.ec2.Instance("bastion", {
    ami: "ami-12345678",
    instanceType: "t3.micro",
    subnetId: publicSubnets[0],
  });
}

export const bastionIp = bastionInstance?.publicIp;

Output Transformations:

// Transform outputs with apply()
const subnetCidrs = pulumi.all(subnets.map(s => s.cidrBlock)).apply(cidrs => {
  return cidrs.join(", ");
});

// Multiple outputs
const clusterEndpoint = pulumi.all([cluster.endpoint, cluster.port]).apply(
  ([endpoint, port]) => `${endpoint}:${port}`
);

---

Python Patterns

Basic Project Setup

# Initialize new project
pulumi new aws-python

# Activate virtual environment
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Deploy
pulumi up

Simple Resources (Python)

# __main__.py
import pulumi
import pulumi_aws as aws

# Create VPC
vpc = aws.ec2.Vpc(
    "main-vpc",
    cidr_block="10.0.0.0/16",
    enable_dns_hostnames=True,
    enable_dns_support=True,
    tags={
        "Name": "main-vpc",
        "ManagedBy": "pulumi",
    }
)

# Create subnet
subnet = aws.ec2.Subnet(
    "private-subnet",
    vpc_id=vpc.id,
    cidr_block="10.0.1.0/24",
    availability_zone="us-east-1a",
    tags={
        "Name": "private-subnet-1a",
    }
)

# Export outputs
pulumi.export("vpc_id", vpc.id)
pulumi.export("subnet_id", subnet.id)

Component Resources (Python)

# vpc.py
import pulumi
import pulumi_aws as aws
from typing import List, Optional, Sequence

class VpcArgs:
    def __init__(
        self,
        cidr_block: str = "10.0.0.0/16",
        availability_zones: Sequence[str] = None,
        enable_nat_gateway: bool = False,
        tags: Optional[dict] = None,
    ):
        self.cidr_block = cidr_block
        self.availability_zones = availability_zones or ["us-east-1a", "us-east-1b"]
        self.enable_nat_gateway = enable_nat_gateway
        self.tags = tags or {}

class Vpc(pulumi.ComponentResource):
    def __init__(
        self,
        name: str,
        args: VpcArgs,
        opts: Optional[pulumi.ResourceOptions] = None,
    ):
        super().__init__("custom:network:Vpc", name, None, opts)

        # Create VPC
        self.vpc = aws.ec2.Vpc(
            f"{name}-vpc",
            cidr_block=args.cidr_block,
            enable_dns_hostnames=True,
            enable_dns_support=True,
            tags={**args.tags, "Name": f"{name}-vpc"},
            opts=pulumi.ResourceOptions(parent=self),
        )

        # Create Internet Gateway
        self.igw = aws.ec2.InternetGateway(
            f"{name}-igw",
            vpc_id=self.vpc.id,
            tags={"Name": f"{name}-igw"},
            opts=pulumi.ResourceOptions(parent=self),
        )

        # Create public subnets
        self.public_subnets = []
        for i, az in enumerate(args.availability_zones):
            subnet = aws.ec2.Subnet(
                f"{name}-public-{i}",
                vpc_id=self.vpc.id,
                cidr_block=f"10.0.{i}.0/24",
                availability_zone=az,
                map_public_ip_on_launch=True,
                tags={"Name": f"{name}-public-{az}", "Type": "public"},
                opts=pulumi.ResourceOptions(parent=self),
            )
            self.public_subnets.append(subnet)

        # Create private subnets
        self.private_subnets = []
        for i, az in enumerate(args.availability_zones):
            subnet = aws.ec2.Subnet(
                f"{name}-private-{i}",
                vpc_id=self.vpc.id,
                cidr_block=f"10.0.{i + 10}.0/24",
                availability_zone=az,
                tags={"Name": f"{name}-private-{az}", "Type": "private"},
                opts=pulumi.ResourceOptions(parent=self),
            )
            self.private_subnets.append(subnet)

        # Create NAT Gateways (optional)
        if args.enable_nat_gateway:
            self.nat_gateways = []
            for i, az in enumerate(args.availability_zones):
                eip = aws.ec2.Eip(
                    f"{name}-nat-eip-{i}",
                    domain="vpc",
                    tags={"Name": f"{name}-nat-eip-{az}"},
                    opts=pulumi.ResourceOptions(parent=self),
                )

                nat_gw = aws.ec2.NatGateway(
                    f"{name}-nat-{i}",
                    allocation_id=eip.id,
                    subnet_id=self.public_subnets[i].id,
                    tags={"Name": f"{name}-nat-{az}"},
                    opts=pulumi.ResourceOptions(parent=self),
                )
                self.nat_gateways.append(nat_gw)

        # Register outputs
        self.register_outputs({
            "vpc_id": self.vpc.id,
            "public_subnet_ids": [s.id for s in self.public_subnets],
            "private_subnet_ids": [s.id for s in self.private_subnets],
        })

Using Component Resources (Python)

# __main__.py
import pulumi
from vpc import Vpc, VpcArgs

# Create VPC
vpc = Vpc(
    "production",
    VpcArgs(
        cidr_block="10.0.0.0/16",
        availability_zones=["us-east-1a", "us-east-1b", "us-east-1c"],
        enable_nat_gateway=True,
        tags={"Environment": "production", "ManagedBy": "pulumi"},
    ),
)

# Export outputs
pulumi.export("vpc_id", vpc.vpc.id)
pulumi.export("public_subnets", [s.id for s in vpc.public_subnets])
pulumi.export("private_subnets", [s.id for s in vpc.private_subnets])

Advanced Python Patterns

Dynamic Resource Creation:

import pulumi
import pulumi_aws as aws

config = pulumi.Config()
subnet_count = config.get_int("subnet_count") or 3

# Create subnets dynamically
subnets = []
for i in range(subnet_count):
    subnet = aws.ec2.Subnet(
        f"subnet-{i}",
        vpc_id=vpc.id,
        cidr_block=f"10.0.{i}.0/24",
        availability_zone=f"us-east-1{chr(97 + i)}",
    )
    subnets.append(subnet)

pulumi.export("subnet_ids", [s.id for s in subnets])

Output Transformations:

# Transform outputs with apply()
subnet_cidrs = pulumi.Output.all(*[s.cidr_block for s in subnets]).apply(
    lambda cidrs: ", ".join(cidrs)
)

# Conditional logic
def get_instance_type(env):
    return "t3.large" if env == "prod" else "t3.micro"

instance_type = pulumi.Output.from_input(environment).apply(get_instance_type)

---

Go Patterns

Basic Project Setup

# Initialize new project
pulumi new aws-go

# Build
go build -o pulumi-infrastructure

# Deploy
pulumi up

Simple Resources (Go)

// main.go
package main

import (
    "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
        // Create VPC
        vpc, err := ec2.NewVpc(ctx, "main-vpc", &ec2.VpcArgs{
            CidrBlock:          pulumi.String("10.0.0.0/16"),
            EnableDnsHostnames: pulumi.Bool(true),
            EnableDnsSupport:   pulumi.Bool(true),
            Tags: pulumi.StringMap{
                "Name":      pulumi.String("main-vpc"),
                "ManagedBy": pulumi.String("pulumi"),
            },
        })
        if err != nil {
            return err
        }

        // Create subnet
        subnet, err := ec2.NewSubnet(ctx, "private-subnet", &ec2.SubnetArgs{
            VpcId:            vpc.ID(),
            CidrBlock:        pulumi.String("10.0.1.0/24"),
            AvailabilityZone: pulumi.String("us-east-1a"),
            Tags: pulumi.StringMap{
                "Name": pulumi.String("private-subnet-1a"),
            },
        })
        if err != nil {
            return err
        }

        // Export outputs
        ctx.Export("vpcId", vpc.ID())
        ctx.Export("subnetId", subnet.ID())

        return nil
    })
}

Component Resources (Go)

// vpc/vpc.go
package vpc

import (
    "fmt"

    "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

type VpcArgs struct {
    CidrBlock          string
    AvailabilityZones  []string
    EnableNatGateway   bool
    Tags               map[string]string
}

type Vpc struct {
    pulumi.ResourceState

    VpcId            pulumi.StringOutput      `pulumi:"vpcId"`
    PublicSubnetIds  pulumi.StringArrayOutput `pulumi:"publicSubnetIds"`
    PrivateSubnetIds pulumi.StringArrayOutput `pulumi:"privateSubnetIds"`
}

func NewVpc(ctx *pulumi.Context, name string, args *VpcArgs, opts ...pulumi.ResourceOption) (*Vpc, error) {
    vpc := &Vpc{}
    err := ctx.RegisterComponentResource("custom:network:Vpc", name, vpc, opts...)
    if err != nil {
        return nil, err
    }

    // Create VPC
    awsVpc, err := ec2.NewVpc(ctx, fmt.Sprintf("%s-vpc", name), &ec2.VpcArgs{
        CidrBlock:          pulumi.String(args.CidrBlock),
        EnableDnsHostnames: pulumi.Bool(true),
        EnableDnsSupport:   pulumi.Bool(true),
        Tags:               pulumi.ToStringMap(args.Tags),
    }, pulumi.Parent(vpc))
    if err != nil {
        return nil, err
    }

    vpc.VpcId = awsVpc.ID().ToStringOutput()

    // Create Internet Gateway
    igw, err := ec2.NewInternetGateway(ctx, fmt.Sprintf("%s-igw", name), &ec2.InternetGatewayArgs{
        VpcId: awsVpc.ID(),
        Tags: pulumi.StringMap{
            "Name": pulumi.String(fmt.Sprintf("%s-igw", name)),
        },
    }, pulumi.Parent(vpc))
    if err != nil {
        return nil, err
    }

    // Create public subnets
    publicSubnetIds := pulumi.StringArray{}
    for i, az := range args.AvailabilityZones {
        subnet, err := ec2.NewSubnet(ctx, fmt.Sprintf("%s-public-%d", name, i), &ec2.SubnetArgs{
            VpcId:               awsVpc.ID(),
            CidrBlock:           pulumi.String(fmt.Sprintf("10.0.%d.0/24", i)),
            AvailabilityZone:    pulumi.String(az),
            MapPublicIpOnLaunch: pulumi.Bool(true),
            Tags: pulumi.StringMap{
                "Name": pulumi.String(fmt.Sprintf("%s-public-%s", name, az)),
                "Type": pulumi.String("public"),
            },
        }, pulumi.Parent(vpc))
        if err != nil {
            return nil, err
        }
        publicSubnetIds = append(publicSubnetIds, subnet.ID().ToStringOutput())
    }
    vpc.PublicSubnetIds = publicSubnetIds.ToStringArrayOutput()

    // Create private subnets
    privateSubnetIds := pulumi.StringArray{}
    for i, az := range args.AvailabilityZones {
        subnet, err := ec2.NewSubnet(ctx, fmt.Sprintf("%s-private-%d", name, i), &ec2.SubnetArgs{
            VpcId:            awsVpc.ID(),
            CidrBlock:        pulumi.String(fmt.Sprintf("10.0.%d.0/24", i+10)),
            AvailabilityZone: pulumi.String(az),
            Tags: pulumi.StringMap{
                "Name": pulumi.String(fmt.Sprintf("%s-private-%s", name, az)),
                "Type": pulumi.String("private"),
            },
        }, pulumi.Parent(vpc))
        if err != nil {
            return nil, err
        }
        privateSubnetIds = append(privateSubnetIds, subnet.ID().ToStringOutput())
    }
    vpc.PrivateSubnetIds = privateSubnetIds.ToStringArrayOutput()

    return vpc, nil
}

---

Stack References

Cross-Stack References (TypeScript)

// infrastructure/networking/index.ts
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const vpc = new aws.ec2.Vpc("main", { cidrBlock: "10.0.0.0/16" });

export const vpcId = vpc.id;
export const vpcCidr = vpc.cidrBlock;

// infrastructure/compute/index.ts
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Reference networking stack
const networkingStack = new pulumi.StackReference("organization/networking/prod");
const vpcId = networkingStack.getOutput("vpcId");

const instance = new aws.ec2.Instance("app", {
  ami: "ami-12345678",
  instanceType: "t3.micro",
  subnetId: networkingStack.getOutput("privateSubnetIds").apply(ids => ids[0]),
});

---

Configuration Management

Stack Configuration (TypeScript)

# Pulumi.prod.yaml
config:
  aws:region: us-east-1
  infrastructure:vpcCidr: "10.0.0.0/16"
  infrastructure:instanceType: "t3.large"
  infrastructure:enableBastion: true

// index.ts
import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();
const vpcCidr = config.require("vpcCidr");
const instanceType = config.get("instanceType") || "t3.micro";
const enableBastion = config.getBoolean("enableBastion") ?? false;

---

Secrets Management

// Set secret
// pulumi config set --secret dbPassword P@ssw0rd!

const config = new pulumi.Config();
const dbPassword = config.requireSecret("dbPassword");

const db = new aws.rds.Instance("database", {
  password: dbPassword,
  // ...
});

---

Testing Patterns

Unit Tests (TypeScript)

// tests/infrastructure.test.ts
import * as pulumi from "@pulumi/pulumi";

pulumi.runtime.setMocks({
  newResource: function(args: pulumi.runtime.MockResourceArgs): {id: string, state: any} {
    return {
      id: args.inputs.name + "_id",
      state: args.inputs,
    };
  },
  call: function(args: pulumi.runtime.MockCallArgs) {
    return args.inputs;
  },
});

describe("Infrastructure", () => {
  let vpcId: pulumi.Output<string>;

  before(async () => {
    const infra = await import("../index");
    vpcId = infra.vpcId;
  });

  it("VPC must have a valid ID", (done) => {
    pulumi.all([vpcId]).apply(([id]) => {
      expect(id).to.contain("vpc");
      done();
    });
  });
});

---

Automation API

// automation-api-example.ts
import * as pulumi from "@pulumi/pulumi/automation";

async function deployInfrastructure() {
  const stackName = "dev";
  const projectName = "my-infrastructure";

  // Create or select stack
  const stack = await pulumi.LocalWorkspace.createOrSelectStack({
    stackName,
    projectName,
    program: async () => {
      // Inline Pulumi program
      const vpc = new aws.ec2.Vpc("main", { cidrBlock: "10.0.0.0/16" });
      return { vpcId: vpc.id };
    },
  });

  // Set configuration
  await stack.setConfig("aws:region", { value: "us-east-1" });

  // Run pulumi up
  const upResult = await stack.up({ onOutput: console.log });
  console.log(`VPC ID: ${upResult.outputs.vpcId.value}`);
}

deployInfrastructure();

---

Best Practices

TypeScript:

• ✅ Use strict TypeScript (strict: true in tsconfig.json)
• ✅ Leverage IDE IntelliSense
• ✅ Create component resources for reusability
• ✅ Use apply() for output transformations

Python:

• ✅ Use type hints for clarity
• ✅ Follow PEP 8 style guide
• ✅ Use virtual environments
• ✅ Create component resources as classes

Go:

• ✅ Handle all errors explicitly
• ✅ Use Go modules for dependencies
• ✅ Follow Go naming conventions
• ✅ Leverage strong typing

All Languages:

• ✅ Use stack references for cross-stack dependencies
• ✅ Store secrets in Pulumi config (encrypted)
• ✅ Write unit tests for infrastructure
• ✅ Use component resources for composition

---

See SKILL.md for links to other IaC topics including Terraform patterns, state management, and testing strategies.

State Management - Remote State, Locking, and Isolation

Comprehensive guide to infrastructure state management patterns across Terraform and Pulumi.

Table of Contents

1. State Fundamentals 2. Remote State Backends 3. State Locking 4. State Isolation Strategies 5. Sensitive Data in State 6. State Operations 7. State Recovery 8. Migration Patterns

---

State Fundamentals

What is State?

Infrastructure state tracks the mapping between your code and real cloud resources:

Terraform State:

• JSON file mapping resource names to cloud resource IDs
• Stores resource metadata (ARNs, IPs, etc.)
• Tracks dependencies between resources
• Contains sensitive data (passwords, keys)

Pulumi State:

• Checkpoint files tracking resource state
• Supports Pulumi Service or self-managed backends
• Encrypted by default
• Includes secrets management

Why Remote State?

Problems with Local State:

• ❌ Team collaboration impossible (state on one machine)
• ❌ No locking (concurrent runs corrupt state)
• ❌ No encryption at rest
• ❌ No versioning/rollback
• ❌ CI/CD requires shared storage

Benefits of Remote State:

• ✅ Team collaboration (shared state)
• ✅ State locking (prevents corruption)
• ✅ Encryption at rest
• ✅ Versioning and rollback
• ✅ CI/CD integration

---

Remote State Backends

Terraform - S3 Backend with DynamoDB Locking

Bootstrap State Backend (One-Time Setup):

# bootstrap/state-bucket.tf
# Run this FIRST with local state, then migrate to S3

provider "aws" {
  region = "us-east-1"
}

# S3 bucket for state files
resource "aws_s3_bucket" "terraform_state" {
  bucket = "company-terraform-state-12345"  # Must be globally unique

  lifecycle {
    prevent_destroy = true  # Never accidentally destroy state bucket
  }

  tags = {
    Name        = "Terraform State Bucket"
    Environment = "shared"
    Purpose     = "terraform-state"
  }
}

# Enable versioning for rollback capability
resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  versioning_configuration {
    status = "Enabled"
  }
}

# Enable encryption at rest
resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.terraform_state.arn
    }
  }
}

# Block public access
resource "aws_s3_bucket_public_access_block" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

# KMS key for encryption
resource "aws_kms_key" "terraform_state" {
  description             = "KMS key for Terraform state encryption"
  deletion_window_in_days = 10
  enable_key_rotation     = true

  tags = {
    Name = "terraform-state-key"
  }
}

resource "aws_kms_alias" "terraform_state" {
  name          = "alias/terraform-state"
  target_key_id = aws_kms_key.terraform_state.key_id
}

# DynamoDB table for state locking
resource "aws_dynamodb_table" "terraform_locks" {
  name         = "terraform-state-locks"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"

  attribute {
    name = "LockID"
    type = "S"
  }

  lifecycle {
    prevent_destroy = true
  }

  tags = {
    Name        = "Terraform State Lock Table"
    Environment = "shared"
  }
}

# Outputs for backend configuration
output "state_bucket_name" {
  value       = aws_s3_bucket.terraform_state.bucket
  description = "Name of the S3 bucket for Terraform state"
}

output "dynamodb_table_name" {
  value       = aws_dynamodb_table.terraform_locks.name
  description = "Name of the DynamoDB table for state locking"
}

output "kms_key_id" {
  value       = aws_kms_key.terraform_state.id
  description = "KMS key ID for state encryption"
}

Bootstrap Deployment:

# Step 1: Initialize with local state
cd bootstrap/
terraform init

# Step 2: Apply to create state backend
terraform apply

# Step 3: Note outputs for backend configuration
terraform output

# Step 4: Migrate to remote state (optional)
# Add backend.tf (below), then:
terraform init -migrate-state

Using S3 Backend:

# backend.tf
terraform {
  backend "s3" {
    bucket         = "company-terraform-state-12345"
    key            = "prod/vpc/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-east-1:123456789012:key/abc123"
    dynamodb_table = "terraform-state-locks"

    # Optional: Workspace-specific state paths
    # workspace_key_prefix = "workspaces"
  }
}

Partial Backend Configuration (Recommended):

# backend.tf (static - committed to Git)
terraform {
  backend "s3" {}
}

# backend-prod.tfbackend (dynamic - not committed)
bucket         = "company-terraform-state-12345"
key            = "prod/vpc/terraform.tfstate"
region         = "us-east-1"
encrypt        = true
kms_key_id     = "arn:aws:kms:us-east-1:123456789012:key/abc123"
dynamodb_table = "terraform-state-locks"

# Initialize with backend config
terraform init -backend-config=backend-prod.tfbackend

Terraform - GCS Backend (Google Cloud)

terraform {
  backend "gcs" {
    bucket = "company-terraform-state"
    prefix = "prod/vpc"

    # Optional: encryption
    encryption_key = "your-base64-encoded-encryption-key"
  }
}

Terraform - Azure Blob Backend

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "companytfstate"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

Terraform Cloud/Enterprise

terraform {
  cloud {
    organization = "company-name"

    workspaces {
      name = "prod-vpc"
    }
  }
}

Pulumi State Backends

Pulumi Service (Default):

# Automatic - no configuration needed
pulumi login

# Deploy
pulumi up

Self-Managed S3 Backend:

# Login to S3 backend
pulumi login s3://company-pulumi-state

# Set region
pulumi stack init prod --secrets-provider=awskms://alias/pulumi-secrets

# Deploy
pulumi up

Azure Blob Backend:

pulumi login azblob://container-name

Google Cloud Storage Backend:

pulumi login gs://bucket-name

Local Backend (Not Recommended for Teams):

pulumi login --local

---

State Locking

Terraform State Locking

How Locking Works: 1. terraform apply acquires lock via DynamoDB 2. Lock prevents concurrent operations 3. Lock released after apply completes 4. Failed operations auto-release after timeout

Manual Lock Management:

# View lock information
terraform force-unlock <lock-id>

# Only use force-unlock if certain no other process is running
# Better: wait for lock to release naturally or investigate

Lock Timeout:

# Set custom lock timeout (default: 0s = no timeout)
terraform apply -lock-timeout=10m

Pulumi State Locking

Pulumi automatically handles locking:

• Pulumi Service: Built-in locking
• Self-managed backends: File-based locking

# Cancel ongoing operation (releases lock)
pulumi cancel

---

State Isolation Strategies

Strategy 1: Directory Separation (Recommended)

Structure:

infrastructure/
├── environments/
│   ├── prod/
│   │   ├── networking/
│   │   │   ├── main.tf
│   │   │   └── backend.tf  # key: prod/networking/terraform.tfstate
│   │   ├── compute/
│   │   │   ├── main.tf
│   │   │   └── backend.tf  # key: prod/compute/terraform.tfstate
│   │   └── data/
│   │       ├── main.tf
│   │       └── backend.tf  # key: prod/data/terraform.tfstate
│   ├── staging/
│   │   ├── networking/
│   │   │   └── backend.tf  # key: staging/networking/terraform.tfstate
│   │   └── ...
│   └── dev/
│       └── ...
└── modules/
    └── ...

Benefits:

• ✅ Complete state isolation
• ✅ No risk of cross-environment changes
• ✅ Clear separation of concerns
• ✅ Easy to understand

Drawbacks:

• ⚠️ Code duplication across environments
• ⚠️ Must keep environments in sync manually

Mitigate with Modules:

# environments/prod/networking/main.tf
module "vpc" {
  source = "../../../modules/vpc"

  environment = "prod"
  cidr_block  = "10.0.0.0/16"
}

# environments/staging/networking/main.tf
module "vpc" {
  source = "../../../modules/vpc"

  environment = "staging"
  cidr_block  = "10.1.0.0/16"
}

Strategy 2: Workspaces

Structure:

infrastructure/
├── main.tf
├── variables.tf
├── backend.tf  # Single backend, workspace-specific keys
└── terraform.tfvars

Backend Configuration:

terraform {
  backend "s3" {
    bucket         = "company-terraform-state"
    key            = "vpc/terraform.tfstate"  # Workspace name prepended automatically
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"
    workspace_key_prefix = "environments"  # Results in: environments/prod/vpc/terraform.tfstate
  }
}

Usage:

# Create workspaces
terraform workspace new prod
terraform workspace new staging
terraform workspace new dev

# Switch workspace
terraform workspace select prod

# List workspaces
terraform workspace list

# Current workspace
terraform workspace show

Workspace-Aware Code:

locals {
  environment = terraform.workspace

  instance_type = {
    prod    = "t3.large"
    staging = "t3.medium"
    dev     = "t3.micro"
  }[terraform.workspace]

  instance_count = {
    prod    = 3
    staging = 2
    dev     = 1
  }[terraform.workspace]
}

resource "aws_instance" "app" {
  count         = local.instance_count
  instance_type = local.instance_type
  # ...
}

Benefits:

• ✅ Single codebase
• ✅ DRY (Don't Repeat Yourself)
• ✅ Easy to add new environments

Drawbacks:

• ⚠️ Easy to accidentally operate on wrong workspace
• ⚠️ Shared backend = potential for errors
• ⚠️ Workspace name must be passed around

Strategy 3: Layered Architecture

Structure:

infrastructure/
├── 1-networking/
│   └── backend.tf  # key: networking/terraform.tfstate
├── 2-security/
│   └── backend.tf  # key: security/terraform.tfstate
├── 3-data/
│   └── backend.tf  # key: data/terraform.tfstate
└── 4-compute/
    └── backend.tf  # key: compute/terraform.tfstate

Cross-Layer References:

# 4-compute/main.tf
data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "company-terraform-state"
    key    = "networking/terraform.tfstate"
    region = "us-east-1"
  }
}

resource "aws_instance" "app" {
  subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
  # ...
}

Benefits:

• ✅ Blast radius reduction
• ✅ Independent layer updates
• ✅ Clear dependencies

Drawbacks:

• ⚠️ More complex cross-layer references
• ⚠️ Must apply layers in order

---

Sensitive Data in State

Handling Secrets in Terraform

Problem: Terraform state contains sensitive data in plaintext.

Solution 1: Encrypt State at Rest

# Always use KMS encryption for S3 backend
terraform {
  backend "s3" {
    bucket     = "company-terraform-state"
    key        = "prod/db/terraform.tfstate"
    region     = "us-east-1"
    encrypt    = true  # AES-256 encryption
    kms_key_id = "arn:aws:kms:us-east-1:123456789012:key/abc123"  # KMS encryption
  }
}

Solution 2: Mark Outputs as Sensitive

resource "random_password" "db" {
  length  = 32
  special = true
}

resource "aws_db_instance" "main" {
  password = random_password.db.result
  # ...
}

# Prevent password from appearing in logs
output "db_password" {
  value     = random_password.db.result
  sensitive = true  # Hidden in terraform output
}

Solution 3: Use Secret References Instead of Values

# BAD: Password stored in state
resource "aws_db_instance" "main" {
  password = "hardcoded-password"  # ❌ Stored in state
}

# GOOD: Reference secret from secret manager
data "aws_secretsmanager_secret_version" "db_password" {
  secret_id = "prod/db/master-password"
}

resource "aws_db_instance" "main" {
  password = data.aws_secretsmanager_secret_version.db_password.secret_string  # ✅ Not stored
}

Handling Secrets in Pulumi

Pulumi encrypts secrets in state automatically:

const config = new pulumi.Config();
const dbPassword = config.requireSecret("dbPassword");  // Encrypted in state

const db = new aws.rds.Instance("database", {
  password: dbPassword,  // Stored encrypted
});

export const password = pulumi.secret(dbPassword);  // Exported as secret

---

State Operations

Terraform State Commands

# List resources in state
terraform state list

# Show resource details
terraform state show aws_vpc.main

# Move resource (rename)
terraform state mv aws_instance.old aws_instance.new

# Remove resource from state (doesn't destroy)
terraform state rm aws_instance.abandoned

# Replace resource (taint)
terraform apply -replace=aws_instance.app

# Import existing resource
terraform import aws_vpc.main vpc-12345678

# Pull state to local file
terraform state pull > terraform.tfstate.backup

# Push local state to remote
terraform state push terraform.tfstate

Pulumi State Commands

# List resources in stack
pulumi stack --show-urns

# Export state to file
pulumi stack export > state.json

# Import state from file
pulumi stack import < state.json

# Refresh state
pulumi refresh

# Remove resource from state
pulumi state delete 'urn:pulumi:...'

# Import existing resource
pulumi import aws:ec2/vpc:Vpc main vpc-12345678

---

State Recovery

Terraform State Recovery

Scenario 1: Corrupted State

# Restore from S3 versioning
aws s3api list-object-versions \
  --bucket company-terraform-state \
  --prefix prod/vpc/terraform.tfstate

# Download previous version
aws s3api get-object \
  --bucket company-terraform-state \
  --key prod/vpc/terraform.tfstate \
  --version-id <version-id> \
  terraform.tfstate.restored

# Restore
terraform state push terraform.tfstate.restored

Scenario 2: Lost State

# Rebuild state by importing all resources
terraform import aws_vpc.main vpc-12345678
terraform import aws_subnet.private[0] subnet-abc123
# ... import all resources

Pulumi State Recovery

# Restore from checkpoint history
pulumi stack export --version <timestamp> > state.json
pulumi stack import < state.json

---

Migration Patterns

Local to Remote State (Terraform)

# Step 1: Add backend configuration
# (Create backend.tf with S3 configuration)

# Step 2: Initialize with migration
terraform init -migrate-state

# Step 3: Verify
terraform plan  # Should show no changes

Workspace to Directory Migration (Terraform)

# For each workspace:
terraform workspace select prod
terraform state pull > prod-state.tfstate

# Create new directory structure
mkdir -p environments/prod
mv prod-state.tfstate environments/prod/terraform.tfstate

# Update backend config and re-init
cd environments/prod
terraform init
terraform plan  # Verify no changes

Terraform to Pulumi Migration

# Convert HCL to Pulumi
pulumi convert --from terraform --language typescript --out ./pulumi

# Import state
cd pulumi
pulumi stack init prod
pulumi import <resources>

---

Best Practices

State Security:

• ✅ Always use remote state for teams
• ✅ Enable encryption at rest (KMS)
• ✅ Enable versioning for rollback
• ✅ Restrict access with IAM policies
• ✅ Never commit state files to Git
• ✅ Use separate state files per environment

State Organization:

• ✅ Use directory separation for true isolation
• ✅ Layer state files by infrastructure concern
• ✅ Keep state files small and focused
• ✅ Document state structure

State Operations:

• ✅ Always back up state before operations
• ✅ Use state locking to prevent corruption
• ✅ Test state migrations in non-prod first
• ✅ Monitor state file access logs

Secrets Management:

• ✅ Mark sensitive outputs as sensitive
• ✅ Use secret references, not values
• ✅ Encrypt state at rest
• ✅ Rotate secrets regularly
• ✅ Audit secret access

---

See SKILL.md for links to drift detection and testing strategies.

Terraform Patterns and Best Practices

Comprehensive guide to Terraform/OpenTofu patterns, HCL best practices, and production-ready infrastructure code.

Table of Contents

1. Project Structure 2. Variable Management 3. Provider Configuration 4. Resource Patterns 5. Data Sources 6. Outputs 7. Locals 8. Dynamic Blocks 9. Count vs For_Each 10. Conditional Resources 11. Backend Configuration 12. Workspace Patterns 13. Dependency Management 14. Error Handling

---

Project Structure

Recommended Directory Layout

Monolithic Structure (Small Projects):

terraform/
├── main.tf           # Primary resources
├── variables.tf      # Input variables
├── outputs.tf        # Outputs
├── versions.tf       # Terraform and provider versions
├── backend.tf        # Backend configuration
├── terraform.tfvars  # Default values (not committed)
├── prod.tfvars       # Production values
├── staging.tfvars    # Staging values
└── dev.tfvars        # Development values

Modular Structure (Medium/Large Projects):

infrastructure/
├── modules/
│   ├── vpc/
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   └── README.md
│   ├── security-group/
│   ├── rds/
│   └── ecs-service/
├── environments/
│   ├── prod/
│   │   ├── main.tf
│   │   ├── backend.tf
│   │   ├── variables.tf
│   │   └── terraform.tfvars
│   ├── staging/
│   └── dev/
└── global/
    └── state-backend/  # Bootstrap S3 + DynamoDB

Layered Structure (Enterprise):

infrastructure/
├── 1-bootstrap/        # State backend, IAM
├── 2-networking/       # VPCs, subnets, routing
├── 3-security/         # Security groups, NACLs, WAF
├── 4-data/             # Databases, caches, queues
├── 5-compute/          # ECS, Lambda, EC2
└── 6-applications/     # Application-specific resources

---

Variable Management

Variable Definitions

variables.tf:

# Required variable with validation
variable "environment" {
  type        = string
  description = "Environment name (prod, staging, dev)"

  validation {
    condition     = contains(["prod", "staging", "dev"], var.environment)
    error_message = "Environment must be prod, staging, or dev."
  }
}

# Optional with default
variable "vpc_cidr" {
  type        = string
  description = "VPC CIDR block"
  default     = "10.0.0.0/16"
}

# Complex types
variable "subnet_config" {
  type = map(object({
    cidr = string
    az   = string
    type = string
  }))
  description = "Subnet configuration map"
  default = {
    private_a = {
      cidr = "10.0.1.0/24"
      az   = "us-east-1a"
      type = "private"
    }
  }
}

# Sensitive variable
variable "database_password" {
  type        = string
  description = "RDS master password"
  sensitive   = true
}

# List variable
variable "availability_zones" {
  type        = list(string)
  description = "List of availability zones"
  default     = ["us-east-1a", "us-east-1b", "us-east-1c"]
}

Variable Precedence

Order of precedence (highest to lowest): 1. -var or -var-file command line flags 2. *.auto.tfvars files (alphabetical order) 3. terraform.tfvars file 4. Environment variables (TF_VAR_name) 5. Default values in variable definitions

Variable Best Practices

# ✅ Good: Descriptive name, validation, documentation
variable "rds_instance_class" {
  type        = string
  description = "RDS instance class (e.g., db.t3.medium)"

  validation {
    condition     = can(regex("^db\\.", var.rds_instance_class))
    error_message = "Instance class must start with 'db.'"
  }
}

# ❌ Bad: Vague name, no validation, no description
variable "size" {
  type = string
}

# ✅ Good: Complex type with clear structure
variable "tags" {
  type        = map(string)
  description = "Resource tags"
  default = {
    ManagedBy = "terraform"
  }
}

# ✅ Good: Nullable for optional resources
variable "enable_monitoring" {
  type        = bool
  description = "Enable CloudWatch monitoring"
  default     = true
  nullable    = false  # Prevent null values
}

---

Provider Configuration

Provider Versioning

versions.tf:

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"  # Allow minor updates, not major
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.5"
    }
  }
}

provider "aws" {
  region = var.aws_region

  default_tags {
    tags = {
      Environment = var.environment
      ManagedBy   = "terraform"
      Project     = var.project_name
    }
  }
}

# Additional provider for cross-region resources
provider "aws" {
  alias  = "us_west_2"
  region = "us-west-2"
}

Provider Aliases for Multi-Region

# Create resources in multiple regions
resource "aws_s3_bucket" "primary" {
  provider = aws.us_east_1
  bucket   = "primary-bucket"
}

resource "aws_s3_bucket" "replica" {
  provider = aws.us_west_2
  bucket   = "replica-bucket"
}

---

Resource Patterns

Resource Naming Convention

# Pattern: <resource_type>.<logical_name>
# Use descriptive names that indicate purpose

# ✅ Good
resource "aws_vpc" "main" {}
resource "aws_subnet" "private" {}
resource "aws_security_group" "web" {}

# ❌ Bad
resource "aws_vpc" "vpc1" {}
resource "aws_subnet" "subnet" {}
resource "aws_security_group" "sg" {}

Resource Dependencies

# Implicit dependency (recommended)
resource "aws_subnet" "private" {
  vpc_id = aws_vpc.main.id  # Terraform infers dependency
}

# Explicit dependency (use sparingly)
resource "aws_instance" "app" {
  ami           = data.aws_ami.latest.id
  instance_type = "t3.micro"

  depends_on = [
    aws_iam_role_policy_attachment.app_policy
  ]
}

Lifecycle Rules

resource "aws_s3_bucket" "state" {
  bucket = "terraform-state-bucket"

  lifecycle {
    # Prevent accidental deletion
    prevent_destroy = true

    # Create new resource before destroying old
    create_before_destroy = true

    # Ignore changes to specific attributes
    ignore_changes = [
      tags["LastModified"],
    ]
  }
}

Provisioners (Use Sparingly)

resource "aws_instance" "web" {
  ami           = data.aws_ami.latest.id
  instance_type = "t3.micro"

  # Local provisioner (runs on Terraform machine)
  provisioner "local-exec" {
    command = "echo ${self.private_ip} >> private_ips.txt"
  }

  # Remote provisioner (use configuration management instead)
  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y nginx"
    ]

    connection {
      type        = "ssh"
      user        = "ubuntu"
      private_key = file("~/.ssh/id_rsa")
      host        = self.public_ip
    }
  }

  # Destroy-time provisioner
  provisioner "local-exec" {
    when    = destroy
    command = "echo 'Instance destroyed' >> cleanup.log"
  }
}

---

Data Sources

Common Data Source Patterns

# Fetch latest AMI
data "aws_ami" "amazon_linux_2" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

# Fetch availability zones
data "aws_availability_zones" "available" {
  state = "available"
}

# Fetch current caller identity
data "aws_caller_identity" "current" {}

# Fetch existing VPC by tag
data "aws_vpc" "selected" {
  tags = {
    Name = "production-vpc"
  }
}

# Fetch remote state from another workspace
data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "company-terraform-state"
    key    = "networking/terraform.tfstate"
    region = "us-east-1"
  }
}

---

Outputs

Output Best Practices

# Simple output
output "vpc_id" {
  description = "VPC ID"
  value       = aws_vpc.main.id
}

# Sensitive output (hidden in logs)
output "database_password" {
  description = "RDS master password"
  value       = aws_db_instance.main.password
  sensitive   = true
}

# Complex output
output "subnet_info" {
  description = "Subnet IDs and CIDR blocks"
  value = {
    private_subnet_ids   = aws_subnet.private[*].id
    private_subnet_cidrs = aws_subnet.private[*].cidr_block
    public_subnet_ids    = aws_subnet.public[*].id
    public_subnet_cidrs  = aws_subnet.public[*].cidr_block
  }
}

# Conditional output
output "alb_dns" {
  description = "ALB DNS name (if created)"
  value       = try(aws_lb.main[0].dns_name, null)
}

---

Locals

Local Value Patterns

locals {
  # Computed values
  common_tags = {
    Environment = var.environment
    ManagedBy   = "terraform"
    Project     = var.project_name
    CostCenter  = var.cost_center
  }

  # Conditional logic
  instance_type = var.environment == "prod" ? "t3.large" : "t3.micro"

  # List transformations
  availability_zones = slice(data.aws_availability_zones.available.names, 0, 3)

  # Map transformations
  subnet_cidrs = {
    for idx, az in local.availability_zones :
    az => cidrsubnet(var.vpc_cidr, 4, idx)
  }

  # Derived names
  resource_prefix = "${var.project_name}-${var.environment}"
  vpc_name        = "${local.resource_prefix}-vpc"
  cluster_name    = "${local.resource_prefix}-ecs"
}

---

Dynamic Blocks

Dynamic Block Patterns

# Dynamic ingress rules
resource "aws_security_group" "web" {
  name        = "web-sg"
  description = "Web server security group"
  vpc_id      = aws_vpc.main.id

  dynamic "ingress" {
    for_each = var.ingress_rules
    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
      description = ingress.value.description
    }
  }
}

# Variable definition
variable "ingress_rules" {
  type = list(object({
    from_port   = number
    to_port     = number
    protocol    = string
    cidr_blocks = list(string)
    description = string
  }))
  default = [
    {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "HTTP"
    },
    {
      from_port   = 443
      to_port     = 443
      protocol    = "tcp"
      cidr_blocks = ["0.0.0.0/0"]
      description = "HTTPS"
    }
  ]
}

---

Count vs For_Each

When to Use Count

# Count: Simple numeric iteration
resource "aws_subnet" "private" {
  count             = 3
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 4, count.index)
  availability_zone = data.aws_availability_zones.available.names[count.index]

  tags = {
    Name = "private-subnet-${count.index}"
  }
}

# Reference: aws_subnet.private[0].id

When to Use For_Each

# For_each: Key-based iteration (preferred)
variable "subnets" {
  type = map(object({
    cidr = string
    az   = string
  }))
  default = {
    private_a = { cidr = "10.0.1.0/24", az = "us-east-1a" }
    private_b = { cidr = "10.0.2.0/24", az = "us-east-1b" }
    private_c = { cidr = "10.0.3.0/24", az = "us-east-1c" }
  }
}

resource "aws_subnet" "private" {
  for_each          = var.subnets
  vpc_id            = aws_vpc.main.id
  cidr_block        = each.value.cidr
  availability_zone = each.value.az

  tags = {
    Name = "private-subnet-${each.key}"
  }
}

# Reference: aws_subnet.private["private_a"].id

Recommendation: Use for_each over count for most cases - adding/removing items won't affect other resources.

---

Conditional Resources

# Create resource conditionally
resource "aws_instance" "bastion" {
  count = var.enable_bastion ? 1 : 0

  ami           = data.aws_ami.latest.id
  instance_type = "t3.micro"
  subnet_id     = aws_subnet.public[0].id
}

# Reference conditional resource
output "bastion_ip" {
  value = var.enable_bastion ? aws_instance.bastion[0].public_ip : null
}

# Conditional attribute
resource "aws_db_instance" "main" {
  allocated_storage = var.environment == "prod" ? 100 : 20
  instance_class    = var.environment == "prod" ? "db.m5.large" : "db.t3.micro"
}

---

Backend Configuration

S3 Backend with DynamoDB Locking

backend.tf:

terraform {
  backend "s3" {
    bucket         = "company-terraform-state"
    key            = "prod/vpc/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    kms_key_id     = "arn:aws:kms:us-east-1:123456789012:key/abc123"
    dynamodb_table = "terraform-locks"

    # Optional: State file versioning rollback
    # Enable versioning on S3 bucket separately
  }
}

Partial Backend Configuration

backend.tf (static):

terraform {
  backend "s3" {}
}

backend-prod.tfbackend (dynamic):

bucket         = "company-terraform-state"
key            = "prod/vpc/terraform.tfstate"
region         = "us-east-1"
encrypt        = true
dynamodb_table = "terraform-locks"

Usage:

terraform init -backend-config=backend-prod.tfbackend

---

Workspace Patterns

# Create workspace
terraform workspace new staging

# List workspaces
terraform workspace list

# Select workspace
terraform workspace select prod

# Show current workspace
terraform workspace show

Use workspace in code:

resource "aws_instance" "app" {
  instance_type = terraform.workspace == "prod" ? "t3.large" : "t3.micro"

  tags = {
    Name        = "app-${terraform.workspace}"
    Environment = terraform.workspace
  }
}

Warning: Workspaces share the same state backend. Use directory separation for true isolation.

---

Dependency Management

Explicit Dependencies

# Use depends_on when implicit dependencies aren't sufficient
resource "aws_iam_role_policy_attachment" "lambda_policy" {
  role       = aws_iam_role.lambda.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_lambda_function" "app" {
  # Implicit: role = aws_iam_role.lambda.arn creates dependency
  role = aws_iam_role.lambda.arn

  # Explicit: Ensure policy is attached before creating function
  depends_on = [aws_iam_role_policy_attachment.lambda_policy]
}

Module Dependencies

module "vpc" {
  source = "./modules/vpc"
}

module "ecs_cluster" {
  source = "./modules/ecs-cluster"

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnet_ids

  # Implicit dependency via output references
}

---

Error Handling

Validation Functions

variable "cidr_block" {
  type = string

  validation {
    condition     = can(cidrhost(var.cidr_block, 0))
    error_message = "Must be a valid IPv4 CIDR block."
  }
}

variable "environment" {
  type = string

  validation {
    condition     = contains(["prod", "staging", "dev"], var.environment)
    error_message = "Environment must be prod, staging, or dev."
  }
}

Try/Can Functions

# Try with fallback
locals {
  # Returns first successful expression
  vpc_id = try(aws_vpc.main[0].id, data.aws_vpc.existing.id)
}

# Can for validation
variable "json_config" {
  type = string

  validation {
    condition     = can(jsondecode(var.json_config))
    error_message = "Must be valid JSON."
  }
}

# Coalescelist for first non-empty list
locals {
  subnet_ids = coalescelist(
    aws_subnet.private[*].id,
    data.aws_subnet.existing[*].id
  )
}

Preconditions and Postconditions

resource "aws_instance" "web" {
  ami           = data.aws_ami.latest.id
  instance_type = var.instance_type

  lifecycle {
    precondition {
      condition     = data.aws_ami.latest.architecture == "x86_64"
      error_message = "AMI must be x86_64 architecture."
    }

    postcondition {
      condition     = self.public_ip != ""
      error_message = "Instance must have a public IP address."
    }
  }
}

---

Advanced Patterns

Remote State Data Source

# Reference outputs from another Terraform workspace
data "terraform_remote_state" "networking" {
  backend = "s3"
  config = {
    bucket = "company-terraform-state"
    key    = "networking/terraform.tfstate"
    region = "us-east-1"
  }
}

# Use outputs
resource "aws_instance" "app" {
  subnet_id = data.terraform_remote_state.networking.outputs.private_subnet_ids[0]
}

Terraform Data Source (Self-Reference)

data "terraform_remote_state" "self" {
  backend = "local"
  config = {
    path = "${path.module}/terraform.tfstate"
  }
}

Import Existing Resources

# Import existing AWS resource
terraform import aws_vpc.main vpc-12345678

# Import module resource
terraform import 'module.vpc.aws_vpc.main' vpc-12345678

---

Testing Terraform Code

Validation Commands

# Format code
terraform fmt -recursive

# Validate syntax
terraform validate

# Plan with detailed output
terraform plan -out=tfplan

# Show plan in JSON
terraform show -json tfplan | jq

# Lint with tflint
tflint --init
tflint

Terratest Example

See examples/terraform/testing/ for Terratest examples.

---

Performance Optimization

Parallelism

# Increase parallelism (default: 10)
terraform apply -parallelism=20

Target Specific Resources

# Plan/apply specific resource
terraform plan -target=aws_instance.web
terraform apply -target=module.vpc

Refresh vs No-Refresh

# Skip refresh for faster plans
terraform plan -refresh=false

# Refresh state only (no changes)
terraform apply -refresh-only

---

Best Practices Summary

Code Organization:

• ✅ Separate environments by directory
• ✅ Use modules for reusable components
• ✅ Pin provider and module versions
• ✅ Keep state files small and focused

Variable Management:

• ✅ Add validation rules for critical variables
• ✅ Provide descriptions for all variables
• ✅ Use sensitive = true for secrets
• ✅ Document variable precedence

Resource Design:

• ✅ Use for_each over count
• ✅ Add lifecycle rules where appropriate
• ✅ Use descriptive resource names
• ✅ Implement default_tags at provider level

State Management:

• ✅ Use remote state with locking
• ✅ Enable encryption and versioning
• ✅ Separate state by layer/environment
• ✅ Never commit state files to Git

Operations:

• ✅ Run terraform fmt before commits
• ✅ Run terraform validate in CI
• ✅ Review terraform plan before apply
• ✅ Use terraform import for existing resources

---

See SKILL.md for links to other IaC topics including Pulumi patterns, state management, and module design.