Mastering Aws Cli

AWS CLI v2 Quick Reference

A unified tool to manage AWS services from the terminal. This guide focuses on CLI v2 features, practical examples, and advanced patterns for experienced developers.

Quick Start

# Verify installation and version
aws --version

# Interactive configuration
aws configure                    # Access keys + region + output format
aws configure sso               # IAM Identity Center (SSO) - recommended

# Verify identity
aws sts get-caller-identity     # Shows Account, UserId, ARN

# Enable auto-prompt for command discovery
aws dynamodb --cli-auto-prompt

Power User Tips

# See all waiter commands for a service
aws ec2 wait help

# Generate command skeleton (fill in the blanks)
aws lambda create-function --generate-cli-skeleton > create-fn.json

# Create CLI alias for common commands
aws configure set cli_alias.whoami "sts get-caller-identity"
aws whoami  # Now works!

# Disable pager for scripting
export AWS_PAGER=""

See Advanced Patterns for JMESPath mastery and automation tricks.

Global Options

Decision Trees

Compute & Containers

Need compute?
├── Serverless functions ────────────► Lambda (references/lambda.md)
├── Docker containers
│   ├── Managed orchestration ───────► ECS (references/ecs.md)
│   ├── Kubernetes ──────────────────► EKS (references/eks.md)
│   └── Container registry ──────────► ECR (references/ecr.md)
└── Virtual machines ────────────────► EC2 (use aws ec2 commands)

Data & Storage

Need data storage?
├── Object/blob storage ─────────────► S3 (references/s3.md)
├── NoSQL (key-value/document) ──────► DynamoDB (references/dynamodb.md)
├── Relational SQL ──────────────────► Aurora/RDS (references/aurora.md)
├── Data catalog & ETL ──────────────► Glue (references/glue.md)
└── Data warehouse ──────────────────► Redshift (aws redshift commands)

Streaming & Messaging

Need streaming/messaging?
├── Kafka-compatible ────────────────► MSK (references/msk.md)
├── Real-time streams ───────────────► Kinesis (references/kinesis.md)
├── Message queues ──────────────────► SQS (aws sqs commands)
└── Pub/Sub notifications ───────────► SNS (aws sns commands)

Security & Access

Need security/access management?
├── Users, roles, policies ──────────► IAM (references/iam-security.md)
├── Secrets & credentials ───────────► Secrets Manager/SSM (references/private-parameters.md)
├── Private network access ──────────► VPC (references/vpc-networking.md)
└── Secure tunneling ────────────────► SSM/Bastion (references/bastion-tunneling.md)

Reference File Navigation

Environment Variables

Credential Precedence

The CLI resolves credentials in this order (first match wins):

1. Command-line options (--profile, explicit credentials) 2. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) 3. Web identity token (EKS IRSA, OIDC) 4. SSO credentials (IAM Identity Center) 5. Credentials file (~/.aws/credentials) 6. Config file (~/.aws/config with credential_process) 7. Container credentials (ECS task role) 8. Instance metadata (EC2 instance profile, IMDSv2)

Common Patterns

Profile Switching

# Use specific profile for one command
aws s3 ls --profile production

# Set default profile for session
export AWS_PROFILE=production

# List configured profiles
aws configure list-profiles

Output Filtering with JMESPath

# Get specific fields
aws ec2 describe-instances \
    --query 'Reservations[*].Instances[*].[InstanceId,State.Name]' \
    --output table

# Filter running instances
aws ec2 describe-instances \
    --query 'Reservations[*].Instances[?State.Name==`running`].InstanceId' \
    --output text

Wait for Resource State

# Wait for instance to be running
aws ec2 wait instance-running --instance-ids i-1234567890abcdef0

# Wait for Lambda function update
aws lambda wait function-updated --function-name my-function

Best Practices

Common Errors Quick Reference

For detailed troubleshooting, see Setup.

When Not to Use

• AWS SDK code — For boto3, AWS SDK for JavaScript, etc., use programming documentation
• CloudFormation/Terraform — This skill covers CLI commands, not IaC templates
• Console UI steps — CLI-focused; use AWS documentation for console walkthroughs
• Pricing/billing — Use AWS pricing calculator or Cost Explorer documentation

Quick Command Reference

# Identity & Access
aws sts get-caller-identity
# → {"Account": "123456789012", "UserId": "AIDAEXAMPLE", "Arn": "arn:aws:iam::123456789012:user/dev"}

aws sts assume-role --role-arn arn:aws:iam::123456789012:role/Admin --role-session-name mysession
# → {"Credentials": {"AccessKeyId": "ASIA...", "SecretAccessKey": "...", "SessionToken": "..."}}

# S3
aws s3 ls
# → 2024-01-15 bucket-name-1
# → 2024-02-20 bucket-name-2

aws s3 sync ./local s3://bucket/prefix --delete

# Lambda
aws lambda invoke --function-name fn response.json
# → {"StatusCode": 200, "ExecutedVersion": "$LATEST"}

aws lambda update-function-code --function-name fn --zip-file fileb://code.zip
# → {"FunctionName": "fn", "LastModified": "2024-12-28T...", "State": "Active"}

# ECS
aws ecs list-clusters
# → {"clusterArns": ["arn:aws:ecs:us-east-1:123456789012:cluster/prod"]}

aws ecs update-service --cluster prod --service api --force-new-deployment

# EKS
aws eks update-kubeconfig --name my-cluster
# → Added new context arn:aws:eks:us-east-1:123456789012:cluster/my-cluster

aws eks list-clusters
# → {"clusters": ["my-cluster", "dev-cluster"]}

# Secrets
aws secretsmanager get-secret-value --secret-id prod/api/key --query SecretString --output text
# → sk_live_xxxxxxxxxxxxx

aws ssm get-parameter --name /app/prod/db/host --with-decryption --query Parameter.Value --output text
# → db.example.com

# Debugging
aws ssm start-session --target i-0123456789abcdef0
# → Starting session with SessionId: user-0a1b2c3d4e5f67890

Mastering AWS CLI

A comprehensive Claude Code skill for AWS CLI v2 quick-reference, designed for experienced developers.

Overview

This skill provides instant access to AWS CLI commands, patterns, and best practices. It covers compute, storage, networking, security, and CI/CD integration with GitHub Actions.

What's Included

• Compute & Containers: Lambda, ECS, EKS, ECR, EC2
• Storage & Databases: S3, DynamoDB, Aurora/RDS
• Streaming & Messaging: MSK (Kafka), Kinesis, SQS, SNS
• Data & ETL: Glue (Catalog/Crawlers/Jobs)
• Security: IAM, STS, Secrets Manager, SSM Parameter Store
• Networking: VPC, Security Groups, SSM Tunneling
• CI/CD: GitHub Actions, OIDC Federation

Installing with Skilz (Universal Installer)

The recommended way to install this skill across different AI coding agents is using the skilz universal installer.

This skill supports Agent Skill Standard which means it supports 14+ coding agents including Claude Code, OpenAI Codex, Cursor, and Gemini.

Install Skilz

pip install skilz

Quick Install from Git

You can use either -g or --git with HTTPS or SSH URLs:

# HTTPS URL
skilz install -g https://github.com/SpillwaveSolutions/mastering-aws-cli

# SSH URL
skilz install --git git@github.com:SpillwaveSolutions/mastering-aws-cli.git

Claude Code

Install to user home (available in all projects):

skilz install -g https://github.com/SpillwaveSolutions/mastering-aws-cli

Install to current project only:

skilz install -g https://github.com/SpillwaveSolutions/mastering-aws-cli --project

OpenCode

Install for OpenCode:

skilz install -g https://github.com/SpillwaveSolutions/mastering-aws-cli --agent opencode

Project-level install:

skilz install -g https://github.com/SpillwaveSolutions/mastering-aws-cli --project --agent opencode

Gemini CLI

Project-level install for Gemini:

skilz install -g https://github.com/SpillwaveSolutions/mastering-aws-cli --agent gemini

OpenAI Codex

Install for OpenAI Codex:

skilz install -g https://github.com/SpillwaveSolutions/mastering-aws-cli --agent codex

Project-level install:

skilz install -g https://github.com/SpillwaveSolutions/mastering-aws-cli --project --agent codex

Install from SkillzWave Marketplace

# Claude to user home dir ~/.claude/skills
skilz install SpillwaveSolutions_mastering-aws-cli/mastering-aws-cli

# Claude skill in project folder ./claude/skills
skilz install SpillwaveSolutions_mastering-aws-cli/mastering-aws-cli --project

# OpenCode install to user home dir ~/.config/opencode/skills
skilz install SpillwaveSolutions_mastering-aws-cli/mastering-aws-cli --agent opencode

# OpenCode project level
skilz install SpillwaveSolutions_mastering-aws-cli/mastering-aws-cli --agent opencode --project

# OpenAI Codex install to user home dir ~/.codex/skills
skilz install SpillwaveSolutions_mastering-aws-cli/mastering-aws-cli --agent codex

# OpenAI Codex project level ./.codex/skills
skilz install SpillwaveSolutions_mastering-aws-cli/mastering-aws-cli --agent codex --project

# Gemini CLI (project level only)
skilz install SpillwaveSolutions_mastering-aws-cli/mastering-aws-cli --agent gemini

See the Skill Listing for installation instructions for all 14+ supported coding agents.

Other Supported Agents

Skilz supports 14+ coding agents including Windsurf, Qwen Code, Aidr, and more.

For the full list of supported platforms, visit SkillzWave.ai/platforms or see the skilz-cli GitHub repository.

Manual Installation

Copy this skill to your Claude Code skills directory:

# User-level installation
cp -r mastering-aws-cli ~/.claude/skills/

# Or create a symlink
ln -s "$(pwd)" ~/.claude/skills/mastering-aws-cli

Usage

The skill activates automatically when you mention AWS-related topics:

"How do I assume an IAM role?"
"Show me ECS deployment commands"
"Set up GitHub Actions with AWS OIDC"
"Deploy a Lambda function from a zip file"
"Configure S3 lifecycle policies"

Skill Structure

mastering-aws-cli/
├── SKILL.md                    # Main skill definition with decision trees
├── README.md                   # This file
└── references/
    ├── setup.md                # Installation, SSO, profiles
    ├── iam-security.md         # Roles, policies, STS
    ├── lambda.md               # Serverless functions
    ├── ecs.md                  # Container orchestration
    ├── eks.md                  # Kubernetes
    ├── ecr.md                  # Container registry
    ├── s3.md                   # Object storage
    ├── dynamodb.md             # NoSQL database
    ├── aurora.md               # Relational databases
    ├── glue.md                 # ETL and data catalog
    ├── msk.md                  # Managed Kafka
    ├── kinesis.md              # Data streams
    ├── vpc-networking.md       # VPC and networking
    ├── bastion-tunneling.md    # SSM tunneling
    ├── github-cicd.md          # GitHub Actions integration
    └── advanced-patterns.md    # JMESPath, waiters, aliases

Quick Reference

Essential Commands

# Identity & Access
aws sts get-caller-identity              # Verify identity
aws configure sso                        # Set up SSO (recommended)
aws sso login --profile prod             # Refresh SSO session

# S3
aws s3 ls                                # List buckets
aws s3 sync ./local s3://bucket/prefix   # Sync directories

# Lambda
aws lambda invoke --function-name fn response.json
aws lambda update-function-code --function-name fn --zip-file fileb://code.zip

# ECS
aws ecs list-clusters
aws ecs update-service --cluster prod --service api --force-new-deployment

# EKS
aws eks update-kubeconfig --name my-cluster
kubectl get pods

# Secrets
aws secretsmanager get-secret-value --secret-id prod/api/key --query SecretString --output text
aws ssm get-parameter --name /app/db/host --with-decryption

Triggers

The skill responds to these keywords:

Progressive Disclosure Architecture

This skill uses a three-level loading system for efficient context usage:

1. Metadata (~100 words) - Always loaded, triggers skill activation 2. SKILL.md (<5K words) - Quick reference with decision trees 3. References (unlimited) - Detailed docs loaded on-demand

When you ask about a specific topic, Claude loads only the relevant reference file.

Version

• Version: 2.1.0
• Author: Spillwave
• License: MIT

Contributing

Contributions welcome! Please:

1. Fork this repository 2. Add or update reference files in references/ 3. Update SKILL.md navigation if adding new files 4. Submit a pull request

Related Skills

• mastering-gcloud-commands - Google Cloud CLI reference
• mastering-github-cli - GitHub CLI reference

---

<a href="https://skillzwave.ai/">SkillzWave: Largest Agentic Marketplace for AI Agent Skills</a> | <a href="https://spillwave.com/">SpillWave: Leaders in AI Agent Development</a>

Advanced Patterns

JMESPath Querying

JMESPath is a query language for JSON. Master these patterns to filter and transform AWS CLI output efficiently.

Query Patterns by Complexity

# SELECTION: Extract fields from results
--query 'Reservations[*].Instances[*].[InstanceId,State.Name]'           # Array output
--query 'Reservations[*].Instances[*].{ID:InstanceId,State:State.Name}'  # Named objects
--query 'Reservations[0].Instances[0].InstanceId'                        # Single value

# FILTERING: Match conditions
--query 'Reservations[*].Instances[?State.Name==`running`].InstanceId'   # Exact match
--query 'Contents[?Size > `1048576`].Key'                                # Comparison
--query 'Roles[?starts_with(RoleName, `Lambda`)].RoleName'               # String functions
--query 'Instances[?contains(Tags[?Key==`Name`].Value|[0], `prod`)]'     # Nested + contains

# SORTING & AGGREGATION: Transform results
--query 'sort_by(Images, &CreationDate)[-1].ImageId'                     # Sort, get last
--query 'reverse(sort_by(Images, &CreationDate))[0:5]'                   # Newest 5
--query 'length(Reservations[*].Instances[*][])'                         # Count
--query 'sum(Volumes[*].Size)'                                           # Sum values
--query 'max_by(Instances[], &LaunchTime).InstanceId'                    # Max by field

# TRANSFORMATION: Reshape output
--query 'Reservations[*].Instances[*].InstanceId[]'                      # Flatten arrays
--query 'Reservations[*].Instances[*].InstanceId | join(`,`, @)'         # Join to string
--query '{ID:InstanceId,HasIP:PublicIpAddress!=null}'                    # Conditional
--query '{Name:FunctionName,Timeout:Timeout||`3`}'                       # Default values
--query '{ID:InstanceId,Name:Tags[?Key==`Name`].Value|[0]}'              # Extract tag

Practical Examples

# List running instances with names
aws ec2 describe-instances \
    --query 'Reservations[*].Instances[?State.Name==`running`].{ID:InstanceId,Name:Tags[?Key==`Name`].Value|[0]}' \
    --output table

# Chain filters: running t3.micro instances
aws ec2 describe-instances \
    --query 'Reservations[*].Instances[?State.Name==`running`] | [?InstanceType==`t3.micro`] | [].{ID:InstanceId,AZ:Placement.AvailabilityZone}'

Pagination & Limiting

Client-side Pagination

# Control page size (memory management)
aws s3api list-objects-v2 --bucket my-bucket --page-size 100

# Limit total items
aws s3api list-objects-v2 --bucket my-bucket --max-items 50

# Starting token for manual pagination
aws s3api list-objects-v2 --bucket my-bucket \
    --max-items 100 \
    --starting-token eyJNYXJrZXIiOiBudWxsLCAiYm90b190cnVuY2F0ZV9hbW91bnQiOiAxMDB9

# No pager (direct stdout)
aws s3 ls --no-cli-pager

# Disable pager globally
export AWS_PAGER=""

Server-side Pagination

# Use service-specific pagination
aws dynamodb scan \
    --table-name MyTable \
    --limit 25 \
    --exclusive-start-key '{"pk": {"S": "last-key"}}'

# List all with loop
aws s3api list-objects-v2 --bucket my-bucket --output json | \
    jq -r '.Contents[].Key' > all-keys.txt

Skeletons & Input JSON

Generate Skeletons

# Generate input skeleton
aws ec2 run-instances --generate-cli-skeleton > run-instance-input.json

# Generate output skeleton (shows expected response)
aws ec2 run-instances --generate-cli-skeleton output > run-instance-output.json

# Generate for complex commands
aws ecs create-service --generate-cli-skeleton > ecs-service.json
aws lambda create-function --generate-cli-skeleton > lambda-function.json

Use Input JSON

# Execute with input file
aws ec2 run-instances --cli-input-json file://run-instance-input.json

# Combine with overrides
aws lambda create-function \
    --cli-input-json file://lambda-base.json \
    --function-name override-name

# YAML input (requires yq or conversion)
yq -o=json input.yaml | aws ecs create-service --cli-input-json file:///dev/stdin

Waiters

Built-in Waiters

# Wait for instance running
aws ec2 start-instances --instance-ids i-123
aws ec2 wait instance-running --instance-ids i-123
echo "Instance is running"

# Wait for instance stopped
aws ec2 wait instance-stopped --instance-ids i-123

# Wait for instance terminated
aws ec2 wait instance-terminated --instance-ids i-123

# Wait for CloudFormation stack
aws cloudformation wait stack-create-complete --stack-name my-stack
aws cloudformation wait stack-update-complete --stack-name my-stack
aws cloudformation wait stack-delete-complete --stack-name my-stack

# Wait for RDS available
aws rds wait db-instance-available --db-instance-identifier mydb

# Wait for ECS service stable
aws ecs wait services-stable \
    --cluster my-cluster \
    --services my-service

# Wait for Lambda function active
aws lambda wait function-active --function-name my-function

# Wait for S3 bucket exists
aws s3api wait bucket-exists --bucket my-bucket

# Wait for image available
aws ec2 wait image-available --image-ids ami-123

Custom Waiter with Loop

#!/bin/bash
# Custom waiter for any condition

wait_for_condition() {
    local max_attempts=60
    local attempt=0
    local sleep_time=5

    while [ $attempt -lt $max_attempts ]; do
        status=$(aws ecs describe-services \
            --cluster my-cluster \
            --services my-service \
            --query 'services[0].deployments[0].rolloutState' \
            --output text)

        if [ "$status" = "COMPLETED" ]; then
            echo "Deployment completed"
            return 0
        elif [ "$status" = "FAILED" ]; then
            echo "Deployment failed"
            return 1
        fi

        echo "Status: $status (attempt $((attempt+1))/$max_attempts)"
        sleep $sleep_time
        ((attempt++))
    done

    echo "Timeout waiting for deployment"
    return 1
}

wait_for_condition

CLI Aliases

Configure Aliases

Create ~/.aws/cli/alias:

[toplevel]
# Identity shortcuts
whoami = sts get-caller-identity
account = sts get-caller-identity --query Account --output text
region = configure get region

# S3 shortcuts
mkbucket = s3 mb
rmbucket = s3 rb --force

# Quick list commands
instances = ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId,State.Name,InstanceType,Tags[?Key==`Name`].Value|[0]]' --output table
volumes = ec2 describe-volumes --query 'Volumes[*].[VolumeId,State,Size,VolumeType]' --output table
buckets = s3api list-buckets --query 'Buckets[*].[Name,CreationDate]' --output table
functions = lambda list-functions --query 'Functions[*].[FunctionName,Runtime,MemorySize]' --output table

# Running instances
running = ec2 describe-instances --filters Name=instance-state-name,Values=running --query 'Reservations[*].Instances[*].[InstanceId,Tags[?Key==`Name`].Value|[0]]' --output table

# Recent logs
logs = logs tail --follow

[command ec2]
# EC2-specific aliases
top = describe-instances --query 'Reservations[*].Instances[*].[InstanceId,State.Name,InstanceType]' --output table
myami = describe-images --owners self --query 'sort_by(Images,&CreationDate)[-5:].{ID:ImageId,Name:Name,Date:CreationDate}' --output table

[command s3]
# S3-specific aliases
disk = api list-objects-v2 --summarize --human-readable --query '{Objects:length(Contents),Size:Size}'

[command ecs]
# ECS shortcuts
tasks = list-tasks
services = list-services

Usage:

aws whoami
aws running
aws ec2 top
aws s3 disk --bucket my-bucket

Debugging

Dry Run

# Check permissions without executing
aws ec2 run-instances --dry-run \
    --image-id ami-123 \
    --instance-type t3.micro

# Returns error if no permission, success if allowed
aws ec2 terminate-instances --dry-run --instance-ids i-123

Debug Mode

# Full debug output (HTTP requests/responses)
aws s3 ls --debug

# Debug to file
aws s3 ls --debug 2>&1 | tee debug.log

# Show just the HTTP traffic
aws s3 ls --debug 2>&1 | grep -E "(HTTP|Request|Response)"

Verbose Credential Info

# Show credential source
aws sts get-caller-identity --debug 2>&1 | grep -i credential

# Check credential chain
aws configure list

Scripting Patterns

Error Handling

#!/bin/bash
set -euo pipefail

# Capture output and status
if output=$(aws s3 ls s3://my-bucket 2>&1); then
    echo "Success: $output"
else
    echo "Failed: $output"
    exit 1
fi

# Check specific error
create_output=$(aws s3 mb s3://my-bucket 2>&1) || {
    if [[ "$create_output" == *"BucketAlreadyOwnedByYou"* ]]; then
        echo "Bucket already exists, continuing..."
    else
        echo "Error: $create_output"
        exit 1
    fi
}

# Retry pattern
retry() {
    local max_attempts=3
    local attempt=1
    local delay=5

    while [ $attempt -le $max_attempts ]; do
        if "$@"; then
            return 0
        fi
        echo "Attempt $attempt failed, retrying in ${delay}s..."
        sleep $delay
        ((attempt++))
        ((delay*=2))
    done

    echo "All $max_attempts attempts failed"
    return 1
}

retry aws lambda invoke --function-name my-function output.json

Parallel Execution

#!/bin/bash
# Process instances in parallel

instances=$(aws ec2 describe-instances \
    --filters Name=instance-state-name,Values=running \
    --query 'Reservations[*].Instances[*].InstanceId' \
    --output text)

# Using xargs for parallel
echo "$instances" | xargs -n1 -P4 -I{} aws ssm send-command \
    --instance-ids {} \
    --document-name "AWS-RunShellScript" \
    --parameters 'commands=["uptime"]'

# Using GNU parallel
echo "$instances" | parallel -j4 aws ssm send-command \
    --instance-ids {} \
    --document-name "AWS-RunShellScript" \
    --parameters 'commands=["df -h"]'

Batch Operations

#!/bin/bash
# Process in batches (API limits)

batch_size=25
items=($(aws sqs list-queues --query 'QueueUrls[*]' --output text))

for ((i=0; i<${#items[@]}; i+=batch_size)); do
    batch=("${items[@]:i:batch_size}")
    echo "Processing batch: ${batch[*]}"

    for queue in "${batch[@]}"; do
        aws sqs get-queue-attributes \
            --queue-url "$queue" \
            --attribute-names ApproximateNumberOfMessages
    done
done

Output Processing

# CSV output
aws ec2 describe-instances \
    --query 'Reservations[*].Instances[*].[InstanceId,InstanceType,State.Name]' \
    --output text | tr '\t' ','

# JSON to CSV with jq
aws ec2 describe-instances | \
    jq -r '.Reservations[].Instances[] | [.InstanceId, .InstanceType, .State.Name] | @csv'

# Process JSON output
aws lambda list-functions | jq -r '.Functions[] |
    select(.Runtime | startswith("python")) |
    "\(.FunctionName): \(.Runtime) - \(.MemorySize)MB"'

# Create report
aws ec2 describe-instances --output json | jq -r '
    ["Instance ID","Name","Type","State"],
    (.Reservations[].Instances[] | [
        .InstanceId,
        (.Tags // [] | map(select(.Key == "Name")) | .[0].Value // "N/A"),
        .InstanceType,
        .State.Name
    ]) | @tsv
' | column -t -s $'\t'

Configuration Management

#!/bin/bash
# Environment-specific config

ENV=${1:-dev}
CONFIG_FILE="config-${ENV}.json"

if [[ ! -f "$CONFIG_FILE" ]]; then
    echo "Config file not found: $CONFIG_FILE"
    exit 1
fi

# Read config
BUCKET=$(jq -r '.bucket' "$CONFIG_FILE")
REGION=$(jq -r '.region' "$CONFIG_FILE")
PROFILE=$(jq -r '.profile // "default"' "$CONFIG_FILE")

# Use in commands
aws s3 sync ./dist "s3://${BUCKET}/" \
    --region "$REGION" \
    --profile "$PROFILE"

Useful Queries Collection

# Find untagged resources
aws ec2 describe-instances \
    --query 'Reservations[*].Instances[?!not_null(Tags)].InstanceId'

# Get total EBS storage
aws ec2 describe-volumes \
    --query 'sum(Volumes[*].Size)' \
    --output text

# Find public S3 buckets
aws s3api list-buckets --query 'Buckets[*].Name' --output text | \
    xargs -n1 -P4 -I{} sh -c \
    'aws s3api get-bucket-acl --bucket {} 2>/dev/null | grep -q AllUsers && echo {}'

# Lambda functions by runtime
aws lambda list-functions \
    --query 'Functions | group_by(@, &Runtime) | [*].{Runtime: [0].Runtime, Count: length(@)}'

# Cost estimation (running instances)
aws ec2 describe-instances \
    --filters Name=instance-state-name,Values=running \
    --query 'Reservations[*].Instances[*].InstanceType' \
    --output text | sort | uniq -c | sort -rn

Best Practices

Aurora & RDS

Aurora Serverless v2

Create Cluster

# Aurora PostgreSQL Serverless v2
aws rds create-db-cluster \
    --db-cluster-identifier my-cluster \
    --engine aurora-postgresql \
    --engine-version 15.4 \
    --master-username admin \
    --master-user-password SecurePassword123! \
    --serverless-v2-scaling-configuration MinCapacity=0.5,MaxCapacity=16 \
    --db-subnet-group-name my-subnet-group \
    --vpc-security-group-ids sg-12345678 \
    --storage-encrypted \
    --enable-cloudwatch-logs-exports postgresql

# Aurora MySQL Serverless v2
aws rds create-db-cluster \
    --db-cluster-identifier mysql-cluster \
    --engine aurora-mysql \
    --engine-version 3.04.0 \
    --master-username admin \
    --master-user-password SecurePassword123! \
    --serverless-v2-scaling-configuration MinCapacity=0.5,MaxCapacity=32 \
    --db-subnet-group-name my-subnet-group \
    --vpc-security-group-ids sg-12345678

Create Instance

# Primary writer instance
aws rds create-db-instance \
    --db-instance-identifier my-primary \
    --db-cluster-identifier my-cluster \
    --engine aurora-postgresql \
    --db-instance-class db.serverless

# Add reader instance
aws rds create-db-instance \
    --db-instance-identifier my-reader-1 \
    --db-cluster-identifier my-cluster \
    --engine aurora-postgresql \
    --db-instance-class db.serverless

# Wait for instance
aws rds wait db-instance-available --db-instance-identifier my-primary

Aurora Provisioned

Create Cluster

# Aurora PostgreSQL with provisioned capacity
aws rds create-db-cluster \
    --db-cluster-identifier prod-cluster \
    --engine aurora-postgresql \
    --engine-version 15.4 \
    --master-username admin \
    --master-user-password SecurePassword123! \
    --db-subnet-group-name my-subnet-group \
    --vpc-security-group-ids sg-12345678 \
    --storage-encrypted \
    --kms-key-id alias/aws/rds \
    --backup-retention-period 7 \
    --preferred-backup-window "03:00-04:00" \
    --preferred-maintenance-window "sun:05:00-sun:06:00"

# Create writer instance
aws rds create-db-instance \
    --db-instance-identifier prod-primary \
    --db-cluster-identifier prod-cluster \
    --engine aurora-postgresql \
    --db-instance-class db.r6g.large \
    --publicly-accessible false

# Create reader instance
aws rds create-db-instance \
    --db-instance-identifier prod-reader-1 \
    --db-cluster-identifier prod-cluster \
    --engine aurora-postgresql \
    --db-instance-class db.r6g.large \
    --publicly-accessible false

Standard RDS

Create Instance

# RDS PostgreSQL
aws rds create-db-instance \
    --db-instance-identifier prod-postgres \
    --db-instance-class db.t3.medium \
    --engine postgres \
    --engine-version 15.4 \
    --allocated-storage 100 \
    --max-allocated-storage 500 \
    --master-username admin \
    --master-user-password SecurePassword123! \
    --db-subnet-group-name my-subnet-group \
    --vpc-security-group-ids sg-12345678 \
    --storage-encrypted \
    --multi-az \
    --publicly-accessible false \
    --backup-retention-period 7

# RDS MySQL
aws rds create-db-instance \
    --db-instance-identifier prod-mysql \
    --db-instance-class db.r6g.large \
    --engine mysql \
    --engine-version 8.0.35 \
    --allocated-storage 100 \
    --max-allocated-storage 1000 \
    --master-username admin \
    --master-user-password SecurePassword123! \
    --storage-type gp3 \
    --iops 3000 \
    --storage-throughput 125

Cluster and Instance Management

Describe and List

# List clusters
aws rds describe-db-clusters

# Describe specific cluster
aws rds describe-db-clusters --db-cluster-identifier my-cluster

# Get cluster endpoint
aws rds describe-db-clusters \
    --db-cluster-identifier my-cluster \
    --query 'DBClusters[0].Endpoint' \
    --output text

# Get reader endpoint
aws rds describe-db-clusters \
    --db-cluster-identifier my-cluster \
    --query 'DBClusters[0].ReaderEndpoint' \
    --output text

# List instances
aws rds describe-db-instances

# Describe specific instance
aws rds describe-db-instances --db-instance-identifier my-primary

Modify Cluster

# Modify scaling configuration
aws rds modify-db-cluster \
    --db-cluster-identifier my-cluster \
    --serverless-v2-scaling-configuration MinCapacity=1,MaxCapacity=32 \
    --apply-immediately

# Enable deletion protection
aws rds modify-db-cluster \
    --db-cluster-identifier my-cluster \
    --deletion-protection

# Change maintenance window
aws rds modify-db-cluster \
    --db-cluster-identifier my-cluster \
    --preferred-maintenance-window "sun:05:00-sun:06:00"

Modify Instance

# Change instance class
aws rds modify-db-instance \
    --db-instance-identifier my-primary \
    --db-instance-class db.r6g.xlarge \
    --apply-immediately

# Scale storage (RDS only)
aws rds modify-db-instance \
    --db-instance-identifier prod-postgres \
    --allocated-storage 200 \
    --apply-immediately

Delete

# Delete instance (skip final snapshot)
aws rds delete-db-instance \
    --db-instance-identifier my-reader-1 \
    --skip-final-snapshot

# Delete cluster (with final snapshot)
aws rds delete-db-cluster \
    --db-cluster-identifier my-cluster \
    --final-db-snapshot-identifier my-cluster-final-snapshot

# Delete cluster (skip final snapshot)
aws rds delete-db-cluster \
    --db-cluster-identifier my-cluster \
    --skip-final-snapshot

Aurora Global Database

Create Global Database

# Create global database from existing cluster
aws rds create-global-cluster \
    --global-cluster-identifier my-global-db \
    --source-db-cluster-identifier arn:aws:rds:us-east-1:123456789012:cluster:my-cluster

# Add secondary region
aws rds create-db-cluster \
    --db-cluster-identifier my-cluster-eu \
    --engine aurora-postgresql \
    --engine-version 15.4 \
    --global-cluster-identifier my-global-db \
    --db-subnet-group-name eu-subnet-group \
    --vpc-security-group-ids sg-eu-12345 \
    --region eu-west-1

# Add instance in secondary region
aws rds create-db-instance \
    --db-instance-identifier my-cluster-eu-1 \
    --db-cluster-identifier my-cluster-eu \
    --engine aurora-postgresql \
    --db-instance-class db.r6g.large \
    --region eu-west-1

Manage Global Database

# Describe global database
aws rds describe-global-clusters --global-cluster-identifier my-global-db

# Failover to secondary (planned)
aws rds failover-global-cluster \
    --global-cluster-identifier my-global-db \
    --target-db-cluster-identifier arn:aws:rds:eu-west-1:123456789012:cluster:my-cluster-eu

# Remove cluster from global database
aws rds remove-from-global-cluster \
    --global-cluster-identifier my-global-db \
    --db-cluster-identifier arn:aws:rds:eu-west-1:123456789012:cluster:my-cluster-eu

# Delete global database
aws rds delete-global-cluster --global-cluster-identifier my-global-db

RDS Proxy

Create Proxy

# Create RDS Proxy
aws rds create-db-proxy \
    --db-proxy-name my-proxy \
    --engine-family POSTGRESQL \
    --auth '[{
        "AuthScheme": "SECRETS",
        "SecretArn": "arn:aws:secretsmanager:us-east-1:123456789012:secret:db-creds",
        "IAMAuth": "DISABLED"
    }]' \
    --role-arn arn:aws:iam::123456789012:role/RDSProxyRole \
    --vpc-subnet-ids subnet-1 subnet-2 \
    --vpc-security-group-ids sg-12345678 \
    --require-tls

# Register target (Aurora cluster)
aws rds register-db-proxy-targets \
    --db-proxy-name my-proxy \
    --db-cluster-identifiers my-cluster

# Register target (RDS instance)
aws rds register-db-proxy-targets \
    --db-proxy-name my-proxy \
    --db-instance-identifiers prod-postgres

Manage Proxy

# Describe proxy
aws rds describe-db-proxies --db-proxy-name my-proxy

# Get proxy endpoint
aws rds describe-db-proxies \
    --db-proxy-name my-proxy \
    --query 'DBProxies[0].Endpoint' \
    --output text

# Describe target groups
aws rds describe-db-proxy-target-groups --db-proxy-name my-proxy

# Delete proxy
aws rds delete-db-proxy --db-proxy-name my-proxy

Snapshots and Cloning

Manual Snapshots

# Create cluster snapshot
aws rds create-db-cluster-snapshot \
    --db-cluster-snapshot-identifier my-cluster-snap-01 \
    --db-cluster-identifier my-cluster

# Create instance snapshot (RDS only)
aws rds create-db-snapshot \
    --db-snapshot-identifier prod-postgres-snap-01 \
    --db-instance-identifier prod-postgres

# List snapshots
aws rds describe-db-cluster-snapshots --db-cluster-identifier my-cluster

# Copy snapshot cross-region
aws rds copy-db-cluster-snapshot \
    --source-db-cluster-snapshot-identifier arn:aws:rds:us-east-1:123456789012:cluster-snapshot:my-cluster-snap-01 \
    --target-db-cluster-snapshot-identifier my-cluster-snap-01-copy \
    --region eu-west-1

# Share snapshot with another account
aws rds modify-db-cluster-snapshot-attribute \
    --db-cluster-snapshot-identifier my-cluster-snap-01 \
    --attribute-name restore \
    --values-to-add 987654321098

Restore from Snapshot

# Restore Aurora cluster
aws rds restore-db-cluster-from-snapshot \
    --db-cluster-identifier restored-cluster \
    --snapshot-identifier my-cluster-snap-01 \
    --engine aurora-postgresql \
    --db-subnet-group-name my-subnet-group \
    --vpc-security-group-ids sg-12345678

# Then create instance
aws rds create-db-instance \
    --db-instance-identifier restored-primary \
    --db-cluster-identifier restored-cluster \
    --engine aurora-postgresql \
    --db-instance-class db.serverless

# Restore RDS instance
aws rds restore-db-instance-from-db-snapshot \
    --db-instance-identifier restored-postgres \
    --db-snapshot-identifier prod-postgres-snap-01 \
    --db-instance-class db.t3.medium

Aurora Cloning

# Fast clone (copy-on-write)
aws rds restore-db-cluster-to-point-in-time \
    --source-db-cluster-identifier my-cluster \
    --db-cluster-identifier test-clone \
    --restore-type copy-on-write \
    --use-latest-restorable-time

# Clone to specific point in time
aws rds restore-db-cluster-to-point-in-time \
    --source-db-cluster-identifier my-cluster \
    --db-cluster-identifier test-clone \
    --restore-type copy-on-write \
    --restore-to-time "2024-01-15T12:00:00Z"

Blue/Green Deployments

Create Blue/Green Deployment

# Create deployment for version upgrade
aws rds create-blue-green-deployment \
    --blue-green-deployment-name pg-upgrade-16 \
    --source arn:aws:rds:us-east-1:123456789012:cluster:my-cluster \
    --target-engine-version 16.1

# Create deployment with instance class change
aws rds create-blue-green-deployment \
    --blue-green-deployment-name scale-up \
    --source arn:aws:rds:us-east-1:123456789012:db:prod-postgres \
    --target-db-instance-class db.r6g.xlarge

Manage Blue/Green Deployment

# Describe deployment
aws rds describe-blue-green-deployments \
    --blue-green-deployment-identifier bgd-12345678

# Switchover (after verification)
aws rds switchover-blue-green-deployment \
    --blue-green-deployment-identifier bgd-12345678 \
    --switchover-timeout 300

# Delete deployment (cleanup old environment)
aws rds delete-blue-green-deployment \
    --blue-green-deployment-identifier bgd-12345678 \
    --delete-target

Parameter Groups

Create Parameter Group

# Create cluster parameter group
aws rds create-db-cluster-parameter-group \
    --db-cluster-parameter-group-name my-aurora-pg-params \
    --db-parameter-group-family aurora-postgresql15 \
    --description "Custom Aurora PostgreSQL parameters"

# Create instance parameter group
aws rds create-db-parameter-group \
    --db-parameter-group-name my-postgres-params \
    --db-parameter-group-family postgres15 \
    --description "Custom PostgreSQL parameters"

Modify Parameters

# Modify cluster parameters
aws rds modify-db-cluster-parameter-group \
    --db-cluster-parameter-group-name my-aurora-pg-params \
    --parameters "ParameterName=log_statement,ParameterValue=all,ApplyMethod=immediate" \
                 "ParameterName=log_min_duration_statement,ParameterValue=1000,ApplyMethod=immediate"

# Apply to cluster
aws rds modify-db-cluster \
    --db-cluster-identifier my-cluster \
    --db-cluster-parameter-group-name my-aurora-pg-params \
    --apply-immediately

IAM Database Authentication

Enable IAM Auth

# Enable on cluster
aws rds modify-db-cluster \
    --db-cluster-identifier my-cluster \
    --enable-iam-database-authentication \
    --apply-immediately

# Enable on RDS instance
aws rds modify-db-instance \
    --db-instance-identifier prod-postgres \
    --enable-iam-database-authentication \
    --apply-immediately

Generate Auth Token

# Generate authentication token
aws rds generate-db-auth-token \
    --hostname my-cluster.cluster-abc123.us-east-1.rds.amazonaws.com \
    --port 5432 \
    --username iam_user \
    --region us-east-1

Bastion Access to Aurora

Port Forwarding via SSM

# Get cluster endpoint
ENDPOINT=$(aws rds describe-db-clusters \
    --db-cluster-identifier my-cluster \
    --query 'DBClusters[0].Endpoint' \
    --output text)

# Start port forwarding through bastion
aws ssm start-session \
    --target i-bastion-instance-id \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters "{\"host\":[\"$ENDPOINT\"],\"portNumber\":[\"5432\"],\"localPortNumber\":[\"5432\"]}"

# In another terminal, connect via psql
psql -h localhost -p 5432 -U admin -d mydb

SSH Tunnel via SSM

# Configure SSH (~/.ssh/config)
# Host i-* mi-*
#     ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"

# Create SSH tunnel
ssh -L 5432:my-cluster.cluster-abc123.us-east-1.rds.amazonaws.com:5432 \
    ec2-user@i-bastion-instance-id -N &

# Connect via tunnel
psql -h localhost -p 5432 -U admin -d mydb

Monitoring

Enable Enhanced Monitoring

aws rds modify-db-instance \
    --db-instance-identifier my-primary \
    --monitoring-interval 60 \
    --monitoring-role-arn arn:aws:iam::123456789012:role/rds-monitoring-role

Enable Performance Insights

aws rds modify-db-instance \
    --db-instance-identifier my-primary \
    --enable-performance-insights \
    --performance-insights-retention-period 7

CloudWatch Logs

# Enable log exports
aws rds modify-db-cluster \
    --db-cluster-identifier my-cluster \
    --cloudwatch-logs-export-configuration \
        EnableLogTypes=postgresql,upgrade

Useful Queries

# Get cluster status
aws rds describe-db-clusters \
    --db-cluster-identifier my-cluster \
    --query 'DBClusters[0].Status'

# List all endpoints
aws rds describe-db-clusters \
    --db-cluster-identifier my-cluster \
    --query 'DBClusters[0].{Writer:Endpoint,Reader:ReaderEndpoint}'

# Get current capacity (Serverless v2)
aws rds describe-db-clusters \
    --db-cluster-identifier my-cluster \
    --query 'DBClusters[0].ServerlessV2ScalingConfiguration'

# Find instances in cluster
aws rds describe-db-instances \
    --filters Name=db-cluster-id,Values=my-cluster \
    --query 'DBInstances[*].{ID:DBInstanceIdentifier,Class:DBInstanceClass,Status:DBInstanceStatus}'

Best Practices

Bastion & Tunneling

SSM Session Manager

The modern, secure replacement for SSH bastion hosts. No open inbound ports required.

Prerequisites

# Install Session Manager plugin
# macOS
brew install --cask session-manager-plugin

# Verify installation
session-manager-plugin

# Required VPC endpoints (for private subnets without NAT)
# - ssm.region.amazonaws.com
# - ssmmessages.region.amazonaws.com
# - ec2messages.region.amazonaws.com

Interactive Shell

# Start session to EC2 instance
aws ssm start-session --target i-0123456789abcdef0

# Start session with specific region
aws ssm start-session \
    --target i-0123456789abcdef0 \
    --region us-west-2

# Start session to on-premises managed instance
aws ssm start-session --target mi-0123456789abcdef0

Port Forwarding (Local)

Forward a local port to a port on the remote instance.

# Forward local port to remote port
aws ssm start-session \
    --target i-0123456789abcdef0 \
    --document-name AWS-StartPortForwardingSession \
    --parameters '{"portNumber":["8080"],"localPortNumber":["8080"]}'

# RDP access (Windows)
aws ssm start-session \
    --target i-windows-instance \
    --document-name AWS-StartPortForwardingSession \
    --parameters '{"portNumber":["3389"],"localPortNumber":["33389"]}'
# Then: mstsc /v:localhost:33389

# VNC access
aws ssm start-session \
    --target i-linux-instance \
    --document-name AWS-StartPortForwardingSession \
    --parameters '{"portNumber":["5901"],"localPortNumber":["5901"]}'

Remote Host Port Forwarding

Access resources through a bastion instance (jump host pattern).

# Access RDS through bastion
aws ssm start-session \
    --target i-bastion-instance \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{
        "host":["mydb.cluster-xyz.us-east-1.rds.amazonaws.com"],
        "portNumber":["5432"],
        "localPortNumber":["5432"]
    }'
# Now connect: psql -h localhost -p 5432 -U admin mydb

# Access Aurora MySQL
aws ssm start-session \
    --target i-bastion-instance \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{
        "host":["aurora-cluster.cluster-xyz.us-east-1.rds.amazonaws.com"],
        "portNumber":["3306"],
        "localPortNumber":["3306"]
    }'
# Now connect: mysql -h 127.0.0.1 -P 3306 -u admin -p

# Access ElastiCache Redis
aws ssm start-session \
    --target i-bastion-instance \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{
        "host":["redis-cluster.xyz.cache.amazonaws.com"],
        "portNumber":["6379"],
        "localPortNumber":["6379"]
    }'
# Now connect: redis-cli -h localhost -p 6379

# Access OpenSearch
aws ssm start-session \
    --target i-bastion-instance \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{
        "host":["search-domain.us-east-1.es.amazonaws.com"],
        "portNumber":["443"],
        "localPortNumber":["9200"]
    }'
# Now: curl https://localhost:9200

# Access internal ALB
aws ssm start-session \
    --target i-bastion-instance \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{
        "host":["internal-alb-123.us-east-1.elb.amazonaws.com"],
        "portNumber":["443"],
        "localPortNumber":["8443"]
    }'

EKS Private Cluster Access

Access private EKS clusters through SSM without exposing the API server publicly.

Update Kubeconfig for Private Cluster

# Get cluster info
aws eks describe-cluster --name my-cluster \
    --query 'cluster.{endpoint:endpoint,ca:certificateAuthority.data}'

# Update kubeconfig (will fail if cluster is private)
aws eks update-kubeconfig --name my-cluster --region us-east-1

Kubectl via SSM Port Forward

# 1. Get private endpoint
ENDPOINT=$(aws eks describe-cluster --name my-cluster \
    --query 'cluster.endpoint' --output text)
# Example: https://ABC123.gr7.us-east-1.eks.amazonaws.com

# 2. Extract hostname
EKS_HOST=$(echo $ENDPOINT | sed 's|https://||')

# 3. Start port forwarding to EKS API (port 443)
aws ssm start-session \
    --target i-bastion-in-eks-vpc \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters "{
        \"host\":[\"$EKS_HOST\"],
        \"portNumber\":[\"443\"],
        \"localPortNumber\":[\"6443\"]
    }"

# 4. In another terminal, modify kubeconfig
# Add to ~/.kube/config:
# clusters:
# - cluster:
#     server: https://127.0.0.1:6443
#     certificate-authority-data: <base64-ca-from-cluster>
#   name: my-cluster

# 5. Run kubectl
kubectl get nodes
kubectl get pods -A

Alternative: SSM Document for EKS

# Create custom SSM document for kubectl
aws ssm create-document \
    --name "EKS-Kubectl" \
    --document-type "Session" \
    --content '{
        "schemaVersion": "1.0",
        "description": "Run kubectl commands via SSM",
        "sessionType": "InteractiveCommands",
        "inputs": {
            "runAsEnabled": true,
            "runAsDefaultUser": "ec2-user"
        }
    }'

# Run kubectl commands directly on bastion
aws ssm start-session \
    --target i-bastion-with-kubectl \
    --document-name EKS-Kubectl

Using SSM as SSH Proxy for EKS

# ~/.ssh/config addition for SSM proxy
Host eks-bastion
    HostName i-bastion-instance-id
    User ec2-user
    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"

# SSH to bastion, then kubectl
ssh eks-bastion
kubectl get nodes

Aurora/RDS Database Access

Direct Port Forwarding

# PostgreSQL (Aurora/RDS)
aws ssm start-session \
    --target i-bastion \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{
        "host":["prod-db.cluster-xyz.us-east-1.rds.amazonaws.com"],
        "portNumber":["5432"],
        "localPortNumber":["5432"]
    }'

# Connect with psql
psql "host=localhost port=5432 dbname=mydb user=admin sslmode=require"

# Connect with DBeaver/pgAdmin
# Host: localhost, Port: 5432, SSL: Required

# MySQL/Aurora MySQL
aws ssm start-session \
    --target i-bastion \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{
        "host":["prod-db.cluster-xyz.us-east-1.rds.amazonaws.com"],
        "portNumber":["3306"],
        "localPortNumber":["3306"]
    }'

# Connect with mysql client
mysql -h 127.0.0.1 -P 3306 -u admin -p --ssl-mode=REQUIRED

JDBC Connection Through Tunnel

# Start tunnel in background
aws ssm start-session \
    --target i-bastion \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters '{
        "host":["aurora.cluster-xyz.us-east-1.rds.amazonaws.com"],
        "portNumber":["5432"],
        "localPortNumber":["5432"]
    }' &

# JDBC URL (use in application/IDE)
# PostgreSQL: jdbc:postgresql://localhost:5432/mydb?ssl=true
# MySQL: jdbc:mysql://localhost:3306/mydb?useSSL=true

Shell Script for DB Tunnel

#!/bin/bash
# db-tunnel.sh - Start database tunnel

DB_HOST="${1:-prod-db.cluster-xyz.us-east-1.rds.amazonaws.com}"
DB_PORT="${2:-5432}"
LOCAL_PORT="${3:-5432}"
BASTION_ID="${BASTION_INSTANCE_ID:-i-0123456789abcdef0}"

echo "Starting tunnel to $DB_HOST:$DB_PORT on localhost:$LOCAL_PORT"
aws ssm start-session \
    --target "$BASTION_ID" \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters "{
        \"host\":[\"$DB_HOST\"],
        \"portNumber\":[\"$DB_PORT\"],
        \"localPortNumber\":[\"$LOCAL_PORT\"]
    }"

SSH Access Methods

EC2 Instance Connect

Push a temporary SSH key (valid 60 seconds).

# Push public key
aws ec2-instance-connect send-ssh-public-key \
    --instance-id i-0123456789abcdef0 \
    --instance-os-user ec2-user \
    --ssh-public-key file://~/.ssh/id_rsa.pub

# SSH within 60 seconds
ssh ec2-user@<public-ip>

# One-liner with AWS CLI
aws ec2-instance-connect send-ssh-public-key \
    --instance-id i-123 \
    --instance-os-user ec2-user \
    --ssh-public-key file://~/.ssh/id_rsa.pub && \
    ssh ec2-user@$(aws ec2 describe-instances --instance-ids i-123 \
        --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)

SSM as SSH Proxy

Configure SSH to tunnel through SSM (no public IP needed).

# ~/.ssh/config
Host i-* mi-*
    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"
    User ec2-user
    IdentityFile ~/.ssh/my-key.pem

Host bastion-prod
    HostName i-0123456789abcdef0
    User ec2-user
    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"

# Usage
ssh i-0123456789abcdef0
ssh bastion-prod

# SCP through SSM
scp -o ProxyCommand="aws ssm start-session --target i-123 --document-name AWS-StartSSHSession --parameters portNumber=22" \
    myfile.txt ec2-user@i-123:/home/ec2-user/

Multi-Hop SSH Through Bastion

# ~/.ssh/config for jump host pattern
Host bastion
    HostName i-bastion-id
    User ec2-user
    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"

Host private-server
    HostName 10.0.1.100
    User ec2-user
    ProxyJump bastion
    IdentityFile ~/.ssh/private-key.pem

# Usage
ssh private-server

Run Command (Remote Execution)

Execute Commands

# Run command on single instance
aws ssm send-command \
    --instance-ids i-0123456789abcdef0 \
    --document-name "AWS-RunShellScript" \
    --parameters 'commands=["df -h","free -m"]'

# Run on multiple instances
aws ssm send-command \
    --instance-ids i-123 i-456 i-789 \
    --document-name "AWS-RunShellScript" \
    --parameters 'commands=["yum update -y"]'

# Run on instances by tag
aws ssm send-command \
    --targets Key=tag:Environment,Values=Production \
    --document-name "AWS-RunShellScript" \
    --parameters 'commands=["systemctl restart nginx"]'

# Run with timeout
aws ssm send-command \
    --instance-ids i-123 \
    --document-name "AWS-RunShellScript" \
    --parameters 'commands=["./long-running-script.sh"]' \
    --timeout-seconds 3600

Get Command Output

# Get command invocation results
COMMAND_ID=$(aws ssm send-command \
    --instance-ids i-123 \
    --document-name "AWS-RunShellScript" \
    --parameters 'commands=["hostname"]' \
    --query 'Command.CommandId' \
    --output text)

# Wait and get output
aws ssm get-command-invocation \
    --command-id $COMMAND_ID \
    --instance-id i-123 \
    --query '{Status:Status,Output:StandardOutputContent}'

# List all command invocations
aws ssm list-command-invocations \
    --command-id $COMMAND_ID \
    --details

Windows Commands

# PowerShell command
aws ssm send-command \
    --instance-ids i-windows-123 \
    --document-name "AWS-RunPowerShellScript" \
    --parameters 'commands=["Get-Process | Sort-Object CPU -Descending | Select-Object -First 10"]'

# Install Windows feature
aws ssm send-command \
    --instance-ids i-windows-123 \
    --document-name "AWS-RunPowerShellScript" \
    --parameters 'commands=["Install-WindowsFeature -Name Web-Server -IncludeManagementTools"]'

Session Manager Preferences

Configure Logging

# Create preferences document
aws ssm update-document \
    --name "SSM-SessionManagerRunShell" \
    --document-version "\$LATEST" \
    --content '{
        "schemaVersion": "1.0",
        "description": "Session Manager Preferences",
        "sessionType": "Standard_Stream",
        "inputs": {
            "s3BucketName": "my-session-logs-bucket",
            "s3KeyPrefix": "session-logs/",
            "s3EncryptionEnabled": true,
            "cloudWatchLogGroupName": "/aws/ssm/session-logs",
            "cloudWatchEncryptionEnabled": true,
            "kmsKeyId": "alias/session-manager-key",
            "runAsEnabled": true,
            "runAsDefaultUser": "ssm-user",
            "idleSessionTimeout": "20",
            "shellProfile": {
                "linux": "cd ~ && bash",
                "windows": ""
            }
        }
    }'

Custom Session Documents

# Create interactive command document
aws ssm create-document \
    --name "Custom-InteractiveSession" \
    --document-type "Session" \
    --content '{
        "schemaVersion": "1.0",
        "description": "Custom interactive session",
        "sessionType": "InteractiveCommands",
        "inputs": {
            "runAsEnabled": true,
            "runAsDefaultUser": "admin"
        }
    }'

Automation Patterns

Dynamic Bastion Discovery

Find bastion instances by tag instead of hardcoding instance IDs.

# Find bastion by tag
BASTION_ID=$(aws ec2 describe-instances \
    --filters "Name=tag:Role,Values=bastion" \
              "Name=instance-state-name,Values=running" \
    --query 'Reservations[0].Instances[0].InstanceId' \
    --output text)

# Verify bastion found
if [ "$BASTION_ID" = "None" ] || [ -z "$BASTION_ID" ]; then
    echo "Error: No running bastion instance found"
    exit 1
fi

# Use in SSM session
aws ssm start-session --target "$BASTION_ID"

# Find by multiple tags (environment + role)
BASTION_ID=$(aws ec2 describe-instances \
    --filters "Name=tag:Environment,Values=production" \
              "Name=tag:Role,Values=bastion" \
              "Name=instance-state-name,Values=running" \
    --query 'Reservations[0].Instances[0].InstanceId' \
    --output text)

Clean Shell Environment

Ensure SSO credentials aren't overridden by static environment variables.

# Clear any static credentials before SSO login
unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
unset AWS_SESSION_TOKEN

# Now SSO login works correctly
export AWS_PROFILE=my-sso-profile
aws sso login

# Verify using SSO credentials (not static)
aws configure list
# Should show: profile my-sso-profile, not environment variables

# Full clean environment script
clean_aws_env() {
    unset AWS_ACCESS_KEY_ID
    unset AWS_SECRET_ACCESS_KEY
    unset AWS_SESSION_TOKEN
    unset AWS_SECURITY_TOKEN
    echo "Cleared static AWS credentials from environment"
}

Credential Validation Patterns

Check if credentials are valid before proceeding.

# Validate credentials (returns 0 if valid, non-zero if expired/invalid)
validate_aws_creds() {
    if aws sts get-caller-identity &>/dev/null; then
        echo "✓ AWS credentials valid"
        aws sts get-caller-identity --query 'Arn' --output text
        return 0
    else
        echo "✗ AWS credentials invalid or expired"
        return 1
    fi
}

# Auto-login if credentials expired (SSO profiles)
ensure_aws_login() {
    if ! aws sts get-caller-identity &>/dev/null; then
        echo "Credentials expired, logging in..."
        aws sso login --profile "${AWS_PROFILE:-default}"
    fi
}

# Use in scripts
ensure_aws_login
aws s3 ls

# Check profile before operations
check_profile() {
    local current=$(aws configure list --query 'profile' --output text 2>/dev/null)
    echo "Current profile: ${AWS_PROFILE:-default}"
    echo "Account: $(aws sts get-caller-identity --query 'Account' --output text)"
    echo "ARN: $(aws sts get-caller-identity --query 'Arn' --output text)"
}

Port Conflict Resolution

Handle "Address already in use" errors.

# Check what's using a port
lsof -i :5432

# Find and show process on port
check_port() {
    local port=$1
    local pid=$(lsof -ti :$port 2>/dev/null)
    if [ -n "$pid" ]; then
        echo "Port $port in use by PID $pid:"
        ps -p $pid -o pid,user,command
        return 1
    else
        echo "Port $port is available"
        return 0
    fi
}

# Kill process on port (use with caution)
free_port() {
    local port=$1
    local pid=$(lsof -ti :$port 2>/dev/null)
    if [ -n "$pid" ]; then
        echo "Killing process $pid on port $port"
        kill -9 $pid
        sleep 1
    fi
}

# Smart port selection (find available port)
find_available_port() {
    local start_port=${1:-5432}
    local port=$start_port
    while lsof -ti :$port &>/dev/null; do
        ((port++))
    done
    echo $port
}

# Example: auto-select port for DB tunnel
LOCAL_PORT=$(find_available_port 5432)
echo "Using port $LOCAL_PORT"

Multiple Cluster Access

Work with multiple EKS clusters simultaneously.

# Each terminal: different cluster
# Terminal 1 (Dev)
export AWS_PROFILE=dev-admin
export KUBECONFIG=~/.kube/config-dev
./connect-cluster.sh dev-cluster

# Terminal 2 (Staging)
export AWS_PROFILE=staging-admin
export KUBECONFIG=~/.kube/config-staging
./connect-cluster.sh staging-cluster

# Terminal 3 (Prod - read only)
export AWS_PROFILE=prod-readonly
export KUBECONFIG=~/.kube/config-prod
./connect-cluster.sh prod-cluster

# Quick cluster context switch script
use_cluster() {
    local env=$1
    case $env in
        dev)
            export AWS_PROFILE=dev-admin
            export KUBECONFIG=~/.kube/config-dev
            ;;
        staging)
            export AWS_PROFILE=staging-admin
            export KUBECONFIG=~/.kube/config-staging
            ;;
        prod)
            export AWS_PROFILE=prod-readonly
            export KUBECONFIG=~/.kube/config-prod
            ;;
        *)
            echo "Unknown environment: $env"
            return 1
            ;;
    esac
    echo "Switched to $env environment"
    kubectl config current-context
}

Session Cleanup on Exit

Ensure SSM sessions are cleaned up when shell exits.

#!/bin/bash
# ssm-connect.sh - Connect with automatic cleanup

SESSION_PID=""
SSM_LOG="/tmp/ssm-session-$$.log"

cleanup() {
    echo "Cleaning up SSM session..."
    [ -n "$SESSION_PID" ] && kill $SESSION_PID 2>/dev/null
    rm -f "$SSM_LOG"
    echo "Session terminated"
}

# Register cleanup on exit
trap cleanup EXIT INT TERM

# Start SSM port forwarding in background
aws ssm start-session \
    --target "$BASTION_ID" \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters "{
        \"host\":[\"$REMOTE_HOST\"],
        \"portNumber\":[\"$REMOTE_PORT\"],
        \"localPortNumber\":[\"$LOCAL_PORT\"]
    }" > "$SSM_LOG" 2>&1 &

SESSION_PID=$!

# Wait for tunnel to establish
sleep 3

# Check if session is running
if ! kill -0 $SESSION_PID 2>/dev/null; then
    echo "Failed to start SSM session"
    cat "$SSM_LOG"
    exit 1
fi

echo "Tunnel established on localhost:$LOCAL_PORT"
echo "Press Ctrl+C or type 'exit' to disconnect"

# Start interactive shell
$SHELL

# Cleanup happens automatically via trap

EKS Connection Script

Complete script for connecting to private EKS clusters.

#!/bin/bash
# eks-connect.sh <cluster-name> [local-port]

CLUSTER_NAME=${1:?Usage: eks-connect.sh <cluster-name> [local-port]}
LOCAL_PORT=${2:-6443}

# Ensure clean environment
unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN

# Validate credentials
if ! aws sts get-caller-identity &>/dev/null; then
    echo "Please login first: aws sso login"
    exit 1
fi

# Get cluster endpoint
ENDPOINT=$(aws eks describe-cluster --name "$CLUSTER_NAME" \
    --query 'cluster.endpoint' --output text)
EKS_HOST=$(echo "$ENDPOINT" | sed 's|https://||')

# Find bastion
BASTION_ID=$(aws ec2 describe-instances \
    --filters "Name=tag:Role,Values=bastion" \
              "Name=instance-state-name,Values=running" \
    --query 'Reservations[0].Instances[0].InstanceId' \
    --output text)

if [ "$BASTION_ID" = "None" ]; then
    echo "Error: No bastion found"
    exit 1
fi

# Check port availability
if lsof -ti :$LOCAL_PORT &>/dev/null; then
    echo "Port $LOCAL_PORT in use, finding alternative..."
    LOCAL_PORT=$(find_available_port $LOCAL_PORT)
fi

echo "Connecting to $CLUSTER_NAME via $BASTION_ID on port $LOCAL_PORT"

# Update kubeconfig
aws eks update-kubeconfig --name "$CLUSTER_NAME" 2>/dev/null
# Override server to use local port
kubectl config set-cluster "$CLUSTER_NAME" --server="https://127.0.0.1:$LOCAL_PORT"

# Start session (cleanup via trap)
trap 'kill $SSM_PID 2>/dev/null; exit' EXIT INT TERM

aws ssm start-session \
    --target "$BASTION_ID" \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters "{
        \"host\":[\"$EKS_HOST\"],
        \"portNumber\":[\"443\"],
        \"localPortNumber\":[\"$LOCAL_PORT\"]
    }" &
SSM_PID=$!

sleep 3

echo "Connected! Run kubectl commands in this shell."
echo "Type 'exit' to disconnect."

# Interactive shell with custom KUBECONFIG
KUBECONFIG=~/.kube/config $SHELL

---

Useful Queries

# List SSM-managed instances
aws ssm describe-instance-information \
    --query 'InstanceInformationList[*].{ID:InstanceId,IP:IPAddress,Platform:PlatformType,Status:PingStatus}'

# Find instances by tag
aws ssm describe-instance-information \
    --filters Key=tag:Environment,Values=Production \
    --query 'InstanceInformationList[*].InstanceId'

# List active sessions
aws ssm describe-sessions \
    --state Active \
    --query 'Sessions[*].{SessionId:SessionId,Target:Target,Owner:Owner}'

# Get session history
aws ssm describe-sessions \
    --state History \
    --filters key=Owner,value=$(aws sts get-caller-identity --query 'Arn' --output text) \
    --query 'Sessions[*].{SessionId:SessionId,Target:Target,StartDate:StartDate}'

# Terminate session
aws ssm terminate-session --session-id session-id-here

Best Practices

DynamoDB

Table Management

Create Table (On-Demand)

# Simple table with on-demand billing
aws dynamodb create-table \
    --table-name Users \
    --attribute-definitions \
        AttributeName=UserId,AttributeType=S \
    --key-schema \
        AttributeName=UserId,KeyType=HASH \
    --billing-mode PAY_PER_REQUEST

# With Global Secondary Index
aws dynamodb create-table \
    --table-name Users \
    --attribute-definitions \
        AttributeName=UserId,AttributeType=S \
        AttributeName=Email,AttributeType=S \
    --key-schema \
        AttributeName=UserId,KeyType=HASH \
    --billing-mode PAY_PER_REQUEST \
    --global-secondary-indexes '[
        {
            "IndexName": "EmailIndex",
            "KeySchema": [{"AttributeName": "Email", "KeyType": "HASH"}],
            "Projection": {"ProjectionType": "ALL"}
        }
    ]'

# Composite key (partition + sort)
aws dynamodb create-table \
    --table-name Orders \
    --attribute-definitions \
        AttributeName=CustomerId,AttributeType=S \
        AttributeName=OrderDate,AttributeType=S \
    --key-schema \
        AttributeName=CustomerId,KeyType=HASH \
        AttributeName=OrderDate,KeyType=RANGE \
    --billing-mode PAY_PER_REQUEST

Create Table (Provisioned)

aws dynamodb create-table \
    --table-name Logs \
    --attribute-definitions AttributeName=LogId,AttributeType=S \
    --key-schema AttributeName=LogId,KeyType=HASH \
    --billing-mode PROVISIONED \
    --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

Manage Tables

# List tables
aws dynamodb list-tables

# Describe table
aws dynamodb describe-table --table-name Users

# Delete table
aws dynamodb delete-table --table-name Users

# Wait for table to be active
aws dynamodb wait table-exists --table-name Users

Update Table

# Switch to provisioned capacity
aws dynamodb update-table \
    --table-name Users \
    --billing-mode PROVISIONED \
    --provisioned-throughput ReadCapacityUnits=10,WriteCapacityUnits=10

# Switch to on-demand
aws dynamodb update-table \
    --table-name Users \
    --billing-mode PAY_PER_REQUEST

# Add GSI
aws dynamodb update-table \
    --table-name Users \
    --attribute-definitions AttributeName=Status,AttributeType=S \
    --global-secondary-index-updates '[
        {
            "Create": {
                "IndexName": "StatusIndex",
                "KeySchema": [{"AttributeName": "Status", "KeyType": "HASH"}],
                "Projection": {"ProjectionType": "ALL"}
            }
        }
    ]'

# Delete GSI
aws dynamodb update-table \
    --table-name Users \
    --global-secondary-index-updates '[
        {"Delete": {"IndexName": "StatusIndex"}}
    ]'

Item Operations

Put Item

# Basic put
aws dynamodb put-item \
    --table-name Users \
    --item '{
        "UserId": {"S": "u-123"},
        "Name": {"S": "Alice"},
        "Age": {"N": "30"},
        "Email": {"S": "alice@example.com"},
        "Tags": {"SS": ["admin", "active"]}
    }'

# Put with condition (only if not exists)
aws dynamodb put-item \
    --table-name Users \
    --item '{"UserId": {"S": "u-123"}, "Name": {"S": "Alice"}}' \
    --condition-expression "attribute_not_exists(UserId)"

# Return old values
aws dynamodb put-item \
    --table-name Users \
    --item '{"UserId": {"S": "u-123"}, "Name": {"S": "Bob"}}' \
    --return-values ALL_OLD

Get Item

# Basic get
aws dynamodb get-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}'

# Get specific attributes only
aws dynamodb get-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --projection-expression "Name, Email"

# Consistent read
aws dynamodb get-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --consistent-read

Update Item

# Update with expression
aws dynamodb update-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --update-expression "SET Age = :age, #status = :status" \
    --expression-attribute-names '{"#status": "Status"}' \
    --expression-attribute-values '{":age": {"N": "31"}, ":status": {"S": "active"}}'

# Increment counter
aws dynamodb update-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --update-expression "SET LoginCount = if_not_exists(LoginCount, :zero) + :inc" \
    --expression-attribute-values '{":zero": {"N": "0"}, ":inc": {"N": "1"}}'

# Add to set
aws dynamodb update-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --update-expression "ADD Tags :newTags" \
    --expression-attribute-values '{":newTags": {"SS": ["premium"]}}'

# Remove attribute
aws dynamodb update-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --update-expression "REMOVE TemporaryField"

# Conditional update
aws dynamodb update-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --update-expression "SET Balance = Balance - :amount" \
    --condition-expression "Balance >= :amount" \
    --expression-attribute-values '{":amount": {"N": "100"}}'

Delete Item

# Basic delete
aws dynamodb delete-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}'

# Conditional delete
aws dynamodb delete-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --condition-expression "#status = :inactive" \
    --expression-attribute-names '{"#status": "Status"}' \
    --expression-attribute-values '{":inactive": {"S": "inactive"}}'

# Return deleted item
aws dynamodb delete-item \
    --table-name Users \
    --key '{"UserId": {"S": "u-123"}}' \
    --return-values ALL_OLD

Query and Scan

Query (Efficient)

# Query by partition key
aws dynamodb query \
    --table-name Orders \
    --key-condition-expression "CustomerId = :cid" \
    --expression-attribute-values '{":cid": {"S": "c-123"}}'

# Query with sort key condition
aws dynamodb query \
    --table-name Orders \
    --key-condition-expression "CustomerId = :cid AND OrderDate BETWEEN :start AND :end" \
    --expression-attribute-values '{
        ":cid": {"S": "c-123"},
        ":start": {"S": "2024-01-01"},
        ":end": {"S": "2024-12-31"}
    }'

# Query with filter
aws dynamodb query \
    --table-name Orders \
    --key-condition-expression "CustomerId = :cid" \
    --filter-expression "Amount > :min" \
    --expression-attribute-values '{":cid": {"S": "c-123"}, ":min": {"N": "100"}}'

# Query GSI
aws dynamodb query \
    --table-name Users \
    --index-name EmailIndex \
    --key-condition-expression "Email = :email" \
    --expression-attribute-values '{":email": {"S": "alice@example.com"}}'

# Reverse order (descending)
aws dynamodb query \
    --table-name Orders \
    --key-condition-expression "CustomerId = :cid" \
    --expression-attribute-values '{":cid": {"S": "c-123"}}' \
    --scan-index-forward false \
    --limit 10

Scan (Use Sparingly)

# Basic scan
aws dynamodb scan --table-name Users

# Scan with filter
aws dynamodb scan \
    --table-name Users \
    --filter-expression "Age > :age" \
    --expression-attribute-values '{":age": {"N": "25"}}'

# Parallel scan (for large tables)
aws dynamodb scan \
    --table-name Users \
    --segment 0 \
    --total-segments 4

PartiQL

# Select
aws dynamodb execute-statement \
    --statement "SELECT * FROM Users WHERE UserId = 'u-123'"

# Select with filter
aws dynamodb execute-statement \
    --statement "SELECT Name, Email FROM Users WHERE Age > 25"

# Insert
aws dynamodb execute-statement \
    --statement "INSERT INTO Users VALUE {'UserId': 'u-456', 'Name': 'Bob'}"

# Update
aws dynamodb execute-statement \
    --statement "UPDATE Users SET Age = 32 WHERE UserId = 'u-123'"

# Delete
aws dynamodb execute-statement \
    --statement "DELETE FROM Users WHERE UserId = 'u-123'"

Batch Operations

Batch Write

# Write multiple items (max 25)
aws dynamodb batch-write-item \
    --request-items '{
        "Users": [
            {"PutRequest": {"Item": {"UserId": {"S": "u-1"}, "Name": {"S": "Alice"}}}},
            {"PutRequest": {"Item": {"UserId": {"S": "u-2"}, "Name": {"S": "Bob"}}}},
            {"DeleteRequest": {"Key": {"UserId": {"S": "u-old"}}}}
        ]
    }'

Batch Get

# Get multiple items (max 100)
aws dynamodb batch-get-item \
    --request-items '{
        "Users": {
            "Keys": [
                {"UserId": {"S": "u-1"}},
                {"UserId": {"S": "u-2"}},
                {"UserId": {"S": "u-3"}}
            ],
            "ProjectionExpression": "UserId, Name, Email"
        }
    }'

Transactions

TransactWrite (All-or-nothing writes)

aws dynamodb transact-write-items \
    --transact-items '[
        {
            "Put": {
                "TableName": "Orders",
                "Item": {"OrderId": {"S": "o-123"}, "CustomerId": {"S": "c-123"}, "Amount": {"N": "100"}}
            }
        },
        {
            "Update": {
                "TableName": "Customers",
                "Key": {"CustomerId": {"S": "c-123"}},
                "UpdateExpression": "SET Balance = Balance - :amount",
                "ConditionExpression": "Balance >= :amount",
                "ExpressionAttributeValues": {":amount": {"N": "100"}}
            }
        }
    ]'

TransactGet (Consistent reads)

aws dynamodb transact-get-items \
    --transact-items '[
        {"Get": {"TableName": "Users", "Key": {"UserId": {"S": "u-123"}}}},
        {"Get": {"TableName": "Orders", "Key": {"OrderId": {"S": "o-123"}}}}
    ]'

DynamoDB Streams

Enable Streams

# Enable streams on table
aws dynamodb update-table \
    --table-name Users \
    --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES

# Stream view types:
# KEYS_ONLY - Only key attributes
# NEW_IMAGE - Item after modification
# OLD_IMAGE - Item before modification
# NEW_AND_OLD_IMAGES - Both before and after

Read Streams

# Get stream ARN
aws dynamodb describe-table \
    --table-name Users \
    --query 'Table.LatestStreamArn' \
    --output text

# List stream shards
aws dynamodbstreams describe-stream \
    --stream-arn arn:aws:dynamodb:us-east-1:123456789012:table/Users/stream/2024-01-01T00:00:00.000

# Get shard iterator
aws dynamodbstreams get-shard-iterator \
    --stream-arn arn:aws:dynamodb:us-east-1:123456789012:table/Users/stream/2024-01-01T00:00:00.000 \
    --shard-id shardId-00000001 \
    --shard-iterator-type TRIM_HORIZON

# Get records
aws dynamodbstreams get-records \
    --shard-iterator <shard-iterator>

Global Tables

Create Global Table

# Create table in first region
aws dynamodb create-table \
    --table-name GlobalUsers \
    --attribute-definitions AttributeName=UserId,AttributeType=S \
    --key-schema AttributeName=UserId,KeyType=HASH \
    --billing-mode PAY_PER_REQUEST \
    --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES \
    --region us-east-1

# Wait for table
aws dynamodb wait table-exists --table-name GlobalUsers --region us-east-1

# Add replica in another region
aws dynamodb update-table \
    --table-name GlobalUsers \
    --replica-updates '[{"Create": {"RegionName": "eu-west-1"}}]' \
    --region us-east-1

# Add more replicas
aws dynamodb update-table \
    --table-name GlobalUsers \
    --replica-updates '[{"Create": {"RegionName": "ap-southeast-1"}}]' \
    --region us-east-1

Manage Global Tables

# Describe global table
aws dynamodb describe-table \
    --table-name GlobalUsers \
    --query 'Table.Replicas'

# Remove replica
aws dynamodb update-table \
    --table-name GlobalUsers \
    --replica-updates '[{"Delete": {"RegionName": "ap-southeast-1"}}]' \
    --region us-east-1

Time to Live (TTL)

Enable TTL

# Enable TTL on attribute
aws dynamodb update-time-to-live \
    --table-name Sessions \
    --time-to-live-specification Enabled=true,AttributeName=ExpiresAt

# Check TTL status
aws dynamodb describe-time-to-live --table-name Sessions

Use TTL

# Put item with TTL (Unix timestamp)
EXPIRES=$(date -d '+24 hours' +%s)
aws dynamodb put-item \
    --table-name Sessions \
    --item "{
        \"SessionId\": {\"S\": \"sess-123\"},
        \"UserId\": {\"S\": \"u-123\"},
        \"ExpiresAt\": {\"N\": \"$EXPIRES\"}
    }"

Backups and Recovery

On-Demand Backup

# Create backup
aws dynamodb create-backup \
    --table-name Users \
    --backup-name Users-2024-01-15

# List backups
aws dynamodb list-backups --table-name Users

# Describe backup
aws dynamodb describe-backup \
    --backup-arn arn:aws:dynamodb:us-east-1:123456789012:table/Users/backup/01234567890123-abc123

# Restore from backup
aws dynamodb restore-table-from-backup \
    --target-table-name Users-Restored \
    --backup-arn arn:aws:dynamodb:us-east-1:123456789012:table/Users/backup/01234567890123-abc123

# Delete backup
aws dynamodb delete-backup \
    --backup-arn arn:aws:dynamodb:us-east-1:123456789012:table/Users/backup/01234567890123-abc123

Point-in-Time Recovery (PITR)

# Enable PITR
aws dynamodb update-continuous-backups \
    --table-name Users \
    --point-in-time-recovery-specification PointInTimeRecoveryEnabled=true

# Check PITR status
aws dynamodb describe-continuous-backups --table-name Users

# Restore to point in time
aws dynamodb restore-table-to-point-in-time \
    --source-table-name Users \
    --target-table-name Users-Restored \
    --restore-date-time 2024-01-15T12:00:00Z

Export to S3

# Export table to S3 (for analytics)
aws dynamodb export-table-to-point-in-time \
    --table-arn arn:aws:dynamodb:us-east-1:123456789012:table/Users \
    --s3-bucket my-exports-bucket \
    --s3-prefix dynamodb-exports/ \
    --export-format DYNAMODB_JSON

# List exports
aws dynamodb list-exports --table-arn arn:aws:dynamodb:us-east-1:123456789012:table/Users

# Describe export
aws dynamodb describe-export \
    --export-arn arn:aws:dynamodb:us-east-1:123456789012:table/Users/export/01234567890123-abc123

Import from S3

# Import from S3
aws dynamodb import-table \
    --s3-bucket-source S3Bucket=my-import-bucket,S3KeyPrefix=imports/ \
    --input-format DYNAMODB_JSON \
    --table-creation-parameters '{
        "TableName": "ImportedUsers",
        "AttributeDefinitions": [{"AttributeName": "UserId", "AttributeType": "S"}],
        "KeySchema": [{"AttributeName": "UserId", "KeyType": "HASH"}],
        "BillingMode": "PAY_PER_REQUEST"
    }'

# List imports
aws dynamodb list-imports

# Describe import
aws dynamodb describe-import --import-arn <import-arn>

Auto Scaling (Provisioned Mode)

# Register scalable target (read capacity)
aws application-autoscaling register-scalable-target \
    --service-namespace dynamodb \
    --resource-id "table/Users" \
    --scalable-dimension "dynamodb:table:ReadCapacityUnits" \
    --min-capacity 5 \
    --max-capacity 1000

# Create scaling policy
aws application-autoscaling put-scaling-policy \
    --service-namespace dynamodb \
    --resource-id "table/Users" \
    --scalable-dimension "dynamodb:table:ReadCapacityUnits" \
    --policy-name "UsersReadScaling" \
    --policy-type TargetTrackingScaling \
    --target-tracking-scaling-policy-configuration '{
        "TargetValue": 70.0,
        "PredefinedMetricSpecification": {
            "PredefinedMetricType": "DynamoDBReadCapacityUtilization"
        },
        "ScaleOutCooldown": 60,
        "ScaleInCooldown": 60
    }'

Useful Queries

# Get table item count
aws dynamodb describe-table \
    --table-name Users \
    --query 'Table.ItemCount'

# Get table size in bytes
aws dynamodb describe-table \
    --table-name Users \
    --query 'Table.TableSizeBytes'

# List all GSIs
aws dynamodb describe-table \
    --table-name Users \
    --query 'Table.GlobalSecondaryIndexes[*].IndexName'

# Check table status
aws dynamodb describe-table \
    --table-name Users \
    --query 'Table.TableStatus'

# Get stream ARN
aws dynamodb describe-table \
    --table-name Users \
    --query 'Table.LatestStreamArn' \
    --output text

Best Practices

Elastic Container Registry (ECR)

Repository Management

Create Repository

# Basic repository with scanning and immutability
aws ecr create-repository \
    --repository-name my-app/backend \
    --image-scanning-configuration scanOnPush=true \
    --image-tag-mutability IMMUTABLE

# With KMS encryption
aws ecr create-repository \
    --repository-name my-app/backend \
    --image-scanning-configuration scanOnPush=true \
    --image-tag-mutability IMMUTABLE \
    --encryption-configuration encryptionType=KMS,kmsKey=alias/ecr-key

# With tags
aws ecr create-repository \
    --repository-name my-app/backend \
    --tags Key=Environment,Value=Production Key=Team,Value=Backend

List and Describe Repositories

# List all repositories
aws ecr describe-repositories

# Describe specific repository
aws ecr describe-repositories --repository-names my-app/backend

# Get repository URI
aws ecr describe-repositories \
    --repository-names my-app/backend \
    --query 'repositories[0].repositoryUri' \
    --output text

Delete Repository

# Delete empty repository
aws ecr delete-repository --repository-name my-app/backend

# Force delete (removes all images)
aws ecr delete-repository --repository-name my-app/backend --force

Authentication

Docker Login

# Get login password and authenticate Docker
aws ecr get-login-password --region us-east-1 | \
    docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com

# For a specific profile
aws ecr get-login-password --region us-east-1 --profile production | \
    docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com

Get Authorization Token Details

# Get base64-encoded token (valid 12 hours)
aws ecr get-authorization-token \
    --query 'authorizationData[0].authorizationToken' \
    --output text | base64 -d

# Get expiration time
aws ecr get-authorization-token \
    --query 'authorizationData[0].expiresAt'

Push/Pull Workflows

Build and Push Image

# Set variables
ACCOUNT_ID=123456789012
REGION=us-east-1
REPO_NAME=my-app/backend
TAG=v1.2.0

# Authenticate
aws ecr get-login-password --region $REGION | \
    docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com

# Build image
docker build -t $REPO_NAME:$TAG .

# Tag for ECR
docker tag $REPO_NAME:$TAG $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:$TAG

# Push
docker push $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:$TAG

# Also tag as latest
docker tag $REPO_NAME:$TAG $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:latest
docker push $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:latest

Pull Image

# Authenticate first
aws ecr get-login-password --region us-east-1 | \
    docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com

# Pull image
docker pull 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app/backend:v1.2.0

Image Management

List Images

# List all images in repository
aws ecr list-images --repository-name my-app/backend

# List with details
aws ecr describe-images --repository-name my-app/backend

# List images sorted by push date
aws ecr describe-images \
    --repository-name my-app/backend \
    --query 'sort_by(imageDetails, &imagePushedAt)[*].{Tag:imageTags[0],Pushed:imagePushedAt,Size:imageSizeInBytes}'

# Find untagged images
aws ecr describe-images \
    --repository-name my-app/backend \
    --filter tagStatus=UNTAGGED

Delete Images

# Delete by tag
aws ecr batch-delete-image \
    --repository-name my-app/backend \
    --image-ids imageTag=v1.0.0

# Delete by digest
aws ecr batch-delete-image \
    --repository-name my-app/backend \
    --image-ids imageDigest=sha256:abc123...

# Delete multiple images
aws ecr batch-delete-image \
    --repository-name my-app/backend \
    --image-ids imageTag=v1.0.0 imageTag=v1.0.1 imageTag=v1.0.2

# Delete all untagged images
aws ecr describe-images \
    --repository-name my-app/backend \
    --filter tagStatus=UNTAGGED \
    --query 'imageDetails[*].imageDigest' \
    --output text | \
    xargs -I {} aws ecr batch-delete-image \
        --repository-name my-app/backend \
        --image-ids imageDigest={}

Get Image Details

# Get specific image by tag
aws ecr describe-images \
    --repository-name my-app/backend \
    --image-ids imageTag=v1.2.0

# Get image manifest
aws ecr batch-get-image \
    --repository-name my-app/backend \
    --image-ids imageTag=v1.2.0 \
    --query 'images[0].imageManifest' \
    --output text

Image Scanning

Basic Scanning (On-Push)

# Enable scan on push for repository
aws ecr put-image-scanning-configuration \
    --repository-name my-app/backend \
    --image-scanning-configuration scanOnPush=true

On-Demand Scanning

# Start manual scan
aws ecr start-image-scan \
    --repository-name my-app/backend \
    --image-id imageTag=v1.2.0

# Wait for scan to complete
aws ecr wait image-scan-complete \
    --repository-name my-app/backend \
    --image-id imageTag=v1.2.0

Get Scan Findings

# Get scan results
aws ecr describe-image-scan-findings \
    --repository-name my-app/backend \
    --image-id imageTag=v1.2.0

# Get only critical and high vulnerabilities
aws ecr describe-image-scan-findings \
    --repository-name my-app/backend \
    --image-id imageTag=v1.2.0 \
    --query 'imageScanFindings.findings[?severity==`CRITICAL` || severity==`HIGH`]'

# Get vulnerability counts by severity
aws ecr describe-image-scan-findings \
    --repository-name my-app/backend \
    --image-id imageTag=v1.2.0 \
    --query 'imageScanFindings.findingSeverityCounts'

Enhanced Scanning (Amazon Inspector)

# Enable enhanced scanning at registry level
aws ecr put-registry-scanning-configuration \
    --scan-type ENHANCED \
    --rules '[{"repositoryFilters":[{"filter":"*","filterType":"WILDCARD"}],"scanFrequency":"SCAN_ON_PUSH"}]'

# Enable continuous scanning for specific repositories
aws ecr put-registry-scanning-configuration \
    --scan-type ENHANCED \
    --rules '[{"repositoryFilters":[{"filter":"prod/*","filterType":"WILDCARD"}],"scanFrequency":"CONTINUOUS_SCAN"}]'

# Get registry scanning configuration
aws ecr get-registry-scanning-configuration

Lifecycle Policies

Create Lifecycle Policy

aws ecr put-lifecycle-policy \
    --repository-name my-app/backend \
    --lifecycle-policy-text file://lifecycle-policy.json

lifecycle-policy.json (Keep tagged, expire old untagged):

{
    "rules": [
        {
            "rulePriority": 1,
            "description": "Expire untagged images older than 1 day",
            "selection": {
                "tagStatus": "untagged",
                "countType": "sinceImagePushed",
                "countUnit": "days",
                "countNumber": 1
            },
            "action": {
                "type": "expire"
            }
        },
        {
            "rulePriority": 2,
            "description": "Keep only 10 dev images",
            "selection": {
                "tagStatus": "tagged",
                "tagPrefixList": ["dev-", "feature-"],
                "countType": "imageCountMoreThan",
                "countNumber": 10
            },
            "action": {
                "type": "expire"
            }
        },
        {
            "rulePriority": 3,
            "description": "Keep last 50 production images",
            "selection": {
                "tagStatus": "tagged",
                "tagPrefixList": ["v", "release-"],
                "countType": "imageCountMoreThan",
                "countNumber": 50
            },
            "action": {
                "type": "expire"
            }
        }
    ]
}

Manage Lifecycle Policies

# Get lifecycle policy
aws ecr get-lifecycle-policy --repository-name my-app/backend

# Preview lifecycle policy (dry run)
aws ecr get-lifecycle-policy-preview \
    --repository-name my-app/backend \
    --lifecycle-policy-text file://lifecycle-policy.json

# Delete lifecycle policy
aws ecr delete-lifecycle-policy --repository-name my-app/backend

Repository Policies (Cross-Account Access)

Set Repository Policy

aws ecr set-repository-policy \
    --repository-name my-app/backend \
    --policy-text file://repo-policy.json

repo-policy.json (Cross-account pull access):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowCrossAccountPull",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::111111111111:root",
                    "arn:aws:iam::222222222222:root"
                ]
            },
            "Action": [
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "ecr:BatchCheckLayerAvailability"
            ]
        }
    ]
}

repo-policy.json (Allow specific role):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowECSTaskRole",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111111111111:role/ECSTaskRole"
            },
            "Action": [
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "ecr:BatchCheckLayerAvailability"
            ]
        }
    ]
}

Manage Repository Policies

# Get current policy
aws ecr get-repository-policy --repository-name my-app/backend

# Delete policy
aws ecr delete-repository-policy --repository-name my-app/backend

Cross-Region Replication

Configure Registry Replication

aws ecr put-replication-configuration \
    --replication-configuration file://replication-config.json

replication-config.json (Single region):

{
    "rules": [
        {
            "destinations": [
                {
                    "region": "eu-west-1",
                    "registryId": "123456789012"
                }
            ],
            "repositoryFilters": [
                {
                    "filter": "prod/",
                    "filterType": "PREFIX_MATCH"
                }
            ]
        }
    ]
}

replication-config.json (Multi-region):

{
    "rules": [
        {
            "destinations": [
                {"region": "eu-west-1", "registryId": "123456789012"},
                {"region": "ap-southeast-1", "registryId": "123456789012"},
                {"region": "us-west-2", "registryId": "123456789012"}
            ],
            "repositoryFilters": [
                {
                    "filter": "prod/",
                    "filterType": "PREFIX_MATCH"
                }
            ]
        }
    ]
}

Cross-Account Replication

{
    "rules": [
        {
            "destinations": [
                {
                    "region": "us-east-1",
                    "registryId": "999999999999"
                }
            ]
        }
    ]
}

Get Replication Configuration

aws ecr describe-registry

Pull Through Cache

Cache images from external registries (Docker Hub, GitHub, Quay, etc.).

Create Pull Through Cache Rule

# Docker Hub (public)
aws ecr create-pull-through-cache-rule \
    --ecr-repository-prefix docker-hub \
    --upstream-registry-url registry-1.docker.io

# GitHub Container Registry
aws ecr create-pull-through-cache-rule \
    --ecr-repository-prefix ghcr \
    --upstream-registry-url ghcr.io

# Quay.io
aws ecr create-pull-through-cache-rule \
    --ecr-repository-prefix quay \
    --upstream-registry-url quay.io

# With credentials (for private registries)
aws ecr create-pull-through-cache-rule \
    --ecr-repository-prefix docker-hub \
    --upstream-registry-url registry-1.docker.io \
    --credential-arn arn:aws:secretsmanager:us-east-1:123456789012:secret:dockerhub-creds

Use Cached Images

# Instead of: docker pull nginx:latest
docker pull 123456789012.dkr.ecr.us-east-1.amazonaws.com/docker-hub/library/nginx:latest

# Instead of: docker pull ghcr.io/owner/image:tag
docker pull 123456789012.dkr.ecr.us-east-1.amazonaws.com/ghcr/owner/image:tag

Manage Pull Through Cache

# List rules
aws ecr describe-pull-through-cache-rules

# Delete rule
aws ecr delete-pull-through-cache-rule --ecr-repository-prefix docker-hub

Registry Settings

Registry Policy

# Set registry policy (for replication permissions)
aws ecr put-registry-policy --policy-text file://registry-policy.json

registry-policy.json:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowReplicationFromAccount",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111111111111:root"
            },
            "Action": [
                "ecr:CreateRepository",
                "ecr:ReplicateImage"
            ],
            "Resource": "arn:aws:ecr:us-east-1:123456789012:repository/*"
        }
    ]
}

Get Registry Settings

# Describe registry
aws ecr describe-registry

# Get registry policy
aws ecr get-registry-policy

Useful Queries

# Get all repository URIs
aws ecr describe-repositories \
    --query 'repositories[*].repositoryUri' \
    --output table

# Find largest images
aws ecr describe-images \
    --repository-name my-app/backend \
    --query 'sort_by(imageDetails, &imageSizeInBytes)[-5:].{Tag:imageTags[0],SizeMB:imageSizeInBytes}' \
    --output table

# Count images per repository
for repo in $(aws ecr describe-repositories --query 'repositories[*].repositoryName' --output text); do
    count=$(aws ecr list-images --repository-name $repo --query 'length(imageIds)' --output text)
    echo "$repo: $count images"
done

# Find images with vulnerabilities
aws ecr describe-images \
    --repository-name my-app/backend \
    --query 'imageDetails[?imageScanFindingsSummary.findingSeverityCounts.CRITICAL > `0`].imageTags'

Image Tagging Best Practices

# Multi-tag strategy in CI/CD
TAG_VERSION="v1.2.3"
TAG_SHA=$(git rev-parse --short HEAD)
TAG_DATE=$(date +%Y%m%d)

docker tag app:latest $ECR_URI:$TAG_VERSION
docker tag app:latest $ECR_URI:$TAG_SHA
docker tag app:latest $ECR_URI:$TAG_VERSION-$TAG_SHA
docker tag app:latest $ECR_URI:latest

docker push $ECR_URI --all-tags

Best Practices

Elastic Container Service (ECS)

Clusters

Create Cluster

# Fargate-only cluster with Container Insights
aws ecs create-cluster \
    --cluster-name production \
    --capacity-providers FARGATE FARGATE_SPOT \
    --default-capacity-provider-strategy \
        capacityProvider=FARGATE,weight=1,base=1 \
        capacityProvider=FARGATE_SPOT,weight=4 \
    --settings name=containerInsights,value=enabled

# EC2 cluster with managed scaling
aws ecs create-cluster \
    --cluster-name ec2-cluster \
    --settings name=containerInsights,value=enabled

Manage Clusters

# List clusters
aws ecs list-clusters

# Describe cluster
aws ecs describe-clusters --clusters production

# Delete cluster (must be empty)
aws ecs delete-cluster --cluster production

Capacity Providers

# Create capacity provider for EC2 Auto Scaling group
aws ecs create-capacity-provider \
    --name my-ec2-capacity \
    --auto-scaling-group-provider \
        autoScalingGroupArn=arn:aws:autoscaling:us-east-1:123456789012:autoScalingGroup:abc123:autoScalingGroupName/my-asg,\
managedScaling='{status=ENABLED,targetCapacity=100}',\
managedTerminationProtection=ENABLED

# Update cluster capacity providers
aws ecs put-cluster-capacity-providers \
    --cluster production \
    --capacity-providers FARGATE FARGATE_SPOT my-ec2-capacity \
    --default-capacity-provider-strategy \
        capacityProvider=FARGATE,weight=1

Task Definitions

Register Task Definition

aws ecs register-task-definition \
    --cli-input-json file://task-def.json

task-def.json (Fargate):

{
    "family": "web-api",
    "networkMode": "awsvpc",
    "requiresCompatibilities": ["FARGATE"],
    "cpu": "256",
    "memory": "512",
    "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
    "taskRoleArn": "arn:aws:iam::123456789012:role/ecsTaskRole",
    "containerDefinitions": [
        {
            "name": "app",
            "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:latest",
            "essential": true,
            "portMappings": [
                {"containerPort": 8080, "protocol": "tcp"}
            ],
            "environment": [
                {"name": "NODE_ENV", "value": "production"}
            ],
            "secrets": [
                {
                    "name": "DB_PASSWORD",
                    "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789012:secret:prod/db-password"
                },
                {
                    "name": "API_KEY",
                    "valueFrom": "arn:aws:ssm:us-east-1:123456789012:parameter/prod/api-key"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/ecs/web-api",
                    "awslogs-region": "us-east-1",
                    "awslogs-stream-prefix": "ecs"
                }
            },
            "healthCheck": {
                "command": ["CMD-SHELL", "curl -f http://localhost:8080/health || exit 1"],
                "interval": 30,
                "timeout": 5,
                "retries": 3,
                "startPeriod": 60
            }
        }
    ]
}

Manage Task Definitions

# List task definition families
aws ecs list-task-definition-families

# List task definitions
aws ecs list-task-definitions --family-prefix web-api

# Describe task definition
aws ecs describe-task-definition --task-definition web-api:1

# Deregister (soft delete)
aws ecs deregister-task-definition --task-definition web-api:1

Services

Create Service

# Fargate service with ALB
aws ecs create-service \
    --cluster production \
    --service-name web-api-svc \
    --task-definition web-api:1 \
    --desired-count 2 \
    --launch-type FARGATE \
    --network-configuration \
        "awsvpcConfiguration={subnets=[subnet-1,subnet-2],securityGroups=[sg-1],assignPublicIp=DISABLED}" \
    --load-balancers \
        "targetGroupArn=arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/my-tg/abc123,containerName=app,containerPort=8080" \
    --health-check-grace-period-seconds 60 \
    --enable-execute-command

# With capacity provider strategy
aws ecs create-service \
    --cluster production \
    --service-name web-api-svc \
    --task-definition web-api:1 \
    --desired-count 4 \
    --capacity-provider-strategy \
        capacityProvider=FARGATE,weight=1,base=1 \
        capacityProvider=FARGATE_SPOT,weight=4 \
    --network-configuration \
        "awsvpcConfiguration={subnets=[subnet-1,subnet-2],securityGroups=[sg-1]}"

Update Service

# Deploy new task definition
aws ecs update-service \
    --cluster production \
    --service web-api-svc \
    --task-definition web-api:2

# Force new deployment (pull latest image)
aws ecs update-service \
    --cluster production \
    --service web-api-svc \
    --force-new-deployment

# Scale
aws ecs update-service \
    --cluster production \
    --service web-api-svc \
    --desired-count 5

# Update deployment configuration
aws ecs update-service \
    --cluster production \
    --service web-api-svc \
    --deployment-configuration \
        "minimumHealthyPercent=50,maximumPercent=200,deploymentCircuitBreaker={enable=true,rollback=true}"

List and Describe Services

# List services
aws ecs list-services --cluster production

# Describe service
aws ecs describe-services \
    --cluster production \
    --services web-api-svc

# Get service events
aws ecs describe-services \
    --cluster production \
    --services web-api-svc \
    --query 'services[0].events[:10]'

Delete Service

# Scale to 0 first
aws ecs update-service \
    --cluster production \
    --service web-api-svc \
    --desired-count 0

# Then delete
aws ecs delete-service \
    --cluster production \
    --service web-api-svc

# Or force delete (stops tasks immediately)
aws ecs delete-service \
    --cluster production \
    --service web-api-svc \
    --force

Tasks

Run Standalone Task

aws ecs run-task \
    --cluster production \
    --task-definition migration-task:1 \
    --launch-type FARGATE \
    --network-configuration \
        "awsvpcConfiguration={subnets=[subnet-1],securityGroups=[sg-1],assignPublicIp=ENABLED}" \
    --overrides \
        '{"containerOverrides":[{"name":"app","command":["python","migrate.py"]}]}'

List and Stop Tasks

# List running tasks
aws ecs list-tasks --cluster production

# List tasks for service
aws ecs list-tasks --cluster production --service-name web-api-svc

# Describe tasks
aws ecs describe-tasks \
    --cluster production \
    --tasks arn:aws:ecs:us-east-1:123456789012:task/production/abc123

# Stop task
aws ecs stop-task \
    --cluster production \
    --task arn:aws:ecs:us-east-1:123456789012:task/production/abc123 \
    --reason "Debugging"

ECS Exec (Container Debugging)

Enable ECS Exec

Service must be created with --enable-execute-command:

aws ecs update-service \
    --cluster production \
    --service web-api-svc \
    --enable-execute-command

Execute Commands

# Interactive shell
aws ecs execute-command \
    --cluster production \
    --task arn:aws:ecs:us-east-1:123456789012:task/production/abc123 \
    --container app \
    --interactive \
    --command "/bin/sh"

# Run single command
aws ecs execute-command \
    --cluster production \
    --task arn:aws:ecs:us-east-1:123456789012:task/production/abc123 \
    --container app \
    --interactive \
    --command "cat /app/config.json"

Check ECS Exec Readiness

# Install amazon-ecs-exec-checker
aws ecs describe-tasks \
    --cluster production \
    --tasks abc123 \
    --query 'tasks[0].containers[0].managedAgents'

Service Auto Scaling

Register Scalable Target

aws application-autoscaling register-scalable-target \
    --service-namespace ecs \
    --resource-id service/production/web-api-svc \
    --scalable-dimension ecs:service:DesiredCount \
    --min-capacity 2 \
    --max-capacity 10

Target Tracking Policy

aws application-autoscaling put-scaling-policy \
    --service-namespace ecs \
    --resource-id service/production/web-api-svc \
    --scalable-dimension ecs:service:DesiredCount \
    --policy-name cpu-scaling \
    --policy-type TargetTrackingScaling \
    --target-tracking-scaling-policy-configuration \
        "TargetValue=70.0,PredefinedMetricSpecification={PredefinedMetricType=ECSServiceAverageCPUUtilization},ScaleOutCooldown=60,ScaleInCooldown=60"

Step Scaling Policy

aws application-autoscaling put-scaling-policy \
    --service-namespace ecs \
    --resource-id service/production/web-api-svc \
    --scalable-dimension ecs:service:DesiredCount \
    --policy-name memory-step-scaling \
    --policy-type StepScaling \
    --step-scaling-policy-configuration \
        "AdjustmentType=ChangeInCapacity,StepAdjustments=[{MetricIntervalLowerBound=0,ScalingAdjustment=2}],Cooldown=60"

Blue-Green Deployments (with CodeDeploy)

Create Deployment Group

aws deploy create-deployment-group \
    --application-name my-ecs-app \
    --deployment-group-name my-dg \
    --service-role-arn arn:aws:iam::123456789012:role/CodeDeployRole \
    --deployment-config-name CodeDeployDefault.ECSAllAtOnce \
    --ecs-services clusterName=production,serviceName=web-api-svc \
    --load-balancer-info \
        "targetGroupPairInfoList=[{targetGroups=[{name=blue-tg},{name=green-tg}],prodTrafficRoute={listenerArns=[arn:aws:elasticloadbalancing:...]}}]" \
    --blue-green-deployment-configuration \
        "terminateBlueInstancesOnDeploymentSuccess={action=TERMINATE,terminationWaitTimeInMinutes=5},deploymentReadyOption={actionOnTimeout=CONTINUE_DEPLOYMENT}"

Useful Queries

# Get task private IPs
aws ecs describe-tasks \
    --cluster production \
    --tasks $(aws ecs list-tasks --cluster production --service-name web-api-svc --query 'taskArns[*]' --output text) \
    --query 'tasks[*].attachments[*].details[?name==`privateIPv4Address`].value' \
    --output text

# Find stopped tasks with reason
aws ecs list-tasks --cluster production --desired-status STOPPED
aws ecs describe-tasks \
    --cluster production \
    --tasks TASK_ARN \
    --query 'tasks[*].{id:taskArn,reason:stoppedReason,code:stopCode}'

# Get container instance ARNs (EC2 launch type)
aws ecs list-container-instances --cluster ec2-cluster

Best Practices

Elastic Kubernetes Service (EKS)

Cluster Management

Create Cluster

# Create cluster with Kubernetes 1.32
aws eks create-cluster \
    --name production-cluster \
    --version 1.32 \
    --role-arn arn:aws:iam::123456789012:role/EKSClusterRole \
    --resources-vpc-config \
        subnetIds=subnet-1,subnet-2,subnet-3,\
securityGroupIds=sg-1,\
endpointPublicAccess=false,\
endpointPrivateAccess=true \
    --kubernetes-network-config serviceIpv4Cidr=10.100.0.0/16 \
    --logging '{"clusterLogging":[{"types":["api","audit","authenticator","controllerManager","scheduler"],"enabled":true}]}'

# Wait for cluster to be active
aws eks wait cluster-active --name production-cluster

Describe and List Clusters

# List clusters
aws eks list-clusters

# Describe cluster
aws eks describe-cluster --name production-cluster

# Get cluster endpoint
aws eks describe-cluster \
    --name production-cluster \
    --query 'cluster.endpoint' \
    --output text

# Get cluster OIDC issuer
aws eks describe-cluster \
    --name production-cluster \
    --query 'cluster.identity.oidc.issuer' \
    --output text

Update Cluster

# Update Kubernetes version
aws eks update-cluster-version \
    --name production-cluster \
    --kubernetes-version 1.33

# Update cluster config (logging)
aws eks update-cluster-config \
    --name production-cluster \
    --logging '{"clusterLogging":[{"types":["api","audit"],"enabled":true}]}'

# Update endpoint access
aws eks update-cluster-config \
    --name production-cluster \
    --resources-vpc-config \
        endpointPublicAccess=true,\
endpointPrivateAccess=true,\
publicAccessCidrs=["203.0.113.0/24"]

Managed Node Groups

Create Node Group

# On-Demand node group
aws eks create-nodegroup \
    --cluster-name production-cluster \
    --nodegroup-name workers-on-demand \
    --node-role arn:aws:iam::123456789012:role/EKSNodeRole \
    --subnets subnet-1 subnet-2 subnet-3 \
    --instance-types m6g.large m6g.xlarge \
    --capacity-type ON_DEMAND \
    --scaling-config minSize=2,maxSize=10,desiredSize=3 \
    --disk-size 50 \
    --labels environment=production,tier=workers

# Spot node group (cost optimization)
aws eks create-nodegroup \
    --cluster-name production-cluster \
    --nodegroup-name workers-spot \
    --node-role arn:aws:iam::123456789012:role/EKSNodeRole \
    --subnets subnet-1 subnet-2 subnet-3 \
    --instance-types m6g.large m6g.xlarge c6g.large c6g.xlarge \
    --capacity-type SPOT \
    --scaling-config minSize=0,maxSize=20,desiredSize=5

# With launch template
aws eks create-nodegroup \
    --cluster-name production-cluster \
    --nodegroup-name custom-workers \
    --node-role arn:aws:iam::123456789012:role/EKSNodeRole \
    --subnets subnet-1 subnet-2 \
    --launch-template name=eks-node-template,version=1 \
    --scaling-config minSize=1,maxSize=5,desiredSize=2

Manage Node Groups

# List node groups
aws eks list-nodegroups --cluster-name production-cluster

# Describe node group
aws eks describe-nodegroup \
    --cluster-name production-cluster \
    --nodegroup-name workers-on-demand

# Update scaling
aws eks update-nodegroup-config \
    --cluster-name production-cluster \
    --nodegroup-name workers-on-demand \
    --scaling-config minSize=3,maxSize=15,desiredSize=5

# Update node group version (rolling update)
aws eks update-nodegroup-version \
    --cluster-name production-cluster \
    --nodegroup-name workers-on-demand \
    --kubernetes-version 1.32

# Delete node group
aws eks delete-nodegroup \
    --cluster-name production-cluster \
    --nodegroup-name workers-spot

Fargate Profiles

Create Fargate Profile

aws eks create-fargate-profile \
    --cluster-name production-cluster \
    --fargate-profile-name app-profile \
    --pod-execution-role-arn arn:aws:iam::123456789012:role/EKSFargatePodRole \
    --subnets subnet-private-1 subnet-private-2 \
    --selectors \
        namespace=production,labels={compute=fargate} \
        namespace=kube-system,labels={k8s-app=kube-dns}

Manage Fargate Profiles

# List profiles
aws eks list-fargate-profiles --cluster-name production-cluster

# Describe profile
aws eks describe-fargate-profile \
    --cluster-name production-cluster \
    --fargate-profile-name app-profile

# Delete profile
aws eks delete-fargate-profile \
    --cluster-name production-cluster \
    --fargate-profile-name app-profile

Add-ons

Manage Add-ons

# List available add-ons
aws eks describe-addon-versions --kubernetes-version 1.32

# List installed add-ons
aws eks list-addons --cluster-name production-cluster

# Create add-on
aws eks create-addon \
    --cluster-name production-cluster \
    --addon-name vpc-cni \
    --addon-version v1.16.0-eksbuild.1 \
    --service-account-role-arn arn:aws:iam::123456789012:role/VPCCNIRole

aws eks create-addon \
    --cluster-name production-cluster \
    --addon-name coredns

aws eks create-addon \
    --cluster-name production-cluster \
    --addon-name kube-proxy

aws eks create-addon \
    --cluster-name production-cluster \
    --addon-name aws-ebs-csi-driver \
    --service-account-role-arn arn:aws:iam::123456789012:role/EBSCSIRole

# Update add-on
aws eks update-addon \
    --cluster-name production-cluster \
    --addon-name vpc-cni \
    --addon-version v1.17.0-eksbuild.1

# Delete add-on
aws eks delete-addon \
    --cluster-name production-cluster \
    --addon-name aws-ebs-csi-driver

Kubeconfig Setup

Basic Configuration

# Update kubeconfig (uses current AWS credentials)
aws eks update-kubeconfig \
    --name production-cluster \
    --region us-east-1

# With custom alias
aws eks update-kubeconfig \
    --name production-cluster \
    --alias prod-cluster

# With role assumption (cross-account or different role)
aws eks update-kubeconfig \
    --name production-cluster \
    --role-arn arn:aws:iam::123456789012:role/EKSAdminRole

# Dry run (show kubeconfig without writing)
aws eks update-kubeconfig \
    --name production-cluster \
    --dry-run

Verify Access

# Test connection
kubectl get nodes
kubectl get pods -A

# Get cluster info
kubectl cluster-info

IAM Roles for Service Accounts (IRSA)

IRSA allows Kubernetes pods to assume IAM roles without using node-level credentials.

Step 1: Create OIDC Provider

# Get OIDC issuer URL
OIDC_URL=$(aws eks describe-cluster \
    --name production-cluster \
    --query 'cluster.identity.oidc.issuer' \
    --output text)

# Extract OIDC ID
OIDC_ID=$(echo $OIDC_URL | sed 's|https://||')

# Create OIDC provider (if not exists)
aws iam create-open-id-connect-provider \
    --url $OIDC_URL \
    --client-id-list sts.amazonaws.com \
    --thumbprint-list $(openssl s_client -servername oidc.eks.us-east-1.amazonaws.com \
        -connect oidc.eks.us-east-1.amazonaws.com:443 2>/dev/null \
        | openssl x509 -fingerprint -sha1 -noout \
        | sed 's/://g' | cut -d= -f2 | tr '[:upper:]' '[:lower:]')

Step 2: Create IAM Role with Trust Policy

# Create trust policy file
cat > trust-policy.json << EOF
{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Principal": {
            "Federated": "arn:aws:iam::123456789012:oidc-provider/${OIDC_ID}"
        },
        "Action": "sts:AssumeRoleWithWebIdentity",
        "Condition": {
            "StringEquals": {
                "${OIDC_ID}:sub": "system:serviceaccount:default:my-service-account",
                "${OIDC_ID}:aud": "sts.amazonaws.com"
            }
        }
    }]
}
EOF

# Create role
aws iam create-role \
    --role-name MyPodRole \
    --assume-role-policy-document file://trust-policy.json

# Attach policy
aws iam attach-role-policy \
    --role-name MyPodRole \
    --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

Step 3: Annotate Service Account

# Create service account with annotation (kubectl)
kubectl create serviceaccount my-service-account
kubectl annotate serviceaccount my-service-account \
    eks.amazonaws.com/role-arn=arn:aws:iam::123456789012:role/MyPodRole

Accessing Private EKS Clusters

Method 1: SSM Port Forwarding

# Get EKS API endpoint (without https://)
EKS_ENDPOINT=$(aws eks describe-cluster \
    --name production-cluster \
    --query 'cluster.endpoint' \
    --output text | sed 's|https://||')

# Start port forwarding through bastion
aws ssm start-session \
    --target i-bastion-instance-id \
    --document-name AWS-StartPortForwardingSessionToRemoteHost \
    --parameters "{\"host\":[\"$EKS_ENDPOINT\"],\"portNumber\":[\"443\"],\"localPortNumber\":[\"9443\"]}"

# In another terminal, update kubeconfig
aws eks update-kubeconfig --name production-cluster

# Modify kubeconfig to use localhost:9443
kubectl config set-cluster arn:aws:eks:us-east-1:123456789012:cluster/production-cluster \
    --server=https://localhost:9443

# Disable certificate verification (for port forwarding)
kubectl config set-cluster arn:aws:eks:us-east-1:123456789012:cluster/production-cluster \
    --insecure-skip-tls-verify=true

# Test access
kubectl get nodes

Method 2: SOCKS Proxy via SSH over SSM

Configure SSH (~/.ssh/config):

Host i-* mi-*
    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"
    User ec2-user
    StrictHostKeyChecking no

Start SOCKS proxy:

# Start SSH SOCKS proxy through SSM bastion
ssh -N -D 127.0.0.1:1080 i-bastion-instance-id &

# Configure kubectl to use proxy
export HTTPS_PROXY=socks5h://127.0.0.1:1080

# Update kubeconfig normally
aws eks update-kubeconfig --name production-cluster

# Use kubectl (traffic goes through SOCKS proxy)
kubectl get nodes

Prerequisites for Private Cluster Access

# Required VPC endpoints
aws ec2 create-vpc-endpoint \
    --vpc-id vpc-xxxxx \
    --service-name com.amazonaws.us-east-1.ssm \
    --vpc-endpoint-type Interface \
    --subnet-ids subnet-xxxxx \
    --security-group-ids sg-xxxxx

aws ec2 create-vpc-endpoint \
    --vpc-id vpc-xxxxx \
    --service-name com.amazonaws.us-east-1.ssmmessages \
    --vpc-endpoint-type Interface \
    --subnet-ids subnet-xxxxx \
    --security-group-ids sg-xxxxx

aws ec2 create-vpc-endpoint \
    --vpc-id vpc-xxxxx \
    --service-name com.amazonaws.us-east-1.ec2messages \
    --vpc-endpoint-type Interface \
    --subnet-ids subnet-xxxxx \
    --security-group-ids sg-xxxxx

Access Entries (EKS Access Management)

Create Access Entry

# Grant cluster access to IAM principal
aws eks create-access-entry \
    --cluster-name production-cluster \
    --principal-arn arn:aws:iam::123456789012:role/DeveloperRole \
    --type STANDARD

# Associate access policy
aws eks associate-access-policy \
    --cluster-name production-cluster \
    --principal-arn arn:aws:iam::123456789012:role/DeveloperRole \
    --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSViewPolicy \
    --access-scope type=namespace,namespaces=default

# Admin access
aws eks associate-access-policy \
    --cluster-name production-cluster \
    --principal-arn arn:aws:iam::123456789012:role/AdminRole \
    --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy \
    --access-scope type=cluster

Manage Access Entries

# List access entries
aws eks list-access-entries --cluster-name production-cluster

# Describe access entry
aws eks describe-access-entry \
    --cluster-name production-cluster \
    --principal-arn arn:aws:iam::123456789012:role/DeveloperRole

# Delete access entry
aws eks delete-access-entry \
    --cluster-name production-cluster \
    --principal-arn arn:aws:iam::123456789012:role/DeveloperRole

Useful Queries

# Get node group status
aws eks describe-nodegroup \
    --cluster-name production-cluster \
    --nodegroup-name workers-on-demand \
    --query 'nodegroup.status'

# List all node groups with status
aws eks list-nodegroups --cluster-name production-cluster \
    --query 'nodegroups[]' --output text | \
    xargs -I {} aws eks describe-nodegroup \
        --cluster-name production-cluster \
        --nodegroup-name {} \
        --query '{name:nodegroup.nodegroupName,status:nodegroup.status}'

# Get cluster CA certificate
aws eks describe-cluster \
    --name production-cluster \
    --query 'cluster.certificateAuthority.data' \
    --output text | base64 -d

# Check cluster health
aws eks describe-cluster \
    --name production-cluster \
    --query 'cluster.{status:status,version:version,endpoint:endpoint}'

Waiters

# Wait for cluster creation
aws eks wait cluster-active --name production-cluster

# Wait for node group creation
aws eks wait nodegroup-active \
    --cluster-name production-cluster \
    --nodegroup-name workers-on-demand

# Wait for node group deletion
aws eks wait nodegroup-deleted \
    --cluster-name production-cluster \
    --nodegroup-name old-workers

Best Practices