Using Cloud Cli

Cloud CLI Patterns

Credentials are pre-configured. Use --help or Context7 for syntax.

BigQuery

# Always estimate cost first
bq query --dry_run --use_legacy_sql=false 'SELECT ...'

# Run query
bq query --use_legacy_sql=false --format=json 'SELECT ...'

# List tables
bq ls project:dataset

# Get table schema
bq show --schema --format=json project:dataset.table

Cost awareness: Charged per bytes scanned. Use --dry_run, partition tables, specify columns.

GCP (gcloud)

# List resources
gcloud compute instances list --format=json

# Describe resource
gcloud compute instances describe INSTANCE --zone=ZONE --format=json

# Create with explicit project
gcloud compute instances create NAME --project=PROJECT --zone=ZONE

# Use --quiet for automation
gcloud compute instances delete NAME --quiet

AWS

# List resources
aws ec2 describe-instances --output json

# With JMESPath filtering
aws ec2 describe-instances --query 'Reservations[].Instances[].InstanceId' --output text

# Explicit region
aws s3 ls s3://bucket --region us-west-2

# Dry run where available
aws ec2 run-instances --dry-run ...

References

• GCP.md - GCP service patterns and common commands
• AWS.md - AWS service patterns and common commands
• scripts/ - Helper scripts for common operations

---

Gotchas

• `gcloud auth login` and `gcloud auth application-default login` are DIFFERENT credentials — a script working for the user can 401 in CI because CI only has the latter.
• AWS CLI v1 vs v2 have different config formats — both may be installed; aws --version reveals which is the default aws binary.
• BigQuery cost estimation requires `--dry_run` — without it, you can run a $100 query thinking it's free. Always estimate first.
• Precedence: AWS_PROFILE vs --profile flag vs AWS_ACCESS_KEY_ID direct env — three-way precedence is order-dependent and rarely documented.
• gcloud impersonation chains via `--impersonate-service-account` need IAM token-creator on EACH hop — missing one hop returns 403 with vague "permission denied".
• `aws sts get-caller-identity` is the cheapest way to verify which credential you're actually using — always check first.

AWS Patterns

S3

# List buckets
aws s3 ls

# List objects
aws s3 ls s3://bucket/prefix/

# Copy
aws s3 cp local_file s3://bucket/path/
aws s3 cp s3://bucket/path/file .

# Sync
aws s3 sync local_dir/ s3://bucket/path/
aws s3 sync s3://bucket/path/ local_dir/

# Remove
aws s3 rm s3://bucket/path/file
aws s3 rm s3://bucket/path/ --recursive

EC2

# List instances
aws ec2 describe-instances \
  --query 'Reservations[].Instances[].[InstanceId,State.Name,PublicIpAddress]' \
  --output table

# Start/Stop
aws ec2 start-instances --instance-ids i-xxxxx
aws ec2 stop-instances --instance-ids i-xxxxx

# Create instance
aws ec2 run-instances \
  --image-id ami-xxxxx \
  --instance-type t3.micro \
  --key-name my-key \
  --security-group-ids sg-xxxxx \
  --subnet-id subnet-xxxxx

# Dry run
aws ec2 run-instances --dry-run ...

Lambda

# List functions
aws lambda list-functions --query 'Functions[].FunctionName' --output table

# Invoke
aws lambda invoke --function-name FUNC output.json
cat output.json

# Update code
aws lambda update-function-code \
  --function-name FUNC \
  --zip-file fileb://function.zip

# View logs
aws logs filter-log-events \
  --log-group-name /aws/lambda/FUNC \
  --start-time $(date -d '1 hour ago' +%s000)

ECS

# List clusters
aws ecs list-clusters

# List services
aws ecs list-services --cluster CLUSTER

# Update service (force deploy)
aws ecs update-service \
  --cluster CLUSTER \
  --service SERVICE \
  --force-new-deployment

# View tasks
aws ecs list-tasks --cluster CLUSTER --service-name SERVICE

IAM

# List roles
aws iam list-roles --query 'Roles[].RoleName' --output table

# Get user
aws sts get-caller-identity

# List attached policies
aws iam list-attached-role-policies --role-name ROLE

Common Flags

JMESPath Examples

# Get instance IDs
--query 'Reservations[].Instances[].InstanceId'

# Filter by tag
--query 'Reservations[].Instances[?Tags[?Key==`Name` && Value==`web`]]'

# Multiple fields
--query 'Reservations[].Instances[].[InstanceId,State.Name]'

# First result
--query 'Reservations[0].Instances[0].InstanceId'

GCP Patterns

BigQuery

# Cost estimation
bq query --dry_run --use_legacy_sql=false \
  'SELECT * FROM `project.dataset.table` WHERE date > "2024-01-01"'

# Standard query
bq query --use_legacy_sql=false --format=json \
  'SELECT col1, col2 FROM `project.dataset.table` LIMIT 100'

# Save to table
bq query --use_legacy_sql=false \
  --destination_table=project:dataset.result \
  'SELECT ...'

# Export to GCS
bq extract --destination_format=NEWLINE_DELIMITED_JSON \
  project:dataset.table gs://bucket/path/*.json

Cost Optimization

• Always --dry_run first to estimate bytes scanned
• Avoid SELECT * - specify columns
• Use partitioned tables with partition filters
• Use clustered tables for frequent filter columns
• LIMIT doesn't reduce scan cost (use in WHERE)

Cloud Storage

# List
gsutil ls gs://bucket/path/

# Copy
gsutil cp local_file gs://bucket/path/
gsutil cp -r local_dir/ gs://bucket/path/

# Sync (like rsync)
gsutil rsync -r local_dir/ gs://bucket/path/

# Download
gsutil cp gs://bucket/path/file .
gsutil cp -r gs://bucket/path/ local_dir/

Compute Engine

# List instances
gcloud compute instances list --format="table(name,zone,status,networkInterfaces[0].accessConfigs[0].natIP)"

# Create
gcloud compute instances create NAME \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --image-family=debian-12 \
  --image-project=debian-cloud

# SSH
gcloud compute ssh INSTANCE --zone=ZONE

# Stop/Start
gcloud compute instances stop INSTANCE --zone=ZONE
gcloud compute instances start INSTANCE --zone=ZONE

Cloud Run

# Deploy
gcloud run deploy SERVICE \
  --image=gcr.io/PROJECT/IMAGE \
  --region=us-central1 \
  --allow-unauthenticated

# List services
gcloud run services list --format=json

# View logs
gcloud run services logs read SERVICE --region=REGION

IAM

# List service accounts
gcloud iam service-accounts list --format=json

# Create service account
gcloud iam service-accounts create NAME --display-name="Description"

# Add role binding
gcloud projects add-iam-policy-binding PROJECT \
  --member="serviceAccount:SA@PROJECT.iam.gserviceaccount.com" \
  --role="roles/storage.objectViewer"

Common Flags