Cloud System Administrator

Cloud System Administrator

When to Use

• Process access requests — IAM roles, group membership, break-glass (per policy)
• Execute credential rotation — access keys, service account keys, certificates
• Run patch and maintenance windows for VMs and managed instance groups
• Operate backups — snapshots, retention checks, test restores
• Triage alerts — CPU, disk, quota, health checks; escalate per runbook
• Handle incidents — connectivity, permission denied, throttling, regional outages
• Request quota/limit increases and track vendor cases
• Perform hygiene — orphaned volumes, old snapshots, untagged resources, idle compute
• Maintain runbooks and execute change tickets for routine infra changes
• Support audit — access reviews, log exports, evidence for reviewers

When NOT to Use

• Design landing zones, migration, or reference architecture → cloud-architect, enterprise-cloud-architect
• Build new VPC, RDS, or serverless stacks from requirements → cloud-engineer
• Terraform module libraries and platform IaC → infrastructure-engineer
• Pipeline failures and GitOps sync → devops
• K8s cluster upgrades and Helm → cluster-deployment-engineer
• Define SEV levels, paging policy, postmortem program → incident-management-engineer
• Security program, IdP design, SIEM → information-security-engineer
• Entitlement design, access review campaigns, federation architecture → iam-specialist
• Application bug fixes → senior-software-engineer
• GL reconciliation of cloud invoices → compute-accounting-manager

Related skills

Core Workflows

1. Scope and operations model

Responsibilities, escalation, change control.

See `references/cloud_sysadmin_scope.md`.

2. IAM and access operations

Requests, reviews, rotation.

See `references/iam_access_operations.md`.

3. Compute and OS maintenance

Patching, instances, disks.

See `references/compute_os_maintenance.md`.

4. Monitoring and incident response

Alerts, triage, runbooks.

See `references/monitoring_incident_response.md`.

5. Backup and restore operations

Snapshots, drills, recovery steps.

See `references/backup_restore_operations.md`.

6. Operational hygiene

Quotas, certs, cleanup, DNS ops.

See `references/operational_hygiene.md`.

Outputs

• Completed change — ticket ID, steps, rollback noted
• Access grant record — who, what role, expiry if temporary
• Incident timeline — detection, actions, resolution, follow-ups
• Restore report — RPO achieved, data validated
• Hygiene report — resources removed, savings estimate
• Runbook update — gaps found during incident

Principles

• Least privilege — grant minimum role; time-bound elevation
• Change control — no prod change without ticket and rollback
• Automate repeat work — scripts over manual clicks where safe
• Document actions — audit trail in ticket and logs
• Escalate architecture — recurring failures may need cloud-engineer or cloud-architect

Backup and restore operations

Table of contents

1. Backup verification 2. Restore procedures 3. Restore drills 4. Ransomware considerations

Backup verification

Daily/weekly checks:

• Backup jobs success rate (RDS, VM snapshots, vault)
• Retention meets policy; no silent policy drift
• Cross-region copy status if required
• Encryption on snapshots and vaults

Alert on failed backup jobs as high priority.

Restore procedures

Before restore in prod:

1. Incident ticket with approver 2. Identify target time (PITR timestamp or snapshot ID) 3. Isolate — restore to new instance/volume first when possible 4. Validate — app team confirms data integrity 5. Cutover — DNS/connection string change per runbook 6. Retain old resource until stable; then decommission

Document RPO achieved vs target.

Restore drills

Quarterly (per tier):

• Restore random snapshot to isolated environment
• Time the procedure; update runbook gaps
• Report to leadership + compliance-engineer if audit requires

Ransomware considerations

• Immutable backups where policy requires
• Do not restore prod without security clearance if encryption event suspected
• Preserve logs and snapshots for investigation — security leads

Cloud sysadmin scope

Table of contents

1. Responsibility boundary 2. Change control 3. Escalation 4. Environments

Responsibility boundary

cloud-engineer builds; cloud-system-administrator runs.

Change control

Standard change types:

Record: what, when, who, rollback, verification.

Escalation

Environments

• Prod — change window, dual control for sensitive actions
• Non-prod — faster path; still no shared admin passwords
• Sandbox — auto-cleanup policies; no prod data

Follow enterprise standards from enterprise-cloud-architect when applicable.

Compute and OS maintenance

Table of contents

1. Patching 2. Instance operations 3. Storage operations 4. Managed services ops

Patching

VM and instance group patching:

1. Scan — missing critical patches report 2. Schedule — maintenance window; notify app owners 3. Snapshot — before major patch waves on prod 4. Apply — rolling update for ASG/MIG/VMSS; drain where required 5. Verify — health checks green; rollback AMI if failed

Document reboot required kernel patches separately.

Instance operations

Common tasks:

• Resize instance type (stop/start or live resize if supported)
• Replace failed instance from launch template
• Attach/detach EBS/disk — verify AZ match
• User-data debug — only in non-prod; no secret exposure in console

Storage operations

• Expand volumes — grow partition/OS after cloud resize
• Delete unattached volumes per hygiene policy (snapshot first if uncertain)
• S3 lifecycle — verify transitions; no accidental prod bucket delete

Managed services ops

Deep RDS performance tuning → DBA/app team; connectivity → network runbook.

IAM access operations

Table of contents

1. Access request workflow 2. Role assignment 3. Reviews and offboarding 4. Credential rotation

Access request workflow

1. Validate ticket — requester, manager approval, justification, duration 2. Map need to predefined role — avoid inline custom policies in prod 3. Apply in correct account/subscription — verify environment tag 4. Confirm MFA and SSO path for human access 5. Notify requester; log in access register

Deny if: no approval, excessive permissions, prod break-glass without incident.

Role assignment

Prefer:

• Group-based roles over user-attached policies
• Assumed roles with session duration limits
• Workload roles for apps — no long-lived keys

Cross-account: verify trust policy and external ID match standard.

Architecture changes to roles → cloud-engineer with security review.

Reviews and offboarding

Quarterly access review:

• List users/roles per account
• Owners attest or revoke
• Remove stale SSO assignments and cloud bindings same day as HR term

Emergency revoke: disable SSO session + detach policies + rotate shared secrets if any.

Credential rotation

Never paste secrets in tickets or chat — use vault references.

Monitoring and incident response

Table of contents

1. Alert triage 2. Runbook execution 3. Incident timeline 4. Vendor outages

Alert triage

First 15 minutes:

1. Acknowledge page; open incident channel/ticket 2. Scope — one resource, one region, one account, or provider-wide? 3. Impact — prod vs non-prod; customer-facing? 4. Recent changes — deploy, IAM, network, maintenance? 5. Dashboards — CPU, errors, saturation, quota metrics

Severity mapping follows org SEV definitions — incident-management-engineer.

Runbook execution

Runbook sections:

• Symptoms — alert names, typical causes
• Diagnosis — commands/console paths (read-only first)
• Mitigation — rollback, failover, scale up, disable feature flag (app team)
• Escalation — when to page cloud-engineer or vendor support
• Verification — all-clear checks

Update runbook after every significant incident.

Incident timeline

Log entries:

• Detection time (alert)
• Actions with timestamps
• Hypothesis changes
• Resolution and customer impact end
• Follow-up tickets (root cause fix)

Blameless postmortem if SEV threshold met — process from incident program.

Vendor outages

Cloud status page check:

• Confirm regional issue vs local misconfig
• Enable communication template if customer impact
• Avoid destructive changes during provider incident unless runbook says failover
• Open support case if single-tenant anomaly

Operational hygiene

Table of contents

1. Cost cleanup 2. Quotas and limits 3. DNS and certificates 4. Tagging compliance

Cost cleanup

Monthly hygiene (non-prod aggressive, prod cautious):

Report estimated savings; finance visibility via tags — compute-accounting-manager for GL not cleanup execution.

Quotas and limits

• Monitor quota utilization dashboards
• Open provider case before hard block at 80% on critical quotas
• Document approved increase and valid until date
• After increase, verify workload success

DNS and certificates

• Track cert expiry 30/14/7 days alerts
• Renew via automation first; manual DNS validation runbook as backup
• Route53/Cloud DNS record changes via change ticket
• Validate propagation after TTL

Tagging compliance

• Report untagged resources to owners
• Enforce via policy where org mandates — remediation SLA
• Required tags: owner, environment, cost center (per enterprise-cloud-architect or local standard)

Do not delete untagged prod resources without owner escalation path.