Incident Responder

Incident Responder (CSIRT)

When to Use

• Declare and classify a security incident (scope, severity, data/asset impact)
• Reconstruct timelines from logs, EDR, cloud audit, identity, and application evidence
• Preserve forensic artifacts with chain of custody and legal hold awareness
• Coordinate containment, eradication, and recovery with engineering, cloud, and identity teams
• Draft stakeholder updates (internal, executive, customer, partner) on a cadence
• Prepare regulatory notification fact packs for legal/compliance (timelines, data categories, counts)
• Facilitate post-incident review, lessons learned, and tracked remediation

When NOT to Use

• Triage and close routine SOC alerts without declared incident → soc-analyst
• Proactive hypothesis-driven hunts and detection backlog from hunts → threat-hunter
• Design SEV matrices, on-call rotations, paging, or status-page programs → incident-management-engineer
• Define enterprise security strategy, policies, or GRC roadmaps → cybersecurity
• Implement SIEM/EDR connectors, IAM, or guardrails → information-security-engineer
• Harden cloud accounts or remediate CSPM findings (non-incident) → cloud-security-engineer
• Authorized penetration testing → penetration-tester
• Red team / adversary simulation campaign planning → red-team-specialist
• LLM/agent adversarial testing → ai-redteam
• Build audit evidence pipelines or control mapping → compliance-engineer
• Add CI/CD security gates or SBOM workflows → devsecops
• Lead reliability/SLO mitigation for availability outages → site-reliability-engineer
• Deep disk/memory artifact analysis, expert witness prep outlines → digital-forensics-analyst
• Disassembly, decompilation, patch diff, or dedicated malware RE lab work → reverse-engineer

Related skills

Core Workflows

1. Intake and declaration

1. Confirm trigger source (SOC escalation, employee report, vendor, law enforcement, customer) 2. Assign incident ID; open record and comms channel 3. Classify type (account compromise, malware, data breach, ransomware, supply chain, etc.) 4. Set severity from impact and urgency; document rationale 5. Notify incident commander, legal, and comms per severity matrix

See `references/incident_declaration_and_severity.md`.

2. Scope and timeline

1. Identify affected users, systems, accounts, regions, and data classes 2. Establish first known compromise and detection timestamps (UTC) 3. Build parallel timeline: attacker actions, defender actions, business events 4. List evidence sources still available; flag gaps and retention risks 5. Update scope statement when new facts emerge (version each change)

See `references/timeline_and_evidence_handling.md`.

3. Containment → eradication → recovery

1. Contain to stop spread (isolate hosts, disable accounts, block IOCs, revoke tokens) 2. Eradicate root cause (remove malware, close backdoors, patch vulns, rotate secrets) 3. Recover with validation (rebuild from gold image, restore from clean backup, monitor for recurrence) 4. Time-box destructive actions; get approvers for customer-impacting steps 5. Hand off long-term hardening to owning engineering/security skills

See `references/containment_eradication_recovery.md`.

4. Stakeholder and regulatory communications

1. Set update cadence by severity (e.g., 30–60 min for SEV1 security) 2. Separate facts from hypotheses in every update 3. Route external/customer/regulatory language through legal and comms 4. Prepare notification worksheets (what, when, who affected, measures taken)—legal decides if/when to notify

See `references/stakeholder_and_regulatory_comms.md`.

5. Post-incident review

1. Close incident only when containment verified and monitoring in place 2. Run blameless review within agreed SLA (e.g., 5 business days for major incidents) 3. Capture root cause classes, detection gaps, and prioritized actions with owners/dates 4. Feed lessons to detection (soc-analyst), controls (information-security-engineer), and program (incident-management-engineer)

See `references/post_incident_review.md`.

When to load references

• Role boundaries and handoffs → references/incident_responder_scope.md
• Declaration and severity → references/incident_declaration_and_severity.md
• Timeline and evidence → references/timeline_and_evidence_handling.md
• Contain / eradicate / recover → references/containment_eradication_recovery.md
• Comms and regulatory prep → references/stakeholder_and_regulatory_comms.md
• Post-incident review → references/post_incident_review.md

Outputs

• Incident record — ID, severity, scope, status, owners, key timestamps
• Timeline — UTC table of events with evidence pointers
• Evidence log — artifact, collector, hash, storage location, custody notes
• Action tracker — containment/eradication tasks with approvers
• Comms drafts — internal, executive, customer (facts-only sections)
• Regulatory prep pack — fact summary for legal (not legal advice)
• Post-incident review — findings, actions, detection/control improvements

Containment, eradication, and recovery

Table of contents

1. Phase overview 2. Containment 3. Eradication 4. Recovery 5. Approval gates 6. Coordination by incident type

Phase overview

Follow contain → eradicate → recover (NIST-style). Parallelize only when actions do not destroy evidence or increase blast radius.

Containment

Identity

• Disable compromised accounts; revoke refresh tokens and API keys
• Force password reset or step-up re-auth for affected populations (scoped, not org-wide unless approved)
• Review IdP rules, forwarding, OAuth grants, and admin roles

Endpoint / server

• Network isolate host (EDR network containment preferred over abrupt power-off when forensics needed)
• Block IOCs at proxy, firewall, EDR policy—document TTL and owner

Cloud

• Suspend compromised keys; rotate access keys; restrict security group rules
• Snapshot volumes before terminate; preserve audit logs (cloud-security-engineer)

Application

• Disable compromised integrations, webhooks, or service accounts
• Feature-flag or drain traffic from affected service if needed

Record every containment action: who approved, UTC time, expected side effect, rollback.

Eradication

• Remove malware, scheduled tasks, webshells, and unauthorized accounts
• Close ingress paths (patch CVE, fix misconfiguration, remove public exposure)
• Rotate secrets: DB passwords, signing keys, CI tokens, encryption keys per key-management policy
• Rebuild from gold image when integrity uncertain
• Validate no persistence via fresh EDR scan and hunt queries

Engage devsecops if pipeline or artifact compromise suspected; ai-redteam if LLM tool abuse or prompt-injection persistence in agents.

Recovery

1. Restore services from known-good backup or rebuild 2. Re-enable users and integrations in waves; monitor for recurrence 3. Increase detection sensitivity temporarily (higher-fidelity alerts to soc-analyst) 4. Confirm business functions and security controls (MFA, logging, backups) 5. Document recovery time and residual risk for post-incident review

Approval gates

Require explicit approver for:

• Org-wide password reset or MFA reset
• Production data deletion or mass customer notification
• Law enforcement coordination
• Paying ransom (policy usually prohibits—legal + exec only)

Coordination by incident type

Incident declaration and severity

Table of contents

1. When to declare 2. Declaration checklist 3. Severity model 4. Reclassification 5. SOC escalation criteria

When to declare

Declare a security incident when any of the following apply:

• Confirmed unauthorized access to production, corporate, or customer systems
• Malware or ransomware execution on managed endpoints or servers
• Suspected or confirmed personal data or regulated data exposure
• Compromise of privileged identity, break-glass, or CI/CD signing secrets
• Active command-and-control or ongoing attacker activity
• Supply-chain or third-party compromise affecting your environment
• Law enforcement or regulator inquiry tied to a security event

Do not require declaration for every SOC alert—soc-analyst closes benign/true-positive alerts per runbook.

Declaration checklist

1. Assign incident ID and title (type + primary asset) 2. Record declarer, time (UTC), and trigger source 3. Set severity with one-line rationale 4. Open incident channel and record link in ticketing system 5. Assign IC and CSIRT lead; page per matrix 6. Notify legal for SEV1–2 or any data exposure suspicion 7. Publish initial scope hypothesis (systems, accounts, data classes)—mark as preliminary 8. Start timeline and action tracker

Severity model

Align with org incident-management-engineer SEV definitions where they exist. Example security-focused mapping:

Severity drivers (use highest that applies):

• Number of users or tenants affected
• Privilege level of compromised accounts
• Data classification (public → confidential → regulated)
• Attacker persistence (active vs contained)
• Regulatory or contractual notification pressure

Reclassification

• Reclassify when scope expands (new systems, new data class) or containment confirmed
• Document old → new severity, reason, and time in incident record
• Adjust comms cadence and staffing immediately on upgrade

SOC escalation criteria

soc-analyst should escalate to CSIRT when:

• Multiple correlated alerts across identity, endpoint, and cloud
• Alert on Tier-0 assets (IdP, billing, prod admin, secrets store, CI signing)
• Unable to rule out data access within SLA (e.g., 30–60 minutes)
• User or customer report corroborates malicious activity
• Playbook step requires IR authority (legal hold, org-wide password reset, mass session revoke)

Escalation ticket must include UTC timestamps, entities, and evidence links.

Incident responder scope

Table of contents

1. Mission 2. In scope 3. Out of scope 4. Handoffs 5. Roles during active incident

Mission

Operate as CSIRT / incident response: manage declared security incidents from declaration through recovery and post-incident review. Optimize for accurate scope, preserved evidence, controlled containment, and clear stakeholder communication—not for designing the org-wide incident program or triaging every SOC alert.

In scope

• Incident declaration, severity, and scope statements
• Timeline reconstruction and narrative for leadership and legal
• Evidence identification, collection, preservation, and chain of custody
• Containment, eradication, recovery coordination (not sole executor of every technical step)
• Stakeholder comms templates and cadence (internal, exec, customer, partner)
• Regulatory notification preparation—fact gathering for legal/compliance; never provide legal advice
• Post-incident review, lessons learned, and remediation tracking

Out of scope

Handoffs

From SOC (`soc-analyst`):

• Escalate when incident criteria met: confirmed malicious activity, data exposure suspected, widespread compromise, ransomware, active C2, privileged abuse, or executive/regulatory trigger
• Handoff package: alert IDs, initial IOCs, affected entities, timestamps, analyst notes, open questions

To program (`incident-management-engineer`):

• Process gaps (escalation delays, paging failures, missing roles) become program improvements—not one-off fixes only

To engineering:

• Containment actions (isolate VM, revoke session, block IP, disable integration) with explicit approvers and rollback notes

Roles during active incident

Document role assignments in the incident record at declaration.

Post-incident review

Table of contents

1. When to run 2. Participants 3. Review agenda 4. Report template 5. Action items 6. Feeding improvements back

When to run

• SEV1–2 security incidents: review within 5 business days of containment (or per incident-management-engineer policy)
• SEV3: optional short review if recurrence risk or detection gap
• SEV4: trend in monthly SOC/CSIRT metrics unless pattern emerges

Do not close the incident record until critical actions have owners or explicit deferral with risk acceptance.

Participants

• Incident commander, CSIRT lead, key technical leads
• SOC representative (soc-analyst) if detection involved
• incident-management-engineer delegate for process gaps
• Legal/privacy if data exposure; comms if external messaging occurred
• Optional: affected product/engineering manager

Exclude blame; focus on systems and decisions.

Review agenda

1. Timeline walkthrough — final UTC timeline vs initial assumptions 2. Impact — customers, data, duration, regulatory touchpoints 3. What went well — fast containment, clear roles, useful runbooks 4. What went poorly — delays, tooling gaps, comms confusion 5. Root cause classes — people/process/technology/third party (multi-factor OK) 6. Detection — MTTD, alert quality, escalation path 7. Containment/eradication — effective actions and near misses 8. Actions — prioritized with owners and dates

Report template

# Post-incident review: INC-#### [Title]

## Summary
[2–3 sentences]

## Severity and duration
- Severity: SEV#
- Declared: UTC
- Contained: UTC
- Closed: UTC

## Impact
- Systems:
- Data:
- Customers:

## Timeline (abbreviated)
| UTC | Event |
|-----|-------|

## Root cause
[Primary and contributing factors]

## Lessons learned
1.
2.

## Action items
| ID | Action | Owner | Due | Type (detect/prevent/process) |
|----|--------|-------|-----|-------------------------------|

Action items

Classify each item:

Track to completion in same system as engineering work; review open items in monthly security ops forum.

Feeding improvements back

• 5 Whys only to actionable depth—stop at controllable fixes
• Share sanitized learnings with engineering and SOC
• Update runbooks and escalation criteria
• Schedule tabletop if systemic gap (program owner: incident-management-engineer)
• Link repeat incidents to reliability or compliance reviews when applicable

Stakeholder and regulatory communications

Table of contents

1. Principles 2. Audiences and cadence 3. Update template 4. Executive summary 5. Customer and partner messaging 6. Regulatory notification preparation 7. What not to say

Principles

• Facts vs hypotheses: label clearly; update when facts change
• Single voice: comms lead drafts; legal approves external/regulatory text
• No legal advice: CSIRT supplies timelines and technical facts; legal decides obligations
• Consistent numbers: one source of truth for affected counts (versioned)
• Privilege: mark sensitive counsel communications appropriately per org policy

Audiences and cadence

Align cadence with incident-management-engineer program; use communication-lead for polished crisis packs.

Update template

Subject: [INC-####] Security incident update — YYYY-MM-DD HH:MM UTC

1. Status — Investigating / Contained / Eradicating / Recovering / Monitoring / Closed 2. Severity — SEV# (unchanged / upgraded / downgraded since last update) 3. Customer impact — Known / None identified / Under investigation 4. Facts since last update — Bullet list with UTC times 5. Actions in progress — Owner per line 6. Next update — Time (UTC) 7. Open questions — Only if useful for decision-makers

Executive summary

One page max:

• What happened (one paragraph, factual)
• Scope: systems, data classes, geographies, tenant count (ranges if uncertain)
• Current status and time to next milestone
• Decisions needed (resources, customer comms, law enforcement)
• Regulatory touchpoints flagged for legal (yes/no/unknown)

Customer and partner messaging

• Do not disclose IOCs or TTPs that aid copycat attacks without security review
• Avoid blaming users; describe protective actions taken
• Provide actionable steps for customers (rotate API keys, review audit logs) only when validated
• Offer support channel and reference number

Route all external copy through legal and comms before send.

Regulatory notification preparation

CSIRT prepares a fact pack for legal/compliance (compliance-engineer supports evidence structure—not legal determinations):

Legal assesses GDPR, state breach laws, sector rules (HIPAA, PCI, etc.), and contract notice clauses. Do not promise notification timelines in technical channels.

What not to say

• Definitive “no data was accessed” before log proof
• Attribution to named threat actors without approved intel
• Legal conclusions or regulatory filing commitments
• Unverified attacker demands or ransom status in customer channels

Timeline and evidence handling

Table of contents

1. Timeline principles 2. Timeline table format 3. Evidence sources 4. Collection and preservation 5. Chain of custody 6. Common pitfalls

Timeline principles

• Use UTC for all timestamps; note analyst local TZ only in metadata if needed
• Separate attacker, defender, and business lanes in the narrative
• Label each entry fact (sourced) vs hypothesis (pending validation)
• Version the timeline when scope changes; never silent edits
• Tie entries to evidence IDs (log query, screenshot hash, ticket)

Timeline table format

Add columns as needed: affected entity, IOC, action owner.

Evidence sources

Prioritize by fidelity and retention:

Coordinate with cloud-security-engineer for cloud-native exports; information-security-engineer for tooling access.

Collection and preservation

1. Identify sources before containment destroys artifacts (volatile memory, live sessions) 2. Snapshot where appropriate: disk image, VM snapshot, memory dump—per legal guidance 3. Export logs with original timestamps; record query window and export job ID 4. Hash files (SHA-256); store in write-once or access-controlled bucket 5. Document who collected, when, from which system, and storage path 6. Avoid modifying original systems; work from copies 7. Legal hold: engage legal before broad deletion or retention changes

Chain of custody

Maintain a log for each artifact:

Transfer only through approved secure channels; no personal cloud drives.

Common pitfalls

• Rebuilding or reimaging hosts before imaging when forensics may be needed
• Letting log retention expire during long investigations—extend retention early
• Conflating detection time with attacker start time—state both explicitly
• Publishing unverified IOCs externally before legal/comms review