Qcsd Production Swarm

QCSD Production Swarm v1.0

Post-release production health assessment and QCSD feedback loop closure.

---

Overview

The Production Swarm assesses release health in the live production environment using DORA metrics, incident RCA, defect prediction, and cross-phase feedback loops. It renders a HEALTHY / DEGRADED / CRITICAL decision and is the only QCSD phase with dual responsibility: assessing current production health AND closing the feedback loop back to Ideation and Refinement phases.

QCSD Phase Positioning

Parameters

• TELEMETRY_DATA: Path to production telemetry, incident reports, and DORA metrics (required)
• RELEASE_ID: Release identifier for tracking (optional)
• OUTPUT_FOLDER: Where to save reports (default: ${PROJECT_ROOT}/Agentic QCSD/production/)
• SLA_DEFINITIONS: Path to SLA/SLO target definitions (optional)

---

ENFORCEMENT RULES - READ FIRST

PROHIBITED BEHAVIORS:

• Summarizing instead of spawning agents
• Skipping agents "for brevity"
• Proceeding before background tasks complete
• Providing your own analysis instead of spawning specialists
• Omitting report sections or using placeholder text

---

Step Execution Protocol

This skill uses a micro-file step architecture. Each step is a self-contained file loaded one at a time to avoid "lost in the middle" context degradation.

Execute steps sequentially by reading each step file with the Read tool.

Steps

1. Flag Detection -- steps/01-flag-detection.md -- Retrieve CI/CD signals, detect telemetry source, evaluate all 7 flags 2. Core Agents -- steps/02-core-agents.md -- Spawn qe-metrics-optimizer, qe-defect-predictor, qe-root-cause-analyzer in parallel 3. Batch 1 Results -- steps/03-batch1-results.md -- Wait for core agents, extract all metrics 4. Conditional Agents -- steps/04-conditional-agents.md -- Spawn flagged conditional agents in parallel 5. Decision Synthesis -- steps/05-decision-synthesis.md -- Apply HEALTHY/DEGRADED/CRITICAL logic 6. Report Generation -- steps/06-report-generation.md -- Generate executive summary and full report 7. Learning Persistence -- steps/07-learning-persistence.md -- Store findings to memory, save persistence record 8. Feedback Loop -- steps/08-feedback-loop.md -- Run learning coordinator then transfer specialist (sequential) 9. Final Output -- steps/09-final-output.md -- Display completion summary with all scores

Execution Instructions

1. Use the Read tool to load the current step file (e.g., Read({ file_path: ".claude/skills/qcsd-production-swarm/steps/01-flag-detection.md" })) 2. Execute the step's instructions completely 3. Verify all success criteria are met before proceeding 4. Pass the step's output as context to the next step 5. If a step fails, halt and report the failure point -- do not skip ahead

Resume Support

To resume from a specific step: specify --from-step N and the orchestrator will skip to step N. Ensure you have the required prerequisite data from prior steps.

---

Agent Inventory

Total: 12 agents (3 core + 7 conditional + 2 feedback)

---

Quality Gate Thresholds

---

Report Filename Mapping

---

Execution Model Options

---

Key Principle

Production health is measured by outcomes, not intentions. This swarm provides evidence-based production assessment and closes the QCSD feedback loop.

Step 1: Flag & Signal Detection

Prerequisites

• Production swarm skill invoked
• TELEMETRY_DATA path provided (or pre-collected telemetry available)
• RELEASE_ID identified (optional)
• OUTPUT_FOLDER determined

Instructions

Step 0: Retrieve CI/CD Phase Signals (Cross-Phase Consumption)

Before analyzing production context, retrieve the most recent CI/CD phase signals from memory. These provide the release readiness baseline that the Production Swarm builds upon.

MCP Method (preferred):

aqe memory search --pattern "qcsd-cicd-*" --namespace "qcsd-cicd" --limit 1 --json

CLI Fallback:

npx --no-install ruflo memory search --query "qcsd-cicd" --namespace qcsd-cicd --limit 1

Extract and record CI/CD baseline (if available):

+-------------------------------------------------------------+
|                CI/CD PHASE BASELINE                          |
+-------------------------------------------------------------+
|                                                              |
|  Retrieved:     [YES / NO - memory query failed]             |
|                                                              |
|  Release Decision:       [RELEASE / REMEDIATE / BLOCK / N/A] |
|  Deployment Risk Score:  [value / N/A]                       |
|  Quality Gate Status:    [PASSED / FAILED / N/A]             |
|  Known Issues:           [list / NONE]                       |
|  Monitoring Recommendations: [list / NONE]                   |
|                                                              |
|  If NO CI/CD baseline: Proceed without baseline.             |
|  Note "NO CI/CD BASELINE AVAILABLE" in report.               |
|                                                              |
+-------------------------------------------------------------+

DO NOT skip this step. If memory retrieval fails, proceed without baseline but document the gap.

---

Step 0.5: Auto-Detect Pre-Collected Telemetry

Before requiring manual telemetry input, check if the GitHub Actions telemetry collection workflow has pre-collected DORA metrics. This runs automatically after every npm publish and weekly on schedule.

Check for pre-collected telemetry:

TELEMETRY_FILE="docs/telemetry/production/latest.json"

If the file exists and is recent (< 7 days old):

• Use it as the primary TELEMETRY_DATA source
• The DORA metrics (deployment frequency, lead time, change failure rate, MTTR) are already

computed from GitHub API — the qe-metrics-optimizer agent should validate and enrich these, not recompute from scratch

• Record: TELEMETRY SOURCE: GHA pre-collected (${collectionTimestamp from JSON})
• Extract the releaseId from the JSON if RELEASE_ID was not provided as a parameter

If the file does not exist or is stale (> 7 days old):

• Proceed with the manually provided TELEMETRY_DATA parameter as currently specified
• Record: TELEMETRY SOURCE: Manual input

This step is non-blocking. If pre-collected telemetry is unavailable, the swarm operates exactly as before. Pre-collected telemetry simply accelerates Phase 2 by giving qe-metrics-optimizer a validated starting point.

---

Step 1: Scan Production Context and Detect Flags

Scan the production telemetry, incident reports, DORA data, and release context to SET these flags. Do not skip any flag.

Flag Detection (Check ALL SEVEN)

HAS_INFRASTRUCTURE_CHANGE = FALSE
  Set TRUE if input mentions RECENT infrastructure changes since last release:
  Kubernetes config changes, container image updates, cloud resource modifications,
  deployment topology changes, scaling policy updates, network rule changes,
  load balancer updates, DNS changes, certificate rotations, CDN changes.
  NOTE: General mentions of infrastructure existing do NOT trigger this flag.
  Only RECENT CHANGES to infrastructure trigger it.

HAS_PERFORMANCE_SLA = FALSE
  Set TRUE if input mentions ANY of: SLA, SLO, SLI, response time requirements,
  latency targets, error budgets, throughput thresholds, availability targets,
  uptime requirements, p95 latency, p99 latency, error rate targets

HAS_REGRESSION_RISK = FALSE
  Set TRUE if input mentions ANY of: user-reported issues, error rate increases,
  degraded functionality, rollback consideration, feature flag incidents,
  A/B test anomalies, customer complaints, support ticket spikes,
  monitoring alerts, degraded performance post-deploy

HAS_RECURRING_INCIDENTS = FALSE
  Set TRUE if input mentions ANY of: repeated incidents, known recurring issues,
  incident patterns, chronic alerts, previously-seen failure modes,
  flapping services, repeat offender modules, recurring pages,
  same-root-cause incidents, deja-vu failures

HAS_MIDDLEWARE = FALSE
  Set TRUE if input mentions ANY of: middleware, ESB, message broker, MQ,
  Kafka, RabbitMQ, integration bus, API gateway, message queue, pub/sub,
  event bus, service bus, ActiveMQ, NATS, Redis Streams

HAS_SAP_INTEGRATION = FALSE
  Set TRUE if input mentions ANY of: SAP, RFC, BAPI, IDoc, OData,
  S/4HANA, EWM, ECC, ABAP, CDS view, Fiori, SAP Cloud Integration,
  SAP PI/PO, SAP Gateway, SAP connector

HAS_AUTHORIZATION = FALSE
  Set TRUE if input mentions ANY of: SoD, segregation of duties,
  role conflict, authorization object, T-code, user role,
  access control matrix, GRC, RBAC policy, permission matrix,
  privilege escalation, role assignment

Validation Checkpoint

Before proceeding, confirm:

+-- I have read the production telemetry and incident reports
+-- I have read the DORA metrics data
+-- I have reviewed the release context and CI/CD phase signals
+-- I have evaluated ALL SEVEN flags
+-- I have recorded which flags are TRUE
+-- I understand which conditional agents will be needed

DO NOT proceed until all checkboxes are confirmed.

MANDATORY: Output Flag Detection Results

You MUST output flag detection results before proceeding:

+-------------------------------------------------------------+
|                    FLAG DETECTION RESULTS                    |
+-------------------------------------------------------------+
|                                                             |
|  HAS_INFRASTRUCTURE_CHANGE: [TRUE/FALSE]                    |
|  Evidence:                  [what triggered it - specific]  |
|                                                             |
|  HAS_PERFORMANCE_SLA:       [TRUE/FALSE]                    |
|  Evidence:                  [what triggered it - specific]  |
|                                                             |
|  HAS_REGRESSION_RISK:       [TRUE/FALSE]                    |
|  Evidence:                  [what triggered it - specific]  |
|                                                             |
|  HAS_RECURRING_INCIDENTS:   [TRUE/FALSE]                    |
|  Evidence:                  [what triggered it - specific]  |
|                                                             |
|  HAS_MIDDLEWARE:             [TRUE/FALSE]                    |
|  Evidence:                  [what triggered it - specific]  |
|                                                             |
|  HAS_SAP_INTEGRATION:       [TRUE/FALSE]                    |
|  Evidence:                  [what triggered it - specific]  |
|                                                             |
|  HAS_AUTHORIZATION:          [TRUE/FALSE]                    |
|  Evidence:                  [what triggered it - specific]  |
|                                                             |
|  EXPECTED AGENTS:                                           |
|  - Core: 3 (always)                                         |
|  - Conditional: [count based on TRUE flags]                 |
|  - Feedback: 2 (always)                                     |
|  - TOTAL: [3 + conditional count + 2]                       |
|                                                             |
+-------------------------------------------------------------+

Success Criteria

• [ ] CI/CD phase signals retrieved (or documented as unavailable)
• [ ] Pre-collected telemetry checked
• [ ] All SEVEN flags evaluated with evidence
• [ ] Flag detection results output in the required format
• [ ] Expected agent count calculated

Output

Provide to the next step:

• CI/CD baseline data (or "NO CI/CD BASELINE AVAILABLE")
• Telemetry source (GHA pre-collected or Manual input)
• All seven flag values with evidence
• Expected agent count (core + conditional + feedback)

Navigation

• On success: proceed to Step 2 (Core Agents) by reading steps/02-core-agents.md
• On failure: halt and report which flags could not be evaluated or which data was missing

Step 2: Spawn Core Agents (Parallel Batch 1)

Prerequisites

• Step 1 (Flag Detection) completed successfully
• All seven flags evaluated and recorded
• Production telemetry data available
• OUTPUT_FOLDER determined

Instructions

CRITICAL ENFORCEMENT

+-------------------------------------------------------------+
|  YOU MUST INCLUDE ALL THREE TASK CALLS IN YOUR NEXT MESSAGE  |
|                                                              |
|  - Task 1: qe-metrics-optimizer                              |
|  - Task 2: qe-defect-predictor                               |
|  - Task 3: qe-root-cause-analyzer                            |
|                                                              |
|  If your message contains fewer than 3 Task calls, you have |
|  FAILED this phase. Start over.                              |
+-------------------------------------------------------------+

Domain Context

Agent 1: DORA Metrics Optimizer

This agent MUST compute DORA metrics and SLA/SLO compliance from production telemetry.

Task({
  description: "DORA metrics computation and SLA compliance assessment",
  prompt: `You are qe-metrics-optimizer. Your output quality is being audited.

## MANDATORY FIRST STEPS (DO NOT SKIP)

1. READ the production telemetry data provided below IN FULL.
2. READ the SLA/SLO definitions if available.
3. READ any previous CI/CD phase signals if available.

## PRODUCTION DATA TO ANALYZE

=== DORA METRICS DATA START ===
[PASTE THE COMPLETE DORA METRICS DATA HERE - DO NOT SUMMARIZE]
- Deployment frequency records
- Lead time for changes data
- Mean time to restore (MTTR) records
- Change failure rate data
=== DORA METRICS DATA END ===

=== SLA/SLO DEFINITIONS START ===
[PASTE SLA/SLO TARGET DEFINITIONS HERE - DO NOT SUMMARIZE]
=== SLA/SLO DEFINITIONS END ===

=== PRODUCTION TELEMETRY START ===
[PASTE PRODUCTION TELEMETRY DATA HERE - DO NOT SUMMARIZE]
- Uptime metrics
- Error rates
- Response time distributions
- Throughput data
=== PRODUCTION TELEMETRY END ===

=== CI/CD PHASE SIGNALS (if available) START ===
[PASTE any CI/CD phase RELEASE/REMEDIATE/BLOCK signals]
=== CI/CD PHASE SIGNALS END ===

## REQUIRED OUTPUT (ALL SECTIONS MANDATORY)

### 1. DORA Dashboard

Compute all four DORA metrics with trends:

| DORA Metric | Current Value | Previous Period | Trend | Classification |
|-------------|---------------|-----------------|-------|----------------|
| Deployment Frequency | X/day or X/week | X/day or X/week | Improving/Declining/Stable | Elite/High/Medium/Low |
| Lead Time for Changes | X hours/days | X hours/days | Improving/Declining/Stable | Elite/High/Medium/Low |
| Mean Time to Restore (MTTR) | X hours | X hours | Improving/Declining/Stable | Elite/High/Medium/Low |
| Change Failure Rate | X% | X% | Improving/Declining/Stable | Elite/High/Medium/Low |

**DORA Classification Thresholds:**
| Metric | Elite | High | Medium | Low |
|--------|-------|------|--------|-----|
| Deployment Frequency | On-demand (multiple/day) | Weekly-Monthly | Monthly-Biannual | Biannual+ |
| Lead Time | < 1 hour | 1 day - 1 week | 1 week - 1 month | 1-6 months |
| MTTR | < 1 hour | < 1 day | < 1 week | 1 week+ |
| Change Failure Rate | 0-5% | 5-10% | 10-15% | 15%+ |

**OVERALL DORA CLASSIFICATION: [Elite/High/Medium/Low]**

### 2. SLA Compliance Matrix

| SLA/SLO | Target | Actual | Compliance | Status | Burn Rate |
|---------|--------|--------|------------|--------|-----------|
| Availability | X% | X% | X% | PASS/WARN/FAIL | [error budget remaining] |
| Response Time (p50) | Xms | Xms | X% | PASS/WARN/FAIL | N/A |
| Response Time (p95) | Xms | Xms | X% | PASS/WARN/FAIL | N/A |
| Response Time (p99) | Xms | Xms | X% | PASS/WARN/FAIL | N/A |
| Error Rate | <= X% | X% | X% | PASS/WARN/FAIL | [error budget remaining] |
| Throughput | >= X req/s | X req/s | X% | PASS/WARN/FAIL | N/A |

**SLA COMPLIANCE: X% (count of passing SLAs / total SLAs)**

### 3. Quality Metrics Optimization

| Quality Metric | Current | Target | Gap | Trend | Action |
|---------------|---------|--------|-----|-------|--------|
| Defect Escape Rate | X% | <= X% | +/-X% | Improving/Declining | [action] |
| Test Effectiveness | X% | >= X% | +/-X% | Improving/Declining | [action] |
| Automation Coverage | X% | >= X% | +/-X% | Improving/Declining | [action] |
| Mean Time to Detect | X hours | <= X hours | +/-X hours | Improving/Declining | [action] |
| Customer-Reported vs Internal | X:Y ratio | <= X:Y | +/-X | Improving/Declining | [action] |

### 4. Composite DORA Score

Calculate normalized score (0-1):

| Metric | Raw Score | Weight | Weighted Score |
|--------|-----------|--------|----------------|
| Deployment Frequency | X/1.0 | 0.25 | X |
| Lead Time | X/1.0 | 0.25 | X |
| MTTR | X/1.0 | 0.25 | X |
| Change Failure Rate | X/1.0 | 0.25 | X |

**COMPOSITE DORA SCORE: X.XX (0-1 scale)**

Scoring guide:
- Elite: 0.85 - 1.0
- High: 0.7 - 0.84
- Medium: 0.4 - 0.69
- Low: 0.0 - 0.39

**MINIMUM: Compute all 4 DORA metrics with classifications and produce SLA compliance matrix.**

## OUTPUT FORMAT

Save your complete analysis in Markdown to:
${OUTPUT_FOLDER}/02-dora-metrics.md

Use the Write tool to save BEFORE completing.
Report MUST be complete - no placeholders.

## VALIDATION BEFORE SUBMITTING

+-- Did I read all production telemetry and DORA data?
+-- Did I compute all 4 DORA metrics with trends?
+-- Did I classify each metric (Elite/High/Medium/Low)?
+-- Did I evaluate SLA/SLO compliance for all targets?
+-- Did I calculate the composite DORA score (0-1)?
+-- Did I assess quality metrics optimization opportunities?
+-- Did I save the report to the correct output path?`,
  subagent_type: "qe-metrics-optimizer",
  run_in_background: true
})

Agent 2: Defect Predictor

This agent MUST analyze defect trends and predict future defect density from production telemetry patterns.

Task({
  description: "ML-powered defect prediction and trend analysis from production data",
  prompt: `You are qe-defect-predictor. Your output quality is being audited.

## PRODUCTION DATA TO ANALYZE

=== DEFECT DATA START ===
[PASTE THE COMPLETE DEFECT/BUG REPORT DATA HERE - DO NOT SUMMARIZE]
- All defects discovered post-release
- Severity classifications
- Component/module mapping
- Discovery date and resolution status
=== DEFECT DATA END ===

=== PRODUCTION TELEMETRY START ===
[PASTE PRODUCTION ERROR LOGS, EXCEPTION DATA, MONITORING ALERTS]
=== PRODUCTION TELEMETRY END ===

=== HISTORICAL DEFECT DATA (if available) START ===
[PASTE historical defect data from previous releases]
=== HISTORICAL DEFECT DATA END ===

## REQUIRED OUTPUT (ALL SECTIONS MANDATORY)

### 1. Defect Trend Analysis

| Period | Defects Found | Severity Distribution | Density (per KLOC) | Trend |
|--------|--------------|----------------------|---------------------|-------|
| Current Release | X | P0:X P1:X P2:X P3:X P4:X | X.XX | - |
| Previous Release | X | P0:X P1:X P2:X P3:X P4:X | X.XX | - |
| 3-Release Average | X | P0:X P1:X P2:X P3:X P4:X | X.XX | - |
| 6-Release Average | X | P0:X P1:X P2:X P3:X P4:X | X.XX | - |

**DEFECT TREND DIRECTION: [declining / stable / increasing]**

### 2. Predicted Defect Density

| Prediction Horizon | Predicted Density | Confidence | Method |
|--------------------|-------------------|------------|--------|
| Next 7 days | X.XX per KLOC | High/Medium/Low | [regression/trend/pattern] |
| Next 30 days | X.XX per KLOC | High/Medium/Low | [regression/trend/pattern] |
| Next release cycle | X.XX per KLOC | High/Medium/Low | [regression/trend/pattern] |

**PREDICTED DEFECT DENSITY: X.XX per KLOC**

### 3. Hotspot Identification

| Component/Module | Defect Count | Density | Risk Rank | Contributing Factors |
|-----------------|-------------|---------|-----------|---------------------|
| [module 1] | X | X.XX | 1 (Highest) | [complexity, churn, coupling, etc.] |
| [module 2] | X | X.XX | 2 | [factors] |
| [module 3] | X | X.XX | 3 | [factors] |
| [module 4] | X | X.XX | 4 | [factors] |
| [module 5] | X | X.XX | 5 | [factors] |

### 4. Pattern Analysis

| Pattern | Occurrences | Modules Affected | Root Cause Category | Preventability |
|---------|-------------|-----------------|--------------------|--------------------|
| [pattern 1] | X | [modules] | [code/design/config/env] | High/Medium/Low |
| [pattern 2] | X | [modules] | [code/design/config/env] | High/Medium/Low |
| [pattern 3] | X | [modules] | [code/design/config/env] | High/Medium/Low |

### 5. Escape Analysis

For each defect found in production, identify which QCSD phase SHOULD have caught it:

| Defect ID | Severity | Escaped From | Why Escaped | Prevention Strategy |
|-----------|----------|-------------|-------------|---------------------|
| DEF-001 | P0/P1/P2/P3/P4 | Ideation/Refinement/Development/Verification | [why not caught] | [what to change] |
| DEF-002 | P0/P1/P2/P3/P4 | Ideation/Refinement/Development/Verification | [why not caught] | [what to change] |

**Escape Summary:**
| Escaped From Phase | Count | Percentage | Key Gap |
|-------------------|-------|------------|---------|
| Ideation | X | X% | [missing risk assessment] |
| Refinement | X | X% | [missing test strategy] |
| Development | X | X% | [missing test coverage] |
| Verification | X | X% | [missing pipeline check] |

**MINIMUM: Analyze defect trends, calculate density, identify at least 5 hotspots, and perform escape analysis for all defects.**

## OUTPUT FORMAT

Save your complete analysis in Markdown to:
${OUTPUT_FOLDER}/03-defect-prediction.md

Use the Write tool to save BEFORE completing.
Report MUST be complete - no placeholders.

## VALIDATION BEFORE SUBMITTING

+-- Did I analyze defect data across multiple release periods?
+-- Did I calculate trend direction (declining/stable/increasing)?
+-- Did I predict future defect density with confidence levels?
+-- Did I identify at least 5 defect hotspots?
+-- Did I perform escape analysis mapping defects to QCSD phases?
+-- Did I identify at least 3 defect patterns?
+-- Did I save the report to the correct output path?`,
  subagent_type: "qe-defect-predictor",
  run_in_background: true
})

Agent 3: Root Cause Analyzer

This agent MUST perform systematic RCA of all production incidents since release. Incident severity tracking is mandatory.

Task({
  description: "Systematic root cause analysis of production incidents",
  prompt: `You are qe-root-cause-analyzer. Your output quality is being audited.

## PRODUCTION DATA TO ANALYZE

=== INCIDENT REPORTS START ===
[PASTE THE COMPLETE INCIDENT REPORTS HERE - DO NOT SUMMARIZE]
- All P0-P4 incidents since release
- Incident timelines
- Resolution actions
- Post-mortems if available
=== INCIDENT REPORTS END ===

=== PRODUCTION LOGS START ===
[PASTE RELEVANT PRODUCTION LOGS AND ERROR DATA]
=== PRODUCTION LOGS END ===

=== MONITORING ALERTS START ===
[PASTE MONITORING ALERTS AND ALERT HISTORY]
=== MONITORING ALERTS END ===

## REQUIRED OUTPUT (ALL SECTIONS MANDATORY)

### 1. Incident Inventory

Complete inventory of all incidents since release:

| Incident ID | Severity | Status | Summary | Duration | Impact | Detected By |
|-------------|----------|--------|---------|----------|--------|-------------|
| INC-001 | P0/P1/P2/P3/P4 | Open/Resolved/Mitigated | [summary] | X hours | [users/revenue/data] | [monitoring/user/support] |
| INC-002 | P0/P1/P2/P3/P4 | Open/Resolved/Mitigated | [summary] | X hours | [users/revenue/data] | [monitoring/user/support] |

**TOTAL INCIDENTS: X (P0: X, P1: X, P2: X, P3: X, P4: X)**
**MAXIMUM OPEN SEVERITY: [P0/P1/P2/P3/P4/NONE]**

### 2. Root Cause Analysis (per incident)

For EACH incident, provide structured RCA:

#### INC-XXX: [Title]

| RCA Dimension | Finding |
|---------------|---------|
| **What happened** | [factual description of the incident] |
| **Timeline** | [detection -> diagnosis -> mitigation -> resolution] |
| **Root cause** | [the underlying technical cause] |
| **Contributing factors** | [what made the incident possible or worse] |
| **Why not detected earlier** | [gap in monitoring, testing, or review] |
| **5-Why Analysis** | 1. Why? -> 2. Why? -> 3. Why? -> 4. Why? -> 5. Why? -> Root |
| **Category** | [code-defect/config-error/infra-failure/capacity/dependency/human-error] |

### 3. Resolution and Prevention Strategies

| Incident | Resolution Applied | Time to Resolve | Prevention Strategy | Owner | Status |
|----------|-------------------|----------------|--------------------|---------|----|
| INC-XXX | [what was done] | X hours | [what prevents recurrence] | [team] | Implemented/Planned/Backlogged |

### 4. Time to Detect and Resolve

| Incident | Time to Detect (TTD) | Time to Diagnose | Time to Mitigate | Time to Resolve (TTR) | Total Duration |
|----------|---------------------|-----------------|------------------|----------------------|----------------|
| INC-XXX | X min/hours | X min/hours | X min/hours | X min/hours | X min/hours |

**Average TTD: X hours**
**Average TTR: X hours**

### 5. Escape Phase Analysis

| Incident | Root Cause | Should Have Been Caught In | Why It Escaped | Gap Type |
|----------|-----------|---------------------------|----------------|----------|
| INC-XXX | [root cause] | Ideation/Refinement/Development/Verification | [reason] | Testing/Monitoring/Review/Design |

### 6. RCA Completeness

| Metric | Value |
|--------|-------|
| Total incidents requiring RCA | X |
| RCAs completed | X |
| RCA completeness | X% |
| Incidents with prevention plans | X |
| Prevention implementation rate | X% |

**RCA COMPLETENESS: X%** (completedRcas / totalIncidents * 100)

**MINIMUM: Inventory ALL incidents, perform 5-Why RCA for each P0/P1, and calculate RCA completeness percentage.**

## OUTPUT FORMAT

Save your complete analysis in Markdown to:
${OUTPUT_FOLDER}/04-root-cause-analysis.md

Use the Write tool to save BEFORE completing.
Report MUST be complete - no placeholders.

## VALIDATION BEFORE SUBMITTING

+-- Did I inventory ALL incidents (P0-P4) since the release?
+-- Did I record the maximum open severity?
+-- Did I perform 5-Why RCA for each P0/P1 incident?
+-- Did I document resolution and prevention for each incident?
+-- Did I calculate time to detect and resolve for each incident?
+-- Did I perform escape phase analysis?
+-- Did I calculate RCA completeness percentage?
+-- Did I save the report to the correct output path?`,
  subagent_type: "qe-root-cause-analyzer",
  run_in_background: true
})

Post-Spawn Confirmation

After sending all three Task calls, you MUST tell the user:

I've launched 3 core agents in parallel:

  qe-metrics-optimizer [Domain: learning-optimization]
   - Computing DORA metrics (deployment frequency, lead time, MTTR, CFR)
   - Evaluating SLA/SLO compliance across all targets
   - Calculating composite DORA score (0-1)

  qe-defect-predictor [Domain: defect-intelligence]
   - Analyzing defect trends and predicting future density
   - Identifying defect hotspots across modules
   - Performing escape analysis (which phase should have caught each defect)

  qe-root-cause-analyzer [Domain: defect-intelligence]
   - Inventorying all P0-P4 incidents since release
   - Performing systematic 5-Why RCA for each incident
   - Calculating RCA completeness percentage

  WAITING for all agents to complete before proceeding...

DO NOT proceed to Step 3 until you have sent this confirmation.

Success Criteria

• [ ] All THREE core agents spawned in a SINGLE message
• [ ] Each agent received complete production data (not summaries)
• [ ] Post-spawn confirmation message sent to user
• [ ] All agents running in background

Output

Provide to the next step:

• Confirmation that all 3 agents are spawned and running
• The Task IDs for tracking (if available)

Navigation

• On success: proceed to Step 3 (Wait & Extract Results) by reading steps/03-batch1-results.md
• On failure: if fewer than 3 agents spawned, respawn missing agents before proceeding

Step 3: Wait for Batch 1 & Extract Results

Prerequisites

• Step 2 (Core Agents) completed: all 3 core agents spawned
• qe-metrics-optimizer, qe-defect-predictor, qe-root-cause-analyzer running in background

Instructions

ENFORCEMENT: NO EARLY PROCEEDING

+-------------------------------------------------------------+
|  YOU MUST WAIT FOR ALL THREE BACKGROUND TASKS TO COMPLETE    |
|                                                              |
|  DO NOT summarize what agents "would" find                   |
|  DO NOT proceed to Step 4 early                              |
|  DO NOT provide your own analysis as substitute              |
|                                                              |
|  WAIT for actual agent results                               |
|  ONLY proceed when all three have returned                   |
+-------------------------------------------------------------+

Results Extraction Checklist

When results return, extract and record:

From qe-metrics-optimizer:
[ ] doraScore = __.__ composite score (0-1)
[ ] doraClassification = Elite/High/Medium/Low
[ ] slaCompliance = __% compliance percentage
[ ] deploymentFrequency = __/day or __/week
[ ] leadTime = __ hours/days
[ ] mttr = __ hours
[ ] changeFailureRate = __%

From qe-defect-predictor:
[ ] defectTrend = declining/stable/increasing
[ ] defectDensity = __.__ per KLOC
[ ] hotspotCount = __ hotspots identified
[ ] escapeCount = __ defects escaped from earlier phases
[ ] predictedDensity = __.__ per KLOC (next period)

From qe-root-cause-analyzer:
[ ] incidentCount = __ total incidents
[ ] maxOpenSeverity = P0/P1/P2/P3/P4/NONE
[ ] rcaCompleteness = __% (completed RCAs / total incidents)
[ ] averageTTD = __ hours (time to detect)
[ ] averageTTR = __ hours (time to resolve)
[ ] openP0P1 = __ count of open P0/P1 incidents

Metrics Summary Box

Output extracted metrics:

+-------------------------------------------------------------+
|                    BATCH 1 RESULTS SUMMARY                   |
+-------------------------------------------------------------+
|                                                              |
|  DORA Score:                 __.__ (Elite/High/Med/Low)       |
|  Deployment Frequency:       __/day or __/week               |
|  Lead Time:                  __ hours/days                   |
|  MTTR:                       __ hours                        |
|  Change Failure Rate:        __%                             |
|  SLA Compliance:             __%                             |
|                                                              |
|  Defect Trend:               declining/stable/increasing     |
|  Defect Density:             __.__ per KLOC                  |
|  Predicted Density:          __.__ per KLOC                  |
|  Hotspots:                   __                              |
|  Defects Escaped:            __                              |
|                                                              |
|  Incidents (total):          __                              |
|  Max Open Severity:          P_/NONE                         |
|  RCA Completeness:           __%                             |
|  Avg Time to Detect:         __ hours                        |
|  Avg Time to Resolve:        __ hours                        |
|  Open P0/P1:                 __                              |
|                                                              |
+-------------------------------------------------------------+

DO NOT proceed to Step 4 until ALL fields are filled.

Success Criteria

• [ ] All three core agents returned results
• [ ] All metrics extracted and recorded in summary box
• [ ] No placeholder values remain in the summary
• [ ] Results saved for use in subsequent steps

Output

Provide to the next step:

• Complete Batch 1 Results Summary with all metrics filled
• Flag values from Step 1 (needed to determine which conditional agents to spawn)

Navigation

• On success: proceed to Step 4 (Conditional Agents) by reading steps/04-conditional-agents.md
• On failure: if any agent failed, retry the failed agent before proceeding

Step 4: Spawn Conditional Agents (Parallel Batch 2)

Prerequisites

• Step 3 (Batch 1 Results) completed with all metrics extracted
• Flag detection results from Step 1 available
• Core agent reports saved to OUTPUT_FOLDER

Instructions

ENFORCEMENT: NO SKIPPING CONDITIONAL AGENTS

+-------------------------------------------------------------+
|  IF A FLAG IS TRUE, YOU MUST SPAWN THAT AGENT                |
|                                                              |
|  HAS_INFRASTRUCTURE_CHANGE = TRUE -> MUST spawn qe-chaos-engineer       |
|  HAS_PERFORMANCE_SLA = TRUE       -> MUST spawn qe-performance-tester   |
|  HAS_REGRESSION_RISK = TRUE       -> MUST spawn qe-regression-analyzer  |
|  HAS_RECURRING_INCIDENTS = TRUE   -> MUST spawn qe-pattern-learner      |
|  HAS_MIDDLEWARE = TRUE             -> MUST spawn qe-middleware-validator  |
|  HAS_SAP_INTEGRATION = TRUE       -> MUST spawn qe-sap-rfc-tester       |
|  HAS_AUTHORIZATION = TRUE         -> MUST spawn qe-sod-analyzer         |
|                                                              |
|  Skipping a flagged agent is a FAILURE of this skill.        |
+-------------------------------------------------------------+

Conditional Domain Mapping

Decision Tree

IF ALL flags are FALSE:
    -> Skip to Step 5 (no conditional agents needed)
    -> State: "No conditional agents needed based on production context"

ELSE:
    -> Spawn ALL applicable agents in ONE message
    -> Count how many you're spawning: __

IF HAS_INFRASTRUCTURE_CHANGE: Chaos Engineer

Spawn qe-chaos-engineer to assess resilience of production systems after recent infrastructure changes. Agent must analyze infrastructure change impact, run resilience tests, assess auto-recovery, evaluate resource utilization, and produce a Chaos Resilience Score (0-100). Save to: ${OUTPUT_FOLDER}/05-chaos-resilience.md

IF HAS_PERFORMANCE_SLA: Performance Tester

Spawn qe-performance-tester to validate production performance against SLA/SLO targets. Agent must assess SLA/SLO compliance, detect performance regressions, analyze error budgets, evaluate latency distributions, and produce a Performance SLA Score (0-100). Save to: ${OUTPUT_FOLDER}/06-performance-sla.md

IF HAS_REGRESSION_RISK: Regression Analyzer

Spawn qe-regression-analyzer to analyze production regressions from user-reported issues and monitoring data. Agent must inventory regressions, analyze error rates, map root causes, assess user impact, and produce a Regression Analysis Score (0-100). Save to: ${OUTPUT_FOLDER}/07-regression-analysis.md

IF HAS_RECURRING_INCIDENTS: Pattern Learner

Spawn qe-pattern-learner to detect recurring incident patterns in production. Agent must identify recurring patterns, analyze flapping services, find repeat offender modules, cluster same-root-cause incidents, and provide recommendations. Save to: ${OUTPUT_FOLDER}/08-pattern-analysis.md

IF HAS_MIDDLEWARE: Middleware Validator

Spawn qe-middleware-validator to validate middleware and message broker health. Agent must inventory middleware components, assess message flow health, evaluate broker health, analyze message loss, and produce a Middleware Health Score (0-100). Save to: ${OUTPUT_FOLDER}/10-middleware-health.md

IF HAS_SAP_INTEGRATION: SAP RFC Tester

Spawn qe-sap-rfc-tester to validate SAP RFC/BAPI integration health. Agent must inventory SAP services, assess RFC/BAPI health, validate data integrity, evaluate connector performance, and produce a SAP Health Score (0-100). Save to: ${OUTPUT_FOLDER}/11-sap-health.md

IF HAS_AUTHORIZATION: SoD Analyzer

Spawn qe-sod-analyzer to validate segregation of duties and authorization controls. Agent must assess compliance status, inventory SoD violations, analyze role assignments, evaluate access control effectiveness, and produce a SoD Compliance Score (0-100). Save to: ${OUTPUT_FOLDER}/12-sod-compliance.md

Agent Count Validation

+-------------------------------------------------------------+
|                   AGENT COUNT VALIDATION                     |
+-------------------------------------------------------------+
|  CORE AGENTS (ALWAYS 3):                                     |
|    [ ] qe-metrics-optimizer - SPAWNED                        |
|    [ ] qe-defect-predictor - SPAWNED                         |
|    [ ] qe-root-cause-analyzer - SPAWNED                      |
|  CONDITIONAL AGENTS (based on flags):                        |
|    [ ] qe-chaos-engineer - [Y/N] (HAS_INFRA)                |
|    [ ] qe-performance-tester - [Y/N] (HAS_PERF_SLA)         |
|    [ ] qe-regression-analyzer - [Y/N] (HAS_REGRESS)         |
|    [ ] qe-pattern-learner - [Y/N] (HAS_RECURRING)           |
|    [ ] qe-middleware-validator - [Y/N] (HAS_MIDDLEWARE)       |
|    [ ] qe-sap-rfc-tester - [Y/N] (HAS_SAP_INTEG)            |
|    [ ] qe-sod-analyzer - [Y/N] (HAS_AUTHORIZATION)          |
|  FEEDBACK AGENTS (ALWAYS 2): PENDING (Step 7)               |
|  VALIDATION:                                                 |
|    Expected so far: [3 + count of TRUE flags]                |
|    Actual spawned:  [count]                                  |
|    Status:          [PASS/FAIL]                              |
+-------------------------------------------------------------+

DO NOT proceed if validation FAILS. Wait for all conditional agents to complete.

Success Criteria

• [ ] All flagged conditional agents spawned in ONE message (or skipped to Step 5 if no flags)
• [ ] Agent count validation PASSED
• [ ] All conditional agents completed and returned results
• [ ] Conditional scores extracted

Output

Provide to the next step:

• regressionCount (from qe-regression-analyzer, or NULL)
• chaosResilience (from qe-chaos-engineer, or NULL, 0-100)
• middlewareHealth (from qe-middleware-validator, or NULL, 0-100)
• sapHealth (from qe-sap-rfc-tester, or NULL, 0-100)
• sodCompliance (from qe-sod-analyzer, or NULL, 0-100)
• Agent count validation result

Navigation

• On success: proceed to Step 5 (Decision Logic) by reading steps/05-decision-synthesis.md
• On failure: spawn missing agents before proceeding

Step 5: Synthesize Results & Determine Recommendation

Prerequisites

• Step 3 (Batch 1 Results) completed with core metrics
• Step 4 (Conditional Agents) completed with all conditional scores
• All agent reports saved to OUTPUT_FOLDER

Instructions

ENFORCEMENT: EXACT DECISION LOGIC

You MUST apply this logic EXACTLY. No interpretation.

STEP 1: Derive composite metrics from agent outputs
-----------------------------------------------------------
doraScore         = doraMetrics.compositeScore from qe-metrics-optimizer
slaCompliance     = slaReport.compliancePercent from qe-metrics-optimizer
incidentSeverity  = maxOpenSeverity from qe-root-cause-analyzer
rcaCompleteness   = completedRcas / totalIncidents from qe-root-cause-analyzer
defectTrend       = trendDirection from qe-defect-predictor
defectDensity     = predictedDensity from qe-defect-predictor
regressionCount   = regressionCount from qe-regression-analyzer (NULL if not ran)
chaosResilience   = resilienceScore from qe-chaos-engineer (NULL if not ran, 0-100)
middlewareHealth   = healthScore from qe-middleware-validator (NULL if not ran, 0-100)
sapHealth          = healthScore from qe-sap-rfc-tester (NULL if not ran, 0-100)
sodCompliance      = complianceScore from qe-sod-analyzer (NULL if not ran, 0-100)

STEP 2: Check CRITICAL conditions (ANY triggers CRITICAL)
-----------------------------------------------------------
IF incidentSeverity in [P0, P1]        -> CRITICAL ("Active P0/P1 incidents")
IF doraScore < 0.4                     -> CRITICAL ("DORA metrics critically low")
IF slaCompliance < 95.0                -> CRITICAL ("SLA compliance below minimum")
IF defectTrend == "increasing"
   AND defectDensity > 5.0             -> CRITICAL ("Accelerating defect trend")
IF middlewareHealth != NULL
   AND middlewareHealth < 20           -> CRITICAL ("Middleware critically unhealthy")
IF sapHealth != NULL
   AND sapHealth < 20                  -> CRITICAL ("SAP integration critically unhealthy")
IF sodCompliance != NULL
   AND sodCompliance < 20             -> CRITICAL ("Critical SoD violations in production")

STEP 3: Check HEALTHY conditions (ALL required for HEALTHY)
-----------------------------------------------------------
IF doraScore >= 0.7
   AND slaCompliance >= 99.0
   AND incidentSeverity in [P3, P4, NONE]
   AND rcaCompleteness >= 80
   AND defectTrend in ["declining", "stable"]
   AND defectDensity <= 2.0
   AND (regressionCount == NULL OR regressionCount <= 2)
   AND (chaosResilience == NULL OR chaosResilience >= 80)
   AND (middlewareHealth == NULL OR middlewareHealth >= 70)
   AND (sapHealth == NULL OR sapHealth >= 70)
   AND (sodCompliance == NULL OR sodCompliance >= 70)
                                       -> HEALTHY

STEP 4: Default
-----------------------------------------------------------
ELSE                                   -> DEGRADED

Decision Recording

METRICS:
- doraScore = __.__ (0-1)
- slaCompliance = __%
- incidentSeverity = P_/NONE (max open)
- rcaCompleteness = __%
- defectTrend = declining/stable/increasing
- defectDensity = __.__ per KLOC
- regressionCount = __ (if applicable, else NULL)
- chaosResilience = __ (if applicable, else NULL, 0-100)
- middlewareHealth = __ (if applicable, else NULL, 0-100)
- sapHealth = __ (if applicable, else NULL, 0-100)
- sodCompliance = __ (if applicable, else NULL, 0-100)

CRITICAL CHECK:
- incidentSeverity in [P0, P1]? __ (YES/NO)
- doraScore < 0.4? __ (YES/NO)
- slaCompliance < 95.0? __ (YES/NO)
- defectTrend == "increasing" AND defectDensity > 5.0? __ (YES/NO)
- middlewareHealth != NULL AND < 20? __ (YES/NO)
- sapHealth != NULL AND < 20? __ (YES/NO)
- sodCompliance != NULL AND < 20? __ (YES/NO)

HEALTHY CHECK (only if no CRITICAL triggered):
- doraScore >= 0.7? __ (YES/NO)
- slaCompliance >= 99.0? __ (YES/NO)
- incidentSeverity in [P3, P4, NONE]? __ (YES/NO)
- rcaCompleteness >= 80? __ (YES/NO)
- defectTrend in ["declining", "stable"]? __ (YES/NO)
- defectDensity <= 2.0? __ (YES/NO)
- regressionCount == NULL OR <= 2? __ (YES/NO)
- chaosResilience == NULL OR >= 80? __ (YES/NO)
- middlewareHealth == NULL OR >= 70? __ (YES/NO)
- sapHealth == NULL OR >= 70? __ (YES/NO)
- sodCompliance == NULL OR >= 70? __ (YES/NO)

FINAL RECOMMENDATION: [HEALTHY / DEGRADED / CRITICAL]
REASON: ___

Degraded Recommendations

If recommendation is DEGRADED, provide specific improvement actions:

If recommendation is CRITICAL, provide mandatory remediation steps:

Success Criteria

• [ ] All metrics derived from actual agent outputs
• [ ] CRITICAL conditions checked exhaustively
• [ ] HEALTHY conditions checked only if no CRITICAL triggered
• [ ] Decision recording completed with all fields filled
• [ ] FINAL RECOMMENDATION determined: HEALTHY, DEGRADED, or CRITICAL
• [ ] Rationale documented

Output

Provide to the next step:

• FINAL RECOMMENDATION (HEALTHY / DEGRADED / CRITICAL)
• Complete metrics summary
• Decision rationale
• Improvement/remediation actions (if DEGRADED or CRITICAL)

Navigation

• On success: proceed to Step 6 (Report Generation) by reading steps/06-report-generation.md
• On failure: if metrics are missing, return to the appropriate step to retrieve them

Step 6: Generate Production Health Report

Prerequisites

• Step 5 (Decision Synthesis) completed with FINAL RECOMMENDATION
• All agent reports saved to OUTPUT_FOLDER
• Complete metrics available

Instructions

ENFORCEMENT: COMPLETE REPORT STRUCTURE

ALL sections below are MANDATORY. No abbreviations.

Generate the following report and save it to ${OUTPUT_FOLDER}/01-executive-summary.md:

# QCSD Production Health Report: [Release Name/ID]

**Generated**: [Date/Time]
**Recommendation**: [HEALTHY / DEGRADED / CRITICAL]
**Agents Executed**: [List all agents that ran]
**Parallel Batches**: [3]
**Release ID**: [RELEASE_ID value]
**Telemetry Source**: [TELEMETRY_DATA path]

---

## Executive Summary

| Metric | Value | Threshold | Status |
|--------|-------|-----------|--------|
| DORA Score | X.XX | >= 0.7 (HEALTHY) | PASS/WARN/FAIL |
| SLA Compliance | X% | >= 99% (HEALTHY) | PASS/WARN/FAIL |
| Incident Severity | P_/NONE | P3/P4/NONE (HEALTHY) | PASS/WARN/FAIL |
| Defect Trend | declining/stable/increasing | declining/stable (HEALTHY) | PASS/WARN/FAIL |
| RCA Completeness | X% | >= 80% (HEALTHY) | PASS/WARN/FAIL |

**Recommendation Rationale**: [1-2 sentences explaining why HEALTHY/DEGRADED/CRITICAL]

---

## DORA Metrics Analysis

[EMBED or LINK the full report from qe-metrics-optimizer]

### DORA Dashboard

| Metric | Value | Classification |
|--------|-------|----------------|
[All 4 DORA metrics from qe-metrics-optimizer]

### SLA Compliance Matrix
[Key findings from agent output]

---

## Defect Prediction Analysis

[EMBED or LINK the full report from qe-defect-predictor]

### Trend Summary

| Dimension | Value | Risk |
|-----------|-------|------|
[Key metrics from agent output]

### Escape Analysis
[Which phases should have caught production defects]

---

## Root Cause Analysis

[EMBED or LINK the full report from qe-root-cause-analyzer]

### Incident Summary

| Metric | Value | Status |
|--------|-------|--------|
[Key metrics from agent output]

### Top Incidents
[Summary of highest-severity incidents]

---

## Conditional Analysis

[INCLUDE ONLY IF APPLICABLE - based on which conditional agents ran]

### Chaos Resilience (IF HAS_INFRASTRUCTURE_CHANGE)
[Full output from qe-chaos-engineer]

### Performance SLA (IF HAS_PERFORMANCE_SLA)
[Full output from qe-performance-tester]

### Regression Analysis (IF HAS_REGRESSION_RISK)
[Full output from qe-regression-analyzer]

### Pattern Analysis (IF HAS_RECURRING_INCIDENTS)
[Full output from qe-pattern-learner]

### Middleware Health (IF HAS_MIDDLEWARE)
[Full output from qe-middleware-validator]

### SAP Health (IF HAS_SAP_INTEGRATION)
[Full output from qe-sap-rfc-tester]

### SoD Compliance (IF HAS_AUTHORIZATION)
[Full output from qe-sod-analyzer]

---

## Feedback Loop Synthesis

[Full output from qe-learning-coordinator and qe-transfer-specialist - Step 8]

### Strategic Feedback (to Ideation)
[DORA trends + defect patterns for risk calibration]

### Tactical Feedback (to Refinement)
[RCA patterns + escape analysis for test strategy improvement]

---

## Recommended Actions

### Immediate Actions (P0 - Blockers)
- [ ] [Action based on findings]

### Short-Term Actions (P1 - Important)
- [ ] [Action based on findings]

### Long-Term Improvements (P2 - Improvement)
- [ ] [Action based on findings]

---

## Appendix: Agent Outputs

[Link to or embed full outputs from each agent]

---

*Generated by QCSD Production Swarm v1.0*
*Execution Model: Task Tool Parallel Swarm*

Report Validation Checklist

Before presenting report:

+-- Executive Summary table is complete with all 5 metrics
+-- Recommendation matches decision logic output
+-- DORA section includes all 4 metrics with classifications
+-- Defect section includes trend direction and escape analysis
+-- RCA section includes incident severity and completeness
+-- Conditional sections included for all spawned agents
+-- Feedback loop section present (will be filled in Step 8)
+-- Recommended actions are specific (not generic)
+-- Report saved to output folder

DO NOT present an incomplete report.

Success Criteria

• [ ] Executive summary with all 5 core metrics
• [ ] DORA, Defect, and RCA sections complete
• [ ] Conditional sections for all spawned agents
• [ ] Recommended actions are specific and actionable
• [ ] Report saved to ${OUTPUT_FOLDER}/01-executive-summary.md

Output

Provide to the next step:

• Confirmation that report is saved
• Complete metrics for learning persistence

Navigation

• On success: proceed to Step 7 (Learning Persistence) by reading steps/07-learning-persistence.md
• On failure: complete missing report sections before proceeding

Step 7: Store Learnings & Persist State

Prerequisites

• Step 6 (Report Generation) completed
• All metrics available from Steps 3-5
• All flag values from Step 1

Instructions

ENFORCEMENT: ALWAYS RUN THIS STEP

+-------------------------------------------------------------+
|  LEARNING PERSISTENCE MUST ALWAYS EXECUTE                    |
|                                                              |
|  This is NOT optional. It runs on EVERY production scan.     |
|  It stores findings for cross-phase feedback loops,          |
|  historical DORA tracking, and pattern learning.             |
|                                                              |
|  DO NOT skip this step for any reason.                       |
|  DO NOT treat this as "nice to have".                        |
|  Enforcement Rule E9 applies.                                |
+-------------------------------------------------------------+

Purpose

Store production findings for:

• Cross-phase feedback loops (Production -> Ideation and Refinement cycles)
• Historical DORA metric tracking across releases
• Defect trend analysis and prediction model improvement
• Incident pattern learning for recurring issue prevention

Auto-Execution Steps (ALL THREE are MANDATORY)

Step 1: Store production findings to memory

You MUST execute this MCP call with actual values from the production analysis:

aqe memory store \
  --key "qcsd-production-${releaseId}-${Date.now()}" \
  --namespace "qcsd-production" \
  --value '{...}' \
  --json

Step 2: Share learnings with feedback agents

aqe memory share \
  --from "qcsd-production-swarm" \
  --to "qe-learning-coordinator,qe-transfer-specialist" \
  --domain "production-health-patterns" \
  --content '{...}' --json

Step 3: Save learning persistence record to output folder

Save a JSON record to ${OUTPUT_FOLDER}/09-learning-persistence.json:

{
  "phase": "QCSD-Production",
  "releaseId": "[release ID]",
  "releaseName": "[release name]",
  "recommendation": "[HEALTHY/DEGRADED/CRITICAL]",
  "memoryKey": "qcsd-production-[releaseId]-[timestamp]",
  "namespace": "qcsd-production",
  "metrics": {
    "doraScore": "[0-1]",
    "slaCompliance": "[0-100]",
    "incidentSeverity": "[P0-P4 or NONE]",
    "rcaCompleteness": "[0-100]",
    "defectTrend": "[declining/stable/increasing]",
    "defectDensity": "[N.NN]",
    "regressionCount": "[N or null]",
    "chaosResilience": "[N or null]",
    "middlewareHealth": "[N or null]",
    "sapHealth": "[N or null]",
    "sodCompliance": "[N or null]"
  },
  "flags": {
    "HAS_INFRASTRUCTURE_CHANGE": "true/false",
    "HAS_PERFORMANCE_SLA": "true/false",
    "HAS_REGRESSION_RISK": "true/false",
    "HAS_RECURRING_INCIDENTS": "true/false",
    "HAS_MIDDLEWARE": "true/false",
    "HAS_SAP_INTEGRATION": "true/false",
    "HAS_AUTHORIZATION": "true/false"
  },
  "agentsInvoked": ["list", "of", "agents"],
  "crossPhaseSignals": {
    "toIdeation": "DORA trends and defect patterns for risk calibration and quality criteria updates",
    "toRefinement": "RCA patterns and escape analysis for test strategy improvement and BDD generation"
  },
  "persistedAt": "[ISO timestamp]"
}

Fallback: CLI Memory Commands

If MCP memory_store tool is unavailable, use CLI instead (STILL MANDATORY):

npx --no-install ruflo memory store \
  --key "qcsd-production-${RELEASE_ID}-$(date +%s)" \
  --value '{"recommendation":"[VALUE]","doraScore":[N],"slaCompliance":[N],"incidentSeverity":"[P_]","defectTrend":"[VALUE]"}' \
  --namespace qcsd-production

npx --no-install ruflo hooks post-task \
  --task-id "qcsd-production-${RELEASE_ID}" \
  --success true

Cross-Phase Signal Consumption

The Production Swarm both consumes and produces signals:

CONSUMES (from CI/CD phase):
+-- Loop 5 (CI/CD): RELEASE/REMEDIATE/BLOCK decisions
+-- Loop 6 (Pipeline History): Previous production assessments

PRODUCES (for other phases):
+-- To Ideation Phase (Strategic): DORA trends + defect patterns
+-- To Refinement Phase (Tactical): RCA patterns + escape analysis

Validation Before Proceeding

+-- Did I execute aqe memory store with actual values?
+-- Did I execute aqe memory share to propagate learnings?
+-- Did I save 09-learning-persistence.json to the output folder?
+-- Does the JSON contain the correct recommendation from Step 5?
+-- Does the JSON contain actual metrics from Steps 2-4?
+-- Does the JSON contain actual flag values from Step 1?
+-- Does the JSON contain crossPhaseSignals with toIdeation and toRefinement?

If ANY validation check fails, DO NOT proceed to Step 8.

Success Criteria

• [ ] Memory store executed with actual values
• [ ] Memory share executed to propagate learnings
• [ ] 09-learning-persistence.json saved with complete data
• [ ] Cross-phase signals documented

Output

Provide to the next step:

• Confirmation that learnings are persisted
• Summary of cross-phase signals for feedback agents

Navigation

• On success: proceed to Step 8 (Feedback Loop Closure) by reading steps/08-feedback-loop.md
• On failure: retry memory persistence before proceeding

Step 8: Feedback Loop Closure (Sequential Batch 3)

Prerequisites

• Step 7 (Learning Persistence) completed
• All production metrics and reports available
• Learnings persisted to memory

Instructions

ENFORCEMENT: ALWAYS RUN BOTH AGENTS IN SEQUENCE

+-------------------------------------------------------------+
|  BOTH FEEDBACK AGENTS MUST ALWAYS RUN -- SEQUENTIALLY        |
|                                                              |
|  This is NOT conditional. It runs on EVERY production scan.  |
|  qe-learning-coordinator synthesizes cross-domain learnings. |
|  qe-transfer-specialist transfers knowledge to target phases.|
|                                                              |
|  DO NOT skip either agent for any reason.                    |
|  DO NOT run only one of the two agents.                      |
|  Enforcement Rule E8 applies: BOTH agents, ALWAYS.           |
|                                                              |
|  CRITICAL DATA DEPENDENCY:                                   |
|  qe-transfer-specialist DEPENDS ON qe-learning-coordinator's |
|  output. They CANNOT run in parallel.                        |
+-------------------------------------------------------------+

SEQUENTIAL ENFORCEMENT

+-------------------------------------------------------------+
|  YOU MUST RUN THESE AGENTS SEQUENTIALLY (NOT IN PARALLEL)    |
|                                                              |
|  Step A: Spawn qe-learning-coordinator (ONE Task call)       |
|  Step B: WAIT for learning coordinator to complete            |
|  Step C: Read 13-feedback-loops.md produced by Step A        |
|  Step D: Spawn qe-transfer-specialist (ONE Task call)        |
|          with learning coordinator's output as input          |
|  Step E: WAIT for transfer specialist to complete             |
|                                                              |
|  qe-transfer-specialist DEPENDS on qe-learning-coordinator's |
|  saved output. Running them in parallel produces garbage.     |
+-------------------------------------------------------------+

Agent 1: Learning Coordinator

Spawn qe-learning-coordinator to synthesize all production findings into cross-domain learnings. This agent must:

1. Create a Learning Synthesis Matrix from ALL agents that ran 2. Consolidate cross-domain patterns 3. Produce Strategic Feedback signals for Ideation phase (DORA trends, defect patterns, SLA data, escape analysis) 4. Produce Tactical Feedback signals for Refinement phase (RCA patterns, escape analysis, hotspot data, regression patterns) 5. Calculate a Learning Quality Score (0-100)

Save output to: ${OUTPUT_FOLDER}/13-feedback-loops.md

WAIT for this agent to complete before spawning the transfer specialist.

Wait for Learning Coordinator

When the learning coordinator returns: 1. Use the Read tool to read ${OUTPUT_FOLDER}/13-feedback-loops.md 2. Extract the Learning Synthesis Matrix 3. Extract the Strategic Feedback signals 4. Extract the Tactical Feedback signals 5. THEN spawn qe-transfer-specialist with this data

Agent 2: Transfer Specialist

PREREQUISITE: qe-learning-coordinator MUST have completed and saved 13-feedback-loops.md.

Spawn qe-transfer-specialist to transfer synthesized production learnings to target QCSD phases. This agent must:

1. Create a Knowledge Transfer Plan mapping learnings to target agents 2. Map transfers to specific target agents in Ideation and Refinement 3. Verify transfer status 4. Assess feedback loop closure for all loops 5. Provide continuous improvement recommendations

Append output to: ${OUTPUT_FOLDER}/13-feedback-loops.md

Post-Completion Confirmation

After BOTH agents have completed sequentially:

Feedback loop closure complete (sequential execution):

  Step A: qe-learning-coordinator [Domain: learning-optimization] - COMPLETE
   - Synthesized cross-domain learnings from all production agents
   - Produced strategic feedback for Ideation phase
   - Produced tactical feedback for Refinement phase
   - Saved output to: 13-feedback-loops.md

  Step B: qe-transfer-specialist [Domain: learning-optimization] - COMPLETE
   - Created knowledge transfer plan to target agents
   - Verified feedback loop closure status
   - Mapped learnings to specific behavioral changes
   - Appended output to: 13-feedback-loops.md

  PROCEEDING to Step 9 (Final Output)...

Success Criteria

• [ ] qe-learning-coordinator spawned, completed, and saved 13-feedback-loops.md
• [ ] Learning coordinator output read before spawning transfer specialist
• [ ] qe-transfer-specialist spawned with learning coordinator data, completed
• [ ] Feedback loop closure status documented
• [ ] Both agents ran SEQUENTIALLY (not in parallel)

Output

Provide to the next step:

• Learning quality score
• Feedback loops closed count
• Transfer completion percentage

Navigation

• On success: proceed to Step 9 (Final Output) by reading steps/09-final-output.md
• On failure: if either agent failed, retry that agent before proceeding

Step 9: Final Output & Completion Summary

Prerequisites

• All previous steps (1-8) completed
• All reports saved to OUTPUT_FOLDER
• FINAL RECOMMENDATION determined
• Feedback loops closed

Instructions

At the very end of swarm execution, ALWAYS output this completion summary:

+---------------------------------------------------------------------+
|                  QCSD PRODUCTION SWARM COMPLETE                       |
+---------------------------------------------------------------------+
|                                                                      |
|  Release Assessed: [Release Name/ID]                                  |
|  Reports Generated: [count]                                           |
|  Output Folder: ${OUTPUT_FOLDER}                                     |
|                                                                      |
|  PRODUCTION HEALTH SCORES:                                            |
|  +-- DORA Score:              __.__ (Elite/High/Med/Low)             |
|  +-- SLA Compliance:          __%                                     |
|  +-- Incident Severity:       P_/NONE                                |
|  +-- Defect Trend:            declining/stable/increasing            |
|  +-- RCA Completeness:        __%                                     |
|  +-- Defect Density:          __.__ per KLOC                         |
|  +-- Learning Quality:        __/100                                   |
|  [IF HAS_INFRASTRUCTURE_CHANGE]                                       |
|  +-- Chaos Resilience:        __/100                                  |
|  [IF HAS_PERFORMANCE_SLA]                                             |
|  +-- Performance SLA:         __/100                                  |
|  [IF HAS_REGRESSION_RISK]                                             |
|  +-- Regression Score:        __/100                                  |
|  [IF HAS_RECURRING_INCIDENTS]                                         |
|  +-- Recurring Patterns:      __ identified                          |
|  [IF HAS_MIDDLEWARE]                                                  |
|  +-- Middleware Health:        __/100                                  |
|  [IF HAS_SAP_INTEGRATION]                                            |
|  +-- SAP Health:               __/100                                  |
|  [IF HAS_AUTHORIZATION]                                              |
|  +-- SoD Compliance:           __/100                                  |
|                                                                      |
|  FEEDBACK LOOPS:                                                      |
|  +-- To Ideation:             [X signals transferred]                |
|  +-- To Refinement:           [X signals transferred]                |
|  +-- Loop Closure:            X/Y closed                             |
|                                                                      |
|  RECOMMENDATION: [HEALTHY / DEGRADED / CRITICAL]                      |
|  REASON: [1-2 sentence rationale]                                     |
|                                                                      |
|  DELIVERABLES:                                                        |
|  +-- 01-executive-summary.md                                          |
|  +-- 02-dora-metrics.md                                               |
|  +-- 03-defect-prediction.md                                          |
|  +-- 04-root-cause-analysis.md                                        |
|  [IF HAS_INFRASTRUCTURE_CHANGE]                                       |
|  +-- 05-chaos-resilience.md                                           |
|  [IF HAS_PERFORMANCE_SLA]                                             |
|  +-- 06-performance-sla.md                                            |
|  [IF HAS_REGRESSION_RISK]                                             |
|  +-- 07-regression-analysis.md                                        |
|  [IF HAS_RECURRING_INCIDENTS]                                         |
|  +-- 08-pattern-analysis.md                                           |
|  +-- 09-learning-persistence.json                                     |
|  [IF HAS_MIDDLEWARE]                                                  |
|  +-- 10-middleware-health.md                                          |
|  [IF HAS_SAP_INTEGRATION]                                            |
|  +-- 11-sap-health.md                                                |
|  [IF HAS_AUTHORIZATION]                                              |
|  +-- 12-sod-compliance.md                                            |
|  +-- 13-feedback-loops.md                                             |
|                                                                      |
+---------------------------------------------------------------------+

IF recommendation is CRITICAL, ALSO output this prominent action box:

+---------------------------------------------------------------------+
|  ACTION REQUIRED: PRODUCTION CRITICAL - IMMEDIATE ATTENTION           |
+---------------------------------------------------------------------+
|                                                                      |
|  The following critical issues MUST be resolved immediately:          |
|                                                                      |
|  1. [Critical issue 1 with specific remediation]                      |
|  2. [Critical issue 2 with specific remediation]                      |
|  3. [Critical issue 3 with specific remediation]                      |
|                                                                      |
|  NEXT STEPS:                                                          |
|  - Activate incident response for all P0/P1 incidents                |
|  - Address all critical issues listed above                           |
|  - Consider rollback if production stability cannot be restored      |
|  - Re-run /qcsd-production-swarm after stabilization                 |
|  - Target: DORA >= 0.7, SLA >= 99%, no P0/P1, density <= 2.0        |
|                                                                      |
+---------------------------------------------------------------------+

IF recommendation is DEGRADED, output this guidance box:

+---------------------------------------------------------------------+
|  DEGRADED: PRODUCTION NEEDS ATTENTION                                 |
+---------------------------------------------------------------------+
|                                                                      |
|  The production environment is functional but requires improvement:   |
|                                                                      |
|  1. [Improvement 1 - must be addressed this sprint]                   |
|  2. [Improvement 2 - must be addressed next sprint]                   |
|                                                                      |
|  MONITORING STRATEGY:                                                 |
|  - Increase monitoring frequency for [specific metrics]              |
|  - Set tighter alert thresholds for [conditions]                     |
|  - Schedule follow-up production assessment in [timeframe]           |
|                                                                      |
|  FEEDBACK LOOP ACTIONS:                                               |
|  - Ideation: Update risk criteria based on production learnings      |
|  - Refinement: Add BDD scenarios for escaped defect patterns         |
|  - Development: Increase test coverage for identified hotspots       |
|                                                                      |
+---------------------------------------------------------------------+

DO NOT end the swarm without displaying the completion summary.

Success Criteria

• [ ] Completion summary displayed with all metrics filled
• [ ] Recommendation-specific action box shown (CRITICAL or DEGRADED)
• [ ] All deliverable files listed
• [ ] Feedback loop status reported

Output

This is the final step. The swarm execution is complete.

Navigation

• This is the terminal step. No further steps.
• To re-run: invoke /qcsd-production-swarm again
• To resume from a specific step: use --from-step N