Failure Analysis

Failure Analysis

A technology-agnostic skill for classifying test failures and guiding root cause investigation.

Error vs Failure

classification:
  ERROR:
    description: "Test couldn't complete execution"
    causes:
      - "Missing dependencies"
      - "Configuration issues"
      - "Environment problems"
      - "Syntax/compilation errors"
    action: "Fix infrastructure before debugging test"

  FAILURE:
    description: "Test ran but assertion didn't pass"
    causes:
      - "Code behavior doesn't match expectation"
      - "Business logic incorrect"
      - "Edge case not handled"
    action: "Debug production code or update test"

Failure Categories

failure_categories:
  assertion_failure:
    pattern: "Expected X but got Y"
    analysis:
      - "Is the expected value correct?"
      - "Is the production code wrong?"
      - "Has the requirement changed?"

  exception_failure:
    pattern: "Unexpected exception thrown"
    analysis:
      - "What triggered the exception?"
      - "Is the exception handling missing?"
      - "Is the test data invalid?"

  timeout_failure:
    pattern: "Test exceeded time limit"
    analysis:
      - "Is there an infinite loop?"
      - "Is external service slow/unavailable?"
      - "Is the timeout too short?"

  setup_failure:
    pattern: "Fixture/setup failed"
    analysis:
      - "Is the database schema current?"
      - "Are required services running?"
      - "Is the test data valid?"

Trend Indicators

trend_indicators:
  flaky_tests:
    detection: "Same test passes/fails without code changes"
    action: "Investigate race conditions or external dependencies"

  slow_tests:
    detection: "Test duration increasing over time"
    action: "Profile and optimize or parallelize"

  cascading_failures:
    detection: "Many tests fail from single cause"
    action: "Fix root cause first"

Handoff to Debugger

debugger_handoff:
  when: "Failures detected AND category is FAILURE"
  provide:
    - "Failed test list"
    - "Error messages and stack traces"
    - "Test input data"
    - "Expected vs actual results"

  exclude:
    - "Passing test details"
    - "Irrelevant log entries"
    - "Unrelated file contents"

Used By Agents

primary_users:
  - test-failure-debugger: "Failure classification and root cause investigation"