Session Analyzer

Session Analyzer Skill

Post-hoc analysis tool for validating Claude Code session behavior against SKILL.md specifications.

Purpose

Analyze completed sessions to verify: 1. Expected vs Actual Behavior - Did the skill follow SKILL.md workflow? 2. Component Invocations - Were SubAgents, Hooks, and Tools called correctly? 3. Artifacts - Were expected files created/deleted? 4. Bug Detection - Any unexpected errors or deviations?

---

Input Requirements

---

Phase 1: Locate Session Files

Step 1.1: Find Session Files

Session files are located in ~/.claude/:

# Main session log
~/.claude/projects/-{encoded-cwd}/{sessionId}.jsonl

# Debug log (detailed)
~/.claude/debug/{sessionId}.txt

# Agent transcripts (if subagents were used)
~/.claude/projects/-{encoded-cwd}/agent-{agentId}.jsonl

Use script to locate files:

${baseDir}/scripts/find-session-files.sh {sessionId}

Step 1.2: Verify Files Exist

Check all required files exist before proceeding. If debug log is missing, analysis will be limited.

---

Phase 2: Parse Target SKILL.md

Step 2.1: Extract Expected Components

Read the target SKILL.md and identify:

From YAML Frontmatter:

• hooks.PreToolUse - Expected PreToolUse hooks and matchers
• hooks.PostToolUse - Expected PostToolUse hooks
• hooks.Stop - Expected Stop hooks
• hooks.SubagentStop - Expected SubagentStop hooks
• allowed-tools - Tools the skill is allowed to use

From Markdown Body:

• SubAgents mentioned (Task(subagent_type="..."))
• Skills called (Skill("..."))
• Artifacts created (.dev-flow/drafts/, .dev-flow/plans/, etc.)
• Workflow steps and conditions

Step 2.2: Build Expected Behavior Checklist

Create checklist from SKILL.md analysis:

## Expected Behavior

### SubAgents
- [ ] Explore agent called (parallel, run_in_background)
- [ ] gap-analyzer called before plan generation
- [ ] reviewer called after plan creation

### Hooks
- [ ] PreToolUse[Edit|Write] triggers plan-guard.sh
- [ ] Stop hook validates reviewer approval

### Artifacts
- [ ] Draft file created at .dev-flow/drafts/{name}.md
- [ ] Plan file created at .dev-flow/plans/{name}.md
- [ ] Draft file deleted after OKAY

### Workflow
- [ ] Interview Mode before Plan Generation
- [ ] User explicit request triggers plan generation
- [ ] Reviewer REJECT causes revision loop

---

Phase 3: Analyze Debug Log

The debug log (~/.claude/debug/{sessionId}.txt) contains detailed execution traces.

Step 3.1: Extract SubAgent Calls

Search patterns:

SubagentStart with query: {agent-name}
SubagentStop with query: {agent-id}

Use script:

${baseDir}/scripts/extract-subagent-calls.sh {debug-log-path}

Step 3.2: Extract Hook Events

Search patterns:

Getting matching hook commands for {HookEvent} with query: {tool-name}
Matched {N} unique hooks for query "{query}"
Hooks: Processing prompt hook with prompt: {prompt}
Hooks: Prompt hook condition was met/not met
permissionDecision: allow/deny

Use script:

${baseDir}/scripts/extract-hook-events.sh {debug-log-path}

Step 3.3: Extract Tool Calls

Search patterns:

executePreToolHooks called for tool: {tool-name}
File {path} written atomically

Step 3.4: Extract Hook Results

For prompt-based hooks, find the model response:

Hooks: Model response: {
  "ok": true/false,
  "reason": "..."
}

---

Phase 4: Verify Artifacts

Step 4.1: Check File Creation

For each expected artifact: 1. Search debug log for FileHistory: Tracked file modification for {path} 2. Search for File {path} written atomically 3. Verify current filesystem state

Step 4.2: Check File Deletion

For files that should be deleted: 1. Search for rm commands in Bash calls 2. Verify file no longer exists on filesystem

---

Phase 5: Compare Expected vs Actual

Step 5.1: Build Comparison Table

| Component | Expected | Actual | Status |
|-----------|----------|--------|--------|
| Explore agent | 2 parallel calls | 2 calls at 09:39:26 | ✅ |
| gap-analyzer | Called before plan | Called at 09:43:08 | ✅ |
| reviewer | Called after plan | 2 calls (REJECT→OKAY) | ✅ |
| PreToolUse hook | Edit\|Write matcher | Triggered for Write | ✅ |
| Stop hook | Validates approval | Returned ok:true | ✅ |
| Draft file | Created then deleted | Created→Deleted | ✅ |
| Plan file | Created | Exists (10KB) | ✅ |

Step 5.2: Identify Deviations

Flag any mismatches:

• Missing component calls
• Wrong order of operations
• Hook failures
• Missing artifacts
• Unexpected errors

---

Phase 6: Generate Report

Report Template

# Session Analysis Report

## Session Info
- **Session ID**: {sessionId}
- **Target Skill**: {skillPath}
- **Analysis Date**: {date}

---

## 1. Expected Behavior (from SKILL.md)

[Summary of expected workflow]

---

## 2. Skill/SubAgent/Hook Verification

### SubAgents
| SubAgent | Expected | Actual | Time | Result |
|----------|----------|--------|------|--------|
| ... | ... | ... | ... | ✅/❌ |

### Hooks
| Hook | Matcher | Triggered | Result |
|------|---------|-----------|--------|
| ... | ... | ... | ✅/❌ |

---

## 3. Artifacts Verification

| Artifact | Path | Expected State | Actual State |
|----------|------|----------------|--------------|
| ... | ... | ... | ✅/❌ |

---

## 4. Issues/Bugs

| Severity | Description | Location |
|----------|-------------|----------|
| ... | ... | ... |

---

## 5. Overall Result

**Verdict**: ✅ PASS / ❌ FAIL

**Summary**: [1-2 sentence summary]

---

Scripts Reference

---

Usage Example

User: "Analyze session 3cc71c9f-d27a-4233-9dbc-c4f07ea6ec5b against .claude/skills/specify/SKILL.md"

1. Find session files
2. Parse SKILL.md → Expected: Explore, gap-analyzer, reviewer, hooks
3. Analyze debug log → Extract actual calls
4. Verify artifacts → Check .dev-flow/
5. Compare → Build verification table
6. Generate report → PASS/FAIL with details

---

Additional Resources

Reference Files

• `references/analysis-patterns.md` - Detailed grep patterns for log analysis
• `references/common-issues.md` - Known issues and troubleshooting

Scripts

• `scripts/find-session-files.sh` - Session file locator
• `scripts/extract-subagent-calls.sh` - SubAgent call extractor
• `scripts/extract-hook-events.sh` - Hook event extractor

Analysis Patterns for Session Analyzer

Detailed grep/search patterns for extracting information from Claude Code debug logs.

---

Debug Log Structure

Debug logs are located at ~/.claude/debug/{sessionId}.txt and contain timestamped entries:

2026-01-13T09:39:26.905Z [DEBUG] {message}

---

SubAgent Patterns

SubAgent Start

# Pattern
grep "SubagentStart with query:" debug.txt

# Example output
2026-01-13T09:39:26.905Z [DEBUG] Getting matching hook commands for SubagentStart with query: Explore

SubAgent Stop

# Pattern
grep "SubagentStop with query:" debug.txt

# With agent ID (session tracking)
grep "agent_id.*agent_transcript_path" debug.txt

SubAgent Session Registration

# Pattern - shows when hooks are registered for subagent
grep "Registered.*frontmatter hook.*from agent" debug.txt

# Example
2026-01-13T09:43:08.203Z [DEBUG] Registered 1 frontmatter hook(s) from agent 'gap-analyzer' for session a373157

---

Hook Patterns

PreToolUse Hook Trigger

# Pattern
grep "executePreToolHooks called for tool:" debug.txt

# Example
2026-01-13T09:39:40.000Z [DEBUG] executePreToolHooks called for tool: Write

Hook Matcher Check

# Pattern
grep "Getting matching hook commands for PreToolUse with query:" debug.txt

# With match count
grep "Matched.*unique hooks for query" debug.txt

# Example
2026-01-13T09:39:40.000Z [DEBUG] Matched 1 unique hooks for query "Write" (1 before deduplication)

Hook Permission Decision

# Pattern
grep "permissionDecision" debug.txt

# Example (allow)
"permissionDecision": "allow"

# Example (deny)
"permissionDecision": "deny"

Prompt-Based Hook Processing

# Pattern - hook is being processed
grep "Hooks: Processing prompt hook with prompt:" debug.txt

# Pattern - model response
grep "Hooks: Model response:" debug.txt

# Pattern - condition result
grep "Prompt hook condition was" debug.txt

# Example (met)
2026-01-13T09:48:09.076Z [DEBUG] Hooks: Prompt hook condition was met

# Example (not met)
2026-01-13T09:45:59.297Z [DEBUG] Hooks: Prompt hook condition was not met: REJECT - ...

Stop Hook Events

# Pattern
grep "Getting matching hook commands for Stop" debug.txt

SubagentStop Hook Events

# Pattern - converted from Stop to SubagentStop
grep "Converting Stop hook to SubagentStop" debug.txt

# Example
2026-01-13T09:43:08.202Z [DEBUG] Converting Stop hook to SubagentStop for agent 'gap-analyzer'

---

Tool Usage Patterns

Tool Execution

# Pattern
grep "executePreToolHooks called for tool:" debug.txt

File Write Operations

# Pattern - file creation/modification
grep "FileHistory: Tracked file modification for" debug.txt

# Pattern - atomic write
grep "File.*written atomically" debug.txt

# Example
2026-01-13T09:39:40.036Z [DEBUG] File /path/to/file.md written atomically

Bash Command Execution

# Pattern - PreToolHooks for Bash
grep "executePreToolHooks called for tool: Bash" debug.txt

---

Skill/Session Patterns

Skill Loading

# Pattern - skill hooks registered
grep "Added session hook for event" debug.txt
grep "Registered.*hooks from skill" debug.txt

# Example
2026-01-13T09:39:14.449Z [DEBUG] Added session hook for event PreToolUse in session 3cc71c9f-...
2026-01-13T09:39:14.449Z [DEBUG] Registered 2 hooks from skill 'specify'

Session Hook Cleanup

# Pattern
grep "Cleared all session hooks for session" debug.txt

---

AskUserQuestion Patterns

# Pattern - PreToolHooks
grep "executePreToolHooks called for tool: AskUserQuestion" debug.txt

# Pattern - PostToolHooks
grep "PostToolUse with query: AskUserQuestion" debug.txt

---

Error Patterns

Hook Errors

# Pattern
grep -i "error\|failed\|exception" debug.txt | grep -i hook

Tool Errors

# Pattern
grep "Tool.*error\|Tool.*failed" debug.txt

---

Reviewer-Specific Patterns

Reviewer Verdict Extraction

# Pattern - look for model response containing OKAY or REJECT
grep -A5 "Hooks: Model response:" debug.txt | grep -E '"ok":|"reason":'

# Example (OKAY)
{
  "ok": true,
  "reason": "Plan approved by reviewer..."
}

# Example (REJECT)
{
  "ok": false,
  "reason": "REJECT - The plan has a critical contradiction..."
}

---

Artifact Patterns

Draft File Operations

# Pattern - draft creation
grep "\.dev-flow/drafts/" debug.txt | grep "written atomically"

# Pattern - draft deletion (look for rm command)
grep "rm.*\.dev-flow/drafts/" debug.txt

Plan File Operations

# Pattern - plan creation
grep "\.dev-flow/plans/" debug.txt | grep "written atomically"

---

Timeline Reconstruction

To reconstruct a session timeline:

# Extract all timestamped events for key operations
grep -E "(SubagentStart|SubagentStop|executePreToolHooks|Prompt hook condition|written atomically)" debug.txt | sort

---

Combined Analysis Query

Full analysis of a specify skill session:

# 1. Check Explore agents
grep "SubagentStart with query: Explore" debug.txt | wc -l

# 2. Check gap-analyzer
grep "SubagentStart with query: gap-analyzer" debug.txt

# 3. Check reviewer calls and results
grep -E "(SubagentStart with query: reviewer|Prompt hook condition)" debug.txt

# 4. Check plan-guard.sh hook
grep "permissionDecision" debug.txt

# 5. Check artifacts
grep -E "(\.dev-flow/drafts/|\.dev-flow/plans/).*written atomically" debug.txt

# 6. Final Stop hook result
grep -A10 "Getting matching hook commands for Stop" debug.txt | tail -20

Common Issues and Troubleshooting

Known issues when analyzing Claude Code sessions and how to diagnose them.

---

Session File Issues

Issue: Debug Log Not Found

Symptom: ~/.claude/debug/{sessionId}.txt doesn't exist

Possible Causes: 1. Session was too old and debug logs were cleaned up 2. Debug logging was disabled 3. Session ID is incorrect

Workaround:

• Use the main session log (.jsonl) for limited analysis
• Main log contains tool calls but not detailed hook execution

Issue: Large Debug Log (>50MB)

Symptom: Log file too large to read entirely

Solution:

• Use grep with specific patterns instead of reading entire file
• Use tail to get recent entries
• Use offset and limit when reading with Read tool

---

SubAgent Analysis Issues

Issue: SubAgent Not Recorded

Symptom: Expected subagent call not found in logs

Possible Causes: 1. SubAgent was never actually called 2. SubAgent type name differs from expected (case-sensitive) 3. Log was truncated

Diagnosis:

# List all unique subagent types
grep "SubagentStart with query:" debug.txt | sed 's/.*query: //' | sort | uniq

Issue: Missing SubAgent Result

Symptom: SubagentStart found but no SubagentStop

Possible Causes: 1. SubAgent is still running (background task) 2. SubAgent crashed 3. Session ended before subagent completed

Diagnosis:

# Count starts vs stops
grep -c "SubagentStart" debug.txt
grep -c "SubagentStop" debug.txt

---

Hook Analysis Issues

Issue: Hook Not Triggered

Symptom: Expected hook not found in Getting matching hook commands entries

Possible Causes: 1. Matcher pattern doesn't match the tool name 2. Hook not registered (skill not loaded) 3. Hook has wrong event type

Diagnosis:

# Check if skill hooks were registered
grep "Registered.*hooks from skill" debug.txt

# Check what hooks are being queried
grep "Getting matching hook commands for" debug.txt | head -20

Issue: Hook Triggered But No Effect

Symptom: Hook matched (count > 0) but expected behavior didn't occur

Possible Causes: 1. Hook script returned error 2. Hook returned allow when should have returned deny 3. Prompt hook condition was not met

Diagnosis:

# Check hook execution result
grep -A5 "Matched.*unique hooks" debug.txt | grep -E "permissionDecision|ok"

Issue: Prompt Hook Always Returns False

Symptom: Prompt hook condition was not met consistently

Possible Causes: 1. Prompt is too vague for model to understand 2. Context doesn't contain expected information 3. Model misinterprets the criteria

Diagnosis:

# See the full model response
grep -A20 "Hooks: Model response:" debug.txt

---

Artifact Issues

Issue: File Not Created

Symptom: Expected artifact file not in written atomically logs

Possible Causes: 1. Write was blocked by PreToolUse hook 2. Path was wrong 3. Write tool was never called

Diagnosis:

# Check if Write was attempted
grep "executePreToolHooks called for tool: Write" debug.txt

# Check permission decision
grep -A10 "executePreToolHooks called for tool: Write" debug.txt | grep "permissionDecision"

Issue: File Exists But Should Be Deleted

Symptom: Draft file still exists after session ended

Possible Causes: 1. Bash rm command was never executed 2. Skill ended before cleanup step 3. Wrong file path in rm command

Diagnosis:

# Check for rm commands
grep "Bash" debug.txt | grep -i "rm"

---

Reviewer-Specific Issues

Issue: Reviewer Never Returns OKAY

Symptom: Multiple REJECT responses, no OKAY

Possible Causes: 1. Plan genuinely has issues that weren't fixed 2. Reviewer criteria too strict 3. Plan edits not addressing reviewer feedback

Diagnosis:

# Extract all reviewer responses
grep -B2 -A10 "Hooks: Model response:" debug.txt | grep -E '"ok"|"reason"'

Issue: Reviewer Called But No Hook Result

Symptom: SubagentStart with query: reviewer found but no Prompt hook condition result

Possible Causes: 1. Reviewer subagent has no Stop hook configured 2. Hook conversion to SubagentStop failed 3. Reviewer is still running

Diagnosis:

# Check if Stop hook was converted
grep "Converting Stop hook to SubagentStop for agent 'reviewer'" debug.txt

---

Timing Issues

Issue: Events Out of Order

Symptom: Timeline doesn't make sense (e.g., Stop before Start)

Possible Causes: 1. Parallel operations (intended behavior) 2. Log entries from different sessions mixed 3. Clock synchronization issues

Solution:

• Filter by session ID if multiple sessions in same timeframe
• Look at specific operation sequences, not global order

Issue: Large Time Gaps

Symptom: Long pauses between operations

Possible Causes: 1. User interaction (AskUserQuestion waiting) 2. API rate limiting 3. Model thinking time

Diagnosis:

# Find gaps > 30 seconds
awk -F'T|Z' '{print $2}' debug.txt | sort | uniq -c | sort -rn | head

---

Analysis Script Issues

Issue: Script Returns Empty JSON

Symptom: Scripts return { "summary": { "total": 0 } }

Possible Causes: 1. Debug log path is wrong 2. Log format changed 3. No matching events in this session

Solution:

• Verify debug log path exists and has content
• Manually grep for expected patterns to verify format

Issue: Script Permission Denied

Symptom: Permission denied when running scripts

Solution:

chmod +x scripts/*.sh

---

Validation Checklist

When analysis seems wrong, verify:

1. Correct Session ID: Double-check the UUID 2. Files Exist: Run find-session-files.sh first 3. Skill Was Loaded: Look for "Registered.hooks from skill" 4. Right Timeframe: Check timestamps match expected session time 5. Complete Session*: Session ended normally (not interrupted)