
Dream
- 17 installs
- 25 repo stars
- Updated July 31, 2026
- hyperb1iss/hyperskills
Reviews recent Claude Code and Codex conversations and consolidates extracted decisions, patterns, and corrections into the Sibyl memory store.
About
Runs a two-phase sleep cycle that extracts structured knowledge from past conversations and consolidates it into Sibyl. A developer uses it to capture decisions, patterns, and anti-patterns before sessions scroll off.
- NREM consolidates, REM discovers (bio-inspired phases)
- Dedup discipline: every write checked against existing entries
Dream by the numbers
- 17 all-time installs (skills.sh)
- +1 installs in the week ending Aug 2, 2026 (Skillselion tracking)
- Ranked #10,886 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 2, 2026 (Skillselion catalog sync)
npx skills add https://github.com/hyperb1iss/hyperskills --skill dreamAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 17 |
|---|---|
| repo stars | ★ 25 |
| Last updated | July 31, 2026 |
| Repository | hyperb1iss/hyperskills ↗ |
What it does
Reviews recent Claude Code and Codex conversations and consolidates extracted decisions, patterns, and corrections into the Sibyl memory store.
Files
Dream: Conversation Review & Knowledge Consolidation
Bio-inspired two-phase sleep cycle that reviews Claude Code and Codex conversations, extracts structured knowledge, and consolidates it into Sibyl. Like biological dreaming: NREM consolidates, REM discovers.
Core insight: Conversations contain ~10x more knowledge than what gets manually captured. Dreams extract decisions, patterns, corrections, anti-patterns, and open questions that would otherwise vanish when sessions scroll off.
How to read this skill: the phases below describe the rhythm of a useful dream cycle, not a procedure to march through. Quick naps compress most of it, deep sleeps stretch it out. The non-negotiable bits are extraction quality (Sibyl entries that meet the bar in references/extraction-guide.md) and dedup discipline (every write checked against existing entries). Process shape adapts; quality bar doesn't.
The Shape
digraph dream {
rankdir=TB;
node [shape=box];
"1. ORIENT" [style=filled, fillcolor="#e8e8ff"];
"2. HARVEST" [style=filled, fillcolor="#ffe8e8"];
"3. NREM: Consolidate" [style=filled, fillcolor="#e8ffe8"];
"4. REM: Explore" [style=filled, fillcolor="#fff8e0"];
"5. REPORT" [style=filled, fillcolor="#e8e8ff"];
"1. ORIENT" -> "2. HARVEST";
"2. HARVEST" -> "3. NREM: Consolidate";
"3. NREM: Consolidate" -> "4. REM: Explore";
"4. REM: Explore" -> "5. REPORT";
}Depth Modes
| Mode | Sessions | Focus | When |
|---|---|---|---|
| Quick nap | Last 1-3 | Extract from today's work | End of day, /dream quick |
| Full sleep | Last 5-15 | Standard consolidation cycle | Default /dream |
| Deep sleep | All since last dream | Cross-project synthesis + REM | /dream deep |
| Lucid | Specific session(s) | Targeted extraction | /dream <session-id> |
---
Phase 1: ORIENT
Get the lay of the land before harvesting. Re-processing already-dreamed sessions wastes tokens and creates duplicate entries.
Common moves
1. Check dream state: when was the last dream cycle?
# Check Claude's auto-dream lock
stat ~/.claude/projects/*/memory/.consolidate-lock 2>/dev/null | grep -A1 "Modify"
# Check Sibyl for recent dream entries
sibyl search "dream report" --type episode --limit 32. Discover conversation sources:
Claude Code sessions:
# Find recent sessions across ALL projects (last 7 days)
find ~/.claude/projects -name "*.jsonl" -not -path "*/subagents/*" -mtime -7 -exec ls -lt {} + | head -30Codex sessions:
# Find recent Codex rollouts
find ~/.codex/sessions -name "rollout-*.jsonl" -mtime -7 -exec ls -lt {} + | head -303. Count the harvest:
- How many sessions since last dream?
- Which projects were active?
- Any notably long or complex sessions? (file size > 100KB = rich conversation)
4. Set dream scope based on depth mode and available sessions.
---
Phase 2: HARVEST
Read conversations and identify extractable knowledge. The trick is reading targeted segments rather than entire JSONL files; most session content is routine, and only specific patterns carry transferable signal.
Reading Claude Code sessions
Claude Code JSONL files contain one JSON object per line. Key message types to look for:
| Content Type | Where to Find | What to Extract |
|---|---|---|
| User corrections | User messages following assistant errors | Anti-patterns, wrong assumptions |
| Technical decisions | Assistant text blocks with rationale | Decision + alternatives considered |
| Tool invocations | tool_use blocks (Bash, Edit, etc.) | Commands that worked, error patterns |
| Debugging chains | Sequences of failed → fixed attempts | Error patterns, root causes |
| Architecture discussion | Longer text blocks with design reasoning | Patterns, system relationships |
| Thinking blocks | type: "thinking" content | Reasoning chains, hidden insights |
Extraction strategy, don't read whole files. Use targeted python extraction:
# Extract all user prompts (most reliable method)
python3 -c "
import json, sys
with open('session.jsonl') as f:
for line in f:
obj = json.loads(line)
if obj.get('type') == 'user':
content = obj.get('message', {}).get('content', '')
if isinstance(content, str) and len(content) > 20 and not content.startswith('<'):
print(content[:300])
"
# Extract assistant decisions and rationale
python3 -c "
import json
with open('session.jsonl') as f:
for line in f:
obj = json.loads(line)
if obj.get('type') == 'assistant':
for block in obj.get('message', {}).get('content', []):
if isinstance(block, dict) and block.get('type') == 'text':
text = block['text']
if any(kw in text.lower() for kw in ['because', 'root cause', 'the issue', 'approach', 'trade-off']):
if len(text) > 100:
print(text[:400])
print('---')
"
# Get session titles (best way to understand session topics at a glance)
for f in ~/.claude/projects/-Users-bliss-dev-*/*.jsonl; do
title=\$(grep -m1 '"ai-title"' "\$f" 2>/dev/null | python3 -c "import sys,json; print(json.loads(next(sys.stdin)).get('aiTitle',''))" 2>/dev/null)
[[ -n "\$title" ]] && echo "\$(du -h "\$f" | cut -f1) \$(basename "\$(dirname "\$f")"): \$title"
done | sort -rh | head -20Why python over grep: Claude Code JSONL has nested JSON structures (content arrays inside message objects). Simple grep patterns like '"role":"user"' match across the entire line, producing false positives from assistant messages that quote user content. Python parsing is slower but precise. Use grep only for initial signal scoring (counts), then python for actual extraction.
For promising sessions (high correction count, long duration, many tool calls), read key segments more deeply using Read tool on the JSONL file with offset/limit.
Reading Codex Sessions
Codex rollouts at ~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl use a different format. See references/conversation-formats.md for the full schema.
# Find Codex sessions with substantial content
find ~/.codex/sessions -name "rollout-*.jsonl" -mtime -7 -size +10k
# Get Codex session metadata (cwd, branch, model)
python3 -c "
import json
with open('rollout.jsonl') as f:
for line in f:
obj = json.loads(line)
if obj.get('type') == 'session_meta':
p = obj['payload']
print(f'cwd: {p.get(\"cwd\")}')
print(f'model: {p.get(\"model_provider\")}')
print(f'branch: {p.get(\"git\", {}).get(\"branch\")}')
break
"
# Extract user messages from Codex (payload.role == 'user')
python3 -c "
import json
with open('rollout.jsonl') as f:
for line in f:
obj = json.loads(line)
if obj.get('type') == 'response_item':
p = obj.get('payload', {})
if p.get('role') == 'user':
for c in p.get('content', []):
if c.get('type') == 'input_text':
text = c['text']
if not text.startswith('#') and not text.startswith('<') and len(text) > 20:
print(text[:200])
"
# Extract function calls
grep '"function_call"' rollout.jsonl | grep -v '"function_call_output"'Signal Scoring
Prioritize sessions for deep reading:
| Signal | Score | How to Detect |
|---|---|---|
| User corrections present | +3 | grep for negation words in user messages |
| Multiple error-fix cycles | +2 | tool_use errors followed by successful retries |
| Long session (>50 messages) | +1 | line count of JSONL |
| Cross-project references | +2 | mentions of other project paths |
| Architecture/design discussion | +2 | grep for design keywords |
| New library/tool adoption | +2 | grep for "install", "add", package names |
| Simple Q&A session | -1 | Short session with no tool calls |
Process top-scored sessions first; quick nap mode usually caps at the top 3. Low-signal sessions can be skipped entirely. Extracting from a Q&A session about syntax produces noise, not knowledge.
---
Phase 3: NREM, Structured Consolidation
Transform raw conversation signal into structured Sibyl entities. This is where the quality bar matters most: a duplicate-laden, vague-titled Sibyl is worse than a smaller, sharper one.
Extraction categories
For each significant finding, classify and write to Sibyl:
1. Decisions (→ Sibyl episode with category decision)
sibyl add "Decision: [what was decided]" \
"[rationale]. Alternatives considered: [list]. Context: [project/feature]. Date: [date]." \
--type episode --category decision --tags "project:[name]"What qualifies: Any technical choice with trade-offs, library selection, architecture pattern, API design, configuration approach.
2. Patterns (→ Sibyl pattern)
sibyl add "Pattern: [name]" \
"[description]. When to use: [context]. Example: [brief code/approach]. Discovered in: [project]." \
--type pattern --category "[domain]" --tags "project:[name]" --languages "[lang]"What qualifies: Reusable approaches that worked well. The bar: would this be useful in a different project?
3. Corrections / Anti-Patterns (→ Sibyl error_pattern)
sibyl add "Anti-pattern: [what went wrong]" \
"Wrong approach: [what was tried]. Why it failed: [root cause]. Correct approach: [what worked]. Context: [project]." \
--type error_pattern --category "[domain]" --tags "project:[name]"What qualifies: Mistakes that were corrected. The user said "no" or "that's wrong" or something broke and was debugged.
4. Rules (→ Sibyl rule)
sibyl add "Rule: [the rule]" \
"[explanation]. Why: [rationale]. Applies to: [scope]. Discovered: [date]." \
--type rule --category "[domain]" --tags "project:[name]"What qualifies: Hard constraints discovered through experience. "Always X when Y." "Never Z because W."
5. Open Questions / Tensions (→ Sibyl episode with category tension)
sibyl add "Tension: [the unresolved question]" \
"Context: [what prompted this]. Options considered: [list]. Blocking: [what it blocks]. Needs: [what would resolve it]." \
--type episode --category tension --tags "project:[name]"What qualifies: Questions that were raised but not answered. Contradictions between approaches. Deferred decisions.
Deduplication
Before writing any entity to Sibyl, check for existing similar entries. This bit is non-negotiable; the value of the graph collapses when duplicates accumulate.
sibyl search "[entity title keywords]" --type [type] --limit 5| Finding | Action |
|---|---|
| No similar entries | Create new entity |
| Similar but older entry | Update existing if new info supersedes, or add relationship |
| Exact duplicate | Skip, log in dream report |
| Contradictory entry | Create tension entity linking both |
Batch Processing
For efficiency, accumulate extractions and write them in batches:
1. Read and extract from all harvested sessions 2. Deduplicate the extraction set itself (multiple sessions may contain the same insight) 3. Check each against Sibyl 4. Write new entities 5. Track what was written for the dream report
---
Phase 4: REM, Creative Exploration
Only in deep mode. Find unexpected connections across projects, the cross-pollination phase that biological REM is named after.
Cross-Project Pattern Detection
# What patterns exist across multiple projects?
sibyl explore --type pattern --limit 50
# What error patterns keep recurring?
sibyl explore --type error_pattern --limit 30
# What tensions are unresolved?
sibyl search "tension" --type episode --limit 20Connection Discovery
Look for:
1. Pattern reuse: A pattern from project A that would solve a problem in project B 2. Contradictory approaches: Project A does X one way, project B does it differently, which is right? 3. Shared infrastructure gaps: Multiple projects hitting the same limitation 4. Knowledge transfer: Something learned in one domain that applies to another
For each discovered connection:
# Record the cross-project insight
sibyl add "Cross-project: [insight]" \
"[description]. Connects: [project A] and [project B]. Implication: [what to do about it]." \
--type episode --category cross-project --tags "project:[A],project:[B]"Staleness Detection
# Find old entities that may be outdated
sibyl explore --type pattern,rule --limit 100For each entity older than 90 days:
- Is the project still active? (check git log)
- Has the technology changed? (check versions)
- Does the pattern still apply? (check current code)
Mark stale entities:
sibyl entity update <entity-id> --tags "stale,needs-review"Importance Decay
Score existing entities by: base_importance * recency_factor * reference_count
- Entities referenced in recent sessions → boost
- Entities not referenced in 60+ days → flag for review
- Entities contradicted by newer findings → mark as superseded
---
Phase 5: REPORT
Generate a dream summary and record the cycle itself. The report serves two audiences: the user (so they can see what landed) and future dreams (which check this entry to avoid re-processing).
Dream Report Structure
## Dream Report: [date]
### Sessions Reviewed
- [count] Claude Code sessions across [count] projects
- [count] Codex sessions
- Time span: [earliest] to [latest]
- Projects: [list]
### Knowledge Extracted
- **[N] decisions** recorded
- **[N] patterns** discovered/updated
- **[N] anti-patterns** captured
- **[N] rules** established
- **[N] tensions** identified
### Highlights
1. [Most significant finding, 1-2 sentences]
2. [Second most significant]
3. [Third most significant]
### Cross-Project Insights (deep mode only)
- [Connection discovered between projects]
- [Pattern that applies more broadly than originally thought]
### Stale Knowledge Flagged
- [Entity that may need review]
### Dream Metrics
- Sessions processed: [N]
- Entities created: [N]
- Entities updated: [N]
- Duplicates skipped: [N]
- Sibyl calls: [N]Record the Dream
# Record the dream cycle itself
sibyl add "Dream Report: [date]" \
"[full dream report content]" \
--type episode --category dream-report --tags "dream,maintenance"Update Memory Files (Optional)
If significant learnings should be immediately available to Claude Code sessions (not just via Sibyl search), write key findings to the relevant project's memory:
# Only for high-impact findings that affect session behavior
# Most knowledge should live in Sibyl, not flat files---
Quick Nap Mode
For fast end-of-day processing:
1. Find today's sessions (Claude + Codex) 2. Grep for corrections and errors only 3. Extract the top 3-5 findings 4. Write to Sibyl 5. One-paragraph dream report
Skip: REM phase, staleness detection, cross-project analysis, memory file updates.
---
Integration Notes
Sibyl Is the Primary Store
Everything goes to Sibyl, not memory/\*.md files. Sibyl provides:
- Semantic search (vector + BM25)
- Relationship modeling (entity connections)
- Temporal awareness (when things were learned)
- Cross-project visibility (shared graph)
- Multi-machine access (network service)
Memory files are only updated for critical session-level behaviors that need to be in Claude Code's native context window.
Conversation Formats
See references/conversation-formats.md for:
- Claude Code JSONL schema (TranscriptMessage types, content blocks)
- Codex rollout JSONL schema (session_meta, response_item, event_msg, turn_context)
- Useful grep patterns for each format
Extraction Quality
See references/extraction-guide.md for:
- What makes a good vs bad extraction
- Sibyl entity type selection guide
- Deduplication strategies
- Examples of high-quality dream extractions
---
Anti-Patterns
| Anti-Pattern | Fix |
|---|---|
| Reading entire JSONL files | Grep first, read targeted segments |
| Extracting trivial Q&A | Only extract non-obvious insights with transfer value |
| Writing to memory/\*.md instead of Sibyl | Sibyl is the primary store, memory files are a narrow exception |
| Skipping dedup check | Always search Sibyl before writing, duplicates degrade graph quality |
| Dream without orient | Always check when last dream ran, avoid re-processing |
| Extracting everything from every session | Score sessions first, process high-signal ones deeply |
| Ignoring Codex sessions | Codex conversations contain valuable engineering knowledge too |
---
What This Skill is NOT
- Not a replacement for Auto Dream. Auto Dream manages memory/\*.md housekeeping. This skill extracts knowledge into Sibyl.
- Not real-time. Dreams process past conversations. For live knowledge capture, use
sibyl adddirectly during sessions. - Not a full conversation replay. We extract signal, not transcripts. Sibyl stores insights, not chat logs.
- Not automatic (yet). Invoke with
/dream. Future: SessionEnd hook for automatic NREM processing.
Conversation Format Reference
Claude Code JSONL
Location: ~/.claude/projects/<encoded-path>/<session-uuid>.jsonl Encoding: Path separators replaced with - (e.g., /Users/bliss/dev/dreamer → -Users-bliss-dev-dreamer)
Message Structure
Each line is a complete JSON object (one message per line):
{
"uuid": "unique-message-id",
"parentUuid": "previous-message-uuid | null",
"isSidechain": false,
"type": "user | assistant | system | progress",
"timestamp": "2026-04-04T22:15:00.000Z",
"cwd": "/Users/bliss/dev/project",
"sessionId": "session-uuid",
"version": "2.1.81",
"gitBranch": "main",
"userType": "external",
"entrypoint": "cli",
"message": {
"role": "user | assistant",
"content": "..."
}
}Content Formats by Role
User messages: message.content is a plain string (the prompt text).
Assistant messages: message.content is an array of typed blocks:
[
{ "type": "thinking", "thinking": "internal reasoning...", "signature": "..." },
{ "type": "text", "text": "visible response..." },
{ "type": "tool_use", "id": "toolu_...", "name": "Bash", "input": { "command": "ls" } }
]Metadata Entries (Non-Message Lines)
These appear in the JSONL but aren't conversation messages:
| Type | Purpose | Key Fields |
|---|---|---|
summary | Compaction summary | leafUuid, summary |
ai-title | Auto-generated session title | aiTitle |
custom-title | User-set title | customTitle |
tag | Session tag | tag |
last-prompt | Last user prompt | lastPrompt |
pr-link | Associated PR | prNumber, prUrl, prRepository |
file-history-snapshot | File state checkpoint | messageId, snapshot |
mode | Coordinator/normal mode | mode |
task-summary | Task summary | summary, timestamp |
Subagent Transcripts
Location: <session-uuid>/subagents/agent-<agentId>.jsonl Metadata: agent-<agentId>.meta.json → {agentType, description, worktreePath?}
Subagent messages have isSidechain: true and an agentId field.
Useful Grep Patterns
# All user prompts in a session (grep top-level type, not nested role)
grep '"type":"user"' session.jsonl | python3 -c "
import sys, json
for l in sys.stdin:
obj = json.loads(l)
content = obj.get('message', {}).get('content', '')
if isinstance(content, str) and len(content) > 10 and not content.startswith('<'):
print(content[:200])
"
# All tool invocations (inside assistant message content arrays)
grep '"tool_use"' session.jsonl | python3 -c "
import sys, json
for l in sys.stdin:
obj = json.loads(l)
for block in obj.get('message', {}).get('content', []):
if isinstance(block, dict) and block.get('type') == 'tool_use':
print(block.get('name', '?'), json.dumps(block.get('input', {}))[:100])
"
# Count messages by type
grep -c '"type":"user"' session.jsonl
grep -c '"type":"assistant"' session.jsonl
# Find error-containing tool outputs (from assistant tool_result blocks)
grep -i "error\|exception\|failed\|traceback" session.jsonl | head -20
# Find session title
grep '"ai-title"\|"custom-title"' session.jsonl
# Find thinking blocks (extended reasoning)
grep '"type":"thinking"' session.jsonl | wc -lSession Discovery
# All sessions for a project, sorted by recency
# Sessions live directly in the project dir, NOT in a sessions/ subdirectory
ls -lt ~/.claude/projects/-Users-bliss-dev-<project>/*.jsonl | head -20
# All projects with sessions in the last 7 days
# Exclude subagent transcripts which live in <session-uuid>/subagents/
find ~/.claude/projects -maxdepth 2 -name "*.jsonl" -not -path "*/subagents/*" -mtime -7 \
| sed 's|/[^/]*\.jsonl$||' | sort -u
# Global history (all prompts across all projects)
tail -20 ~/.claude/history.jsonl | python3 -c "import sys,json; [print(json.loads(l).get('display','')[:100]) for l in sys.stdin]"
# Session sizes (bigger = richer conversations)
find ~/.claude/projects -maxdepth 2 -name "*.jsonl" -not -path "*/subagents/*" -mtime -7 \
-exec ls -lhS {} + | head -20
# Get AI-generated session titles (great for understanding session topics)
for f in ~/.claude/projects/-Users-bliss-dev-*/*.jsonl; do
title=$(grep -m1 '"ai-title"' "$f" 2>/dev/null | python3 -c "import sys,json; print(json.loads(next(sys.stdin)).get('aiTitle',''))" 2>/dev/null)
[[ -n "$title" ]] && echo "$(du -h "$f" | cut -f1) $(basename "$(dirname "$f")"): $title"
done | sort -rh | head -20---
Codex CLI JSONL
Location: ~/.codex/sessions/YYYY/MM/DD/rollout-<ISO-timestamp>-<session-uuid>.jsonl
Event Structure
Each line has a timestamp, type, and type-specific payload:
{"timestamp": "2026-04-04T22:15:00Z", "type": "session_meta", ...}
{"timestamp": "2026-04-04T22:15:01Z", "type": "response_item", ...}
{"timestamp": "2026-04-04T22:15:02Z", "type": "event_msg", ...}
{"timestamp": "2026-04-04T22:15:03Z", "type": "turn_context", ...}Event Types
`session_meta` — One per file, session header:
payload.id: Session UUIDpayload.cwd: Working directorypayload.cli_version: CLI versionpayload.originator:codex_cli_rs|codex_execpayload.model_provider: Provider name (e.g.,openai)payload.base_instructions: System prompt text (NOTsystem_prompt)payload.source: Source identifierpayload.git:{branch, origin_url, ...}— git context for the session
`response_item` — Conversation turns:
payload.type:message|function_call|function_call_output|reasoning- For messages:
payload.role=developer|user|assistant,payload.content[].type=input_text|output_text - For function calls:
payload.name,payload.arguments(JSON string),payload.call_id - For function outputs:
payload.call_id,payload.output - For reasoning:
payload.encrypted_content(opaque, not readable)
`event_msg` — Lifecycle events:
task_started,task_complete,token_count,user_message,agent_message
`turn_context` — Per-turn metadata:
cwd,date,timezone,approval_policy,sandbox_policymodel_name,personality,reasoning_effort,user_instructions
Useful Grep Patterns
# User messages from Codex
grep '"type":"event_msg"' rollout.jsonl | grep '"user_message"'
# Function calls (tool usage)
grep '"function_call"' rollout.jsonl | grep -v '"function_call_output"'
# Function outputs
grep '"function_call_output"' rollout.jsonl
# Assistant text responses
grep '"type":"response_item"' rollout.jsonl | grep '"output_text"'
# Session metadata
grep '"type":"session_meta"' rollout.jsonl
# Model being used
grep '"turn_context"' rollout.jsonl | head -1
# Session discovery
find ~/.codex/sessions -name "rollout-*.jsonl" -mtime -7 -exec ls -lhS {} + | head -20Codex SQLite (Supplementary)
`~/.codex/state_5.sqlite` — Thread index with columns:
id,title,model,cwd,git_branch,git_origin_urlfirst_user_message,tokens_used,created_at,updated_at
# List recent Codex threads
sqlite3 ~/.codex/state_5.sqlite "SELECT id, title, model, cwd, datetime(created_at, 'unixepoch') FROM threads ORDER BY created_at DESC LIMIT 20"Codex History (Supplementary)
`~/.codex/history.jsonl` — Flat prompt log:
{ "session_id": "uuid", "ts": 1712300000, "text": "user prompt text" }---
Cross-Format Comparison
| Feature | Claude Code | Codex |
|---|---|---|
| Location | ~/.claude/projects/*/<uuid>.jsonl | ~/.codex/sessions/YYYY/MM/DD/*.jsonl |
| Message format | message.content (string or array) | payload.content[].type |
| Tool calls | type: "tool_use" in content array | type: "function_call" as response_item |
| Tool results | Separate tool_result message | function_call_output response_item |
| Thinking/reasoning | type: "thinking" (readable) | type: "reasoning" (encrypted) |
| Session metadata | ai-title, tag, pr-link entries | session_meta header + turn_context |
| Subagents | Separate subagents/ directory | Not applicable |
| Retention | 30 days default | No auto-cleanup |
| Index DB | None (JSONL only) | SQLite state_5.sqlite |
Dream Extraction Guide
What to Extract vs Skip
High-Value Extractions
| Signal | Sibyl Type | Example |
|---|---|---|
| User corrects assistant's approach | error_pattern | "Don't use uv pip — use uv add for project dependencies" |
| Technical decision with trade-offs | episode (category: decision) | "Chose Temporal over BullMQ because workflow visibility matters more than simplicity" |
| Non-obvious debugging insight | pattern | "FalkorDB WRONGTYPE errors mean the key schema changed — run FLUSHALL on dev" |
| Reusable code pattern | pattern | "Use select! with heartbeat future for long-running Temporal activities" |
| Hard constraint discovered | rule | "Never commit .env files — gradial uses SOPS for secrets" |
| Unresolved question deferred | episode (category: tension) | "Should Sibyl use Graphiti's built-in community detection or custom?" |
| New tool/library adoption | episode (category: decision) | "Adopted better-auth for v2 — replacing next-auth due to multi-tenant needs" |
| Performance finding | pattern | "Batch Sibyl writes via REST API, not individual CLI calls — 10x faster" |
| Configuration quirk | error_pattern | "moon workspace requires .moon/toolchains.yml even if empty" |
Skip These (Low/No Value)
| Signal | Why Skip |
|---|---|
| Simple Q&A ("what does X do?") | No transfer value — answer is in the docs |
| File reads / directory listings | Ephemeral navigation, not knowledge |
| Routine git operations | Git history captures this |
| Typo corrections | Not a pattern or learning |
| Boilerplate generation | The code is the artifact, not the conversation |
| "Make it work" debugging with no root cause | No insight to capture if root cause unknown |
| Conversations that only resulted in reading code | Reading isn't learning unless something non-obvious was found |
---
Quality Bar for Extractions
Bad Extractions (Don't Write These)
"Fixed the auth bug"
→ No: What bug? What was the root cause? What's the transferable insight?
"Used React for the frontend"
→ No: This is a project fact derivable from package.json, not a learning.
"Updated the README"
→ No: The git commit says this. No knowledge to capture.Good Extractions
"JWT refresh tokens fail silently when Redis TTL expires before token expiry.
Root cause: token service catches WRONGTYPE error but swallows it.
Fix: Add explicit type check before SET, regenerate token on type mismatch.
Applies to: Any service using Redis for JWT storage with independent TTLs."
→ Yes: Root cause, fix, transferability.
"Temporal activity futures need periodic heartbeats, not just start/completion markers.
Pattern: Wrap the activity future in a select! loop emitting heartbeats every 30s.
Without this, Temporal marks the activity as failed after the heartbeat timeout."
→ Yes: Non-obvious behavior, concrete pattern, prevents future mistakes.
"Chose FalkorDB over Neo4j for Sibyl because: (1) Redis-compatible protocol for
existing infra, (2) built-in vector similarity search, (3) 10x faster for small
graphs (<1M nodes). Trade-off: less mature ecosystem, fewer community resources."
→ Yes: Decision with rationale, alternatives, trade-offs.The Transfer Test
Before writing an extraction, ask: "Would this be useful in a different project or a different session?"
- Yes → Write it
- Maybe → Write it with narrow scope tags
- No → Skip it
---
Entity Type Selection Guide
digraph entity_selection {
rankdir=TB;
node [shape=diamond];
Q1 [label="Is it a reusable\napproach that worked?"];
Q2 [label="Is it something\nthat went wrong?"];
Q3 [label="Is it a hard\nconstraint?"];
Q4 [label="Is it a decision\nwith trade-offs?"];
Q5 [label="Is it unresolved?"];
node [shape=box, style=filled];
pattern [label="pattern", fillcolor="#e8ffe8"];
error [label="error_pattern", fillcolor="#ffe8e8"];
rule [label="rule", fillcolor="#fff8e0"];
decision [label="episode\n(category: decision)", fillcolor="#e8e8ff"];
tension [label="episode\n(category: tension)", fillcolor="#ffe8ff"];
skip [label="Skip\n(not extractable)", fillcolor="#f0f0f0"];
Q1 -> pattern [label="yes"];
Q1 -> Q2 [label="no"];
Q2 -> error [label="yes"];
Q2 -> Q3 [label="no"];
Q3 -> rule [label="yes"];
Q3 -> Q4 [label="no"];
Q4 -> decision [label="yes"];
Q4 -> Q5 [label="no"];
Q5 -> tension [label="yes"];
Q5 -> skip [label="no"];
}---
Deduplication Strategy
Before Writing to Sibyl
1. Exact match check:
sibyl search "[exact entity title]" --type [type] --limit 32. Semantic similarity check:
sibyl search "[key concepts from the extraction]" --type [type] --limit 53. Decision matrix:
| Search Result | Action |
|---|---|
| No matches | Create new entity |
| Same topic, older info | Update existing entity (note: Sibyl tracks temporal validity) |
| Same topic, same info | Skip — already captured |
| Same topic, contradictory info | Create new entity + tension entity linking both |
| Related but distinct | Create new entity with RELATED_TO relationship |
Within a Single Dream Cycle
Multiple sessions may contain the same insight (e.g., same bug hit twice). Deduplicate within the extraction batch before writing to Sibyl:
1. Group extractions by topic/keyword 2. Merge duplicates — keep the richest description 3. Note multiple source sessions in the entity metadata
---
Tagging Conventions
Consistent tags make future search and REM exploration effective.
Required Tags
project:<name>— Which project this relates to (e.g.,project:sibyl,project:v2)- Source conversation type —
source:claudeorsource:codex
Recommended Tags
domain:<area>— Technical domain (e.g.,domain:auth,domain:graph,domain:deployment)stack:<tech>— Technology involved (e.g.,stack:temporal,stack:react,stack:kubernetes)confidence:<level>— How sure are we?high,medium,low
Dream-Specific Tags
dream— All entities created during dream cyclesdream-date:YYYY-MM-DD— When the dream cycle ranstale— Flagged for review during REM phaseneeds-review— Low-confidence extraction requiring human validationcross-project— REM-discovered cross-project connections
---
Examples: Full Extraction from a Conversation
Input: Claude Code Session Excerpt
User: "the SessionEnd hook isn't firing when I close the terminal"
Assistant: [investigates, finds the issue]
Assistant: "The problem is that SessionEnd only fires on clean exits —
if the terminal is killed (SIGKILL), the hook never runs. You need to
also handle SIGTERM in your hook registration..."
User: "ah that explains why the data was missing. let's add SIGTERM handling"Extractions
1. Error Pattern:
sibyl add "SessionEnd hook doesn't fire on terminal kill" \
"Claude Code SessionEnd hook only fires on clean exits (user types exit, Ctrl+D, /clear). Terminal kill (SIGKILL, closing window) bypasses the hook entirely. SIGTERM may or may not fire depending on the terminal emulator. Workaround: Also register a SIGTERM handler in hook scripts, and use a heartbeat/watchdog pattern for critical post-session processing." \
--type error_pattern --category hooks --tags "project:dreamer,source:claude,stack:claude-code"2. Pattern:
sibyl add "Pattern: Heartbeat watchdog for session-end processing" \
"Instead of relying solely on SessionEnd hook (which can miss unclean exits), use a dual approach: (1) SessionEnd hook for immediate processing, (2) Background watchdog that detects stale session PIDs and runs cleanup. Check ~/.claude/sessions/<pid>.json for active sessions." \
--type pattern --category hooks --tags "project:dreamer,source:claude,stack:claude-code"