Knowledge Synthesis

Knowledge Synthesis

Extract, organize, and distribute insights across multi-agent systems. Turns raw interaction data, logs, and outcomes into actionable knowledge through pattern recognition, best practice codification, and structured retrieval.

When to Use This Skill

• Synthesizing findings from multiple agents or research sessions
• Building or updating a shared knowledge base
• Identifying recurring success or failure patterns in workflows
• Codifying best practices from empirical evidence
• Structuring data for optimal retrieval (RAG optimization)
• Cross-domain knowledge transfer between projects or teams

Quick Reference

---

Workflow

Phase 1: Discovery     → Mine interactions, logs, and outcomes for patterns
Phase 2: Codification  → Document best practices, build knowledge graph
Phase 3: Dissemination → Surface insights to relevant agents/teams
Phase 4: Feedback      → Capture adoption feedback, refine the knowledge base

---

Phase 1: Knowledge Discovery

Map the landscape before extracting insights:

1. Scope sources -- identify which interactions, logs, artifacts, and outcomes to mine 2. Classify signals -- tag each finding by value (high/medium/low), novelty, and confidence 3. Identify patterns -- look for recurring success patterns, failure modes, and decision trees 4. Document contradictions -- note where sources disagree or outcomes diverge

Discovery Checklist

• [ ] All relevant interaction logs identified
• [ ] Outcomes mapped to the workflows that produced them
• [ ] Recurring patterns tagged with confidence levels
• [ ] Contradictions and edge cases flagged

---

Phase 2: Codification

Transform raw patterns into structured, retrievable knowledge:

1. Write Knowledge Nuggets -- concise, actionable summaries with context and evidence 2. Build decision trees -- for common choice points, document the decision logic 3. Create playbooks -- step-by-step guides for patterns that recur frequently 4. Update indices -- structure data for retrieval (embeddings, tags, graph links)

Knowledge Nugget Template

## [Pattern Name]

**Context**: When does this pattern apply?
**Evidence**: What interactions/outcomes support it? [cite sources]
**Action**: What should agents do when they encounter this situation?
**Confidence**: High | Medium | Low
**Tags**: [domain], [workflow-type], [agent-role]

---

Phase 3: Dissemination

Surface the right insights to the right consumers:

• Route knowledge nuggets to agents whose workflows they affect
• Integrate high-confidence patterns into skill references and playbooks
• Flag low-confidence patterns for further validation
• Update retrieval indices so future queries find new knowledge

---

Phase 4: Feedback Loop

Close the loop to keep the knowledge base accurate:

• Monitor adoption -- are agents applying the patterns?
• Capture corrections -- when a pattern proves wrong, update or retract it
• Track retrieval quality -- are the right nuggets surfacing for the right queries?
• Refine confidence scores based on real-world outcomes

---

Grounded Responses and Citations

When answering questions based on the knowledge base, provide grounded responses:

1. Use numbered citation markers (e.g., [1], [2]) inline 2. Append a References section listing the source and relevant snippet 3. Cite the specific session, log, or artifact that provided evidence

Example:

The retry logic reduces failures by 40% in high-latency environments [1].

>

References:

[1] "Session 2025-03-12" -- "After adding exponential backoff, error rate dropped from 12% to 7%"

---

Anti-Patterns

• Do not synthesize from a single data point -- require multiple corroborating sources
• Do not codify patterns without confidence ratings
• Do not overwrite existing knowledge without citing the new evidence
• Do not skip the feedback loop -- unvalidated knowledge degrades over time

Synthesis Workflow Reference

Detailed techniques for each phase of knowledge synthesis.

Pattern Recognition Across Interactions

Signal Detection

Look for these pattern types across agent interactions and outcomes:

Confidence Scoring

Assign confidence based on evidence strength:

Cross-Domain Transfer

When a pattern from one domain applies to another:

1. Identify the abstract principle behind the domain-specific pattern 2. Map the principle to the target domain's vocabulary and constraints 3. Validate with at least one concrete example in the target domain 4. Document both the source and target domain applications

---

RAG Optimization Techniques

Structuring Knowledge for Retrieval

Organize knowledge artifacts so retrieval systems can find the right content:

Chunking Strategy

• One knowledge nugget per chunk (avoid mixing topics)
• Include context preamble in each chunk (don't rely on surrounding chunks)
• Keep chunks between 200-800 tokens for embedding quality
• Use descriptive headings that match likely query terms

Metadata Enrichment

• Tag every artifact with: domain, workflow type, agent roles, confidence level
• Include synonyms and alternative phrasings in metadata
• Add "related to" links between associated artifacts
• Timestamp with creation and last-validated dates

Embedding Quality

• Write titles that capture the core insight (not just topic)
• Front-load the most important information in each chunk
• Use consistent terminology across related chunks
• Include concrete examples -- they improve semantic search relevance

Retrieval Testing

After adding or updating knowledge:

1. Formulate 3-5 natural-language queries that should surface the new content 2. Run each query and verify the content appears in top results 3. Identify any queries that miss -- adjust chunk text or metadata 4. Test negative queries that should NOT surface this content

---

Citation Methodology

Citation Format

Use a numbered reference system throughout all synthesized outputs:

Inline: "Pattern X reduces errors by 30% [1] and improves throughput [2]."

References:
[1] Session 2025-03-12, agent: search-specialist -- "Error rate: 12% → 8.4%"
[2] Benchmark run #47, workflow: data-pipeline -- "Throughput: 150 → 195 ops/sec"

Source Hierarchy

When multiple sources exist, prefer:

1. Direct measurement -- logs, metrics, test results 2. Agent output -- synthesized findings from specialist agents 3. Documented convention -- established practices in project docs 4. Inferred pattern -- derived from indirect evidence

Citation Integrity Rules

• Never cite a source you haven't read or verified
• When paraphrasing, ensure the citation accurately represents the original
• If a source is ambiguous, quote the relevant passage directly
• Update citations when source material changes

---

Knowledge Graph Construction

Node Types

Edge Types

Graph Maintenance

• Prune nodes with no edges quarterly
• Merge duplicate patterns when discovered
• Update confidence scores as new evidence arrives
• Archive (don't delete) deprecated patterns with deprecation reason

---

Insight Extraction

The Extraction Pipeline

Raw Data → Filter → Cluster → Abstract → Validate → Codify

1. Filter -- Remove noise (routine logs, expected outcomes, duplicate signals) 2. Cluster -- Group related signals by theme, workflow, or agent 3. Abstract -- Extract the underlying principle from concrete instances 4. Validate -- Check against existing knowledge for contradictions 5. Codify -- Write the insight as a Knowledge Nugget with citations

Extraction Questions

For each cluster of signals, ask:

• What is the common thread across these instances?
• Does this pattern hold outside the specific context where it was observed?
• What would falsify this pattern?
• Who needs to know about this?

---

Continuous Learning Loops

The Learning Cycle

Observe → Hypothesize → Test → Codify → Apply → Observe (repeat)

Triggering a Learning Review

Schedule reviews when:

• A new workflow completes 10+ runs
• An agent reports an unexpected outcome
• A pattern's confidence score changes (up or down)
• Cross-domain transfer is attempted for the first time

Review Checklist

• [ ] New patterns identified since last review
• [ ] Existing patterns with changed confidence levels
• [ ] Patterns that were applied and their outcomes
• [ ] Knowledge gaps identified by failed retrievals
• [ ] Stale artifacts that need re-validation

---

Best Practice Codification

When to Codify

A pattern is ready for codification when:

• Confidence is High (5+ corroborating instances)
• No unresolved contradictions remain
• The pattern has been applied successfully in at least 2 contexts
• A clear, actionable description can be written

Codification Template

# Best Practice: [Name]

## Context
When [situation], and [conditions apply].

## Practice
Do [specific action] because [evidence-backed reason].

## Evidence
- [Citation 1]: [outcome]
- [Citation 2]: [outcome]
- [Citation 3]: [outcome]

## Caveats
- Does not apply when [exception]
- Requires [prerequisite]

## Related
- See also: [related pattern]
- Supersedes: [older pattern, if applicable]

Versioning Practices

• Use semantic versioning for major practice revisions
• Document what changed and why in each revision
• Keep previous versions accessible for reference
• Note when a practice is deprecated and what replaces it