Context Engineering

Context Engineering

Context engineering curates the smallest high-signal token set for LLM tasks. The goal: maximize reasoning quality while minimizing token usage.

When to Activate

• Designing/debugging agent systems
• Context limits constrain performance
• Optimizing cost/latency
• Building multi-agent coordination
• Implementing memory systems
• Evaluating agent performance
• Developing LLM-powered pipelines

Core Principles

1. Context quality > quantity - High-signal tokens beat exhaustive content 2. Attention is finite - U-shaped curve favors beginning/end positions 3. Progressive disclosure - Load information just-in-time 4. Isolation prevents degradation - Partition work across sub-agents 5. Measure before optimizing - Know your baseline

Key Metrics

• Token utilization: Warning at 70%, trigger optimization at 80%
• Token variance: Explains 80% of agent performance variance
• Multi-agent cost: ~15x single agent baseline
• Compaction target: 50-70% reduction, <5% quality loss
• Cache hit target: 70%+ for stable workloads

Four-Bucket Strategy

1. Write: Save context externally (scratchpads, files) 2. Select: Pull only relevant context (retrieval, filtering) 3. Compress: Reduce tokens while preserving info (summarization) 4. Isolate: Split across sub-agents (partitioning)

Anti-Patterns

• Exhaustive context over curated context
• Critical info in middle positions
• No compaction triggers before limits
• Single agent for parallelizable tasks
• Tools without clear descriptions

Guidelines

1. Place critical info at beginning/end of context 2. Implement compaction at 70-80% utilization 3. Use sub-agents for context isolation, not role-play 4. Design tools with clear descriptions (what, when, inputs, returns) 5. Optimize for tokens-per-task, not tokens-per-request 6. Validate with probe-based evaluation 7. Monitor token usage in production 8. Start minimal, add complexity only when proven necessary

Skill Coordination

When multiple skills are active:

• Load only relevant skill content
• Use skill metadata for discovery
• Avoid loading full skill definitions unless needed
• Reference skills by pattern detection, not direct names

References

For detailed guidance, see:

• references/fundamentals.md - Context anatomy, attention mechanics
• references/degradation.md - Debugging failures, lost-in-middle, poisoning
• references/optimization.md - Compaction, masking, caching, partitioning
• references/compression.md - Long sessions, summarization strategies
• references/memory.md - Cross-session persistence, knowledge graphs
• references/multi-agent.md - Coordination patterns, context isolation
• references/evaluation.md - Testing agents, LLM-as-Judge, metrics
• references/tool-design.md - Tool consolidation, description engineering

Context Compression

Long sessions, summarization strategies, and compression techniques.

Summarization Strategies

Incremental Summarization

• Summarize message history periodically
• Keep recent messages detailed
• Compress older messages
• Maintain key information

Hierarchical Summarization

• Summarize at multiple levels
• Keep high-level overview
• Compress details
• Preserve critical information

Long Session Management

Challenges

• Context accumulates over time
• Token usage grows
• Performance degrades
• Information loss

Solutions

• Summarize at intervals (every 20 messages)
• Archive old context
• Use external storage
• Implement memory systems

Compression Techniques

Text Compression

• Remove redundancy
• Use abbreviations
• Condense explanations
• Preserve meaning

Structured Compression

• Extract key facts
• Use structured formats
• Remove verbose descriptions
• Keep essential information

Memory Systems

External Storage

• Save context to files
• Use databases for persistence
• Implement knowledge graphs
• Cross-session memory

Retrieval

• Semantic search
• Relevance filtering
• Just-in-time loading
• Context reconstruction

Best Practices

1. Summarize at 70% utilization 2. Archive old context 3. Use external storage for long-term memory 4. Implement retrieval systems 5. Monitor compression quality 6. Validate after compression

Context Degradation

Failure patterns when context exceeds optimal size or structure.

Lost-in-the-Middle

Problem

Information in the middle of context receives less attention than beginning/end.

Symptoms

• Agent misses information in middle sections
• Inconsistent behavior with same information
• Better performance with shorter context

Solutions

• Place critical info at beginning/end
• Use progressive disclosure
• Split large contexts across sub-agents
• Summarize middle sections

Context Poisoning

Problem

Malicious or low-quality content degrades performance.

Symptoms

• Agent follows incorrect instructions from context
• Performance degrades with more context
• Inconsistent outputs

Solutions

• Validate context sources
• Filter low-quality content
• Use trusted sources only
• Monitor for anomalies

Attention Dilution

Problem

Too much information dilutes attention to critical content.

Symptoms

• Agent misses important details
• Performance degrades with more context
• Inconsistent focus

Solutions

• Curate high-signal content
• Remove redundant information
• Use progressive disclosure
• Focus on task-relevant content

Token Limit Exceeded

Problem

Context exceeds model's token limit.

Symptoms

• Truncation of context
• Missing information
• Errors or failures

Solutions

• Implement compaction at 70-80% utilization
• Use summarization
• Split across multiple requests
• Use sub-agents for isolation

Detection

Warning Signs

• Token utilization >70%
• Performance degradation
• Missing information
• Inconsistent outputs

Monitoring

• Track token usage
• Monitor performance metrics
• Detect degradation patterns
• Set up alerts

Agent Evaluation

Testing agents, LLM-as-Judge, metrics, and evaluation strategies.

Evaluation Metrics

Quality Metrics

• Task completion rate
• Output correctness
• Code quality
• Test pass rate

Performance Metrics

• Token usage
• Response time
• Cost per task
• Cache hit rate

Reliability Metrics

• Error rate
• Retry rate
• Failure recovery
• Consistency

LLM-as-Judge

Pattern

• Use LLM to evaluate agent outputs
• Structured evaluation criteria
• Consistent scoring
• Automated assessment

Benefits

• Scalable evaluation
• Consistent criteria
• Fast feedback
• Cost-effective

Implementation

• Define evaluation criteria
• Create evaluation prompts
• Run evaluations
• Aggregate results

Probe-Based Evaluation

Concept

• Test specific capabilities
• Isolated evaluation
• Targeted assessment
• Identify gaps

Types

• Functional Probes - Test specific functions
• Integration Probes - Test component interactions
• End-to-End Probes - Test complete workflows

Evaluation Workflow

1. Define criteria - What to measure 2. Create test cases - Representative scenarios 3. Run evaluations - Execute tests 4. Analyze results - Identify issues 5. Iterate - Improve based on findings

Best Practices

1. Test with real scenarios 2. Measure before and after changes 3. Use multiple evaluation methods 4. Track metrics over time 5. Set up automated evaluation 6. Review and refine criteria

Context Fundamentals

Context = all input provided to LLM for task completion.

Anatomy of Context

Attention Mechanics

• U-shaped curve: Beginning/end get more attention than middle
• Attention budget: n^2 relationships for n tokens depletes with growth
• Position encoding: Interpolation allows longer sequences with degradation
• First-token sink: BOS token absorbs large attention budget

Progressive Disclosure Levels

1. Metadata (~100 words) - Always in context 2. SKILL.md body (<5k words) - When skill triggers 3. Bundled resources (Unlimited) - As needed

Token Budget Allocation

Document Management

Strong identifiers: customer_pricing_rates.json not data/file1.json Chunk at semantic boundaries: Paragraphs, sections, not arbitrary lengths Include metadata: Source, date, relevance score

Guidelines

1. Treat context as finite with diminishing returns 2. Place critical info at attention-favored positions 3. Use file-system-based access for large documents 4. Pre-load stable content, just-in-time load dynamic 5. Design with explicit token budgets 6. Monitor usage, implement compaction triggers at 70-80%

Memory Systems

Cross-session persistence, knowledge graphs, and memory architectures.

Memory Architecture

Types

• Short-term - Current session context
• Long-term - Cross-session persistence
• Episodic - Specific events and experiences
• Semantic - General knowledge and facts

Knowledge Graphs

Structure

• Entities (nodes)
• Relationships (edges)
• Properties (attributes)
• Hierarchical organization

Benefits

• Structured knowledge representation
• Relationship queries
• Incremental updates
• Cross-domain connections

External Storage

Options

• Files - Simple, file-system based
• Databases - Structured, queryable
• Vector stores - Semantic search
• Graph databases - Relationship queries

Implementation

• Save important context
• Index for retrieval
• Update incrementally
• Query by relevance

Retrieval Strategies

Semantic Search

• Vector embeddings
• Similarity matching
• Relevance ranking
• Context reconstruction

Structured Queries

• Database queries
• Graph traversals
• Filtered retrieval
• Aggregated results

Cross-Session Memory

Persistence

• Save session summaries
• Store key decisions
• Archive important context
• Link related sessions

Retrieval

• Search past sessions
• Load relevant context
• Reconstruct history
• Continue from previous work

Multi-Agent Patterns

Coordination patterns, context isolation, and multi-agent architectures.

Context Isolation

Benefits

• Prevent context degradation
• Enable parallel processing
• Isolate failures
• Reduce token usage per agent

Patterns

• Task Partitioning - Split work by task
• Domain Partitioning - Split by domain
• Pipeline Partitioning - Sequential stages

Coordination Patterns

Orchestration

• Central coordinator
• Manages workflow
• Coordinates sub-agents
• Handles failures

Choreography

• Decentralized coordination
• Event-driven communication
• Self-organizing
• Loose coupling

Cost Considerations

Multi-Agent Cost

• ~15x single agent baseline
• Each agent has own context
• Parallel execution overhead
• Coordination complexity

When to Use

• Large, independent tasks
• Parallelizable work
• Context limits reached
• Performance critical

Communication Patterns

Message Passing

• Direct communication
• Event-driven
• Request/response
• Async messaging

Shared State

• External storage
• Database
• Cache
• File system

Best Practices

1. Use sub-agents for context isolation, not role-play 2. Partition by task, not by role 3. Minimize inter-agent communication 4. Use external storage for shared state 5. Monitor costs and performance 6. Start with single agent, split when needed

Context Optimization

Compaction, masking, caching, and partitioning techniques.

Compaction

Techniques

• Summarization - Condense information
• Extraction - Pull key facts only
• Pruning - Remove low-value content
• Deduplication - Remove redundant information

Targets

• 50-70% token reduction
• <5% quality loss
• Trigger at 70-80% utilization

Implementation

def compact_context(context: str, target_reduction: float = 0.6) -> str:
    # Summarize long sections
    # Extract key facts
    # Remove redundancy
    # Return compacted version

Masking

Purpose

Hide irrelevant information while preserving structure.

Techniques

• Mask low-priority sections
• Hide implementation details
• Show only relevant context
• Preserve structure for navigation

Caching

Benefits

• Reduce redundant processing
• Faster response times
• Lower token usage
• 70%+ cache hit target

Cache Strategies

• Cache system prompts
• Cache tool definitions
• Cache frequently accessed docs
• Invalidate on updates

Partitioning

Sub-Agent Isolation

• Split work across sub-agents
• Each sub-agent has isolated context
• Prevents context degradation
• Enables parallel processing

When to Partition

• Large, independent tasks
• Parallelizable work
• Context limits reached
• Performance optimization needed

Selective Loading

Just-in-Time Loading

• Load information when needed
• Don't pre-load everything
• Use file-system access
• Progressive disclosure

Filtering

• Filter by relevance
• Remove outdated content
• Focus on task-specific info
• Use semantic search

Measurement

Metrics

• Token utilization
• Cache hit rate
• Response quality
• Performance metrics

Tools

• Token counters
• Performance profilers
• Quality evaluators
• Monitoring dashboards

Tool Design

Tool consolidation, description engineering, and tool design best practices.

Tool Description Engineering

Four-Question Framework

1. What - What does the tool do? 2. When - When should it be used? 3. Inputs - What parameters does it accept? 4. Returns - What does it return?

Best Practices

• Clear, concise descriptions
• Specific use cases
• Example inputs/outputs
• Error conditions

Tool Consolidation

Benefits

• Fewer tool definitions
• Lower token usage
• Simpler agent reasoning
• Better tool selection

Patterns

• Combine related tools
• Use parameters for variation
• Generic tools with options
• Tool hierarchies

Tool Selection

Factors

• Task requirements
• Available tools
• Tool capabilities
• Performance considerations

Optimization

• Clear descriptions aid selection
• Group related tools
• Prioritize common tools
• Monitor tool usage

Best Practices

1. Design tools with clear descriptions 2. Consolidate related tools 3. Use parameters for flexibility 4. Document inputs and outputs 5. Include examples 6. Monitor tool usage and effectiveness