Context Degradation

Context Degradation Patterns

Language models exhibit predictable degradation as context grows. Understanding these patterns is essential for diagnosing failures and designing resilient systems.

Degradation Patterns

Lost-in-Middle

Information at beginning and end receives reliable attention. Middle content suffers dramatically reduced recall.

Mitigation:

[CURRENT TASK]                      # At start (high attention)
- Goal: Generate quarterly report
- Deadline: End of week

[DETAILED CONTEXT]                  # Middle (less attention)
- 50 pages of data
- Supporting evidence

[KEY FINDINGS]                      # At end (high attention)
- Revenue up 15%
- Growth in Region A

Context Poisoning

Once errors enter context, they compound through repeated reference.

Entry pathways:

1. Tool outputs with errors 2. Retrieved docs with incorrect info 3. Model-generated summaries with hallucinations

Symptoms:

• Tool calls with wrong parameters
• Strategies that take effort to undo
• Hallucinations that persist despite correction

Recovery:

• Truncate to before poisoning point
• Explicitly note poisoning and re-evaluate
• Restart with clean context

Context Distraction

Even a single irrelevant document reduces performance. Models must attend to everything—they cannot "skip" irrelevant content.

Mitigation:

• Filter for relevance before loading
• Use namespacing for organization
• Access via tools instead of context

Degradation Thresholds

The Four-Bucket Approach

Counterintuitive Findings

1. Shuffled haystacks outperform coherent - Coherent context creates false associations 2. Single distractors have outsized impact - Step function, not proportional 3. Needle-question similarity matters - Dissimilar content degrades faster

When Larger Contexts Hurt

• Performance degrades non-linearly after threshold
• Cost grows exponentially with context length
• Cognitive bottleneck remains regardless of size

Best Practices

1. Monitor context length and performance correlation 2. Place critical information at beginning or end 3. Implement compaction triggers before degradation 4. Validate retrieved documents for accuracy 5. Use versioning to prevent outdated info clash 6. Segment tasks to prevent confusion 7. Design for graceful degradation 8. Test with progressively larger contexts