
Rag Implementer
- 74 installs
- 14 repo stars
- Updated March 2, 2026
- oakoss/agent-skills
Helps with ai & agent building tasks during AI-assisted development.
About
rag-implementer is a Claude Code skill for ai & agent building. It helps solo builders move faster with AI-assisted coding.
- rag-implementer
- AI & Agent Building
- AI-coding skill
Rag Implementer by the numbers
- 74 all-time installs (skills.sh)
- +3 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #5,508 of 16,546 AI & Agent Building skills by installs in the Skillselion catalog
- Data as of Aug 4, 2026 (Skillselion catalog sync)
npx skills add https://github.com/oakoss/agent-skills --skill rag-implementerAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 74 |
|---|---|
| repo stars | ★ 14 |
| Last updated | March 2, 2026 |
| Repository | oakoss/agent-skills ↗ |
What it does
Helps with ai & agent building tasks during AI-assisted development.
Files
RAG Implementer
Build production-ready retrieval-augmented generation systems. RAG = Retrieval + Context Assembly + Generation. Use RAG when LLMs need access to fresh, domain-specific, or proprietary knowledge not in their training data. Do not use RAG when simpler alternatives (FAQ pages, keyword search, semantic search) suffice. For KB architecture selection and governance, use the knowledge-base-manager skill. For knowledge graph implementation, use the knowledge-graph-builder skill.
Overview
Before building RAG, validate the need: try FAQ pages, keyword search, concierge MVP, or simple semantic search first. Only proceed with RAG for 50k+ documents with validated user demand and $200-500/month budget. RAG systems range from Naive (prototype) through Advanced (production) to Modular (enterprise), each tier adding complexity and cost.
The RAG pipeline has three core stages. First, retrieval finds relevant documents using hybrid search (semantic + keyword). Second, context assembly ranks, deduplicates, and compresses retrieved chunks into an optimal prompt. Third, generation produces a grounded response with source attribution. Each stage has distinct failure modes: retrieval can miss relevant documents (low recall), context assembly can overwhelm the model (lost in the middle), and generation can hallucinate despite good context (low faithfulness).
Modern RAG extends beyond basic vector similarity. Hybrid search combining dense embeddings with sparse BM25 is now the baseline. Re-ranking with cross-encoders improves precision after initial retrieval. Contextual chunking and late chunking preserve document-level semantics that fixed-size chunking loses. GraphRAG enables multi-hop reasoning over entity relationships by building knowledge graphs from documents. Proposition chunking breaks documents into atomic facts for precise retrieval of individual claims.
Choose techniques based on your query complexity and document structure. Start with hybrid search and re-ranking as the foundation, then layer contextual chunking, GraphRAG, or query expansion as needed. Measure everything: Precision@K, Recall@K, faithfulness, and end-to-end latency. The difference between a good and bad chunking strategy alone can create a 9% gap in recall performance.
Quick Reference
| Phase | Goal | Key Actions |
|---|---|---|
| 1. Knowledge Base Design | Structured knowledge foundation | Map sources, define chunking, add metadata |
| 2. Embedding Strategy | Semantic understanding | Select model, benchmark on domain data |
| 3. Vector Store | Scalable storage | Choose DB, configure index, plan scaling |
| 4. Retrieval Pipeline | Beyond simple similarity | Hybrid retrieval, query enhancement, re-ranking |
| 5. Context Assembly | Optimal LLM context | Rank, synthesize, compress, mitigate "lost in the middle" |
| 6. Evaluation | Measure performance | Precision@K, Recall@K, faithfulness, latency |
| 7. Production Deploy | Enterprise reliability | Containerize, cache, graceful degradation, security |
| 8. Continuous Improvement | Ongoing enhancement | Auto-updates, fine-tuning, optimization |
| Decision | Options |
|---|---|
| Vector DB (managed) | Pinecone |
| Vector DB (self-hosted) | Weaviate, Qdrant |
| Vector DB (lightweight) | Chroma |
| Vector DB (existing Postgres) | pgvector |
| Vector DB (billion-scale) | Milvus / Zilliz |
| Embedding (general) | text-embedding-3-large (3072 dim) |
| Embedding (cost-optimized) | text-embedding-3-small (1536 dim) |
| Embedding (code) | Voyage Code 3 |
| Embedding (multilingual) | multilingual-e5-large, Cohere embed-v4 |
| Chunking (fixed) | 500-1000 tokens, 50-100 overlap |
| Chunking (semantic) | Paragraph/section/topic boundaries |
| Chunking (recursive) | Markdown headers, code blocks |
| Chunking (contextual) | LLM-generated summaries prepended to each chunk |
| Chunking (late) | Full-document embedding, then pool by chunk boundaries |
| Cost Tier | Time | Monthly Cost | Scale |
|---|---|---|---|
| Naive RAG (prototype) | 1-2 weeks | $50-150 | <10k documents |
| Advanced RAG (production) | 3-4 weeks | $200-500 | 10k-1M documents |
| Modular RAG (enterprise) | 6-8 weeks | $500-2000+ | 1M+ documents |
| Advanced Technique | When to Use |
|---|---|
| Hybrid search | Always -- combine semantic + keyword (BM25) for better recall |
| Re-ranking | When initial retrieval returns noisy results |
| Contextual retrieval | Documents with ambiguous references or pronouns |
| Late chunking | Efficiency-focused pipelines with anaphoric references |
| GraphRAG | Multi-hop reasoning over structured knowledge relationships |
| Proposition chunking | Fact-dense documents requiring atomic retrieval units |
| Query expansion / HyDE | Queries that are short, ambiguous, or under-specified |
Common Mistakes
| Mistake | Correct Pattern |
|---|---|
| Building RAG before validating user need | Try simpler alternatives first (FAQ, keyword search, concierge MVP); only build RAG with validated demand |
| Using a single retrieval method (semantic only) | Implement hybrid retrieval combining semantic search with keyword (BM25) for better recall |
| Dumping all available data into the knowledge base | Curate data sources carefully; filter noise, select authoritative content, and maintain quality |
| Ignoring the "lost in the middle" problem | Place critical information at the start and end of context; compress mid-section |
| Skipping evaluation metrics before production | Establish baselines for Precision@K, Recall@K, faithfulness, and hallucination rate before deploying |
Using text-embedding-3-large at full 3072 dimensions without benchmarking | Test at reduced dimensions (1024 or 1536) first -- often comparable accuracy at lower cost |
| Fixed-size chunking for all document types | Match chunking strategy to document structure; use semantic or recursive chunking for structured content |
| Ignoring metadata filtering | Attach rich metadata (source, date, category) and filter before or during vector search |
Embedding Model Notes
text-embedding-3-large (3072 dimensions) remains OpenAI's most capable embedding model. It supports Matryoshka dimensionality reduction via the dimensions API parameter -- 1024 dimensions often delivers near-full accuracy at one-third storage cost. text-embedding-3-small (1536 dimensions) is a cost-effective alternative at $0.02 per million tokens. For code search, Voyage Code 3 outperforms general-purpose models. For multilingual workloads, consider multilingual-e5-large or Cohere embed-v4. Always benchmark on your domain data; general benchmarks do not predict domain-specific performance.
Vector Store Notes
Pinecone for managed simplicity, Weaviate or Qdrant for self-hosted with hybrid search, Chroma for prototyping, pgvector for teams already on PostgreSQL (practical limit around 10-100M vectors), and Milvus/Zilliz for billion-scale deployments. Choose index type based on tradeoffs: HNSW for speed (higher memory), IVF for scale (requires training), flat for exact search on small datasets only.
Most vector databases now achieve 10-100ms query latency on 1-10M vector datasets. Start with the simplest option that fits your scale requirements and migrate only when you hit concrete performance limits.
Delegation
- Discover data sources and assess knowledge base quality: Use
Exploreagent to catalog documents, evaluate data freshness, and identify authoritative content - Implement retrieval pipeline with hybrid search and re-ranking: Use
Taskagent to build embedding, indexing, retrieval, and evaluation components - Design RAG architecture and vector store topology: Use
Planagent to select embedding models, vector databases, chunking strategies, and deployment architecture
For KB architecture selection, curation workflows, and governance, use theknowledge-base-managerskill. For knowledge graph implementation (ontology, entity extraction, graph databases), use theknowledge-graph-builderskill.
References
- Architecture patterns and prerequisites
- Chunking strategies and knowledge base design
- Retrieval methods and pipeline design
- Evaluation metrics and quality gates
- Production deployment and continuous improvement
Architecture Patterns and Prerequisites
Prerequisites: Validate the Need for RAG
Before implementing RAG, confirm:
- Problem validated with users
- Users need AI search (tested with simpler alternatives)
- ROI justified (cost vs benefit calculated)
Try These FIRST (Before RAG)
1. FAQ Page / Documentation (1 day, $0)
- Create well-organized FAQ or docs
- Add search with Cmd+F
- Works for: <50 common questions, static content
2. Simple Keyword Search (2-3 days, $0-20/month)
- Use Algolia, Typesense, or PostgreSQL full-text search
- Good enough for 80% of use cases
- Works for: <100k documents, keyword matching sufficient
3. Manual Curation / Concierge MVP (1 week, $0)
- Manually answer user questions
- Build FAQ from common questions
- Works for: <100 users, validating if users want AI
4. Simple Semantic Search (1 week, $30-50/month)
- Use OpenAI embeddings + Postgres pgvector
- Skip complex retrieval, re-ranking, etc.
- Works for: <50k documents, basic semantic search
Decision Tree
Do users need to search your content?
|
+- No -> Don't build RAG
|
+- Yes
+- <50 items? -> FAQ page ($0)
|
+- >50 items?
+- Keyword search enough? -> Use Algolia ($0-20/mo)
|
+- Need semantic understanding?
+- <50k docs? -> Simple semantic search via pgvector ($30/mo)
|
+- >50k docs?
+- Validated with users? -> Build RAG
+- Not validated? -> Test with Concierge MVP firstArchitecture Tiers
Naive RAG (Prototype)
- Time: 1-2 weeks
- Cost: $50-150/month
- Scale: <10k documents
- Components: Basic embedding + vector store + simple retrieval
Advanced RAG (Production)
- Time: 3-4 weeks
- Cost: $200-500/month
- Scale: 10k-1M documents
- Components: Hybrid search, re-ranking, monitoring
Modular RAG (Enterprise)
- Time: 6-8 weeks
- Cost: $500-2000+/month
- Scale: 1M+ documents
- Components: Multiple knowledge bases, specialized modules
Modular RAG Architecture
- Search Module: Query understanding, reformulation, and hybrid retrieval
- Memory Module: Long-term conversation persistence and context accumulation
- Routing Module: Query routing to specialized knowledge bases or retrieval strategies
- Predict Module: Anticipatory pre-loading based on context
- Graph Module: Knowledge graph traversal for multi-hop reasoning (GraphRAG)
Hybrid RAG + Fine-tuning
- RAG for dynamic, frequently changing knowledge
- Fine-tuning for domain-specific reasoning patterns
- Combine strengths for maximum effectiveness
GraphRAG
- Build knowledge graphs by extracting entities and relationships from documents
- Enable multi-hop reasoning: "What projects did employees in department X work on?"
- Combine graph traversal with vector similarity for structured + unstructured queries
- Best suited for datasets with rich entity relationships (org charts, product catalogs, research papers)
Key RAG Principles
1. Relevance Over Volume -- Quality curation over massive datasets; remove outdated content continuously 2. Semantic Understanding -- Use embeddings for true semantic matching, recognize query intent 3. Multi-Modal Intelligence -- Handle text, images, code, tables; enable cross-modal retrieval 4. Temporal Awareness -- Prioritize recent info for time-sensitive topics 5. Transparency and Trust -- Always provide source citations and confidence levels
Chunking Strategies and Knowledge Base Design
Phase 1: Knowledge Base Design
Goal: Create well-structured knowledge foundation
Actions:
- Map data sources (internal: docs, databases, APIs / external: web, feeds)
- Filter noise, select authoritative content (prevent "data dump fallacy")
- Define chunking strategy: semantic chunking based on structure
- Add metadata: tags, timestamps, source identifiers, categories
Validation:
- All data sources catalogued and prioritized
- Data quality assessed (accuracy, completeness, freshness)
- Chunking strategy tested with sample documents
- Metadata schema validated for search effectiveness
Chunking Strategies
Fixed-Size Chunking
- 500-1000 tokens per chunk
- 50-100 token overlap between chunks
- Simple to implement, works well for uniform content
- Risk: splits may break semantic boundaries
Semantic Chunking
- Split by paragraph, section headers, or topic boundaries
- Preserves meaning within chunks
- Better for structured documents (technical docs, articles)
Recursive Chunking
- Split by structure: markdown headers, code blocks, list items
- Falls back to smaller units when chunks are too large
- Best for mixed-format documents
Contextual Chunking
- Chunk first, then use an LLM to generate a brief context summary for each chunk
- Prepend the summary to the chunk before embedding (e.g., "This chunk discusses authentication in a Node.js API guide")
- Resolves ambiguous references (pronouns, acronyms) that lose meaning when isolated
- Higher computational cost at indexing time but improves retrieval accuracy
Late Chunking
- Embed the full document first so every token captures complete document context
- Pool token embeddings within chunk boundaries after full-document encoding
- Improves retrieval accuracy by 10-12% on documents with anaphoric references
- More efficient than contextual chunking but may sacrifice some relevance
Proposition Chunking
- Break content into atomic, self-contained factual statements
- Each proposition stands alone without needing surrounding context
- Best for fact-dense documents (knowledge bases, encyclopedias, technical specs)
- Significantly improves precision for factual queries
Phase 2: Embedding Strategy
Goal: Choose optimal embedding approach for semantic understanding
Actions:
- Select embedding model based on domain
- Plan multi-modal needs (text, code, images, tables)
- Decide on fine-tuning: use domain data if general embeddings underperform
- Establish similarity benchmarks
Model Selection
| Use Case | Model | Dimensions |
|---|---|---|
| General text | text-embedding-3-large | 3072 (reducible via API) |
| Cost-optimized | text-embedding-3-small | 1536 |
| Code search | Voyage Code 3 | 1024-2048 |
| Multilingual | multilingual-e5-large | 1024 |
| Multimodal | Cohere embed-v4 | 1024 |
Both OpenAI embedding models support Matryoshka dimensionality reduction via the dimensions API parameter. For text-embedding-3-large, 1024 dimensions offers near-full accuracy at one-third the storage cost.
Phase 3: Vector Store Architecture
Goal: Implement scalable vector database
Actions:
- Choose vector DB based on requirements
- Configure index: HNSW for speed, IVF for scale
- Plan scalability: data growth and query volume
- Implement backup, recovery, security
Vector DB Decision Matrix
| Requirement | Recommended |
|---|---|
| Managed cloud | Pinecone |
| Self-hosted, feature-rich | Weaviate |
| Lightweight, local dev | Chroma |
| Cost-conscious, existing Postgres | pgvector |
| High-performance, production | Qdrant |
| Billion-scale vectors | Milvus / Zilliz |
Index Configuration
- HNSW: Best for speed, higher memory usage
- IVF: Better for large-scale, requires training step
- Flat: Exact search, only viable for small datasets
Evaluation Metrics and Quality Gates
Phase 6: Evaluation and Metrics
Goal: Measure RAG system performance across all dimensions
Retrieval Quality
- Precision@K: Fraction of top-K results that are relevant
- Recall@K: Fraction of relevant docs in top-K
- MRR (Mean Reciprocal Rank): Average rank of first relevant result
- NDCG: Ranking quality with graded relevance
Generation Quality
- Faithfulness: Generated content accuracy vs. sources
- Answer Relevance: Response relevance to query
- Context Utilization: How effectively LLM uses retrieved info
- Hallucination Rate: Frequency of unsupported claims
System Performance
- End-to-End Latency: Query to answer (<3 seconds target)
- Retrieval Latency: Time to retrieve and rank (<500ms)
- Token Efficiency: Information density per token
- Cost Per Query: Combined retrieval + generation costs
Validation
- Baseline metrics established
- A/B testing framework for config comparisons
- Automated evaluation pipeline deployed
- Human evaluation protocols for ground truth
Quality Gates
Before Production
- Accuracy >85% on evaluation dataset
- End-to-end latency 95th percentile <5 seconds
- Retrieval latency <500ms
Ongoing Monitoring
- User satisfaction >4.0/5.0
- Reliability: 99.5% uptime
- Cost: Within 10% of budget
Critical Success Rules
Non-Negotiable:
1. Source attribution for every response 2. Validate generated content against sources (prevent hallucination) 3. Filter sensitive data before retrieval 4. Respond within latency thresholds (<3 seconds) 5. Monitor and optimize costs continuously 6. Comply with security policies 7. Graceful degradation on failures 8. Full testing before production
Production Deployment and Continuous Improvement
Phase 7: Production Deployment
Goal: Deploy with enterprise-grade reliability and security
Deployment
- Containerize with Docker/Kubernetes
- Implement load balancing across RAG instances
- Add caching for frequent queries
- Graceful degradation: fallback to base model on component failure
Security
- Role-based access controls for knowledge base
- Data masking and PII protection
- Audit logging for compliance
- Prompt injection defense
Monitoring
- Real-time metrics dashboard (latency, cost, accuracy)
- Query analysis for patterns and failure modes
- Cost tracking and optimization alerts
- Performance profiling for bottlenecks
Validation
- Production handles expected traffic
- Security prevents unauthorized access
- Monitoring provides actionable insights
- Incident response procedures tested
Phase 8: Continuous Improvement
Goal: Establish processes for ongoing enhancement
Data Pipeline
- Automated knowledge base updates (real-time or scheduled)
- Quality monitoring: detect data drift and degradation
- Source diversification: add new data sources
- Feedback integration: user corrections and preferences
Model Evolution
- Evaluate and migrate to improved embeddings
- Fine-tune on domain data regularly
- Upgrade architecture: Naive to Advanced to Modular RAG
- Expand multi-modal support (images, audio, video)
Optimization
- Analyze query patterns, optimize for common needs
- Improve cache hit rates
- Tune vector indices regularly
- Balance performance vs. costs
Validation
- Automated improvement pipelines functioning
- Performance trends show improvement
- User satisfaction increasing
- System adapts to changing needs
Related Resources
Related Skills:
multi-agent-architect- For complex RAG orchestrationknowledge-graph-builder- For structured knowledge integrationperformance-optimizer- For RAG system optimization
Retrieval Methods and Pipeline Design
Phase 4: Retrieval Pipeline
Goal: Build sophisticated retrieval beyond simple similarity search
Actions:
- Implement hybrid retrieval: semantic search + keyword (BM25)
- Add query enhancement: expansion, reformulation, multi-query
- Apply contextual filtering: metadata, temporal constraints, relevance ranking
- Design for query types: factual (precision), analytical (breadth), creative (diversity)
- Handle edge cases: no relevant results found
Advanced Techniques
- Re-ranking: Use cross-encoder after initial retrieval (e.g.,
cross-encoder/ms-marco-MiniLM-L-12-v2) to improve precision - Query routing: Route different query types to specialized retrieval strategies
- Ensemble methods: Combine multiple retrieval approaches with reciprocal rank fusion
- Adaptive retrieval: Adjust top-k based on query complexity
- Query expansion / HyDE: Generate hypothetical answers to expand sparse queries into richer representations
- GraphRAG: Build knowledge graphs from documents; traverse entity relationships for multi-hop reasoning queries
- Contextual retrieval: Prepend LLM-generated context summaries to chunks before embedding to resolve ambiguous references
- ColBERT-style late interaction: Token-level similarity scoring between queries and documents for fine-grained matching
Validation
- Retrieval accuracy tested across diverse query types
- Hybrid retrieval outperforms single-method baselines
- Query latency meets requirements (<500ms ideal)
- Edge cases and fallbacks tested
Parent Document Retriever
Store small chunks for embedding and retrieval but return the full parent document (or a larger section) for context. Small chunks produce more precise embeddings; large context windows give the LLM enough surrounding information to generate accurate answers.
When to use: Long documents where individual passages lose meaning without surrounding context. Legal contracts, technical manuals, research papers with cross-referencing sections.
When to avoid: Short documents where chunks already capture the full context, or when token budget is tight.
interface ParentDocumentStore {
parentDocuments: Map<string, string>;
childChunks: Map<string, { text: string; parentId: string }>;
}
function buildParentDocumentIndex(
documents: { id: string; text: string }[],
chunkSize: number,
chunkOverlap: number,
): ParentDocumentStore {
const store: ParentDocumentStore = {
parentDocuments: new Map(),
childChunks: new Map(),
};
for (const doc of documents) {
store.parentDocuments.set(doc.id, doc.text);
const chunks = splitIntoChunks(doc.text, chunkSize, chunkOverlap);
for (let i = 0; i < chunks.length; i++) {
const chunkId = `${doc.id}_chunk_${i}`;
store.childChunks.set(chunkId, {
text: chunks[i],
parentId: doc.id,
});
}
}
return store;
}
async function parentDocumentRetrieval(
query: string,
store: ParentDocumentStore,
vectorDb: VectorStore,
topK: number,
): Promise<string[]> {
const childResults = await vectorDb.similaritySearch(query, topK);
const parentIds = new Set<string>();
for (const result of childResults) {
const chunk = store.childChunks.get(result.id);
if (chunk) parentIds.add(chunk.parentId);
}
return [...parentIds].map((id) => store.parentDocuments.get(id)!);
}A common variant uses a mid-level parent: instead of returning the full document, return the section or page containing the matched chunk. This balances precision with context.
Contextual Compression
After retrieval, extract only the relevant portions from each document using an LLM. Reduces noise in the context window so the generator sees focused, high-signal content.
When to use: Retrieved chunks contain relevant information buried inside irrelevant surrounding text. Common with larger chunk sizes or parent document retrieval.
When to avoid: Latency-sensitive pipelines where the extra LLM call is too expensive, or when chunks are already tightly scoped (proposition chunking).
async function compressRetrievedDocuments(
query: string,
documents: { text: string; source: string }[],
llm: LLMClient,
): Promise<{ text: string; source: string }[]> {
const compressed: { text: string; source: string }[] = [];
for (const doc of documents) {
const extraction = await llm.complete({
prompt: [
`Given the following question and document, extract only the parts `,
`of the document that are directly relevant to answering the question. `,
`If nothing is relevant, respond with "IRRELEVANT".\n\n`,
`Question: ${query}\n\n`,
`Document:\n${doc.text}`,
].join(''),
});
if (extraction.trim() !== 'IRRELEVANT') {
compressed.push({ text: extraction, source: doc.source });
}
}
return compressed;
}For higher throughput, batch the compression calls or use a smaller model (e.g., GPT-4o-mini or Claude Haiku) dedicated to extraction. The compression step typically adds 200-500ms latency per document but can reduce total context tokens by 50-70%.
Multi-Query Retrieval
Generate multiple query variations from the original question, retrieve for each, then deduplicate and merge results. Captures different facets of ambiguous or complex queries that a single embedding would miss.
async function multiQueryRetrieval(
originalQuery: string,
llm: LLMClient,
vectorDb: VectorStore,
topK: number,
numVariations = 3,
): Promise<RetrievalResult[]> {
const variations = await llm.complete({
prompt: [
`Generate ${numVariations} different versions of the following question `,
`to help retrieve relevant documents from a vector database. `,
`Each version should approach the question from a different angle.\n\n`,
`Original question: ${originalQuery}\n\n`,
`Return only the questions, one per line.`,
].join(''),
});
const queries = [originalQuery, ...variations.trim().split('\n')];
const allResults = new Map<string, RetrievalResult>();
for (const query of queries) {
const results = await vectorDb.similaritySearch(query, topK);
for (const result of results) {
const existing = allResults.get(result.id);
if (!existing || result.score > existing.score) {
allResults.set(result.id, result);
}
}
}
return [...allResults.values()].sort((a, b) => b.score - a.score);
}Multi-query retrieval pairs well with reciprocal rank fusion (below) for combining results instead of naive max-score deduplication.
Maximal Marginal Relevance (MMR)
Balance relevance and diversity in results to avoid redundant passages. MMR iteratively selects documents that are both relevant to the query and dissimilar to already-selected documents.
Formula: MMR = argmax[lambda * sim(query, doc) - (1 - lambda) * max(sim(doc, selected))]
lambda = 1.0: pure relevance (equivalent to standard similarity search)lambda = 0.0: pure diversity (maximum dissimilarity from selected docs)lambda = 0.5-0.7: typical production range balancing both
function mmrSelection(
queryEmbedding: number[],
candidates: { id: string; embedding: number[]; text: string }[],
k: number,
lambda = 0.6,
): typeof candidates {
const selected: typeof candidates = [];
const remaining = [...candidates];
for (let i = 0; i < k && remaining.length > 0; i++) {
let bestIdx = 0;
let bestScore = -Infinity;
for (let j = 0; j < remaining.length; j++) {
const relevance = cosineSimilarity(
queryEmbedding,
remaining[j].embedding,
);
let maxSimilarity = 0;
for (const sel of selected) {
const sim = cosineSimilarity(remaining[j].embedding, sel.embedding);
maxSimilarity = Math.max(maxSimilarity, sim);
}
const mmrScore = lambda * relevance - (1 - lambda) * maxSimilarity;
if (mmrScore > bestScore) {
bestScore = mmrScore;
bestIdx = j;
}
}
selected.push(remaining[bestIdx]);
remaining.splice(bestIdx, 1);
}
return selected;
}MMR is especially valuable when retrieved chunks come from similar sections of the same document. Without MMR, the top-K results might all contain near-identical information, wasting context window tokens.
Cross-Encoder Reranking
Initial retrieval uses bi-encoders (separate query and document embeddings) for speed. Reranking uses a cross-encoder that processes query and document together for higher accuracy, at the cost of being ~100x slower per pair.
Pipeline: Retrieve 50-100 candidates with bi-encoder, then rerank the top candidates with a cross-encoder, return the top-K reranked results.
Cohere Rerank API
import { CohereClient } from 'cohere-ai';
async function cohereRerank(
query: string,
documents: { text: string; id: string }[],
topN: number,
): Promise<{ id: string; text: string; relevanceScore: number }[]> {
const cohere = new CohereClient({ token: process.env.COHERE_API_KEY });
const response = await cohere.rerank({
query,
documents: documents.map((d) => d.text),
topN,
model: 'rerank-v3.5',
});
return response.results.map((r) => ({
id: documents[r.index].id,
text: documents[r.index].text,
relevanceScore: r.relevanceScore,
}));
}Local Cross-Encoder Reranking
async function crossEncoderRerank(
query: string,
documents: { text: string; id: string }[],
model: CrossEncoderModel,
topN: number,
): Promise<{ id: string; text: string; score: number }[]> {
const pairs = documents.map((doc) => ({
id: doc.id,
text: doc.text,
score: model.predict(query, doc.text),
}));
return pairs.sort((a, b) => b.score - a.score).slice(0, topN);
}When to Rerank vs When Not To
| Reranking adds value | Skip reranking |
|---|---|
| Initial retrieval returns >20 candidates | Result set is already small (<10) |
| Noisy results from hybrid search fusion | Latency budget is under 100ms total |
| Domain-specific queries where bi-encoders struggle | Bi-encoder is fine-tuned on domain data |
| High-stakes answers (legal, medical, compliance) | Cost per query must stay under $0.001 |
Reranking typically adds 100-300ms latency. For Cohere Rerank, cost is ~$1 per 1000 search queries (reranking 100 documents each).
Reciprocal Rank Fusion (RRF)
Combine results from multiple retrieval methods (semantic search, BM25, metadata filters) into a single ranked list without needing normalized scores. RRF is score-agnostic, making it ideal for fusing results from systems with incompatible score scales.
Formula: RRF_score(doc) = sum(1 / (k + rank_i(doc))) for each retrieval method i
The constant k (typically 60) dampens the impact of high rankings from any single method.
function reciprocalRankFusion(
rankedLists: { id: string; text: string }[][],
k = 60,
): { id: string; text: string; score: number }[] {
const scores = new Map<string, { text: string; score: number }>();
for (const list of rankedLists) {
for (let rank = 0; rank < list.length; rank++) {
const doc = list[rank];
const existing = scores.get(doc.id);
const rrfScore = 1 / (k + rank + 1);
if (existing) {
existing.score += rrfScore;
} else {
scores.set(doc.id, { text: doc.text, score: rrfScore });
}
}
}
return [...scores.entries()]
.map(([id, { text, score }]) => ({ id, text, score }))
.sort((a, b) => b.score - a.score);
}Hybrid Search with RRF
async function hybridSearchWithRRF(
query: string,
vectorDb: VectorStore,
bm25Index: BM25Index,
topK: number,
): Promise<{ id: string; text: string; score: number }[]> {
const [semanticResults, keywordResults] = await Promise.all([
vectorDb.similaritySearch(query, topK * 2),
bm25Index.search(query, topK * 2),
]);
const fused = reciprocalRankFusion([semanticResults, keywordResults]);
return fused.slice(0, topK);
}RRF is the default fusion method in Elasticsearch and Weaviate hybrid search. It consistently outperforms simple score averaging or weighted combination because it handles score distribution mismatches between retrieval methods.
Phase 5: Context Assembly
Goal: Transform retrieved chunks into optimal LLM context
Actions:
- Rank and select: prioritize by relevance score, recency, source authority
- Synthesize: merge related chunks, avoid redundancy
- Compress: use LLMLingua or similar for token optimization
- Mitigate "lost in the middle": place critical info at start/end
- Adapt dynamically: adjust context based on conversation history
Context Engineering Integration
- Blend RAG results with system instructions and user prompts
- Maintain conversation coherence across multi-turn interactions
- Implement context persistence for follow-up queries
- Balance context size vs. information density
Validation
- Context relevance validated against human judgments
- Token optimization maintains accuracy
- Multi-turn conversations maintain coherence
- Assembly latency <200ms
Standard RAG Response Format
{
"answer": "Generated response incorporating retrieved information",
"sources": [
{
"content": "Retrieved text chunk",
"source": "Document/URL identifier",
"relevance_score": 0.95,
"chunk_id": "unique_identifier"
}
],
"confidence": 0.87,
"retrieval_metadata": {
"chunks_retrieved": 5,
"retrieval_time_ms": 150,
"generation_time_ms": 800
}
}Retrieval Method Selection Guide
| Method | Latency Impact | Best For | Pair With |
|---|---|---|---|
| Hybrid search (RRF) | +10-50ms | All production systems (baseline) | Reranking |
| Parent document | +20-50ms | Long documents, context-dependent passages | Contextual compression |
| Multi-query | +500-1500ms | Ambiguous or complex queries | RRF |
| MMR | +10-30ms | Reducing redundancy in results | Any retrieval method |
| Cross-encoder reranking | +100-300ms | Noisy initial results, high-stakes answers | Hybrid search |
| Contextual compression | +200-500ms | Large chunks, parent document retrieval | Parent document |
| HyDE | +500-1000ms | Short or vague queries | Hybrid search |