Rag Implementer

RAG Implementer

Build production-ready retrieval-augmented generation systems.

Core Principle

RAG = Retrieval + Context Assembly + Generation

Use RAG when you need LLMs to access fresh, domain-specific, or proprietary knowledge that wasn't in their training data.

---

⚠️ Prerequisites & Cost Reality Check

STOP: Have You Validated the Need for RAG?

Before implementing RAG, confirm:

• [ ] Problem validated - Completed product-strategist Phase 1 (problem discovery)
• [ ] Users need AI search - Tested with simpler alternatives (see below)
• [ ] ROI justified - Calculated cost vs benefit of RAG vs alternatives

Try These FIRST (Before RAG)

RAG is powerful but expensive. Try cheaper alternatives first:

1. FAQ Page / Documentation (1 day, $0)

• Create well-organized FAQ or docs
• Add search with Cmd+F
• Works for: <50 common questions, static content
• Test: Do users find answers? If yes, stop here.

2. Simple Keyword Search (2-3 days, $0-20/month)

• Use Algolia, Typesense, or PostgreSQL full-text search
• Good enough for 80% of use cases
• Works for: <100k documents, keyword matching sufficient
• Test: Do users get relevant results? If yes, stop here.

3. Manual Curation (Concierge MVP) (1 week, $0)

• Manually answer user questions
• Build FAQ from common questions
• Works for: <100 users, validating if users want AI
• Test: Do users value your answers enough to pay? If yes, consider RAG.

4. Simple Semantic Search (1 week, $30-50/month)

• Use OpenAI embeddings + Postgres pgvector
• Skip complex retrieval, re-ranking, etc.
• Works for: <50k documents, basic semantic search
• Test: Are embeddings better than keyword search? If no, stop here.

Cost Reality Check

Naive RAG (Prototype):

• Time: 1-2 weeks
• Cost: $50-150/month (vector DB + embeddings + API calls)
• When: Prototype, <10k documents, proof of concept

Advanced RAG (Production):

• Time: 3-4 weeks
• Cost: $200-500/month (hybrid search, re-ranking, monitoring)
• When: Production, 10k-1M documents, validated demand

Modular RAG (Enterprise):

• Time: 6-8 weeks
• Cost: $500-2000+/month (multiple KBs, specialized modules)
• When: Enterprise, 1M+ documents, mission-critical

Decision Tree: Do You Really Need RAG?

Do users need to search your content?
│
├─ No → Don't build RAG ❌
│
└─ Yes
   ├─ <50 items? → FAQ page ✅ ($0)
   │
   └─ >50 items?
      ├─ Keyword search enough? → Use Algolia ✅ ($0-20/mo)
      │
      └─ Need semantic understanding?
         ├─ <50k docs? → Simple semantic (pgvector) ✅ ($30/mo)
         │
         └─ >50k docs?
            ├─ Validated with users? → Build RAG ✅
            └─ Not validated? → Test with Concierge MVP first ⚠️

Validation Checklist

Only proceed with RAG implementation if:

• [ ] Tested simpler alternatives (FAQ, keyword search, manual curation)
• [ ] Users confirmed they need AI-powered search (not just you think they do)
• [ ] Calculated ROI: cost of RAG < value users get
• [ ] Have >50k documents OR complex semantic search requirements
• [ ] Budget: $200-500/month for infrastructure
• [ ] Time: 3-4 weeks for production implementation

If any checkbox is unchecked: Go back to product-strategist or mvp-builder skills to validate first.

See also: PLAYBOOKS/validation-first-development.md for step-by-step validation process.

---

8-Phase RAG Implementation

Phase 1: Knowledge Base Design

Goal: Create well-structured knowledge foundation

Actions:

• Map data sources (internal: docs, databases, APIs / external: web, feeds)
• Filter noise, select authoritative content (prevent "data dump fallacy")
• Define chunking strategy: semantic chunking based on structure
• Add metadata: tags, timestamps, source identifiers, categories

Validation:

• [ ] All data sources catalogued and prioritized
• [ ] Data quality assessed (accuracy, completeness, freshness)
• [ ] Chunking strategy tested with sample documents
• [ ] Metadata schema validated for search effectiveness

Common Chunking Strategies:

• Fixed-size: 500-1000 tokens, 50-100 token overlap
• Semantic: By paragraph, section headers, or topic boundaries
• Recursive: Split by structure (markdown headers, code blocks)

---

Phase 2: Embedding Strategy

Goal: Choose optimal embedding approach for semantic understanding

Actions:

• Select embedding model: text-embedding-3-large (1536 dim) for general, domain-specific for specialized
• Plan multi-modal needs (text, code, images, tables)
• Decide on fine-tuning: use domain data if general embeddings underperform
• Establish similarity benchmarks

Validation:

• [ ] Embedding model benchmarked on domain data
• [ ] Retrieval accuracy tested with known query-document pairs
• [ ] Storage and compute costs validated

Model Selection:

• General: OpenAI text-embedding-3-large, text-embedding-3-small
• Code: code-search-babbage-code-001 or StarEncoder
• Multilingual: multilingual-e5-large

---

Phase 3: Vector Store Architecture

Goal: Implement scalable vector database

Actions:

• Choose vector DB (Pinecone, Weaviate, Qdrant, Chroma, pgvector)
• Configure index: HNSW for speed, IVF for scale
• Plan scalability: data growth and query volume
• Implement backup, recovery, security

Validation:

• [ ] Vector store benchmarked under expected load
• [ ] Index optimized for retrieval speed and accuracy
• [ ] Backup and recovery tested
• [ ] Security controls implemented

Vector DB Decision:

• Managed cloud → Pinecone
• Self-hosted, feature-rich → Weaviate
• Lightweight, local → Chroma
• Cost-conscious → pgvector (Postgres extension)
• High-performance → Qdrant

---

Phase 4: Retrieval Pipeline

Goal: Build sophisticated retrieval beyond simple similarity search

Actions:

• Implement hybrid retrieval: semantic search + keyword (BM25)
• Add query enhancement: expansion, reformulation, multi-query
• Apply contextual filtering: metadata, temporal constraints, relevance ranking
• Design for query types: factual (precision), analytical (breadth), creative (diversity)
• Handle edge cases: no relevant results found

Advanced Techniques:

• Re-ranking: Use cross-encoder after initial retrieval (e.g., cross-encoder/ms-marco-MiniLM-L-12-v2)
• Query routing: Route different query types to specialized strategies
• Ensemble methods: Combine multiple retrieval approaches
• Adaptive retrieval: Adjust top-k based on query complexity

Validation:

• [ ] Retrieval accuracy tested across diverse query types
• [ ] Hybrid retrieval outperforms single-method baselines
• [ ] Query latency meets requirements (<500ms ideal)
• [ ] Edge cases and fallbacks tested

---

Phase 5: Context Assembly

Goal: Transform retrieved chunks into optimal LLM context

Actions:

• Rank and select: prioritize by relevance score, recency, source authority
• Synthesize: merge related chunks, avoid redundancy
• Compress: use LLMLingua or similar for token optimization
• Mitigate "lost in the middle": place critical info at start/end
• Adapt dynamically: adjust context based on conversation history

Context Engineering Integration:

• Blend RAG results with system instructions and user prompts
• Maintain conversation coherence across multi-turn interactions
• Implement context persistence for follow-up queries
• Balance context size vs. information density

Validation:

• [ ] Context relevance validated against human judgments
• [ ] Token optimization maintains accuracy
• [ ] Multi-turn conversations maintain coherence
• [ ] Assembly latency <200ms

---

Phase 6: Evaluation & Metrics

Goal: Measure RAG system performance comprehensively

Retrieval Quality:

• Precision@K: Fraction of top-K results that are relevant
• Recall@K: Fraction of relevant docs in top-K
• MRR (Mean Reciprocal Rank): Average rank of first relevant result
• NDCG: Ranking quality with graded relevance

Generation Quality:

• Faithfulness: Generated content accuracy vs. sources
• Answer Relevance: Response relevance to query
• Context Utilization: How effectively LLM uses retrieved info
• Hallucination Rate: Frequency of unsupported claims

System Performance:

• End-to-End Latency: Query to answer (<3 seconds target)
• Retrieval Latency: Time to retrieve and rank (<500ms)
• Token Efficiency: Information density per token
• Cost Per Query: Combined retrieval + generation costs

Validation:

• [ ] Baseline metrics established
• [ ] A/B testing framework for config comparisons
• [ ] Automated evaluation pipeline deployed
• [ ] Human evaluation protocols for ground truth

---

Phase 7: Production Deployment

Goal: Deploy with enterprise-grade reliability and security

Deployment:

• Containerize with Docker/Kubernetes
• Implement load balancing across RAG instances
• Add caching for frequent queries
• Graceful degradation: fallback to base model on component failure

Security:

• Role-based access controls for knowledge base
• Data masking and PII protection
• Audit logging for compliance
• Prompt injection defense

Monitoring:

• Real-time metrics dashboard (latency, cost, accuracy)
• Query analysis for patterns and failure modes
• Cost tracking and optimization alerts
• Performance profiling for bottlenecks

Validation:

• [ ] Production handles expected traffic
• [ ] Security prevents unauthorized access
• [ ] Monitoring provides actionable insights
• [ ] Incident response procedures tested

---

Phase 8: Continuous Improvement

Goal: Establish processes for ongoing enhancement

Data Pipeline:

• Automated knowledge base updates (real-time or scheduled)
• Quality monitoring: detect data drift and degradation
• Source diversification: add new data sources
• Feedback integration: user corrections and preferences

Model Evolution:

• Evaluate and migrate to improved embeddings
• Fine-tune on domain data regularly
• Upgrade architecture: Naive → Advanced → Modular RAG
• Expand multi-modal support (images, audio, video)

Optimization:

• Analyze query patterns, optimize for common needs
• Improve cache hit rates
• Tune vector indices regularly
• Balance performance vs. costs

Validation:

• [ ] Automated improvement pipelines functioning
• [ ] Performance trends show improvement
• [ ] User satisfaction increasing
• [ ] System adapts to changing needs

Key RAG Principles

1. Relevance Over Volume

• Quality curation > massive datasets
• Remove outdated/low-quality content continuously
• Prioritize most relevant info to prevent "lost in the middle"

2. Semantic Understanding

• Use embeddings for true semantic matching, not just keywords
• Recognize query intent (factual, analytical, creative)
• Adapt retrieval strategy based on context

3. Multi-Modal Intelligence

• Handle text, images, code, tables, structured data
• Enable cross-modal retrieval (text query → image results)
• Preserve document structure and formatting

4. Temporal Awareness

• Prioritize recent info for time-sensitive topics
• Maintain historical access when relevant
• Integrate real-time data feeds for dynamic domains

5. Transparency & Trust

• Always provide source citations
• Indicate confidence levels
• Explain why specific information was selected

Standard RAG Response Format

{
  "answer": "Generated response incorporating retrieved information",
  "sources": [
    {
      "content": "Retrieved text chunk",
      "source": "Document/URL identifier",
      "relevance_score": 0.95,
      "chunk_id": "unique_identifier"
    }
  ],
  "confidence": 0.87,
  "retrieval_metadata": {
    "chunks_retrieved": 5,
    "retrieval_time_ms": 150,
    "generation_time_ms": 800
  }
}

Critical Success Rules

Non-Negotiable:

1. ✅ Source attribution for every response 2. ✅ Validate generated content against sources (prevent hallucination) 3. ✅ Filter sensitive data before retrieval 4. ✅ Respond within latency thresholds (<3 seconds) 5. ✅ Monitor and optimize costs continuously 6. ✅ Comply with security policies 7. ✅ Graceful degradation on failures 8. ✅ Comprehensive testing before production

Quality Gates:

• Before Production: >85% accuracy on evaluation dataset
• Ongoing: User satisfaction >4.0/5.0
• Performance: 95th percentile <5 seconds
• Reliability: 99.5% uptime
• Cost: Within 10% of budget

Advanced Patterns

Modular RAG Architecture

• Search Module: Query understanding and reformulation
• Memory Module: Long-term conversation persistence
• Routing Module: Query routing to specialized knowledge bases
• Predict Module: Anticipatory pre-loading based on context

Hybrid RAG + Fine-tuning

• RAG for dynamic, frequently changing knowledge
• Fine-tuning for domain-specific reasoning patterns
• Combine strengths for maximum effectiveness

Related Resources

Related Skills:

• multi-agent-architect - For complex RAG orchestration
• knowledge-graph-builder - For structured knowledge integration
• performance-optimizer - For RAG system optimization

Related Patterns:

• META/DECISION-FRAMEWORK.md - Vector DB and embedding selection
• STANDARDS/architecture-patterns/rag-pattern.md - RAG architecture details (when created)

Related Playbooks:

• PLAYBOOKS/deploy-rag-system.md - RAG deployment procedure (when created)

RAG Implementer - Quick Start

Version: 1.0.0 Category: AI-Native Development Difficulty: Intermediate

What This Skill Does

Guides implementation of production-ready Retrieval-Augmented Generation (RAG) systems from data preparation through deployment and monitoring.

When to Use

Use this skill when you need to:

• Build knowledge-intensive applications
• Create document search and Q&A systems
• Ground LLM responses in external/proprietary data
• Access fresh information not in LLM training data
• Reduce hallucinations with source attribution

Quick Start

Fastest path to RAG:

1. Define knowledge scope (Phase 1)

• Identify data sources
• Choose chunking strategy (500-1000 tokens)
• Add metadata for filtering

2. Choose embedding model (Phase 2)

• General: text-embedding-3-large (OpenAI)
• Code: code-search-babbage-code-001
• Test on sample queries

3. Set up vector store (Phase 3)

• Managed: Pinecone
• Self-hosted: Weaviate or Qdrant
• Lightweight: Chroma or pgvector

4. Build retrieval pipeline (Phase 4)

• Hybrid search: semantic + keyword (BM25)
• Re-rank top results with cross-encoder
• Return top-5 chunks

5. Assemble context (Phase 5)

• Rank by relevance + recency
• Remove redundancy
• Stay under token limits

6. Evaluate (Phase 6)

• Measure: Precision@5, Recall@10, MRR
• Test faithfulness and hallucination rate
• Monitor latency (<3 seconds)

Time to production: 1-2 weeks for MVP

File Structure

rag-implementer/
├── SKILL.md           # Main skill instructions (start here)
└── README.md          # This file

Prerequisites

Knowledge:

• Understanding of embeddings and vector similarity
• Basic LLM concepts
• API integration experience

Tools:

• Vector database account (Pinecone, Weaviate, etc.)
• LLM API access (Anthropic Claude, OpenAI)
• Embedding API access (OpenAI, Cohere, etc.)

Related Skills:

• None required, but multi-agent-architect helps for complex systems

Success Criteria

You've successfully used this skill when:

• ✅ All 8 RAG phases completed with validation gates checked
• ✅ Retrieval quality: Precision@5 >70%, MRR >0.6
• ✅ Generation quality: Faithfulness >85%, hallucination <10%
• ✅ System performance: End-to-end latency <3 seconds
• ✅ Source citations included in all responses
• ✅ Monitoring and evaluation pipelines deployed

Common Workflows

Workflow 1: Document Q&A System

1. Use rag-implementer Phases 1-3 for data prep and vector store 2. Implement hybrid retrieval (Phase 4) 3. Deploy with monitoring (Phase 7) 4. Use deployment-advisor for hosting decisions

Workflow 2: Code Search Assistant

1. Use rag-implementer with code-specific embeddings 2. Add syntax-aware chunking (Phase 1) 3. Implement multi-modal retrieval for code + docs 4. Use api-designer for building search API

Workflow 3: Real-Time Knowledge Base

1. Use rag-implementer with streaming data sources 2. Implement automated updates (Phase 8) 3. Add temporal awareness for freshness weighting 4. Use performance-optimizer for latency reduction

Key Concepts

RAG vs. Fine-tuning vs. Prompting:

• RAG: Dynamic knowledge, frequently updated data, source attribution needed
• Fine-tuning: Static knowledge, reasoning patterns, domain-specific behavior
• Prompting: General knowledge already in LLM, task-specific instructions

3 RAG Architectures:

1. Naive RAG: Simple retrieval → context → generation 2. Advanced RAG: Query enhancement, re-ranking, hybrid search 3. Modular RAG: Specialized modules (search, memory, routing, prediction)

8 RAG Phases:

1. Knowledge base design 2. Embedding strategy 3. Vector store setup 4. Retrieval pipeline 5. Context assembly 6. Evaluation & metrics 7. Production deployment 8. Continuous improvement

Troubleshooting

Skill not activating?

• Try explicitly requesting: "Use the rag-implementer skill to..."
• Mention keywords: "RAG", "vector database", "retrieval", "embeddings"

Low retrieval accuracy?

• Check embedding model performance on domain data
• Implement hybrid search (semantic + keyword)
• Add re-ranking with cross-encoder
• Review chunking strategy

High latency?

• Reduce top-k retrieval count (try 3-5 instead of 10)
• Implement caching for frequent queries
• Use faster vector DB (Qdrant for performance)
• Optimize context assembly

High hallucination rate?

• Validate faithfulness against sources
• Add explicit source citation requirements
• Implement confidence scoring
• Use temperature=0 for factual queries

Context too large?

• Use context compression (LLMLingua)
• Remove redundant chunks
• Prioritize by relevance score
• Increase chunk overlap to reduce count

Version History

• 1.0.0 (2025-10-21): Initial release, adapted from RAG Framework

License

Part of ai-dev-standards repository.