Ai Choosing Architecture

Choose the Right DSPy Architecture

When NOT to use this skill

• Already know what module to use — go to the matching /dspy-* skill
• Fixing errors in existing code — use /ai-fixing-errors
• Learning a specific module — use the matching /dspy-* skill
• Need a project plan, not an architecture decision — use /ai-planning

---

Step 1: Answer 3 questions

Before recommending anything, get answers to these three questions from the user (or infer them from context):

1. What goes in and what comes out? Input type and format, output type and format. 2. Does the AI need external tools? Search, APIs, databases, calculators, code execution? 3. How complex is the reasoning? Simple mapping, moderate analysis, complex multi-step logic?

---

Step 2: Pick the module

Walk the decision tree:

Does it need tools?
├── Yes: Does it need to write and run code?
│   ├── Yes → CodeAct
│   └── No → ReAct
└── No: How complex is the reasoning?
    ├── Simple (direct mapping) → Predict
    ├── Moderate (needs explanation) → ChainOfThought
    ├── Complex (math/computation) → ProgramOfThought
    └── Very complex (compare approaches) → MultiChainComparison

Module tradeoff summary:

For the full module list including Refine, RLM, and Parallel, see reference.md.

---

Step 3: Single module vs pipeline

Use this table to decide whether one module is enough or a pipeline is warranted:

Rule of thumb: start with a single module. Add pipeline stages only when you have measured a quality gap that a single module cannot close.

Verification: After implementing the chosen architecture, run dspy.Evaluate(devset, metric=your_metric) on 20-50 examples to confirm the module choice was correct before optimizing.

---

Step 4: Architecture-to-optimizer pairing

---

Step 5: Generate the recommendation

Output the recommendation in this format:

## Architecture Recommendation

**Module:** dspy.ChainOfThought (or whatever was chosen)
**Why:** [1-2 sentences tying the module to the task]
**Skeleton:**
[minimal code showing the module or pipeline structure]

**Optimizer path:**
1. Start with BootstrapFewShot (quick baseline)
2. Move to MIPROv2 if accuracy needs to improve

**Alternative considered:** [what else was considered and why it was not chosen]

---

Skeleton code templates

1. Single Predict (simplest)

import dspy

class MyTask(dspy.Signature):
    """One sentence describing the task."""
    input_text: str = dspy.InputField()
    output_label: str = dspy.OutputField()

predictor = dspy.Predict(MyTask)
result = predictor(input_text="...")
print(result.output_label)

2. Single ChainOfThought (default choice)

import dspy

class MyTask(dspy.Signature):
    """One sentence describing the task."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

cot = dspy.ChainOfThought(MyTask)
result = cot(question="...")
print(result.answer)

3. ReAct with tools

import dspy

def search(query: str) -> str:
    """Search external knowledge base."""
    ...

def lookup(term: str) -> str:
    """Look up a term in a database."""
    ...

class MyAgentTask(dspy.Signature):
    """Answer questions using search and lookup tools."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

agent = dspy.ReAct(MyAgentTask, tools=[search, lookup])
result = agent(question="...")
print(result.answer)

4. Two-stage pipeline (classify then generate)

import dspy

class Classify(dspy.Signature):
    """Classify the input into a category."""
    text: str = dspy.InputField()
    category: str = dspy.OutputField()

class Generate(dspy.Signature):
    """Generate a response given the category and original text."""
    text: str = dspy.InputField()
    category: str = dspy.InputField()
    response: str = dspy.OutputField()

class ClassifyThenGenerate(dspy.Module):
    def __init__(self):
        self.classify = dspy.Predict(Classify)
        self.generate = dspy.ChainOfThought(Generate)

    def forward(self, text: str) -> dspy.Prediction:
        category = self.classify(text=text).category
        response = self.generate(text=text, category=category).response
        return dspy.Prediction(category=category, response=response)

5. Three-stage RAG pipeline (retrieve, reason, generate)

import dspy

retriever = dspy.Retrieve(k=3)

class Reason(dspy.Signature):
    """Given context passages, identify the key facts relevant to the question."""
    question: str = dspy.InputField()
    context: list[str] = dspy.InputField()
    key_facts: str = dspy.OutputField()

class Answer(dspy.Signature):
    """Answer the question using the identified key facts."""
    question: str = dspy.InputField()
    key_facts: str = dspy.InputField()
    answer: str = dspy.OutputField()

class RAGPipeline(dspy.Module):
    def __init__(self):
        self.retrieve = retriever
        self.reason = dspy.ChainOfThought(Reason)
        self.answer = dspy.ChainOfThought(Answer)

    def forward(self, question: str) -> dspy.Prediction:
        passages = self.retrieve(question).passages
        key_facts = self.reason(question=question, context=passages).key_facts
        answer = self.answer(question=question, key_facts=key_facts).answer
        return dspy.Prediction(answer=answer, passages=passages)

---

Gotchas

1. Defaulting to ChainOfThought for everything. Predict is better for simple classification or extraction where reasoning adds noise, not signal. If the correct output is a fixed label from a known set, CoT can hallucinate reasoning that leads it astray.

2. Using ReAct when a pipeline suffices. ReAct is for tasks that need dynamic tool selection at runtime. If you know the steps upfront (e.g., always retrieve then answer), use a pipeline — it is cheaper, faster, and easier to optimize.

3. Over-engineering with MultiChainComparison. MCC runs 3-5x the cost of a single pass. Only reach for it after measuring that single-pass accuracy is insufficient for your use case.

4. Building a pipeline before proving a single module works. Always start with the simplest module that could work. Measure it on your eval set. Add pipeline stages only when you have a specific, measured quality gap.

5. Ignoring cost implications early. A ReAct agent with 10 tool calls costs roughly 10x a single Predict call. Factor cost and latency into architecture decisions before you build, not after.

---

Cross-references

Install any skill: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>

• For full module comparison tables and complete code templates, see reference.md
• For worked architecture decisions with real examples, see examples.md
• Ready to build? Use the matching /dspy-* skill for your chosen module
• Need to implement a pipeline? Use /ai-building-pipelines
• Want to plan the full project? Use /ai-planning
• Need to review existing code? Use /ai-auditing-code
• Install `/ai-do` if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do

ai-choosing-architecture: Worked Examples

---

Example 1: Support ticket classifier

The task

A SaaS company receives ~2,000 support tickets per day. They want to automatically assign each ticket to one of four queues: billing, technical, account, other. A human agent reviews the assignment before acting on it.

3 questions answered

1. What goes in and what comes out? In: plain-text ticket body (50-500 words). Out: a single label from a fixed set of four values. 2. Does the AI need external tools? No. The category is determinable from the ticket text alone. 3. How complex is the reasoning? Simple. The mapping from text to category is direct. A human can usually classify a ticket in under five seconds.

Decision tree walkthrough

Does it need tools? → No
How complex is the reasoning? → Simple (direct mapping)
→ Predict

ChainOfThought is tempting here but wrong. The reasoning trace adds tokens and cost but does not improve label accuracy — the model may even talk itself into the wrong category by over-analyzing edge cases. Predict is the right choice.

Recommendation

Module: dspy.Predict Why: Direct label assignment from a fixed taxonomy. Reasoning adds noise, not signal. Predict is faster and cheaper.

Skeleton:

import dspy

class ClassifyTicket(dspy.Signature):
    """Classify a support ticket into exactly one support queue."""
    ticket_text: str = dspy.InputField()
    queue: str = dspy.OutputField(
        desc="One of: billing, technical, account, other"
    )

classifier = dspy.Predict(ClassifyTicket)

# Optimize with BootstrapFewShot on 100-200 labeled tickets
def exact_match(example, pred, trace=None):
    return example.queue == pred.queue

optimizer = dspy.BootstrapFewShot(metric=exact_match, max_bootstrapped_demos=6)
optimized_classifier = optimizer.compile(classifier, trainset=labeled_tickets)

Optimizer path: 1. BootstrapFewShot — quick baseline, should reach 85-90% accuracy 2. MIPROv2 if accuracy needs to improve beyond that

Alternative considered: ChainOfThought — rejected because the task is a fixed-label classification and reasoning does not help; it adds cost and can introduce label drift on edge cases.

---

Example 2: Customer Q&A over product docs

The task

A B2B software company wants a chatbot that answers customer questions using their product documentation (300 markdown files, ~2M tokens total). Answers must be grounded in the docs, not hallucinated.

3 questions answered

1. What goes in and what comes out? In: a natural-language question. Out: a plain-text answer with source references. 2. Does the AI need external tools? Yes — retrieval from the doc corpus. The docs are too large to fit in context. 3. How complex is the reasoning? Moderate. The model needs to identify the relevant parts of retrieved passages and synthesize a coherent answer.

Decision tree walkthrough

Does it need tools? → Yes (retrieval)
Does it need to write and run code? → No
→ ReAct ... but wait

ReAct is the first branch for tool use, but retrieval-augmented generation is a well-known pattern where the tool call sequence is fixed and known upfront: always retrieve, then answer. A pipeline is preferable to ReAct here because:

• The steps are predetermined (no dynamic tool selection needed)
• Pipelines are cheaper and easier to optimize than ReAct
• The retrieval step can be a standard DSPy Retrieve module

Revised decision: Use a RAG pipeline (Retrieve + ChainOfThought).

Recommendation

Module: RAG pipeline — dspy.Retrieve + dspy.ChainOfThought Why: The retrieval step is always the same (fetch top-k passages). ChainOfThought synthesizes the answer with visible reasoning, making hallucination easier to detect.

Skeleton:

import dspy

lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

retriever = dspy.Retrieve(k=5)

class AnswerFromDocs(dspy.Signature):
    """Answer the customer question using only the provided documentation passages."""
    question: str = dspy.InputField()
    passages: list[str] = dspy.InputField(desc="Relevant documentation excerpts")
    answer: str = dspy.OutputField(desc="Clear answer grounded in the passages")
    source_hint: str = dspy.OutputField(desc="Which passage(s) supported this answer")

class DocQA(dspy.Module):
    def __init__(self):
        self.retrieve = retriever
        self.answer = dspy.ChainOfThought(AnswerFromDocs)

    def forward(self, question: str) -> dspy.Prediction:
        passages = self.retrieve(question).passages
        result = self.answer(question=question, passages=passages)
        return dspy.Prediction(
            answer=result.answer,
            source_hint=result.source_hint,
        )

qa = DocQA()
result = qa(question="How do I configure SSO with Okta?")
print(result.answer)

Optimizer path: 1. BootstrapFewShot on 50-100 QA pairs 2. MIPROv2 if answer quality needs to improve (optimize both the reasoning and answer stages end-to-end)

Alternative considered: Single ChainOfThought with full docs in context — rejected because the corpus is too large. ReAct — rejected because the retrieval pattern is fixed and a pipeline is simpler and cheaper.

---

Example 3: AI research assistant

The task

A market research firm wants an AI assistant that can research any company on demand: find recent news, pull financial data, and summarize competitive positioning. The set of data sources to consult varies by question.

3 questions answered

1. What goes in and what comes out? In: a research question (e.g., "What is Stripe's competitive position in payments?"). Out: a structured research summary. 2. Does the AI need external tools? Yes — web search, financial data APIs, news feeds. The specific tools needed depend on the question. 3. How complex is the reasoning? High. The agent must decide which sources to consult, in what order, and how to synthesize conflicting information.

Decision tree walkthrough

Does it need tools? → Yes
Does it need to write and run code? → No
→ ReAct

Unlike the doc QA example, here the tool selection is dynamic. A question about a private company needs different tools than a question about a public company. ReAct's ability to choose tools at runtime is the right fit.

Recommendation

Module: dspy.ReAct Why: The task genuinely requires dynamic tool selection. The agent decides at runtime which sources to consult based on what it learns from each tool call.

Skeleton:

import dspy

lm = dspy.LM("openai/gpt-4o")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

def search_news(query: str) -> str:
    """Search recent news articles. Returns titles and snippets."""
    ...

def get_financial_data(ticker: str) -> str:
    """Get key financial metrics for a public company by stock ticker."""
    ...

def search_web(query: str) -> str:
    """General web search for any topic."""
    ...

class ResearchCompany(dspy.Signature):
    """Research a company and produce a structured competitive summary."""
    question: str = dspy.InputField()
    summary: str = dspy.OutputField(
        desc="Structured summary covering: overview, recent news, competitive position"
    )

agent = dspy.ReAct(
    ResearchCompany,
    tools=[search_news, get_financial_data, search_web],
    max_iters=8,
)

result = agent(question="What is Stripe competitive position in payments as of 2024?")
print(result.summary)

Optimizer path: 1. BootstrapFewShot with 10-20 research examples — agents need fewer demos than classifiers 2. Stay with BootstrapFewShot; MIPROv2 can be unstable for multi-turn agent traces

Alternative considered: Fixed RAG pipeline — rejected because the sources vary by question type. Pipeline with hardcoded stages — rejected because the number and order of API calls is not known upfront.

---

Example 4: Essay grading system

The task

An online learning platform grades student essays on a 1-5 rubric covering: thesis clarity, argument strength, evidence quality, and writing mechanics. Grades must be consistent — the same essay should always get the same score.

3 questions answered

1. What goes in and what comes out? In: a student essay (200-1,000 words). Out: four integer scores (1-5) plus brief justifications. 2. Does the AI need external tools? No. Grading is based solely on the essay text. 3. How complex is the reasoning? High. Rubric application requires careful analysis of multiple dimensions. Consistency across runs is critical.

Decision tree walkthrough

Does it need tools? → No
How complex is the reasoning? → Very complex (multi-dimensional, must be consistent)
→ MultiChainComparison or ChainOfThought + BestOfN

MultiChainComparison generates N reasoning chains and compares them to pick the most consistent answer — good for subjective tasks where consistency matters. BestOfN with a scoring function is an alternative if a reliable scorer exists.

For essay grading, ChainOfThought with BestOfN (where the scorer checks rubric adherence and internal consistency) is often more practical than MCC because you can define the scoring criteria explicitly.

Recommendation

Module: dspy.ChainOfThought + dspy.BestOfN Why: ChainOfThought applies the rubric with visible reasoning. BestOfN samples multiple grading attempts and selects the most internally consistent one, addressing the consistency requirement.

Skeleton:

import dspy

lm = dspy.LM("openai/gpt-4o")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

RUBRIC = """
Score each dimension 1-5:
- thesis_clarity: Is the main argument clearly stated?
- argument_strength: Is the argument logically sound?
- evidence_quality: Is evidence relevant and well-cited?
- writing_mechanics: Is grammar, spelling, and structure correct?
"""

class GradeEssay(dspy.Signature):
    """Grade a student essay using the provided rubric. Be consistent and fair."""
    essay: str = dspy.InputField()
    rubric: str = dspy.InputField()
    thesis_clarity: int = dspy.OutputField(desc="Score 1-5")
    argument_strength: int = dspy.OutputField(desc="Score 1-5")
    evidence_quality: int = dspy.OutputField(desc="Score 1-5")
    writing_mechanics: int = dspy.OutputField(desc="Score 1-5")
    justification: str = dspy.OutputField(desc="2-3 sentences explaining the scores")

grader_module = dspy.ChainOfThought(GradeEssay)

def consistency_score(args, prediction) -> float:
    """Score a prediction by checking that all scores are in range and the
    justification references at least two rubric dimensions."""
    scores = [
        prediction.thesis_clarity,
        prediction.argument_strength,
        prediction.evidence_quality,
        prediction.writing_mechanics,
    ]
    if not all(1 <= s <= 5 for s in scores):
        return 0.0
    rubric_terms = ["thesis", "argument", "evidence", "mechanics", "writing"]
    mentions = sum(1 for t in rubric_terms if t in prediction.justification.lower())
    return min(1.0, mentions / 2)

grader = dspy.BestOfN(
    module=grader_module,
    N=3,
    reward_fn=consistency_score,
    threshold=0.8,
)

result = grader(essay=student_essay, rubric=RUBRIC)
print(result.thesis_clarity, result.argument_strength)
print(result.justification)

Optimizer path: 1. BootstrapFewShot on 50 human-graded essays 2. MIPROv2 if inter-rater agreement with human graders is below target

Alternative considered: MultiChainComparison — considered but BestOfN with an explicit scorer is preferred here because the scoring criteria are well-defined and inspectable. Single ChainOfThought — rejected because consistency across runs was measured to be insufficient without sampling.

ai-choosing-architecture: Reference

Full module comparison table

When to use each module (extended notes)

Predict — Use when the task is a direct mapping and reasoning would add noise. Examples: sentiment classification, named entity extraction, format conversion, label assignment from a known taxonomy. If the correct answer is deterministic given the input, Predict is the right choice.

ChainOfThought — Use as the default for any task where the model benefits from thinking before answering. Open-ended question answering, summarization, explanation generation, and tasks with ambiguous inputs all benefit from CoT. The reasoning trace also makes debugging easier.

ProgramOfThought — Use when the task involves arithmetic, counting, unit conversion, statistical computation, or any problem where writing a small Python program is more reliable than generating prose. ProgramOfThought generates executable code and runs it; the code output becomes the answer.

ReAct — Use when the correct answer depends on information that is not in the context at call time. Examples: answering questions about current events (needs search), looking up customer records (needs database), checking live prices (needs API). Each tool call is a round-trip LM invocation, so keep the tool set small and focused.

CodeAct — Use when the task itself is to write and run code, or when the reasoning loop should be expressed as code execution steps rather than natural language. More powerful than ReAct for coding-heavy workflows.

MultiChainComparison — Use when single-pass accuracy is measurably insufficient and cost is not the primary constraint. Runs N independent reasoning chains and selects the most consistent answer. Effective for high-stakes single answers (medical triage, legal classification).

BestOfN — Use when you have a reliable scoring function (a reward model, a unit test, a regex check). Runs the module up to N times at temperature 1.0, returns the first output exceeding threshold or the highest-scoring output. Requires module, N, reward_fn, and threshold parameters.

Refine — Use when you want sampling with reward-guided selection. Runs the module up to N times at temperature 1.0, selects the first output exceeding a reward threshold or the highest-scoring output overall. Similar to BestOfN but integrated with DSPy's refinement tracking. Requires module, N, reward_fn, and threshold parameters.

Parallel — Use for fan-out patterns: summarizing many documents independently, classifying a batch of items, running the same task over multiple inputs simultaneously. Reduces wall-clock time when the sub-tasks are independent.

---

Architecture templates

Template 1: Simple classifier (Predict)

import dspy

lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

class ClassifyTicket(dspy.Signature):
    """Classify a support ticket into exactly one category."""
    ticket_text: str = dspy.InputField(desc="The raw support ticket text")
    category: str = dspy.OutputField(
        desc="One of: billing, technical, account, other"
    )

classifier = dspy.Predict(ClassifyTicket)

# Basic usage
result = classifier(ticket_text="My invoice shows the wrong amount.")
print(result.category)  # "billing"

# With optimization
def accuracy_metric(example, pred, trace=None):
    return example.category == pred.category

optimizer = dspy.BootstrapFewShot(metric=accuracy_metric, max_bootstrapped_demos=4)
optimized = optimizer.compile(classifier, trainset=trainset)

Template 2: Reasoning task (ChainOfThought)

import dspy

lm = dspy.LM("openai/gpt-4o")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

class AnswerQuestion(dspy.Signature):
    """Answer the question based on the provided context."""
    context: str = dspy.InputField(desc="Background information")
    question: str = dspy.InputField(desc="The question to answer")
    answer: str = dspy.OutputField(desc="A clear, concise answer")

answerer = dspy.ChainOfThought(AnswerQuestion)

result = answerer(
    context="DSPy is a framework for programming language models...",
    question="What is DSPy used for?"
)
print(result.reasoning)  # The chain-of-thought trace
print(result.answer)

# Optimize with MIPROv2 for best quality
optimizer = dspy.MIPROv2(metric=your_metric, auto="medium")
optimized = optimizer.compile(answerer, trainset=trainset, valset=valset)

Template 3: Tool-using agent (ReAct)

import dspy

lm = dspy.LM("openai/gpt-4o")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

# Define tools as plain Python functions with docstrings
def search_web(query: str) -> str:
    """Search the web and return a summary of results."""
    # your search implementation
    ...

def fetch_page(url: str) -> str:
    """Fetch the content of a web page."""
    # your fetch implementation
    ...

def calculate(expression: str) -> str:
    """Evaluate a mathematical expression and return the result."""
    return str(eval(expression))  # use a safe evaluator in production

class ResearchTask(dspy.Signature):
    """Research a topic and provide a well-sourced answer."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="Detailed answer with sources cited")

agent = dspy.ReAct(ResearchTask, tools=[search_web, fetch_page, calculate])

result = agent(question="What was the GDP of Germany in 2023?")
print(result.answer)

# For agents, BootstrapFewShot is usually enough
optimizer = dspy.BootstrapFewShot(metric=your_metric, max_bootstrapped_demos=2)
optimized = optimizer.compile(agent, trainset=trainset)

Template 4: Classify-then-generate pipeline

import dspy

lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

class ClassifyIntent(dspy.Signature):
    """Classify the user intent from the support message."""
    message: str = dspy.InputField()
    intent: str = dspy.OutputField(
        desc="One of: question, complaint, cancellation_request, praise"
    )

class DraftReply(dspy.Signature):
    """Draft a professional support reply given the message and its intent."""
    message: str = dspy.InputField()
    intent: str = dspy.InputField()
    reply: str = dspy.OutputField(desc="Professional, empathetic reply under 150 words")

class SupportResponder(dspy.Module):
    def __init__(self):
        self.classify = dspy.Predict(ClassifyIntent)
        self.draft = dspy.ChainOfThought(DraftReply)

    def forward(self, message: str) -> dspy.Prediction:
        intent = self.classify(message=message).intent
        reply = self.draft(message=message, intent=intent).reply
        return dspy.Prediction(intent=intent, reply=reply)

responder = SupportResponder()
result = responder(message="I have been charged twice for my subscription.")
print(result.intent)
print(result.reply)

Template 5: RAG pipeline (retrieve, reason, generate)

import dspy

lm = dspy.LM("openai/gpt-4o")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

# Configure your retriever (colbert, faiss, weaviate, etc.)
retriever = dspy.Retrieve(k=5)

class IdentifyRelevantFacts(dspy.Signature):
    """From the retrieved passages, identify the facts most relevant to the question."""
    question: str = dspy.InputField()
    passages: list[str] = dspy.InputField(desc="Retrieved context passages")
    relevant_facts: str = dspy.OutputField(
        desc="Bullet-point list of facts directly relevant to the question"
    )

class SynthesizeAnswer(dspy.Signature):
    """Synthesize a final answer from the identified relevant facts."""
    question: str = dspy.InputField()
    relevant_facts: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="Clear, well-supported answer")
    citations: str = dspy.OutputField(desc="Which facts were used")

class RAGPipeline(dspy.Module):
    def __init__(self):
        self.retrieve = retriever
        self.identify = dspy.ChainOfThought(IdentifyRelevantFacts)
        self.synthesize = dspy.ChainOfThought(SynthesizeAnswer)

    def forward(self, question: str) -> dspy.Prediction:
        passages = self.retrieve(question).passages
        facts = self.identify(question=question, passages=passages).relevant_facts
        result = self.synthesize(question=question, relevant_facts=facts)
        return dspy.Prediction(
            answer=result.answer,
            citations=result.citations,
            passages=passages,
        )

rag = RAGPipeline()
result = rag(question="What are the refund policy terms?")
print(result.answer)

---

Optimizer pairing details

BootstrapFewShot

• Best for: Getting a quick baseline on any architecture. Works well when you have 20-100 labeled examples.
• How it works: Runs the program on training examples, collects traces where the metric passes, and uses those traces as few-shot demonstrations.
• When to move on: If accuracy plateaus after 4-8 demos, switch to MIPROv2.

MIPROv2

• Best for: Single modules and pipelines where you have 50+ examples and care about maximum quality.
• How it works: Bayesian optimization over both the instruction and the few-shot demonstrations. Tries many prompt variants and picks the best.
• Cost: More expensive to run than BootstrapFewShot (many optimization calls). Run it once, save the optimized program.

BootstrapFinetune

• Best for: High-traffic production pipelines where inference cost matters. Generates training data from successful traces and fine-tunes the LM weights.
• Requires: A fine-tunable model (GPT-4o fine-tune, local model, etc.).

BetterTogether

• Best for: Combining prompt optimization and fine-tuning. Alternates between MIPROv2 (prompt) and BootstrapFinetune (weights) to get the best of both.
• Use when: You need both high quality and low inference cost.

---

Cost estimation rules of thumb

These are rough multipliers. Actual cost depends on token counts, model choice, and retrieval corpus size. Always measure on real inputs before scaling.