Dspy Multi Chain Comparison

Get Better Answers by Comparing Multiple Reasoning Chains

Guide the user through using dspy.MultiChainComparison to improve answer quality. Instead of relying on a single chain of thought, you generate several independent reasoning chains yourself and then hand them to this module, which selects the best final answer by comparing them.

What is MultiChainComparison

dspy.MultiChainComparison is a DSPy module that:

1. Takes M pre-generated reasoning chains -- you generate these yourself first (one ChainOfThought call with n=M), then pass them in 2. Compares the candidates -- a single comparison/synthesis LM call evaluates all chains and picks the best answer 3. Returns a single answer -- the output looks the same as any other DSPy module

Important: MultiChainComparison does NOT generate the chains for you. You generate the M completions first, then pass them as the first positional argument to the module. The module itself makes exactly 1 LM call -- the synthesis step.

Think of it as getting multiple opinions from different experts, then having a judge pick the most convincing one. The diversity of reasoning paths surfaces better answers than any single chain alone.

When to use MultiChainComparison

Use it when:

• Quality matters more than speed -- you can afford extra LM calls for a better answer
• Tasks have genuine ambiguity -- multiple valid approaches exist and you want the best one
• Single CoT is unreliable -- the model sometimes reasons poorly and you want redundancy
• High-stakes decisions -- recommendations, diagnoses, critical analysis where being wrong is costly

Do NOT use it when:

• Latency is critical -- generating M chains (one ChainOfThought call with n=M) plus the comparison call is slower than a single ChainOfThought
• The task is straightforward -- simple classification, extraction, or lookup does not benefit from multiple chains
• Cost is a hard constraint -- generating M chains plus the synthesis call costs more than a single ChainOfThought
• You need deterministic output -- the comparison step adds variability

Basic usage

Using MultiChainComparison is a two-step process. First you generate M reasoning chains yourself with one ChainOfThought call (set n=M), then you pass the resulting .completions list as the first positional argument to the MultiChainComparison instance:

import dspy

lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

# Step 1: generate M=5 reasoning chains yourself (one ChainOfThought call with n=M)
generate = dspy.ChainOfThought("problem -> recommendation", n=5)
completions = generate(
    problem="We need to migrate from MongoDB to PostgreSQL for our 50GB dataset with complex joins"
).completions  # list of M completions

# Step 2: pass the completions to MultiChainComparison to synthesize the best answer
compare = dspy.MultiChainComparison("problem -> recommendation", M=5)
result = compare(
    completions,
    problem="We need to migrate from MongoDB to PostgreSQL for our 50GB dataset with complex joins",
)
print(result.recommendation)

Make sure M on the MultiChainComparison instance matches the number of completions you generated (n on the ChainOfThought generator).

With a class-based signature -- same two-step pattern:

import dspy

class TechRecommendation(dspy.Signature):
    """Recommend the best technical approach for this problem."""
    problem: str = dspy.InputField(desc="Technical problem or decision to make")
    constraints: str = dspy.InputField(desc="Budget, timeline, or technical constraints")
    recommendation: str = dspy.OutputField(desc="The recommended approach with justification")

inputs = dict(
    problem="Our API response times are over 2 seconds under load",
    constraints="Small team, no budget for new infrastructure",
)

# Step 1: generate the chains
generate = dspy.ChainOfThought(TechRecommendation, n=3)
completions = generate(**inputs).completions

# Step 2: compare and synthesize
compare = dspy.MultiChainComparison(TechRecommendation, M=3)
result = compare(completions, **inputs)
print(result.recommendation)

How it works internally

The work is split across two steps -- the first is yours, the second is the module's:

1. You generate the chains -- one ChainOfThought call with n=M produces M completions, each with its own reasoning and output fields. This is a single LM call (the provider returns M samples). 2. You pass the completions in -- compare(completions, **inputs) hands the M pre-generated chains to MultiChainComparison. 3. The module runs one comparison step -- a single LM call sees all candidate chains and selects/synthesizes the best answer.

The comparison step is the key differentiator. Rather than picking randomly or voting, the model actively evaluates the quality of each reasoning chain before choosing. MultiChainComparison itself contributes exactly 1 LM call -- the synthesis. It does not generate the chains.

Input --> ChainOfThought(n=M) --> reasoning_1 + answer_1 --|
                                  reasoning_2 + answer_2 --|--> MultiChainComparison --> best answer
                                  reasoning_3 + answer_3 --|   (1 synthesis LM call)
       (1 LM call returning M completions)

Configuring the number of chains

By default, MultiChainComparison expects 3 chains. M tells the module how many completions to expect (it must match the n you used when generating them). You can adjust M and temperature:

# Constructor signature
dspy.MultiChainComparison(signature, M=3, temperature=0.7, **config)

• M — number of reasoning chains the module expects to receive (default 3); set the ChainOfThought generator's n to the same value.
• temperature — sampling temperature for the comparison step (default 0.7). When generating chains, set the temperature on the ChainOfThought generator; higher values produce more diverse chains, which gives the comparison step more to work with.

Guidelines for choosing M (the chains are 1 ChainOfThought call with n=M; MultiChainComparison adds 1 synthesis call):

Using MultiChainComparison in a module

Wrap it in a dspy.Module to combine with other steps:

import dspy
from typing import Literal

class RiskAssessment(dspy.Signature):
    """Assess the risk level of this proposed change."""
    change_description: str = dspy.InputField(desc="What is being changed")
    system_context: str = dspy.InputField(desc="The system being modified")
    risk_level: Literal["low", "medium", "high", "critical"] = dspy.OutputField()
    risk_factors: str = dspy.OutputField(desc="Key risks identified")
    mitigation: str = dspy.OutputField(desc="Recommended mitigation steps")

class ChangeReviewer(dspy.Module):
    def __init__(self, M=3):
        self.M = M
        self.classify = dspy.Predict("change_description -> change_type: str")
        # Generator produces M chains; MCC synthesizes the best answer
        self.generate = dspy.ChainOfThought(RiskAssessment, n=M)
        self.assess = dspy.MultiChainComparison(RiskAssessment, M=M)

    def forward(self, change_description, system_context):
        change_type = self.classify(change_description=change_description).change_type
        inputs = dict(
            change_description=f"[{change_type}] {change_description}",
            system_context=system_context,
        )
        # Step 1: generate M reasoning chains
        completions = self.generate(**inputs).completions
        # Step 2: compare and synthesize the best answer
        result = self.assess(completions, **inputs)
        return dspy.Prediction(
            change_type=change_type,
            risk_level=result.risk_level,
            risk_factors=result.risk_factors,
            mitigation=result.mitigation,
        )

# Usage
lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

reviewer = ChangeReviewer()
result = reviewer(
    change_description="Drop the users_v1 table and migrate all queries to users_v2",
    system_context="Production e-commerce platform with 10k daily active users",
)
print(f"Risk: {result.risk_level}")
print(f"Factors: {result.risk_factors}")
print(f"Mitigation: {result.mitigation}")

Cost and latency tradeoffs

The two-step pattern trades speed and cost for quality. The chain generation is 1 ChainOfThought call (with n=M) and MultiChainComparison adds 1 synthesis call. Here is a rough comparison with M=3:

Note: even though there are only 2 LM calls, the generate call samples M completions, so token cost scales with M (roughly M generations + 1 synthesis).

Strategies to manage cost:

• Use a cheaper model for chains, an expensive model for comparison -- the comparison step benefits most from a strong model, while the chains can be sampled cheaply:

cheap_lm = dspy.LM("openai/gpt-4o-mini")  # or any smaller model
expensive_lm = dspy.LM("openai/gpt-4o")  # or "anthropic/claude-sonnet-4-5-20250929", etc.

# Generate the M chains with the cheap model
generate = dspy.ChainOfThought("problem -> recommendation", n=5)
generate.set_lm(cheap_lm)

# Run the synthesis/comparison with the expensive model
compare = dspy.MultiChainComparison("problem -> recommendation", M=5)
compare.set_lm(expensive_lm)

problem = "Our API response times are over 2 seconds under load"
completions = generate(problem=problem).completions
result = compare(completions, problem=problem)
print(result.recommendation)

• Use MultiChainComparison selectively -- route only hard tasks through it:

class AdaptiveReasoner(dspy.Module):
    def __init__(self, M=3):
        self.M = M
        self.classify_difficulty = dspy.Predict("question -> difficulty: str")
        self.fast = dspy.ChainOfThought("question -> answer")
        self.generate = dspy.ChainOfThought("question -> answer", n=M)
        self.compare = dspy.MultiChainComparison("question -> answer", M=M)

    def forward(self, question):
        difficulty = self.classify_difficulty(question=question).difficulty.lower()
        if "hard" in difficulty or "complex" in difficulty:
            completions = self.generate(question=question).completions
            return self.compare(completions, question=question)
        return self.fast(question=question)

Optimizing MultiChainComparison

MultiChainComparison modules are optimizable like any other DSPy module. Because the chains are generated by a separate ChainOfThought step, wrap both steps in a dspy.Module so the optimizer can tune the prompts for both the generation and the comparison/synthesis steps:

def quality_metric(example, prediction, trace=None):
    return prediction.answer.strip().lower() == example.answer.strip().lower()

class CompareReasoner(dspy.Module):
    def __init__(self, M=3):
        self.generate = dspy.ChainOfThought("question -> answer", n=M)
        self.compare = dspy.MultiChainComparison("question -> answer", M=M)

    def forward(self, question):
        completions = self.generate(question=question).completions
        return self.compare(completions, question=question)

program = CompareReasoner(M=3)

optimizer = dspy.BootstrapFewShot(metric=quality_metric, max_bootstrapped_demos=4)
optimized = optimizer.compile(program, trainset=trainset)

# Save for production
optimized.save("optimized_mcc.json")

For best results with MIPROv2:

optimizer = dspy.MIPROv2(metric=quality_metric, auto="medium")
optimized = optimizer.compile(program, trainset=trainset)

When NOT to use MultiChainComparison

Pick a simpler alternative when:

MultiChainComparison is most valuable when the problem genuinely benefits from diverse perspectives -- not when there is a single clearly correct approach.

Gotchas

1. Claude calls MultiChainComparison directly with raw inputs. MultiChainComparison does NOT generate the chains. You must generate M completions first (dspy.ChainOfThought(sig, n=M)), grab .completions, then pass that list as the first positional argument: compare(completions, **inputs). Calling compare(problem=...) without completions is wrong. 2. Claude forgets to sample diverse chains. Diversity comes from the ChainOfThought generator. Generate with n=M and a non-zero temperature on the generator — with temperature=0 the chains are near-identical and the comparison step adds cost with no quality gain. Keep the default temperature=0.7 or higher. 3. Claude uses MultiChainComparison for simple tasks. For straightforward classification, extraction, or lookup, the generate-plus-synthesize pattern adds cost with no quality improvement. Use dspy.Predict or dspy.ChainOfThought for simple tasks and reserve MultiChainComparison for genuinely ambiguous or high-stakes decisions. 4. Claude sets M too high. Beyond M=5, diminishing returns set in quickly — each additional chain adds a sampled generation but contributes marginal diversity. Start with M=3 and only increase if evaluation shows improvement. Also keep M on the module equal to the generator's n. 5. Claude ignores the cost during optimization. Optimizing a wrapper module that generates M chains plus a synthesis call means every trial makes 2 LM calls and samples M completions. With many trials this adds up. Use auto="light" for MIPROv2 or keep trial counts low.

Cross-references

Install any skill: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>

• ChainOfThought is the single-chain version and the right default -- see /dspy-chain-of-thought
• Reasoning strategies including when to pick MultiChainComparison vs other approaches -- see /ai-reasoning
• Improving accuracy with evaluation and optimization -- see /ai-improving-accuracy
• Install `/ai-do` if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do

Additional resources

• dspy.MultiChainComparison API docs
• For API details, see reference.md
• For worked examples, see examples.md

dspy-multi-chain-comparison -- Worked Examples

Example 1: Complex analysis with multi-chain comparison

An architecture review module that uses multiple reasoning chains to evaluate a proposed system design. Each chain considers the design independently, and the comparison step picks the most thorough analysis.

import dspy
from typing import Literal
from pydantic import BaseModel, Field


# --- Signatures ---

class ArchitectureReview(dspy.Signature):
    """Review a proposed system architecture and identify strengths, weaknesses, and recommendations."""
    design_description: str = dspy.InputField(desc="The proposed architecture or design")
    requirements: str = dspy.InputField(desc="Key requirements the design must satisfy")
    strengths: str = dspy.OutputField(desc="What the design does well")
    weaknesses: str = dspy.OutputField(desc="Gaps, risks, or areas of concern")
    recommendation: str = dspy.OutputField(desc="Concrete next steps to improve the design")
    overall_rating: Literal["strong", "adequate", "needs_work", "risky"] = dspy.OutputField()


# --- Module ---

class ArchitectureReviewer(dspy.Module):
    """Review a system design using multiple reasoning chains for thorough analysis."""

    def __init__(self, num_chains=3):
        self.num_chains = num_chains
        self.extract_requirements = dspy.ChainOfThought(
            "design_description -> key_requirements: str"
        )
        # Generate the M chains, then compare/synthesize the best answer
        self.generate = dspy.ChainOfThought(ArchitectureReview, n=num_chains)
        self.review = dspy.MultiChainComparison(ArchitectureReview, M=num_chains)

    def forward(self, design_description, requirements=""):
        # If no explicit requirements, extract them from the design
        if not requirements.strip():
            extracted = self.extract_requirements(design_description=design_description)
            requirements = extracted.key_requirements

        inputs = dict(
            design_description=design_description,
            requirements=requirements,
        )
        # Step 1: generate M independent reasoning chains
        completions = self.generate(**inputs).completions
        # Step 2: compare them and synthesize the strongest review
        result = self.review(completions, **inputs)

        return dspy.Prediction(
            strengths=result.strengths,
            weaknesses=result.weaknesses,
            recommendation=result.recommendation,
            overall_rating=result.overall_rating,
        )


# --- Usage ---

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

reviewer = ArchitectureReviewer(num_chains=3)

result = reviewer(
    design_description="""
    We're building a real-time analytics dashboard. The plan:
    - React frontend with WebSocket connections to a Node.js API
    - Node.js API queries PostgreSQL directly for all data
    - PostgreSQL stores raw events (500M rows, growing 2M/day)
    - Cron job runs nightly to compute aggregate tables
    - Single server deployment behind Nginx
    """,
    requirements="""
    - Dashboard must load in under 2 seconds
    - Support 200 concurrent users
    - Show data no more than 5 minutes old
    - Budget: $500/month infrastructure
    """,
)

print(f"Rating: {result.overall_rating}")
print(f"\nStrengths:\n{result.strengths}")
print(f"\nWeaknesses:\n{result.weaknesses}")
print(f"\nRecommendation:\n{result.recommendation}")


# --- Optimization ---

def review_metric(example, prediction, trace=None):
    """Score based on whether the review catches known issues."""
    # Check that known weaknesses are mentioned
    known_issues = example.known_issues  # list of strings
    weaknesses_text = prediction.weaknesses.lower()

    issues_found = sum(
        1 for issue in known_issues
        if issue.lower() in weaknesses_text
    )
    coverage = issues_found / len(known_issues) if known_issues else 0

    # Check rating matches expected
    rating_correct = prediction.overall_rating == example.expected_rating

    return 0.6 * coverage + 0.4 * rating_correct


# trainset = [
#     dspy.Example(
#         design_description="...",
#         requirements="...",
#         known_issues=["single point of failure", "no caching layer"],
#         expected_rating="needs_work",
#     ).with_inputs("design_description", "requirements"),
# ]
# optimizer = dspy.BootstrapFewShot(metric=review_metric, max_bootstrapped_demos=4)
# optimized = optimizer.compile(reviewer, trainset=trainset)
# optimized.save("architecture_reviewer.json")

Key points:

• MultiChainComparison shines here because architecture reviews benefit from multiple perspectives -- one chain might catch scalability issues while another notices security gaps
• The comparison step picks the analysis that covers the most ground, not just the first one generated
• Pairing with ChainOfThought for requirement extraction shows how to compose MultiChainComparison inside a larger module
• The metric checks whether the review catches known issues, not just whether it produces any output

Example 2: Decision-making with multiple perspectives

A hiring decision module that evaluates a candidate from multiple angles. Each chain reasons independently about the candidate's fit, and the comparison step synthesizes the most balanced assessment.

import dspy
from typing import Literal
from pydantic import BaseModel, Field


# --- Signatures ---

class CandidateEvaluation(dspy.Signature):
    """Evaluate a job candidate based on their profile and the role requirements."""
    candidate_profile: str = dspy.InputField(desc="Candidate's experience, skills, and background")
    role_requirements: str = dspy.InputField(desc="What the role needs")
    team_context: str = dspy.InputField(desc="Current team composition and gaps")
    strengths_for_role: str = dspy.OutputField(desc="How the candidate's strengths match the role")
    concerns: str = dspy.OutputField(desc="Potential gaps or risks")
    growth_areas: str = dspy.OutputField(desc="Where the candidate would need to develop")
    hiring_recommendation: Literal["strong_yes", "yes", "maybe", "no"] = dspy.OutputField()


class InterviewQuestions(dspy.Signature):
    """Generate targeted interview questions based on the evaluation."""
    evaluation_summary: str = dspy.InputField(desc="Summary of the candidate evaluation")
    concerns: str = dspy.InputField(desc="Areas of concern to probe")
    questions: list[str] = dspy.OutputField(
        desc="3-5 targeted interview questions to address the concerns"
    )


# --- Module ---

class HiringAdvisor(dspy.Module):
    """Evaluate a candidate from multiple perspectives and generate interview questions."""

    def __init__(self, M=4):
        self.M = M
        # Generate the M chains, then compare/synthesize the best evaluation
        self.generate = dspy.ChainOfThought(CandidateEvaluation, n=M)
        self.evaluate = dspy.MultiChainComparison(CandidateEvaluation, M=M)
        self.generate_questions = dspy.ChainOfThought(InterviewQuestions)

    def forward(self, candidate_profile, role_requirements, team_context):
        # Stage 1: Multi-perspective evaluation
        # Each chain may weigh different factors -- technical depth, culture fit,
        # growth potential, risk tolerance. The comparison picks the most balanced view.
        eval_inputs = dict(
            candidate_profile=candidate_profile,
            role_requirements=role_requirements,
            team_context=team_context,
        )
        # Step 1: generate M reasoning chains
        completions = self.generate(**eval_inputs).completions
        # Step 2: compare and synthesize the most balanced assessment
        evaluation = self.evaluate(completions, **eval_inputs)

        # Stage 2: Generate targeted questions based on concerns
        questions_result = self.generate_questions(
            evaluation_summary=f"Strengths: {evaluation.strengths_for_role}",
            concerns=evaluation.concerns,
        )

        return dspy.Prediction(
            strengths=evaluation.strengths_for_role,
            concerns=evaluation.concerns,
            growth_areas=evaluation.growth_areas,
            recommendation=evaluation.hiring_recommendation,
            interview_questions=questions_result.questions,
        )


# --- Usage ---

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

advisor = HiringAdvisor()

result = advisor(
    candidate_profile="""
    5 years at a Series B startup as senior backend engineer.
    Strong in Python, PostgreSQL, Redis. Built their event processing pipeline
    handling 10k events/sec. Led a team of 3 for the last year.
    No experience with Go or Kubernetes. Has given conference talks on
    distributed systems. Left because the company was acquired.
    """,
    role_requirements="""
    Staff engineer role at a Series C company. Need someone who can:
    - Design and lead implementation of a new microservices platform
    - Mentor a team of 6 engineers (mix of mid and senior)
    - Work primarily in Go with Kubernetes on AWS
    - Handle ambiguity and drive technical decisions
    """,
    team_context="""
    Current team: 6 engineers, mostly mid-level, strong in Go but weak on
    system design. No staff engineer currently. Tech lead left 2 months ago.
    Team morale is low and they need a technical leader.
    """,
)

print(f"Recommendation: {result.recommendation}")
print(f"\nStrengths:\n{result.strengths}")
print(f"\nConcerns:\n{result.concerns}")
print(f"\nGrowth areas:\n{result.growth_areas}")
print(f"\nInterview questions:")
for i, q in enumerate(result.interview_questions, 1):
    print(f"  {i}. {q}")


# --- Optimization ---

def hiring_metric(example, prediction, trace=None):
    """Score based on recommendation accuracy and concern coverage."""
    # Check recommendation matches
    rec_correct = prediction.recommendation == example.expected_recommendation

    # Check that key concerns are raised
    concerns_text = prediction.concerns.lower()
    expected_concerns = example.expected_concerns  # list of strings
    concerns_found = sum(
        1 for c in expected_concerns
        if c.lower() in concerns_text
    )
    concern_coverage = concerns_found / len(expected_concerns) if expected_concerns else 1.0

    # Check interview questions are relevant (at least 3 generated)
    has_questions = len(prediction.interview_questions) >= 3

    return 0.4 * rec_correct + 0.4 * concern_coverage + 0.2 * has_questions


# trainset = [
#     dspy.Example(
#         candidate_profile="...",
#         role_requirements="...",
#         team_context="...",
#         expected_recommendation="yes",
#         expected_concerns=["no go experience", "no kubernetes experience"],
#     ).with_inputs("candidate_profile", "role_requirements", "team_context"),
# ]
# optimizer = dspy.MIPROv2(metric=hiring_metric, auto="medium")
# optimized = optimizer.compile(advisor, trainset=trainset)
# optimized.save("hiring_advisor.json")

Key points:

• Using M=4 chains gives more perspective diversity for a nuanced people-decision -- each chain may emphasize different aspects (technical fit, leadership readiness, culture alignment, risk)
• The comparison step picks the most balanced assessment rather than the most optimistic or pessimistic one
• Chaining MultiChainComparison into ChainOfThought for follow-up questions shows the natural composition pattern -- use the expensive multi-chain step where it matters most, then use cheaper single-chain steps for downstream tasks
• The metric checks both the recommendation and whether specific concerns are raised, ensuring the evaluation is thorough, not just confident

Condensed from dspy.ai/api/modules/MultiChainComparison/. Verify against upstream for latest.

dspy.MultiChainComparison — API Reference

Constructor

dspy.MultiChainComparison(
    signature,                    # Task signature (str or dspy.Signature class)
    M: int = 3,                  # Number of reasoning chains to generate and compare
    temperature: float = 0.7,    # Sampling temperature for chain generation
    **config,                    # Additional config passed to internal Predict
)

How it works

MultiChainComparison does NOT generate the reasoning chains. The caller generates M completions first (one dspy.ChainOfThought(signature, n=M) call) and passes the resulting .completions list to the module. The constructor dynamically builds a comparison signature that:

1. Appends M input fields for each reasoning attempt (reasoning_attempt_1 through reasoning_attempt_M) 2. Prepends an output field for synthesized "Accurate Reasoning" 3. The forward() method receives the M pre-generated completions and uses a single Predict call to synthesize/select the best answer

The module itself makes exactly 1 LM call -- the synthesis step.

Key methods

completions is the required first positional argument: a list of M pre-generated completions, obtained from dspy.ChainOfThought(signature, n=M)(...).completions.

Cost model

MultiChainComparison itself makes 1 LM call (the synthesis). Generating the M chains is a separate step -- 1 ChainOfThought call with n=M -- so the full pattern is 2 LM calls. Token cost still scales with M because the generate call samples M completions: