Dspy Program Of Thought

Solve Problems by Generating and Executing Code with dspy.ProgramOfThought

Guide the user through using DSPy's ProgramOfThought module, which has the LM write Python code to solve a problem and then executes that code to produce the answer.

What is ProgramOfThought

dspy.ProgramOfThought is a module that asks the LM to express its reasoning as executable Python code instead of natural language. The generated code runs in a sandboxed environment, and the execution result becomes the output.

This is fundamentally different from ChainOfThought:

• ChainOfThought -- the LM reasons in natural language, then produces an answer. Good for qualitative reasoning but prone to arithmetic and counting errors.
• ProgramOfThought -- the LM writes Python code that computes the answer. The code runs, and the result is exact. Good for anything where computation produces a more reliable answer than verbal reasoning.

Think of it as: the LM becomes a programmer that writes a small script to solve your problem, rather than trying to solve it in its head.

When to use ProgramOfThought

Use ProgramOfThought when the task involves:

• Math and arithmetic -- compound interest, tax calculations, unit conversions, statistics
• Counting and aggregation -- "how many items match this condition", tallying, grouping
• Data manipulation -- sorting, filtering, transforming structured data
• Date/time reasoning -- days between dates, business day calculations, timezone math
• Precise string operations -- regex matching, character counting, formatting
• Logic puzzles -- constraint satisfaction, combinatorics, permutations

Do not use it when:

• The task is purely qualitative (summarization, classification, creative writing)
• No computation is needed -- use dspy.Predict or dspy.ChainOfThought instead
• You need tool use or external API calls -- use dspy.ReAct instead

Basic usage

import dspy

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

# Inline signature
solver = dspy.ProgramOfThought("question -> answer")

result = solver(question="What is 15% tip on a $84.50 dinner bill split 3 ways?")
print(result.answer)  # Precise computed result

ProgramOfThought works with any signature -- inline strings or class-based:

class MathProblem(dspy.Signature):
    """Solve the given math problem by writing and executing Python code."""
    problem: str = dspy.InputField(desc="A math word problem")
    answer: float = dspy.OutputField(desc="The numerical answer")

solver = dspy.ProgramOfThought(MathProblem)
result = solver(problem="A store has a 20% off sale. An item costs $45. What is the sale price after 8% tax?")
print(result.answer)

How it works

When you call a ProgramOfThought module, here is what happens:

1. Code generation -- the LM receives the signature and inputs, then generates Python code that computes the answer 2. Sandbox execution -- DSPy executes the generated code in a restricted Python environment 3. Result extraction -- the output of the code execution is captured and returned as the prediction

The LM does not directly produce the answer. It produces code, and the code produces the answer. This means arithmetic is done by Python (exact), not by the LM (approximate).

What the sandbox provides

The generated code runs with access to Python's standard library. This includes math, datetime, collections, itertools, re, json, statistics, and other built-in modules. External packages like numpy or pandas are not available unless they are installed in the environment.

Retry on execution failure

If the generated code raises an exception, ProgramOfThought can retry by generating new code. You can control the number of retries:

solver = dspy.ProgramOfThought("question -> answer", max_iters=5)

The default is 3 iterations. On each retry, the LM sees the error traceback from the previous attempt, which helps it self-correct.

Using ProgramOfThought in a module

Wrap ProgramOfThought in a custom module to combine computation with other reasoning steps:

import dspy


class FinancialAnalyzer(dspy.Module):
    def __init__(self):
        self.compute = dspy.ProgramOfThought("scenario, question -> result: float")
        self.explain = dspy.ChainOfThought("scenario, question, result -> explanation")

    def forward(self, scenario, question):
        # Step 1: Compute the exact numerical answer
        computed = self.compute(scenario=scenario, question=question)

        # Step 2: Explain the result in plain language
        explained = self.explain(
            scenario=scenario,
            question=question,
            result=str(computed.result),
        )

        return dspy.Prediction(
            result=computed.result,
            explanation=explained.explanation,
        )


lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

analyzer = FinancialAnalyzer()
result = analyzer(
    scenario="Revenue was $1.2M in Q1, $1.5M in Q2, $1.1M in Q3, $1.8M in Q4.",
    question="What is the year-over-year growth rate if last year's total was $4.8M?",
)
print(result.result)
print(result.explanation)

This pattern -- compute first, explain second -- gives you both precision and readability.

Optimizing ProgramOfThought

ProgramOfThought modules work with DSPy optimizers just like any other module. The optimizer tunes the instructions and few-shot examples that guide code generation:

def metric(example, prediction, trace=None):
    return abs(float(prediction.answer) - float(example.answer)) < 0.01

optimizer = dspy.BootstrapFewShot(metric=metric, max_bootstrapped_demos=4)
optimized_solver = optimizer.compile(solver, trainset=trainset)

Optimization improves the quality of the generated code by showing the LM examples of good code-generation patterns.

Limitations

• Standard library only -- generated code cannot import packages that are not installed. If your task needs pandas or numpy, ensure they are in the environment.
• No side effects -- the sandbox restricts file I/O, network access, and other side effects. The code is meant to compute a value, not interact with the world.
• Code generation cost -- generating code takes more tokens than a direct answer. For trivial arithmetic (2 + 2), ChainOfThought is faster and cheaper.
• LM capability matters -- weaker models generate buggier code. Use a capable model (GPT-4o, Claude Sonnet, etc.) for complex computations.
• Output is the code's return value -- the generated code must produce a result that maps to your signature's output fields.

ProgramOfThought vs ChainOfThought -- when to use which

Rule of thumb: if you would reach for a calculator or a spreadsheet, use ProgramOfThought.

Cross-references

• dspy.Predict for simple direct LM calls -- see /dspy-predict
• dspy.ChainOfThought for natural language reasoning -- see /dspy-chain-of-thought
• Building modules that combine ProgramOfThought with other steps -- see /dspy-modules
• Reasoning patterns and when to add structured thinking -- see /ai-reasoning
• For worked examples, see examples.md

dspy-program-of-thought -- Worked Examples

Example 1: Financial calculation

Compute compound interest with monthly contributions -- the kind of calculation where LMs routinely make mistakes but code gets right every time.

import dspy


class InvestmentCalculator(dspy.Signature):
    """Calculate the future value of an investment given the parameters."""
    initial_deposit: float = dspy.InputField(desc="Starting amount in dollars")
    monthly_contribution: float = dspy.InputField(desc="Amount added each month")
    annual_rate: float = dspy.InputField(desc="Annual interest rate as a decimal, e.g. 0.07 for 7%")
    years: int = dspy.InputField(desc="Number of years")
    future_value: float = dspy.OutputField(desc="Total value at the end, rounded to 2 decimal places")


lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

calc = dspy.ProgramOfThought(InvestmentCalculator)

result = calc(
    initial_deposit=10000.0,
    monthly_contribution=500.0,
    annual_rate=0.07,
    years=20,
)
print(f"Future value: ${result.future_value:,.2f}")
# The generated code computes compound interest with monthly compounding
# and contributions -- exact to the penny.

Key points:

• Typed signature fields (float, int) guide the LM to produce code that returns the right type
• The desc fields tell the LM what each parameter means, so the generated code uses them correctly
• Compound interest with contributions is a multi-step formula that LMs frequently get wrong in natural language reasoning

Example 2: Data analysis with computation

Analyze sales data to compute aggregates, rankings, and derived metrics. This is the kind of task where you would normally reach for a spreadsheet.

import dspy


class SalesAnalysis(dspy.Signature):
    """Analyze the sales data and answer the question with a computed result."""
    sales_data: str = dspy.InputField(desc="Sales data as a text table or JSON string")
    question: str = dspy.InputField(desc="An analytical question about the data")
    answer: str = dspy.OutputField(desc="The computed answer")


lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

analyzer = dspy.ProgramOfThought(SalesAnalysis)

sales_csv = """
Region,Q1,Q2,Q3,Q4
North,120000,145000,132000,168000
South,98000,103000,115000,125000
East,156000,142000,138000,171000
West,87000,95000,102000,118000
""".strip()

# Question 1: Aggregation
result = analyzer(
    sales_data=sales_csv,
    question="Which region had the highest total annual sales, and what was the total?",
)
print(result.answer)

# Question 2: Growth analysis
result = analyzer(
    sales_data=sales_csv,
    question="What is the quarter-over-quarter growth rate for each region in Q4 vs Q3? Return as percentages.",
)
print(result.answer)

# Question 3: Ranking
result = analyzer(
    sales_data=sales_csv,
    question="Rank the quarters by total sales across all regions, from highest to lowest.",
)
print(result.answer)

Key points:

• The LM generates code to parse the CSV, compute aggregates, and format the result
• Each question produces different code -- the LM adapts its computation to the question
• This avoids the common problem of LMs miscounting or misadding numbers in tables

Example 3: Date/time reasoning

Date calculations involve edge cases (leap years, month lengths, timezone offsets) that trip up natural language reasoning. Code handles them correctly via Python's datetime module.

import dspy


class DateCalculator(dspy.Signature):
    """Solve date and time problems by computing the answer."""
    question: str = dspy.InputField(desc="A question involving dates, times, or durations")
    answer: str = dspy.OutputField(desc="The computed answer")


lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

solver = dspy.ProgramOfThought(DateCalculator)

# Leap year awareness
result = solver(
    question="How many days are there between February 15, 2024 and March 15, 2024?"
)
print(result.answer)  # 29 (2024 is a leap year)

# Business day calculation
result = solver(
    question="If a project starts on Monday, January 6, 2025 and takes 45 business days "
             "(excluding weekends), what date does it end?"
)
print(result.answer)

# Age calculation with edge cases
result = solver(
    question="Someone was born on February 29, 2000. How old are they on March 1, 2025? "
             "Give the answer in years and days."
)
print(result.answer)

# Duration between timestamps
result = solver(
    question="A server went down at 2025-01-15 23:47:12 UTC and came back at "
             "2025-01-16 02:13:45 UTC. How long was the outage in hours and minutes?"
)
print(result.answer)

Key points:

• Python's datetime module handles leap years, month boundaries, and weekday logic correctly
• Business day calculations require looping and weekday checks -- natural language reasoning almost always miscounts
• The LM generates different code for each question type (duration, business days, age calculation)
• No external libraries needed -- datetime is in the standard library and always available in the sandbox