Ai Fine Tuning

Fine-Tune Models on Your Data

Guide the user through deciding whether to fine-tune, preparing data, running fine-tuning with DSPy, distilling to cheaper models, and deploying. Fine-tuning is powerful but expensive — always confirm prerequisites first.

Should you fine-tune?

Before writing any code, walk through these questions with the user:

1. Have you optimized prompts first? If not, use /ai-improving-accuracy — prompt optimization is 10x cheaper and often sufficient. 2. Do you have 500+ labeled examples? Fine-tuning with less data usually overfits. Collect more data first. 3. Is your baseline accuracy above 50%? If your prompt-optimized program is below 50%, your task definition or data has problems. Fix those first. 4. What's the goal — quality or cost?

• Quality: You've maxed out prompt optimization and need more accuracy
• Cost: You want a small cheap model to match an expensive one

When to fine-tune

• You've already optimized prompts with MIPROv2 and hit a ceiling
• You have 500+ labeled examples (1000+ is better)
• Your baseline is >50% and you need to push higher
• You want to distill an expensive model into a cheaper one (10-50x cost savings)
• Your domain has specialized vocabulary or patterns the base model doesn't know
• You need faster inference (smaller fine-tuned models are faster)

When NOT to fine-tune

• You haven't tried prompt optimization yet — start with /ai-improving-accuracy
• You have fewer than 500 examples — need more data? Use /ai-generating-data to bootstrap synthetic examples, or use BootstrapFewShot or MIPROv2 instead
• Your baseline is below 50% — your data or task definition needs work
• You're still iterating on what the task is — fine-tuning locks you in
• You don't have a clear metric — you can't evaluate fine-tuning without one
• Your use case changes frequently — fine-tuned models don't adapt to new instructions easily

Prerequisites checklist

Before starting, confirm:

• [ ] Data: 500+ labeled examples (1000+ recommended), split 80/10/10 (train/dev/test)
• [ ] Baseline: Prompt-optimized program with measured accuracy (use /ai-improving-accuracy)
• [ ] Metric: Clear, automated metric that scores predictions
• [ ] Compute: API access (OpenAI fine-tuning API) or local GPUs (for open-source models)
• [ ] Budget: OpenAI fine-tuning costs ~$0.008/1K tokens for GPT-4o-mini; local needs 1+ GPU

Step 1: Prepare your data and baseline

Build a strong baseline first

Always compare fine-tuning against a prompt-optimized baseline:

import dspy

lm = dspy.LM("openai/gpt-4o")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

# Define your program
class Classify(dspy.Signature):
    """Classify the support ticket."""
    text: str = dspy.InputField()
    category: str = dspy.OutputField()

program = dspy.ChainOfThought(Classify)

# Prepare data
import json
with open("labeled_data.json") as f:
    data = json.load(f)

examples = [dspy.Example(text=x["text"], category=x["category"]).with_inputs("text") for x in data]

# Split: 80% train, 10% dev, 10% test
n = len(examples)
trainset = examples[:int(n * 0.8)]
devset = examples[int(n * 0.8):int(n * 0.9)]
testset = examples[int(n * 0.9):]

# Measure baseline
def metric(example, prediction, trace=None):
    return prediction.category.lower() == example.category.lower()

from dspy.evaluate import Evaluate
evaluator = Evaluate(devset=devset, metric=metric, num_threads=4, display_progress=True)
baseline_score = evaluator(program)
print(f"Baseline: {baseline_score:.1f}%")

Optimize prompts first (your comparison point)

optimizer = dspy.MIPROv2(metric=metric, auto="medium")
prompt_optimized = optimizer.compile(program, trainset=trainset)
prompt_score = evaluator(prompt_optimized)
print(f"Prompt-optimized: {prompt_score:.1f}%")

If prompt optimization gets you to your quality goal, stop here. Fine-tuning is only worth it if you need to go further.

Step 2: BootstrapFinetune (core fine-tuning)

The main fine-tuning workflow in DSPy. It bootstraps successful reasoning traces from your training data, filters them by your metric, and fine-tunes the model weights.

optimizer = dspy.BootstrapFinetune(metric=metric, num_threads=24)
finetuned = optimizer.compile(program, trainset=trainset)

# Evaluate the fine-tuned model
finetuned_score = evaluator(finetuned)
print(f"Baseline:         {baseline_score:.1f}%")
print(f"Prompt-optimized: {prompt_score:.1f}%")
print(f"Fine-tuned:       {finetuned_score:.1f}%")

How it works

1. Bootstrap traces: Runs your program on each training example, keeping traces where the metric passes 2. Filter by metric: Only successful traces become training data 3. Fine-tune weights: Sends traces to the model provider's fine-tuning API 4. Return optimized program: The program now uses the fine-tuned model

Requirements

• A fine-tunable model (OpenAI gpt-4o-mini, gpt-4o; or local open-source models)
• 500+ training examples (more traces bootstrapped = better fine-tuning)
• A metric that reliably identifies good outputs

Step 3: Model distillation (expensive to cheap)

Train a small, cheap model to mimic an expensive model. This is the biggest cost saver — 10-50x reduction with 85-95% quality retention.

Teacher-student pattern

# Step 1: Teacher — expensive model, high quality
teacher_lm = dspy.LM("openai/gpt-4o")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=teacher_lm)

# Build and optimize the teacher
teacher = dspy.ChainOfThought(Classify)
optimizer = dspy.MIPROv2(metric=metric, auto="medium")
teacher_optimized = optimizer.compile(teacher, trainset=trainset)

teacher_score = evaluator(teacher_optimized)
print(f"Teacher (GPT-4o): {teacher_score:.1f}%")

# Step 2: Student — fine-tune cheap model on teacher's outputs
student_lm = dspy.LM("openai/gpt-4o-mini")  # or another fine-tunable model
dspy.configure(lm=student_lm)

student = dspy.ChainOfThought(Classify)
ft_optimizer = dspy.BootstrapFinetune(metric=metric, num_threads=24)
student_finetuned = ft_optimizer.compile(student, trainset=trainset, teacher=teacher_optimized)

student_score = evaluator(student_finetuned)
print(f"Student (GPT-4o-mini, fine-tuned): {student_score:.1f}%")

Typical results

The fine-tuned student costs 33x less and retains ~95% of teacher quality.

Small models can dramatically outperform frontier models on narrow tasks. In a Yale project parsing 3.6M historical names, GPT-4 and Gemini achieved ~70% accuracy. Fine-tuned Qwen models (0.8B-4B parameters) hit 94-96% — beating frontier models by 25+ points while running locally. The key insight: for well-defined extraction tasks with enough training data (500K+ synthetic examples), tiny fine-tuned models dominate.

Step 4: BetterTogether (maximum quality)

BetterTogether alternates between prompt optimization and weight optimization, getting more out of both. Based on the BetterTogether paper (arXiv 2407.10930v2), this approach yields 5-78% gains over either technique alone.

optimizer = dspy.BetterTogether(
    metric=metric,
    p=dspy.MIPROv2(metric=metric),
    w=dspy.BootstrapFinetune(metric=metric),
)
best = optimizer.compile(program, trainset=trainset, strategy="p -> w -> p")

best_score = evaluator(best)
print(f"Prompt-only:    {prompt_score:.1f}%")
print(f"Fine-tune-only: {finetuned_score:.1f}%")
print(f"BetterTogether: {best_score:.1f}%")

How it works

The strategy string "p -> w -> p" controls the sequence — p maps to MIPROv2 (prompt optimizer) and w maps to BootstrapFinetune (weight optimizer):

1. Round 1 (p): Optimize prompts (instructions + few-shot examples) 2. Round 2 (w): Fine-tune weights using the optimized prompts 3. Round 3 (p): Re-optimize prompts for the fine-tuned model 4. Each round builds on the previous, creating synergy between prompt and weight optimization

If you omit the optimizer kwargs, BetterTogether defaults to p=BootstrapFewShotWithRandomSearch and w=BootstrapFinetune.

When to use BetterTogether

• You want the absolute best quality and have the compute budget
• Fine-tuning alone didn't close the gap to your quality target
• You have 500+ examples and a reliable metric

Step 5: Evaluate and deploy

Thorough evaluation

Always evaluate on the held-out test set (not dev set):

test_evaluator = Evaluate(devset=testset, metric=metric, num_threads=4, display_progress=True)

print(f"Test set results:")
print(f"  Baseline:         {test_evaluator(program):.1f}%")
print(f"  Prompt-optimized: {test_evaluator(prompt_optimized):.1f}%")
print(f"  Fine-tuned:       {test_evaluator(finetuned):.1f}%")

Save and load for production

# Save
finetuned.save("finetuned_program.json")

# Load later
from my_module import MyProgram
production = MyProgram()
production.load("finetuned_program.json")
result = production(text="New support ticket...")

When fine-tuning goes wrong

Can't bootstrap enough traces

If the base model fails on most training examples, there aren't enough successful traces to fine-tune on.

Fixes:

• Use a stronger model for bootstrapping (GPT-4o instead of GPT-4o-mini)
• Relax your metric during bootstrapping (accept partial credit)
• Simplify your task (break multi-step into single steps)

Output format errors from small models

Small fine-tuned models (<4B params) often produce JSON syntax errors — unclosed braces, missing quotes, trailing commas. Switch to YAML output format during fine-tuning to eliminate these entirely. YAML is more forgiving to generate and parses reliably from small models.

Model overfits (high train accuracy, low test accuracy)

Fixes:

• Add more training data
• Reduce fine-tuning epochs (if provider allows)
• Use a larger base model (less prone to overfitting)
• Simplify your output format

Fine-tuning didn't improve over prompt optimization

Fixes:

• Check that bootstrapping produced enough successful traces (need 200+)
• Try BetterTogether instead of BootstrapFinetune alone
• Verify your metric actually correlates with quality
• Try a different base model

Infrastructure choices

OpenAI API (easiest)

Works with gpt-4o-mini and gpt-4o. DSPy handles the fine-tuning API calls automatically:

lm = dspy.LM("openai/gpt-4o-mini")  # or any fine-tunable model via API

• Pros: No GPU needed, simple setup, fast
• Cons: Data sent to OpenAI, ongoing per-token costs, limited model choices

Local fine-tuning (own your model)

For open-source models (Llama, Mistral, etc.) using LoRA/QLoRA:

lm = dspy.LM("together_ai/meta-llama/Llama-3-70b-chat-hf")

• Pros: Data stays private, no per-token costs after training, full control
• Cons: Needs GPU(s), more setup, slower iteration

Cloud GPU platforms

AWS SageMaker, Google Cloud, Lambda Labs, or Together AI for training:

• Pros: Scalable, no hardware to manage
• Cons: Costs vary, setup per platform

Gotchas

• Skipping prompt optimization and jumping straight to fine-tuning. Claude defaults to recommending fine-tuning when users mention quality issues. Always confirm the user has run MIPROv2 or similar prompt optimization first — fine-tuning without a prompt-optimized baseline wastes compute and makes it impossible to measure whether fine-tuning actually helped.
• Using the dev set for final evaluation. Claude often evaluates the fine-tuned model on the same dev set used during optimization. Always evaluate on a held-out test set that was never seen during training or prompt optimization. Report both dev and test scores so the user can spot overfitting.
• Passing `teacher=` without an optimized teacher program. When using BootstrapFinetune for distillation, Claude sometimes passes the unoptimized base program as the teacher. The teacher must be the prompt-optimized version — otherwise the student learns from mediocre traces and fine-tuning underperforms.
• Forgetting that `BootstrapFinetune` needs a fine-tunable model. Not all models support fine-tuning via API. Claude sometimes configures dspy.LM("anthropic/claude-sonnet-4-5-20250929") for BootstrapFinetune, but Anthropic does not offer a fine-tuning API. Use OpenAI models or local open-source models for weight optimization.
• Not checking how many traces were bootstrapped. If bootstrapping only produces 50 successful traces from 1000 examples, the fine-tuning data is too small. Check the bootstrap log output and aim for 200+ successful traces. If too few succeed, use a stronger teacher model or relax the metric.

Cross-references

Install any skill: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>

• Build a strong baseline before fine-tuning — see /ai-improving-accuracy
• BootstrapFinetune API details — see /dspy-bootstrap-finetune
• BetterTogether optimizer — see /dspy-better-together
• Cost reduction beyond distillation — see /ai-cutting-costs
• Generate synthetic training data — see /ai-generating-data
• Fix fine-tuning or evaluation errors — see /ai-fixing-errors
• Install `/ai-do` if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do

Additional resources

• For worked examples (classification, distillation, BetterTogether), see examples.md
• For BootstrapFinetune, BetterTogether, and MIPROv2 API details, see reference.md

Fine-Tuning Examples

Worked examples showing the full fine-tuning workflow for different use cases.

Example 1: Classification fine-tuning (ticket sorting)

Train a small model to sort support tickets as well as an expensive model.

Setup and data

import dspy
from dspy.evaluate import Evaluate

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

class ClassifyTicket(dspy.Signature):
    """Classify the support ticket into a category."""
    text: str = dspy.InputField()
    category: str = dspy.OutputField(
        desc="one of: billing, technical, account, feature_request, other"
    )

program = dspy.ChainOfThought(ClassifyTicket)

# Load labeled data (1000 tickets)
import json
with open("tickets.json") as f:
    data = json.load(f)

examples = [
    dspy.Example(text=x["text"], category=x["category"]).with_inputs("text")
    for x in data
]

# Split 80/10/10
trainset = examples[:800]
devset = examples[800:900]
testset = examples[900:]

def metric(example, prediction, trace=None):
    return prediction.category.strip().lower() == example.category.strip().lower()

evaluator = Evaluate(devset=devset, metric=metric, num_threads=4, display_progress=True)

Step 1: Measure baseline

baseline_score = evaluator(program)
print(f"Baseline (GPT-4o-mini, no optimization): {baseline_score:.1f}%")
# Expected: ~65-75%

Step 2: Optimize prompts (comparison point)

optimizer = dspy.MIPROv2(metric=metric, auto="medium")
prompt_optimized = optimizer.compile(program, trainset=trainset)
prompt_score = evaluator(prompt_optimized)
print(f"Prompt-optimized: {prompt_score:.1f}%")
# Expected: ~80-88%

Step 3: Fine-tune

ft_optimizer = dspy.BootstrapFinetune(metric=metric, num_threads=24)
finetuned = ft_optimizer.compile(program, trainset=trainset)
finetuned_score = evaluator(finetuned)
print(f"Fine-tuned: {finetuned_score:.1f}%")
# Expected: ~88-95%

Results comparison

test_eval = Evaluate(devset=testset, metric=metric, num_threads=4, display_progress=True)
print(f"Test set results:")
print(f"  Baseline:         {test_eval(program):.1f}%")
print(f"  Prompt-optimized: {test_eval(prompt_optimized):.1f}%")
print(f"  Fine-tuned:       {test_eval(finetuned):.1f}%")

---

Example 2: RAG distillation (GPT-4o to GPT-4o-mini)

Distill an expensive RAG pipeline into a cheap model that runs 15x cheaper.

Build teacher with expensive model

import dspy
from dspy.evaluate import Evaluate

teacher_lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=teacher_lm)

class AnswerFromDocs(dspy.Signature):
    """Answer the question based on the provided context."""
    context: str = dspy.InputField()
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

def retrieve(query: str, k: int = 3) -> list[str]:
    """Retrieve relevant passages. Replace with your retriever (ColBERTv2, vector DB, etc.)."""
    rm = dspy.ColBERTv2(url="http://your-server:port")
    return rm(query, k=k)

class RAG(dspy.Module):
    def __init__(self):
        self.generate = dspy.ChainOfThought(AnswerFromDocs)

    def forward(self, question):
        context = retrieve(question)
        return self.generate(context=context, question=question)

teacher = RAG()

# Load data
import json
with open("qa_data.json") as f:
    data = json.load(f)

examples = [
    dspy.Example(question=x["question"], answer=x["answer"]).with_inputs("question")
    for x in data
]

trainset = examples[:800]
devset = examples[800:900]
testset = examples[900:]

def metric(example, prediction, trace=None):
    gold_tokens = set(example.answer.lower().split())
    pred_tokens = set(prediction.answer.lower().split())
    if not gold_tokens or not pred_tokens:
        return float(gold_tokens == pred_tokens)
    precision = len(gold_tokens & pred_tokens) / len(pred_tokens)
    recall = len(gold_tokens & pred_tokens) / len(gold_tokens)
    if precision + recall == 0:
        return 0.0
    return 2 * (precision * recall) / (precision + recall)

evaluator = Evaluate(devset=devset, metric=metric, num_threads=4, display_progress=True)

# Optimize teacher
optimizer = dspy.MIPROv2(metric=metric, auto="medium")
teacher_optimized = optimizer.compile(teacher, trainset=trainset)
teacher_score = evaluator(teacher_optimized)
print(f"Teacher (GPT-4o, optimized): {teacher_score:.1f}%")
# Expected: ~82%

Distill to student

# Switch to cheap model
student_lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=student_lm)

student = RAG()
ft_optimizer = dspy.BootstrapFinetune(metric=metric, num_threads=24)
student_finetuned = ft_optimizer.compile(student, trainset=trainset, teacher=teacher_optimized)

student_score = evaluator(student_finetuned)
print(f"Student (GPT-4o-mini, fine-tuned): {student_score:.1f}%")
# Expected: ~76%

Results

93% quality retention at 33x lower cost.

---

Example 3: BetterTogether for maximum quality

Use alternating prompt + weight optimization for a multi-step reasoning task.

Setup

import dspy
from dspy.evaluate import Evaluate

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

class SolveWord(dspy.Signature):
    """Solve the math word problem step by step."""
    problem: str = dspy.InputField()
    answer: float = dspy.OutputField()

program = dspy.ChainOfThought(SolveWord)

# Load math problems
import json
with open("math_problems.json") as f:
    data = json.load(f)

examples = [
    dspy.Example(problem=x["problem"], answer=x["answer"]).with_inputs("problem")
    for x in data
]

trainset = examples[:800]
devset = examples[800:900]
testset = examples[900:]

def metric(example, prediction, trace=None):
    try:
        return abs(float(prediction.answer) - float(example.answer)) < 0.01
    except (ValueError, TypeError):
        return False

evaluator = Evaluate(devset=devset, metric=metric, num_threads=4, display_progress=True)

Run all three approaches

# 1. Baseline
baseline_score = evaluator(program)
print(f"Baseline: {baseline_score:.1f}%")

# 2. Prompt optimization only
prompt_opt = dspy.MIPROv2(metric=metric, auto="medium")
prompt_optimized = prompt_opt.compile(program, trainset=trainset)
prompt_score = evaluator(prompt_optimized)
print(f"Prompt-only: {prompt_score:.1f}%")

# 3. Fine-tuning only
ft_opt = dspy.BootstrapFinetune(metric=metric, num_threads=24)
finetuned = ft_opt.compile(program, trainset=trainset)
ft_score = evaluator(finetuned)
print(f"Fine-tune-only: {ft_score:.1f}%")

# 4. BetterTogether (prompt + weight optimization)
bt_opt = dspy.BetterTogether(
    metric=metric,
    p=dspy.MIPROv2(metric=metric),
    w=dspy.BootstrapFinetune(metric=metric),
)
best = bt_opt.compile(program, trainset=trainset, strategy="p -> w -> p")
best_score = evaluator(best)
print(f"BetterTogether: {best_score:.1f}%")

Results

BetterTogether gets +10 pts beyond the best individual approach because prompt optimization and weight optimization complement each other.

---

Example 4: Troubleshooting low bootstrap success

When your base model is too weak to bootstrap enough successful traces.

Problem: 30% baseline, bootstrapping fails

import dspy
from dspy.evaluate import Evaluate

lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=lm)

class ExtractEntities(dspy.Signature):
    """Extract all named entities from the text."""
    text: str = dspy.InputField()
    entities: list[str] = dspy.OutputField()

program = dspy.ChainOfThought(ExtractEntities)

# Baseline is only ~30% — too weak for bootstrapping
evaluator = Evaluate(devset=devset, metric=metric, num_threads=4, display_progress=True)
print(f"Baseline: {evaluator(program):.1f}%")  # ~30%

# BootstrapFinetune will struggle — only ~30% of traces pass the metric
# Not enough successful traces to fine-tune well

Fix 1: Use a stronger model for bootstrapping

Use an expensive model to generate traces, then fine-tune the cheap model on them:

# Bootstrap with the strong model
strong_lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=strong_lm)

strong_program = dspy.ChainOfThought(ExtractEntities)

# Now bootstrap — GPT-4o will succeed on ~70% of examples
# giving us plenty of traces to fine-tune GPT-4o-mini
weak_lm = dspy.LM("openai/gpt-4o-mini")
dspy.configure(lm=weak_lm)

optimizer = dspy.BootstrapFinetune(metric=metric, num_threads=24)
finetuned = optimizer.compile(program, trainset=trainset, teacher=strong_program)
print(f"Fine-tuned (with strong teacher): {evaluator(finetuned):.1f}%")
# Expected: ~55% (up from 30%)

Fix 2: Relax metric for bootstrapping

Use a lenient metric during trace collection, strict metric for evaluation:

def lenient_metric(example, prediction, trace=None):
    """Accept partial matches during bootstrapping."""
    gold = set(e.lower() for e in example.entities)
    pred = set(e.lower() for e in prediction.entities)
    if not gold:
        return float(len(pred) == 0)
    # Accept if at least half the entities are found
    recall = len(gold & pred) / len(gold)
    return recall >= 0.5

def strict_metric(example, prediction, trace=None):
    """Require exact match for evaluation."""
    gold = set(e.lower() for e in example.entities)
    pred = set(e.lower() for e in prediction.entities)
    return gold == pred

# Bootstrap with lenient metric (more traces pass)
optimizer = dspy.BootstrapFinetune(metric=lenient_metric, num_threads=24)
finetuned = optimizer.compile(program, trainset=trainset)

# Evaluate with strict metric
strict_evaluator = Evaluate(devset=devset, metric=strict_metric, num_threads=4, display_progress=True)
print(f"Fine-tuned (lenient bootstrap): {strict_evaluator(finetuned):.1f}%")
# Expected: ~48% (up from 30%)

Summary: when bootstrapping fails

Fine-Tuning API Reference

Condensed from dspy.ai/api/optimizers/. Verify against upstream for latest.

dspy.BootstrapFinetune

Fine-tunes model weights using bootstrapped reasoning traces.

optimizer = dspy.BootstrapFinetune(metric=metric, num_threads=24)
finetuned = optimizer.compile(student, trainset=trainset, teacher=teacher_optimized)

Constructor

compile()

optimizer.compile(student, trainset, teacher=None) -> Module

dspy.BetterTogether

Alternates prompt optimization and weight optimization for maximum quality.

optimizer = dspy.BetterTogether(
    metric=metric,
    p=dspy.MIPROv2(metric=metric),
    w=dspy.BootstrapFinetune(metric=metric),
)
best = optimizer.compile(program, trainset=trainset, strategy="p -> w -> p")

Constructor

Default optimizers (if none provided): p=BootstrapFewShotWithRandomSearch(metric=metric), w=BootstrapFinetune(metric=metric).

compile()

optimizer.compile(student, trainset, strategy="p -> w -> p", valset=None) -> Module

Strategy strings: "p -> w" (prompt then weights), "p -> w -> p" (cyclic), "w -> p" (reverse).

dspy.MIPROv2

Best prompt optimizer. Optimizes instructions and few-shot examples.

optimizer = dspy.MIPROv2(metric=metric, auto="medium")
optimized = optimizer.compile(program, trainset=trainset)

Constructor (key parameters)

Links

• BootstrapFinetune API docs
• BetterTogether API docs
• MIPROv2 API docs