Dspy Data

Work with DSPy Data: Examples, Predictions, and Datasets

Guide the user through creating, loading, and managing data for DSPy programs. Data is the fuel for DSPy optimizers — getting it right is the difference between a program that works and one that doesn't.

What are Examples

dspy.Example is DSPy's data container. Think of it as a dictionary with one extra feature: you can mark which fields are inputs and which are outputs. This distinction is critical because optimizers need to know what to feed into your program (inputs) and what to compare against (outputs).

import dspy

# An Example holds named fields — like a dict
example = dspy.Example(question="What is DSPy?", answer="A framework for programming LMs")

# Access fields with dot notation or bracket notation
print(example.question)       # "What is DSPy?"
print(example["answer"])      # "A framework for programming LMs"

Every DSPy optimizer, evaluator, and metric function expects data as a list of dspy.Example objects.

Creating Examples

Create examples with keyword arguments. Every keyword becomes a field.

# Simple question-answer pair
ex = dspy.Example(question="What color is the sky?", answer="Blue")

# Classification example with more fields
ex = dspy.Example(
    text="The product broke after one day",
    label="negative",
    category="quality",
)

# Fields can be any Python type
ex = dspy.Example(
    query="hiking trails near Portland",
    results=["Forest Park", "Eagle Creek"],
    count=2,
)

You can also create an Example from a dictionary:

data = {"question": "What is Python?", "answer": "A programming language"}
ex = dspy.Example(**data)

with_inputs() — marking input fields

with_inputs() tells DSPy which fields are inputs (what the program receives) and which are outputs (what the program should produce). This is required for optimizers and evaluation.

# Mark "question" as input — "answer" becomes the expected output
ex = dspy.Example(question="What is DSPy?", answer="A framework").with_inputs("question")

# Multiple input fields
ex = dspy.Example(
    context="DSPy is a Python framework...",
    question="What is DSPy?",
    answer="A framework for programming LMs",
).with_inputs("context", "question")

What happens without with_inputs()

If you skip with_inputs(), optimizers won't know which fields to pass to your program and which to hold back for scoring. You'll get errors or wrong results. Always call it.

How it works

After with_inputs("question"):

• example.inputs() returns an Example with only {"question": "What is DSPy?"}
• example.labels() returns an Example with only {"answer": "A framework"}

DSPy uses .inputs() to feed data into your module and .labels() to check the output against expected values.

ex = dspy.Example(question="What is DSPy?", answer="A framework").with_inputs("question")

print(ex.inputs())   # Example(question="What is DSPy?")
print(ex.labels())   # Example(answer="A framework")

Prediction — what modules return

When you call a DSPy module, it returns a dspy.Prediction, which extends Example. A Prediction has the same dot-access and dict-like behavior.

classify = dspy.ChainOfThought("text -> label")
result = classify(text="Great product!")

# result is a Prediction
print(result.label)       # "positive"
print(result.reasoning)   # chain-of-thought reasoning (added by ChainOfThought)

# Predictions work like Examples
print(result.keys())      # dict_keys(['reasoning', 'label'])

In metric functions, prediction is always a Prediction and example is always an Example:

def metric(example, prediction, trace=None):
    # example has the gold fields you defined
    # prediction has the fields your module produced
    return prediction.label == example.label

Building datasets

A dataset in DSPy is just a Python list of Example objects. Use list comprehensions to build them.

# From parallel lists
questions = ["What is Python?", "What is DSPy?", "What is an LM?"]
answers = ["A programming language", "A framework for LMs", "A language model"]

trainset = [
    dspy.Example(question=q, answer=a).with_inputs("question")
    for q, a in zip(questions, answers)
]

# From a list of dicts
raw_data = [
    {"text": "Love it!", "label": "positive"},
    {"text": "Terrible.", "label": "negative"},
    {"text": "It's okay.", "label": "neutral"},
]

trainset = [
    dspy.Example(**row).with_inputs("text")
    for row in raw_data
]

Loading from CSV

import csv

def load_csv_as_examples(filepath, input_fields):
    """Load a CSV file into a list of dspy.Example objects."""
    examples = []
    with open(filepath, "r") as f:
        reader = csv.DictReader(f)
        for row in reader:
            ex = dspy.Example(**row).with_inputs(*input_fields)
            examples.append(ex)
    return examples

# Usage
trainset = load_csv_as_examples("tickets.csv", input_fields=["message"])

With pandas (if you prefer):

import pandas as pd

df = pd.read_csv("tickets.csv")
trainset = [
    dspy.Example(**row.to_dict()).with_inputs("message")
    for _, row in df.iterrows()
]

Handling CSV quirks

# Skip rows with missing values
trainset = [
    dspy.Example(**row).with_inputs("text")
    for row in csv.DictReader(open("data.csv"))
    if row["text"] and row["label"]  # skip blanks
]

# Rename columns to match your signature
trainset = [
    dspy.Example(
        text=row["customer_message"],
        label=row["assigned_category"],
    ).with_inputs("text")
    for row in csv.DictReader(open("data.csv"))
]

Loading from JSON

import json

def load_json_as_examples(filepath, input_fields):
    """Load a JSON array file into dspy.Example objects."""
    with open(filepath, "r") as f:
        data = json.load(f)
    return [
        dspy.Example(**item).with_inputs(*input_fields)
        for item in data
    ]

# Usage — file contains [{"question": "...", "answer": "..."}, ...]
trainset = load_json_as_examples("qa_pairs.json", input_fields=["question"])

For JSON Lines (one JSON object per line):

def load_jsonl_as_examples(filepath, input_fields):
    """Load a JSONL file into dspy.Example objects."""
    examples = []
    with open(filepath, "r") as f:
        for line in f:
            if line.strip():
                item = json.loads(line)
                examples.append(dspy.Example(**item).with_inputs(*input_fields))
    return examples

Loading from HuggingFace

The HuggingFace datasets library gives you access to thousands of ready-to-use datasets.

pip install datasets

from datasets import load_dataset

# Load a dataset
dataset = load_dataset("hotpotqa/hotpot_qa", "fullwiki")

# Convert to DSPy Examples
trainset = [
    dspy.Example(
        question=x["question"],
        answer=x["answer"],
    ).with_inputs("question")
    for x in dataset["train"]
]

Common HuggingFace patterns

# Limit the number of examples (large datasets)
trainset = [
    dspy.Example(question=x["question"], answer=x["answer"]).with_inputs("question")
    for x in list(dataset["train"])[:500]
]

# Rename fields to match your signature
trainset = [
    dspy.Example(
        text=x["sentence"],
        label="positive" if x["label"] == 1 else "negative",
    ).with_inputs("text")
    for x in dataset["train"]
]

# Filter rows
trainset = [
    dspy.Example(question=x["question"], answer=x["answer"]).with_inputs("question")
    for x in dataset["train"]
    if len(x["answer"]) > 0  # skip empty answers
]

Built-in datasets

DSPy ships with a few standard datasets for prototyping and benchmarking. These return pre-built dspy.Example objects — no HuggingFace dependency needed.

from dspy.datasets import HotPotQA

# Multi-hop question answering
dataset = HotPotQA(train_seed=1, train_size=200, dev_size=50, test_size=0)
trainset = dataset.train
devset = dataset.dev

Constructor parameters (all optional): train_seed, train_size, dev_size, test_size.

Note: You still need .with_inputs() if you're passing these to an optimizer:

trainset = [ex.with_inputs("question") for ex in dataset.train]

Train/dev splits

Optimizers train on trainset and you evaluate on devset. Keep them separate to measure real performance.

Random split

import random

def train_dev_split(examples, train_ratio=0.8, seed=42):
    """Split a list of examples into train and dev sets."""
    random.seed(seed)
    shuffled = list(examples)
    random.shuffle(shuffled)
    split_idx = int(len(shuffled) * train_ratio)
    return shuffled[:split_idx], shuffled[split_idx:]

# Usage
all_examples = load_csv_as_examples("data.csv", input_fields=["text"])
trainset, devset = train_dev_split(all_examples)
print(f"Train: {len(trainset)}, Dev: {len(devset)}")

Stratified split (preserves label distribution)

Use this when your categories are imbalanced (e.g., 90% "general", 10% "urgent").

from collections import defaultdict

def stratified_split(examples, label_field, train_ratio=0.8, seed=42):
    """Split examples while preserving the distribution of a label field."""
    random.seed(seed)
    buckets = defaultdict(list)
    for ex in examples:
        buckets[ex[label_field]].append(ex)

    trainset, devset = [], []
    for label, items in buckets.items():
        random.shuffle(items)
        split_idx = int(len(items) * train_ratio)
        trainset.extend(items[:split_idx])
        devset.extend(items[split_idx:])

    random.shuffle(trainset)
    random.shuffle(devset)
    return trainset, devset

# Usage
trainset, devset = stratified_split(all_examples, label_field="category")

Using HuggingFace's built-in splits

Many HuggingFace datasets come pre-split:

dataset = load_dataset("hotpotqa/hotpot_qa", "fullwiki")
trainset = [dspy.Example(**x).with_inputs("question") for x in dataset["train"].select(range(500))]
devset = [dspy.Example(**x).with_inputs("question") for x in dataset["validation"].select(range(200))]

Common patterns

Accessing fields

ex = dspy.Example(question="What?", answer="That", source="wiki")

# Dot access
ex.question

# Dict-style access
ex["question"]

# Get all field names
ex.keys()  # dict_keys(['question', 'answer', 'source'])

# Check if a field exists
"question" in ex  # True

Converting to/from dicts

# Example to dict
d = dict(ex)  # {"question": "What?", "answer": "That", "source": "wiki"}

# Dict to Example
ex = dspy.Example(**d).with_inputs("question")

Filtering examples

# Keep only examples where the answer is short
short_answers = [ex for ex in trainset if len(ex.answer.split()) < 20]

# Keep only a specific category
urgent_only = [ex for ex in trainset if ex.category == "urgent"]

# Remove duplicates (by a field)
seen = set()
unique = []
for ex in trainset:
    if ex.question not in seen:
        seen.add(ex.question)
        unique.append(ex)

Inspecting your dataset

# Quick summary
print(f"Total examples: {len(trainset)}")
print(f"Fields: {trainset[0].keys()}")
print(f"First example: {trainset[0]}")

# Label distribution
from collections import Counter
labels = Counter(ex.label for ex in trainset)
print(f"Label distribution: {labels}")

Gotchas

• Claude forgets `with_inputs()` on every Example. Without it, optimizers cannot distinguish inputs from expected outputs. Every example passed to an optimizer or evaluator must have with_inputs() called. Claude often creates examples and only calls with_inputs() on the first one or skips it entirely when building lists inline.
• Claude calls `with_inputs()` with output field names. Mark only the fields your module receives as input — not the fields it should produce. If your signature is question -> answer, call .with_inputs("question"), not .with_inputs("question", "answer"). Including output fields means the optimizer has nothing to score against.
• Claude uses `Literal[list]` instead of `Literal[tuple(list)]` for dynamic categories. When building categories from data (CATEGORIES = list(set(...))), the type annotation must be Literal[tuple(CATEGORIES)], not Literal[CATEGORIES]. The latter silently fails to constrain the output.
• Claude passes raw dicts to optimizers instead of `dspy.Example` objects. DSPy optimizers and evaluators require dspy.Example objects, not plain Python dicts. Always convert with dspy.Example(**row).with_inputs(...).
• Claude creates train/dev splits without shuffling first. If data is sorted by label or date, taking the first 80% as train and last 20% as dev creates a biased split. Always shuffle with a fixed seed before splitting.

Additional resources

• dspy.Example API docs
• dspy.Prediction API docs
• reference.md — constructor signatures, method tables, Prediction details
• examples.md — worked examples with CSV, HuggingFace, and manual data

Cross-references

Install any skill: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>

• `/dspy-evaluate` — evaluate your program on a devset with metrics
• `/ai-generating-data` — generate synthetic training data when you have none
• `/ai-improving-accuracy` — use optimizers that consume your trainset to boost quality
• Install `/ai-do` if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do

dspy-data Examples

Three complete, copy-paste-ready examples for working with DSPy data.

Example 1: Load from a CSV file

Load a CSV of customer support tickets, create DSPy Examples, and split into train/dev sets.

Assumes a CSV file `tickets.csv` with columns: `message`, `category`, `priority`

import dspy
import csv
import random

# --- Load CSV into dspy.Example objects ---

def load_csv_as_examples(filepath, input_fields):
    """Load a CSV file into a list of dspy.Example objects."""
    examples = []
    with open(filepath, "r") as f:
        reader = csv.DictReader(f)
        for row in reader:
            # Skip rows with missing required fields
            if all(row.get(field) for field in reader.fieldnames):
                ex = dspy.Example(**row).with_inputs(*input_fields)
                examples.append(ex)
    return examples

all_examples = load_csv_as_examples("tickets.csv", input_fields=["message"])
print(f"Loaded {len(all_examples)} examples")
print(f"Fields: {all_examples[0].keys()}")
print(f"First example: {all_examples[0]}")

# --- Split into train and dev sets ---

random.seed(42)
random.shuffle(all_examples)
split_idx = int(len(all_examples) * 0.8)
trainset = all_examples[:split_idx]
devset = all_examples[split_idx:]
print(f"Train: {len(trainset)}, Dev: {len(devset)}")

# --- Check label distribution ---

from collections import Counter
train_labels = Counter(ex.category for ex in trainset)
dev_labels = Counter(ex.category for ex in devset)
print(f"Train label distribution: {train_labels}")
print(f"Dev label distribution:   {dev_labels}")

# --- Use with an optimizer ---

lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

from typing import Literal

CATEGORIES = list(set(ex.category for ex in trainset))

class ClassifyTicket(dspy.Signature):
    """Classify the support ticket into the correct category."""
    message: str = dspy.InputField(desc="The customer support message")
    category: Literal[tuple(CATEGORIES)] = dspy.OutputField(desc="The ticket category")

classifier = dspy.ChainOfThought(ClassifyTicket)

def metric(example, prediction, trace=None):
    return prediction.category == example.category

optimizer = dspy.BootstrapFewShot(metric=metric, max_bootstrapped_demos=4)
optimized = optimizer.compile(classifier, trainset=trainset)

# Evaluate on dev set
from dspy.evaluate import Evaluate
evaluator = Evaluate(devset=devset, metric=metric, num_threads=4, display_progress=True)
score = evaluator(optimized)
print(f"Dev accuracy: {score}%")

Example 2: Load from HuggingFace

Load the Stanford Sentiment Treebank (SST-2) dataset from HuggingFace and convert to DSPy Examples for a sentiment classification task.

import dspy
from datasets import load_dataset

# --- Load the dataset ---

dataset = load_dataset("glue", "sst2")

# --- Convert to DSPy Examples ---
# SST-2 has "sentence" and "label" (0=negative, 1=positive)

def convert_sst2(split, max_examples=None):
    """Convert HuggingFace SST-2 split to DSPy Examples."""
    items = list(split)
    if max_examples:
        items = items[:max_examples]
    return [
        dspy.Example(
            text=x["sentence"],
            label="positive" if x["label"] == 1 else "negative",
        ).with_inputs("text")
        for x in items
    ]

# Use a subset for speed — DSPy optimizers don't need huge datasets
trainset = convert_sst2(dataset["train"], max_examples=200)
devset = convert_sst2(dataset["validation"], max_examples=100)

print(f"Train: {len(trainset)}, Dev: {len(devset)}")
print(f"First train example: {trainset[0]}")

# --- Check distribution ---

from collections import Counter
print(f"Train labels: {Counter(ex.label for ex in trainset)}")
print(f"Dev labels:   {Counter(ex.label for ex in devset)}")

# --- Build a classifier and optimize ---

lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

from typing import Literal

class SentimentClassify(dspy.Signature):
    """Classify the sentiment of the text."""
    text: str = dspy.InputField(desc="The text to classify")
    label: Literal["positive", "negative"] = dspy.OutputField(desc="The sentiment")

classifier = dspy.ChainOfThought(SentimentClassify)

def metric(example, prediction, trace=None):
    return prediction.label == example.label

# Quick optimization with BootstrapFewShot
optimizer = dspy.BootstrapFewShot(metric=metric, max_bootstrapped_demos=4)
optimized = optimizer.compile(classifier, trainset=trainset)

# Evaluate
from dspy.evaluate import Evaluate
score = Evaluate(devset=devset, metric=metric, num_threads=4)(optimized)
print(f"Dev accuracy: {score}%")

Example 3: Manual creation for a classification task

Hand-craft examples when you're prototyping or have a small number of known cases. Demonstrates with_inputs(), .inputs(), and .labels().

import dspy

# --- Hand-craft examples ---

examples = [
    dspy.Example(
        email="Hi, I can't log into my account. I've tried resetting my password twice.",
        intent="account_access",
        urgency="high",
    ),
    dspy.Example(
        email="When is the next billing cycle? I want to upgrade my plan.",
        intent="billing",
        urgency="low",
    ),
    dspy.Example(
        email="Your API is returning 500 errors on the /users endpoint since 3am.",
        intent="bug_report",
        urgency="critical",
    ),
    dspy.Example(
        email="Could you add dark mode? Would really help with late night coding.",
        intent="feature_request",
        urgency="low",
    ),
    dspy.Example(
        email="I was charged twice for my subscription this month.",
        intent="billing",
        urgency="high",
    ),
    dspy.Example(
        email="The dashboard loads really slowly, takes over 30 seconds.",
        intent="bug_report",
        urgency="medium",
    ),
    dspy.Example(
        email="Thanks for the quick fix on that export bug!",
        intent="feedback",
        urgency="low",
    ),
    dspy.Example(
        email="URGENT: Our entire team is locked out of the platform right now.",
        intent="account_access",
        urgency="critical",
    ),
    dspy.Example(
        email="Is there documentation for the new webhooks feature?",
        intent="question",
        urgency="low",
    ),
    dspy.Example(
        email="We need SSO integration before we can renew our enterprise contract.",
        intent="feature_request",
        urgency="high",
    ),
]

# --- Mark input fields ---
# "email" is the input; "intent" and "urgency" are the expected outputs

examples = [ex.with_inputs("email") for ex in examples]

# --- Verify input/output split ---

first = examples[0]
print("Full example:", first)
print("Inputs only: ", first.inputs())   # Example(email="Hi, I can't log into...")
print("Labels only: ", first.labels())   # Example(intent="account_access", urgency="high")

# --- Split into train and dev ---

trainset = examples[:8]
devset = examples[8:]
print(f"\nTrain: {len(trainset)}, Dev: {len(devset)}")

# --- Build and test a classifier ---

lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)

from typing import Literal

INTENTS = ["account_access", "billing", "bug_report", "feature_request", "feedback", "question"]
URGENCY_LEVELS = ["low", "medium", "high", "critical"]

class ClassifyEmail(dspy.Signature):
    """Classify the customer email by intent and urgency level."""
    email: str = dspy.InputField(desc="The customer email to classify")
    intent: Literal[tuple(INTENTS)] = dspy.OutputField(desc="The primary intent")
    urgency: Literal[tuple(URGENCY_LEVELS)] = dspy.OutputField(desc="The urgency level")

classifier = dspy.ChainOfThought(ClassifyEmail)

# Test on one example
result = classifier(email=devset[0].email)
print(f"\nPredicted intent: {result.intent}, urgency: {result.urgency}")
print(f"Expected  intent: {devset[0].intent}, urgency: {devset[0].urgency}")

# --- Metric that checks both fields ---

def metric(example, prediction, trace=None):
    intent_correct = prediction.intent == example.intent
    urgency_correct = prediction.urgency == example.urgency
    # Score: 1.0 if both right, 0.5 if one right, 0.0 if neither
    return (intent_correct + urgency_correct) / 2.0

# Optimize
optimizer = dspy.BootstrapFewShot(metric=metric, max_bootstrapped_demos=3)
optimized = optimizer.compile(classifier, trainset=trainset)

# Evaluate
from dspy.evaluate import Evaluate
score = Evaluate(devset=devset, metric=metric, num_threads=1)(optimized)
print(f"\nDev score: {score}")

Condensed from dspy.ai/api/primitives/Example/ and dspy.ai/api/primitives/Prediction/. Verify against upstream for latest.

dspy.Example — API Reference

Constructor

dspy.Example(base=None, **kwargs)

Methods

Input/output management

Data access

Data manipulation

Access patterns

ex = dspy.Example(question="What?", answer="That")

# Dot notation
ex.question           # "What?"

# Dict-style access
ex["question"]        # "What?"

# Membership test
"question" in ex      # True

---

dspy.Prediction — API Reference

Constructor

dspy.Prediction(*args, **kwargs)

Inherits from Example. Returned by all DSPy modules (Predict, ChainOfThought, etc.). Adds completion tracking and LM usage metadata. Supports comparison (<, >, <=, >=) and arithmetic (+, /) on predictions with a score field.

Additional methods (beyond Example)

All Example methods (with_inputs, inputs, labels, keys, values, items, get, toDict, copy, without) are also available on Prediction.

Key differences from Example

• Prediction strips _demos and _input_keys from internal state
• Prediction adds _completions and _lm_usage tracking
• Use toDict() to serialize Prediction output to a plain dict (handles nested Pydantic models)

---

Built-in datasets

from dspy.datasets import HotPotQA, GSM8k, Colors

All constructors accept: train_seed, train_size, dev_size, test_size (all optional).

Access splits via .train, .dev, .test attributes. Returns lists of dspy.Example objects.