Ai Auditing Code

Audit Your DSPy Code

When NOT to use this skill:

• Code is crashing or throwing errors -> use /ai-fixing-errors first
• Need to measure or improve accuracy -> use /ai-improving-accuracy
• Have not built anything yet -> use /ai-kickoff or /ai-choosing-architecture
• Want to learn a specific DSPy API -> use the matching /dspy-* skill

---

Step 1: Ask 2 questions

Before reading any code, ask:

1. Point me to the code — which files contain your DSPy code? 2. What are your concerns? — general review, a specific worry, or a pre-launch check?

Do not proceed until you have answers to both.

---

Step 2: Read the code

Read all DSPy-related files the user points you to. Build a mental model of:

• Signatures defined and their field names/types
• Modules defined and how they compose
• How forward() methods pass data between sub-modules
• Where data is loaded and how examples are constructed
• Whether a metric exists and how it is implemented
• Whether an optimizer is used and how it is called

---

Step 3: Run the 7-category audit

For each category, check the items in reference.md. Mark each finding:

• CRITICAL — causes silent failures, wrong results, or data loss (e.g., missing with_inputs(), metric always returns True)
• WARNING — suboptimal but works (e.g., Predict where ChainOfThought would help, no error handling)
• INFO — style or convention suggestions (e.g., naming, file organization)

Category 1: Signature Design

Check that field names are descriptive (not output, result, text), types are correct, complex outputs use Pydantic models, and ambiguous fields have a desc= argument.

Category 2: Module Composition

Check that modules are registered in __init__, forward() passes sub-module outputs as typed fields (not strings), and there is no raw string manipulation of LM outputs.

Category 3: Data Pipeline

Check that with_inputs() is called on every dspy.Example, a train/dev split exists, example field names match the signature, and data loading is reproducible (no random shuffles without a seed).

Category 4: Metric Design

Check that the metric handles edge cases (empty strings, None), returns a float between 0 and 1 or a bool, accepts the trace parameter, and can be tested independently of the module.

Category 5: Optimizer Usage

Check that the right optimizer is chosen for the dataset size, the trainset is large enough, the metric is passed correctly, and the optimized program is saved with program.save().

Category 6: Production Readiness

Check that LM calls are wrapped in error handling, timeouts are set, there is a fallback for LM failures, and costs have been estimated before deployment.

Category 7: Anti-patterns

Check for f-string prompt construction instead of signatures, direct lm() calls instead of using modules, hardcoded prompt strings alongside DSPy code, and mixed raw API calls with DSPy modules.

---

Step 4: Generate the findings report

After completing the audit, produce this report:

## DSPy Code Audit: [Project/Module Name]

### Summary
- X findings: Y critical, Z warnings, W info
- Overall assessment: [Ready for production / Needs fixes before production / Needs significant rework]

### Critical Findings
1. **[Category] — [Issue]**
   - File: path/to/file.py:line
   - Problem: ...
   - Fix: ...
   - Code:
     Before: <code>
     After:  <code>

### Warnings
1. **[Category] — [Issue]**
   - File: path/to/file.py:line
   - Problem: ...
   - Fix: ...

### Info
1. **[Category] — [Suggestion]**
   - ...

### Recommended Next Steps
1. Fix all critical findings
2. Address warnings in priority order
3. Run /ai-improving-accuracy to measure baseline quality after fixes

---

Step 5: Offer to fix

After presenting the report, ask:

"Would you like me to apply the critical and warning fixes directly to your code?"

If yes, make the fixes. Do not silently apply fixes during the audit itself.

---

Gotchas

1. Do not rewrite code during the audit. Audit first, present findings, then fix on request. Silent refactoring while reviewing confuses users about what changed and why.

2. Do not rate everything CRITICAL. Reserve CRITICAL for things that cause silent failures, wrong results, or data loss. Suboptimal patterns are WARNING. Style issues are INFO.

3. Do not audit accuracy instead of code. This skill reviews code structure and patterns — not whether the AI produces correct answers. For accuracy measurement, send the user to /ai-improving-accuracy.

4. Respect domain context. A simple Predict module may be entirely correct for a simple task. Do not recommend ChainOfThought everywhere or assume complexity is always better.

5. Do not suggest MIPROv2 for every finding. Not every issue requires an optimizer-level fix. Many issues are plain code bugs that need to be corrected before any optimization makes sense.

---

Additional resources

• Full 7-category checklist with code examples: reference.md
• Worked audit examples (ticket classifier, RAG pipeline, content generator): examples.md

Cross-references

Install any skill: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>

• Code is crashing? Fix it first with /ai-fixing-errors
• Measure accuracy after fixing — see /ai-improving-accuracy
• Plan your AI feature — see /ai-planning
• Pick the right DSPy pattern — see /ai-choosing-architecture
• Signature design patterns — see /dspy-signatures
• Module composition — see /dspy-modules
• Optimizer selection — see /dspy-optimizers
• Install `/ai-do` if you do not have it — it routes any AI problem to the right skill and is the fastest way to work: npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do

ai-auditing-code: Worked Examples

Three worked audits showing the full process: code being reviewed, findings report, and fixes applied.

---

Example 1: Ticket Classifier

Code being audited

# classifier.py
import dspy

lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)

class Classify(dspy.Signature):
    input: str = dspy.InputField()
    output: str = dspy.OutputField()

class TicketClassifier(dspy.Module):
    def forward(self, text):
        result = dspy.Predict(Classify)(input=text)
        return result.output

examples = [
    dspy.Example(input="my card was charged twice", output="billing"),
    dspy.Example(input="app keeps crashing", output="technical"),
    dspy.Example(input="reset my password", output="account"),
]

def my_metric(example, prediction):
    return prediction.output == example.output

optimizer = dspy.MIPROv2(metric=my_metric, auto="medium")
compiled = optimizer.compile(TicketClassifier(), trainset=examples)

Findings report

## DSPy Code Audit: TicketClassifier

### Summary
- 5 findings: 3 critical, 1 warning, 1 info
- Overall assessment: Needs fixes before production

### Critical Findings

1. **Data Pipeline — missing with_inputs()**
   - File: classifier.py:14-16
   - Problem: Examples do not call with_inputs(), so DSPy cannot distinguish input
     fields from output fields. The optimizer will treat all fields as inputs.
   - Fix: Add .with_inputs("input") to each example.
   - Code:
     Before: dspy.Example(input="my card was charged twice", output="billing")
     After:  dspy.Example(input="my card was charged twice", output="billing").with_inputs("input")

2. **Metric Design — metric ignores trace parameter**
   - File: classifier.py:18
   - Problem: Metric does not accept the trace parameter. This causes a TypeError
     during MIPROv2 optimization, which passes trace as a keyword argument.
   - Fix: Add trace=None to the metric signature.
   - Code:
     Before: def my_metric(example, prediction):
     After:  def my_metric(example, prediction, trace=None):

3. **Optimizer Usage — MIPROv2 with 3 examples**
   - File: classifier.py:21
   - Problem: MIPROv2 requires at least 50 examples to work effectively. With 3
     examples the optimizer cannot bootstrap meaningful demonstrations and will
     likely produce worse results than the baseline.
   - Fix: Use BootstrapFewShot for small datasets.
   - Code:
     Before: optimizer = dspy.MIPROv2(metric=my_metric, auto="medium")
     After:  optimizer = dspy.BootstrapFewShot(metric=my_metric, max_bootstrapped_demos=2)

### Warnings

1. **Signature Design — generic field names**
   - File: classifier.py:7-8
   - Problem: Fields named "input" and "output" give the LM no context about the
     task. This reduces accuracy because the LM cannot infer expected content.
   - Fix: Rename to ticket_text and category with descriptive desc= arguments.

### Info

1. **Module Composition — Predict created inside forward()**
   - File: classifier.py:12
   - Problem: dspy.Predict(Classify) is constructed on every call to forward().
     This works but prevents DSPy from tracking the module for optimization.
   - Fix: Move to __init__: self.classify = dspy.Predict(Classify)

### Recommended Next Steps
1. Add with_inputs() to all examples
2. Fix metric signature to accept trace parameter
3. Switch to BootstrapFewShot or add more training examples
4. Rename signature fields to be descriptive
5. Run /ai-improving-accuracy to measure baseline quality

Fixes applied

# classifier.py — after fixes
import dspy

lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)

class ClassifyTicket(dspy.Signature):
    """Classify a customer support ticket into a category."""
    ticket_text: str = dspy.InputField(desc="raw text of the customer support ticket")
    category: str = dspy.OutputField(desc="one of: billing, technical, account, general")

class TicketClassifier(dspy.Module):
    def __init__(self):
        self.classify = dspy.Predict(ClassifyTicket)

    def forward(self, ticket_text):
        return self.classify(ticket_text=ticket_text)

examples = [
    dspy.Example(ticket_text="my card was charged twice", category="billing").with_inputs("ticket_text"),
    dspy.Example(ticket_text="app keeps crashing", category="technical").with_inputs("ticket_text"),
    dspy.Example(ticket_text="reset my password", category="account").with_inputs("ticket_text"),
]

def my_metric(example, prediction, trace=None):
    pred = getattr(prediction, "category", None)
    true = getattr(example, "category", None)
    if pred is None or true is None:
        return False
    return pred.strip().lower() == true.strip().lower()

optimizer = dspy.BootstrapFewShot(metric=my_metric, max_bootstrapped_demos=2)
compiled = optimizer.compile(TicketClassifier(), trainset=examples)
compiled.save("ticket_classifier.json")

---

Example 2: RAG Pipeline

Code being audited

# rag.py
import dspy

lm = dspy.LM("openai/gpt-4o")
rm = dspy.ColBERTv2(url="http://my-colbert-server/")
dspy.configure(lm=lm, rm=rm)

class GenerateAnswer(dspy.Signature):
    context: str = dspy.InputField()
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

class RAGPipeline(dspy.Module):
    def forward(self, question):
        docs = dspy.Retrieve(k=3)(question)
        context = ""
        for doc in docs.passages:
            context = context + doc + " "
        result = dspy.ChainOfThought(GenerateAnswer)(
            context=context,
            question=question
        )
        return result

# training
trainset = load_training_data()  # returns 200 examples
optimizer = dspy.MIPROv2(metric=answer_metric, auto="medium")
compiled = optimizer.compile(RAGPipeline(), trainset=trainset)
print("done")  # optimized program is not saved

Findings report

## DSPy Code Audit: RAGPipeline

### Summary
- 4 findings: 2 critical, 1 warning, 1 info
- Overall assessment: Needs fixes before production

### Critical Findings

1. **Optimizer Usage — optimized program not saved**
   - File: rag.py:25
   - Problem: The compiled program exists only in memory. When the script exits,
     all optimization work is lost. The program must be saved to disk to be
     reused.
   - Fix: Call compiled.save() after compilation.
   - Code:
     Before: print("done")
     After:  compiled.save("rag_pipeline.json")

2. **Module Composition — sub-modules created inside forward()**
   - File: rag.py:14-15
   - Problem: dspy.Retrieve and dspy.ChainOfThought are constructed on every
     forward() call. DSPy cannot track these for optimization — the optimizer
     cannot tune their prompts or demonstrations.
   - Fix: Move both to __init__.

### Warnings

1. **Module Composition — string concatenation of retrieved docs**
   - File: rag.py:16-17
   - Problem: Concatenating passages with string addition and trailing spaces is
     fragile. If passages contain special characters or are very long, the
     context field may be malformed. Use str.join() instead.
   - Fix: context = "\n\n".join(docs.passages)

### Info

1. **Production Readiness — no error handling around LM calls**
   - File: rag.py:18-21
   - Problem: If the retrieval server is down or the LM times out, the pipeline
     raises an unhandled exception. A fallback response keeps the application
     running.

### Recommended Next Steps
1. Save the compiled program after optimization
2. Move Retrieve and ChainOfThought to __init__
3. Replace string concatenation with str.join()
4. Add try/except around the forward() body

Fixes applied

# rag.py — after fixes
import dspy

lm = dspy.LM("openai/gpt-4o", timeout=30)
rm = dspy.ColBERTv2(url="http://my-colbert-server/")
dspy.configure(lm=lm, rm=rm)

class GenerateAnswer(dspy.Signature):
    """Answer a question using the provided context passages."""
    context: str = dspy.InputField(desc="retrieved passages relevant to the question")
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

class RAGPipeline(dspy.Module):
    def __init__(self):
        self.retrieve = dspy.Retrieve(k=3)
        self.generate = dspy.ChainOfThought(GenerateAnswer)

    def forward(self, question):
        try:
            docs = self.retrieve(question)
            context = "\n\n".join(docs.passages)
            return self.generate(context=context, question=question)
        except Exception as e:
            import logging
            logging.error(f"RAG pipeline failed: {e}")
            return dspy.Prediction(answer="I was unable to retrieve an answer at this time.")

# training
trainset = load_training_data()
optimizer = dspy.MIPROv2(metric=answer_metric, auto="medium")
compiled = optimizer.compile(RAGPipeline(), trainset=trainset)
compiled.save("rag_pipeline.json")

---

Example 3: Content Generator

Code being audited

# generator.py
import openai
import dspy

lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)

SYSTEM_PROMPT = """You are a helpful content writer. Write in a friendly tone.
Always include a call to action. Use bullet points where appropriate."""

class DraftContent(dspy.Signature):
    topic: str = dspy.InputField()
    draft: str = dspy.OutputField()

class ContentGenerator(dspy.Module):
    def __init__(self):
        self.draft = dspy.Predict(DraftContent)

    def forward(self, topic, audience):
        # use raw OpenAI for the outline because DSPy is slow
        client = openai.OpenAI()
        outline_response = client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": SYSTEM_PROMPT},
                {"role": "user", "content": f"Create an outline for: {topic}, audience: {audience}"}
            ]
        )
        outline = outline_response.choices[0].message.content

        # inject outline into DSPy call
        prompt = f"Topic: {topic}\nOutline: {outline}\nWrite the full article now."
        result = lm(prompt)
        return result[0]["content"]

Findings report

## DSPy Code Audit: ContentGenerator

### Summary
- 5 findings: 3 critical, 1 warning, 1 info
- Overall assessment: Needs significant rework

### Critical Findings

1. **Anti-patterns — raw OpenAI API call inside DSPy module**
   - File: generator.py:18-25
   - Problem: The outline step uses openai.chat.completions.create() directly.
     This call is invisible to DSPy — it cannot be optimized, traced, cached, or
     swapped to a different model. The module cannot be compiled.
   - Fix: Replace with a DSPy signature and Predict module.

2. **Anti-patterns — direct lm() call with f-string prompt**
   - File: generator.py:27-28
   - Problem: Calling lm(prompt) with a manually constructed f-string bypasses
     the entire DSPy signature system. The output is raw LM text that has to be
     unpacked manually. This call cannot be optimized.
   - Fix: Define a signature for the drafting step and use a Predict or
     ChainOfThought module.

3. **Anti-patterns — hardcoded SYSTEM_PROMPT alongside DSPy code**
   - File: generator.py:6-9
   - Problem: SYSTEM_PROMPT is a hardcoded string that gets passed to the raw
     OpenAI call. This creates two separate prompt systems in the same file.
     DSPy manages prompts through signatures and optimizers — hardcoded strings
     will not be updated when the program is compiled.
   - Fix: Remove SYSTEM_PROMPT. Encode style instructions in the signature
     docstring or as field desc= values, which the optimizer can tune.

### Warnings

1. **Signature Design — audience field missing from signature**
   - File: generator.py:12-13
   - Problem: The forward() method accepts an audience parameter but the
     signature has no audience field. The audience value is injected via the
     f-string workaround rather than being a proper signature input. The LM
     never sees audience in a structured way.
   - Fix: Add audience as an InputField to the signature.

### Info

1. **Production Readiness — no error handling**
   - File: generator.py:18-29
   - Problem: Both the raw OpenAI call and the lm() call can raise exceptions.
     No error handling is present.

### Recommended Next Steps
1. Replace raw OpenAI call with a DSPy Outline signature
2. Replace lm() call with a DSPy Draft signature
3. Remove SYSTEM_PROMPT; encode intent in signature docstrings
4. Add audience to the signature
5. Add error handling

Fixes applied

# generator.py — after fixes
import dspy

lm = dspy.LM("openai/gpt-4o", timeout=30)
dspy.configure(lm=lm)

class CreateOutline(dspy.Signature):
    """Create a structured outline for a content piece in a friendly, engaging tone."""
    topic: str = dspy.InputField()
    audience: str = dspy.InputField(desc="description of the target audience for this content")
    outline: str = dspy.OutputField(desc="structured outline with main sections and key points")

class DraftContent(dspy.Signature):
    """Write a full article from an outline. Include a call to action at the end."""
    topic: str = dspy.InputField()
    audience: str = dspy.InputField(desc="description of the target audience")
    outline: str = dspy.InputField(desc="structured outline to follow")
    article: str = dspy.OutputField(desc="complete article ready for publication")

class ContentGenerator(dspy.Module):
    def __init__(self):
        self.outline = dspy.ChainOfThought(CreateOutline)
        self.draft = dspy.ChainOfThought(DraftContent)

    def forward(self, topic, audience):
        try:
            outline_result = self.outline(topic=topic, audience=audience)
            draft_result = self.draft(
                topic=topic,
                audience=audience,
                outline=outline_result.outline
            )
            return draft_result
        except Exception as e:
            import logging
            logging.error(f"Content generation failed for topic={topic!r}: {e}")
            raise

ai-auditing-code: 7-Category Checklist

Full checklist items and before/after code examples for each audit category.

---

Category 1: Signature Design

• [ ] Field names are descriptive — not input, output, result, text, data
• [ ] Output fields use specific types: str, int, float, bool, list[str], or Pydantic model
• [ ] Complex structured outputs use a pydantic.BaseModel, not a plain str that gets parsed later
• [ ] Ambiguous fields have a desc= argument explaining what the field should contain
• [ ] Input fields that are not part of the LM reasoning are not placed in the signature

Before — generic field names:

class Classify(dspy.Signature):
    input: str = dspy.InputField()
    output: str = dspy.OutputField()

After — descriptive field names with desc:

class ClassifyTicket(dspy.Signature):
    """Classify a customer support ticket into a category."""
    ticket_text: str = dspy.InputField(desc="raw text of the customer support ticket")
    category: str = dspy.OutputField(desc="one of: billing, technical, account, general")

Before — plain string output that gets parsed:

class ExtractEntities(dspy.Signature):
    document: str = dspy.InputField()
    entities: str = dspy.OutputField(desc="comma-separated list of entities")

After — typed Pydantic output:

from pydantic import BaseModel

class EntityList(BaseModel):
    names: list[str]
    organizations: list[str]

class ExtractEntities(dspy.Signature):
    document: str = dspy.InputField()
    entities: EntityList = dspy.OutputField()

---

Category 2: Module Composition

• [ ] All sub-modules are assigned in __init__ so DSPy can track them
• [ ] forward() passes outputs of one sub-module as typed fields to the next, not as strings
• [ ] No str(prediction) or prediction.completions[0] hacks to extract values
• [ ] No string concatenation to build inputs for sub-modules
• [ ] Module returns a dspy.Prediction or the fields directly, not a plain dict

Before — sub-module created inside forward(), not tracked:

class MyPipeline(dspy.Module):
    def forward(self, question):
        answer = dspy.ChainOfThought("question -> answer")(question=question)
        return answer

After — sub-module registered in __init__:

class MyPipeline(dspy.Module):
    def __init__(self):
        self.answer = dspy.ChainOfThought("question -> answer")

    def forward(self, question):
        return self.answer(question=question)

Before — string concatenation between modules:

def forward(self, docs):
    summaries = [self.summarize(doc=d).summary for d in docs]
    combined = "\n".join(summaries)
    return self.conclude(summaries=combined)

After — pass structured data:

def forward(self, docs):
    summaries = [self.summarize(doc=d).summary for d in docs]
    return self.conclude(summaries=summaries)  # pass list, let DSPy handle it

---

Category 3: Data Pipeline

• [ ] dspy.Example(...).with_inputs(...) is called on every example
• [ ] The field names in examples match the signature's input/output field names exactly
• [ ] A train/dev split exists; evaluation is not done on the training set
• [ ] Data loading uses a fixed random seed for reproducibility
• [ ] There are at least 20 examples in the training set (50+ recommended for MIPROv2)

Before — missing with_inputs():

examples = [
    dspy.Example(question="What is 2+2?", answer="4"),
    dspy.Example(question="Capital of France?", answer="Paris"),
]

After — with_inputs() called:

examples = [
    dspy.Example(question="What is 2+2?", answer="4").with_inputs("question"),
    dspy.Example(question="Capital of France?", answer="Paris").with_inputs("question"),
]

Before — no train/dev split:

examples = load_all_examples()
compiled = optimizer.compile(program, trainset=examples)
evaluate(program, devset=examples)  # evaluating on training data

After — proper split:

import random
random.seed(42)
all_examples = load_all_examples()
random.shuffle(all_examples)
split = int(0.8 * len(all_examples))
trainset = all_examples[:split]
devset = all_examples[split:]
compiled = optimizer.compile(program, trainset=trainset)
evaluate(program, devset=devset)

---

Category 4: Metric Design

• [ ] Metric function signature is metric(example, prediction, trace=None)
• [ ] Metric returns float in range 0.0–1.0 or bool
• [ ] Metric handles None and empty string values without crashing
• [ ] When trace is not None (optimization mode), metric can apply stricter criteria
• [ ] Metric logic can be tested with a simple unit test, independent of the module

Before — metric ignores trace parameter, always returns True:

def my_metric(example, prediction):
    return True

After — proper metric with trace parameter and real logic:

def my_metric(example, prediction, trace=None):
    if not prediction.answer:
        return False
    expected = example.answer.strip().lower()
    actual = prediction.answer.strip().lower()
    match = expected in actual or actual in expected
    if trace is not None:
        # stricter during optimization: require exact match
        return expected == actual
    return match

Before — metric crashes on edge cases:

def accuracy_metric(example, prediction, trace=None):
    return prediction.label.lower() == example.label.lower()

After — handles None and missing fields:

def accuracy_metric(example, prediction, trace=None):
    pred_label = getattr(prediction, "label", None)
    true_label = getattr(example, "label", None)
    if pred_label is None or true_label is None:
        return False
    return pred_label.strip().lower() == true_label.strip().lower()

---

Category 5: Optimizer Usage

• [ ] Optimizer is matched to dataset size: BootstrapFewShot for small datasets (under 50 examples), MIPROv2 for larger ones
• [ ] trainset passed to compile() contains dspy.Example objects with with_inputs() called
• [ ] Metric is passed to the optimizer, not hardcoded or skipped
• [ ] Optimized program is saved with program.save("path/program.json")
• [ ] The uncompiled baseline is evaluated before compilation for comparison

Before — MIPROv2 with 10 examples:

optimizer = dspy.MIPROv2(metric=my_metric, auto="medium")
compiled = optimizer.compile(program, trainset=small_set)  # only 10 examples

After — BootstrapFewShot for small datasets:

optimizer = dspy.BootstrapFewShot(metric=my_metric, max_bootstrapped_demos=4)
compiled = optimizer.compile(program, trainset=small_set)

Before — optimized program not saved:

compiled = optimizer.compile(program, trainset=trainset)
# program is lost when the script exits

After — save the optimized program:

compiled = optimizer.compile(program, trainset=trainset)
compiled.save("optimized_program.json")
# later: program.load("optimized_program.json")

---

Category 6: Production Readiness

• [ ] LM calls are wrapped in try/except with a meaningful fallback or error message
• [ ] Timeouts are configured on the LM client
• [ ] There is a retry strategy or fallback model for LM failures
• [ ] Token usage and estimated costs have been calculated for expected traffic
• [ ] Logging captures inputs/outputs for debugging production issues

Before — no error handling:

def classify(ticket_text):
    result = program(ticket_text=ticket_text)
    return result.category

After — error handling with fallback:

def classify(ticket_text):
    try:
        result = program(ticket_text=ticket_text)
        return result.category
    except Exception as e:
        logger.error(f"Classification failed: {e}", extra={"ticket_text": ticket_text})
        return "general"  # safe fallback category

Before — no timeout set:

lm = dspy.LM("openai/gpt-4o")
dspy.configure(lm=lm)

After — timeout configured:

lm = dspy.LM("openai/gpt-4o", timeout=30)
dspy.configure(lm=lm)

---

Category 7: Anti-patterns

• [ ] No f-string prompt construction that bypasses the signature system
• [ ] No direct lm("some prompt") calls — all LM calls go through a module
• [ ] No hardcoded prompt strings sitting alongside DSPy signatures in the same file
• [ ] No mixing of raw openai.chat.completions.create() calls with DSPy modules in the same pipeline
• [ ] No manual JSON parsing of LM outputs that should be typed fields

Before — f-string prompt instead of signature:

prompt = f"Classify this ticket: {ticket_text}. Categories: billing, technical, account."
result = lm(prompt)
category = result[0]["content"]

After — DSPy signature:

class ClassifyTicket(dspy.Signature):
    """Classify a customer support ticket."""
    ticket_text: str = dspy.InputField()
    category: str = dspy.OutputField(desc="one of: billing, technical, account, general")

classifier = dspy.Predict(ClassifyTicket)
result = classifier(ticket_text=ticket_text)
category = result.category

Before — direct lm() call:

lm = dspy.LM("openai/gpt-4o")
response = lm(f"Summarize: {doc}")

After — module with signature:

class Summarize(dspy.Signature):
    document: str = dspy.InputField()
    summary: str = dspy.OutputField()

summarizer = dspy.Predict(Summarize)
result = summarizer(document=doc)

Before — manual JSON parsing:

class ExtractData(dspy.Signature):
    text: str = dspy.InputField()
    result: str = dspy.OutputField(desc='JSON with keys "name" and "age"')

prediction = extractor(text=text)
data = json.loads(prediction.result)  # fragile, breaks if LM adds prose

After — typed Pydantic field:

from pydantic import BaseModel

class PersonData(BaseModel):
    name: str
    age: int

class ExtractData(dspy.Signature):
    text: str = dspy.InputField()
    result: PersonData = dspy.OutputField()

prediction = extractor(text=text)
data = prediction.result  # already a PersonData instance