Dspy Langtrace

Langtrace — Open-Source LLM Observability for DSPy

Guide the user through setting up Langtrace for automatic DSPy tracing and observability.

What is Langtrace

Langtrace is an open-source LLM observability platform with first-class DSPy auto-instrumentation. One line of code traces all DSPy LM calls, retrievals, module executions, token counts, and cost — no manual decorators needed.

• Cloud: Managed at app.langtrace.ai
• Self-hosted: Run your own instance with Docker
• Open source: github.com/Scale3-Labs/langtrace

What gets traced automatically

When to use Langtrace

Use Langtrace when:

• You want the easiest DSPy tracing setup (one line)
• You need auto-instrumentation without decorating every function
• You want a cloud dashboard with no infrastructure to manage
• You need self-hosted tracing for data privacy

Do NOT use Langtrace when:

• You need deep evaluation/evals features — see /dspy-phoenix (Phoenix has built-in evals)
• Your team is already invested in W&B for experiment tracking — see /dspy-weave
• You need the full ML lifecycle (model registry, deployment) — see /dspy-mlflow

Setup

Install

pip install langtrace-python-sdk

Cloud setup (quickest)

1. Sign up at app.langtrace.ai 2. Create a project and copy your API key 3. Add two lines to your code:

from langtrace_python_sdk import langtrace

langtrace.init(api_key="your-key")  # or set LANGTRACE_API_KEY env var

# That's it — all DSPy calls are now traced automatically
import dspy
dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

program = dspy.ChainOfThought("question -> answer")
result = program(question="What is DSPy?")
# View traces at app.langtrace.ai

Self-hosted setup (Docker)

For teams that need data to stay on-premises:

# Clone and start Langtrace
git clone https://github.com/Scale3-Labs/langtrace.git
cd langtrace
docker compose up -d

Then point your SDK at your local instance:

from langtrace_python_sdk import langtrace

langtrace.init(api_host="http://localhost:3000")

# All traces go to your self-hosted instance

Environment variable configuration

export LANGTRACE_API_KEY="your-key"           # Cloud API key
# OR
export LANGTRACE_API_HOST="http://localhost:3000"  # Self-hosted URL

from langtrace_python_sdk import langtrace

langtrace.init()  # Picks up from environment variables

Tracing a DSPy pipeline

Langtrace auto-instruments the entire call tree. No changes to your DSPy code:

from langtrace_python_sdk import langtrace

langtrace.init(api_key="your-key")

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

class RAGPipeline(dspy.Module):
    def __init__(self):
        self.retrieve = dspy.Retrieve(k=3)
        self.answer = dspy.ChainOfThought("context, question -> answer")

    def forward(self, question):
        context = self.retrieve(question).passages
        return self.answer(context=context, question=question)

pipeline = RAGPipeline()
result = pipeline(question="How do refunds work?")
# Langtrace captures:
#   - The top-level RAGPipeline call
#   - The Retrieve call (query, passages, latency)
#   - The ChainOfThought LM call (prompt, response, tokens, cost)

Tracing optimization runs

Langtrace traces optimizer internals too — useful for understanding what MIPROv2 or GEPA tried:

from langtrace_python_sdk import langtrace

langtrace.init(api_key="your-key")

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

trainset = [...]  # your training examples

program = dspy.ChainOfThought("question -> answer")
optimizer = dspy.MIPROv2(metric=my_metric, auto="light")
optimized = optimizer.compile(program, trainset=trainset)
# Every LM call the optimizer makes is traced — see which candidates it tried

Viewing traces in the Langtrace UI

The Langtrace dashboard shows:

• Trace timeline: waterfall view of every step in a request
• Token counts & cost: per-call and aggregate
• Latency breakdown: which step is slowest
• Prompt/response viewer: full text of every LM interaction
• Filters: by time range, latency, status, and custom attributes

Adding custom attributes

Tag traces with metadata for filtering:

from langtrace_python_sdk import langtrace, with_langtrace_root_span

@with_langtrace_root_span("customer-query")
def handle_query(user_id, question):
    # Custom attributes appear in the UI for filtering
    langtrace.inject_additional_attributes({
        "user_id": user_id,
        "environment": "production",
    })
    return pipeline(question=question)

Langtrace vs Phoenix vs Jaeger

Decision guide

Want DSPy tracing?
|
+- Easiest setup, auto-instrument everything? -> Langtrace
+- Need built-in evaluation features? -> Arize Phoenix (/dspy-phoenix)
+- Team already uses W&B? -> W&B Weave (/dspy-weave)
+- Need full ML lifecycle (registry, deploy)? -> MLflow (/dspy-mlflow)
+- Team already uses Jaeger? -> Jaeger (see /ai-tracing-requests)

Cross-references

• Arize Phoenix (open-source with evals) — /dspy-phoenix
• W&B Weave (team dashboards, experiment tracking) — /dspy-weave
• MLflow (full ML lifecycle) — /dspy-mlflow
• Aggregate monitoring (not per-request) — /ai-monitoring
• Per-request debugging (inspect_history, JSONL traces) — /ai-tracing-requests
• For worked examples, see examples.md

Langtrace Examples

Trace a RAG pipeline and find slow requests

Setup

from langtrace_python_sdk import langtrace

langtrace.init(api_key="your-key")

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

class SupportBot(dspy.Module):
    def __init__(self):
        self.retrieve = dspy.Retrieve(k=5)
        self.answer = dspy.ChainOfThought("context, question -> answer")

    def forward(self, question):
        context = self.retrieve(question).passages
        return self.answer(context=context, question=question)

bot = SupportBot()

Run queries (all automatically traced)

questions = [
    "How do I reset my password?",
    "What's your refund policy?",
    "Can I upgrade my plan mid-cycle?",
    "How do I export my data?",
]

for q in questions:
    result = bot(question=q)
    print(f"Q: {q}\nA: {result.answer}\n")

Find slow requests in the Langtrace UI

1. Go to app.langtrace.ai (or your self-hosted URL) 2. Open your project 3. Sort traces by latency (descending) 4. Click a slow trace to see the waterfall view 5. Check which step is the bottleneck:

• Retrieve slow? Your vector DB may need optimization or the query is too broad
• LM call slow? The model may be overloaded or the prompt is too long

Trace with custom metadata for production filtering

from langtrace_python_sdk import langtrace, with_langtrace_root_span

langtrace.init(api_key="your-key")

import dspy

dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))

bot = SupportBot()

@with_langtrace_root_span("support-request")
def handle_support(user_id, plan, question):
    langtrace.inject_additional_attributes({
        "user_id": user_id,
        "plan": plan,
        "source": "api",
    })
    return bot(question=question)

# In production, filter traces by plan="enterprise" to debug issues for key accounts
result = handle_support("user-123", "enterprise", "How do I set up SSO?")

Self-hosted: Docker Compose with Postgres persistence

# docker-compose.override.yml — add to the Langtrace repo's docker-compose.yml
version: "3.8"
services:
  langtrace:
    environment:
      - DATABASE_URL=postgresql://langtrace:secret@db:5432/langtrace
    depends_on:
      - db
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: langtrace
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: langtrace
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

docker compose up -d

from langtrace_python_sdk import langtrace

langtrace.init(api_host="http://localhost:3000")

# Traces persist across restarts in the Postgres volume