Langfuse

Langfuse Skill

Debug AI agents and LLM applications through Langfuse observability.

This skill is the agent-facing companion to langfuse-mcp. It tells Claude Code and Codex when to use Langfuse, which MCP tool to call first, and how to move from broad trace discovery to a concrete root-cause hypothesis.

Triggers: langfuse, traces, debug AI, find exceptions, set up langfuse, what went wrong, why is it slow, datasets, evaluation sets

What This Skill Provides

• Setup steps for connecting langfuse-mcp to Claude Code or Codex.
• Playbooks for exception triage, trace inspection, latency analysis, sessions, prompts, and datasets.
• A quick reference for the highest-value MCP tools.
• Links to full setup and tool references for deeper troubleshooting.

Use the playbooks before guessing at individual tools. Start broad, identify the relevant trace/session/observation, then drill into the exact failure or slow path.

Setup

Step 1: Get credentials from https://cloud.langfuse.com → Settings → API Keys

If self-hosted, use your instance URL for LANGFUSE_HOST and create keys there.

Step 2: Install MCP (pick one):

Requires Python 3.10 or newer. CI verifies Python 3.10 through 3.14.

# Claude Code (project-scoped, shared via .mcp.json)
claude mcp add \
  --scope project \
  --env LANGFUSE_PUBLIC_KEY=pk-... \
  --env LANGFUSE_SECRET_KEY=sk-... \
  --env LANGFUSE_HOST=https://cloud.langfuse.com \
  langfuse -- uvx langfuse-mcp

# Codex CLI (user-scoped, stored in ~/.codex/config.toml)
codex mcp add langfuse \
  --env LANGFUSE_PUBLIC_KEY=pk-... \
  --env LANGFUSE_SECRET_KEY=sk-... \
  --env LANGFUSE_HOST=https://cloud.langfuse.com \
  -- uvx langfuse-mcp

Add --python 3.14 before langfuse-mcp if you want to pin a CI-verified interpreter explicitly.

Step 3: Restart CLI, verify with /mcp (Claude) or codex mcp list (Codex)

Step 4: Test: fetch_traces(age=60)

Read-Only Mode

For safer observability without risk of modifying prompts or datasets, enable read-only mode:

# CLI flag
langfuse-mcp --read-only

# Or environment variable
LANGFUSE_MCP_READ_ONLY=true

This disables write tools: create_text_prompt, create_chat_prompt, update_prompt_labels, create_dataset, create_dataset_item, delete_dataset_item.

Default Output Mode

If you want MCP clients to default to writing full payloads to files when they omit output_mode, configure:

langfuse-mcp --default-output-mode full_json_file

# Or via environment variable
LANGFUSE_MCP_DEFAULT_OUTPUT_MODE=full_json_file

For manual .mcp.json setup or troubleshooting, see references/setup.md.

---

Playbooks

"Where are the errors?"

find_exceptions(age=1440, group_by="file")

→ Shows error counts by file. Pick the worst offender.

find_exceptions_in_file(filepath="src/ai/chat.py", age=1440)

→ Lists specific exceptions. Grab a trace_id.

get_exception_details(trace_id="...")

→ Full stacktrace and context.

---

"What happened in this interaction?"

fetch_traces(age=60, user_id="...")

→ Find the trace. Note the trace_id.

If you don't know the user_id, start with:

fetch_traces(age=60)

fetch_trace(trace_id="...", include_observations=true)

→ See all LLM calls in the trace.

fetch_observation(observation_id="...")

→ Inspect a specific generation's input/output.

---

"Why is it slow?"

fetch_observations(age=60, type="GENERATION")

→ Find recent LLM calls. Look for high latency.

fetch_observation(observation_id="...")

→ Check token counts, model, timing.

---

"What's this user experiencing?"

get_user_sessions(user_id="...", age=1440)

→ List their sessions.

get_session_details(session_id="...")

→ See all traces in the session.

---

"Manage datasets"

list_datasets()

→ See all datasets.

get_dataset(name="evaluation-set-v1")

→ Get dataset details.

list_dataset_items(dataset_name="evaluation-set-v1", page=1, limit=10)

→ Browse items in the dataset.

create_dataset(name="qa-test-cases", description="QA evaluation set")

→ Create a new dataset.

create_dataset_item(
  dataset_name="qa-test-cases",
  input={"question": "What is 2+2?"},
  expected_output={"answer": "4"}
)

→ Add test cases.

create_dataset_item(
  dataset_name="qa-test-cases",
  item_id="item_123",
  input={"question": "What is 3+3?"},
  expected_output={"answer": "6"}
)

→ Upsert: updates existing item by id or creates if missing.

---

"Manage prompts"

list_prompts()

→ See all prompts with labels.

get_prompt(name="...", label="production")

→ Fetch current production version.

create_text_prompt(name="...", prompt="...", labels=["staging"])

→ Create new version in staging.

update_prompt_labels(name="...", version=N, labels=["production"])

→ Promote to production. (Rollback = re-apply label to older version)

---

Quick Reference

age = minutes to look back (max 10080 = 7 days)

---

Troubleshooting

MCP connection fails

• Verify credentials: check LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST
• Restart CLI after adding/updating MCP config
• Test MCP independently: fetch_traces(age=60) — if this fails, the issue is MCP, not the skill
• See references/setup.md for detailed troubleshooting

No traces found

• Increase the age parameter (default lookback may be too short)
• Verify your application is sending traces to the correct Langfuse project
• Check LANGFUSE_HOST points to the right instance (cloud vs self-hosted)

Permission denied

• Regenerate API keys from Langfuse dashboard
• Ensure keys have the required scopes for the operation
• Write operations require read-write keys (not read-only mode)

---

References

• references/tool-reference.md — Full parameter docs, filter semantics, response schemas
• references/setup.md — Manual setup, troubleshooting, advanced configuration

Langfuse MCP Setup Reference

Detailed setup instructions, troubleshooting, and configuration options.

Manual .mcp.json Setup

If you prefer manual configuration over claude mcp add:

{
  "mcpServers": {
    "langfuse": {
      "command": "uvx",
      "args": ["langfuse-mcp"],
      "env": {
        "LANGFUSE_PUBLIC_KEY": "pk-...",
        "LANGFUSE_SECRET_KEY": "sk-...",
        "LANGFUSE_HOST": "https://cloud.langfuse.com"
      }
    }
  }
}

Important: If .mcp.json already exists, merge the langfuse entry into the existing mcpServers object. Don't overwrite the file.

Add to .gitignore

grep -q '.mcp.json' .gitignore 2>/dev/null || echo '.mcp.json' >> .gitignore

Never commit credentials to version control.

---

Troubleshooting

Authentication Errors

• Public key must start with pk-
• Secret key must start with sk-
• Host must match your Langfuse instance (cloud vs self-hosted)

Python Version Errors

If MCP fails to connect, check your Python version. Langfuse MCP requires Python 3.10 or newer and CI currently verifies Python 3.10 through 3.14.

If your default interpreter is older than Python 3.10, pin a CI-verified Python in the uvx command:

uvx --python 3.14 langfuse-mcp

Or verify manually:

uvx --python 3.14 langfuse-mcp --help

Timeout Errors

Increase the timeout:

# Claude Code with timeout
claude mcp add \
  --scope project \
  --env LANGFUSE_PUBLIC_KEY=pk-... \
  --env LANGFUSE_SECRET_KEY=sk-... \
  --env LANGFUSE_HOST=https://cloud.langfuse.com \
  langfuse -- uvx langfuse-mcp --timeout 60

# Or in .mcp.json
"args": ["langfuse-mcp", "--timeout", "60"]

Empty Results

• Check age parameter (minutes, not hours/days)
• Verify filters match your data
• Try fetch_traces(age=1440) with no filters to confirm data exists
• Data older than 7 days (10080 minutes) cannot be retrieved

MCP Not Found

• Restart Claude Code / Codex CLI after adding MCP
• Run /mcp (Claude) or codex mcp list (Codex) to verify
• Check .mcp.json syntax (valid JSON, correct paths)
• Codex CLI uses ~/.codex/config.toml for config; verify that file instead of .mcp.json

---

Date-to-Minutes Conversion

---

Glossary

---

Tool Groups

Load specific tool groups to reduce token overhead:

langfuse-mcp --tools traces,prompts

---

Environment Variables

---

Skill Installation Scopes

Install globally:

cp -r skills/langfuse ~/.claude/skills/   # Claude Code
cp -r skills/langfuse ~/.codex/skills/    # Codex CLI

---

Security Notes

• Never commit .mcp.json with real credentials
• Rotate keys if leaked
• full_json_file exports may contain sensitive user data
• Use environment variable injection in CI/CD rather than hardcoded values

Langfuse MCP Tool Reference

Reference documentation for the Langfuse MCP tools. The top-level README lists the current full tool inventory; this reference focuses on the most common debugging and management workflows.

Tools by Category

\Tools marked with \ are disabled in read-only mode (`--read-only` or `LANGFUSE_MCP_READ_ONLY=true`).

Output Modes

Some tools support output modes via the output_mode parameter:

Tools with output_mode: fetch_traces, fetch_trace, fetch_observations, fetch_observation, fetch_sessions, get_session_details, get_user_sessions, find_exceptions_in_file, get_exception_details, list_dataset_items, get_dataset_item

Tools without output_mode: find_exceptions, get_error_count, list_prompts, get_prompt, get_prompt_unresolved, create_text_prompt, create_chat_prompt, update_prompt_labels, list_datasets, get_dataset, create_dataset, create_dataset_item, delete_dataset_item, get_data_schema

Filter Semantics

Filter behavior depends on the Langfuse API:

Note: list_prompts uses exact name matching. Other tools pass filters to the Langfuse API.

Sort Order

Sort order depends on the Langfuse API. Traces and observations are typically sorted by timestamp descending (newest first).

Pagination

Some tools support pagination via page and limit parameters. Check individual tool docs.

Tools with pagination: fetch_traces, fetch_observations, fetch_sessions, list_prompts, list_datasets, list_dataset_items

Tools without pagination: find_exceptions, find_exceptions_in_file, get_exception_details, get_user_sessions

Tip: For large results, use output_mode="full_json_file" to avoid context overflow.

---

Traces

fetch_traces

Search and filter traces with pagination.

Parameters:

Returns: List of trace objects with metadata including pagination info.

Example:

fetch_traces(age=1440, user_id="user_123", include_observations=true)

---

fetch_trace

Fetch a specific trace by ID.

Parameters:

Returns: Single trace object with all details.

Example:

fetch_trace(trace_id="abc-123", include_observations=true, output_mode="full_json_file")

---

Observations

fetch_observations

Search and filter observations (spans, generations, events).

Parameters:

Returns: List of observation objects.

Example:

fetch_observations(age=60, type="GENERATION", name="chat-completion")

---

fetch_observation

Fetch a specific observation by ID.

Parameters:

Returns: Single observation object with full details.

---

Sessions

fetch_sessions

List recent sessions with pagination.

Parameters:

Returns: List of session summaries.

---

get_session_details

Get detailed session info by ID.

Parameters:

Returns: Session object with all traces.

---

get_user_sessions

Get all sessions for a user.

Parameters:

Returns: List of sessions for the user.

---

Exceptions

find_exceptions

Find exceptions grouped by file, function, or type.

Parameters:

Returns: List of {group: string, count: int} objects, sorted by count descending (top 50).

Note: Does not support output_mode parameter.

Example:

find_exceptions(age=1440, group_by="type")

---

find_exceptions_in_file

Find exceptions in a specific file.

Parameters:

Returns: List of exception details (top 10, newest first):

• observation_id, trace_id, timestamp
• exception_type, exception_message, exception_stacktrace
• function, line_number

Example:

find_exceptions_in_file(filepath="src/ai/chat.py", age=1440)

---

get_exception_details

Get detailed exception info for a trace/span.

Parameters:

Returns: List of exceptions with full context including observation details.

---

get_error_count

Get total error count.

Parameters:

Returns:

{
  "data": {
    "age_minutes": 60,
    "from_timestamp": "2024-01-15T09:30:00Z",
    "to_timestamp": "2024-01-15T10:30:00Z",
    "trace_count": 5,
    "observation_count": 12,
    "exception_count": 18
  },
  "metadata": {...}
}

Note: Does not support output_mode parameter.

---

Prompts

list_prompts

List and filter prompts in the project.

Parameters:

Returns: List of prompt metadata:

• name, type ("text" or "chat")
• versions, labels, tags
• lastUpdatedAt, lastConfig

---

get_prompt

Fetch a specific prompt with resolved dependencies.

Parameters:

Returns: Prompt object:

• id, name, version, type
• prompt (string for text, list for chat)
• labels, tags, config

Example:

get_prompt(name="chat-system", label="production")

---

get_prompt_unresolved

Fetch a prompt WITHOUT resolving dependencies.

Returns raw prompt content with dependency tags intact (e.g., @@@langfusePrompt:name=xxx@@@) when the SDK supports resolve=false. If the SDK doesn't support this parameter, returns the resolved prompt and sets metadata.resolved=true to indicate fallback behavior.

Parameters: Same as get_prompt.

Returns: Same structure but with dependency tags preserved in prompt content. Check metadata.resolved to verify if unresolved content was returned.

---

create_text_prompt

Create a new text prompt version.

Parameters:

Config Example:

{
  "model": "gpt-4",
  "temperature": 0.7,
  "max_tokens": 1000,
  "top_p": 1.0
}

Returns: Created prompt object.

Note: Prompts are immutable. Creating a new version is the only way to update content. Labels are unique across versions - assigning a label here removes it from other versions.

Example:

create_text_prompt(
  name="greeting",
  prompt="Hello {{name}}, welcome to {{app}}!",
  labels=["staging"],
  config={"model": "gpt-4", "temperature": 0.7},
  commit_message="feat: add personalized greeting"
)

---

create_chat_prompt

Create a new chat prompt version.

Parameters:

Prompt Format:

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "{{user_input}}"}
]

Returns: Created prompt object.

Example:

create_chat_prompt(
  name="assistant",
  prompt=[
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "{{question}}"}
  ],
  labels=["staging"],
  config={"model": "gpt-4", "temperature": 0.3}
)

---

update_prompt_labels

Update labels for a specific prompt version.

Parameters:

Returns: Updated prompt object.

Note: This is the only supported mutation for existing prompts. Labels are unique across versions - adding a label here removes it from other versions.

Example (promote to production):

update_prompt_labels(name="greeting", version=3, labels=["production"])

Example (rollback):

update_prompt_labels(name="greeting", version=2, labels=["production"])

---

Datasets

list_datasets

List all datasets with pagination.

Parameters:

Returns: List of dataset objects with metadata.

Example:

list_datasets(page=1, limit=20)

---

get_dataset

Get a dataset by name.

Parameters:

Returns: Dataset object with full details.

Example:

get_dataset(name="evaluation-set-v1")

---

list_dataset_items

List items in a dataset with optional filters.

Parameters:

Returns: List of dataset items.

Example:

list_dataset_items(dataset_name="evaluation-set-v1", page=1, limit=10)

---

get_dataset_item

Get a specific dataset item by ID.

Parameters:

Returns: Dataset item object with full details.

Example:

get_dataset_item(item_id="item-abc-123")

---

create_dataset

Create a new dataset.

Parameters:

Returns: Created dataset object.

Example:

create_dataset(name="qa-evaluation-set", description="QA test cases for v2.0")

---

create_dataset_item

Create or upsert a dataset item.

Parameters:

Returns: Created or updated dataset item object.

Example:

create_dataset_item(
  dataset_name="qa-evaluation-set",
  input={"question": "What is the capital of France?"},
  expected_output={"answer": "Paris"},
  metadata={"category": "geography"}
)

---

delete_dataset_item

Delete a dataset item.

Parameters:

Returns: Confirmation of deletion.

Example:

delete_dataset_item(item_id="item-abc-123")

---

Schema

get_data_schema

Get schema information for response structures.

Parameters: None (dummy parameter for compatibility).

Returns: Documentation of response schemas for all tools.

---

Response Format

Tools return responses in one of two formats depending on output_mode:

For compact and full_json_file modes:

{
  "data": [...] | {...},
  "metadata": {
    "item_count": 10,
    "page": 1,
    "total": 100,
    "next_page": 2,
    "file_path": "/tmp/langfuse_mcp/traces_2024...json",
    "file_info": {...}
  }
}

For full_json_string mode:

Returns a string containing serialized JSON (not an object). Parse it if you need structured access.

---

Example Response (compact)

{
  "data": [
    {
      "id": "trace-abc-123",
      "name": "chat-completion",
      "user_id": "user-456",
      "timestamp": "2024-01-15T10:30:00Z",
      "observations": ["obs-1", "obs-2"]
    }
  ],
  "metadata": {
    "item_count": 1,
    "page": 1,
    "total": 47,
    "next_page": 2,
    "file_path": null,
    "file_info": null
  }
}

Example Response (full_json_file)

{
  "data": [{"id": "trace-abc-123", "...": "truncated"}],
  "metadata": {
    "item_count": 1,
    "page": 1,
    "total": 47,
    "file_path": "/tmp/langfuse_mcp/traces_20240115_103000.json",
    "file_info": {
      "size_bytes": 15234,
      "created_at": "2024-01-15T10:30:05Z"
    }
  }
}