Langfuse

Langfuse

This skill helps you use Langfuse effectively across all common workflows: instrumenting applications, migrating prompts, debugging traces, and accessing data programmatically.

Core Principles

Follow these principles for ALL Langfuse work:

1. Documentation First: NEVER implement based on memory. Always fetch current docs before writing code (Langfuse updates frequently) See the section below on how to access documentation. 2. CLI for Data Access: Use langfuse-cli when querying/modifying Langfuse data. See the section below on how to use the CLI. 3. Best Practices by Use Case: Check the relevant reference file below for use-case-specific guidelines before implementing 4. Use latest Langfuse versions: Unless the user specified otherwise or there's a good reason, always use the latest version of Langfuse SDKs/APIs.

Use case specific references

• instrumenting an existing function/application: references/instrumentation.md
• migrating prompts from a codebase into Langfuse: references/prompt-migration.md
• capturing user feedback (thumbs, ratings, implicit signals) as scores on traces: references/user-feedback.md
• further tips on using the Langfuse CLI: references/cli.md
• upgrading or migrating Langfuse SDKs to the latest version: references/sdk-upgrade.md
• submitting feedback about this skill: references/skill-feedback.md

1. Langfuse API via CLI

Use the langfuse-cli to interact with the full Langfuse REST API from the command line. Run via npx (no install required):

Start by discovering the schema and available arguments:

# Discover all available resources
npx langfuse-cli api __schema

# List actions for a resource
npx langfuse-cli api <resource> --help

# Show args/options for a specific action
npx langfuse-cli api <resource> <action> --help

Credentials

Set environment variables before making calls:

export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
export LANGFUSE_HOST=https://cloud.langfuse.com # example for EU cloud. For US cloud it's us.cloud.langfuse.com, and can also be a self-hosted URL. The server must always be specified in order to access Langfuse.

If not set, ask the user for their API keys (found in Langfuse UI → Settings → API Keys).

Detailed CLI Reference

For common workflows, tips, and full usage patterns, see references/cli.md.

2. Langfuse Documentation

Three methods to access Langfuse docs, in order of preference. Always prefer your application's native web fetch and search tools (e.g., WebFetch, WebSearch, mcp_fetch, etc.) over curl when available. The URLs and patterns below work with any fetching method — the curl examples are just illustrative.

2a. Documentation Index (llms.txt)

Fetch the full index of all documentation pages:

curl -s https://langfuse.com/llms.txt

Returns a structured list of every doc page with titles and URLs. Use this to discover the right page for a topic, then fetch that page directly.

Alternatively, you can start on https://langfuse.com/docs and explore the site to find the page you need.

2b. Fetch Individual Pages as Markdown

Any page listed in llms.txt can be fetched as markdown by appending .md to its path or by using Accept: text/markdown in the request headers. Use this when you know which page contains the information needed. Returns clean markdown with code examples and configuration details.

curl -s "https://langfuse.com/docs/observability/overview.md"
curl -s "https://langfuse.com/docs/observability/overview" -H "Accept: text/markdown"

2c. Search Documentation

When you need to find information across all docs and github issues/discussions without knowing the specific page:

curl -s "https://langfuse.com/api/search-docs?query=<url-encoded-query>"

Example:

curl -s "https://langfuse.com/api/search-docs?query=How+do+I+trace+LangGraph+agents"

Returns a JSON response with:

• query: the original query
• answer: a JSON string containing an array of matching documents, each with:
• url: link to the doc page
• title: page title
• source.content: array of relevant text excerpts from the page

Search is a great fallback if you cannot find the relevant pages or need more context. Especially useful when debugging issues as all GitHub Issues and Discussions are also indexed. Responses can be large — extract only the relevant portions.

Documentation Workflow

1. Start with llms.txt to orient — scan for relevant page titles 2. Fetch specific pages when you identify the right one 3. Fall back to search when the topic is unclear and you want more context

Skill Feedback

When the user expresses that something about this skill is not working as expected, gives incorrect guidance, is missing information, or could be improved — offer to submit feedback to the Langfuse skill maintainers. This includes when:

• The skill gave wrong or outdated instructions
• A workflow didn't produce the expected result
• The user wishes the skill covered something it doesn't
• The user explicitly says something like "this should work differently" or "this is wrong"

Do NOT trigger this for issues with Langfuse itself (the product) — only for issues with this skill's instructions and behavior.

When triggered, follow the process in references/skill-feedback.md.

Langfuse CLI Reference

Install

# Run directly (recommended)
npx langfuse-cli api <resource> <action>
bunx langfuse-cli api <resource> <action>

# Or install globally
npm i -g langfuse-cli
langfuse api <resource> <action>

Discovery

# List all resources and auth info
langfuse api __schema

# List actions for a resource
langfuse api <resource> --help

# Show args/options for a specific action
langfuse api <resource> <action> --help

# Preview the curl command without executing
langfuse api <resource> <action> --curl

Credentials

Set environment variables:

export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
export LANGFUSE_HOST=https://cloud.langfuse.com  

Tips

• Use --json for machine-readable JSON output
• Use --curl to preview the HTTP request without executing
• Pagination: use --limit and --page on list endpoints
• All list commands support filtering — check <resource> <action> --help for available options
• Prefer observations-v2s over observations — the v2 endpoint returns richer data
• Prefer metrics-v2s over metrics — the v2 endpoint returns richer data
• Prefer score-v2s over scores — the v1 scores resource only supports create/delete; use score-v2s for list and get operations

Langfuse Observability

Instrument LLM applications with Langfuse tracing, following best practices and tailored to your use case.

When to Use

• Setting up Langfuse in a new project
• Auditing existing Langfuse instrumentation
• Adding observability to LLM calls

Workflow

1. Assess Current State

Check the project:

• Is Langfuse SDK installed?
• What LLM frameworks are used? (OpenAI SDK, LangChain, LlamaIndex, Vercel AI SDK, etc.)
• Is there existing instrumentation?

No integration yet: Set up Langfuse using a framework integration if available. Integrations capture more context automatically and require less code than manual instrumentation.

Integration exists: Audit against baseline requirements below.

2. Verify Baseline Requirements

Every trace should have these fundamentals:

Framework integrations (OpenAI, LangChain, etc.) handle model name, tokens, and observation types automatically. Prefer integrations over manual instrumentation.

Docs: https://langfuse.com/docs/tracing

3. Explore Traces First

Once baseline instrumentation is working, encourage the user to explore their traces in the Langfuse UI before adding more context:

"Your traces are now appearing in Langfuse. Take a look at a few of them—see what data is being captured, what's useful, and what's missing. This will help us decide what additional context to add."

This helps the user:

• Understand what they're already getting
• Form opinions about what's missing
• Ask better questions about what they need

4. Discover Additional Context Needs

Determine what additional instrumentation would be valuable. Infer from code when possible, only ask when unclear.

Infer from code:

Only ask when not obvious from code:

• "How do you know when a response is good vs bad?" → Determines scoring approach
• "What would you want to filter by in a dashboard?" → Surfaces non-obvious tags
• "Are there different user segments you'd want to compare?" → Customer tiers, plans, etc.

Additions and their value:

These are NOT baseline requirements—only add what's relevant based on inference or user input.

5. Guide to UI

After adding context, point users to relevant UI features:

• Traces view: See individual requests
• Sessions view: See grouped conversations (if session_id added)
• Dashboard: Build filtered views using tags
• Scores: Filter by quality metrics

Framework Integrations

Prefer these over manual instrumentation:

Full list: https://langfuse.com/docs/integrations

Always Explain Why

When suggesting additions, explain the user benefit:

"I recommend adding session_id to your traces.

Why: This groups messages from the same conversation together.
You'll be able to see full conversation flows in the Sessions view,
making it much easier to debug multi-turn interactions.

Learn more: https://langfuse.com/docs/tracing-features/sessions"

Common Mistakes

Langfuse Prompt Migration

Migrate hardcoded prompts to Langfuse for version control, A/B testing, and deployment-free iteration.

Prerequisites

Verify credentials before starting:

echo $LANGFUSE_PUBLIC_KEY   # pk-...
echo $LANGFUSE_SECRET_KEY   # sk-...
echo $LANGFUSE_HOST         # https://cloud.langfuse.com or self-hosted

If not set, ask user to configure them first.

Migration Flow

1. Scan codebase for prompts
2. Analyze templating compatibility
3. Propose structure (names, subprompts, variables)
4. User approves
5. Create prompts in Langfuse
6. Refactor code to use get_prompt()
7. Link prompts to traces (if tracing enabled)
8. Verify application works

Step 1: Find Prompts

Search for these patterns:

Step 2: Check Templating Compatibility

CRITICAL: Langfuse only supports simple {{variable}} substitution. No conditionals, loops, or filters.

Decision Tree

Contains {% if %}, {% for %}, or filters?
├─ No → Direct migration
└─ Yes → Choose:
    ├─ Option A (RECOMMENDED): Move logic to code, pass pre-computed values
    └─ Option B: Store raw template, compile client-side with Jinja2
        └─ ⚠️ Loses: Playground preview, UI experiments

Simplifying Complex Templates

Conditionals → Pre-compute in code:

# Instead of {% if user.is_premium %}...{% endif %} in prompt
# Use {{tier_message}} and compute value in code before compile()

Loops → Pre-format in code:

# Instead of {% for tool in tools %}...{% endfor %} in prompt
# Use {{tools_list}} and format the list in code before compile()

For external templating details, fetch: https://langfuse.com/faq/all/using-external-templating-libraries

Step 3: Propose Structure

Naming Conventions

Identify Subprompts

Extract when:

• Same text in 2+ prompts
• Represents distinct component (personality, safety rules, format)
• Would need to change together

Variable Extraction

Step 4: Present Plan to User

Format:

Found N prompts across M files:

src/chat.py:
  - System prompt (47 lines) → 'chat-assistant'

src/support/triage.py:
  - Triage prompt (34 lines) → 'support/triage'
    ⚠️ Contains {% if %} - will simplify

Subprompts to extract:
  - '_base-personality' - used by: chat-assistant, support/triage

Variables to add:
  - {{user_name}} - hardcoded in 2 prompts

Proceed?

Step 5: Create Prompts in Langfuse

Use langfuse.create_prompt() with:

• name: Your chosen name
• prompt: Template text (or message array for chat type)
• type: "text" or "chat"
• labels: ["production"] (they're already live)
• config: Optional model settings

Labeling strategy:

• production → All migrated prompts
• staging → Add later for testing
• latest → Auto-applied by Langfuse

For full API: fetch https://langfuse.com/docs/prompts/get-started

Step 6: Refactor Code

Replace hardcoded prompts with:

prompt = langfuse.get_prompt("name", label="production")
messages = prompt.compile(var1=value1, var2=value2)

Key points:

• Always use label="production" (not latest) for stability
• Call .compile() to substitute variables
• For chat prompts, result is message array ready for API

For SDK examples (Python/JS/TS): fetch https://langfuse.com/docs/prompts/get-started

Step 7: Link Prompts to Traces

If codebase uses Langfuse tracing, link prompts so you can see which version produced each response.

Detect Existing Tracing

Look for:

• @observe() decorators
• langfuse.trace() calls
• from langfuse.openai import openai (instrumented client)

Link Methods

Verify in UI

1. Go to Traces → select a trace 2. Click on Generation 3. Check Prompt field shows name and version

For tracing details: fetch https://langfuse.com/docs/prompts/get-started#link-with-langfuse-tracing

Step 8: Verify Migration

Checklist

• [ ] All prompts created with production label
• [ ] Code fetches with label="production"
• [ ] Variables compile without errors
• [ ] Subprompts resolve correctly
• [ ] Application behavior unchanged
• [ ] Generations show linked prompt in UI (if tracing)

Common Issues

Out of Scope

• Prompt engineering (writing better prompts)
• Evaluation setup
• A/B testing workflow
• Non-LLM string templates

Langfuse SDK Upgrade Guide

Assist users in upgrading their Langfuse SDK to the latest version. The Python and JS/TS SDKs share the same architectural changes but differ in syntax.

When to Use

• User asks to upgrade/migrate their Langfuse SDK
• User is on an older SDK version and encounters deprecated APIs
• User wants to adopt the latest Langfuse features

Migration Docs

Always fetch the latest migration guide before starting — these pages are the source of truth:

• Python (v3 → v4): https://langfuse.com/docs/observability/sdk/upgrade-path/python-v3-to-v4
• JS/TS (v4 → v5): https://langfuse.com/docs/observability/sdk/upgrade-path/js-v4-to-v5

Fetch the relevant page as markdown before implementing any changes:

curl -s "https://langfuse.com/docs/observability/sdk/upgrade-path/python-v3-to-v4.md"
curl -s "https://langfuse.com/docs/observability/sdk/upgrade-path/js-v4-to-v5.md"

Upgrade Checklist

Work through each item in order. Skip items that don't apply to the user's codebase.

Both SDKs

• [ ] Update the SDK package to the latest version
• [ ] Audit span filtering: Non-LLM spans (HTTP, DB, queues) no longer export by default. If the user relied on these, configure a custom should_export_span / shouldExportSpan filter
• [ ] Replace `update_current_trace()` / `updateActiveTrace()`: Split into three calls:
• propagate_attributes() / propagateAttributes() for correlating attributes (user_id, session_id, tags, metadata, trace_name)
• set_current_trace_io() / setActiveTraceIO() for input/output (deprecated — prefer setting I/O on root observation directly)
• set_current_trace_as_public() / setActiveTraceAsPublic() for public flag
• [ ] Replace `.update_trace()` / `.updateTrace()` on observation objects (same decomposition as above)
• [ ] Update API namespace references: observations_v_2 / observationsV2 → observations, score_v_2 / scoreV2 → scores, metrics_v_2 / metricsV2 → metrics. Legacy v1 APIs moved to api.legacy.*
• [ ] Validate metadata format: Must be dict[str, str] / Record<string, string> with values ≤200 characters
• [ ] Move `release` and `environment` from code parameters to environment variables (LANGFUSE_RELEASE, LANGFUSE_TRACING_ENVIRONMENT)
• [ ] Enable debug logging during migration to catch issues (debug=True in Python, LANGFUSE_DEBUG="true" in JS/TS)
• [ ] Test trace hierarchies to verify no spans are unexpectedly dropped

Python-specific

• [ ] Replace `start_span()` / `start_generation()` with start_observation() (use as_type="generation" for generations)
• [ ] Replace `start_as_current_span()` / `start_as_current_generation()` with start_as_current_observation()
• [ ] Replace dataset `item.run()` with dataset.run_experiment(name=..., task=...)
• [ ] Remove `CallbackHandler(update_trace=...)` parameter — use propagate_attributes() wrapper instead
• [ ] Upgrade to Pydantic v2 — the SDK now requires it. Use pydantic.v1 compatibility shim if migrating gradually
• [ ] Update removed types: TraceMetadata, ObservationParams removed from langfuse.types. Import MapValue, ModelUsage, PromptClient from langfuse.model

JS/TS-specific

• [ ] Update LangChain `CallbackHandler` — traceMetadata now requires string values; internal behavior uses propagateAttributes() instead of direct trace updates
• [ ] Update OpenAI integration — traceMethod wrapper now uses propagateAttributes() internally; wrap entire execution in propagateAttributes() if relying on parent attribute inheritance

Key API Changes Reference

Correlating attributes (both SDKs)

Before:

# Python
langfuse.update_current_trace(name="trace-name", user_id="user-123", session_id="session-abc", tags=["tag1"])

// JS/TS
updateActiveTrace({ name: "trace-name", userId: "user-123", sessionId: "session-456", tags: ["prod"] });

After:

# Python
from langfuse import propagate_attributes

with propagate_attributes(trace_name="trace-name", user_id="user-123", session_id="session-abc", tags=["tag1"]):
    result = call_llm("hello")

// JS/TS
import { propagateAttributes } from "langfuse";

await propagateAttributes(
  { traceName: "trace-name", userId: "user-123", sessionId: "session-456", tags: ["prod"] },
  async () => { /* traced code */ }
);

Span/Generation creation (Python)

Before:

langfuse.start_span(name="x")
langfuse.start_generation(name="x", model="gpt-4")

After:

langfuse.start_observation(name="x")
langfuse.start_observation(name="x", as_type="generation", model="gpt-4")

Dataset experiments (Python)

Before:

for item in dataset.items:
    with item.run(run_name="my-run") as span:
        result = my_llm(item.input)
        span.update(output=result)

After:

def my_task(*, item, **kwargs):
    return my_llm(item.input)

dataset.run_experiment(name="my-run", task=my_task)

Span filtering (both SDKs)

To restore pre-upgrade "export all" behavior:

# Python
langfuse = Langfuse(should_export_span=lambda span: True)

// JS/TS
const spanProcessor = new LangfuseSpanProcessor({ shouldExportSpan: () => true });

To extend defaults with custom scopes:

# Python
from langfuse.span_filter import is_default_export_span

langfuse = Langfuse(
    should_export_span=lambda span: (
        is_default_export_span(span)
        or span.instrumentation_scope.name.startswith("my_framework")
    )
)

// JS/TS
import { isDefaultExportSpan } from "@langfuse/otel";

shouldExportSpan: ({ otelSpan }) =>
  isDefaultExportSpan(otelSpan) || otelSpan.instrumentationScope.name.startsWith("my_framework")

Common Pitfalls

Best Practices

1. Always fetch the migration docs first — they are the canonical source and may have been updated since this guide was written 2. Enable debug logging during migration to surface dropped spans and trace hierarchy issues 3. Use `propagate_attributes()` as the primary mechanism for setting trace-level correlating attributes 4. Set input/output on root observations directly rather than using deprecated trace-level setters 5. Compose custom span filters with is_default_export_span / isDefaultExportSpan to extend defaults rather than replacing them entirely 6. Test thoroughly — run the application with debug logging, check the Langfuse UI for missing or orphaned spans, verify metadata appears correctly 7. Migrate incrementally — upgrade the SDK first, fix breaking changes, then adopt new patterns

Skill Feedback

Follow these steps exactly:

1. Ask permission: Ask the user if they'd like you to submit feedback to the skill maintainers. Make it clear this is about the skill (the agent instructions), not about Langfuse the product. If they decline, move on. 2. Draft feedback: Write the feedback using the form structure below. Present the draft to the user and ask if they'd like to change anything before submitting. 3. Submit: Once approved, submit via gh CLI as described below. Share the resulting discussion URL with the user.

Feedback Form Structure

Draft the feedback using these two fields:

Describe your idea or feedback (required) A clear description of what went wrong or what could be improved. Include:

• What the user was trying to do
• What the skill did vs what was expected
• Any specific instructions that were incorrect or missing

What would the ideal outcome look like? (optional) What the correct behavior or guidance should be.

Format the body as markdown with the two field labels as headings.

Submitting

Create a GitHub Discussion on the langfuse/skills repository using the GraphQL API:

gh api graphql -f query='
mutation($repoId: ID!, $categoryId: ID!, $title: String!, $body: String!) {
  createDiscussion(input: {repositoryId: $repoId, categoryId: $categoryId, title: $title, body: $body}) {
    discussion { url }
  }
}' \
  -f repoId="$(gh api graphql -f query='{ repository(owner: "langfuse", name: "skills") { id } }' --jq '.data.repository.id')" \
  -f categoryId="$(gh api graphql -f query='{ repository(owner: "langfuse", name: "skills") { discussionCategories(first: 10) { nodes { id name } } } }' --jq '.data.repository.discussionCategories.nodes[] | select(.name == "Ideas & Improvements") | .id')" \
  -f title="<concise title>" \
  -f body="<formatted feedback>"

If the gh CLI is not authenticated or the request fails, give the user this link to create the discussion manually:

https://github.com/langfuse/skills/discussions/new?category=ideas-improvements

After submission, share the discussion URL with the user.

User Feedback

Tracing must already be set up — feedback is stored as scores on traces.

Docs: https://langfuse.com/docs/observability/features/user-feedback

Workflow

1. Determine What Feedback to Capture

If the user has asked for something specific, go with that. Otherwise, look at the application and present a few UX options for how feedback could work, then ask the user which they prefer before implementing.

Common UX patterns to suggest:

This table is not exhaustive — if the application suggests a different feedback pattern that fits better, propose that instead. Present 2–3 options that match the application's use case and ask the user which approach they'd like. This decision shapes everything downstream (score names, data types, frontend components), so it's important to align early.

Feedback can be explicit (user rates via thumbs, stars, etc.) or implicit (derived from behavior like copying output, retrying, or escalating to support). Both are stored as scores. Explicit feedback requires the trace ID to reach the frontend; implicit feedback is logged server-side where the event already happens.

2. Choose Score Names

Name reflects the signal source, not what you hope it measures (e.g., user-thumbs not response-quality — a thumbs down doesn't tell you what was wrong). Avoid generic names like feedback or score.

Rules:

• Lowercase with hyphens
• One consistent name per feedback type across the entire app
• If capturing multiple signals, each gets its own distinct name

3. Implement Score Creation

For implicit feedback (server-side): Use langfuse.create_score() / langfuse.score.create() wherever the event is already handled in application code. Fetch SDK docs for current API: https://langfuse.com/docs/evaluation/evaluation-methods/scores-via-sdk

For explicit feedback (frontend): Use LangfuseWeb in the browser. It uses the public key only — no secret key exposed.

import { LangfuseWeb } from "langfuse";

const langfuse = new LangfuseWeb({
  publicKey: process.env.NEXT_PUBLIC_LANGFUSE_PUBLIC_KEY!,
  baseUrl: process.env.NEXT_PUBLIC_LANGFUSE_HOST,
});

langfuse.score({
  traceId,
  name: "user-thumbs",
  value: 1, // 1 = positive, 0 = negative
  dataType: "BOOLEAN",
  comment: optionalUserComment,
});

The trace ID must be available in the frontend for this to work. For Vercel AI SDK, the non-obvious pattern is using generateMessageId:

import { getActiveTraceId } from "@langfuse/tracing";

// Inside route handler wrapped with observe()
return result.toUIMessageStreamResponse({
  generateMessageId: () => getActiveTraceId() || crypto.randomUUID(),
});

4. Verify

Trigger a feedback action and check the trace's Scores tab in Langfuse. Confirm the score name, value, and data type are correct.

Point users to what they can do with feedback data: filter traces by low scores, use score analytics for trends, build annotation queues for team review.

Common Mistakes