Langsmith

LangSmith

When to use this skill

• The user already chose LangSmith and needs the smallest useful packet for tracing, evals, review, prompt ownership, or trace propagation.
• A request mentions LangChain tracing, `@traceable` / `traceable`, `wrap_openai` / `wrapOpenAI`, annotation queues, run trees, trace IDs, datasets, or experiment comparison.
• The team needs to decide whether a change needs debugging traces, an offline eval gate, an online review queue, Prompt Hub / prompt-registry work, or multi-service lineage.
• The user has some LangSmith setup already, but confidence is still low and they need an audit / gap review.

Do not use this skill as the main workflow when:

• The real job is generic dashboards / alerts / SLOs / telemetry ownership → monitoring-observability
• The real job is existing-log root-cause triage → log-analysis
• The real job is code-level reproduction / bug isolation → debugging
• The real job is deployment / rollout choreography → deployment-automation
• The real job is generic exported-metrics interpretation or KPI synthesis → data-analysis
• The real job is runtime guardrails / approvals / policy middleware → the relevant security or policy skill

Core idea

langsmith should behave like a workflow packet router, not a giant SDK encyclopedia.

1. Label the request before naming APIs. 2. Choose one primary packet. 3. Pick the smallest instrumentation / eval / review surface that answers the question. 4. Return a compact operator brief. 5. Route adjacent observability, debugging, deployment, and policy work outward fast.

Read these support docs before choosing the packet:

• references/intake-packets-and-route-outs.md
• references/modes-and-routing.md
• references/python-sdk.md
• references/typescript-sdk.md
• references/cli.md

Instructions

Step 1: Normalize the request

Convert the prompt into this intake packet first:

langsmith_packet:
  primary_packet: trace-debug | eval | review | prompt-registry | propagation | audit
  app_shape: chat | rag | tool-calling | agent | batch-job | multi-service | mixed | unknown
  runtime: python | typescript | mixed | cli-only | unknown
  request_type: setup | debug | pre-ship gate | production review | prompt decision | cross-service tracing | audit
  current_evidence: traces | datasets | evaluators | feedback queues | prompt versions | exported runs | none
  confidence_problem: missing traces | noisy traces | no regression gate | weak review rubric | prompt drift | broken lineage | mixed | unknown
  route_after: stay-here | monitoring-observability | log-analysis | debugging | deployment-automation | data-analysis

Choose one primary packet for the run. If two seem plausible, pick the one that reduces uncertainty fastest.

Step 2: Choose the packet

Packet rules:

• Prefer trace-debug when the team cannot explain the run tree.
• Prefer eval when the next risk is shipping without a regression gate.
• Prefer review when real-user output needs human or structured quality feedback.
• Prefer prompt-registry only when prompt/version choice is the actual bottleneck.
• Prefer propagation when one request fans out beyond a single service boundary.
• Prefer audit when the system already “has LangSmith” but confidence is still low.

Step 3: Pick the smallest useful surface

Rules:

• Prefer the smallest surface that answers the present question.
• Always name project/workspace/environment grouping.
• Preserve run_id, trace_id, and metadata if humans or downstream tools will review runs later.
• Include shutdown flush / wait behavior for CLIs, workers, and short-lived jobs.

Step 4: Choose the eval or review layer only when needed

Do not pretend one layer replaces the others.

Step 5: Return one compact LangSmith brief

Default response shape:

## LangSmith brief
- Packet: trace-debug | eval | review | prompt-registry | propagation | audit
- App shape: ...
- Runtime: ...
- Current evidence: ...

## Chosen surface
- wrappers / decorators / manual trace / dataset+experiment / review queue / prompt registry / propagation headers

## Required config
- `LANGSMITH_API_KEY`
- project/workspace/environment
- endpoint/workspace overrides if self-hosted

## Required IDs / metadata
- `run_id`, `trace_id`, tags, metadata fields, user/session identifiers as needed

## Verification step
- one trace query, one experiment comparison, one queue review check, or one prompt-version sanity check

## Route-out
- next neighboring skill if the bottleneck moves

Keep it compact. The point is to leave the user with one smallest next artifact, not a wall of SDK notes.

Step 6: Packet-specific heuristics

For trace-debug

• Start from the missing visibility surface: app span, tool calls, retrieval, provider call, or worker boundary.
• Prefer wrappers plus a few high-value decorators over tracing everything blindly.
• Return the first trace query/filter the team should run once instrumentation lands.

For eval

• Define the smallest dataset that reflects the risky change.
• Separate deterministic checks from LLM-as-judge or human review.
• Name baseline/candidate comparison, experiment naming, concurrency/cost limits, and the ship/no-ship decision point.

For review

• Define feedback keys, scale semantics, reviewer ownership, and queue cadence.
• Preserve run_id / trace_id so feedback attaches to the right artifact later.
• Route broader support-ops workflows out if the issue is bigger than output review.

For prompt-registry

• Explicitly decide whether Prompt Hub / LangSmith should own the prompt or whether Git/app config should stay canonical.
• Connect prompt versions to evals or experiments; do not treat registry as a separate island.
• Be explicit when public hub browsing is reference-only rather than the team's source of truth.

For propagation

• Name parent/child headers, metadata handoff rules, and which services must preserve trace context.
• Keep distributed-systems reliability and alerting concerns routed to generic observability where appropriate.

For audit

• Rank gaps by severity: missing coverage, bad metadata, absent evals, weak review flow, prompt/version ambiguity, broken flush or lineage.
• Recommend the first ratchet, not a giant reinstall plan.

Step 7: Route out aggressively

Switch as soon as LangSmith is no longer the main bottleneck:

• Generic telemetry / dashboards / alerts / SLOs → monitoring-observability
• Existing-log root-cause triage → log-analysis
• Reproduction and bug isolation → debugging
• Rollout / deploy / environment promotion → deployment-automation
• Exported metrics, experiments, or KPI interpretation → data-analysis
• Policy / approval / runtime guardrails → the relevant security or policy skill

Examples

Example 1: Trace-debug packet

Prompt:

Add LangSmith to this FastAPI RAG service so we can debug why answer quality regressed after a retriever change.

Good response shape:

• choose trace-debug
• recommend wrappers plus targeted higher-level spans if retrieval/tools must be visible
• define project/tags/metadata and a flush step for short-lived runs
• route generic service-health observability outward

Example 2: Eval packet

Prompt:

Before we ship this prompt rewrite, build a LangSmith eval workflow with a baseline, a candidate, and one judge for subjective quality.

Good response shape:

• choose eval
• define dataset, evaluator mix, experiment naming, and verification step
• separate deterministic checks from LLM-as-judge and human review
• keep rollout orchestration out of scope

Example 3: Review packet

Prompt:

We already have LangSmith traces, but support leads still do not trust the AI assistant. Set up a review workflow with human scoring.

Good response shape:

• choose review
• define feedback keys, rubric, queue ownership, and run-ID preservation
• keep the answer grounded in review workflow rather than generic prompt management

Example 4: Prompt-registry packet

Prompt:

Should these prompts live in LangSmith Prompt Hub or stay in Git?

Good response shape:

• choose prompt-registry
• compare platform-owned vs Git-owned prompt source of truth
• connect the choice to experiments and release confidence
• stay honest when Prompt Hub is optional

Example 5: Propagation packet

Prompt:

One chat request fans out to workers and tools. How do we keep one trace through LangSmith?

Good response shape:

• choose propagation
• define parent/child IDs, metadata propagation, and service boundaries
• route generic distributed-systems observability outward when needed

Best practices

1. Start with the packet, not the SDK call. 2. Treat LangSmith as a trace/eval/review substrate, not the entire production control plane. 3. Preserve run_id, trace_id, tags, and metadata deliberately. 4. Prefer the smallest useful surface that answers the present question. 5. Keep deterministic checks, LLM judges, and human review distinct. 6. Be explicit when Prompt Hub is optional or reference-only. 7. Include flush/wait behavior for scripts, workers, and cron-style jobs. 8. End with one verification step and one route-out.

References

• references/intake-packets-and-route-outs.md
• references/modes-and-routing.md
• references/python-sdk.md
• references/typescript-sdk.md
• references/cli.md
• Official docs: https://docs.langchain.com/langsmith
• SDK repo: https://github.com/langchain-ai/langsmith-sdk
• OpenEvals: https://github.com/langchain-ai/openevals

LangSmith CLI Reference

Source: https://docs.langchain.com/langsmith/langsmith-cli

Install: curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh

Installation

# Recommended (install script)
curl -sSL https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh | sh

# Go install
go install github.com/langchain-ai/langsmith-cli/cmd/langsmith@latest

Authentication

export LANGSMITH_API_KEY="lsv2_..."
export LANGSMITH_ENDPOINT="..."        # Self-hosted instances only
export LANGSMITH_PROJECT="my-project"  # Default project

Project Commands

langsmith project list
langsmith project list --name-contains "staging"
langsmith project create --name "my-project"
langsmith project delete --name "old-project"

Trace Commands

# List traces
langsmith trace list
langsmith trace list --limit 50 --last-n-minutes 60
langsmith trace list --error                      # Only errored traces
langsmith trace list --full --include-metadata    # With full content

# Get specific trace
langsmith trace get <trace-id>

# Export traces
langsmith trace export -o traces.json
langsmith trace export --format pretty -o traces.txt

Run Commands (individual spans)

# List runs
langsmith run list
langsmith run list --run-type llm
langsmith run list --name "OpenAI Call"
langsmith run list --project "my-project"

# Get specific run
langsmith run get <run-id>

# Export
langsmith run export -o runs.json

Thread Commands (conversations)

langsmith thread list
langsmith thread list --project "my-project"
langsmith thread get <thread-id>

Dataset Commands

# List datasets
langsmith dataset list

# Create dataset
langsmith dataset create --name "My Dataset"

# Export dataset
langsmith dataset export --name "My Dataset" -o dataset.json

# Import dataset
langsmith dataset import --name "My Dataset" -i dataset.json

Evaluator Commands

langsmith evaluator list
langsmith evaluator upload

Experiment Commands

langsmith experiment list
langsmith experiment list --dataset "My Dataset"
langsmith experiment get <experiment-id>

Output Formats

langsmith trace list                    # JSON (default)
langsmith trace list --format pretty    # Human-readable table
langsmith trace export -o output.json   # Write JSON to file

References

• CLI Docs
• langsmith-cli GitHub

Intake packets and route-outs

Packet chooser

Use one primary packet per run.

Surface ladder

• Quick trace coverage → provider wrappers + a few high-value decorators
• Named workflow spans → traceable() / decorators around application functions
• Scoped audit visibility → manual tracing context or targeted trace blocks
• Pre-ship confidence → dataset + evaluator + experiment packet
• Human-review flow → annotation queue + feedback criteria + reviewer ownership
• Cross-service lineage → explicit header / metadata propagation
• Audit/export workflow → CLI list/get/export flows

Required fields to preserve

• project/workspace/environment
• run_id
• trace_id
• tags and metadata fields that support later filtering
• user/session/request identifiers when review or analytics need them

Quick route-outs

• Generic telemetry, dashboards, alerts, or SLOs → monitoring-observability
• Existing-log root-cause work → log-analysis
• Reproduction and code-path diagnosis → debugging
• Deployment or rollout choreography → deployment-automation
• Exported experiment/KPI interpretation → data-analysis
• Runtime policy, approvals, or guardrails → relevant security/policy skill

Smell test

If your next paragraph is mostly about service-health dashboards, incident response, deployment steps, or generic product metrics instead of LangSmith packets, this skill has already done its job.

LangSmith Modes and Routing

Core rule

Start by deciding which LangSmith workflow is failing or needed. Do not start with API snippets.

Modes

1. Debugging traces

Use when the question is “what happened?” or “why did this LLM/app/agent run fail?”

Focus on:

• whether trace coverage includes the app span plus nested LLM/tool/retrieval calls
• project / environment / tag / metadata consistency
• wrapper vs decorator vs manual trace choice
• explicit flush behavior for short-lived jobs

Return:

• minimum instrumentation packet
• metadata/tag standards
• first trace query to verify success

2. Offline evals

Use when a prompt/model/app change needs a regression gate before release.

Focus on:

• dataset shape and coverage
• deterministic vs LLM-as-judge vs pairwise evaluation
• experiment naming and baseline/candidate comparison
• concurrency/cost bounds

Return:

• dataset packet
• evaluator packet
• experiment packet

3. Online feedback and review

Use when traces exist but nobody trusts output quality in production.

Focus on:

• feedback keys and scales
• annotation queue and rubric design
• reviewer ownership and cadence
• run ID / trace ID preservation

Return:

• queue/rubric packet
• required labels
• review flow

4. Prompt registry and experiment comparison

Use when prompt/version choice is the bottleneck.

Focus on:

• whether Prompt Hub should own prompts at all
• version pinning and experiment comparison
• Git/config alternative if LangSmith registry is not the right fit

Return:

• registry decision
• comparison packet
• route-out note if Git/app config should stay canonical

5. Multi-service / agent propagation

Use when one user request fans out across services, workers, tools, or agent hops.

Focus on:

• header/context propagation
• consistent metadata across spans
• what LangSmith should trace versus what generic observability should cover

Return:

• propagation packet
• required headers / IDs
• route-out to generic observability if needed

6. Audit / gap review

Use when LangSmith is installed but confidence is still low.

Focus on:

• missing instrumentation coverage
• noisy or useless metadata
• missing evals or weak review schema
• missing prompt/version clarity
• missing flush or run-ID preservation

Return:

• severity-ranked gap list
• next ratchet

Route-outs

• Generic dashboards / alerts / SLOs / telemetry ownership → monitoring-observability
• Existing-log root-cause triage → log-analysis
• Code-level reproduction / bug isolation → debugging
• Runtime guardrails / approval logic → policy/security skill
• Rollout / deployment orchestration → deployment-automation
• Exported experiment/KPI interpretation → data-analysis

Minimum evidence packet

For any LangSmith task, try to leave the user with:

• selected mode
• chosen instrumentation surface
• required env/config
• required metadata / IDs / tags
• eval / feedback / queue design if applicable
• verification step
• route-outs

LangSmith Python SDK Reference

Source: https://docs.langchain.com/langsmith | SDK: langsmith v0.7.17+

Install: pip install -U langsmith

Environment Variables

@traceable Decorator

Full signature:

from langsmith import traceable

@traceable(
    run_type: Literal['tool', 'chain', 'llm', 'retriever', 'embedding', 'prompt', 'parser'] = 'chain',
    name: str | None = None,            # Defaults to function name
    metadata: dict | None = None,
    tags: list[str] | None = None,
    client: Client | None = None,
    reduce_fn: Callable | None = None,  # For streaming/generator outputs
    project_name: str | None = None,
    process_inputs: Callable | None = None,   # Custom input serializer
    process_outputs: Callable | None = None,  # Custom output serializer
)
def my_function(...):
    ...

trace() Context Manager

from langsmith import trace

inputs = {"question": "What is 2+2?"}
with trace(name="My Trace", inputs=inputs) as run:
    result = my_pipeline(**inputs)
    run.outputs = {"answer": result}
    trace_id = run.id

tracing_context() — Selective Tracing

import langsmith as ls

# Enable for this block only
with ls.tracing_context(enabled=True, project_name="test"):
    result = chain.invoke(...)

# Disable despite env var
with ls.tracing_context(enabled=False):
    result = chain.invoke(...)

# Use custom client
with ls.tracing_context(client=my_client, project_name="isolated"):
    result = chain.invoke(...)

wrap_openai() and Other Wrappers

from langsmith.wrappers import wrap_openai, wrap_anthropic
from openai import OpenAI
import anthropic

openai_client = wrap_openai(OpenAI())
anthropic_client = wrap_anthropic(anthropic.Anthropic())
# wrap_gemini() also available

Client — Full API

from langsmith import Client
client = Client(api_key="...", api_url="https://api.smith.langchain.com")

Datasets

# Create
dataset = client.create_dataset("name", description="...")

# Add examples
client.create_examples(
    dataset_id=dataset.id,
    examples=[{"inputs": {...}, "outputs": {...}}]
)

# List, clone, delete
for ds in client.list_datasets(): ...
client.clone_public_dataset("https://smith.langchain.com/public/...")
client.delete_dataset(dataset_id=dataset.id)

Evaluation

# Synchronous
results = client.evaluate(
    target_fn,           # Callable: dict -> dict
    data="dataset-name", # str name, UUID, or iterator
    evaluators=[eval_fn],
    experiment_prefix="v1",
    description="...",
    max_concurrency=4
)

# Async
results = await client.aevaluate(target_fn, data="...", evaluators=[...])

# Evaluate existing experiment runs
results = client.evaluate_existing(runs, evaluators=[...])

# Pairwise comparison
results = client.evaluate_comparative("exp-name", data="...",
    candidate=fn_v2, baseline=fn_v1)

Evaluator Function Signatures

# With reference outputs (offline)
def evaluator(inputs: dict, outputs: dict, reference_outputs: dict) -> dict | bool:
    return {"key": "correctness", "score": 1, "comment": "..."}
    # Or: return outputs["answer"] == reference_outputs["answer"]

# Without reference outputs (online)
def online_evaluator(inputs: dict, outputs: dict) -> dict:
    return {"key": "quality", "score": 0.8}

Feedback

# Attach to a run
client.create_feedback(
    key="thumbs_up",
    score=1,
    run_id=run_id,
    trace_id=trace_id,
    comment="Accurate"
)

# Feedback config
client.create_feedback_config("accuracy", feedback_config={
    "type": "continuous", "min": 0, "max": 1
})
client.create_feedback_config("label", feedback_config={
    "type": "categorical",
    "categories": [{"value": 1, "label": "Pass"}, {"value": 0, "label": "Fail"}]
})
client.create_feedback_config("notes", feedback_config={"type": "freeform"})

# List, update, delete
for cfg in client.list_feedback_configs(): print(cfg.feedback_key)
client.update_feedback_config("accuracy", is_lower_score_better=True)
client.delete_feedback_config("accuracy")

Annotation Queues

queue = client.create_annotation_queue(
    name="Review Queue",
    description="...",
    rubric_instructions="Score carefully.",
    rubric_items=[{
        "feedback_key": "accuracy",
        "description": "How accurate?",
        "score_descriptions": {"0": "Wrong", "1": "Correct"},
        "is_required": True,
    }]
)

Prompt Hub

from langchain_core.prompts import ChatPromptTemplate

# Push
prompt = ChatPromptTemplate([("system", "..."), ("user", "{q}")])
client.push_prompt("my-prompt", object=prompt)

# Pull (latest or specific version)
p = client.pull_prompt("my-prompt")
p = client.pull_prompt("my-prompt:commit-abc123")

Projects

client.create_project("my-project")
client.delete_project(project_name="old-project")
for proj in client.list_projects(): print(proj.name)

Utility Functions

from langsmith.run_helpers import get_current_run_tree
from langchain_core.tracers.langchain import wait_for_all_tracers

# Get current run (inside @traceable)
@traceable
def my_func():
    rt = get_current_run_tree()
    headers = rt.to_headers()   # For distributed tracing
    return headers

# Wait for async trace flush (important in scripts)
wait_for_all_tracers()

LangChain Integration

# Auto-trace: just set env vars
import os
os.environ["LANGSMITH_TRACING"] = "true"

from langchain_openai import ChatOpenAI
chain = ChatOpenAI()  # All invocations automatically traced

# Add metadata at invocation time
chain.invoke(
    {"input": "..."},
    {
        "tags": ["production"],
        "metadata": {"version": "2.0"},
        "run_name": "MyChain",
        "run_id": str(uuid.uuid4()),
    }
)

Distributed Tracing

import langsmith
from langsmith.run_helpers import get_current_run_tree

@langsmith.traceable
def parent(inputs):
    rt = get_current_run_tree()
    headers = rt.to_headers()
    # Pass headers to child microservice

@langsmith.traceable
def child(x, headers):
    with langsmith.tracing_context(parent=headers):
        return do_work(x)

openevals — Prebuilt Evaluators

pip install -U openevals

from openevals.llm import create_llm_as_judge
from openevals.prompts import CORRECTNESS_PROMPT, CONCISENESS_PROMPT

judge = create_llm_as_judge(
    prompt=CORRECTNESS_PROMPT,
    model="openai:o3-mini",
    feedback_key="correctness",
)
result = judge(inputs=inputs, outputs=outputs, reference_outputs=ref)

Available prompts: CORRECTNESS_PROMPT, CONCISENESS_PROMPT, HELPFULNESS_PROMPT, HARMFULNESS_PROMPT

References

• Official Docs
• SDK GitHub
• ReadTheDocs API Reference
• openevals GitHub

LangSmith TypeScript/JavaScript SDK Reference

Source: https://docs.langchain.com/langsmith | SDK: langsmith (npm)

Install: npm install langsmith

Environment Variables

export LANGSMITH_API_KEY="lsv2_..."
export LANGSMITH_TRACING=true
export LANGSMITH_PROJECT="my-project"  # optional
export LANGSMITH_ENDPOINT="..."         # self-hosted only

traceable() Wrapper

import { traceable } from "langsmith/traceable";

// Wrap an async function
const pipeline = traceable(
    async (question: string): Promise<string> => {
        const result = await myLLM(question);
        return result;
    },
    {
        name: "My Pipeline",
        // run_type, tags, metadata also available
    }
);

await pipeline("What is LangSmith?");

wrapOpenAI() and Other Wrappers

import { wrapOpenAI } from "langsmith/wrappers";
import { OpenAI } from "openai";

const client = wrapOpenAI(new OpenAI());
// All chat.completions.create() calls are now traced

Client — Core Methods

import { Client } from "langsmith";

const client = new Client({
    apiKey: process.env.LANGSMITH_API_KEY,
    // apiUrl: "https://api.smith.langchain.com"  // self-hosted
});

Datasets

// Create
const dataset = await client.createDataset("My Dataset");

// Add examples
await client.createExamples({
    inputs: [{ question: "Capital of France?" }],
    outputs: [{ answer: "Paris" }],
    datasetId: dataset.id
});

// List
for await (const ds of client.listDatasets()) {
    console.log(ds.name);
}

// Delete
await client.deleteDataset({ datasetId: dataset.id });

Evaluation

import { evaluate } from "langsmith/evaluation";
import type { EvaluationResult } from "langsmith/evaluation";

// Target function
async function myApp(inputs: Record<string, any>) {
    return { answer: await callLLM(inputs.question) };
}

// Evaluator
function exactMatch({ inputs, outputs, referenceOutputs }: {
    inputs: Record<string, any>;
    outputs: Record<string, any>;
    referenceOutputs?: Record<string, any>;
}): EvaluationResult {
    return {
        key: "exact_match",
        score: outputs.answer === referenceOutputs?.answer ? 1 : 0
    };
}

// Run experiment
await evaluate(myApp, {
    data: "My Dataset",
    evaluators: [exactMatch],
    experimentPrefix: "v1-baseline",
    maxConcurrency: 4
});

Feedback

// Attach feedback to a run
await client.createFeedback(
    runId,          // string UUID
    "thumbs_up",    // feedback key
    {
        score: 1,
        comment: "Great response"
    }
);

Feedback Config

// Continuous (0-1 slider)
await client.createFeedbackConfig({
    feedbackKey: "accuracy",
    feedbackConfig: { type: "continuous", min: 0, max: 1 }
});

// Categorical
await client.createFeedbackConfig({
    feedbackKey: "label",
    feedbackConfig: {
        type: "categorical",
        categories: [
            { value: 1, label: "Pass" },
            { value: 0, label: "Fail" }
        ]
    }
});

// Freeform
await client.createFeedbackConfig({
    feedbackKey: "notes",
    feedbackConfig: { type: "freeform" }
});

// List
for await (const cfg of client.listFeedbackConfigs()) {
    console.log(cfg.feedbackKey);
}

Annotation Queues

const queue = await client.createAnnotationQueue({
    name: "Review Queue",
    rubricItems: [
        {
            feedback_key: "accuracy",
            description: "How accurate is the response?",
            is_required: true
        }
    ]
});

Prompt Hub

import { Client } from "langsmith";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { pull } from "langchain/hub";

const client = new Client();

// Push
const prompt = ChatPromptTemplate.fromMessages([
    ["system", "You are a helpful assistant."],
    ["user", "{question}"],
]);
await client.pushPrompt("my-prompt", { object: prompt });

// Pull
const pulled = await pull("my-prompt");
const formatted = await pulled.invoke({ question: "What is LangSmith?" });

Projects

await client.createProject({ projectName: "my-project" });
await client.deleteProject({ projectName: "old-project" });

for await (const proj of client.listProjects()) {
    console.log(proj.name);
}

LangChain (TypeScript) Integration

import { LangChainTracer } from "@langchain/core/tracers/tracer_langchain";
import { getLangchainCallbacks } from "langsmith/langchain";

// Static tracer
const tracer = new LangChainTracer({ projectName: "My Project" });
await chain.invoke({ input: "..." }, { callbacks: [tracer] });

// Dynamic callbacks inside traceable (recommended)
const main = traceable(async (input: string) => {
    const callbacks = await getLangchainCallbacks();
    return await chain.invoke({ input }, { callbacks });
}, { name: "main" });

Wait for Async Trace Flush

import { awaitAllCallbacks } from "@langchain/core/callbacks/promises";

try {
    const result = await llm.invoke("...");
} finally {
    await awaitAllCallbacks();  // Flush all pending traces
}

Custom Run ID for Feedback

import { v4 as uuidv4 } from "uuid";

const runId = uuidv4();
const result = await chain.invoke(
    { input: "..." },
    { runId }
);

// Later attach feedback
await client.createFeedback(runId, "correctness", { score: 1 });

References

• Official Docs
• langsmith npm
• SDK GitHub
• JS Reference Docs