mlflow/skills
9 skills4.8k installs594 starsGitHub
Install
npx skills add https://github.com/mlflow/skillsSkills in this repo
1Searching Mlflow Docssearching-mlflow-docs is a documentation-retrieval skill for the MLflow MLOps platform. It first fetches the official llms.txt index at mlflow.org/docs/latest/llms.txt, identifies the matching page path, then pulls the corresponding Markdown source and returns verbatim code examples for APIs, tracing, experiment tracking, and integrations with LangChain, LangGraph, and OpenAI. Developers reach for searching-mlflow-docs when a coding agent would otherwise hallucinate MLflow APIs or outdated integration patterns. The workflow is read-only against public docs—no local MLflow server required—and triggers on questions like how to use MLflow with a specific framework or where to find an API reference.604installs2Mlflow Onboardingmlflow-onboarding is an MLflow agent skill that determines whether a developer is building GenAI agents and apps or running traditional ML and deep learning, then guides the matching MLflow quickstart path for tracking, projects, experiments, and deployment patterns. The skill accepts an optional experiment ID to refine the use-case branch and triggers on phrases like “get started with MLflow,” “set up MLflow tracking,” or “add MLflow to my project.” It helps engineers choose the right onboarding tutorials before deeper skills such as instrumenting-with-mlflow-tracing or trace analysis. Install from mlflow/skills alongside related observability skills in the same repository. Developers reach for mlflow-onboarding at the start of MLflow adoption when the codebase lacks tracking configuration and the team needs a structured first integration rather than piecemeal documentation searches across unrelated MLflow docs.579installs3Agent Evaluationagent-evaluation is a skill from mlflow/skills for defining MLflow evaluation suites, datasets, and scorers that benchmark LLM agent performance before production promotion. The workflow covers measuring tool-use correctness, answer quality against labeled examples, and regression thresholds that fail CI when agent behavior degrades across versions. Developers reach for agent-evaluation when shipping agentic features that call tools, retrieve context, or chain steps and need reproducible quality gates instead of ad-hoc prompt spot checks. The skill integrates with MLflow's experiment tracking so evaluation runs, metrics, and scorer results are logged and compared across agent model or prompt revisions.573installs4Instrumenting With Mlflow Tracinginstrumenting-with-mlflow-tracing is an MLflow-maintained agent skill in the mlflow/skills repository that walks coding agents through a four-step instrumentation workflow: detect the LLM framework, add the correct autolog call, configure experiment tracking, and verify spans, inputs, outputs, and latency in the MLflow UI. The skill targets Python and TypeScript GenAI apps using OpenAI, Anthropic, LangChain, LangGraph, LiteLLM, and related stacks, with reference patterns for async tracing, multi-thread context propagation, PII redaction, sampling, and production deployment. Developers reach for it when debugging agent chains, comparing prompt experiments, or preparing LLM apps for evaluation. Install via `npx skills add mlflow/skills`; the repo also ships companion skills for trace analysis, chat-session debugging, and trace retrieval. MLflow documents a production-focused mlflow-tracing SDK that reduces install footprint versus the full mlflow package while preserving tracing capabilities.563installs5Analyzing Mlflow Traceanalyzing-mlflow-trace is an MLflow agent skill from mlflow/skills that investigates one trace ID to answer why an AI or agent run failed, returned wrong output, or regressed in production. The workflow fetches the trace, reconstructs the span tree, reviews assessments for quality signals, inspects per-span inputs outputs status and timing including tool errors and retrieval steps, then correlates span names with local source to suggest concrete code or prompt fixes. Developers reach for analyzing-mlflow-trace when they already have MLflow tracing enabled and a specific problematic trace—not when they need fleet-wide evaluation or first-time instrumentation. The mlflow/skills repository ships multiple GenAI observability skills alongside trace analysis, including instrumentation, chat-session debugging, and trace retrieval. Trigger phrases include analyze this trace, debug trace, and why did this trace fail.535installs6Retrieving Mlflow Tracesretrieving-mlflow-traces is an MLflow agent skill for querying GenAI trace data when debugging agent or LLM pipeline failures. It distinguishes single-fetch via mlflow traces get --trace-id from search via mlflow traces search --experiment-id with filter-string queries on trace.status, execution_time_ms, metadata session keys, tags, and span names. The workflow requires running mlflow traces search --help first to match the installed MLflow version, then building filters like trace.status = 'ERROR' or traces slower than 1000 ms. TraceInfo exposes status, execution_time_ms, tags, metadata, and assessments, while spans form a tree of named operations with attributes and timestamps. Developers invoke it when logs reference a trace ID, failed traces need session grouping, or slow spans require performance filtering.532installs7Analyzing Mlflow Sessionanalyzing-mlflow-session is an MLflow skills package workflow for debugging multi-turn chat conversations stored as linked traces under metadata key mlflow.trace.session. The skill teaches a three-step reconstruction: discover input and output schema from the first trace, extract fields across all session traces with mlflow traces search --extract-fields instead of downloading every full trace, then fetch full JSON only for suspect turns via mlflow traces get. It includes bash helpers discover_schema.sh and inspect_turn.sh, explains root-span versus child-span attribute conventions, and warns that piped CLI output can silently truncate above roughly 30KB. Developers reach for analyzing-mlflow-session when users report wrong answers mid-conversation, when assessments flag session-level quality issues, or when turn-level errors propagate across later messages in LangChain, OpenAI, or custom traced agents. The workflow also covers codebase searches for session ID assignment, context window construction, and cross-turn state that commonly causes propagated chat failures.526installs8Querying Mlflow Metricsquerying-mlflow-metrics is an MLflow skills workflow centered on scripts/fetch_metrics.py for querying aggregated trace metrics from an MLflow tracking server. It supports five core metrics—trace_count, latency, input_tokens, output_tokens, and total_tokens—with eight aggregations including COUNT, SUM, AVG, MIN, MAX, P50, P95, and P99, plus optional grouping by trace_name or trace_status and hourly or daily buckets via --time-interval. Developers can query SPANS or ASSESSMENTS views for span_count or assessment_value trends, filter windows with --start-time values like -24h or -7d, and emit table or JSON output with -o json. Reach for querying-mlflow-metrics when reviewing LLM cost spikes, P95 latency regressions, error-rate breakdowns by trace_status, or average assessment scores by evaluator name during ML ops reviews. The bundled references/api_reference.md documents filter syntax when you extend queries beyond the default table output examples in the skill README.520installs9Mlflow Agentmlflow-agent is a dispatcher skill from mlflow/skills with disable-model-invocation set true so it never executes MLflow work directly. Instead it reads developer intent and routes to one of eight sub-skills: instrumenting-with-mlflow-tracing, agent-evaluation, analyze-mlflow-trace, analyze-mlflow-chat-session, retrieving-mlflow-traces, querying-mlflow-metrics, mlflow-onboarding, or searching-mlflow-docs. Triggers include use mlflow, trace my agent, evaluate my agent, and add mlflow to my project. The dispatcher asks at most one clarifying question, handles multi-skill sequences in logical order such as setup then instrument then evaluate, and invokes the matched sub-skill via the Skill tool. Developers reach for mlflow-agent when they know they need MLflow for agent observability but have not chosen which tracing, metrics, or evaluation workflow to run.413installs