Roboflow Inference

For agents — source-of-truth: This skill is authored in `roboflow/computer-vision-skills` and shipped with the Roboflow plugin. If your client has loaded the plugin (you'll see roboflow:<name> skills in your available skills list), use those local skills — they're read fresh from disk every session. The same content served as MCP resources at roboflow://skills/<name>/... is a fallback for clients without the plugin and may lag this repo. Don't call `ReadMcpResourceTool` for `roboflow://skills/...` URIs when a local `roboflow:<name>` skill is available.

Tip: If you're connected to the Roboflow MCP server, prefer its inference tools over raw HTTP — auth is handled. For workflows the headline tool is `workflows_run` (run a saved workflow by workflow_id — the workflow URL slug; workspace is inferred from the API key — see Finding your workspace slug). For single-model calls use models_infer. workflow_specs_run and workflow_specs_validate exist for narrow inline-spec exceptions described under "Authoring Workflows" below.

Inference & Deployment

Prefer Workflows for integration code. Unlike a direct models_infer call, a workflow composes model + visualization + logic in one deployable unit, benefits from server-side optimizations, and keeps active learning and post-processing as a zero-friction addition — similar to the extensibility benefits of a microservice over a point-to-point call. Recommend a workflow by default; use models_infer when the user explicitly asks for direct inference or needs a quick one-off check. See workflows and workflow-templates.

Authoring Workflows — don't paste JSON into chat or scripts. Workflows are authored on the Roboflow platform (storage, versioning, and retrieval go through the platform) and run from code by identifier. Two authoring modes — propose / infer the right one from session context, never silently pick:

>

- Mode A — Agent-driven (MCP, in-session) — for demos, previews, or when the user is committed to in-session "vibe coding". Agent designs the blocks, uses MCP authoring tools to create+save the workflow on the platform during the session (ground the design with workflow_blocks_list / workflow_blocks_get_schema; validate with workflow_specs_validate), then runs it.

- Mode B — Platform-driven (Roboflow app + in-app agent) — better default for non-trivial / sophisticated cases, when the user prefers visual iteration, when they aren't committed to agent-driven authoring this session, or as the fallback when Mode A hits an issue. Agent proposes the block design and hands the user a link to the Workflows builder; the user builds (manually or with the more context-grounded in-app agent), tests in the preview, saves, and shares the workspace + workflow URL slugs back (both visible in the builder URL: app.roboflow.com/<workspace-slug>/workflows/<workflow-slug>).

>

Either mode lands at the same run path: workflows_run (MCP) or client.run_workflow(workspace_name=..., workflow_id=...) (SDK). Inline specs (workflow_specs_run) are an exception, not a default — only when the user explicitly asks for a throwaway run, and validate the spec first with workflow_specs_validate. See workflows "Authoring & Deployment" for the full flow.

For live video (webcam, RTSP, file): the MCP workflows_run tool only handles single static images. For live video, present the user with three options (don't pick one silently): (A) WebRTC → serverless GPU, (B) WebRTC → local inference server, or (C) in-process InferencePipeline. They have different setup costs, dep sizes, and latency characteristics — surface a brief 1-line summary of each and let the user choose. See roboflow://skills/inference/workflows ("Video Stream" section) for full code and the comparison table.

Deployment Options

When to Use Which

• Serverless -- default choice. Zero setup, auto-scales, 20MB upload limit. Use models_infer or workflows_run MCP tools.
• Dedicated -- need consistent latency, large models (Florence 2), or high throughput. Development and production tiers available. Subdomain: <name>.roboflow.cloud.
• Self-hosted -- deploy Roboflow Inference via Docker on your own hardware (Jetson, cloud VMs, RPi). Same API surface as serverless -- just change api_url.
• Batch Processing -- runs a Workflow on uploaded images/videos asynchronously. No real-time requirement. Results delivered as JSON.
• Real-time video (webcam/RTSP/file) -- three deployment options; ask the user which one before writing code:
• (A) Serverless GPU + WebRTC — zero setup, just an API key; per-minute credits, plan-tiered (webrtc-gpu-small/medium/large).
• (B) Local inference server + WebRTC — pip install inference-cli && inference server start (Docker recommended); lowest latency, isolates the heavy CV/model deps inside the server.
• (C) `InferencePipeline` in-process — pip install inference in a venv (prefer uv); runs the workflow loop directly in the user's Python process, no separate server. Heavy deps (torch, opencv, onnxruntime) install locally.

All three have a slower first run (model download / warmup) before subsequent runs hit cached state — tell the user this so they don't think the script is hung.

• See roboflow://skills/inference/workflows ("Video Stream" section) for full code and a comparison table.

MCP Tools for Inference

Local tooling: when MCP isn't enough

For most operations, prefer the Roboflow MCP tools above — they handle auth and need nothing installed locally. Reach for local Python packages only for the gaps: integration scripts (inference-sdk), Batch Processing / Data Staging (inference-cli), the self-hosted server (inference-cli), and asset scripts that need typed Python objects.

See `local-tooling` for what to install for which use case, the recommended uv-based env setup, conda / venv fallbacks, and common pitfalls.

Response Shapes by Task

For canonical response shapes (object detection, classification, segmentation, keypoint) with all fields including class_id, detection_id, class_confidence, see roboflow://skills/api-reference/inference.

Large Response Handling

Instance segmentation `points` arrays are the main culprit for bloated responses. Each detection includes a polygon with potentially hundreds of coordinate pairs. A single image with many detections can return megabytes of JSON.

Mitigation strategies:

1. Use Workflows instead of direct inference -- add a polygon simplification or property extraction block to reduce output before it reaches the client 2. Filter classes -- use class_filter to only return classes you need 3. Raise confidence threshold -- fewer detections = smaller response 4. Post-process -- if consuming raw responses, drop or simplify the points array when you only need bounding boxes 5. Avoid returning raw segmentation results through LLM context -- extract only the fields you need (class counts, bounding boxes) and discard polygon data

Workflow image outputs are a second culprit. Visualization blocks (bounding box, polygon, mask, label, halo, …) emit rendered images as base64-encoded blobs inside the response — a 720p annotated frame is hundreds of KB of JSON-escaped string. When you call workflows_run / workflow_specs_run via MCP, this routinely overflows the tool-result token budget. Decode every image-shaped output ({"type": "base64", "value": "..."}) and write it to disk instead of carrying it through agent context. Don't hard-code field names — the output keys are whatever the workflow author declared via JsonField; iterate output.keys() and shape-check.

Batch Processing

What it is. A Roboflow-managed cloud service that runs a Workflow over a batch of images or videos asynchronously, provisioning the infrastructure for you. "Ideal for asynchronously processing large amounts of data." — Roboflow docs.

Problem it solves. Bulk inference over thousands to millions of files without standing up your own GPUs, queues, or autoscaler. You hand Roboflow a Workflow plus a batch of inputs, pay per job, and get JSON results back when the job finishes.

Pick it when the data is stored (not live), per-file cost matters more than per-file latency, and minutes-to-hours per job is acceptable. Pick something else when you need real-time per-request results (use Serverless or Dedicated) or air-gapped/on-prem processing (use Self-hosted).

Surfaces: Roboflow web UI, inference rf-cloud CLI, and REST API.

Flow

1. Have a saved Workflow in your workspace. 2. Stage inputs as a Data Staging batch (local directory, JSONL of signed URLs, or cloud-storage path on S3 / GCS / Azure). 3. Submit a job referencing the Workflow + input batch; choose CPU or GPU. 4. Monitor — poll job status or register a webhook. 5. Export the output batch as JSON.

CLI

The inference rf-cloud CLI exposes two subcommand groups: data-staging (manage input/output batches) and batch-processing (submit and monitor jobs). Run any command with --help for the full option list.

Minimal end-to-end:

# Stage images
inference rf-cloud data-staging create-batch-of-images \
  --images-dir ./my-images --batch-id my-batch

# Submit
inference rf-cloud batch-processing process-images-with-workflow \
  --workflow-id my-workflow --batch-id my-batch
# -> prints JOB_ID

# Monitor
inference rf-cloud batch-processing show-job-details --job-id JOB_ID

# Export results
inference rf-cloud data-staging export-batch \
  --target-dir ./results --batch-id OUTPUT_BATCH_ID

Data Staging commands — see `batch-staging` for nuances (data sources, JSONL reference format, multipart batches, webhook notifications):

Batch Processing (job) commands — see `batch-jobs` for nuances (compute configuration, workflow parameters, image-output persistence, aggregation format, video FPS, restarts, TRT compilation):

Notes and constraints

• Async only — minutes-to-hours latency depending on volume and hardware. Not for real-time.
• Pricing — per job; GPU jobs cost more than CPU. See `plans-and-pricing`.
• Image-references ingest requires signed URLs from trusted sources; arbitrary public URLs are rejected — stage to a local directory or cloud-storage path instead.

Full reference: Roboflow Batch Processing docs.

Batch Processing Jobs — CLI Nuances

Tip: Run inference rf-cloud batch-processing <command> --help for the canonical option list. This page covers cross-cutting concepts and per-command nuances, not every flag.

A job runs a Workflow (or compiles a model) over an input batch and produces one or more output batches — one per stage. Jobs progress through stages; each stage has tasks. Logs, notifications, and show-job-details give visibility while the job runs. Inputs come from Data Staging.

Compute configuration

Common to process-images-with-workflow, process-videos-with-workflow, and restart-job:

• --machine-type / -mt — cpu or gpu. GPU is faster, costs more credits.
• --workers-per-machine — workers per machine. More workers = better resource utilization, but risk of OOM for memory-heavy Workflows. Prefer this over --machine-size.
• --machine-size / -ms — (deprecated; removal scheduled in inference 0.42.0) legacy enum xs / s / m / l / xl, mapped to workers-per-machine of 8 / 4 / 2 / 1 / 1 respectively.
• --max-runtime-seconds — hard cap on processing duration; the job aborts when exceeded.
• --max-parallel-tasks — concurrency ceiling across the job.

Agent guidance. Defaults (CPU, platform-default workers, no runtime cap) are fine for demos and small smoke tests. For real workloads, ask the user — --machine-type, --workers-per-machine, --max-runtime-seconds, and --max-parallel-tasks materially affect cost and throughput, and the right values depend on Workflow heaviness, batch size, and budget. Same for --job-id / --job-name: invent for demos, ask for real workloads.

Workflow parameterization

• --workflow-id / -w — the saved Workflow's identifier (required).
• --workflow-params — path to a JSON file with parameters; use this when the Workflow takes typed parameters (numbers, lists, nested objects) that are awkward to pass on the CLI.
• --image-input-name — name of the image-input parameter in the Workflow. Only needed when the Workflow's input isn't named image.

Image outputs persistence

Workflows often produce annotated images. By default these images are not persisted:

• --save-image-outputs — persist all image outputs.
• --image-outputs-to-save <name> — persist only the named outputs (repeat the flag per output).

Persisted images land in a separate part of the output batch — fetch with data-staging export-batch --part-name <name>.

Result aggregation

--aggregation-format chooses the per-task results format: csv or jsonl. JSONL is the typical default; CSV is convenient when results are flat scalars.

Video-only options

process-videos-with-workflow adds:

• --max-video-fps — subsample to this FPS before inference. Lower = faster processing, less granular tracking.

Notifications

--notifications-url registers a webhook for job-state events (started, stage transitions, completed, failed). Recommended for long-running jobs instead of polling show-job-details.

Inference backend

--inference-backend / -ib selects between old-inference (legacy) and inference-models (newer). Defaults to the platform default — only override on Roboflow's guidance.

Commands

list-jobs

Workspace-wide job list. --max-pages / -p paginates.

show-job-details

Snapshot of a single job: planned vs current stage, output batch IDs per stage, terminal state, restart history. Polling target.

Polling for completion

show-job-details prints a Rich table — convenient to read, painful to parse. For automation, use the asset script `bin/poll_batch_job.py`: it calls inference_cli's get_batch_job_metadata in a loop, prints stage transitions and the latest notification message as the job moves through stages, and exits 0 on success / 1 on terminal error / 2 on timeout.

pip install inference-cli
export ROBOFLOW_API_KEY=...
skills/inference/bin/poll_batch_job.py JOB_ID        # direct; relative to skill dir: bin/poll_batch_job.py
# or: python skills/inference/bin/poll_batch_job.py JOB_ID
# optional: --interval 30 --max-wait 7200

For long-running jobs, prefer --notifications-url at job submission and let the webhook deliver state changes — polling burns API calls.

process-images-with-workflow

Submit an image-batch job. Required: --batch-id / -b, --workflow-id / -w. Common optional: compute (--machine-type, --workers-per-machine), aggregation (--aggregation-format), image outputs (--save-image-outputs + --image-outputs-to-save), --workflow-params, --notifications-url. Prints the new job ID on success.

inference rf-cloud batch-processing process-images-with-workflow \
  -b my-batch -w my-workflow \
  --machine-type gpu --workers-per-machine 2 \
  --aggregation-format jsonl \
  --save-image-outputs --image-outputs-to-save annotated

process-videos-with-workflow

Same shape as the images variant, plus --max-video-fps.

inference rf-cloud batch-processing process-videos-with-workflow \
  -b my-videos -w my-workflow --max-video-fps 5 -mt gpu

fetch-logs

Pull job logs. --log-severity {info,warning,error} filters by severity; --output-file / -o writes JSONL to disk.

abort-job

Terminate a running job by ID. Idempotent; safe to call on already-finished jobs.

restart-job

Restart a failed job, reusing the original workflow + input batch. Optional overrides (compute only): --machine-type, --workers-per-machine, --max-runtime-seconds, --max-parallel-tasks.

trt-compile

Pre-compile a Roboflow model to TensorRT for one or more NVIDIA devices. Required: --model-id / -m, --device / -d (repeat per device — supported: nvidia-l4, nvidia-t4, nvidia-l40s). Optional: --notifications-url, --job-name. The output is a job — track with show-job-details.

inference rf-cloud batch-processing trt-compile \
  -m my-project/3 -d nvidia-l4 -d nvidia-t4

REST reference

OpenAPI spec: <https://openapi.gitbook.com/o/-MABnPmH89-NX2aB8mq4/spec/batch-processing-rest-cli.yaml>. Endpoints used by these commands:

• POST /batch-processing/v1/external/{workspace}/jobs/{job_id} — start a job. Body includes type, jobInput.batchId, computeConfiguration.machineType, processingSpecification.workflowId (and the persistence / aggregation / video-FPS fields above).
• GET /batch-processing/v1/external/{workspace}/jobs/{job_id} — current job status (pending / processing / completed / failed) with progress.
• GET /batch-processing/v1/external/{workspace}/jobs/{job_id}/stages — stage list with output batch IDs per stage.
• GET /batch-processing/v1/external/{workspace}/jobs/{job_id}/stages/{stage_id}/tasks — per-stage tasks (paginated).

All endpoints accept the API key as the api_key query parameter.

Data Staging — CLI Nuances

Tip: Run inference rf-cloud data-staging <command> --help for the canonical option list. This page covers cross-cutting concepts and per-command nuances, not every flag.

Data Staging is the storage layer for Batch Processing. You ingest input files into staging batches, run jobs that produce output batches, and export the outputs back out.

Batches

A batch holds either images or videos. Two flavors:

• Input batches — what you create with create-batch-of-images / create-batch-of-videos. Job inputs.
• Output batches — produced automatically by jobs (one per job stage). Often multipart (e.g. predictions JSONL + persisted image outputs as separate parts).

--batch-id must be lowercase letters with - / _. Batches expire (see expiryDate from show-batch-details).

Agent guidance. For one-off demo or ad-hoc runs, invent a sensible --batch-id and --batch-name yourself (e.g. a short descriptor plus a timestamp). For real / production workloads, ask the user — they likely have naming conventions (project prefix, date, ticket ID) and/or need the batch findable in the web UI later.

Data sources for input batches

Selected via --data-source / -ds:

Cloud storage paths — --bucket-path accepts S3/GCS/Azure URLs with optional glob:

s3://my-bucket/run-2026-05-06/**/*.jpg
gs://my-bucket/inbox/
az://my-container/frames/

References file — JSONL (one JSON object per line). Each line needs name and url:

{"name": "frame_001.jpg", "url": "https://signed.example.com/..."}
{"name": "frame_002.jpg", "url": "https://signed.example.com/..."}

URLs must be signed URLs from trusted domains (e.g. presigned S3 / GCS). Public URLs from arbitrary domains are rejected at ingest. For arbitrary public URLs, download them to a local directory first and use local-directory.

Webhook notifications (URL ingests only)

--notifications-url registers a webhook for ingest events. --notification-category filters categories (ingest-status, files-status); pass the flag multiple times to select more than one. Both options are only meaningful for `references-file` and `cloud-storage` ingests — not for local uploads. Use --ingest-id to label the ingest so events are correlatable.

Multipart batches

Output batches from jobs can have multiple parts (e.g. predictions JSONL, persisted image outputs). Filter operations to a part with --part-name / -pn on list-batch-content and export-batch. list-batch-content without --part-name returns metadata across all parts.

Commands

list-batches

Workspace-wide batch list. --pages / -p and --page-size paginate.

create-batch-of-images / create-batch-of-videos

Create an input batch. Pick the data source and matching option:

• local-directory (default) → --images-dir / --videos-dir
• cloud-storage → --bucket-path
• references-file → --references (JSONL of {name, url})

Optional: --batch-name (display name), --ingest-id (label the ingest), --notifications-url (URL ingests only).

# Local
inference rf-cloud data-staging create-batch-of-images \
  -b my-batch -i ./images

# Cloud storage
inference rf-cloud data-staging create-batch-of-images \
  -b my-batch -ds cloud-storage -bp 's3://my-bucket/run/**/*.jpg'

# Signed-URL JSONL
inference rf-cloud data-staging create-batch-of-images \
  -b my-batch -ds references-file -r ./refs.jsonl

show-batch-details

One-shot metadata: type, content type, created/expiry dates.

list-batch-content

Lists per-file metadata (download URLs, names, parts). --part-name / -pn filters multipart batches; --limit / -l caps the number of entries; --output-file / -o writes JSONL to disk instead of printing.

list-ingest-details

Per-shard status for the ingest of a batch. Use this when a references-file or cloud-storage ingest is partially failing — it surfaces which shards are stuck or errored.

export-batch

Downloads files from a batch to --target-dir / -t. --part-name / -pn filters parts; --override-existing re-downloads files that already exist locally (default: skip).

inference rf-cloud data-staging export-batch \
  -b OUTPUT_BATCH_ID -t ./results --part-name predictions

REST reference

OpenAPI spec: <https://openapi.gitbook.com/o/-MABnPmH89-NX2aB8mq4/spec/batch-processing-rest-cli.yaml>. Endpoints used by these commands:

• POST /data-staging/v1/external/{workspace}/batches/{batch_id}/upload/image — single-image upload (multipart form; recommended up to ~5,000 images)
• POST /data-staging/v1/external/{workspace}/batches/{batch_id}/bulk-upload/image-files — request a signed URL for a .tar upload (high-volume)
• POST /data-staging/v1/external/{workspace}/batches/{batch_id}/upload/video — request a signed URL to upload a video
• POST /data-staging/v1/external/{workspace}/batches/{batch_id}/bulk-upload/image-references — register a list of signed URLs (powers references-file)
• GET /data-staging/v1/external/{workspace}/batches/{batch_id}/count — file count
• GET /data-staging/v1/external/{workspace}/batches/{batch_id}/shards — shard statuses (paginated)
• GET /data-staging/v1/external/{workspace}/batches/{batch_id}/parts — list parts of a multipart batch
• GET /data-staging/v1/external/{workspace}/batches/{batch_id}/list — list download URLs (filterable by partName, paginated)

All endpoints accept the API key as the api_key query parameter.

Local Tooling — When MCP Isn't Enough

Tip: Prefer the Roboflow MCP server for anything it covers — it handles auth and needs nothing installed locally. This page is for the gaps where you need local Python tooling.

When you need local tooling

Reach for local Python packages only when an operation has no MCP equivalent.

Confirm the target env with the user first

Before installing anything, ask which Python env to install into — the user owns that decision. Don't assume uv, conda, or venv based on what looks cleanest; an agent that silently picks an env can pollute the system Python, break a pre-existing project env, or duplicate dependencies the user already has.

Ask (or infer from explicit prior signals like CLAUDE.md/memory) and always(!) confirm:

• Is there an existing project env to reuse? (look for .venv/, pyproject.toml + uv.lock, an active conda env, an environment.yml)
• If creating a new env, which manager — uv, conda, or stdlib venv?
• Which Python version, if not already pinned?

Only after the user has confirmed should you run pip install / uv pip install / conda install. The recommendations below are defaults to propose, not defaults to act on.

Recommended setup: uv

uv is the recommended Python package + env manager for these tools. Fast, reproducible, no boilerplate. Default to uv unless the user explicitly asks for something else.

# Install uv (one-time; see docs.astral.sh/uv for alternative installers)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create a project env with the right Python version
uv venv --python 3.12

# Install only what you need
uv pip install inference-sdk         # integration scripts
uv pip install inference-cli         # batch processing / data staging / self-hosted server
# (or both)

# Run anything via uv (auto-uses the project .venv)
uv run python script.py
uv run inference rf-cloud data-staging list-batches

For anything beyond a one-off, declare deps in pyproject.toml:

[project]
name = "my-roboflow-integration"
requires-python = ">=3.10"
dependencies = [
    "inference-sdk>=1.2",
    # "inference-cli>=1.2",   # add if you also need rf-cloud / self-hosted server
]

Then uv sync reproduces the env on any machine; uv.lock pins exact versions.

Why uv

• Fast (seconds, not minutes) — single Rust-based resolver/installer.
• Deterministic resolution and lock file (uv.lock).
• Manages Python versions per project — no system-Python coupling.
• One command (uv run) handles both Python and console scripts.
• Same UX on Linux, macOS, Windows.

Escape hatch: conda / venv

Use these only when uv is genuinely off-limits (existing pipeline, org policy, user explicitly insists). They work — they're just slower and more error-prone.

# conda
conda create -n my-roboflow python=3.12 -y
conda activate my-roboflow
pip install inference-sdk inference-cli

# stdlib venv
python3.12 -m venv .venv
source .venv/bin/activate
pip install inference-sdk inference-cli

In all cases: never install into the system Python or a pre-existing shared env. Both packages have heavy dependency trees that conflict easily with unrelated projects.

Self-hosted server: Docker prerequisite

inference server start / stop / status are thin wrappers around Docker — they pull and run the roboflow/roboflow-inference-server-* image. Before invoking any of them, verify Docker is installed and the daemon is running. Don't just pip install inference-cli and assume the server will come up.

Quick check:

docker info --format '{{.ServerVersion}}'   # exits 0 only if the daemon is reachable

Note: inference server start itself will detect the running container, agent should just check if Docker daemon is running.

If it fails:

• Not installed — point the user to Docker Desktop (macOS/Windows) or the appropriate docker-ce package (Linux). Don't install Docker silently; it's a system-level dependency the user owns.
• Installed but daemon not running — on macOS/Windows, ask the user to launch Docker Desktop; on Linux, sudo systemctl start docker (or rootless equivalent - but ALWAYS ASK FOR CONFIRMATION OF SUCH OPERATION). Wait for the daemon before retrying.
• Permission denied on the socket (Linux) — user is not in the docker group; surface the error rather than sudo-ing around it.

Common pitfalls

• API key not picked up — both packages read ROBOFLOW_API_KEY from the environment of the process running them. Export it in that shell, or use an .env loader explicitly. Never hardcode it in source; keep .env in .gitignore.
• Same code, different deployment — api_url is the only thing that changes between Serverless (https://serverless.roboflow.com), Dedicated (https://<name>.roboflow.cloud), and Self-hosted (http://localhost:9001).

Workflow Templates

Source-of-truth note: This page ships with the Roboflow plugin. If your client has the plugin loaded, prefer the local skill (roboflow:inference) over fetching roboflow://skills/inference/workflow-templates via ReadMcpResourceTool — the MCP resources are a fallback for non-plugin clients and may lag the source repo.

Quick-reference catalog of built-in workflow templates. Use these as starting points when building workflows via the editor or workflow_specs_run.

Detection & Counting

SAHI (Small Object Detection)

Segmentation & Masking

Multi-Model Pipelines

Conditional Branching

Zone Analytics

Privacy & Safety

Data Collection

Parallel Comparison

Block Index

All blocks referenced across templates:

Workflows

Source-of-truth note: This page ships with the Roboflow plugin. If your client has the plugin loaded, prefer the local skill (roboflow:inference) over fetching roboflow://skills/inference/workflows via ReadMcpResourceTool — the MCP resources are a fallback for non-plugin clients and may lag the source repo.

Tip: If you're connected to the Roboflow MCP server, prefer `workflows_run` (saved workflow by workflow_id — the workflow URL slug; workspace is inferred from the API key — see Finding your workspace slug) over raw HTTP. workflow_specs_run is an inline-spec escape hatch for explicit one-offs only; see "Authoring & Deployment" below.

What Are Workflows

Composable, multi-step computer vision pipelines built in a visual editor. Chain models, logic, visualization, and integrations into a single deployable unit.

Why Workflows over direct inference:

• Chain multiple models (detect -> crop -> classify)
• Add post-processing (counting, filtering, tracking)
• Visualize results (bounding boxes, labels, masks)
• Integrate external services (notifications, storage)
• Single deploy for the entire pipeline

Key Concepts

Block Reference

Use workflow_blocks_list to get the live catalog. Below are the ~30 most common blocks grouped by category.

Two block identifiers — don't mix them up. The "Workflow type" column below is the value you put in the type field of a workflow JSON spec (e.g. roboflow_core/sam3@v3). workflow_blocks_get_schema does not accept this — it requires the long manifest key returned by workflow_blocks_list (e.g. inference__core__workflows__core_steps__models__foundation__segment_anything3__v3__BlockManifest). Same block, two different identifiers for two different APIs.

Models

Visualization

Transformation

Analytics (Video)

Video analytics pattern: Model -> Byte Tracker -> Analytics block. Always insert a tracker between model and counter/zone.

Logic & Data

Output & Integration

Block Configuration

Key parameters on model blocks:

How Blocks Connect

No explicit edges. Connections are selector strings in step input properties:

• $inputs.{name} -- workflow input
• $steps.{step_name}.{output} -- step output
• $steps.{step_name}.* -- all outputs (used in workflow outputs)

Step names: derive from block type, strip roboflow_core/ and @vX, lowercase with underscores.

Authoring & Deployment

Two ways to author — both end with a saved workflow on the platform

Workflows can be authored two ways. The agent should propose the right one (or infer from prior session signals) — never silently pick. Both paths land at the same place: a workflow saved on the Roboflow platform, identified by its workspace + workflow URL slugs (visible in the builder URL: app.roboflow.com/<workspace-slug>/workflows/<workflow-slug>). Storage, versioning, and retrieval always go through the platform; the run path is the same regardless of how the workflow was authored.

Mode A — Agent-driven (MCP, in-session)

Use when: the session is for a demo or preview, or the user is committed to in-session "vibe coding" and wants the agent to drive the whole authoring loop end-to-end.

How: agent designs the block list, calls Roboflow MCP workflow-authoring tools to create and save the workflow on the platform during the session, and runs it. Ground the design in real types: use workflow_blocks_list / workflow_blocks_get_schema for manifest types and required props, and workflow_specs_validate to catch shape errors before saving.

Mode B — Platform-driven (Roboflow app + in-app agent)

Use when: the workflow is non-trivial, the user prefers to see and adjust it visually, the user isn't committed to agent-driven authoring this session, or Mode A has hit an issue and a fallback is needed. This is also the better default for sophisticated cases — the builder's in-app agent is more tightly context-grounded than a generic external agent.

How: agent proposes the block design (block list, how they connect, expected inputs/outputs) and hands the user a direct link to the Workflows builder (Workflows tab → "Create a Workflow"). The user builds manually or works with the in-app workflow agent, tests via the built-in preview, saves, and shares the workspace + workflow URL slugs back (both visible in the builder URL: app.roboflow.com/<workspace-slug>/workflows/<workflow-slug>). The agent then runs it from code.

Running a saved workflow

Whichever mode authored it, run it the same way:

• MCP: workflows_run with workflow_id (and optional parameters). The workspace is inferred from the API key — there is no separate workspace argument. (See Finding your workspace slug if you need to know which workspace a key resolves to.)
• SDK: client.run_workflow(workspace_name=..., workflow_id=..., images=..., parameters=...).

`workflow_id` is the workflow URL slug, not the document ID. workflows_create / workflows_get return both — only the slug is recognised at run time. Find it in the url field of those responses, or in the browser address bar at https://app.roboflow.com/<workspace-slug>/workflows/<workflow-slug>.

`workspace_name` (SDK only) is your workspace URL slug — the path segment immediately after app.roboflow.com/ when you're signed into the dashboard. If you only have an API key and need the slug programmatically, see Finding your workspace slug below.

Finding your workspace slug

If you only have an API key — no dashboard access in front of you — hit the root REST endpoint to resolve the workspace it belongs to:

curl -s "https://api.roboflow.com/?api_key=YOUR_API_KEY"

The response includes a workspace field whose url (slug) is what you pass as workspace_name in the SDK and what appears in app.roboflow.com/<workspace-slug>/.... Useful for: SDK scripts started from just a key, verifying which workspace a key belongs to, and CI environments where no human ever opens the dashboard.

Inline specs — exception only

workflow_specs_run (MCP) and client.run_workflow(specification=...) (SDK) accept an inline spec without ever touching the platform. Reserve for narrow cases the user has explicitly authorised: throwaway one-offs or programmatic generation where saving is genuinely impractical. Validate first with workflow_specs_validate. Default for everything else: author via Mode A or Mode B, then call workflows_run.

Deploy

SDK Code

import base64
from pathlib import Path
from inference_sdk import InferenceHTTPClient

client = InferenceHTTPClient(
    api_url="https://serverless.roboflow.com",
    api_key="API_KEY",
)

result = client.run_workflow(
    workspace_name="my-workspace",          # URL slug; resolve from an API key alone via `curl https://api.roboflow.com/?api_key=...` (see "Finding your workspace slug")
    workflow_id="my-workflow",               # workflow URL slug, NOT the document id
    images={"image": "path/to/image.jpg"},   # local path, base64, or https:// URL — http:// is rejected
    parameters={                             # see "Runtime parameters" below
        "classes": ["cat", "dog"],
        "confidence": 0.35,
    },
)

# `result` is a list with one entry per input image. Each entry is a dict
# keyed by the workflow's output names — whatever the author declared via
# `JsonField` in the spec. Don't hard-code names; read `output.keys()`.
output = result[0]

# Image-shaped outputs come back as base64-encoded blobs and can be hundreds
# of KB each. Decode and write to disk rather than carrying them in memory
# or through agent context.
for name, value in output.items():
    if isinstance(value, dict) and value.get("type") == "base64":
        Path(f"{name}.jpg").write_bytes(base64.b64decode(value["value"]))

Runtime parameters. The parameters dict at run time must match the WorkflowParameter declarations inside the workflow spec — same names, same types, and (for selectors) kinds the consuming block accepts. Anything not declared in the spec is ignored; a wrong type fails at runtime. If you didn't author the workflow, fetch its definition with workflows_get and read the inputs block to see what parameters it exposes.

Image input constraint. URL inputs must be https:// — plain http:// is rejected with a RuntimeInputError. Local paths and base64 strings work without that restriction.

Video Stream (Webcam / RTSP / File) — WebRTC

For real-time video — webcam, RTSP, or file — use the WebRTC API in inference_sdk.webrtc. It opens a peer connection to either the serverless GPU fleet or a local inference server, streams frames up, and returns annotated frames + workflow data over the data channel.

Reasoning trap to avoid: the MCP workflows_run tool only handles single static images. That's expected — it does not mean you should fall back to InferencePipeline as the default for live video. WebRTC (Variants A or B below) is the default because it isolates the heavy CV/model deps inside an inference server. InferencePipeline (Variant C) is a lower-level option for in-process Python embedding — pick it only when in-process execution is a specific requirement.

Always ask the user which variant before generating the script. There are three: (A) serverless WebRTC, (B) local-server WebRTC, (C) in-process InferencePipeline. Surface a brief 1-line summary of each from the comparison table below — don't silently pick one. Variants A and B differ only in api_url and a few StreamConfig fields; Variant C is structurally different (in-process Python, no network).

Tell the user: first run is slower than subsequent runs for any of these — there's a model load / warmup step before the first frame is processed. Subsequent runs reuse cached state. Useful to mention so they don't think the script is hung when nothing happens for a few seconds.

Variant A — Serverless GPU (hosted)

Best for: zero infra setup, bursty/occasional use, getting started.

import cv2
from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import WebcamSource, StreamConfig, VideoMetadata

client = InferenceHTTPClient.init(
    api_url="https://serverless.roboflow.com",
    api_key="YOUR_API_KEY",
)

source = WebcamSource(resolution=(1280, 720))  # or RTSPSource / FileSource

config = StreamConfig(
    stream_output=["annotated_image"],          # frames returned to client
    data_output=["active_count", "new_instances", "event_log", "complete_events"],  # workflow outputs over datachannel
    processing_timeout=3600,                    # seconds; session ends after this
    requested_plan="webrtc-gpu-medium",         # webrtc-gpu-small | webrtc-gpu-medium | webrtc-gpu-large
    requested_region="us",                      # us | eu | ap
)

session = client.webrtc.stream(
    source=source,
    workflow="my-workflow-id",
    workspace="my-workspace",
    image_input="image",                        # name of the image input on the workflow
    config=config,
)

@session.on_frame
def show_frame(frame, metadata: VideoMetadata):
    cv2.imshow("Workflow Output", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        session.close()

@session.on_data()
def on_data(data: dict, metadata: VideoMetadata):
    print(f"Frame {metadata.frame_id}: {data}")

session.run()  # blocks until the session closes

Pick data_output to match the workflow output names the user's workflow exposes (e.g. counts, event logs, tracking ids). Look these up via workflows_get if unsure.

Variant B — Local inference server

Best for: predictable latency on local GPU/CPU.

Prereqs — start the inference server first:

pip install inference-cli
inference server start     # serves inference server on http://localhost:9001

Then the same script with two changes: api_url points at localhost, and StreamConfig drops requested_plan / requested_region (those are serverless-only).

import cv2
from inference_sdk import InferenceHTTPClient
from inference_sdk.webrtc import WebcamSource, StreamConfig, VideoMetadata

client = InferenceHTTPClient.init(
    api_url="http://localhost:9001",
    api_key="YOUR_API_KEY",
)

source = WebcamSource(resolution=(1280, 720))

config = StreamConfig(
    stream_output=["annotated_image"],
    data_output=["active_count", "new_instances", "event_log", "complete_events"],
    processing_timeout=3600,
)

session = client.webrtc.stream(
    source=source,
    workflow="my-workflow-id",
    workspace="my-workspace",
    image_input="image",
    config=config,
)

@session.on_frame
def show_frame(frame, metadata: VideoMetadata):
    cv2.imshow("Workflow Output", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        session.close()

@session.on_data()
def on_data(data: dict, metadata: VideoMetadata):
    print(f"Frame {metadata.frame_id}: {data}")

session.run()

Variant C — InferencePipeline (in-process Python)

Best for: embedding the workflow loop directly in your own Python application, single-host setups where standing up a separate inference server (Variant B) is overkill, or environments where you can't expose an HTTP/WebRTC port. The pipeline runs in-process — predictions are delivered to a callback in your script, not over a network channel.

Trade-off: requires installing the full inference Python package locally, which pulls in heavy CV/model dependencies (torch, opencv, onnxruntime, model files, etc.). On GPU especially, this is the most fragile install of the three options. If you can run the inference server (Variant B), prefer that — same model deps, but isolated. Reach for InferencePipeline only when in-process execution is a hard requirement and the user has confirmed they're OK installing the inference package locally.

Setup — prefer `uv` for the venv, and pin Python to 3.12:

# Preferred: uv is much faster and keeps deps in an isolated venv.
# Pin Python to 3.12 — newer versions (3.13+) often lack onnxruntime wheels,
# so `uv pip install inference` will fail on a default-Python (3.13+) venv.
uv venv --python 3.12
uv pip install inference                  # CPU; or `inference-gpu` for CUDA

# Without uv, fall back to stdlib venv (slower, but works):
# python3.12 -m venv .venv && .venv/bin/pip install inference

First run is slower than subsequent runs — inference downloads model weights and warms up the ONNX runtime on first invocation. Tell the user this so they don't think the script is hung. Subsequent runs reuse cached weights.

import cv2
from inference import InferencePipeline


def on_prediction(result, video_frame):
    # `result` is a dict of workflow outputs — pull whatever your workflow exposes.
    if (annotated := result.get("annotated_image")) is not None:
        cv2.imshow("Workflow Output", annotated.numpy_image)
        cv2.waitKey(1)
    if (count := result.get("active_count")) is not None:
        print(f"active_count = {count}")


pipeline = InferencePipeline.init_with_workflow(
    api_key="YOUR_API_KEY",
    workspace_name="my-workspace",
    workflow_id="my-workflow-id",
    video_reference=0,                # 0 = default webcam; can be RTSP URL or file path
    image_input_name="image",         # name of the workflow's image input (default "image")
    on_prediction=on_prediction,
)

pipeline.start()
pipeline.join()                       # blocks until video source ends or pipeline terminates

Choosing among the three

How to present this to the user: surface all three with a one-line summary of each (use the "Best for" row), then let the user pick. Don't default-pick; the right answer depends on whether they have Docker, want zero local install, or want the script to be self-contained.

When to Use Workflows vs Direct Inference

Recommend Workflows for integration code, production apps, multi-step pipelines, video, post-processing, and active learning. Workflows compose model + logic + visualization in one call, benefit from server-side optimizations, and keep active learning and other blocks as a zero-friction addition without changing your API surface. Use `models_infer` for quick checks or when the user explicitly prefers direct inference.

MCP Tools