Vss Generate Video Calibration

Purpose

Run AutoMagicCalib end-to-end on local files, RTSP streams, or the bundled sample dataset and (when needed) deploy the AMC microservice.

Instructions

Follow the routing tables and step-by-step workflows below. Each section that ends in workflow, quick start, or flow is intended to be executed top-to-bottom. Detailed reference material lives in references/; load only the reference needed for the selected input mode.

Examples

Worked end-to-end examples are kept under evals/ (each *.json manifest contains a runnable scenario) and inline in the per-workflow curl blocks below. Run a Tier-3 evaluation with nv-base validate <this-skill-dir> --agent-eval to replay them.

Limitations

• Requires the matching VSS profile / microservice to be deployed and reachable from the caller.
• NGC-hosted models and NIMs may be subject to rate-limits, GPU memory requirements, and license restrictions.
• Concurrency, GPU memory, and storage limits depend on the host hardware and the profile's compose file.

Troubleshooting

• Error: REST call returns connection refused. Cause: target microservice not running. Solution: probe /docs or /health; redeploy via vss-deploy-profile or the matching vss-deploy-* skill.
• Error: HTTP 401/403 from NGC pulls. Cause: missing/expired NGC_CLI_API_KEY. Solution: docker login nvcr.io and re-export the key before retrying.
• Error: container OOM or model fails to load. Cause: insufficient GPU memory for the selected profile. Solution: switch to a smaller variant or free GPUs via docker compose down.

VSS Generate Video Calibration

Run AutoMagicCalib over one of three input sources and drive the calibration through the microservice REST API. The input-resolution work differs per source; everything from verify_project onward is identical and lives in this file. Pick the right input-mode reference and pair it with the Shared Calibration Tail below.

Shared helper references are loaded only when needed:

• Read `references/common-steps.md` when a mode reference needs the shared create_project, video-upload, or handoff snippets.
• Read `references/calibration-tail.md` when you need the reusable Python implementation of the verify → calibrate → poll → results tail.

Input Routing

Match the user's request to a mode, then load that mode's reference for input collection, mode-specific API calls, and the full Python script.

Disambiguation rule: if the user is asking to launch / deploy / set up AMC (no calibration verb) → deploy. If they provide RTSP URLs → rtsp. If they mention local files / a videos directory → videos. If they ask to verify install or test the bundled sample → sample-dataset. Combined intents (e.g. "launch AMC and calibrate my videos") → walk deploy first, then the calibration mode. When ambiguous, ask via AskUserQuestion.

Prerequisites (shared across calibration modes)

• AMC microservice + UI running. If not, walk `references/deploy-auto-calibration-service.md` first.
• Microservice reachable at http://<HOST_IP>:${VSS_AUTO_CALIBRATION_PORT:-8010}/v1/ready → {"code":0,...}.
• Projects directory writable by the container user. If you didn't just deploy (so Step 5 of the deploy reference hasn't run), confirm the write test in `references/deploy-auto-calibration-service.md` § Step 5 — otherwise the first create_project returns [Errno 13] Permission denied.
• Python 3 with requests installed (each input-mode reference includes a self-healing venv fallback for direct runs).

Mode-specific prerequisites (VIOS for rtsp, sample zip for sample-dataset) live in the respective references.

Shared Calibration Tail

The verify → calibrate → poll → results sequence is identical regardless of input mode. After the mode-specific reference has uploaded videos / ingested RTSP clips / uploaded the bundled sample, run this tail. Use `references/calibration-tail.md` for the shared Python snippet.

Step A — Verify Project

POST /v1/verify_project/<project_id>

Response: {"project_state": "READY"} — must be READY before calibrating. If not READY, re-check that videos + alignment + layout are present (either via API or via UI manual alignment).

Step B — Start Calibration

Confirm the plan before calibrating. Whether the settings file and detector were auto-detected or asked, present a short summary and confirm via AskUserQuestion before the POST /calibrate. The resolved values are the defaults, so confirming is one click — but the user can switch the detector or skip an auto-detected settings file. Summarize:

• Detector — resnet or transformer (the value to be sent).
• Calibration settings — the file being applied (path), or default parameters (with the option to tune them in the UI first — see below).
• Optional overrides — ground-truth zip and focal lengths, if any.

The sample-dataset install-check run uses a fixed resnet and can proceed without this confirmation.

POST /v1/calibrate/<project_id>
Content-Type: application/json

{"detector_type": "resnet"}   # or "transformer"

detector_type is a separate /calibrate parameter — not consumed by /v1/config/<id>. If the user provided a calibration settings file, parse it for "detector" / "detector_type" and use that value. If the file doesn't specify one, the default (resnet) is the value shown in the confirmation above — the user can switch it there before calibrating. If there's no settings file at all, ask the user via AskUserQuestion:

• resnet — default, fast.
• transformer — slower, better under heavy occlusion.

UI Step 3 (Parameters) does NOT cover detector choice; never assume the user picked one in the UI.

Also when there's no settings file, ask whether to tune the calibration parameters first (AskUserQuestion):

• Proceed with the default parameters — well-suited to typical warehouse scenes; recommended unless the user has specific tuning in mind.
• Adjust parameters in the UI first — open the project, go to Step 3: Parameters, change values, and click Save; then continue.

Wait for the user's choice — and, if they choose to tune, for them to confirm they've Saved — before calling /calibrate.

Step C — Poll for Completion

GET /v1/get_project_info/<project_id>

Poll every 10 s. project_info.project_state:

When calibration starts, surface the project ID, the UI URL (http://<HOST_IP>:${VSS_AUTO_CALIBRATION_UI_PORT:-5000}), and the log endpoint so the user can watch progress while the run proceeds. During RUNNING, emit a progress line at least once a minute with elapsed time so a long run doesn't look stalled. On ERROR, fetch and show the last lines of GET /v1/amc/calibrate/<id>/log before stopping. Live logs can also be streamed via GET /v1/calibrate/<project_id>/log/<type>/stream.

Typical time: 10–60 min (your-own videos), 10–30 min (bundled sample).

Step D — Results

GET /v1/get_project_info/<project_id>                    # project state
GET /v1/result/<project_id>/evaluation_statistics        # only if GT uploaded
GET /v1/result/<project_id>/overlay_image                # visual overlay (PNG)
GET /v1/amc/calibrate/<project_id>/log                   # calibration log

Evaluation response includes Average L2 distance(m) and Average reprojection error 0(px). Evaluation metrics are produced only when a ground-truth `GT.zip` was uploaded — a missing evaluation_statistics result is normal otherwise and is not the end of result reporting.

After COMPLETED, always give the user a way to review the result for that exact project, regardless of whether metrics exist:

• UI — http://<HOST_IP>:${VSS_AUTO_CALIBRATION_UI_PORT:-5000}; open the project, then the Results page to view the overlay.
• Overlay image on disk — ${VSS_APPS_DIR}/services/auto-calibration/projects/project_<id>/output/multi_view_results/BA_output/results_ba_scaled_world/overlay_img_*.png (single-camera projects use output/single_view_results/cam_00/verification_map_overlay.png).
• Project files — ${VSS_APPS_DIR}/services/auto-calibration/projects/project_<id>/.

Step E — VGGT Refinement

After the AMC run completes, always check vggt_state in project info. VGGT model staging is optional during setup and must not block the AMC result, but post-AMC handling follows the state:

• If vggt_state == "READY" and the user explicitly requested VGGT refinement or staged VGGT during this setup flow, run VGGT refinement without asking again.
• If vggt_state == "READY" but VGGT was already staged before this request and the user has not asked for VGGT-refined output, ask via AskUserQuestion whether to run refinement before starting it.
• If VGGT is not ready, skip refinement and mention that VGGT refinement is available after staging the model (see `references/deploy-auto-calibration-service.md` Step 2).

POST /v1/vggt/calibrate/<project_id>
GET  /v1/get_project_info/<project_id>                    # poll vggt_state
GET  /v1/vggt_results/<project_id>/evaluation_statistics  # VGGT metrics

Settings File + Detector Pattern

Optional across all three modes. When the user provides a JSON settings file (typically exported from UI Step 3 Download), POST it verbatim:

POST /v1/config/<project_id>
Content-Type: application/json

<file contents, posted as-is>

The file replaces what the user would otherwise tune in UI Step 3 (rectification, bundle-adjustment, evaluation knobs, detector, …). After a successful POST, also parse the file for "detector" / "detector_type" — if it's "resnet" or "transformer", use that value for the /calibrate call in Step B (detector is a separate API parameter, not consumed by /config).

Non-2xx is surfaced — do not silently fall back. Skip this call entirely if the user chose the UI-fallback path.

UI Fallback Pattern

When alignment / layout files aren't on disk, direct the user to the appropriate AMC UI step:

• Settings missing → "Open UI project <project_id>, go to Step 3: Parameters, tune via the settings dialog (or accept defaults), click Save." Also: before the /calibrate call, ask the user via AskUserQuestion whether to use the resnet or transformer detector — Step 3 doesn't cover detector choice.
• Layout missing → "Open UI project <project_id>, go to Step 2: Video Configuration, upload layout.png only (do NOT re-upload videos — they're already attached via API/RTSP), click Save."
• Alignment missing → "Open UI project <project_id>, go to Step 4: Alignment, either upload alignment_data.json or mark correspondence points on the layout, click Save."

Wait for user confirmation. For alignment/layout, verify on disk before continuing:

# Project state lives under $VSS_APPS_DIR/services/auto-calibration/projects
# (the path bind-mounted into the MS container in
#  deploy/docker/services/auto-calibration/ms/compose.yml).
HOST_PROJECTS="${VSS_APPS_DIR}/services/auto-calibration/projects"

ls "$HOST_PROJECTS/project_<project_id>/manual_adjustment/"
# Expected: alignment_data.json, layout.png

Success Criteria

• project_state == "COMPLETED" after polling.
• If manual alignment was used: ${VSS_APPS_DIR}/services/auto-calibration/projects/project_<id>/manual_adjustment/ contains alignment_data.json + layout.png.
• If GT was uploaded: evaluation returns typical thresholds (Average L2 distance(m) < 1.5, Average reprojection error 0(px) < 5 for your data; < 10 for the bundled sample).
• No ERROR state.

Key Output Files

Under ${VSS_APPS_DIR}/services/auto-calibration/projects/project_<project_id>/:

project_<project_id>/
├── manual_adjustment/
│   ├── alignment_data.json
│   └── layout.png
├── output/
│   ├── single_view_results/cam_XX/
│   │   ├── camInfo_hyper_XX.yaml
│   │   └── trajDump_Stream_0_3d.txt
│   ├── multi_view_results/BA_output/results_ba/
│   │   ├── initial/camInfo_XX.yaml
│   │   └── refined/camInfo_XX.yaml          # ← final calibration
│   └── multi_view_results/BA_output/results_ba_scaled_world/
│       └── overlay_img_XX.png               # ← visual overlay for review
└── calibration.log

Cross-cutting Troubleshooting

Mode-specific issues live in each reference's own troubleshooting table.

For Downstream Skills — MV3DT Export

Downstream consumers (e.g. a Multi-View 3D Tracking skill owned by another team) fetch the MV3DT-format calibration output directly from the microservice. This skill returns the project_id; the downstream skill calls:

GET /v1/result/{project_id}/mv3dt_result?result_type=amc
# Response: application/zip — mv3dt_output.zip containing transforms.yml

For VGGT-refined output (only available if VGGT ran to COMPLETED, see Step E):

GET /v1/result/{project_id}/mv3dt_result?result_type=vggt
# Response: application/zip — vggt_mv3dt_output.zip

Downstream skill flow: 1. Call this skill with the user's inputs; capture the printed project_id. 2. Wait for the skill to return (it polls until COMPLETED internally). 3. GET /v1/result/{project_id}/mv3dt_result?result_type=amc — save the ZIP locally. 4. If VGGT also ran, optionally fetch ?result_type=vggt for the refined MV3DT.

Related Skills

• `vss-manage-video-io-storage` — VIOS API skill; only the rtsp calibration mode depends on VIOS being reachable.

Root README.md "Custom Dataset" and "Calibration Workflow (UI)" sections document input-video guidelines and the UI-driven alternative to this API flow.

bump:1

Evaluation Report

Evaluation of the vss-generate-video-calibration skill before publication through NVSkills-Eval.

This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use.

Evaluation Summary

• Skill: vss-generate-video-calibration
• Evaluation date: 2026-06-10
• NVSkills-Eval profile: external
• Environment: astra-sandbox
• Dataset: 3 evaluation tasks
• Attempts per task: 1
• Pass threshold: 50%
• Overall verdict: PASS

Agents Used

• claude-code
• codex

Metrics Used

Reported benchmark dimensions:

• Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
• Correctness: checks whether the agent follows the expected workflow and produces the correct final output.
• Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
• Effectiveness: checks whether the agent performs measurably better with the skill than without it.
• Efficiency: checks whether the agent uses fewer tokens and avoids redundant work.

Underlying evaluation signals used in this run:

• security (Security): checks for unsafe operations, secret leakage, and unauthorized access.
• skill_execution (Skill Execution): verifies that the agent loaded the expected skill and workflow.
• skill_efficiency (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage.
• accuracy (Accuracy): grades final-answer correctness against the reference answer.
• goal_accuracy (Goal Accuracy): checks whether the overall user task completed successfully.
• behavior_check (Behavior Check): verifies expected behavior steps, including safety expectations.
• token_efficiency (Token Efficiency): compares token usage with and without the skill.

Test Tasks

The benchmark dataset contained 3 evaluation tasks:

• Positive tasks: 3 tasks where the skill was expected to activate.
• Negative tasks: 0 tasks where no skill was expected.
• Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred.

Task composition is derived from the evaluation dataset when possible. Entries with expected_skill set are treated as positive skill-activation cases, while entries with expected_skill: null are treated as negative activation cases.

Results

Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available.

Tier 1: Static Validation Summary

Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 7 total findings.

Top findings:

• MEDIUM QUALITY/quality_correctness: SKILL_SPEC recommended field missing: 'metadata.author' (skills/vss-generate-video-calibration/SKILL.md)
• MEDIUM QUALITY/quality_efficiency: Deeply nested references in common-steps.md (skills/vss-generate-video-calibration/SKILL.md)
• MEDIUM SCHEMA/author_missing: Author not specified in metadata (skills/vss-generate-video-calibration/SKILL.md)
• MEDIUM SECURITY/Unknown (SQP-2): The RTSP capture script sets ssl_verify=False when communicating with the MS endpoint and handles RTSP URLs that may con (references/rtsp.md:227)
• MEDIUM SECURITY/Unknown (SQP-2): The skill automatically executes curl -LsSf https://astral.sh/uv/install.sh | sh without explicit user consent when th (references/sample-dataset.md:132)

Tier 2: Deduplication Summary

Tier 2 validation passed. NVSkills-Eval ran 2 checks and found 0 total findings.

Notable observations:

• Context Deduplication: Collected 7 file(s)
• Inter-Skill Deduplication: Parsed skill 'vss-generate-video-calibration': 182 char description

Publication Recommendation

The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change.

Shared Calibration Tail (Python)

The verify → calibrate → poll → results sequence is identical across all three input modes (videos, RTSP, sample-dataset). The mode-specific references stop after their last upload step and reference this snippet.

Assumes s, BASE_URL, project_id, and DETECTOR_TYPE are already bound from the preceding mode-specific Python.

import os
import time
from urllib.parse import urlparse

# Verify the project before calibration
s.post(f"{BASE_URL}/verify_project/{project_id}").raise_for_status()

# Step B — Start calibration (detector_type is a /calibrate argument; not consumed by /v1/config)
s.post(f"{BASE_URL}/calibrate/{project_id}",
       json={"detector_type": DETECTOR_TYPE}).raise_for_status()

# Surface where to watch progress before the long poll begins.
_host = urlparse(BASE_URL).hostname or "<HOST_IP>"
_ui_port = os.environ.get("VSS_AUTO_CALIBRATION_UI_PORT", "5000")
_root = BASE_URL.rsplit("/v1", 1)[0]
print("[B] Calibration started")
print(f"    Project:  {project_id}")
print(f"    Detector: {DETECTOR_TYPE}")
print(f"    UI:       http://{_host}:{_ui_port}")
print(f"    Logs:     GET {BASE_URL}/amc/calibrate/{project_id}/log   (Swagger UI: {_root}/docs)")

# Step C — Poll until COMPLETED (10–60 min typical). Poll every 10s, and print a
# heartbeat at least once a minute so a long RUNNING state still shows progress.
start, last_state, last_beat = time.time(), "", 0.0
while time.time() - start < 5400:
    info = s.get(f"{BASE_URL}/get_project_info/{project_id}").json()
    st = info["project_info"]["project_state"]
    mins, secs = divmod(int(time.time() - start), 60)
    if st != last_state or time.time() - last_beat >= 60:
        print(f"    [{mins:>3}m {secs:02d}s] {st}", flush=True)
        last_state, last_beat = st, time.time()
    if st == "COMPLETED":
        print(f"[C] Completed in {mins}m {secs:02d}s"); break
    if st == "ERROR":
        # Surface the tail of the calibration log so the failure is actionable.
        try:
            log_lines = s.get(f"{BASE_URL}/amc/calibrate/{project_id}/log").text.splitlines()
            print("    --- last calibration log lines ---")
            for line in log_lines[-20:]:
                print(f"    {line}")
        except Exception:
            pass
        raise RuntimeError(f"Calibration ERROR — full log: GET {BASE_URL}/amc/calibrate/{project_id}/log")
    time.sleep(10)
else:
    raise RuntimeError(
        f"Calibration still running after {int((time.time() - start) // 60)} min — "
        f"inspect GET {BASE_URL}/amc/calibrate/{project_id}/log or the UI at http://{_host}:{_ui_port}"
    )

# Step D — Results + review
print("\n=== Calibration complete ===")
print(f"Project:  {project_id}")
print(f"Detector: {DETECTOR_TYPE}")

# Evaluation metrics are only produced when a ground-truth GT.zip was uploaded.
# A missing result here is normal (no GT) — it is not the end of result reporting.
r = s.get(f"{BASE_URL}/result/{project_id}/evaluation_statistics")
_stats = r.json().get("statistics") if r.status_code == 200 else None
if _stats:
    print("Evaluation metrics:")
    for k, v in _stats.items():
        print(f"    {k}: {v}")
else:
    print("Evaluation metrics: not available — upload a ground-truth GT.zip before calibrating to get L2 / reprojection metrics.")

# Always point to the visual overlay so the user can validate calibration quality.
_projects_dir = os.environ.get(
    "PROJECTS_DIR",
    f"{os.environ.get('VSS_APPS_DIR', '<VSS_APPS_DIR>')}/services/auto-calibration/projects",
)
_proj_path = f"{_projects_dir}/project_{project_id}"
print("\nReview the calibration:")
print(f"    UI:            http://{_host}:{_ui_port}  — open project {project_id}, then the Results page to view the overlay")
print(f"    Overlay image: {_proj_path}/output/multi_view_results/BA_output/results_ba_scaled_world/overlay_img_*.png")
print(f"    Project files: {_proj_path}")

See SKILL.md Shared Calibration Tail for the REST equivalents and the meaning of each project state.

Common Calibration Steps

Shared snippets used by all three input-mode references (videos, RTSP, sample-dataset). Each mode reference points here for the common create_project, upload_videos, and handoff steps to avoid duplication.

Create project

POST /v1/create_project
Content-Type: application/x-www-form-urlencoded

project_name=<your_project_name>

Save the returned project_id — every subsequent endpoint takes it.

Python equivalent:

r = s.post(f"{BASE_URL}/create_project", data={"project_name": PROJECT_NAME})
r.raise_for_status()
project_id = r.json()["project_id"]

Upload videos

Videos must be named cam_00.mp4, cam_01.mp4, … contiguous, no gaps.

POST /v1/upload_video_files/<project_id>
Content-Type: multipart/form-data

files=@cam_00.mp4
files=@cam_01.mp4
...

For the sample-dataset mode the bundled zip already contains the cameras in the correct order; the mode reference just feeds them into this endpoint.

Hand off to the shared calibration tail

Once the mode-specific reference has uploaded videos, alignment, and layout (plus any optional GT zip / focal lengths), continue with the Shared Calibration Tail — see SKILL.md Step A onward for the REST flow and `calibration-tail.md` for the shared Python snippet (verify → calibrate → poll → results).

Deploy auto-calibration service

Use this reference when the user wants to deploy AMC (launch the microservice + UI). The parent skill (`../SKILL.md (see ../SKILL.md`)) routes here on triggers like "launch AMC" / "deploy auto-calibration" / "set up auto-magic-calib".

Deploys the vss-auto-calibration service — AMC microservice + web UI from pre-built release images. The compose tree lives at `deploy/docker/services/auto-calibration/`, and AMC is enabled only by auto_calib, bp_wh_auto_calib_2d, bp_wh_auto_calib_3d, or bp_wh_auto_calib_mv3dt. AMC is a service inside the warehouse-operations industry profile — env vars live in `deploy/docker/industry-profiles/warehouse-operations/.env`.

What's different from base VSS

• Standalone microservice — not part of the VSS agent stack. AMC ships its own MS + UI containers. The VSS agent, NIMs, VST, RTVI, etc. are not brought up by this skill — only the AMC backend and its web UI.
• AMC piggybacks on the `warehouse-operations` industry profile. Warehouse calibration profiles load the env automatically; running auto_calib standalone requires the same env to be present.
• Default ports: MS at ${VSS_AUTO_CALIBRATION_PORT} (default 8010); UI at ${VSS_AUTO_CALIBRATION_UI_PORT} (default 5000). MS uses network_mode: host, so 8010 is also the host port.
• VIOS auto-wired. When deployed with a warehouse calibration profile, VIOS_BASE_URL is fetched from ${VST_INTERNAL_URL}. No manual VIOS config needed if VST is running in the same compose.
• Optional VGGT model. AMC works without VGGT, but model-based refinement needs vggt_1B_commercial.pt at $VSS_DATA_DIR/auto-calib/vggt/ (the path the MS container mounts read-only). Skip this step unless the user explicitly wants VGGT.

What gets deployed

Image references are illustrative. The compose files above are the source of truth for the exact image repo and tag — they may differ by release. Don't pull a hand-typed path; read the resolved path from docker compose config / resolved.yml (Step 3) and let docker compose up pull it.

Env recipe

Set in `deploy/docker/industry-profiles/warehouse-operations/.env` (the values below are the in-repo defaults):

Deployment flow

Standard compose-centric workflow: env overrides → docker compose --env-file .env config dry-run → review → docker compose up.

Step 1 — NGC login

AMC pulls its images from the vss-core namespace on nvcr.io (the exact org — e.g. nvidia for published releases — is whatever the compose files in the table above reference). The user's NGC key must have access to that namespace.

The credential source is the NGC_CLI_API_KEY environment variable in the current shell/env file. Confirm it is set before logging in (this prints only SET/NOT SET, never the key):

echo "NGC_CLI_API_KEY: $([ -n "${NGC_CLI_API_KEY}" ] && echo SET || echo 'NOT SET')"
echo "$NGC_CLI_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin

Credential handling. State that you are logging in with NGC_CLI_API_KEY from the current env before you run it. If the var is NOT SET, or docker login fails / a pull later returns 401, stop and ask the user for a valid NGC key (AskUserQuestion) — do not reuse an NGC key seen earlier in the conversation unless the user explicitly confirms reusing it. Never echo, log, or persist the raw key (--password-stdin keeps it out of argv and shell history; keep it out of any file you write).

Step 2 — (Optional) Stage the VGGT model

Skip this step unless the user explicitly asks for VGGT-refined output.

2a. Accept the model license (one-time, manual): visit https://huggingface.co/facebook/VGGT-1B-Commercial and click "Agree and access repository".

2b. Get a HuggingFace read token: https://huggingface.co/settings/tokens (starts with hf_…). Ask the user for it via AskUserQuestion.

2c. Download into the VSS data dir:

# venv with huggingface_hub
python3 -m venv /tmp/amc-hf-venv
/tmp/amc-hf-venv/bin/pip install --quiet huggingface_hub

# Download into the path the MS expects to mount
mkdir -p "${VSS_DATA_DIR}/auto-calib/vggt"
/tmp/amc-hf-venv/bin/hf download facebook/VGGT-1B-Commercial \
  --local-dir "${VSS_DATA_DIR}/auto-calib/vggt/" \
  --token <HF_TOKEN>

# Verify
ls -lh "${VSS_DATA_DIR}/auto-calib/vggt/vggt_1B_commercial.pt"
# Should show ~4.7GB file

Do not log or echo the HuggingFace token value. Pass it inline to the hf CLI via --token rather than storing it on disk or in shell history.

Step 2b — If VIOS is already running, confirm VIOS_BASE_URL

AMC's RTSP-stream calibration path calls VIOS over ${VIOS_BASE_URL}. The warehouse-operations .env defaults to VIOS_BASE_URL=${VST_INTERNAL_URL} (which resolves to http://${HOST_IP}:${VST_PORT}). That default is correct when VIOS/VST comes up as part of the same compose stack — but if you're standing AMC up next to a pre-existing VIOS (separate image / different namespace / from another compose project), the default may point at nothing.

Detect first:

docker ps --format '{{.Names}}\t{{.Image}}' | grep -E "vst|vios|sensor-ms" || echo "(no VIOS detected)"

If VIOS is running, before the dry-run in Step 3:

1. Confirm VIOS_BASE_URL is set in industry-profiles/warehouse-operations/.env. If the file leaves it commented out or empty, set it explicitly:

   grep -E "^VIOS_BASE_URL=" deploy/docker/industry-profiles/warehouse-operations/.env \
     || echo 'VIOS_BASE_URL=${VST_INTERNAL_URL}' >> deploy/docker/industry-profiles/warehouse-operations/.env

2. Verify the URL actually points at the running VIOS. The default assumes ${HOST_IP}:${VST_PORT} — check both:

   grep -E "^(HOST_IP|VST_PORT)=" deploy/docker/industry-profiles/warehouse-operations/.env
   docker port vst-ingress 2>/dev/null   # or whichever VIOS ingress container is running
   curl -sf -o /dev/null -w "%{http_code}\n" "http://${HOST_IP}:${VST_PORT}/"

If VST_PORT doesn't match what the existing VIOS ingress publishes, override either VST_PORT or set VIOS_BASE_URL directly to the running URL (e.g. VIOS_BASE_URL=http://10.34.3.199:30888) — don't leave the variable form pointing at the wrong port.

If you don't intend to use AMC's RTSP-stream path (only sample-dataset or pre-recorded videos), VIOS_BASE_URL is unused and you can skip this step.

Step 3 — Enable an auto-calibration compose profile and deploy

Pick the profile that matches the intent, then run the same generate → confirm image access → bring up sequence:

cd deploy/docker
export COMPOSE_PROFILES=auto_calib   # or a bp_wh_auto_calib_* profile from the table above

# 1. Generate the resolved compose for review
docker compose --env-file industry-profiles/warehouse-operations/.env config > resolved.yml
# Review resolved.yml — confirm vss-auto-calibration and vss-auto-calibration-ui appear

# 2. Confirm the NGC key can access the AMC images before bringing the stack up.
#    Image references are read from the resolved compose, so this tracks the release tag automatically.
AMC_IMAGES=$(docker compose --env-file industry-profiles/warehouse-operations/.env config --images | grep auto-calibration)
if [ -z "$AMC_IMAGES" ]; then
  echo "No auto-calibration images found in the resolved compose."
  echo "Confirm COMPOSE_PROFILES is exported and the chosen profile includes vss-auto-calibration before continuing."
  exit 1
fi
for img in $AMC_IMAGES; do
  echo "Checking access: $img"
  if ! docker pull "$img"; then
    echo
    echo "NGC login succeeded, but this key does not have access to the required AutoMagicCalib image:"
    echo "  $img"
    echo "Provide an NGC key with access to the vss-core namespace, then retry."
    exit 1
  fi
done

# 3. Bring up the stack (images are already local from the access check)
docker compose --env-file industry-profiles/warehouse-operations/.env up -d

Step 4 — Verify

PORT=$(grep ^VSS_AUTO_CALIBRATION_PORT deploy/docker/industry-profiles/warehouse-operations/.env | cut -d= -f2)
UI_PORT=$(grep ^VSS_AUTO_CALIBRATION_UI_PORT deploy/docker/industry-profiles/warehouse-operations/.env | cut -d= -f2)
HOST_IP=$(hostname -I | awk '{print $1}')

# MS ready (cold pulls can take a bit after compose returns)
READY_URL="http://localhost:${PORT:-8010}/v1/ready"
for i in $(seq 1 24); do
  if curl -sf "$READY_URL"; then
    break
  fi
  echo "Waiting for AMC microservice readiness... ($i/24)"
  sleep 5
done
curl -sf "$READY_URL"
# Expected: {"code":0,"message":"VSS Auto Calibration Microservice is ready"}

# UI reachable
curl -s -o /dev/null -w "%{http_code}\n" "http://localhost:${UI_PORT:-5000}/"
# Expected: 200

# Containers healthy
docker ps --filter name=vss-auto-calibration --format '{{.Names}}\t{{.Status}}'
# Expected:
#   vss-auto-calibration       Up XXs (healthy)
#   vss-auto-calibration-ui    Up XXs

echo "Microservice: http://${HOST_IP}:${PORT:-8010}"
echo "Web UI:       http://${HOST_IP}:${UI_PORT:-5000}"

Step 5 — Confirm the projects directory is writable

AMC stores each project under a host directory bind-mounted into the container. The container runs as UID 1000 (triton-server), so that directory must be writable by UID 1000 — otherwise the first POST /v1/create_project returns [Errno 13] Permission denied. On a fresh checkout this almost always fails the first time: a git clone leaves services/auto-calibration/projects owned by the cloning user (whatever their UID is), and unless that happens to be UID 1000 the container can't write. Treat the write-test failing as the expected default on a new host and apply the scoped ACL below. Check this once after the stack is healthy, before any calibration run:

PROJECTS_DIR="${VSS_APPS_DIR}/services/auto-calibration/projects"
mkdir -p "$PROJECTS_DIR"

# Write-test as the container user, against the actual bind-mount destination
# inside the container (resolved from `docker inspect`, so this is robust to the
# container's WorkingDir and to release path changes — do NOT hardcode it).
DEST=$(docker inspect vss-auto-calibration \
  --format '{{range .Mounts}}{{println .Source .Destination}}{{end}}' \
  | awk -v s="$PROJECTS_DIR" '$1==s {print $2}')
if [ -z "$DEST" ]; then
  WORKDIR=$(docker inspect vss-auto-calibration --format "{{.Config.WorkingDir}}")
  if [ -z "$WORKDIR" ]; then
    echo "ERROR: could not determine container working directory — is vss-auto-calibration running?" >&2
    exit 1
  fi
  DEST="${WORKDIR%/}/projects"
fi

docker exec vss-auto-calibration sh -c \
  "touch '$DEST/.amc_write_test' && rm -f '$DEST/.amc_write_test'" \
  && echo "projects directory is writable" \
  || echo "projects directory is not writable by the container — apply the ACL below"

The projects dir mounts under the container working directory. Use the mount destination resolved from docker inspect; a workdir-relative path with the working-directory basename prefixed can resolve to a nested non-existent path and mask a permission failure.

If the write test does not succeed (the common case on a fresh host — see above), grant the container user access with a narrow ACL (ask the user before changing host permissions). This adds write access for UID 1000 only and leaves existing ownership intact:

setfacl -m u:1000:rwx "$PROJECTS_DIR"     # prefix with sudo if the directory is root-owned

Re-run the write test to confirm, then continue. Prefer this scoped ACL over a broad chmod -R 777.

Success criteria

• curl http://localhost:${VSS_AUTO_CALIBRATION_PORT:-8010}/v1/ready returns {"code":0,"message":"VSS Auto Calibration Microservice is ready"}.
• vss-auto-calibration reports (healthy) in docker ps (the compose healthcheck has a generous start_period: 1000s).
• Web UI at http://<HOST_IP>:${VSS_AUTO_CALIBRATION_UI_PORT:-5000} renders the AutoMagicCalib interface.

Key Output

• Microservice: http://<HOST_IP>:${VSS_AUTO_CALIBRATION_PORT:-8010} — Swagger at /docs
• Web UI: http://<HOST_IP>:${VSS_AUTO_CALIBRATION_UI_PORT:-5000} — project management, file upload, calibration, results
• Project state: ${VSS_APPS_DIR}/services/auto-calibration/projects/ (bind-mounted into the MS container)
• VGGT model (optional): ${VSS_DATA_DIR}/auto-calib/vggt/vggt_1B_commercial.pt (read-only mount)

Troubleshooting

Stopping the services

cd deploy/docker
COMPOSE_PROFILES=auto_calib docker compose --env-file industry-profiles/warehouse-operations/.env down

# Or, if running as part of warehouse auto-calibration, tear down that profile:
COMPOSE_PROFILES=bp_wh_auto_calib_2d docker compose --env-file industry-profiles/warehouse-operations/.env down

What comes next

Once the AMC stack is up and healthy, the parent skill picks one of three calibration modes based on what the user has:

• sample-dataset.md — bundled sample (recommended first run; sanity-checks the install).
• videos.md — pre-recorded MP4s.
• rtsp.md — live RTSP streams (requires VIOS).

Agent behavior: if the user's original prompt asked to both deploy AND calibrate (e.g. "launch AMC and test the sample dataset", "set up auto-magic-calib and calibrate my videos at /data/videos/"), proceed immediately to one of the calibration-mode references once the readiness probe passes — don't stop at "deploy succeeded" and wait for re-prompt. If the user only asked to deploy, surface the URLs (MS + UI) and the three calibration options above so they can pick.

vss-generate-video-calibration — RTSP Mode (live camera streams)

Load this reference when the user wants to calibrate from live RTSP camera streams. The MS records each stream through VIOS, ingests the recorded clips, then runs the normal AMC calibration. Skip to the Shared Calibration Tail in SKILL.md once the RTSP capture + ingest is done and alignment/layout are uploaded.

For local MP4s instead, see videos.md. For verifying the install with the bundled sample, see sample-dataset.md.

Mode-specific Prerequisites

• VIOS is running and reachable — Step 1 probes the default port 30888 first, then falls back to VIOS_BASE_URL from the MS container env / compose files. If none work, point the user at the `vss-manage-video-io-storage (see ../../vss-manage-video-io-storage/SKILL.md`) skill, else ask them to deploy VIOS.
• MS knows where VIOS is — VIOS_BASE_URL is set in the MS container's environment (auto-wired from ${VST_INTERNAL_URL} under bp_wh_* blueprints; otherwise set explicitly in `deploy/docker/industry-profiles/warehouse-operations/.env`). Required at runtime; Step 1 only uses the 30888 probe to detect whether VIOS is up locally.
• RTSP URLs reachable from the VIOS host — verify with the user before starting capture.

The shared prerequisites (AMC microservice, Python+requests) come from the SKILL.md Prerequisites section.

Step 1 — Verify VIOS Is Reachable

Confirm VIOS is up before doing anything else. Probe in this order — stop at the first hit:

export REPO_ROOT=$(git rev-parse --show-toplevel)
VIOS_BASE_URL=""

# 1a. Default port probe — standard VIOS one-click deployment listens on 30888.
if curl -sf http://localhost:30888/vst/api/v1/sensor/list >/dev/null 2>&1; then
  # Use HOST_IP from the warehouse-operations env (not `localhost` — the MS container can't reach host `localhost`)
  ENV_FILE="$REPO_ROOT/deploy/docker/industry-profiles/warehouse-operations/.env"
  HOST_IP=$(grep ^HOST_IP "$ENV_FILE" 2>/dev/null | cut -d= -f2)
  VIOS_BASE_URL="http://${HOST_IP:-localhost}:30888"
  echo "VIOS detected at default port: $VIOS_BASE_URL"
fi

# 1b. Fallback — VIOS_BASE_URL from the running MS container env (authoritative if set).
if [ -z "$VIOS_BASE_URL" ]; then
  VIOS_BASE_URL=$(docker exec vss-auto-calibration printenv VIOS_BASE_URL 2>/dev/null)
fi

# 1c. Fallback — grep compose files (useful when MS isn't running yet).
if [ -z "$VIOS_BASE_URL" ]; then
  VIOS_BASE_URL=$(grep -hR '^\s*-\?\s*VIOS_BASE_URL' "$REPO_ROOT/deploy/docker/services/auto-calibration" 2>/dev/null \
    | sed -E 's/.*VIOS_BASE_URL[=:]\s*//' | head -1)
fi

# 1d. Confirm VIOS actually responds at whatever URL we resolved.
if [ -n "$VIOS_BASE_URL" ]; then
  curl -sf "${VIOS_BASE_URL}/vst/api/v1/sensor/list" >/dev/null \
    && echo "VIOS up at $VIOS_BASE_URL" \
    || { echo "VIOS_BASE_URL=$VIOS_BASE_URL is set but not responding"; VIOS_BASE_URL=""; }
fi

If VIOS still can't be reached (all four checks failed): 1. Look for a VIOS setup skill: ls skills/ | grep -i vios. If found (e.g. vios), invoke it. 2. Otherwise, ask the user to deploy VIOS and share the base URL via AskUserQuestion. Do not proceed until ${VIOS_BASE_URL}/vst/api/v1/sensor/list returns 200.

If VIOS was detected on 30888 but the MS container env is unset, the capture endpoint will still return 503 until VIOS_BASE_URL is set. The cleanest fix is to deploy alongside a bp_wh_* blueprint (which auto-wires it from ${VST_INTERNAL_URL}). Otherwise set VIOS_BASE_URL=http://<HOST_IP>:30888 in `deploy/docker/industry-profiles/warehouse-operations/.env` and re-run docker compose --env-file ... up -d from deploy/docker/.

Step 2 — Collect Inputs From User

Required

1. RTSP URLs — one per camera. Example: rtsp://<nvstreamer-host>:31556/stream/cam_00.mp4 or rtsp://user:pass@<cam-ip>:554/stream. 2. Camera names — short label per stream (used as camera_name in the capture request), e.g. cam_00, cam_01, … 3. Duration seconds — recording window (minimum 60). Pick at least 2–3 min of moving objects for decent calibration. 4. Microservice URL — e.g. http://<HOST_IP>:8010. 5. Project name — short descriptive string.

Anchor-File Pattern (ask config first, then auto-scan its dir for alignment)

Because there's no local videos directory to anchor the scan, ask the user for the calibration settings file first. Then look in its directory for alignment/layout:

UI fallback details for any of these live in SKILL.md UI Fallback Pattern.

Required when no calibration-settings file is provided

6. Detector type — see SKILL.md § Step B — Start Calibration for the choice and the AskUserQuestion fallback. 7. Parameter tuning — also ask whether to proceed with the default calibration parameters or tune them in the UI (Step 3: Parameters) first. See SKILL.md § Step B for the exact prompt.

Optional

7. `sensor_id` per stream — if VIOS already has the sensor registered, pass the ID to skip re-registration. Leave null and the MS auto-registers via VIOS. 8. Ground truth zip (GT.zip) and focal lengths — same options as the videos mode.

VGGT refinement is handled after AMC completes by SKILL.md Step E. Do not collect a separate RTSP-mode VGGT flag; staging the model is optional during deployment, and missing VGGT must not block the AMC run.

For nvstreamer setup details and sensor pre-registration, see your VIOS deployment docs.

Step 3 — Initialize RTSP Run

Before capture, allocate an AMC project using `common-steps.md`. The RTSP capture request uses that project_id.

Step 4 — Start RTSP Capture

POST /v1/rtsp/capture/<project_id>
Content-Type: application/json

{
  "streams": [
    {"rtsp_url": "rtsp://.../cam_00", "camera_name": "cam_00", "sensor_id": null},
    {"rtsp_url": "rtsp://.../cam_01", "camera_name": "cam_01", "sensor_id": null}
  ],
  "duration_seconds": 180,
  "vios_token": null,
  "ssl_verify": false
}

Response shape: {"code": 0, "message": "...", "session": {"session_id": "...", "status": "STARTING", ...}}. Save session.session_id. The same nested-session shape is returned by GET /v1/rtsp/capture/<project_id>/<session_id>, so unwrap it on every poll too.

Session lifecycle:

STARTING → RECORDING → COMPLETED → INGESTING → INGESTED
                                ↘ ERROR
RECORDING → CANCELLED (via /stop)

Step 5 — Poll Capture Status, Then Ingest

Poll every ~10 s until session state is COMPLETED:

GET /v1/rtsp/capture/<project_id>/<session_id>

Then ingest the recorded clips as the project's video files:

POST /v1/rtsp/capture/<project_id>/<session_id>/ingest

When this returns successfully, the project has the clips attached — same state as if you'd called /v1/upload_video_files/<project_id> with local MP4s.

Need to stop early? POST /v1/rtsp/capture/<project_id>/<session_id>/stop — the partial clip can still be ingested.

Other session endpoints:

• GET /v1/rtsp/sessions/<project_id> — list all sessions for a project.
• DELETE /v1/rtsp/session/<project_id>/<session_id> — delete a session record.

Step 6 — Apply Config, Upload Alignment / Layout

Resolve the config path (asked in Step 2) and use it as the anchor to scan for alignment + layout.

Calibration settings: see Settings File + Detector Pattern.

Alignment + layout (resolved via same-dir scan of the config path, or user-provided, or UI fallback):

POST /v1/upload_alignment/<project_id>    alignment_file=<alignment_data.json>
POST /v1/upload_layout/<project_id>       layout_file=<layout.png>

Other optional uploads (same as the videos mode):

POST /v1/upload_gt_file/<project_id>      gt_file=<GT.zip>                 # optional
POST /v1/upload_focal_length/<project_id> focal_length=<f0>&focal_length=<f1>...  # optional

UI fallback details — see SKILL.md UI Fallback Pattern. Note for RTSP: the "Layout missing → UI Step 2" instruction says to upload layout.png ONLY; do not touch the video section because clips are already ingested from RTSP capture.

Step 7 — Hand off to the Shared Calibration Tail

Continue with SKILL.md Step A onward (verify → calibrate → poll → results). Use `calibration-tail.md` for the shared Python snippet; `common-steps.md` § Hand off has the reusable handoff note.

---

RTSP Mode Python Script

from pathlib import Path
import os
import time

import requests

# --- Edit these ---
BASE_URL       = "http://<HOST_IP>:<MS_PORT>/v1"   # default MS_PORT 8010
PROJECT_NAME   = "rtsp_calibration_run"

# One entry per camera
STREAMS = [
    {"rtsp_url": "rtsp://<host>:31556/.../cam_00.mp4", "camera_name": "cam_00", "sensor_id": None},
    {"rtsp_url": "rtsp://<host>:31557/.../cam_01.mp4", "camera_name": "cam_01", "sensor_id": None},
]
DURATION_SECONDS = 180                 # >= 60

# Anchor file — ask user for this path. Leave None if they don't have one (→ UI Step 3 fallback).
CONFIG_FILE    = None                                   # e.g. Path("/path/to/settings.json")
# If CONFIG_FILE is set, the skill scans its parent directory for alignment + layout.
ALIGNMENT_JSON = None
LAYOUT_PNG     = None
GT_ZIP         = None                                   # optional
FOCAL_LENGTHS  = None                                   # optional: [1269.0, 1099.5]
DETECTOR_TYPE  = "resnet"                               # overridden below if CONFIG_FILE pins it

VSS_APPS_DIR = Path(os.environ.get("VSS_APPS_DIR", Path.cwd()))
PROJECTS_DIR = Path(os.environ.get("PROJECTS_DIR", VSS_APPS_DIR / "services" / "auto-calibration" / "projects"))

# Auto-scan alignment+layout from the same dir as CONFIG_FILE
def _resolve_local(override, candidate_names, scan_dir, label):
    if override and Path(override).exists():
        return Path(override)
    if scan_dir is None:
        return None
    hits = [scan_dir / n for n in candidate_names if (scan_dir / n).exists()]
    if len(hits) == 1:
        print(f"    auto-detected {label}: {hits[0]}")
        return hits[0]
    if len(hits) > 1:
        print(f"    multiple {label} candidates in {scan_dir}: {hits} — skipping auto-detect")
    return None

_scan_dir = CONFIG_FILE.parent if (CONFIG_FILE and Path(CONFIG_FILE).exists()) else None
ALIGNMENT_JSON = _resolve_local(ALIGNMENT_JSON, ["alignment_data.json"], _scan_dir, "alignment")
LAYOUT_PNG     = _resolve_local(LAYOUT_PNG,     ["layout.png"],           _scan_dir, "layout")

s = requests.Session()

# Open an RTSP calibration project
r = s.post(f"{BASE_URL}/create_project", data={"project_name": PROJECT_NAME})
r.raise_for_status()
project_id = r.json()["project_id"]
print(f"[3] Created project {project_id}")

# Step 4 — Start RTSP capture
r = s.post(f"{BASE_URL}/rtsp/capture/{project_id}", json={
    "streams": STREAMS,
    "duration_seconds": DURATION_SECONDS,
    "vios_token": None,
    "ssl_verify": False,
})
r.raise_for_status()
session = r.json().get("session") or r.json()  # response nests session_id/status under "session"
session_id = session["session_id"]
print(f"[4] Capture session {session_id} — duration {DURATION_SECONDS}s")

# Step 5a — Poll capture status
print(f"[5] Polling capture status (~{DURATION_SECONDS + 60}s)...")
start = time.time(); last = ""
while time.time() - start < DURATION_SECONDS + 600:
    info = s.get(f"{BASE_URL}/rtsp/capture/{project_id}/{session_id}").json()
    sess = info.get("session") or info
    state = sess.get("status") or sess.get("state")
    elapsed = int(time.time() - start)
    if state != last:
        print(f"    [{elapsed:>4}s] {state}", flush=True); last = state
    if state == "COMPLETED":
        break
    if state in {"ERROR", "CANCELLED"}:
        raise RuntimeError(f"Capture {state}: {info}")
    time.sleep(10)
else:
    raise RuntimeError("Capture poll timed out")

# Step 5b — Ingest clips into project
r = s.post(f"{BASE_URL}/rtsp/capture/{project_id}/{session_id}/ingest")
r.raise_for_status()
print(f"[5] Ingested clips: {r.json()}")

# Step 6 — Config + alignment + layout + optional extras
if CONFIG_FILE and Path(CONFIG_FILE).exists():
    r = s.post(f"{BASE_URL}/config/{project_id}",
               data=Path(CONFIG_FILE).read_bytes(),
               headers={"Content-Type": "application/json"})
    r.raise_for_status()
    print(f"[6] Applied calibration config from {Path(CONFIG_FILE).name}")
    try:
        import json as _json
        _cfg = _json.loads(Path(CONFIG_FILE).read_text())
        _det = _cfg.get("detector") or _cfg.get("detector_type")
        if _det in ("resnet", "transformer"):
            DETECTOR_TYPE = _det
            print(f"    Detector overridden from config: {DETECTOR_TYPE}")
    except Exception:
        pass

if ALIGNMENT_JSON and ALIGNMENT_JSON.exists():
    with open(ALIGNMENT_JSON, "rb") as f:
        s.post(f"{BASE_URL}/upload_alignment/{project_id}",
               files={"alignment_file": (ALIGNMENT_JSON.name, f, "application/json")}).raise_for_status()
if LAYOUT_PNG and LAYOUT_PNG.exists():
    with open(LAYOUT_PNG, "rb") as f:
        s.post(f"{BASE_URL}/upload_layout/{project_id}",
               files={"layout_file": (LAYOUT_PNG.name, f, "image/png")}).raise_for_status()
if GT_ZIP and Path(GT_ZIP).exists():
    with open(GT_ZIP, "rb") as f:
        s.post(f"{BASE_URL}/upload_gt_file/{project_id}",
               files={"gt_file": (Path(GT_ZIP).name, f, "application/zip")}, timeout=120).raise_for_status()
if FOCAL_LENGTHS:
    s.post(f"{BASE_URL}/upload_focal_length/{project_id}",
           data={"focal_length": FOCAL_LENGTHS}).raise_for_status()

# UI fallback for anything not resolved — run the canonical block from
# videos.md § "Step 5 — UI fallback for anything not resolved" (builds ui_tasks,
# prompts for the detector, and verifies the manual_adjustment alignment files).
# RTSP difference: videos are already ingested from the RTSP capture, so in UI
# Step 2 (Video Configuration) upload layout.png ONLY — do not re-upload videos.

# Run the shared tail now; see Step 7 above.

Mode-specific Troubleshooting

See the Cross-cutting Troubleshooting table in SKILL.md for issues that span all modes.

vss-generate-video-calibration — Sample-Dataset Mode (verify install)

Load this reference when the user wants to verify a fresh AMC install by running calibration on the bundled sample dataset (sdg_08_2_sample_data_010926.zip, 4 synthetic warehouse cameras with ground truth). Useful before throwing real data at it.

For your own pre-recorded MP4s, see videos.md. For live RTSP streams, see rtsp.md.

The sample includes GT, so the run produces evaluation metrics (L2 distance, reprojection error) — no calibration parameter tuning needed.

Mode-specific Prerequisites

• Sample zip present at `assets/sdg_08_2_sample_data_010926.zip` — the VSS repo does not ship this file. See Obtain the sample zip below.
• Python 3 with `requests` available — or use the Swagger UI walkthrough below.
• The inline run block self-heals: if requests is missing it creates a throwaway venv under ${TMPDIR:-/tmp}/amc-sample-test-venv (nothing written to the repo).
• If python3 -m venv itself fails with ensurepip not available, the inline block falls back to `uv` (sudo-free, installed via curl -LsSf https://astral.sh/uv/install.sh | sh). If neither path is available: sudo apt install -y python3-venv python3-pip as a last resort.

The shared AMC microservice prereq comes from the SKILL.md Prerequisites section.

Quick Start for Agents

"launch AMC and test sample dataset" (or similar):

1. Walk deploy-auto-calibration-service.md first to bring up the AMC stack. 2. Wait for /v1/ready to return OK. 3. Extract sample data (snippet below) — idempotent, safe to re-run. 4. Run the inline block in Run Inline (No File Written). Do not save it as a .py file — pipe via heredoc so the user's repo stays clean. 5. Report final metrics + UI URL for manual inspection.

"test sample dataset" (MS already running):

1. Detect backend: scan ports 8000–8009 (and 8010) for a /v1/ready response. 2. If none → walk deploy-auto-calibration-service.md first. 3. Extract sample data if not already cached. 4. Run the inline block (heredoc-piped Python — no file written). 5. Report metrics.

Detect Running Backend

MS_HOST="${HOST_IP:-localhost}"
MS_PORT=""
for port in {8000..8009}; do
  if curl -s "http://${MS_HOST}:$port/v1/ready" | grep -q '"code":0'; then
    MS_PORT=$port; break
  fi
done
if [ -z "$MS_PORT" ] && curl -s "http://${MS_HOST}:8010/v1/ready" | grep -q '"code":0'; then
  MS_PORT=8010
fi
[ -z "$MS_PORT" ] && { echo "No running backend. Walk deploy-auto-calibration-service.md first to bring up AMC."; exit 1; }
echo "Backend on ${MS_HOST}:$MS_PORT"

Obtain the sample zip

The zip is not committed to the VSS repo. It lives in the standalone AMC repo on GitHub, where it ships via git-lfs:

• Canonical source: <https://github.com/NVIDIA-AI-IOT/auto-magic-calib/blob/main/assets/sdg_08_2_sample_data_010926.zip>
• Raw LFS download: <https://github.com/NVIDIA-AI-IOT/auto-magic-calib/raw/main/assets/sdg_08_2_sample_data_010926.zip>
• File size: ~154 MB

Pick the path that fits your setup:

export REPO_ROOT=$(git rev-parse --show-toplevel)
mkdir -p "$REPO_ROOT/assets"
TARGET="$REPO_ROOT/assets/sdg_08_2_sample_data_010926.zip"

# (a) Reuse an existing AMC checkout on the same host (cheapest, no network)
if [ -f "$HOME/auto-magic-calib/assets/sdg_08_2_sample_data_010926.zip" ]; then
  ln -sf "$HOME/auto-magic-calib/assets/sdg_08_2_sample_data_010926.zip" "$TARGET"

# (b) Pull from GitHub LFS directly (no AMC checkout needed)
else
  curl -L -o "$TARGET" \
    https://github.com/NVIDIA-AI-IOT/auto-magic-calib/raw/main/assets/sdg_08_2_sample_data_010926.zip
fi

# (c) Or: clone the AMC repo with LFS into a sibling dir and symlink — useful if you
# also want the AMC scripts/docs:
#   git lfs install
#   git clone https://github.com/NVIDIA-AI-IOT/auto-magic-calib.git ../auto-magic-calib
#   ln -sf "$PWD/../auto-magic-calib/assets/sdg_08_2_sample_data_010926.zip" "$TARGET"

# Verify (~154 MB)
ls -lh "$TARGET"

The VSS repo deliberately doesn't bundle the zip (size + version-skew across AMC releases). Don't commit it here — assets/sdg_08_2_sample_data_010926.zip should stay gitignored if you copy it in.

Locate + Extract Sample Data (idempotent)

export REPO_ROOT=$(git rev-parse --show-toplevel)

SAMPLE_ZIP="$REPO_ROOT/assets/sdg_08_2_sample_data_010926.zip"
[ -f "$SAMPLE_ZIP" ] || { echo "Sample zip not found at $SAMPLE_ZIP"; exit 1; }

# Cache directory next to the zip.
SAMPLE_DIR="$(dirname "$SAMPLE_ZIP")/.cache/sdg_08_2_sample_data_010926"

if [ ! -d "$SAMPLE_DIR" ]; then
  mkdir -p "$SAMPLE_DIR"
  unzip -q "$SAMPLE_ZIP" -d "$SAMPLE_DIR"
fi
ls "$SAMPLE_DIR"
# Expected (possibly inside a wrapper folder): alignment_data/  GT.zip  videos/

Run Inline (No File Written)

Run the test on the fly — pipe Python into python3 via heredoc so nothing is saved into the user's repo. The block below is fully self-contained: it resolves REPO_ROOT via git rev-parse, reads MS_PORT from the warehouse-operations .env, picks (or creates) a Python with requests installed, and then pipes the inline script. Safe to copy/paste verbatim. Each invocation creates a fresh project.

# Resolve env
export REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
ENV_FILE="$REPO_ROOT/deploy/docker/industry-profiles/warehouse-operations/.env"
export MS_PORT="$(grep ^VSS_AUTO_CALIBRATION_PORT "$ENV_FILE" 2>/dev/null | cut -d= -f2)"
export MS_PORT="${MS_PORT:-8010}"
export BASE_URL="http://${HOST_IP:-localhost}:${MS_PORT}/v1"
# Optional: export SAMPLE_DIR=/abs/path/to/extracted/sample to override autodetection

# Pick a python3 that has `requests`; create a throwaway venv if needed (no repo files written)
PY=python3
"$PY" -c 'import requests' 2>/dev/null || {
  VENV="${TMPDIR:-/tmp}/amc-sample-test-venv"
  # Try the stdlib venv first.
  if python3 -m venv "$VENV" 2>/dev/null; then
    "$VENV/bin/pip" install --quiet requests
    PY="$VENV/bin/python3"
  # Fall back to uv (sudo-free, user-local install). Same fallback as the /deploy skill.
  elif command -v uv >/dev/null 2>&1 \
      || curl -LsSf https://astral.sh/uv/install.sh | sh; then
    export PATH="$HOME/.local/bin:$PATH"
    uv venv "$VENV"
    uv pip install --python "$VENV/bin/python" --quiet requests
    PY="$VENV/bin/python3"
  # Last resort: stdlib venv via apt (requires sudo).
  else
    echo "Need python3-venv or uv. Try one of:" >&2
    echo "  curl -LsSf https://astral.sh/uv/install.sh | sh   (no sudo)" >&2
    echo "  sudo apt install -y python3-venv python3-pip" >&2
    exit 1
  fi
}

"$PY" - <<'PY'
import os
import sys
import time
from pathlib import Path

import requests

# REPO_ROOT comes from the surrounding shell; fall back to cwd when missing
# (no `__file__` to lean on when fed via stdin).
REPO_ROOT = Path(os.environ.get("REPO_ROOT") or Path.cwd())
MS_PORT = os.environ.get("MS_PORT", "8010")
BASE_URL = os.environ.get("BASE_URL", f"http://{os.environ.get('HOST_IP', 'localhost')}:{MS_PORT}/v1")

# Sample zip lives in assets/.
def _find_sample_dir() -> Path:
    candidate = REPO_ROOT / "assets" / ".cache" / "sdg_08_2_sample_data_010926"
    if candidate.exists():
        return candidate
    sys.exit(
        "Sample data not extracted. Run the extraction snippet from this reference first, "
        "or pass SAMPLE_DIR= explicitly."
    )

# NOTE: do NOT write `Path(os.environ.get("SAMPLE_DIR", "")) or _find_sample_dir()`
# — Path("") evaluates to Path('.') which is truthy, so the `or` never falls
# through and the script silently picks `.` (typically the repo root). Rglobbing
# `cam_*.mp4` from there can sweep dozens of stale videos from prior test runs.
_env_sample = os.environ.get("SAMPLE_DIR")
SAMPLE_DIR = Path(_env_sample).resolve() if _env_sample else _find_sample_dir()

# Locate sample files (handle an optional wrapper folder from unzip)
def _find(path: Path, name: str) -> Path:
    hits = list(path.rglob(name))
    if not hits:
        sys.exit(f"Could not find {name} under {path}")
    return hits[0]

# Anchor video discovery on the canonical `videos/` directory if present
# (non-recursive). Only fall back to rglob if no `videos/` folder exists,
# and assert a sane upper bound so a misconfigured SAMPLE_DIR fails loud
# instead of uploading every cam_*.mp4 in the tree.
videos_dirs = list(SAMPLE_DIR.rglob("videos"))
videos_dir = next((d for d in videos_dirs if d.is_dir()), None)
if videos_dir is not None:
    videos = sorted(videos_dir.glob("cam_*.mp4"))
else:
    videos = sorted(SAMPLE_DIR.rglob("cam_*.mp4"))

alignment = _find(SAMPLE_DIR, "alignment_data.json")
layout = _find(SAMPLE_DIR, "layout.png")
gt_zip = _find(SAMPLE_DIR, "GT.zip")

assert len(videos) >= 2, f"Need >=2 cam_XX.mp4 under {SAMPLE_DIR}, found {len(videos)}"
# Sample dataset has 4 cameras — bail if SAMPLE_DIR is so wide we'd upload
# unrelated videos. Override SAMPLE_DIR explicitly if you need a different one.
assert len(videos) <= 16, (
    f"Found {len(videos)} cam_*.mp4 under {SAMPLE_DIR} — looks like SAMPLE_DIR "
    "is too broad (probably picked up stale test caches). Set SAMPLE_DIR to the "
    "extracted sample folder explicitly and re-run."
)
print(f"Base URL:   {BASE_URL}")
print(f"Sample dir: {SAMPLE_DIR}")
print(f"Videos:     {[v.name for v in videos]}")

s = requests.Session()

# Create the sample-dataset project
project_name = f"sample_test_{int(time.time())}"
r = s.post(f"{BASE_URL}/create_project", data={"project_name": project_name})
r.raise_for_status()
project_id = r.json()["project_id"]
print(f"[1] Created project {project_name} → {project_id}")

# Upload the bundled sample cameras; order defines camera indices.
upload_parts, open_files = [], []
try:
    for video_path in videos:
        handle = video_path.open("rb")
        open_files.append(handle)
        upload_parts.append(("files", (video_path.name, handle, "video/mp4")))
    r = s.post(f"{BASE_URL}/upload_video_files/{project_id}", files=upload_parts, timeout=300)
finally:
    for handle in open_files:
        handle.close()
r.raise_for_status()
print(f"[2] Uploaded {len(videos)} videos")

# Step 3 — Upload alignment JSON
with open(alignment, "rb") as f:
    r = s.post(f"{BASE_URL}/upload_alignment/{project_id}",
               files={"alignment_file": (alignment.name, f, "application/json")})
    r.raise_for_status()
print(f"[3] Uploaded alignment JSON")

# Step 4 — Upload layout PNG
with open(layout, "rb") as f:
    r = s.post(f"{BASE_URL}/upload_layout/{project_id}",
               files={"layout_file": (layout.name, f, "image/png")})
    r.raise_for_status()
print(f"[4] Uploaded layout PNG")

# Step 5 — Upload GT zip (enables evaluation metrics)
with open(gt_zip, "rb") as f:
    r = s.post(f"{BASE_URL}/upload_gt_file/{project_id}",
               files={"gt_file": (gt_zip.name, f, "application/zip")}, timeout=120)
    r.raise_for_status()
print(f"[5] Uploaded GT zip")

# Shared Calibration Tail — see references/calibration-tail.md for the snippet
# (verify_project → calibrate → poll → fetch evaluation_statistics)
# Note: detector_type is hard-coded to "resnet" for the sample dataset.
DETECTOR_TYPE = "resnet"
# Run the snippet from references/calibration-tail.md here.
# Then fetch the evaluation statistics:
r = s.get(f"{BASE_URL}/result/{project_id}/evaluation_statistics")
if r.status_code == 200:
    stats = r.json().get("statistics", r.json())
    print(f"\n[D] Evaluation statistics:")
    for k, v in stats.items():
        print(f"    {k}: {v}")
else:
    print(f"\n[D] evaluation_statistics returned {r.status_code}: {r.text[:200]}")

print(f"\nProject ID: {project_id}")
print("Inspect in UI: open the project in the web UI to view results and overlay videos")
PY

Why heredoc, not a `.py` file? The reference is meant to run on demand against any user's checkout — writing run_sample_test.py into the repo would dirty their working tree. The <<'PY' quoting prevents shell expansion inside the script. Re-run the same block any time; each run creates a fresh project.

Alternative: Swagger UI Walkthrough

The microservice exposes an interactive OpenAPI UI at `http://<HOST_IP>:<MS_PORT>/docs`. If you prefer clicking through the API by hand:

1. Open http://<HOST_IP>:<MS_PORT>/docs in a browser (default MS_PORT is 8010). 2. Unzip sdg_08_2_sample_data_010926.zip into a cache directory next to it. 3. Execute these endpoints in order, copying the project_id from step 1 into subsequent paths:

This is the same sequence the Python script runs, just executed manually.

Success Criteria

• Project reaches project_state == "COMPLETED" within ~30 min.
• /v1/result/{id}/evaluation_statistics returns non-empty statistics (GT was uploaded).
• No ERROR state encountered.

Representative metrics for the sample (yours should be similar):

Average L2 distance(m)               : < 1.5
Average reprojection error 0(px)     : < 10

Monitoring Progress

PROJECT_ID=<id_from_step_1>
# Calibration log lives under the projects dir, relative to the container
# working directory. Use projects/...; do not prefix it with the
# working-directory basename.
docker exec vss-auto-calibration tail -F projects/project_${PROJECT_ID}/calibration.log

Or stream MS logs:

docker logs -f vss-auto-calibration

Mode-specific Troubleshooting

See the Cross-cutting Troubleshooting table in SKILL.md for issues that span all modes.

vss-generate-video-calibration — Videos Mode (pre-recorded MP4s)

Load this reference when the user has local MP4 files to calibrate. Skip to the Shared Calibration Tail in SKILL.md once videos + alignment + layout are uploaded.

For live RTSP streams, see rtsp.md. For verifying the install with the bundled sample, see sample-dataset.md.

What to Ask the User

Required

1. Videos directory — a folder containing cam_00.mp4, cam_01.mp4, … (time-synchronized, 1920×1080 recommended). The skill reads cam_*.mp4 from here and uploads them sorted alphabetically. 2. Microservice URL — e.g. http://<HOST_IP>:8010. 3. Project name — short descriptive string.

Auto-Detected (ask only if not found)

The skill scans the videos directory and its parent directory for these files and uses them silently if exactly one match is found. Ask the user only if missing or ambiguous; if they don't have the file, fall back to the UI (see SKILL.md UI Fallback Pattern):

See the Settings File + Detector Pattern section in SKILL.md for the parsing rule.

Required when no calibration-settings file is provided

4. Detector type — see SKILL.md § Step B — Start Calibration for the resnet vs transformer choice and the AskUserQuestion fallback. When a config file is provided, the script extracts the detector automatically. 5. Parameter tuning — also ask whether to proceed with the default calibration parameters or tune them in the UI (Step 3: Parameters) first. See SKILL.md § Step B for the exact prompt.

Optional

5. Ground truth zip — GT.zip with _World_Cameras_Camera_XX/ folders (enables evaluation metrics). 6. Focal lengths — one per camera, e.g. 1269.0, 1099.5, 1099.5.

VGGT refinement is handled after AMC completes by SKILL.md Step E. Do not collect a separate videos-mode VGGT flag; staging the model is optional during deployment, and missing VGGT must not block the AMC run.

Root README.md "Custom Dataset" section documents input-video guidelines and ground-truth format.

API Call Sequence (videos mode)

Step 1 — Initialize Videos Run

Create the project with the shared request in `common-steps.md`, then keep project_id for the upload calls.

Step 2 — Upload Videos (required)

See `common-steps.md` § Upload videos.

Important: upload sorted alphabetically — the server assigns camera

indices by upload order. The multipart/form-data part name is files.

Step 3 — Resolve Local Files (Auto-Scan, Ask, or UI)

For each of calibration-settings, alignment, and layout, run this resolution:

1. Auto-scan VIDEO_DIR and VIDEO_DIR.parent for the candidate filenames (table above). 2. If exactly one match, use it silently and print what was found. 3. If zero or multiple matches, ask the user for an explicit path via AskUserQuestion. If they don't have the file, mark it for UI fallback. 4. UI fallback: see SKILL.md UI Fallback Pattern.

Step 4 — Upload Resolved Files

For each file that was resolved locally:

Calibration settings:

POST /v1/config/<project_id>
Content-Type: application/json

<file contents, posted as-is>

After a successful POST, also parse the file for "detector" / "detector_type" and override DETECTOR_TYPE for the /calibrate call (see Settings File + Detector Pattern).

Alignment JSON:

POST /v1/upload_alignment/<project_id>
alignment_file: ("alignment_data.json", <bytes>, "application/json")

Layout PNG:

POST /v1/upload_layout/<project_id>
layout_file: ("layout.png", <bytes>, "image/png")

Ground truth (optional, enables evaluation):

POST /v1/upload_gt_file/<project_id>
gt_file: ("GT.zip", <bytes>, "application/zip")

Focal lengths (optional, overrides GeoCalib estimates):

POST /v1/upload_focal_length/<project_id>
focal_length=1269.0&focal_length=1099.5&...

Step 5 — Hand off to the Shared Calibration Tail

Once uploads are done (and any UI fallback confirmed on disk), continue with SKILL.md Step A onward (verify → calibrate → poll → results). Use `calibration-tail.md` for the shared Python snippet.

---

Videos Mode Python Script

import os
import time
from pathlib import Path

import requests

# --- Edit these ---
BASE_URL       = "http://<HOST_IP>:<MS_PORT>/v1"   # default MS_PORT 8010
PROJECT_NAME   = "my_calibration_run"
VIDEO_DIR      = Path("/path/to/videos")
# Optional explicit overrides (leave as None to trigger auto-scan, then ask-user, then UI fallback)
CONFIG_FILE    = None                                   # e.g. Path("/path/to/settings.json")
                                                        # Full settings override — replaces UI Step 3 (rectification, BA, eval, detector, ...).
                                                        # If the file pins a detector, it's also extracted for the calibrate call below.
ALIGNMENT_JSON = None                                   # e.g. Path("/path/to/alignment_data.json")
LAYOUT_PNG     = None                                   # e.g. Path("/path/to/layout.png")
GT_ZIP         = None                                   # optional: Path("/path/to/GT.zip")
FOCAL_LENGTHS  = None                                   # optional: [1269.0, 1099.5]
DETECTOR_TYPE  = "resnet"                               # "resnet" or "transformer" (overridden if CONFIG_FILE pins it)
RUN_VGGT_IF_READY = False  # Set True if the user requested VGGT or staged VGGT in this run

# Projects dir on the host (for verifying manual alignment output).
# Bind-mounted into the MS container from $VSS_APPS_DIR/services/auto-calibration/projects
# (see deploy/docker/services/auto-calibration/ms/compose.yml).
VSS_APPS_DIR = Path(os.environ.get("VSS_APPS_DIR", Path.cwd()))
PROJECTS_DIR = Path(os.environ.get("PROJECTS_DIR", VSS_APPS_DIR / "services" / "auto-calibration" / "projects"))

VIDEO_FILES = sorted(VIDEO_DIR.glob("cam_*.mp4"))
assert VIDEO_FILES, f"No cam_*.mp4 files under {VIDEO_DIR}"

# --- Auto-scan helper ---
def _resolve_local(override, candidate_names, scan_dirs, label):
    if override and Path(override).exists():
        return Path(override)
    hits = []
    for d in scan_dirs:
        for name in candidate_names:
            p = d / name
            if p.exists():
                hits.append(p)
    if len(hits) == 1:
        print(f"    auto-detected {label}: {hits[0]}")
        return hits[0]
    if len(hits) > 1:
        print(f"    multiple {label} candidates in {scan_dirs}: {hits} — skipping auto-detect")
    return None

_scan_dirs = [VIDEO_DIR, VIDEO_DIR.parent]
CONFIG_FILE    = _resolve_local(CONFIG_FILE,    ["calibration_settings.json", "settings.json", "config.json", "calibration_config.json"], _scan_dirs, "config")
ALIGNMENT_JSON = _resolve_local(ALIGNMENT_JSON, ["alignment_data.json"],                                       _scan_dirs, "alignment")
LAYOUT_PNG     = _resolve_local(LAYOUT_PNG,     ["layout.png"],                                                _scan_dirs, "layout")

s = requests.Session()

# Create the videos-mode project
r = s.post(f"{BASE_URL}/create_project", data={"project_name": PROJECT_NAME})
r.raise_for_status()
project_id = r.json()["project_id"]
print(f"[1] Created project: {project_id}")

# Upload videos alphabetically so camera indices are stable
files, handles = [], []
for v in VIDEO_FILES:
    f = open(v, "rb"); handles.append(f)
    files.append(("files", (v.name, f, "video/mp4")))
r = s.post(f"{BASE_URL}/upload_video_files/{project_id}", files=files, timeout=300)
for f in handles: f.close()
r.raise_for_status()
print(f"[2] Uploaded {len(VIDEO_FILES)} videos")

# Step 3/4 — Upload resolved files
if CONFIG_FILE and CONFIG_FILE.exists():
    r = s.post(f"{BASE_URL}/config/{project_id}",
               data=CONFIG_FILE.read_bytes(),
               headers={"Content-Type": "application/json"})
    r.raise_for_status()
    print(f"[3] Applied calibration config from {CONFIG_FILE.name}")
    try:
        import json as _json
        _cfg = _json.loads(CONFIG_FILE.read_text())
        _det = _cfg.get("detector") or _cfg.get("detector_type")
        if _det in ("resnet", "transformer"):
            DETECTOR_TYPE = _det
            print(f"    Detector overridden from config: {DETECTOR_TYPE}")
    except Exception:
        pass

if ALIGNMENT_JSON and ALIGNMENT_JSON.exists():
    with open(ALIGNMENT_JSON, "rb") as f:
        s.post(f"{BASE_URL}/upload_alignment/{project_id}",
               files={"alignment_file": (ALIGNMENT_JSON.name, f, "application/json")}).raise_for_status()
    print(f"[3] Uploaded alignment: {ALIGNMENT_JSON.name}")

if LAYOUT_PNG and LAYOUT_PNG.exists():
    with open(LAYOUT_PNG, "rb") as f:
        s.post(f"{BASE_URL}/upload_layout/{project_id}",
               files={"layout_file": (LAYOUT_PNG.name, f, "image/png")}).raise_for_status()
    print(f"[3] Uploaded layout: {LAYOUT_PNG.name}")

if GT_ZIP and GT_ZIP.exists():
    with open(GT_ZIP, "rb") as f:
        s.post(f"{BASE_URL}/upload_gt_file/{project_id}",
               files={"gt_file": (GT_ZIP.name, f, "application/zip")}, timeout=120).raise_for_status()
    print(f"[3] Uploaded GT zip")

if FOCAL_LENGTHS:
    s.post(f"{BASE_URL}/upload_focal_length/{project_id}",
           data={"focal_length": FOCAL_LENGTHS}).raise_for_status()
    print(f"[3] Uploaded focal lengths: {FOCAL_LENGTHS}")

# Step 5 — UI fallback for anything not resolved
ui_tasks = []
if not CONFIG_FILE:
    ui_tasks.append("Step 3 (Parameters): tune settings or accept defaults, then Save.")
    # Agent should ask via AskUserQuestion; the input() is the direct-run fallback.
    if DETECTOR_TYPE == "resnet":
        _choice = input("    Detector [resnet/transformer] (default resnet): ").strip().lower()
        if _choice in ("resnet", "transformer"):
            DETECTOR_TYPE = _choice
        print(f"    Using detector: {DETECTOR_TYPE}")
if not ALIGNMENT_JSON or not LAYOUT_PNG:
    ui_tasks.append("Step 2 (Video Configuration): upload layout.png only — videos already uploaded via API, do not re-upload. Then Save. Step 4 (Alignment): upload alignment_data.json or mark correspondence points, then Save.")
if ui_tasks:
    print(f"\n[5] UI action required for project {project_id}:")
    for t in ui_tasks:
        print(f"    - {t}")
    input("    Press Enter when done...")
    if not ALIGNMENT_JSON or not LAYOUT_PNG:
        manual_dir = PROJECTS_DIR / f"project_{project_id}" / "manual_adjustment"
        assert (manual_dir / "alignment_data.json").exists() and (manual_dir / "layout.png").exists(), (
            f"Alignment files missing under {manual_dir}. Re-check UI Step 4 and click Save."
        )
        print(f"    Alignment files verified at {manual_dir}")

# Paste references/calibration-tail.md here before VGGT refinement.

# Step E — VGGT refinement
info = s.get(f"{BASE_URL}/get_project_info/{project_id}").json()
vggt_state = info.get("project_info", {}).get("vggt_state", "INIT")
if vggt_state == "READY" and RUN_VGGT_IF_READY:
    s.post(f"{BASE_URL}/vggt/calibrate/{project_id}").raise_for_status()
    print("\n[E] VGGT started")
    t0 = time.time()
    while time.time() - t0 < 900:
        vs = s.get(f"{BASE_URL}/get_project_info/{project_id}").json() \
            .get("project_info", {}).get("vggt_state", "INIT")
        if vs == "COMPLETED":
            print("     VGGT done"); break
        if vs == "ERROR":
            raise RuntimeError("VGGT failed")
        time.sleep(10)
elif vggt_state == "READY":
    print("\n[E] VGGT is ready. Ask whether to run refinement; set RUN_VGGT_IF_READY=True for direct-mode runs.")
else:
    print(f"\n[E] VGGT not ready (state={vggt_state}) — skipping. VGGT refinement is available after staging the model.")

print(f"\nProject: {project_id}")
print(f"Final camera parameters: ${{VSS_APPS_DIR}}/services/auto-calibration/projects/project_{project_id}/output/multi_view_results/BA_output/results_ba/refined/camInfo_XX.yaml")

Mode-specific Troubleshooting

See the Cross-cutting Troubleshooting table in SKILL.md for issues that span all modes.

Description: <br>

Use to run AutoMagicCalib on local MP4s, RTSP, or the bundled sample dataset, and to deploy vss-auto-calibration when needed. <br>

This skill is ready for commercial/non-commercial use. <br>

Owner

NVIDIA <br>

License/Terms of Use: <br>

Apache-2.0 <br>

Use Case: <br>

Developers and engineers use this skill to run automated camera calibration (AutoMagicCalib) on video inputs via the VSS auto-calibration microservice REST API. <br>

Deployment Geography for Use: <br>

Global <br>

Known Risks and Mitigations: <br>

Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br> Mitigation: Review and scan skill before deployment. <br>

Reference(s): <br>

• VSS Documentation <br>
• GitHub Repository <br>

Skill Output: <br>

Output Type(s): [API Calls, Shell commands, Configuration instructions] <br> Output Format: [Markdown with inline bash and Python code blocks] <br> Output Parameters: [1D] <br> Other Properties Related to Output: [None] <br>

Evaluation Agents Used: <br>

• claude-code <br>
• codex <br>

Evaluation Tasks: <br>

Evaluated against 3 internal evaluation tasks (3 positive skill-activation cases, 0 negative cases). <br>

Evaluation Metrics Used: <br>

Reported benchmark dimensions: <br>

• Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. <br>
• Correctness: Checks whether the agent follows the expected workflow and produces the correct final output. <br>
• Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant. <br>
• Effectiveness: Checks whether the agent performs measurably better with the skill than without it. <br>
• Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work. <br>

Underlying evaluation signals used in this run: <br>

• security: Checks for unsafe operations, secret leakage, and unauthorized access. <br>
• skill_execution: Verifies that the agent loaded the expected skill and workflow. <br>
• skill_efficiency: Checks routing quality, decoy avoidance, and redundant tool usage. <br>
• accuracy: Grades final-answer correctness against the reference answer. <br>
• goal_accuracy: Checks whether the overall user task completed successfully. <br>
• behavior_check: Verifies expected behavior steps, including safety expectations. <br>
• token_efficiency: Compares token usage with and without the skill. <br>

Evaluation Results: <br>

Skill Version(s): <br>

3.2.0 (source: frontmatter) <br>

Ethical Considerations: <br>

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

(For Release on NVIDIA Platforms Only) <br> Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here. <br>