Vss Search Archive

Purpose

Run the top-level VSS fusion search across archived video, ingest new clips / RTSP streams for search, and delete search-ingested sources.

Prerequisites

• Active VSS deployment reachable on $HOST_IP (see vss-deploy-profile and references/).
• vss-manage-video-io-storage skill installed (used to list and manage video sources before search).
• NGC credentials in $NGC_CLI_API_KEY and $NVIDIA_API_KEY for any image pulls.
• curl, jq, and Docker available on the caller.

Instructions

Follow the routing tables and step-by-step workflows below. Each section that ends in workflow, quick start, or flow is intended to be executed top-to-bottom. Detailed reference material lives in references/.

Examples

Worked end-to-end examples are kept under evals/ (each *.json manifest contains a runnable scenario) and inline in the per-workflow curl blocks below. Run a Tier-3 evaluation with nv-base validate <this-skill-dir> --agent-eval to replay them.

Limitations

• Requires the matching VSS profile / microservice to be deployed and reachable from the caller.
• NGC-hosted models and NIMs may be subject to rate-limits, GPU memory requirements, and license restrictions.
• Concurrency, GPU memory, and storage limits depend on the host hardware and the profile's compose file.

Troubleshooting

• Error: REST call returns connection refused. Cause: target microservice not running. Solution: probe /docs or /health; redeploy via vss-deploy-profile or the matching vss-deploy-* skill.
• Error: HTTP 401/403 from NGC pulls. Cause: missing/expired NGC_CLI_API_KEY. Solution: docker login nvcr.io and re-export the key before retrying.
• Error: container OOM or model fails to load. Cause: insufficient GPU memory for the selected profile. Solution: switch to a smaller variant or free GPUs via docker compose down.

Video Search Workflows

Alpha Feature — not recommended for production use.

Search video archives by natural language using Cosmos Embed1 embeddings. Requires the search profile — deploy with the vss-deploy-profile skill (-p search). These videos sources can be ingested files or RTSP streams.

When to Use

• "Find all instances of forklifts"
• "When did someone enter the restricted area?"
• "Show me people near the loading dock"
• "Search for vehicles between 8am and noon"
• Any natural-language search across video archives
• "Ingest <file> for search" / "upload this video for search"
• "Add this RTSP stream for search" / "register <rtsp_url> for search"
• "Delete <file> from search" / "remove this video and embeddings"

---

Deployment prerequisite

This skill requires the VSS search profile running on the host at $HOST_IP. Before any request:

1. Probe the stack:

   curl -sf --max-time 5 "http://${HOST_IP}:8000/docs" >/dev/null \
     && curl -sf --max-time 5 "http://${HOST_IP}:9200/" >/dev/null

(The second check confirms Elasticsearch is up — unique to the search profile.)

2. If the probe fails, ask the user:

"The VSS `search` profile isn't running on `$HOST_IP`. Shall I deploy it now using the `/vss-deploy-profile` skill with `-p search`?"

• If yes → hand off to the /vss-deploy-profile skill. Return here once it succeeds.
• If no → stop. Do not run this skill against a missing or wrong-profile stack.

(If your caller has granted explicit pre-authorization to deploy autonomously — e.g. the request says "pre-authorized to deploy prerequisites", or you are running in a non-interactive evaluation harness with that permission — skip the confirmation and invoke /vss-deploy-profile directly.)

3. If the probe passes, proceed.

---

Ingestion prerequisite (required before any /generate)

For a source to be searchable it must be ingested through the VSS agent backend, not through VIOS alone. The agent's ingest routes own the VIOS upload + RTVI-CV register + RTVI-embed pipeline as one transaction; a bare VIOS PUT only stores the bytes and never wires them into Elasticsearch.

Confirm the source exists in VIOS first (Mandatory workflow Step 2). If it is missing, ingest it with one of the recipes below before firing /generate. After ingest succeeds, the source appears in sensor/list under the name you provided and can be referenced from the natural-language query the agent forwards to its search-tool decomposer — you do NOT need to construct a structured video_sources payload yourself.

File upload — universal three-step flow

Use the timestamped upload form below. The VSS agent/search profile uses 2025-01-01T00:00:00.000Z as the uploaded video_file base timestamp; VIOS storage and embeddings must share that timeline, otherwise screenshot URLs and critic frame fetches can fail.

FILENAME="<filename.mp4>"
FILE_PATH="/path/to/${FILENAME}"

# 1. Ask the agent for the chunked-upload URL
UPLOAD_URL=$(curl -s -X POST "http://${HOST_IP}:8000/api/v1/videos" \
  -H "Content-Type: application/json" \
  -d "{\"filename\":\"${FILENAME}\"}" | jq -r .url)

# 2. Chunked POST the file to that VST URL (nvstreamer protocol).
#    The final-chunk response carries sensorId.
IDENTIFIER=$(uuidgen 2>/dev/null || cat /proc/sys/kernel/random/uuid)
UPLOAD_RESPONSE=$(curl -s -X POST "${UPLOAD_URL}" \
  -H "nvstreamer-chunk-number: 1" \
  -H "nvstreamer-total-chunks: 1" \
  -H "nvstreamer-is-last-chunk: true" \
  -H "nvstreamer-identifier: ${IDENTIFIER}" \
  -H "nvstreamer-file-name: ${FILENAME}" \
  -F "mediaFile=@${FILE_PATH};filename=${FILENAME}" \
  -F "filename=${FILENAME}" \
  -F 'metadata={"timestamp":"2025-01-01T00:00:00"}')

# 3. Tell the agent the upload finished — this fans out to RTVI-CV + RTVI-embed
SENSOR=$(printf '%s' "${UPLOAD_RESPONSE}" | jq -r .sensorId)
[ -z "${SENSOR}" ] || [ "${SENSOR}" = "null" ] \
  && { echo "Upload failed: no sensorId in response: ${UPLOAD_RESPONSE}"; exit 1; }
printf '%s' "${UPLOAD_RESPONSE}" \
  | jq --arg filename "${FILENAME}" '. + {filename: $filename}' \
  | curl -s -X POST "http://${HOST_IP}:8000/api/v1/videos/${SENSOR}/complete" \
      -H "Content-Type: application/json" \
      -d @- | jq .

Wait for the /complete response (it returns chunks_processed > 0 once embeddings land). Only then is the video searchable.

The deprecated PUT /api/v1/videos-for-search/{filename} route is also wired in for legacy callers (single-shot, agent-driven), but its OpenAPI entry is flagged deprecated. Prefer the three-step flow above for new work.

RTSP stream — single endpoint

curl -s -X POST "http://${HOST_IP}:8000/api/v1/rtsp-streams/add" \
  -H "Content-Type: application/json" \
  -d '{
    "sensorUrl": "rtsp://<host>:<port>/<path>",
    "name": "<sensor-name>",
    "username": "",
    "password": "",
    "location": "",
    "tags": ""
  }' | jq .

The response shape is {status, message, error} — no sensorId (the agent keys the stream by the name you provided). On any step's failure earlier steps roll back. The start_embedding_generation step is fire-and-verify: a 2xx confirms the request was accepted and the embedding pipeline is running in the background, not that the stream is searchable yet. Search hits will start appearing only after enough chunks land in Elasticsearch — poll with a low-top_k query a few seconds in if you need a readiness signal.

Delete source — agent-backed cleanup

Delete through the agent backend, not bare VIOS, so VIOS storage and search embeddings are cleaned up together.

# For video files: video_id is the VIOS sensor/video UUID
curl -s -X DELETE "http://${HOST_IP}:8000/api/v1/videos/<video_id>" | jq .

# For RTSP streams: name is the registered source name
curl -s -X DELETE "http://${HOST_IP}:8000/api/v1/rtsp-streams/delete/<name>" | jq .

---

How Search Works

1. Ingest — Files come in through the agent's three-step universal flow; RTSP streams through /api/v1/rtsp-streams/add. Both routes hand the source to RTVI-CV (attribute detection) and RTVI-Embed (Cosmos Embed1) which generates vector embeddings for video segments. 2. Index — Embeddings are stored in Elasticsearch via the Kafka pipeline. 3. Query — Natural-language queries are embedded and matched against stored vectors by similarity. 4. Results — Timestamped video segments ranked by relevance, with clip playback links.

This search orchestrated by VSS agent can lead to 3 behaviors:

• Attribute-only: when the LLM decomposes the query and finds only appearance attributes with no action (e.g. "person wearing red jacket")
• Embed-only: when the query has no extractable attributes (e.g. "show me forklifts")
• Fusion: when the query has both an action and attributes (e.g., "person in red jacket running"), it runs embed search first, then reranks using attribute search

---

Mandatory workflow

When using this skill, ALWAYS follow this high-level workflow: 1. Resolve inputs from user instructions — HARD STOP if `$HOST_IP` is not explicitly provided. See § Input resolution below. Do NOT default to localhost, 127.0.0.1, the host the agent itself is running on, or any other guess. Do NOT issue a POST http://.../generate request until the user has supplied an endpoint. Respond to the user with a single question asking for HOST_IP / the VSS agent endpoint and wait. 2. Resolve the source — HARD STOP before any `/generate` call. If the user query references a specific video / sensor name (e.g. "the airport video", "warehouse_cam_3", "sample warehouse"), verify it's actually registered in VIOS before firing POST .../generate. List sources via the vss-manage-video-io-storage skill.

Then:

• If the named source (or a clearly substring-matching name) IS in the list → proceed to step 3. Forward the user's natural-language query verbatim — the agent's own search tool decomposer (services/agent/src/vss_agents/tools/search.py) extracts video_sources from the prose given the available sources, so the skill does NOT need to construct a structured video sources payload.
• If the named source is NOT in the list → STOP. Do NOT fire /generate as a probe. Respond to the user with the registered source names and ask whether they meant one of those, want to ingest the missing source (point them at Ingestion prerequisite and run the matching file or RTSP recipe through the agent backend, not bare VIOS), or want to abandon the query. Wait for clarification.
• If the query names no specific source ("find forklifts in the ingested videos", "search across all sources") → skip the substring check, but sensor/list must still return non-empty (otherwise no sources are ingested → HARD STOP).

3. Run the search(es) via approach chosen 4. Present the results to the user query. Format response as a professional inspection report but name it Video Search Results: — Use clear section headers

• Organize findings individually with supporting detail, and close with a summary
• Use tables where comparisons help. Write like a technical report, not a chat message.
• If criteria results are non-null, then in addition to a column "Critic result" ("confirmed" | "rejected" | "skipped"), include a column "Criteria" with all the criteria for this search result ({criteria_n}: ✓ | ✗)

5. CRITICAL: Verify the results and explain this to the user concisely. If search fails, or returns unexpected results (i.e. videos that do not appear to match user query, zero matches, zero videos returned, error etc.), STOP. Do not proceed without reading troubleshooting.md to iterate with feedback loops until proper results are found and presented like a professional inspection report. 6. Final verifications:

• ALWAYS inform user that final and further verifications can be run. Present this as a Verification Step
• ONLY IF user agrees, download screenshots using the screenshot_url of the best candidates (highest similarity scores) from the search hits (JSON results) to /tmp. Read them and verify if they correspond to the user query

Input resolution

Infer these inputs only from the conversation or user query (no other files unless provided). If some cannot be inferred, ask the user immediately:

• $HOST_IP: where the VSS agent backend runs

---

Gotchas

• ALWAYS step into the troubleshooting step of the workflow immediately if anything unexpected happens, read troubleshooting.md
• Queries work best with concrete visual descriptions (objects, actions, locations). Augment user queries if needed to enhance the quality of the questions, expanding potential details
• The skill assumes video sources are already ingested through the agent backend (see Ingestion prerequisite). It MAY run the agent-backed ingest recipes when the user explicitly asks ("ingest <file> for search", "add <rtsp_url> for search"); it does NOT search the local filesystem for files the user didn't name, and it does NOT use the bare-VIOS PUT path (no embeddings get generated). Workflow step 2 still makes confirming "this source exists in VIOS" a hard precondition before /generate.
• Use vss-query-analytics skill to cross-reference search results with incident/alert data

---

Search via REST API

Default to using this REST API approach, unless user specifies otherwise.

# Consider only ingested video file sources by default
curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"input_message": "find all instances of forklifts"}' | jq .

More Examples

Use the messages request shape when passing structured request options such as search_source_type; the input_message shortcut does not accept extra fields.

# Search by object
curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"input_message": "find vehicles in the parking lot"}' | jq .

# Search by action
curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"input_message": "show me people running"}' | jq .

# Search by time context
curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"input_message": "what happened at the entrance between 2pm and 3pm?"}' | jq .

# Consider only RTSP sources with `search_source_type` filter i.e. live camera streams
curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "find all instances of forklifts"}], "search_source_type": "rtsp"}' | jq .

Advanced control knobs

If user query is ambiguous, user wants more guidance or when fine-grained control is needed, augment the user input_message by calling out explicitly certain options in plain-text and steering the agent in the desired direction. Available control axes:

Pick and choose some of these tuning options. Adjust them as needed for the user’s situation and query. For examples of discovery modes leveraging these, see discovery_modes.md.

---

Search via Agent UI

Open http://${HOST_IP}:3000/ and type natural-language queries:

find all instances of forklifts
show me people near the loading dock
when did a truck arrive at the gate?
find someone wearing a red jacket

Results include timestamped clips with similarity scores.

bump:2

Evaluation Report

Evaluation of the vss-search-archive skill before publication through NVSkills-Eval.

This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use.

Evaluation Summary

• Skill: vss-search-archive
• Evaluation date: 2026-06-15
• NVSkills-Eval profile: external
• Environment: astra-sandbox
• Dataset: 1 evaluation tasks
• Attempts per task: 1
• Pass threshold: 50%
• Overall verdict: PASS

Agents Used

• claude-code
• codex

Metrics Used

Reported benchmark dimensions:

• Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
• Correctness: checks whether the agent follows the expected workflow and produces the correct final output.
• Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
• Effectiveness: checks whether the agent performs measurably better with the skill than without it.
• Efficiency: checks whether the agent uses fewer tokens and avoids redundant work.

Underlying evaluation signals used in this run:

• security (Security): checks for unsafe operations, secret leakage, and unauthorized access.
• skill_execution (Skill Execution): verifies that the agent loaded the expected skill and workflow.
• skill_efficiency (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage.
• accuracy (Accuracy): grades final-answer correctness against the reference answer.
• goal_accuracy (Goal Accuracy): checks whether the overall user task completed successfully.
• behavior_check (Behavior Check): verifies expected behavior steps, including safety expectations.
• token_efficiency (Token Efficiency): compares token usage with and without the skill.

Test Tasks

The benchmark dataset contained 1 evaluation tasks:

• Positive tasks: 1 tasks where the skill was expected to activate.
• Negative tasks: 0 tasks where no skill was expected.
• Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred.

Task composition is derived from the evaluation dataset when possible. Entries with expected_skill set are treated as positive skill-activation cases, while entries with expected_skill: null are treated as negative activation cases.

Results

Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available.

Tier 1: Static Validation Summary

Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 1 total findings.

Top findings:

• LOW SCHEMA/author_format: Author must be of the form 'Name <email@host>' (skills/vss-search-archive/SKILL.md)

Tier 2: Deduplication Summary

This tier was not run or did not produce findings in this report.

Publication Recommendation

The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change.

Examples of discovery modes

Wide-net discovery — cast the widest net, fast

For exploratory searches when recall matters more than precision. Start broad (high result count e.g. 50–100, low similarity threshold e.g. 0.1, critic disabled exceptionally) then refine based on returned results.

curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"input_message": "find unusual activity, return top 100 results, any similarity, disable critic"}' | jq .

Typical follow-ups:

• Take the most promising results and re-run with high-precision mode (higher similarity threshold, lower top_k to filter noise)
• Scope to cameras/time — if certain cameras or time windows surfaced interesting results, re-run narrowed to those specific video sources and time ranges
• Search based on attributes — if a person of interest appeared in the results, follow up with an appearance-based query (e.g., "person wearing red jacket and blue jeans") to find other occurrences across cameras.

Narrow to specific cameras and/or time — scope to a known incident

When the camera location and time window are known. Reduces search space and returns faster, more relevant results.

Specify camera names as the video sources in the user input. Set explicit time range, keep critic enabled. For RTSP camera streams, use the RTSP messages + search_source_type request shape from the main SKILL.md instead of the input_message shortcut.

curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"input_message": "find person carrying a box at loading_dock_cam and warehouse_entrance between 10pm and 6am"}' | jq .

High-precision search — raise the similarity bar

When false positives are very costly (e.g., compliance audits, PPE verification) and there must be very low tolerance. Low result count, high similarity threshold (e.g. 0.5+) plus critic gives the tightest filter.

curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"input_message": "find person wearing high-visibility vest, top 5 results, minimum similarity 0.5"}' | jq .

Metadata-based filtering — filter by camera tags

Only useful when cameras are tagged with location or category metadata (e.g., "parking lot", "warehouse", "lobby"). Reduce pollution of the semantic search.

When considering this mode, first check if cameras have metadata or tags set using the vss-manage-video-io-storage skill to list sensors and show their descriptions. If no tags exist, offer the user the option to add metadata tags via the vss-manage-video-io-storage skill before relying on this type of filtering.

Mention the camera metadata tag (location, category) explicitly in the query. Can add other filters (camera names, time-ranges for further scoping etc.)

curl -s -X POST http://${HOST_IP}:8000/generate \
  -H "Content-Type: application/json" \
  -d '{"input_message": "find person running, only from cameras tagged as parking lot, top 10 results"}' | jq .

Troubleshooting feedback loop

Isolate the problem encountered in vss-search-archive then iterate to resolve it. Examples of useful flows below.

Gotchas

• ALWAYS use the method to list video sources with VST first with vss-manage-video-io-storage, before making curl requests to check Elasticsearch embeddings.
• If the video source is not ingested yet, NEVER use VST-only upload APIs because they will not generate embeddings. Use the agent video ingest handshake described below for video files (or rtsp-streams/add for RTSP streams), and use the term "ingest" instead of "upload" to avoid confusion.
• NEVER try to guess the URL or VST API to check what is available in the system. Use the vss-manage-video-io-storage skill instead to list video sources and manage streams feeding into the search pipeline

# NEVER guess commands like
# curl -s "http://<ip>:30888/vst/api/v1/sensors" 
# curl -s "http://<ip>:30888/vst/api/v2/sensors?pageSize=50"

Failure modes or unexpected results

• Video source(s) not returned or empty results
• Video source(s) returned, all with low similarity scores and/or a few with high scores. But sensor/stream names do not match the user query. Hence, not certain if these are correct answers, needs further verifications.
• Errors due to backend services all or partially not working

Troubleshooting flows

Target specific components. Infer from the conversation where (${HOST_IP}, ${PORT}) the service or model in question runs when running the commands below. If unable to infer, ask user to know ${HOST_IP} and ${PORT}.

The components in the externally accessible section should be reachable by their ${HOST_IP}. But if they are not (ports blocked by firewall for security), ask user if they are accessible via ssh and run those commands through ssh. Otherwise ask user how they prefer to reach them.

If further investigation is required, refer to the full components from the vss-deploy-profile skill and choose which one to investigate.

Externally accessible

• Ensure VST is running and ensure video source(s) of interest were ingested by listing them in VST via the vss-manage-video-io-storage skill.

If not, offer the user the option to ingest them via the full pipeline video ingest handshake below if they are video files (or rtsp-streams/add for RTSP streams).

• If a video source in the system has no embeddings, it means it has not been ingested through the full pipeline. STOP and ask user if video can be re-ingested and if user can provide video source. If yes, carefully follow:
• First delete it through the agent backend (avoid two copies; cleans indexes/embeddings too):

      # For video files
      # video_id = sensor / video UUID, same ID as in VST
      curl -s -X DELETE "http://${HOST_IP}:8000/api/v1/videos/<video_id>" | jq .

      # For RTSP streams
      curl -s -X DELETE "http://${HOST_IP}:8000/api/v1/rtsp-streams/delete/<name>" | jq .

• Then re-ingest the video source using the File upload or RTSP stream flow in the main SKILL.md under Ingestion prerequisite. Follow those steps exactly — they include the required nvstreamer chunked-upload headers and metadata.

• Further verifications to determine if returned video sources match the user query. Each step to go deeper:
• Check their source names, their video description / tags via the vss-manage-video-io-storage skill
• Download screenshots using the screenshot_url of the best candidates (highest similarity scores) from the search hits (JSON results) to /tmp. Read them and verify if they correspond to the user query

• Potentially retry by augmenting the user input with a lower similary threshold to include more results. This helps seeing if a clip of interest was filtered out due to a lower score

• Check if LLM/VLM are working:

# Ports are usually:
# - LLM: 30081
# - VLM: 30082
curl -s http://${HOST_IP}:${PORT}/v1/models | jq .

curl -s -X POST http://${HOST_IP}:${PORT}/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "<MODEL_NAME>", "max_tokens": 128, "messages": [{"role": "user", "content": "Hello!"}]}' | jq .

• Check if embeddings for that video source appear in Elasticsearch:

# List all indices with doc counts
curl -s "http://${HOST_IP}:9200/_cat/indices?h=index,docs.count,store.size&v"

# Count uploaded video_file embeddings
curl -s "http://${HOST_IP}:9200/mdx-embed-filtered-2025-01-01/_count"

# Count RTSP embeddings for a source name; RTSP streams use date-based indices
curl -s "http://${HOST_IP}:9200/mdx-embed-filtered-*,-mdx-embed-filtered-2025-01-01/_count" \
  -H "Content-Type: application/json" \
  -d '{"query": {"query_string": {"query": "*<sensor-name>*"}}}'

# Sample one uploaded video_file embedding doc (without the vector)
curl -s "http://${HOST_IP}:9200/mdx-embed-filtered-2025-01-01/_search?size=1&pretty" \
  -H "Content-Type: application/json" \
  -d '{"_source": {"excludes": ["embedding"]}, "query": {"match_all": {}}}'

# Sample one RTSP embedding doc for a source name (without the vector)
curl -s "http://${HOST_IP}:9200/mdx-embed-filtered-*,-mdx-embed-filtered-2025-01-01/_search?size=1&pretty" \
  -H "Content-Type: application/json" \
  -d '{"_source": {"excludes": ["embedding"]}, "query": {"query_string": {"query": "*<sensor-name>*"}}}'

Description: <br>

Use this skill to run top-level VSS fusion search on archived video, or to ingest video files / RTSP streams for search. <br>

This skill is for demonstration purposes and not for production usage. <br>

Owner

NVIDIA <br>

License/Terms of Use: <br>

Apache 2.0 OR MIT <br>

Use Case: <br>

Developers and engineers who need to search archived video content using natural-language queries, ingest video files or RTSP streams for search indexing, and manage search-ingested video sources. <br>

Deployment Geography for Use: <br>

Global <br>

Known Risks and Mitigations: <br>

Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. <br> Mitigation: Review and scan skill before deployment. <br>

Reference(s): <br>

• Discovery Modes <br>
• Troubleshooting <br>
• Video Search and Summarization GitHub <br>

Skill Output: <br>

Output Type(s): [API Calls, Shell commands, Analysis] <br> Output Format: [Markdown with inline bash code blocks] <br> Output Parameters: [1D] <br> Other Properties Related to Output: [None] <br>

Evaluation Agents Used: <br>

• claude-code <br>
• codex <br>

Evaluation Tasks: <br>

Evaluated against 1 task in the NVSkills-Eval external profile on the astra-sandbox environment. <br>

Evaluation Metrics Used: <br>

Reported benchmark dimensions: <br>

• Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. <br>
• Correctness: Checks whether the agent follows the expected workflow and produces the correct final output. <br>
• Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant. <br>
• Effectiveness: Checks whether the agent performs measurably better with the skill than without it. <br>
• Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work. <br>

Underlying evaluation signals used in this run: <br>

• security: Checks for unsafe operations, secret leakage, and unauthorized access. <br>
• skill_execution: Verifies that the agent loaded the expected skill and workflow. <br>
• skill_efficiency: Checks routing quality, decoy avoidance, and redundant tool usage. <br>
• accuracy: Grades final-answer correctness against the reference answer. <br>
• goal_accuracy: Checks whether the overall user task completed successfully. <br>
• behavior_check: Verifies expected behavior steps, including safety expectations. <br>
• token_efficiency: Compares token usage with and without the skill. <br>

Evaluation Results: <br>

Skill Version(s): <br>

3.2.0 (source: frontmatter) <br>

Ethical Considerations: <br>

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. <br>

(For Release on NVIDIA Platforms Only) <br> Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here. <br>