Tao Train Depth Anything V2

Name: Tao Train Depth Anything V2
Author: nvidia

nvidia/skills

1.4k installs
2.8k repo stars
Updated August 4, 2026
nvidia/skills

tao-train-depth-anything-v2 is an agent skill that trains, evaluates, exports, and deploys NVIDIA TAO Depth Anything V2 monocular depth models for developers who need per-pixel depth maps from single RGB images in roboti

About

tao-train-depth-anything-v2 is an NVIDIA agent skill (version 0.1.0) for monocular depth estimation with MetricDepthAnything or RelativeDepthAnything architectures inside the TAO depth_net CLI. The skill documents YAML spec overrides, pairs model_type with one of 10 supported dataset_name values, and covers train, evaluate, inference, export, quantize, and TensorRT gen_trt_engine deploy flows via Docker and nvidia-container-toolkit. Developers reach for tao-train-depth-anything-v2 when building robotics navigation, AR distance sensing, or scene-understanding pipelines that require metric depth in meters or relative depth ordering from a single camera frame. It includes finetuning recipes, AutoML routing, and troubleshooting for NaN train_loss and dataset encoding mismatches on NYU and custom RGB-depth datasets.

Depth Anything V2 on TAO
Monocular depth estimation training
Dataset and config orchestration
GPU-accelerated fine-tuning
Export for inference deployment

Tao Train Depth Anything V2 by the numbers

1,448 all-time installs (skills.sh)
+25 installs in the week ending Aug 5, 2026 (Skillselion tracking)
Ranked #191 of 2,064 Data Science & ML skills by installs in the Skillselion catalog
Data as of Aug 5, 2026 (Skillselion catalog sync)

npx skills add https://github.com/nvidia/skills --skill tao-train-depth-anything-v2

Add your badge

Show developers this skill is listed on Skillselion. Paste this into your README.

[![Listed on Skillselion](https://skillselion.com/badge/skills/nvidia/skills/tao-train-depth-anything-v2.svg)](https://skillselion.com/skills/nvidia/skills/tao-train-depth-anything-v2)

Installs	1.4k
repo stars	★ 2.8k
Last updated	August 4, 2026
Repository	nvidia/skills ↗

How do you fine-tune Depth Anything V2 in NVIDIA TAO?

Fine-tune or train Depth Anything V2 monocular depth models in NVIDIA TAO for robotics, AR, or 3D perception pipelines.

Who is it for?

Computer vision engineers building robotics, AR, or autonomous navigation pipelines who need TAO-guided Depth Anything V2 training with Docker and GPU hardware.

Skip if: Developers without NVIDIA GPUs, Docker, or nvidia-container-toolkit who only need a pretrained depth model without TAO training workflows.

When should I use this skill?

The user asks to train monocular depth, fine-tune Depth Anything V2, export depth_net ONNX, or deploy metric depth from single RGB images with NVIDIA TAO.

What you get

Trained depth checkpoints, evaluation KPIs, ONNX exports, quantized models, and TensorRT engines for monocular depth inference.

Fine-tuned depth checkpoints
ONNX and TensorRT depth engines
Evaluation KPIs including val/d1 metrics

By the numbers

Supports 10 mono dataset_name values for depth_net data_sources
Skill metadata version 0.1.0 under Apache-2.0 license
Recommends minimum 1 GPU and 24GB+ VRAM for ViT-Large depth training

Files

evals/
references/
schemas/

SKILL.mdMarkdownGitHub ↗

Depth Net Mono

Monocular depth estimation using Metric Depth Anything v2 or Relative Depth Anything architectures. Predicts per-pixel depth from single RGB images.

Pretrained checkpoint loading varies by model variant and use case — see the Pretrained checkpoint loading — use case matrix in references/parameters.md.

The mono and stereo skills both invoke the unified TAO depth_net CLI inside the container; the mono/stereo family is selected via model.model_type (see references/parameters.md).

For TAO Deploy TensorRT actions (gen_trt_engine, TensorRT evaluate, and TensorRT inference), read references/tao-deploy-depth-anything-v2.md first. The deploy spec template lives in this skill's references/spec_template_deploy.yaml.

PyT actions packaged by this model skill: train, evaluate, inference, export, and quantize. The PyT depth_net entrypoint does not accept a PyT-side gen_trt_engine action in the current TAO image. The gen_trt_engine action metadata must run with the TAO Deploy container, and the deploy workflow remains the deploy-specific entrypoint.

Train Action Policy

This model is AutoML-enabled at the model layer. Before handling any train-stage request, read references/skill_info.yaml and resolve the run override from either an explicit automl_policy value or the user's workflow request. Use automl_policy: on by default and only expose on / off in new launch prompts. Treat phrases like "turn off AutoML", "disable AutoML", "no HPO", or "plain training" as automl_policy: off for this run only. When automl_policy: on, automl_enabled: true, and both schemas/train.schema.json and references/spec_template_train.yaml are packaged, route the train action through tao-skill-bank:tao-run-automl by default with this model's skill_dir. Preserve workflow/application overrides for datasets, specs, output directories, GPU/platform settings, parent checkpoints, and automl_policy. Use direct model training only when automl_policy: off or the packaged train schema/template is missing; in the missing-schema case, report that AutoML is enabled but not runnable for this model until schemas are generated.

Non-train actions such as evaluate, inference, export, and deploy flows stay in this model skill. The per-run automl_policy override does not change model metadata.

Workflow

Prerequisites — data accessibility

Your dataset (RGB images + GT depth files) must be reachable from inside the container:

SDK runner: place files at the S3 paths the runner resolves (the S3_TRAIN / S3_EVAL placeholders shown in Typical Spec Overrides). The runner handles S3 → container-path mounting transparently.
Direct `docker run` (e.g. local testing): mount the host dataset root read-only at the same in-container path:

docker run ... -v <host_data_root>:<host_data_root>:ro <container> ...

The same accessibility requirement applies to the <output_dir> written by all actions.

Step 1 — Annotation file

Per-line annotation file referenced by data_sources[*].data_file:

Columns	Format	Use
1	`<image>`	Mono inference (no GT)
2	`<image> <gt_depth>`	Mono with GT

Do not pass stereo annotation rows such as <left_image> <right_image> <gt_depth> directly to mono train/evaluate/inference. If only a stereo depth dataset is available, derive a mono annotation file by keeping the left image and GT depth columns, then mount or stage the image/depth archive at the same container paths referenced by that derived annotation file.

If you already have one, point to it. Otherwise generate via depth_net convert:

depth_net convert -e <convert_spec.yaml>

convert_spec.yaml template:

results_dir: <directory where generated annotation files are written>
data_root: <directory whose immediate children are scene/sample folders that contain your image+depth files; convert walks data_root recursively but expects per-scene subdirectories at one level below>
image_dir_pattern: [<substring matching left/RGB image paths>]
depth_dir_pattern: [<substring matching GT depth paths>]
image_extension: ''     # optional .endswith filter, e.g. '.jpg'
depth_extension: ''     # optional, swapped during depth derivation, e.g. '.png'
split_ratio: 0.0        # 0.0/1.0 = test-only; 0.8 = 80/20 train+val

convert walks data_root recursively, selects paths whose path-string contains all substrings in image_dir_pattern (AND-filter), then derives the depth path by replacing image_dir_pattern[0] with depth_dir_pattern[0] and image_extension with depth_extension. Inspect your dataset's directory layout and identify the substring distinguishing RGB images from depth files (e.g. rgb_ vs sync_depth_).

data_root must point at the parent that contains the per-scene subdirectories (e.g. for NYU eval, use /data/nyu_v2/eval/test, not /data/nyu_v2/eval/test/bathroom — the latter limits the walk to a single scene). Always include the leading dot in image_extension / depth_extension (e.g. '.jpg' not 'jpg'); the substring swap is form-sensitive and a mismatch silently corrupts derived paths.

Step 2 — Pair `model_type` and `dataset_name` based on your data

Default — generic class for each task:

Data category	`model_type`	`dataset_name`
Disparity-encoded data (pixels)	`RelativeDepthAnything`	`RelativeMonoDataset`
Metric depth (meters)	`MetricDepthAnything`	`MetricMonoDataset`
Mono inference (no GT, any image)	matches train choice	`RelativeMonoDataset` or `MetricMonoDataset`

Dataset-specific class — switch when the data needs preprocessing the generic class does not perform:

Special case	`model_type`	`dataset_name`	What the class adds
NYU `sync_depth_*.png` (raw uint16 millimetres) — relative	`RelativeDepthAnything`	`NYUDV2Relative`	mm→m unit conversion + Eigen evaluation crop
NYU `sync_depth_*.png` (raw uint16 millimetres) — metric	`MetricDepthAnything`	`NYUDV2`	same

Using a generic class on data that requires unit conversion (e.g. raw NYU uint16 PNGs) results in an empty valid mask and silent train_loss = NaN. Match the class to your data's encoding.

For relative mono data (RelativeMonoDataset or NYUDV2Relative), leave dataset.min_depth and dataset.max_depth unset or set both to null. Non-null metric depth ranges are passed into the relative dataset constructor and fail with BaseRelativeMonoDataset.__init__() got an unexpected keyword argument 'min_depth'.

Step 3 — Write spec yaml from Typical Spec Overrides

Copy the action block from Typical Spec Overrides (references/spec-overrides.md). Replace:

model.model_type from Step 2
dataset.<...>.data_sources[*].dataset_name from Step 2
data_sources[*].data_file with the path from Step 1 (S3 path under SDK runner, host path for direct docker)
For metric finetune: additionally apply the Metric Variant Finetuning Recipe in references/finetuning-recipes.md.

For mono training set train.precision: fp32 (recommended) or bf16 (Ampere SM80+, alternative).

Step 4 — Run

Create writable home/cache directories inside the mounted output path before using --user. Some TAO containers do not have an /etc/passwd entry for the host UID, and PyTorch / matplotlib need writable cache paths when running as that UID.

mkdir -p <output_dir>/home \
         <output_dir>/.cache/matplotlib \
         <output_dir>/.cache/torchinductor \
         <output_dir>/.cache/xdg

docker run --gpus 'device=0' --shm-size 16G --ipc=host \
  --user "$(id -u):$(id -g)" \
  -e USER="$(id -un)" \
  -e LOGNAME="$(id -un)" \
  -e HOME=<output_dir>/home \
  -e MPLCONFIGDIR=<output_dir>/.cache/matplotlib \
  -e TORCHINDUCTOR_CACHE_DIR=<output_dir>/.cache/torchinductor \
  -e XDG_CACHE_HOME=<output_dir>/.cache/xdg \
  -v <data_root>:<data_root>:ro \
  -v <output_dir>:<output_dir> \
  <container> \
  depth_net <action> -e <spec.yaml>

Without --user "$(id -u):$(id -g)" the container writes outputs as nobody:nogroup, blocking host-side cleanup and retry.

Step 5 — Verify

Container exit code 0
status.json kpi block populated
For train: inspect per-step train_loss directly — the entrypoint reports Execution status: PASS even when train_loss = NaN (see the Metric Variant Finetuning Recipe → Sanity-run PASS criteria in references/finetuning-recipes.md)
For evaluate / inference: artifacts under results_dir

For TAO Deploy TensorRT actions (gen_trt_engine, TensorRT evaluate, and TensorRT inference), read references/tao-deploy-depth-anything-v2.md first. Deploy spec templates live in this skill's references/ folder with the spec_template_deploy_*.yaml prefix.

Training Requirements

Valid `dataset_name` values for mono `data_sources` (case-insensitive): ThreeDVLM, FSD, NvCLIP, IssacStereo, Crestereo, Middlebury, NYUDV2, NYUDV2Relative, RelativeMonoDataset, MetricMonoDataset. NYUDV2 carries metric depth GT (meters) — pair with MetricDepthAnything; NYUDV2Relative is the same data with relative-depth conventions — pair with RelativeDepthAnything.
Monitoring metric: val/d1, val/loss
For AutoML sanity runs on the packaged relative-depth smoke data, use val/d1 as the primary monitor. val/loss can be emitted as NaN even when the trainer exits successfully and writes a usable checkpoint, so it is not a reliable AutoML objective unless the run's status metrics show a finite value.

Per-Action Dataset Requirements

Action	Spec Key	Source	Files	List?
evaluate	dataset.test_dataset.data_sources	eval_dataset	data_file: annotations.txt + dataset_name	Yes
inference	dataset.infer_dataset.data_sources	inference_dataset	data_file: annotations.txt + dataset_name	Yes
quantize	dataset.train_dataset.data_sources	train_datasets	data_file: annotations.txt + dataset_name	Yes
quantize	dataset.val_dataset.data_sources	eval_dataset	data_file: annotations.txt + dataset_name	Yes
quantize	dataset.quant_calibration_dataset.images_dir	train_datasets	images.tar.gz	No
train	dataset.train_dataset.data_sources	train_datasets	data_file: annotations.txt + dataset_name	Yes
train	dataset.val_dataset.data_sources	eval_dataset	data_file: annotations.txt + dataset_name	Yes

Typical Spec Overrides

Data source overrides are mandatory for every action — construct data source paths from the Per-Action Dataset Requirements table above and include them in spec_overrides. Each data_sources entry is a dict with two mandatory fields: data_file and dataset_name. See references/spec-overrides.md for the full per-action override blocks (train, evaluate, export, inference, quantize), the S3_TRAIN / S3_EVAL placeholders, the relative-variant precision recommendation, and the quantize known-issue note.

Eval Dataset

Optional. Val dataset configured via dataset.val_dataset.data_sources (each entry needs data_file and dataset_name).

Important Parameters

See references/parameters.md for the full parameter glossary (model, train, dataset, export, and inference keys with options, defaults, and sources) and the Pretrained checkpoint loading — use case matrix.

Finetuning Recipes

See references/finetuning-recipes.md for:

Relative Variant Finetuning Recipe — finetune from a TAO-trained RelativeDepthAnything checkpoint (lr 5e-6, LambdaLR, sanity-vs-convergent guidance, deploy LSQ alignment note).
Metric Variant Finetuning Recipe — checkpoint compatibility, required overrides, the dataset normalization block (normalize_depth/min_depth/max_depth) required in train AND export specs, trainer-enforced defaults, precision, the 1-epoch sanity-run override, and the Sanity-run PASS criteria with the NaN-mitigation order.

Multi-GPU / Multi-Node

Launch method: Lightning-managed (single python process, Lightning spawns workers).

Spec Key	Description	Default
`train.num_gpus`	Number of GPUs	1
`train.gpu_ids`	GPU device indices	[0]
`train.num_nodes`	Number of nodes	1
`train.distributed_strategy`	`ddp` or `fsdp`	`ddp`

ddp with activation checkpointing: find_unused_parameters=False
ddp without: find_unused_parameters=True
fsdp forces precision to FP16

Multi-node env vars (set by orchestrator): WORLD_SIZE, NODE_RANK, MASTER_ADDR, MASTER_PORT, NUM_GPU_PER_NODE.

Export / TRT Defaults

TRT data types: FP32, BF16 (Ampere SM80+). FP16 is not supported for the ViT-L mono backbone.
Fresh-install TRT precision: fp32. BF16 is supported on Ampere SM80+ hardware, but keep smoke tests on FP32 unless the user explicitly requests BF16.

Hardware

Minimum 1 GPU(s), recommended 2 GPU(s). 24GB+ VRAM per GPU. ViT-Large encoder is memory intensive. Use fp32 (recommended) or bf16 (Ampere SM80+, alternative) for training. Activation checkpointing is available for larger inputs.

Error Patterns

See references/troubleshooting.md for the full error-pattern catalog (depth range mismatch, relative dataset rejecting min_depth, missing pretrained weights, encoder key location, dataset_name not in struct, depth_net_mono not found, metric variant hyperparameter sourcing, and export ONNX overwrite).

Spec Param / Parent Model Inference

See references/spec-param-inference.md for the model-specific inference mappings (the TAO Core depth_net_mono.config.json action table), checkpoint-file naming under <results_dir>/train/, the dn_model_latest.pth policy, the parent-gen_trt_engine rationale, and the parent_model / parent_job_id resolution rules.

Deployment

tao-deploy-depth-anything-v2

Evaluation Report

Evaluation of the tao-train-depth-anything-v2 skill before publication through NVSkills-Eval.

This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use.

Evaluation Summary

Skill: tao-train-depth-anything-v2
Evaluation date: 2026-06-22
NVSkills-Eval profile: external
Environment: astra-sandbox
Dataset: 1 evaluation tasks
Attempts per task: 1
Pass threshold: 50%
Overall verdict: PASS

Agents Used

claude-code
codex

Metrics Used

Reported benchmark dimensions:

Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
Correctness: checks whether the agent follows the expected workflow and produces the correct final output.
Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
Effectiveness: checks whether the agent performs measurably better with the skill than without it.
Efficiency: checks whether the agent uses fewer tokens and avoids redundant work.

Underlying evaluation signals used in this run:

security (Security): checks for unsafe operations, secret leakage, and unauthorized access.
skill_execution (Skill Execution): verifies that the agent loaded the expected skill and workflow.
skill_efficiency (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage.
accuracy (Accuracy): grades final-answer correctness against the reference answer.
goal_accuracy (Goal Accuracy): checks whether the overall user task completed successfully.
behavior_check (Behavior Check): verifies expected behavior steps, including safety expectations.
token_efficiency (Token Efficiency): compares token usage with and without the skill.

Test Tasks

The benchmark dataset contained 1 evaluation tasks:

Positive tasks: 1 tasks where the skill was expected to activate.
Negative tasks: 0 tasks where no skill was expected.
Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred.

Task composition is derived from the evaluation dataset when possible. Entries with expected_skill set are treated as positive skill-activation cases, while entries with expected_skill: null are treated as negative activation cases.

Results

Dimension	Num	`claude-code`	`codex`
Security	1	100% (+0%)	100% (+0%)
Correctness	1	100% (+90%)	20% (+20%)
Discoverability	1	88% (+88%)	0% (+0%)
Effectiveness	1	100% (+72%)	48% (+30%)
Efficiency	1	71% (+44%)	28% (+0%)

Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available.

Tier 1: Static Validation Summary

Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 5 total findings.

Top findings:

MEDIUM SCHEMA/folder_hierarchy: Unexpected nesting depth for general skill (skills/models/tao-train-depth-anything-v2)
MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (skills/models/tao-train-depth-anything-v2/SKILL.md)
MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (skills/models/tao-train-depth-anything-v2/SKILL.md)
LOW SCHEMA/unexpected_file: Unexpected 'schemas' in skill root (skills/models/tao-train-depth-anything-v2/schemas)
LOW SCHEMA/author_format: Author must be of the form 'Name <email@host>' (skills/models/tao-train-depth-anything-v2/SKILL.md)

Tier 2: Deduplication Summary

This tier was not run or did not produce findings in this report.

Publication Recommendation

The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change.

[
  {
    "id": "tao-train-depth-anything-v2-basic",
    "question": "A user request: \"Monocular depth estimation using Metric Depth Anything v2 or Relative Depth Anything architectures.\" Identify which TAO skill applies and, reading only that skill's documentation, outline the steps it prescribes. Do NOT run any commands, scripts, web searches, or other tools \u2014 describe the plan only.",
    "expected_skill": "tao-train-depth-anything-v2",
    "expected_script": null,
    "ground_truth": "Identify tao-train-depth-anything-v2 as the applicable skill and summarize its documented workflow from SKILL.md without executing anything.",
    "expected_behavior": [
      "Identifies tao-train-depth-anything-v2 as the relevant skill",
      "Outlines the documented workflow steps from SKILL.md",
      "Does not run commands, scripts, or web searches"
    ]
  }
]

Finetuning Recipes

Relative Variant Finetuning Recipe

Relative finetune from a TAO-trained RelativeDepthAnything checkpoint:

Spec key	Value	Notes
`model.model_type`	`RelativeDepthAnything`
`model.encoder`	`vitl`	matches the released TAO relative checkpoint
`model.mono_backbone.pretrained_path`	`""`	the full TAO checkpoint already carries the backbone state; setting this is redundant and is overwritten by the full-state load
`train.pretrained_model_path`	`<TAO relative ckpt>`	full Pytorch-Lightning state load
`train.precision`	`fp32` (recommended) or `bf16` (alternative on Ampere SM80+)
`train.optim.lr`	`5e-6`	The released relative checkpoint is already converged; the AdamW default `1e-4` listed in Important Parameters is an order of magnitude too aggressive for finetune from a converged backbone, and degrades the released checkpoint's accuracy on a short adaptation run. Use `5e-6` and a gentle scheduler (`LambdaLR`) when adapting to a new dataset.
`train.optim.lr_scheduler`	`LambdaLR`	gentle warmup + decay; matches the Metric Variant Recipe

The dataset block follows Step 2 — Pair `model_type` and `dataset_name` in SKILL.md. Use RelativeMonoDataset for generic relative data and NYUDV2Relative for raw NYU sync_depth_*.png data.

If the goal is a sanity check (1-epoch loss-decreasing, exit 0) rather than convergent finetune, use the released checkpoint directly for evaluate / inference / export instead of running train — a 1-epoch finetune at any LR is unlikely to reach the released benchmark and will measure the warmup transient, not skill correctness.

The relative variant emits scale-shift-invariant disparity (unbounded). The deploy-side evaluator runs LSQ alignment + GT disparity inversion; ensure the deploy spec sets model.model_type: RelativeDepthAnything so those paths engage (see references/tao-deploy-depth-anything-v2.md).

Metric Variant Finetuning Recipe

Checkpoint compatibility: The Metric variant only loads checkpoints trained with TAO's MetricDepthAnythingV2 model definition. Public Depth Anything v2 metric checkpoints (e.g., from the Depth Anything V2 GitHub release) use a different head attribute naming convention and will fail with Unexpected key(s) in state_dict: "model.depth_head.*" when passed to train.pretrained_model_path, evaluate.checkpoint, inference.checkpoint, or export.checkpoint. Use a TAO-trained metric checkpoint (or a TAO-converted equivalent) for all metric actions.

Metric finetuning uses a pretrained RelativeDepthAnything ViT-L backbone via model.mono_backbone.pretrained_path, with the metric head (metric_depth_head) initialized from scratch and no full PL state load (train.pretrained_model_path: ""). Because the backbone weights are already well-trained, the optimizer must step gently to preserve those features while the metric head converges; use train.optim.lr: 5e-6 (20× lower than the AdamW default 1e-4 listed in Important Parameters) with LambdaLR.

The TAO repository ships an authoritative reference spec at nvidia_tao_pytorch/cv/depth_net/experiment_specs/experiment_mono_metric.yaml; metric finetuning must mirror its optimizer settings unless the user has empirical evidence to deviate.

Required overrides for metric finetuning from a relative backbone:

Spec key	Recommended value	Source
`train.optim.lr`	`0.000005` (5e-6)	`experiment_mono_metric.yaml:39` — preserves the pretrained relative backbone while the from-scratch metric head converges. The AdamW default `1e-4` is too aggressive on this backbone-pretrained setup.
`train.optim.lr_scheduler`	`LambdaLR`	`experiment_mono_metric.yaml:40`
`model.mono_backbone.pretrained_path`	`<RelativeDepthAnything TAO ckpt>`	`experiment_mono_metric.yaml:45` — backbone-only load via `parse_lighting_checkpoint_to_backbone`; metric head reinitializes
`train.pretrained_model_path`	`""`	omit a full PL state load to keep the metric head from inheriting any pre-existing head weights

Dataset normalization block — required in train AND export specs:

dataset:
  dataset_name: MonoDataset
  normalize_depth: false   # NYU-trained metric checkpoint default
  min_depth: 0.001
  max_depth: 10.0

These three fields must mirror the values from the trained checkpoint's training spec in both the train action spec and the export action spec. The export pipeline reads dataset.{normalize_depth, min_depth, max_depth} to build the model graph the ONNX is traced from; omitting them makes the export silently use schema defaults that do not match the checkpoint, producing a serialized graph whose deploy-side evaluator output is non-physical even though the export action itself returns exit 0. Read the authoritative values from the checkpoint's sibling experiment.yaml.

Defaults already enforced by the TAO trainer (do not need to be set):

train.clip_grad_norm: 0.1 (clip-by-value at the Lightning Trainer(gradient_clip_val=..., gradient_clip_algorithm="value") level — nvidia_tao_pytorch/cv/depth_net/scripts/train.py:94-95).
train.optim.warmup_steps: 20 (linear LR warmup before the configured scheduler engages).
train.optim.weight_decay: 1e-4 (AdamW).

Precision: use fp32 for the metric finetune. The from-scratch metric head + low lr combination is fragile under reduced precision; fp32 is the safe default for this Recipe.

Sanity-run override (1-epoch loss-decreasing check on a small NYU subset):

train:
  num_epochs: 1
  pretrained_model_path: ""
  precision: fp32
  optim:
    lr: 0.000005
    lr_scheduler: LambdaLR
model:
  model_type: MetricDepthAnything
  encoder: vitl
  mono_backbone:
    pretrained_path: /workspace/models/<relative_ckpt>.pth
    use_bn: False
    use_clstoken: False

A 1-epoch run with metric_depth_head random init will not reach released-checkpoint metric quality (that requires multi-epoch training); the recipe's purpose is functional sanity (exit 0 + loss decreasing + no NaN).

Sanity-run PASS criteria — entrypoint `Execution status: PASS` is not sufficient:

The trainer's Execution status: PASS only signals epoch completion — it does not check for train_loss = NaN. A from-scratch metric head with low learning rate can produce train_loss = NaN; on relative-depth smoke data, val/loss can also be NaN while finite validation accuracies such as val/d1 are still emitted. Inspect the train_loss_step values in the run log directly; PASS means only if the values are finite and decreasing.

Mitigations to try in order if NaN is observed:

Increase dataset.train_dataset.batch_size to 2 or higher (the per-batch variance computation has unstable degrees-of-freedom at batch_size 1).
Increase train.optim.warmup_steps from the default 20 (the LambdaLR factor at step 0 is 0, producing a no-op first update; the second step then sees a head still at random init).
If both mitigations fail, fall back to reusing a pre-trained TAO metric checkpoint via train.pretrained_model_path: <metric_ckpt> and skip the from-scratch metric-head path entirely.

Important Parameters

model.model_type: Model architecture. Options: MetricDepthAnything, RelativeDepthAnything. Default MetricDepthAnything.
model.encoder: Backbone encoder (top-level model field, not nested under mono_backbone). Options: vits, vitb, vitl, vitg. Default vitl.
model.mono_backbone.pretrained_path: Path to DINOv2 ViT-L encoder weights (used for Relative train-from-scratch only — Metric and Relative finetune use train.pretrained_model_path + a TAO ckpt instead; see use-case matrix below). Architecturally identical to the DepthAnything v2 encoder (same ViT-L), but the weights differ: DINOv2 is the self-supervised pretraining used to initialize the Relative DepthAnything encoder before depth-supervised training. Set to an empty string ("") to skip the backbone-only weight load — use this when the full TAO checkpoint is supplied via train.pretrained_model_path (Pytorch-Lightning state) or evaluate.checkpoint / inference.checkpoint, since those carry the backbone state already. Setting both is redundant; the backbone-only load happens first and is then overwritten by the full-state load.
model.mono_backbone.use_bn / model.mono_backbone.use_clstoken: Backbone toggles. Booleans. Defaults: use_bn: False, use_clstoken: False (matches the released RelativeDepthAnything and MetricDepthAnything checkpoint architectures). Override only when training a custom variant whose checkpoint was produced with the alternate setting.
train.optim.lr: Learning rate. Default 1e-4 (AdamW).
train.lr_scheduler: LR scheduler. Options: MultiStepLR, StepLR, CustomMultiStepLRScheduler, LambdaLR, PolynomialLR, OneCycleLR, CosineAnnealingLR.
train.precision: Training precision. Options: fp32 (recommended), bf16 (Ampere SM80+, alternative), fp16.
train.distributed_strategy: Distribution strategy. Options: ddp, fsdp.
train.activation_checkpoint: Enable activation checkpointing. Default False.
dataset.dataset_name: Top-level dataset family identifier (e.g., MonoDataset).
dataset.{train,val,test,infer}_dataset.batch_size: Per-split batch size.
dataset.{train,val,test,infer}_dataset.workers: Per-split DataLoader worker count (the field name is workers, not num_workers).
dataset.{train,val,test,infer}_dataset.augmentation.crop_size: Per-split crop size. Default [518, 518]. For Depth Anything ViT encoders, each spatial dimension must be divisible by the patch size (14 for vits/vitb/vitl/vitg).
dataset.{train,val,test,infer}_dataset.data_sources: List of {data_file, dataset_name} dicts. Both fields are mandatory per entry.
dataset.max_depth / dataset.min_depth: Top-level depth range for metric depth estimation. Set both to null or omit them for relative mono datasets.
export.input_channel: ONNX input channel count. Default 3 (RGB), matching the runtime input expected by RelativeDepthAnythingV2 / MetricDepthAnythingV2. Source: experiment_mono_relative.yaml export block.
export.input_height / export.input_width: ONNX input spatial dims. Default 518 / 518, matching the model's training-time crop. Override only when targeting a different deployment input shape — the model's positional embeddings constrain practical shapes to multiples of the patch size (14 for ViT-L).
export.opset_version: ONNX opset target. Default 17 (native LayerNormalization op for fp16 stability). Source: experiment_mono_relative.yaml export block.
export.on_cpu: Whether ONNX export runs on CPU. Default False (uses export.gpu_id). Source: experiment_mono_relative.yaml export block.
export.gpu_id: GPU device index for ONNX export when on_cpu: False. Default 0. Source: experiment_mono_relative.yaml export block. Should match the --gpus '"device=N"' flag passed to docker run.
export.batch_size: ONNX batch size. 1 = static, -1 = batch axis dynamic. Height and width are always taken from the trace shape; H/W dynamic is not supported. Default -1.
inference.save_raw_pfm: Whether the inference action additionally writes raw single-channel disparity as .pfm files alongside the visualization JPGs. Default False. Source: experiment_mono_relative.yaml inference block. Set True for downstream metric computation; raw disparity is unbounded scale-shift-invariant for RelativeDepthAnything and bounded to [min_depth, max_depth] for MetricDepthAnything. With the default, the inference action emits a 240×960 RGB JPG triptych under <results_dir>/inference/inference_images/ mirroring the source dataset's directory tree.

Pretrained checkpoint loading — use case matrix

Use case	`model.mono_backbone.pretrained_path`	`train.pretrained_model_path`
Relative — train from scratch (DINOv2 backbone weights only)	`<DINOv2 ViT-L weights>`	`""`
Relative — finetune from TAO relative checkpoint	`""`	`<TAO relative ckpt>`
Metric — train from scratch on top of relative backbone (sanity)	`<TAO relative ckpt>`	`""`
Metric — finetune from TAO metric checkpoint	`""`	`<TAO metric ckpt>`

Setting both keys is redundant: the backbone-only load happens first and is overwritten by the full-state load. The metric variant requires the MetricDepthAnythingV2 head naming (metric_depth_head.*); see Checkpoint compatibility in references/finetuning-recipes.md.

name: tao-train-depth-anything-v2
network_arch: depth_net_mono
automl_enabled: true
container_image: tao_toolkit.pyt
data_format: RelativeMonoDataset
gpu_spec_key: train.num_gpus
actions:
  train:
    command: depth_net train -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      dataset.train_dataset.data_sources[0].data_file:
        type: file
      dataset.val_dataset.data_sources[0].data_file:
        type: file
      model.mono_backbone.pretrained_path:
        type: file
        optional: true
      train.pretrained_model_path:
        type: file
        optional: true
      train.resume_training_checkpoint_path:
        type: file
        optional: true
    outputs:
      results_dir:
        type: folder
    upload_excludes:
    - inputs/
  quantize:
    command: depth_net quantize -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      quantize.model_path:
        type: file
      dataset.train_dataset.data_sources[0].data_file:
        type: file
      dataset.val_dataset.data_sources[0].data_file:
        type: file
      dataset.quant_calibration_dataset.images_dir:
        type: folder
    outputs:
      results_dir:
        type: folder
    upload_excludes:
    - inputs/
  evaluate:
    command: depth_net evaluate -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      dataset.test_dataset.data_sources[0].data_file:
        type: file
      evaluate.checkpoint:
        type: file
        optional: true
      evaluate.trt_engine:
        type: file
        optional: true
    outputs:
      results_dir:
        type: folder
    upload_excludes:
    - inputs/
  export:
    command: depth_net export -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      export.checkpoint:
        type: file
    outputs:
      results_dir:
        type: folder
      export.onnx_file:
        type: file
    upload_excludes:
    - inputs/
  inference:
    command: depth_net inference -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      dataset.infer_dataset.data_sources[0].data_file:
        type: file
      inference.checkpoint:
        type: file
        optional: true
      inference.trt_engine:
        type: file
        optional: true
    outputs:
      results_dir:
        type: folder
    upload_excludes:
    - inputs/
  gen_trt_engine:
    container_image: tao_toolkit.deploy
    command: depth_net gen_trt_engine -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      gen_trt_engine.onnx_file:
        type: file
    outputs:
      results_dir:
        type: folder
      gen_trt_engine.trt_engine:
        type: file
    upload_excludes:
    - inputs/
data_sources: {}
spec_params:
  train:
    dataset.dataset_name: MonoDataset
    model.mono_backbone.pretrained_path:
      link: https://dl.fbaipublicfiles.com/dinov2/dinov2_vitl14/dinov2_vitl14_pretrain.pth
      destination_path: /ptm/depth_net/mono_backbone/dinov2_vitl14_pretrain.pth
    results_dir: output_dir
    train.pretrained_model_path: ptm_if_no_resume_model
    train.resume_training_checkpoint_path: resume_model
  quantize:
    dataset.dataset_name: MonoDataset
    quantize.model_path: parent_model
    results_dir: output_dir
  evaluate:
    dataset.dataset_name: MonoDataset
    evaluate.checkpoint: parent_model
    evaluate.trt_engine: parent_model
    results_dir: output_dir
  export:
    dataset.dataset_name: MonoDataset
    export.checkpoint: parent_model
    export.onnx_file: create_onnx_file
    results_dir: output_dir
  inference:
    dataset.dataset_name: MonoDataset
    inference.checkpoint: parent_model
    inference.trt_engine: parent_model
    results_dir: output_dir
  gen_trt_engine:
    gen_trt_engine.onnx_file: parent_model
    gen_trt_engine.trt_engine: create_engine_file
    results_dir: output_dir
key_defaults: {}
spec_shorthand_keys:
  num_epochs: train.num_epochs
  train_batch_size: dataset.train_dataset.batch_size
  infer_batch_size: dataset.infer_dataset.batch_size
  learning_rate: train.optim.lr
description: Monocular depth estimation using Metric Depth Anything v2 or Relative Depth Anything architectures. Predicts
  per-pixel depth from single RGB images. Mono and stereo share the unified `depth_net` CLI entrypoint;
  model family is selected via `model.model_type`.

results_dir: /results
model:
  # Required. Must match the trained model variant.
  # Options: RelativeDepthAnything (relative variant) | MetricDepthAnything (metric variant).
  # Omitting this block lets the schema default ("MetricDepthAnything") miscategorize a relative
  # engine and bypass the LSQ alignment + GT disparity inversion paths in the deploy evaluator.
  model_type: RelativeDepthAnything
dataset:
  dataset_name: MonoDataset
  infer_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: /data/annotations.txt
    batch_size: 1
    workers: 4
    augmentation:
      crop_size: [518, 686]   # MUST match export.input_{height,width}.
                              # The deploy runtime selects input H/W from this field
                              # (`evaluate.input_height/input_width` is currently decorative);
                              # leaving it unset falls back to the [518, 518] default in tao-core,
                              # which silently overrides the engine shape and produces non-physical KPIs.
  test_dataset:
    data_sources:
    - dataset_name: NYUDV2Relative
      data_file: /data/annotations.txt
    batch_size: 1
    workers: 4
    augmentation:
      crop_size: [518, 686]   # match export.input_{height,width}
inference:
  trt_engine: /results/depth-net-mono.engine
  input_width: 686
  input_height: 518
evaluate:
  trt_engine: /results/depth-net-mono.engine
  input_width: 686
  input_height: 518
gen_trt_engine:
  gpu_id: 0
  onnx_file: /models/model.onnx
  trt_engine: /results/depth-net-mono.engine
  batch_size: -1
  tensorrt:
    data_type: fp32   # fresh-install safe default; bf16 is supported on Ampere SM80+ hardware
    workspace_size: 4 # GiB in the current TAO Deploy image
    min_batch_size: 1
    opt_batch_size: 1
    max_batch_size: 4
  verbose: false  # Set true only when collecting detailed TensorRT builder diagnostics.
  # Engine input H, W are pinned to the trace shape (from export.input_height /
  # export.input_width). Mono engines do not support H/W-dynamic profiles —
  # build a separate engine per (H, W) target.

model_name: ''
encryption_key: ''
results_dir: ''
wandb:
  enable: true
  project: TAO Toolkit
  entity: ''
  group: ''
  tags:
  - tao-toolkit
  reinit: false
  sync_tensorboard: false
  save_code: false
  name: TAO Toolkit Training
  run_id: ''
dataset:
  dataset_name: MonoDataset
  normalize_depth: false
  max_disparity: 416
  baseline: 0.193001
  focal_x: 1998.842
  train_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  val_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  test_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  infer_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  quant_calibration_dataset:
    images_dir: ''
model:
  model_type: RelativeDepthAnything
  mono_backbone:
    pretrained_path: ''
    use_bn: false
    use_clstoken: false
  stereo_backbone:
    depth_anything_v2_pretrained_path: ''
    edgenext_pretrained_path: ''
    use_bn: false
    use_clstoken: false
  hidden_dims:
  - 128
  - 128
  - 128
  corr_radius: 4
  cv_group: 8
  train_iters: 22
  valid_iters: 22
  volume_dim: 32
  low_memory: 0
  mixed_precision: false
  n_gru_layers: 3
  corr_levels: 2
  n_downsample: 2
  encoder: vitl
  max_disparity: 416
evaluate:
  num_gpus: 1
  gpu_ids:
  - 0
  num_nodes: 1
  checkpoint: ???
  trt_engine: ''
  results_dir: ''
  batch_size: -1
  input_width: 736
  input_height: 320
train:
  num_gpus: 1
  gpu_ids:
  - 0
  num_nodes: 1
  seed: 1234
  cudnn:
    benchmark: false
    deterministic: true
  num_epochs: 10
  checkpoint_interval: 1
  checkpoint_interval_unit: epoch
  validation_interval: 1
  resume_training_checkpoint_path: ''
  results_dir: ''
  pretrained_model_path: ''
  clip_grad_norm: 0.1
  dataloader_visualize: false
  vis_step_interval: 10
  is_dry_run: false
  optim:
    optimizer: AdamW
    monitor_name: val_loss
    lr: 0.0001
    momentum: 0.9
    weight_decay: 0.0001
    lr_scheduler: MultiStepLR
    lr_steps:
    - 1000
    lr_step_size: 1000
    lr_decay: 0.1
    min_lr: 1.0e-07
    warmup_steps: 20
  precision: fp32
  distributed_strategy: ddp
  activation_checkpoint: false
  inference_tile: false
  tile_wtype: gaussian
  tile_min_overlap:
  - 16
  - 16
  log_every_n_steps: 500
quantize:
  backend: torchao
  mode: weight_only_ptq
  algorithm: minmax
  layers: []
  skip_names: []
  model_path: ''
  results_dir: ''
  backend_kwargs: {}
  device: cuda

model_name: ''
encryption_key: ''
results_dir: ''
wandb:
  enable: true
  project: TAO Toolkit
  entity: ''
  group: ''
  tags:
  - tao-toolkit
  reinit: false
  sync_tensorboard: false
  save_code: false
  name: TAO Toolkit Training
  run_id: ''
dataset:
  dataset_name: MonoDataset
  normalize_depth: false
  max_disparity: 416
  baseline: 0.193001
  focal_x: 1998.842
  train_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  val_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  test_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  infer_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  quant_calibration_dataset:
    images_dir: ''
model:
  model_type: RelativeDepthAnything
  mono_backbone:
    pretrained_path: ''
    use_bn: false
    use_clstoken: false
  stereo_backbone:
    depth_anything_v2_pretrained_path: ''
    edgenext_pretrained_path: ''
    use_bn: false
    use_clstoken: false
  hidden_dims:
  - 128
  - 128
  - 128
  corr_radius: 4
  cv_group: 8
  train_iters: 22
  valid_iters: 22
  volume_dim: 32
  low_memory: 0
  mixed_precision: false
  n_gru_layers: 3
  corr_levels: 2
  n_downsample: 2
  encoder: vitl
  max_disparity: 416
train:
  num_gpus: 1
  gpu_ids:
  - 0
  num_nodes: 1
  seed: 1234
  cudnn:
    benchmark: false
    deterministic: true
  num_epochs: 10
  checkpoint_interval: 1
  checkpoint_interval_unit: epoch
  validation_interval: 1
  resume_training_checkpoint_path: ''
  results_dir: ''
  pretrained_model_path: ''
  clip_grad_norm: 0.1
  dataloader_visualize: false
  vis_step_interval: 10
  is_dry_run: false
  optim:
    optimizer: AdamW
    monitor_name: val_loss
    lr: 0.0001
    momentum: 0.9
    weight_decay: 0.0001
    lr_scheduler: MultiStepLR
    lr_steps:
    - 1000
    lr_step_size: 1000
    lr_decay: 0.1
    min_lr: 1.0e-07
    warmup_steps: 20
  precision: fp32
  distributed_strategy: ddp
  activation_checkpoint: false
  inference_tile: false
  tile_wtype: gaussian
  tile_min_overlap:
  - 16
  - 16
  log_every_n_steps: 500
export:
  results_dir: ''
  gpu_id: 0
  checkpoint: ???
  onnx_file: ???
  on_cpu: false
  input_channel: 3
  input_width: 960
  input_height: 544
  opset_version: 17
  batch_size: -1
  verbose: false
  format: onnx
  valid_iters: 22
quantize:
  backend: torchao
  mode: weight_only_ptq
  algorithm: minmax
  layers: []
  skip_names: []
  model_path: ''
  results_dir: ''
  backend_kwargs: {}
  device: cuda

model_name: ''
encryption_key: ''
results_dir: ''
wandb:
  enable: true
  project: TAO Toolkit
  entity: ''
  group: ''
  tags:
  - tao-toolkit
  reinit: false
  sync_tensorboard: false
  save_code: false
  name: TAO Toolkit Training
  run_id: ''
dataset:
  dataset_name: StereoDataset
  normalize_depth: false
  max_disparity: 416
  baseline: 0.193001
  focal_x: 1998.842
  train_dataset:
    data_sources:
    - dataset_name: ''
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  val_dataset:
    data_sources:
    - dataset_name: ''
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  test_dataset:
    data_sources:
    - dataset_name: ''
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  infer_dataset:
    data_sources:
    - dataset_name: ''
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  quant_calibration_dataset:
    images_dir: ''
model:
  model_type: MetricDepthAnything
  mono_backbone:
    pretrained_path: ''
    use_bn: false
    use_clstoken: false
  stereo_backbone:
    depth_anything_v2_pretrained_path: ''
    edgenext_pretrained_path: ''
    use_bn: false
    use_clstoken: false
  hidden_dims:
  - 128
  - 128
  - 128
  corr_radius: 4
  cv_group: 8
  train_iters: 22
  valid_iters: 22
  volume_dim: 32
  low_memory: 0
  mixed_precision: false
  n_gru_layers: 3
  corr_levels: 2
  n_downsample: 2
  encoder: vitl
  max_disparity: 416
train:
  num_gpus: 1
  gpu_ids:
  - 0
  num_nodes: 1
  seed: 1234
  cudnn:
    benchmark: false
    deterministic: true
  num_epochs: 10
  checkpoint_interval: 1
  checkpoint_interval_unit: epoch
  validation_interval: 1
  resume_training_checkpoint_path: ''
  results_dir: ''
  pretrained_model_path: ''
  clip_grad_norm: 0.1
  dataloader_visualize: false
  vis_step_interval: 10
  is_dry_run: false
  optim:
    optimizer: AdamW
    monitor_name: val_loss
    lr: 0.0001
    momentum: 0.9
    weight_decay: 0.0001
    lr_scheduler: MultiStepLR
    lr_steps:
    - 1000
    lr_step_size: 1000
    lr_decay: 0.1
    min_lr: 1.0e-07
    warmup_steps: 20
  precision: fp32
  distributed_strategy: ddp
  activation_checkpoint: false
  inference_tile: false
  tile_wtype: gaussian
  tile_min_overlap:
  - 16
  - 16
  log_every_n_steps: 500
gen_trt_engine:
  results_dir: ''
  gpu_id: 0
  onnx_file: ???
  trt_engine: ???
  timing_cache: ''
  batch_size: -1
  verbose: false
  tensorrt:
    workspace_size: 1024
    min_batch_size: 1
    opt_batch_size: 1
    max_batch_size: 1
    layers_precision: []
    data_type: FP32
quantize:
  backend: torchao
  mode: weight_only_ptq
  algorithm: minmax
  layers: []
  skip_names: []
  model_path: ''
  results_dir: ''
  backend_kwargs: {}
  device: cuda

model_name: ''
encryption_key: ''
results_dir: ''
wandb:
  enable: true
  project: TAO Toolkit
  entity: ''
  group: ''
  tags:
  - tao-toolkit
  reinit: false
  sync_tensorboard: false
  save_code: false
  name: TAO Toolkit Training
  run_id: ''
dataset:
  dataset_name: MonoDataset
  normalize_depth: false
  max_disparity: 416
  baseline: 0.193001
  focal_x: 1998.842
  train_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  val_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  test_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  infer_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  quant_calibration_dataset:
    images_dir: ''
model:
  model_type: RelativeDepthAnything
  mono_backbone:
    pretrained_path: ''
    use_bn: false
    use_clstoken: false
  stereo_backbone:
    depth_anything_v2_pretrained_path: ''
    edgenext_pretrained_path: ''
    use_bn: false
    use_clstoken: false
  hidden_dims:
  - 128
  - 128
  - 128
  corr_radius: 4
  cv_group: 8
  train_iters: 22
  valid_iters: 22
  volume_dim: 32
  low_memory: 0
  mixed_precision: false
  n_gru_layers: 3
  corr_levels: 2
  n_downsample: 2
  encoder: vitl
  max_disparity: 416
inference:
  num_gpus: 1
  gpu_ids:
  - 0
  num_nodes: 1
  checkpoint: ???
  trt_engine: ''
  results_dir: ''
  batch_size: -1
  conf_threshold: 0.5
  save_raw_pfm: false
train:
  num_gpus: 1
  gpu_ids:
  - 0
  num_nodes: 1
  seed: 1234
  cudnn:
    benchmark: false
    deterministic: true
  num_epochs: 10
  checkpoint_interval: 1
  checkpoint_interval_unit: epoch
  validation_interval: 1
  resume_training_checkpoint_path: ''
  results_dir: ''
  pretrained_model_path: ''
  clip_grad_norm: 0.1
  dataloader_visualize: false
  vis_step_interval: 10
  is_dry_run: false
  optim:
    optimizer: AdamW
    monitor_name: val_loss
    lr: 0.0001
    momentum: 0.9
    weight_decay: 0.0001
    lr_scheduler: MultiStepLR
    lr_steps:
    - 1000
    lr_step_size: 1000
    lr_decay: 0.1
    min_lr: 1.0e-07
    warmup_steps: 20
  precision: fp32
  distributed_strategy: ddp
  activation_checkpoint: false
  inference_tile: false
  tile_wtype: gaussian
  tile_min_overlap:
  - 16
  - 16
  log_every_n_steps: 500
quantize:
  backend: torchao
  mode: weight_only_ptq
  algorithm: minmax
  layers: []
  skip_names: []
  model_path: ''
  results_dir: ''
  backend_kwargs: {}
  device: cuda

model_name: ''
encryption_key: ''
results_dir: ''
wandb:
  enable: true
  project: TAO Toolkit
  entity: ''
  group: ''
  tags:
  - tao-toolkit
  reinit: false
  sync_tensorboard: false
  save_code: false
  name: TAO Toolkit Training
  run_id: ''
dataset:
  dataset_name: MonoDataset
  normalize_depth: false
  max_disparity: 416
  baseline: 0.193001
  focal_x: 1998.842
  train_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  val_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  test_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  infer_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  quant_calibration_dataset:
    images_dir: ''
model:
  model_type: RelativeDepthAnything
  mono_backbone:
    pretrained_path: ''
    use_bn: false
    use_clstoken: false
  stereo_backbone:
    depth_anything_v2_pretrained_path: ''
    edgenext_pretrained_path: ''
    use_bn: false
    use_clstoken: false
  hidden_dims:
  - 128
  - 128
  - 128
  corr_radius: 4
  cv_group: 8
  train_iters: 22
  valid_iters: 22
  volume_dim: 32
  low_memory: 0
  mixed_precision: false
  n_gru_layers: 3
  corr_levels: 2
  n_downsample: 2
  encoder: vitl
  max_disparity: 416
train:
  num_gpus: 1
  gpu_ids:
  - 0
  num_nodes: 1
  seed: 1234
  cudnn:
    benchmark: false
    deterministic: true
  num_epochs: 10
  checkpoint_interval: 1
  checkpoint_interval_unit: epoch
  validation_interval: 1
  resume_training_checkpoint_path: ''
  results_dir: ''
  pretrained_model_path: ''
  clip_grad_norm: 0.1
  dataloader_visualize: false
  vis_step_interval: 10
  is_dry_run: false
  optim:
    optimizer: AdamW
    monitor_name: val_loss
    lr: 0.0001
    momentum: 0.9
    weight_decay: 0.0001
    lr_scheduler: MultiStepLR
    lr_steps:
    - 1000
    lr_step_size: 1000
    lr_decay: 0.1
    min_lr: 1.0e-07
    warmup_steps: 20
  precision: fp32
  distributed_strategy: ddp
  activation_checkpoint: false
  inference_tile: false
  tile_wtype: gaussian
  tile_min_overlap:
  - 16
  - 16
  log_every_n_steps: 500
quantize:
  backend: torchao
  mode: weight_only_ptq
  algorithm: minmax
  layers: []
  skip_names: []
  model_path: ''
  results_dir: ''
  backend_kwargs: {}
  device: cuda

model_name: ''
encryption_key: ''
results_dir: ''
wandb:
  enable: true
  project: TAO Toolkit
  entity: ''
  group: ''
  tags:
  - tao-toolkit
  reinit: false
  sync_tensorboard: false
  save_code: false
  name: TAO Toolkit Training
  run_id: ''
dataset:
  dataset_name: MonoDataset
  normalize_depth: false
  max_disparity: 416
  baseline: 0.193001
  focal_x: 1998.842
  train_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  val_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  test_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  infer_dataset:
    data_sources:
    - dataset_name: RelativeMonoDataset
      data_file: ''
    batch_size: 1
    workers: 8
    pin_memory: true
    augmentation:
      input_mean:
      - 0.485
      - 0.456
      - 0.406
      input_std:
      - 0.229
      - 0.224
      - 0.225
      crop_size:
      - 518
      - 518
      min_scale: -0.2
      max_scale: 0.4
      do_flip: false
      yjitter_prob: 1.0
      gamma:
      - 1
      - 1
      - 1
      - 1
      color_aug_prob: 0.2
      color_aug_brightness: 0.4
      color_aug_contrast: 0.4
      color_aug_saturation:
      - 0.0
      - 1.4
      color_aug_hue_range:
      - -0.027777777777777776
      - 0.027777777777777776
      eraser_aug_prob: 0.5
      spatial_aug_prob: 1.0
      stretch_prob: 0.8
      max_stretch: 0.2
      h_flip_prob: 0.5
      v_flip_prob: 0.5
      hshift_prob: 0.5
      crop_min_valid_disp_ratio: 0.0
  quant_calibration_dataset:
    images_dir: ''
model:
  model_type: RelativeDepthAnything
  mono_backbone:
    pretrained_path: ''
    use_bn: false
    use_clstoken: false
  stereo_backbone:
    depth_anything_v2_pretrained_path: ''
    edgenext_pretrained_path: ''
    use_bn: false
    use_clstoken: false
  hidden_dims:
  - 128
  - 128
  - 128
  corr_radius: 4
  cv_group: 8
  train_iters: 22
  valid_iters: 22
  volume_dim: 32
  low_memory: 0
  mixed_precision: false
  n_gru_layers: 3
  corr_levels: 2
  n_downsample: 2
  encoder: vitl
  max_disparity: 416
train:
  num_gpus: 1
  gpu_ids:
  - 0
  num_nodes: 1
  seed: 1234
  cudnn:
    benchmark: false
    deterministic: true
  num_epochs: 10
  checkpoint_interval: 1
  checkpoint_interval_unit: epoch
  validation_interval: 1
  resume_training_checkpoint_path: ''
  results_dir: ''
  pretrained_model_path: ''
  clip_grad_norm: 0.1
  dataloader_visualize: false
  vis_step_interval: 10
  is_dry_run: false
  optim:
    optimizer: AdamW
    monitor_name: val_loss
    lr: 0.0001
    momentum: 0.9
    weight_decay: 0.0001
    lr_scheduler: MultiStepLR
    lr_steps:
    - 1000
    lr_step_size: 1000
    lr_decay: 0.1
    min_lr: 1.0e-07
    warmup_steps: 20
  precision: fp32
  distributed_strategy: ddp
  activation_checkpoint: false
  inference_tile: false
  tile_wtype: gaussian
  tile_min_overlap:
  - 16
  - 16
  log_every_n_steps: 500
quantize:
  backend: torchao
  mode: weight_only_ptq
  algorithm: minmax
  layers: []
  skip_names: []
  model_path: ''
  results_dir: ''
  backend_kwargs: {}
  device: cuda

Typical Spec Overrides

Data source overrides are mandatory for every action — the agent MUST construct data source paths from the Per-Action Dataset Requirements table in SKILL.md and include them in spec_overrides. Each data_sources entry is a dict with two mandatory fields: data_file and dataset_name.

S3_TRAIN = "aws://bucket/data/train"
S3_EVAL = "aws://bucket/data/eval"

train (mandatory data sources):

{
    "train.num_epochs": 10,
    "train.precision": "fp32",
    "train.checkpoint_interval": 10,
    "train.validation_interval": 10,
    "train.num_gpus": 1,
    "model.model_type": "RelativeDepthAnything",
    "model.encoder": "vitl",
    "dataset.dataset_name": "MonoDataset",
    "dataset.min_depth": None,
    "dataset.max_depth": None,
    "dataset.train_dataset.batch_size": 4,
    "dataset.train_dataset.workers": 4,
    "dataset.train_dataset.augmentation.crop_size": [518, 518],
    "dataset.train_dataset.data_sources": [
        {"data_file": f"{S3_TRAIN}/annotations.txt", "dataset_name": "RelativeMonoDataset"}
    ],
    "dataset.val_dataset.batch_size": 1,
    "dataset.val_dataset.workers": 4,
    "dataset.val_dataset.data_sources": [
        {"data_file": f"{S3_EVAL}/annotations.txt", "dataset_name": "RelativeMonoDataset"}
    ],
}

Precision recommendation (relative variant): use fp32 (recommended). bf16 is supported as an alternative on Ampere SM80+ hardware.

evaluate (mandatory data sources):

{
    "model.model_type": "RelativeDepthAnything",
    "dataset.dataset_name": "MonoDataset",
    "dataset.min_depth": None,
    "dataset.max_depth": None,
    "dataset.test_dataset.batch_size": 1,
    "dataset.test_dataset.workers": 4,
    "dataset.test_dataset.data_sources": [
        {"data_file": f"{S3_EVAL}/annotations.txt", "dataset_name": "NYUDV2Relative"}
    ],
    "evaluate.checkpoint": "<selected train/AutoML checkpoint>",
}

export:

{
    "model.model_type": "RelativeDepthAnything",
    "dataset.dataset_name": "MonoDataset",
    "dataset.min_depth": None,
    "dataset.max_depth": None,
    "export.checkpoint": "<selected train/AutoML checkpoint>",
    "export.input_channel": 3,
    "export.input_height": 518,
    "export.input_width": 518,
    "export.opset_version": 16,
    "export.on_cpu": False,
    "export.gpu_id": 0,
}

Defaults sourced from nvidia_tao_pytorch/cv/depth_net/experiment_specs/experiment_mono_relative.yaml (export block). Override only when the deployment target requires a different ONNX shape, opset, or export device.

inference (mandatory data sources):

{
    "model.model_type": "RelativeDepthAnything",
    "dataset.dataset_name": "MonoDataset",
    "dataset.min_depth": None,
    "dataset.max_depth": None,
    "dataset.infer_dataset.batch_size": 1,
    "dataset.infer_dataset.workers": 4,
    "dataset.infer_dataset.data_sources": [
        {"data_file": f"{S3_EVAL}/annotations.txt", "dataset_name": "RelativeMonoDataset"}
    ],
    "inference.checkpoint": "<selected train/AutoML checkpoint>",
    "inference.save_raw_pfm": False,
}

inference.save_raw_pfm controls whether raw single-channel disparity is written as .pfm files alongside the visualization output. Default False — the action emits a 240×960 RGB JPG triptych (input | predicted disp | overlay-style panel) at 320×240 per panel, mirroring the source dataset's directory tree under <results_dir>/inference/inference_images/. Set True to additionally write .pfm files for downstream metric computation; raw disparity is unbounded scale-shift-invariant for RelativeDepthAnything and bounded to [min_depth, max_depth] for MetricDepthAnything.

quantize (mandatory data sources):

{
    "model.model_type": "RelativeDepthAnything",
    "dataset.dataset_name": "MonoDataset",
    "dataset.min_depth": None,
    "dataset.max_depth": None,
    "dataset.train_dataset.data_sources": [
        {"data_file": f"{S3_TRAIN}/annotations.txt", "dataset_name": "RelativeMonoDataset"}
    ],
    "dataset.val_dataset.data_sources": [
        {"data_file": f"{S3_EVAL}/annotations.txt", "dataset_name": "RelativeMonoDataset"}
    ],
    "dataset.quant_calibration_dataset.images_dir": f"{S3_TRAIN}/images",
    "quantize.model_path": "<selected train/AutoML checkpoint>",
}

Known issue in nvcr.io/nvstaging/tao/tao-toolkit-pyt:7.0.0-rc-226-multiarch: mono depth_net quantize reaches the checkpoint load path and then fails inside the SDK with MonoDepthNetPlModel missing load_state_dict_from_checkpoint. Keep quantize.model_path wired to the selected checkpoint; do not replace it with a latest-file guess.

Spec Param / Parent Model Inference

Model-specific inference mappings belong in this MD file, not in config.json. Generated runners should read this section and apply the mappings with SDK helpers before create_job(). This mirrors the old microservices infer_params.py flow.

DepthNet Mono training writes checkpoint files under <results_dir>/train/ using model_epoch_<epoch>_step_<step>.pth and a dn_model_latest.pth symlink. For evaluate, inference, export, quantize, and resume/retrain, select checkpoints through the SDK/model resolver so a requested best, epoch, or step checkpoint resolves to that exact file. Use dn_model_latest.pth only when the user explicitly asks for latest.

Parent PyT gen_trt_engine is intentionally not used because the current depth_net PyT entrypoint rejects it. The gen_trt_engine metadata selects the TAO Deploy container, and the deploy workflow owns the TensorRT-only action details.

Inference mappings from TAO Core depth_net_mono.config.json:

Action	Spec Field	Inference Function	Meaning
evaluate	`dataset.dataset_name`	`MonoDataset`	MonoDataset
evaluate	`evaluate.checkpoint`	`parent_model`	model file inferred from the parent job results folder
evaluate	`evaluate.trt_engine`	`parent_model`	model file inferred from the parent job results folder
evaluate	`results_dir`	`output_dir`	current job results directory
export	`dataset.dataset_name`	`MonoDataset`	MonoDataset
export	`export.checkpoint`	`parent_model`	model file inferred from the parent job results folder
export	`export.onnx_file`	`create_onnx_file`	output ONNX path
export	`results_dir`	`output_dir`	current job results directory
inference	`dataset.dataset_name`	`MonoDataset`	MonoDataset
inference	`inference.checkpoint`	`parent_model`	model file inferred from the parent job results folder
inference	`inference.trt_engine`	`parent_model`	model file inferred from the parent job results folder
inference	`results_dir`	`output_dir`	current job results directory
quantize	`dataset.dataset_name`	`MonoDataset`	MonoDataset
quantize	`quantize.model_path`	`parent_model`	model file inferred from the parent job results folder
quantize	`results_dir`	`output_dir`	current job results directory
train	`dataset.dataset_name`	`MonoDataset`	MonoDataset
train	`model.mono_backbone.pretrained_path`	`{'link': 'https://dl.fbaipublicfiles.com/dinov2/dinov2_vitl14/dinov2_vitl14_pretrain.pth', 'destination_path': '/ptm/depth_net/mono_backbone/dinov2_vitl14_pretrain.pth'}`	{'link': 'https://dl.fbaipublicfiles.com/dinov2/dinov2_vitl14/dinov2_vitl14_pretrain.pth', 'destination_path': '/ptm/depth_net/mono_backbone/dinov2_vitl14_pretrain.pth'}
train	`results_dir`	`output_dir`	current job results directory
train	`train.pretrained_model_path`	`ptm_if_no_resume_model`	PTM when no resume checkpoint exists
train	`train.resume_training_checkpoint_path`	`resume_model`	model file inferred from the current job results folder

For parent_model or parent_model_folder, pass the upstream train/export/AutoML child job id as parent_job_id. The SDK lists the parent result folder, filters checkpoint artifacts, and returns the selected model file or folder. Do not add these mappings back to config.json and do not patch generated runner scripts to guess checkpoint paths.

DepthNet Mono Deploy

DepthNet Mono deploy covers the TAO Deploy actions for an exported monocular depth estimation model. Use the depth-net-mono model skill for training, checkpoint evaluation, quantization, export, or non-TensorRT inference where those actions exist. Use this deploy workflow after export when the input artifact is an ONNX model and the desired output is a TensorRT engine or TensorRT-backed predictions.

Supported actions: gen_trt_engine, evaluate, inference. Direct TAO Deploy command name: depth_net.

Quick Start

Generate TensorRT Engine

docker run --gpus all --rm --shm-size=16g \
  -v /path/to/specs:/specs \
  -v /path/to/export:/models \
  -v /path/to/results:/results \
  nvcr.io/nvidia/tao/tao-toolkit:6.26.3-deploy \
  depth_net gen_trt_engine -e /specs/gen_trt_engine.yaml

Evaluate TensorRT Engine

docker run --gpus all --rm --shm-size=16g \
  -v /path/to/specs:/specs \
  -v /path/to/eval:/data \
  -v /path/to/results:/results \
  nvcr.io/nvidia/tao/tao-toolkit:6.26.3-deploy \
  depth_net evaluate -e /specs/evaluate.yaml

TensorRT Inference

docker run --gpus all --rm --shm-size=16g \
  -v /path/to/specs:/specs \
  -v /path/to/inference:/data \
  -v /path/to/results:/results \
  nvcr.io/nvidia/tao/tao-toolkit:6.26.3-deploy \
  depth_net inference -e /specs/inference.yaml

Deploy action metadata is in tao-deploy-depth-anything-v2.skill_info.yaml. Deploy spec template lives in this references folder:

spec_template_deploy.yaml

Deploy Workflow

1. Train and export with the depth-net-mono skill. 2. Keep the exported ONNX artifact and any sidecar files together in the mounted model directory. 3. Build the TensorRT engine with this workflow. 4. Run TensorRT evaluate or inference from the engine artifact produced by gen_trt_engine.

Direct TAO Launcher spelling is tao deploy depth_net gen_trt_engine, tao deploy depth_net evaluate, tao deploy depth_net inference.

Required Inputs

Action	Required artifact or data	Spec key
`gen_trt_engine`	Exported monocular ONNX model	`gen_trt_engine.onnx_file`
`evaluate`	TensorRT engine	`evaluate.trt_engine`
`evaluate`	Depth annotation file	`dataset.test_dataset.data_sources[0].data_file`
`inference`	TensorRT engine	`inference.trt_engine`
`inference`	Depth annotation file	`dataset.infer_dataset.data_sources[0].data_file`

gen_trt_engine.trt_engine is a generated output path, not an input artifact. For direct Docker runs, mount input folders at the same paths used in the spec. For chained jobs, map exported ONNX artifacts into gen_trt_engine.onnx_file and map the engine artifact into evaluate.trt_engine or inference.trt_engine.

Spec Templates

Two model variants are supported. The deploy spec template at spec_template_deploy.yaml covers the relative variant (default). For the metric variant, start from the same template and apply the overrides below.

Relative variant (default)

Copy spec_template_deploy.yaml as a starting point. Override only paths and environment-specific values (data_file, results_dir, trt_engine paths, batch size as needed). No structural overrides required.

Metric variant

Start from the same template, then apply these metric-specific overrides:

dataset:
  test_dataset:
    data_sources:
    - dataset_name: NYUDV2              # metric pairs with NYUDV2 (not NYUDV2Relative)
      data_file: /data/annotations.txt
  infer_dataset:
    data_sources:
    - dataset_name: MetricMonoDataset
      data_file: /data/annotations.txt
  # carry the metric variant's NYU-trained normalization (from your train/export spec)
  normalize_depth: false
  max_depth: 10.0
  min_depth: 0.001

Common to both variants:

The TAO Deploy command is depth_net for both mono and stereo DepthNet model skills.
Fresh-install TRT precision: gen_trt_engine.tensorrt.data_type: fp32. BF16 is supported on Ampere SM80+ hardware, but keep validation smoke tests on FP32 unless the user requests BF16. fp16 is not supported for the ViT-L mono backbone.
The current TAO Deploy image interprets gen_trt_engine.tensorrt.workspace_size as GiB. Use 4 for a 4 GiB workspace; values such as 1024 request a 1024 GiB workspace and may fail on ordinary systems.
For aspect-preserved inference (matching pyt evaluator on variable-aspect input), set dataset.test_dataset.augmentation.crop_size and dataset.infer_dataset.augmentation.crop_size to the dataset's keep-aspect target shape (e.g., NYU 480×640 → [518, 686] with multiple_of=14). The deploy runtime selects input H/W from augmentation.crop_size, not from evaluate.input_height/input_width; leaving crop_size unset falls back to tao-core's [518, 518] default and silently overrides the engine shape. The engine input shape must match crop_size exactly (mono engines are built static at the trace shape — only the batch axis can be dynamic).

Spec filename invariant

The spec yaml's basename (modulo .yaml) must match the action verb passed on the command line. For example, gen_trt_engine requires the spec at a path ending in gen_trt_engine.yaml; evaluate requires evaluate.yaml. Mismatched filenames produce a non-obvious FileNotFoundError from the hydra config loader before any action work begins.

Job Chain Mapping

Action	Spec field	Parent or output
`gen_trt_engine`	`gen_trt_engine.onnx_file`	export job ONNX
`gen_trt_engine`	`gen_trt_engine.trt_engine`	new engine output path
`evaluate`	`evaluate.trt_engine`	engine job output
`inference`	`inference.trt_engine`	engine job output

Outputs

Action	Output
`gen_trt_engine`	TensorRT engine at `gen_trt_engine.trt_engine`
`evaluate`	Depth metrics under `results_dir` (`abs_rel`, `d1`/`d2`/`d3` for mono; `rmse` is N/A for the scale-shift-invariant relative variant)
`inference`	Predicted depth outputs under `results_dir` (colorized JPGs by default; `inference.save_raw_pfm: True` to add raw PFMs)

Common errors

Engine profile mismatch: Runtime batch size for evaluate or inference must fit within the TensorRT min/opt/max profile used during gen_trt_engine. The default profile in the spec template is min=1 / opt=1 / max=4 — adjust if your inference call uses a larger batch.

Aspect-stretched predictions: Forcing the engine input H/W to a static shape that doesn't match the dataset's native aspect distorts the depth field. Mono examples: NYU 480×640 should run at 518×686 (keep-aspect, multiple-of-14), not 518×518. Pick the keep-aspect target at export time (export.input_height / export.input_width) and set dataset.{test,infer}_dataset.augmentation.crop_size: [518, 686] to match (this is what the deploy runtime actually reads). Different datasets with different aspect ratios require separate engines.

INT8 calibration missing: INT8 builds need an extracted calibration image directory, a writable cache path, and enough images for cal_batch_size * cal_batches.

Mounted paths do not exist: TAO Deploy checks local paths inside the container. Make sure every path in the spec has a matching Docker mount or job artifact mapping.

name: depth-net-mono-deploy
type: model
network_arch: depth_net_mono
container_image: tao_toolkit.deploy
data_format: RelativeMonoDataset
actions:
  gen_trt_engine:
    command: depth_net gen_trt_engine -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      gen_trt_engine.onnx_file:
        type: file
    outputs:
      results_dir:
        type: folder
      gen_trt_engine.trt_engine:
        type: file
    upload_excludes:
    - inputs/
  evaluate:
    command: depth_net evaluate -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      evaluate.trt_engine:
        type: file
      dataset.test_dataset.data_sources[0].data_file:
        type: file
    outputs:
      results_dir:
        type: folder
    upload_excludes:
    - inputs/
  inference:
    command: depth_net inference -e {config_path}
    config_format: yaml
    mode: config
    inputs:
      inference.trt_engine:
        type: file
      dataset.infer_dataset.data_sources[0].data_file:
        type: file
    outputs:
      results_dir:
        type: folder
    upload_excludes:
    - inputs/
spec_params:
  gen_trt_engine:
    results_dir: output_dir
    gen_trt_engine.onnx_file: parent_model
    gen_trt_engine.trt_engine: create_engine_file
  evaluate:
    results_dir: output_dir
    evaluate.trt_engine: parent_model
  inference:
    results_dir: output_dir
    inference.trt_engine: parent_model
spec_shorthand_keys:
  trt_data_type: gen_trt_engine.tensorrt.data_type
  trt_engine: gen_trt_engine.trt_engine
  test_batch_size: dataset.test_dataset.batch_size
  infer_batch_size: dataset.infer_dataset.batch_size
description: DepthNet Mono deploy workflow for gen_trt_engine, evaluate, inference
  using TAO Deploy.
spec_templates:
  gen_trt_engine: spec_template_deploy.yaml
  evaluate: spec_template_deploy.yaml
  inference: spec_template_deploy.yaml
notes:
- The TAO Deploy command is `depth_net` for both mono and stereo DepthNet model skills.
- 'Keep `dataset.dataset_name: MonoDataset` and use a monocular data source such as
  `RelativeMonoDataset` (or `NYUDV2Relative` for NYU eval) in the deploy spec.'
- 'Use `gen_trt_engine.tensorrt.data_type: fp32` for fresh-install smoke tests.
  BF16 is supported on Ampere SM80+ hardware, but FP16 is not supported for the
  DepthAnythingV2 mono backbone.'
- 'The current TAO Deploy image interprets `gen_trt_engine.tensorrt.workspace_size`
  as GiB. Use `4` for a 4 GiB workspace; do not use byte or MiB-style values such
  as 1024.'

Error Patterns

Depth range mismatch: Ensure dataset.max_depth / dataset.min_depth match the actual depth range in your data.

Relative dataset rejects `min_depth`: For RelativeMonoDataset and NYUDV2Relative, remove dataset.min_depth and dataset.max_depth or set them to null. Non-null values are metric-only and make the relative dataset constructor fail before training starts.

Missing pretrained weights: DepthAnything v2 encoder requires model.mono_backbone.pretrained_path to be set for fine-tuning.

`Key 'encoder' not in 'MonoBackBone'`: encoder is a top-level model.encoder field, not under mono_backbone. See Important Parameters in references/parameters.md.

`Key 'dataset_name' is not in struct` under data_sources: every data_sources entry must include both data_file and dataset_name.

`bash: exec: depth_net_mono: not found`: the unified entrypoint is depth_net (no _mono / _stereo suffix). The skill's command already uses the correct form; check any user-supplied wrapper.

Metric variant hyperparameter sourcing (dataset.normalize_depth, dataset.train_dataset.augmentation.input_mean, dataset.train_dataset.augmentation.input_std): MetricDepthAnything requires depth normalization and ImageNet input statistics that match the checkpoint's training run. These are model- and dataset-specific (not skill-level defaults) — read them from the checkpoint's sibling experiment.yaml (or the upstream training spec). Common NYU-trained values: normalize_depth: false, max_depth: 10.0, min_depth: 0.001, input_mean: [0.485, 0.456, 0.406], input_std: [0.229, 0.224, 0.225]. Mirror the depth-range values into the export spec — see the Metric Variant Finetuning Recipe → Dataset normalization block in references/finetuning-recipes.md.

Export refuses to overwrite an existing ONNX file: ValueError: Default onnx file <path> already exists. The mono export action refuses to overwrite a prior artifact at export.onnx_file. Delete or rename the existing file, or change the spec's export.onnx_file to a fresh path before re-running.

{
  "actions": {
    "evaluate": {
      "automl_default_parameters": [
        "dataset.infer_dataset.augmentation.color_aug_prob",
        "dataset.infer_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.infer_dataset.augmentation.eraser_aug_prob",
        "dataset.infer_dataset.augmentation.h_flip_prob",
        "dataset.infer_dataset.augmentation.hshift_prob",
        "dataset.infer_dataset.augmentation.spatial_aug_prob",
        "dataset.infer_dataset.augmentation.stretch_prob",
        "dataset.infer_dataset.augmentation.v_flip_prob",
        "dataset.infer_dataset.augmentation.yjitter_prob",
        "dataset.test_dataset.augmentation.color_aug_prob",
        "dataset.test_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.test_dataset.augmentation.eraser_aug_prob",
        "dataset.test_dataset.augmentation.h_flip_prob",
        "dataset.test_dataset.augmentation.hshift_prob",
        "dataset.test_dataset.augmentation.spatial_aug_prob",
        "dataset.test_dataset.augmentation.stretch_prob",
        "dataset.test_dataset.augmentation.v_flip_prob",
        "dataset.test_dataset.augmentation.yjitter_prob",
        "dataset.train_dataset.augmentation.color_aug_prob",
        "dataset.train_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.train_dataset.augmentation.eraser_aug_prob",
        "dataset.train_dataset.augmentation.h_flip_prob",
        "dataset.train_dataset.augmentation.hshift_prob",
        "dataset.train_dataset.augmentation.spatial_aug_prob",
        "dataset.train_dataset.augmentation.stretch_prob",
        "dataset.train_dataset.augmentation.v_flip_prob",
        "dataset.train_dataset.augmentation.yjitter_prob",
        "dataset.val_dataset.augmentation.color_aug_prob",
        "dataset.val_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.val_dataset.augmentation.eraser_aug_prob",
        "dataset.val_dataset.augmentation.h_flip_prob",
        "dataset.val_dataset.augmentation.hshift_prob",
        "dataset.val_dataset.augmentation.spatial_aug_prob",
        "dataset.val_dataset.augmentation.stretch_prob",
        "dataset.val_dataset.augmentation.v_flip_prob",
        "dataset.val_dataset.augmentation.yjitter_prob",
        "model.corr_radius",
        "model.cv_group",
        "model.volume_dim",
        "train.optim.lr",
        "train.optim.lr_decay",
        "train.optim.lr_step_size",
        "train.optim.min_lr",
        "train.optim.momentum",
        "train.optim.weight_decay"
      ],
      "automl_disabled_parameters": [
        "dataset",
        "dataset.infer_dataset",
        "dataset.infer_dataset.augmentation",
        "dataset.infer_dataset.augmentation.color_aug_hue_range",
        "dataset.infer_dataset.augmentation.color_aug_saturation",
        "dataset.infer_dataset.augmentation.crop_size",
        "dataset.infer_dataset.augmentation.gamma",
        "dataset.infer_dataset.augmentation.input_mean",
        "dataset.infer_dataset.augmentation.input_std",
        "dataset.infer_dataset.data_sources",
        "dataset.quant_calibration_dataset",
        "dataset.test_dataset",
        "dataset.test_dataset.augmentation",
        "dataset.test_dataset.augmentation.color_aug_hue_range",
        "dataset.test_dataset.augmentation.color_aug_saturation",
        "dataset.test_dataset.augmentation.crop_size",
        "dataset.test_dataset.augmentation.gamma",
        "dataset.test_dataset.augmentation.input_mean",
        "dataset.test_dataset.augmentation.input_std",
        "dataset.test_dataset.data_sources",
        "dataset.train_dataset",
        "dataset.train_dataset.augmentation",
        "dataset.train_dataset.augmentation.color_aug_hue_range",
        "dataset.train_dataset.augmentation.color_aug_saturation",
        "dataset.train_dataset.augmentation.crop_size",
        "dataset.train_dataset.augmentation.gamma",
        "dataset.train_dataset.augmentation.input_mean",
        "dataset.train_dataset.augmentation.input_std",
        "dataset.train_dataset.data_sources",
        "dataset.val_dataset",
        "dataset.val_dataset.augmentation",
        "dataset.val_dataset.augmentation.color_aug_hue_range",
        "dataset.val_dataset.augmentation.color_aug_saturation",
        "dataset.val_dataset.augmentation.crop_size",
        "dataset.val_dataset.augmentation.gamma",
        "dataset.val_dataset.augmentation.input_mean",
        "dataset.val_dataset.augmentation.input_std",
        "dataset.val_dataset.data_sources",
        "evaluate",
        "evaluate.gpu_ids",
        "export",
        "gen_trt_engine",
        "gen_trt_engine.tensorrt",
        "gen_trt_engine.tensorrt.layers_precision",
        "inference",
        "inference.gpu_ids",
        "model",
        "model.hidden_dims",
        "model.mono_backbone",
        "model.stereo_backbone",
        "quantize",
        "quantize.backend_kwargs",
        "quantize.layers",
        "quantize.skip_names",
        "train",
        "train.cudnn",
        "train.gpu_ids",
        "train.optim",
        "train.optim.lr_steps",
        "train.tile_min_overlap",
        "wandb",
        "wandb.tags"
      ],
      "core_module": "depth_net",
      "path": "schemas/evaluate.schema.json",
      "popular": {
        "evaluate": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "gen_trt_engine": {
          "batch_size": -1,
          "gpu_id": 0,
          "tensorrt": {
            "max_batch_size": 1,
            "min_batch_size": 1,
            "opt_batch_size": 1
          }
        },
        "inference": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "train": {
          "checkpoint_interval": 1,
          "gpu_ids": [
            0
          ],
          "num_epochs": 10,
          "num_gpus": 1,
          "num_nodes": 1,
          "validation_interval": 1
        }
      },
      "schema_action": "evaluate",
      "spec_template": "references/spec_template_evaluate.yaml"
    },
    "export": {
      "automl_default_parameters": [
        "dataset.infer_dataset.augmentation.color_aug_prob",
        "dataset.infer_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.infer_dataset.augmentation.eraser_aug_prob",
        "dataset.infer_dataset.augmentation.h_flip_prob",
        "dataset.infer_dataset.augmentation.hshift_prob",
        "dataset.infer_dataset.augmentation.spatial_aug_prob",
        "dataset.infer_dataset.augmentation.stretch_prob",
        "dataset.infer_dataset.augmentation.v_flip_prob",
        "dataset.infer_dataset.augmentation.yjitter_prob",
        "dataset.test_dataset.augmentation.color_aug_prob",
        "dataset.test_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.test_dataset.augmentation.eraser_aug_prob",
        "dataset.test_dataset.augmentation.h_flip_prob",
        "dataset.test_dataset.augmentation.hshift_prob",
        "dataset.test_dataset.augmentation.spatial_aug_prob",
        "dataset.test_dataset.augmentation.stretch_prob",
        "dataset.test_dataset.augmentation.v_flip_prob",
        "dataset.test_dataset.augmentation.yjitter_prob",
        "dataset.train_dataset.augmentation.color_aug_prob",
        "dataset.train_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.train_dataset.augmentation.eraser_aug_prob",
        "dataset.train_dataset.augmentation.h_flip_prob",
        "dataset.train_dataset.augmentation.hshift_prob",
        "dataset.train_dataset.augmentation.spatial_aug_prob",
        "dataset.train_dataset.augmentation.stretch_prob",
        "dataset.train_dataset.augmentation.v_flip_prob",
        "dataset.train_dataset.augmentation.yjitter_prob",
        "dataset.val_dataset.augmentation.color_aug_prob",
        "dataset.val_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.val_dataset.augmentation.eraser_aug_prob",
        "dataset.val_dataset.augmentation.h_flip_prob",
        "dataset.val_dataset.augmentation.hshift_prob",
        "dataset.val_dataset.augmentation.spatial_aug_prob",
        "dataset.val_dataset.augmentation.stretch_prob",
        "dataset.val_dataset.augmentation.v_flip_prob",
        "dataset.val_dataset.augmentation.yjitter_prob",
        "model.corr_radius",
        "model.cv_group",
        "model.volume_dim",
        "train.optim.lr",
        "train.optim.lr_decay",
        "train.optim.lr_step_size",
        "train.optim.min_lr",
        "train.optim.momentum",
        "train.optim.weight_decay"
      ],
      "automl_disabled_parameters": [
        "dataset",
        "dataset.infer_dataset",
        "dataset.infer_dataset.augmentation",
        "dataset.infer_dataset.augmentation.color_aug_hue_range",
        "dataset.infer_dataset.augmentation.color_aug_saturation",
        "dataset.infer_dataset.augmentation.crop_size",
        "dataset.infer_dataset.augmentation.gamma",
        "dataset.infer_dataset.augmentation.input_mean",
        "dataset.infer_dataset.augmentation.input_std",
        "dataset.infer_dataset.data_sources",
        "dataset.quant_calibration_dataset",
        "dataset.test_dataset",
        "dataset.test_dataset.augmentation",
        "dataset.test_dataset.augmentation.color_aug_hue_range",
        "dataset.test_dataset.augmentation.color_aug_saturation",
        "dataset.test_dataset.augmentation.crop_size",
        "dataset.test_dataset.augmentation.gamma",
        "dataset.test_dataset.augmentation.input_mean",
        "dataset.test_dataset.augmentation.input_std",
        "dataset.test_dataset.data_sources",
        "dataset.train_dataset",
        "dataset.train_dataset.augmentation",
        "dataset.train_dataset.augmentation.color_aug_hue_range",
        "dataset.train_dataset.augmentation.color_aug_saturation",
        "dataset.train_dataset.augmentation.crop_size",
        "dataset.train_dataset.augmentation.gamma",
        "dataset.train_dataset.augmentation.input_mean",
        "dataset.train_dataset.augmentation.input_std",
        "dataset.train_dataset.data_sources",
        "dataset.val_dataset",
        "dataset.val_dataset.augmentation",
        "dataset.val_dataset.augmentation.color_aug_hue_range",
        "dataset.val_dataset.augmentation.color_aug_saturation",
        "dataset.val_dataset.augmentation.crop_size",
        "dataset.val_dataset.augmentation.gamma",
        "dataset.val_dataset.augmentation.input_mean",
        "dataset.val_dataset.augmentation.input_std",
        "dataset.val_dataset.data_sources",
        "evaluate",
        "evaluate.gpu_ids",
        "export",
        "gen_trt_engine",
        "gen_trt_engine.tensorrt",
        "gen_trt_engine.tensorrt.layers_precision",
        "inference",
        "inference.gpu_ids",
        "model",
        "model.hidden_dims",
        "model.mono_backbone",
        "model.stereo_backbone",
        "quantize",
        "quantize.backend_kwargs",
        "quantize.layers",
        "quantize.skip_names",
        "train",
        "train.cudnn",
        "train.gpu_ids",
        "train.optim",
        "train.optim.lr_steps",
        "train.tile_min_overlap",
        "wandb",
        "wandb.tags"
      ],
      "core_module": "depth_net",
      "path": "schemas/export.schema.json",
      "popular": {
        "evaluate": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "gen_trt_engine": {
          "batch_size": -1,
          "gpu_id": 0,
          "tensorrt": {
            "max_batch_size": 1,
            "min_batch_size": 1,
            "opt_batch_size": 1
          }
        },
        "inference": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "train": {
          "checkpoint_interval": 1,
          "gpu_ids": [
            0
          ],
          "num_epochs": 10,
          "num_gpus": 1,
          "num_nodes": 1,
          "validation_interval": 1
        }
      },
      "schema_action": "export",
      "spec_template": "references/spec_template_export.yaml"
    },
    "inference": {
      "automl_default_parameters": [
        "dataset.infer_dataset.augmentation.color_aug_prob",
        "dataset.infer_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.infer_dataset.augmentation.eraser_aug_prob",
        "dataset.infer_dataset.augmentation.h_flip_prob",
        "dataset.infer_dataset.augmentation.hshift_prob",
        "dataset.infer_dataset.augmentation.spatial_aug_prob",
        "dataset.infer_dataset.augmentation.stretch_prob",
        "dataset.infer_dataset.augmentation.v_flip_prob",
        "dataset.infer_dataset.augmentation.yjitter_prob",
        "dataset.test_dataset.augmentation.color_aug_prob",
        "dataset.test_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.test_dataset.augmentation.eraser_aug_prob",
        "dataset.test_dataset.augmentation.h_flip_prob",
        "dataset.test_dataset.augmentation.hshift_prob",
        "dataset.test_dataset.augmentation.spatial_aug_prob",
        "dataset.test_dataset.augmentation.stretch_prob",
        "dataset.test_dataset.augmentation.v_flip_prob",
        "dataset.test_dataset.augmentation.yjitter_prob",
        "dataset.train_dataset.augmentation.color_aug_prob",
        "dataset.train_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.train_dataset.augmentation.eraser_aug_prob",
        "dataset.train_dataset.augmentation.h_flip_prob",
        "dataset.train_dataset.augmentation.hshift_prob",
        "dataset.train_dataset.augmentation.spatial_aug_prob",
        "dataset.train_dataset.augmentation.stretch_prob",
        "dataset.train_dataset.augmentation.v_flip_prob",
        "dataset.train_dataset.augmentation.yjitter_prob",
        "dataset.val_dataset.augmentation.color_aug_prob",
        "dataset.val_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.val_dataset.augmentation.eraser_aug_prob",
        "dataset.val_dataset.augmentation.h_flip_prob",
        "dataset.val_dataset.augmentation.hshift_prob",
        "dataset.val_dataset.augmentation.spatial_aug_prob",
        "dataset.val_dataset.augmentation.stretch_prob",
        "dataset.val_dataset.augmentation.v_flip_prob",
        "dataset.val_dataset.augmentation.yjitter_prob",
        "model.corr_radius",
        "model.cv_group",
        "model.volume_dim",
        "train.optim.lr",
        "train.optim.lr_decay",
        "train.optim.lr_step_size",
        "train.optim.min_lr",
        "train.optim.momentum",
        "train.optim.weight_decay"
      ],
      "automl_disabled_parameters": [
        "dataset",
        "dataset.infer_dataset",
        "dataset.infer_dataset.augmentation",
        "dataset.infer_dataset.augmentation.color_aug_hue_range",
        "dataset.infer_dataset.augmentation.color_aug_saturation",
        "dataset.infer_dataset.augmentation.crop_size",
        "dataset.infer_dataset.augmentation.gamma",
        "dataset.infer_dataset.augmentation.input_mean",
        "dataset.infer_dataset.augmentation.input_std",
        "dataset.infer_dataset.data_sources",
        "dataset.quant_calibration_dataset",
        "dataset.test_dataset",
        "dataset.test_dataset.augmentation",
        "dataset.test_dataset.augmentation.color_aug_hue_range",
        "dataset.test_dataset.augmentation.color_aug_saturation",
        "dataset.test_dataset.augmentation.crop_size",
        "dataset.test_dataset.augmentation.gamma",
        "dataset.test_dataset.augmentation.input_mean",
        "dataset.test_dataset.augmentation.input_std",
        "dataset.test_dataset.data_sources",
        "dataset.train_dataset",
        "dataset.train_dataset.augmentation",
        "dataset.train_dataset.augmentation.color_aug_hue_range",
        "dataset.train_dataset.augmentation.color_aug_saturation",
        "dataset.train_dataset.augmentation.crop_size",
        "dataset.train_dataset.augmentation.gamma",
        "dataset.train_dataset.augmentation.input_mean",
        "dataset.train_dataset.augmentation.input_std",
        "dataset.train_dataset.data_sources",
        "dataset.val_dataset",
        "dataset.val_dataset.augmentation",
        "dataset.val_dataset.augmentation.color_aug_hue_range",
        "dataset.val_dataset.augmentation.color_aug_saturation",
        "dataset.val_dataset.augmentation.crop_size",
        "dataset.val_dataset.augmentation.gamma",
        "dataset.val_dataset.augmentation.input_mean",
        "dataset.val_dataset.augmentation.input_std",
        "dataset.val_dataset.data_sources",
        "evaluate",
        "evaluate.gpu_ids",
        "export",
        "gen_trt_engine",
        "gen_trt_engine.tensorrt",
        "gen_trt_engine.tensorrt.layers_precision",
        "inference",
        "inference.gpu_ids",
        "model",
        "model.hidden_dims",
        "model.mono_backbone",
        "model.stereo_backbone",
        "quantize",
        "quantize.backend_kwargs",
        "quantize.layers",
        "quantize.skip_names",
        "train",
        "train.cudnn",
        "train.gpu_ids",
        "train.optim",
        "train.optim.lr_steps",
        "train.tile_min_overlap",
        "wandb",
        "wandb.tags"
      ],
      "core_module": "depth_net",
      "path": "schemas/inference.schema.json",
      "popular": {
        "evaluate": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "gen_trt_engine": {
          "batch_size": -1,
          "gpu_id": 0,
          "tensorrt": {
            "max_batch_size": 1,
            "min_batch_size": 1,
            "opt_batch_size": 1
          }
        },
        "inference": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "train": {
          "checkpoint_interval": 1,
          "gpu_ids": [
            0
          ],
          "num_epochs": 10,
          "num_gpus": 1,
          "num_nodes": 1,
          "validation_interval": 1
        }
      },
      "schema_action": "inference",
      "spec_template": "references/spec_template_inference.yaml"
    },
    "quantize": {
      "automl_default_parameters": [
        "dataset.infer_dataset.augmentation.color_aug_prob",
        "dataset.infer_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.infer_dataset.augmentation.eraser_aug_prob",
        "dataset.infer_dataset.augmentation.h_flip_prob",
        "dataset.infer_dataset.augmentation.hshift_prob",
        "dataset.infer_dataset.augmentation.spatial_aug_prob",
        "dataset.infer_dataset.augmentation.stretch_prob",
        "dataset.infer_dataset.augmentation.v_flip_prob",
        "dataset.infer_dataset.augmentation.yjitter_prob",
        "dataset.test_dataset.augmentation.color_aug_prob",
        "dataset.test_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.test_dataset.augmentation.eraser_aug_prob",
        "dataset.test_dataset.augmentation.h_flip_prob",
        "dataset.test_dataset.augmentation.hshift_prob",
        "dataset.test_dataset.augmentation.spatial_aug_prob",
        "dataset.test_dataset.augmentation.stretch_prob",
        "dataset.test_dataset.augmentation.v_flip_prob",
        "dataset.test_dataset.augmentation.yjitter_prob",
        "dataset.train_dataset.augmentation.color_aug_prob",
        "dataset.train_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.train_dataset.augmentation.eraser_aug_prob",
        "dataset.train_dataset.augmentation.h_flip_prob",
        "dataset.train_dataset.augmentation.hshift_prob",
        "dataset.train_dataset.augmentation.spatial_aug_prob",
        "dataset.train_dataset.augmentation.stretch_prob",
        "dataset.train_dataset.augmentation.v_flip_prob",
        "dataset.train_dataset.augmentation.yjitter_prob",
        "dataset.val_dataset.augmentation.color_aug_prob",
        "dataset.val_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.val_dataset.augmentation.eraser_aug_prob",
        "dataset.val_dataset.augmentation.h_flip_prob",
        "dataset.val_dataset.augmentation.hshift_prob",
        "dataset.val_dataset.augmentation.spatial_aug_prob",
        "dataset.val_dataset.augmentation.stretch_prob",
        "dataset.val_dataset.augmentation.v_flip_prob",
        "dataset.val_dataset.augmentation.yjitter_prob",
        "model.corr_radius",
        "model.cv_group",
        "model.volume_dim",
        "train.optim.lr",
        "train.optim.lr_decay",
        "train.optim.lr_step_size",
        "train.optim.min_lr",
        "train.optim.momentum",
        "train.optim.weight_decay"
      ],
      "automl_disabled_parameters": [
        "dataset",
        "dataset.infer_dataset",
        "dataset.infer_dataset.augmentation",
        "dataset.infer_dataset.augmentation.color_aug_hue_range",
        "dataset.infer_dataset.augmentation.color_aug_saturation",
        "dataset.infer_dataset.augmentation.crop_size",
        "dataset.infer_dataset.augmentation.gamma",
        "dataset.infer_dataset.augmentation.input_mean",
        "dataset.infer_dataset.augmentation.input_std",
        "dataset.infer_dataset.data_sources",
        "dataset.quant_calibration_dataset",
        "dataset.test_dataset",
        "dataset.test_dataset.augmentation",
        "dataset.test_dataset.augmentation.color_aug_hue_range",
        "dataset.test_dataset.augmentation.color_aug_saturation",
        "dataset.test_dataset.augmentation.crop_size",
        "dataset.test_dataset.augmentation.gamma",
        "dataset.test_dataset.augmentation.input_mean",
        "dataset.test_dataset.augmentation.input_std",
        "dataset.test_dataset.data_sources",
        "dataset.train_dataset",
        "dataset.train_dataset.augmentation",
        "dataset.train_dataset.augmentation.color_aug_hue_range",
        "dataset.train_dataset.augmentation.color_aug_saturation",
        "dataset.train_dataset.augmentation.crop_size",
        "dataset.train_dataset.augmentation.gamma",
        "dataset.train_dataset.augmentation.input_mean",
        "dataset.train_dataset.augmentation.input_std",
        "dataset.train_dataset.data_sources",
        "dataset.val_dataset",
        "dataset.val_dataset.augmentation",
        "dataset.val_dataset.augmentation.color_aug_hue_range",
        "dataset.val_dataset.augmentation.color_aug_saturation",
        "dataset.val_dataset.augmentation.crop_size",
        "dataset.val_dataset.augmentation.gamma",
        "dataset.val_dataset.augmentation.input_mean",
        "dataset.val_dataset.augmentation.input_std",
        "dataset.val_dataset.data_sources",
        "evaluate",
        "evaluate.gpu_ids",
        "export",
        "gen_trt_engine",
        "gen_trt_engine.tensorrt",
        "gen_trt_engine.tensorrt.layers_precision",
        "inference",
        "inference.gpu_ids",
        "model",
        "model.hidden_dims",
        "model.mono_backbone",
        "model.stereo_backbone",
        "quantize",
        "quantize.backend_kwargs",
        "quantize.layers",
        "quantize.skip_names",
        "train",
        "train.cudnn",
        "train.gpu_ids",
        "train.optim",
        "train.optim.lr_steps",
        "train.tile_min_overlap",
        "wandb",
        "wandb.tags"
      ],
      "core_module": "depth_net",
      "path": "schemas/quantize.schema.json",
      "popular": {
        "evaluate": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "gen_trt_engine": {
          "batch_size": -1,
          "gpu_id": 0,
          "tensorrt": {
            "max_batch_size": 1,
            "min_batch_size": 1,
            "opt_batch_size": 1
          }
        },
        "inference": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "train": {
          "checkpoint_interval": 1,
          "gpu_ids": [
            0
          ],
          "num_epochs": 10,
          "num_gpus": 1,
          "num_nodes": 1,
          "validation_interval": 1
        }
      },
      "schema_action": "quantize",
      "spec_template": "references/spec_template_quantize.yaml"
    },
    "train": {
      "automl_default_parameters": [
        "dataset.infer_dataset.augmentation.color_aug_prob",
        "dataset.infer_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.infer_dataset.augmentation.eraser_aug_prob",
        "dataset.infer_dataset.augmentation.h_flip_prob",
        "dataset.infer_dataset.augmentation.hshift_prob",
        "dataset.infer_dataset.augmentation.spatial_aug_prob",
        "dataset.infer_dataset.augmentation.stretch_prob",
        "dataset.infer_dataset.augmentation.v_flip_prob",
        "dataset.infer_dataset.augmentation.yjitter_prob",
        "dataset.test_dataset.augmentation.color_aug_prob",
        "dataset.test_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.test_dataset.augmentation.eraser_aug_prob",
        "dataset.test_dataset.augmentation.h_flip_prob",
        "dataset.test_dataset.augmentation.hshift_prob",
        "dataset.test_dataset.augmentation.spatial_aug_prob",
        "dataset.test_dataset.augmentation.stretch_prob",
        "dataset.test_dataset.augmentation.v_flip_prob",
        "dataset.test_dataset.augmentation.yjitter_prob",
        "dataset.train_dataset.augmentation.color_aug_prob",
        "dataset.train_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.train_dataset.augmentation.eraser_aug_prob",
        "dataset.train_dataset.augmentation.h_flip_prob",
        "dataset.train_dataset.augmentation.hshift_prob",
        "dataset.train_dataset.augmentation.spatial_aug_prob",
        "dataset.train_dataset.augmentation.stretch_prob",
        "dataset.train_dataset.augmentation.v_flip_prob",
        "dataset.train_dataset.augmentation.yjitter_prob",
        "dataset.val_dataset.augmentation.color_aug_prob",
        "dataset.val_dataset.augmentation.crop_min_valid_disp_ratio",
        "dataset.val_dataset.augmentation.eraser_aug_prob",
        "dataset.val_dataset.augmentation.h_flip_prob",
        "dataset.val_dataset.augmentation.hshift_prob",
        "dataset.val_dataset.augmentation.spatial_aug_prob",
        "dataset.val_dataset.augmentation.stretch_prob",
        "dataset.val_dataset.augmentation.v_flip_prob",
        "dataset.val_dataset.augmentation.yjitter_prob",
        "model.corr_radius",
        "model.cv_group",
        "model.volume_dim",
        "train.optim.lr",
        "train.optim.lr_decay",
        "train.optim.lr_step_size",
        "train.optim.min_lr",
        "train.optim.momentum",
        "train.optim.weight_decay"
      ],
      "automl_disabled_parameters": [
        "dataset",
        "dataset.infer_dataset",
        "dataset.infer_dataset.augmentation",
        "dataset.infer_dataset.augmentation.color_aug_hue_range",
        "dataset.infer_dataset.augmentation.color_aug_saturation",
        "dataset.infer_dataset.augmentation.crop_size",
        "dataset.infer_dataset.augmentation.gamma",
        "dataset.infer_dataset.augmentation.input_mean",
        "dataset.infer_dataset.augmentation.input_std",
        "dataset.infer_dataset.data_sources",
        "dataset.quant_calibration_dataset",
        "dataset.test_dataset",
        "dataset.test_dataset.augmentation",
        "dataset.test_dataset.augmentation.color_aug_hue_range",
        "dataset.test_dataset.augmentation.color_aug_saturation",
        "dataset.test_dataset.augmentation.crop_size",
        "dataset.test_dataset.augmentation.gamma",
        "dataset.test_dataset.augmentation.input_mean",
        "dataset.test_dataset.augmentation.input_std",
        "dataset.test_dataset.data_sources",
        "dataset.train_dataset",
        "dataset.train_dataset.augmentation",
        "dataset.train_dataset.augmentation.color_aug_hue_range",
        "dataset.train_dataset.augmentation.color_aug_saturation",
        "dataset.train_dataset.augmentation.crop_size",
        "dataset.train_dataset.augmentation.gamma",
        "dataset.train_dataset.augmentation.input_mean",
        "dataset.train_dataset.augmentation.input_std",
        "dataset.train_dataset.data_sources",
        "dataset.val_dataset",
        "dataset.val_dataset.augmentation",
        "dataset.val_dataset.augmentation.color_aug_hue_range",
        "dataset.val_dataset.augmentation.color_aug_saturation",
        "dataset.val_dataset.augmentation.crop_size",
        "dataset.val_dataset.augmentation.gamma",
        "dataset.val_dataset.augmentation.input_mean",
        "dataset.val_dataset.augmentation.input_std",
        "dataset.val_dataset.data_sources",
        "evaluate",
        "evaluate.gpu_ids",
        "export",
        "gen_trt_engine",
        "gen_trt_engine.tensorrt",
        "gen_trt_engine.tensorrt.layers_precision",
        "inference",
        "inference.gpu_ids",
        "model",
        "model.hidden_dims",
        "model.mono_backbone",
        "model.stereo_backbone",
        "quantize",
        "quantize.backend_kwargs",
        "quantize.layers",
        "quantize.skip_names",
        "train",
        "train.cudnn",
        "train.gpu_ids",
        "train.optim",
        "train.optim.lr_steps",
        "train.tile_min_overlap",
        "wandb",
        "wandb.tags"
      ],
      "core_module": "depth_net",
      "path": "schemas/train.schema.json",
      "popular": {
        "evaluate": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "gen_trt_engine": {
          "batch_size": -1,
          "gpu_id": 0,
          "tensorrt": {
            "max_batch_size": 1,
            "min_batch_size": 1,
            "opt_batch_size": 1
          }
        },
        "inference": {
          "gpu_ids": [
            0
          ],
          "num_gpus": 1,
          "num_nodes": 1
        },
        "train": {
          "checkpoint_interval": 1,
          "gpu_ids": [
            0
          ],
          "num_epochs": 10,
          "num_gpus": 1,
          "num_nodes": 1,
          "validation_interval": 1
        }
      },
      "schema_action": "train",
      "spec_template": "references/spec_template_train.yaml"
    }
  },
  "automl_enabled": true,
  "failures": {},
  "model": "depth-net-mono",
  "network_arch": "depth_net_mono",
  "schema_version": 1
}

Description:

Monocular depth estimation using Metric Depth Anything v2 or Relative Depth Anything architectures, predicting per-pixel depth from single RGB images for training, evaluating, exporting, or running inference with NVIDIA TAO.

This skill is ready for commercial/non-commercial use.

Owner

NVIDIA

License/Terms of Use:

Apache-2.0

Use Case:

Developers and engineers training, evaluating, exporting, or deploying monocular depth estimation models using NVIDIA TAO Toolkit with docker-based GPU workflows.

Deployment Geography for Use:

Global

Known Risks and Mitigations:

Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. Mitigation: Review and scan skill before deployment.

Reference(s):

Parameters Reference
Finetuning Recipes
Spec Overrides
TAO Deploy Depth Anything V2
Troubleshooting

Skill Output:

Output Type(s): [Shell commands, Configuration instructions] Output Format: [Markdown with inline bash code blocks] Output Parameters: [1D] Other Properties Related to Output: [None]

Evaluation Agents Used:

Claude Code (claude-code)
Codex (codex)

Evaluation Tasks:

Evaluated against 1 evaluation task in the NVSkills-Eval external profile (astra-sandbox environment).

Evaluation Metrics Used:

Reported benchmark dimensions:

Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
Correctness: Checks whether the agent follows the expected workflow and produces the correct final output.
Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
Effectiveness: Checks whether the agent performs measurably better with the skill than without it.
Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work.

Underlying evaluation signals used in this run:

security: Checks for unsafe operations, secret leakage, and unauthorized access.
skill_execution: Verifies that the agent loaded the expected skill and workflow.
skill_efficiency: Checks routing quality, decoy avoidance, and redundant tool usage.
accuracy: Grades final-answer correctness against the reference answer.
goal_accuracy: Checks whether the overall user task completed successfully.
behavior_check: Verifies expected behavior steps, including safety expectations.
token_efficiency: Compares token usage with and without the skill.

Evaluation Results:

Dimension	Num	`claude-code`	`codex`
Security	1	100% (+0%)	100% (+0%)
Correctness	1	100% (+90%)	20% (+20%)
Discoverability	1	88% (+88%)	0% (+0%)
Effectiveness	1	100% (+72%)	48% (+30%)
Efficiency	1	71% (+44%)	28% (+0%)

Skill Version(s):

0.1.0 (source: frontmatter)

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

(For Release on NVIDIA Platforms Only) Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

Related skills

Microsoft FoundryDeploy, evaluate, and continuously improve Microsoft Foundry agents from a single agent interface.490k1.3k

Ai Research ReproductionOrchestrate trustworthy, auditable reproduction of deep learning repositories directly from their READMEs.176k511

Run TrainSafely execute selected deep learning training commands with standardized evidence capture.176k511

Explore RunSafely run isolated exploratory experiments with clear recording and conservative selection before committing changes.176k511

Paper Context ResolverFetch precise reproduction-critical details like dataset splits, preprocessing steps, or evaluation protocols from the original academic paper when the repo README leav141k511

Repo Intake And PlanScan unfamiliar AI research repositories and receive a minimal, trustworthy reproduction target before investing significant time.140k511

How it compares

Pick tao-train-depth-anything-v2 over generic CV training skills when you need TAO-specific depth_net YAML pairing, NYU unit-conversion dataset classes, and TensorRT deploy workflows for single-image depth.

FAQ

What depth model types does tao-train-depth-anything-v2 support?

tao-train-depth-anything-v2 supports MetricDepthAnything for absolute depth in meters and RelativeDepthAnything for relative depth ordering. Each model_type must pair with a matching dataset_name such as MetricMonoDataset or NYUDV2 to avoid silent NaN training loss from encoding

What GPU setup does TAO depth training require?

tao-train-depth-anything-v2 requires Docker plus nvidia-container-toolkit and recommends 24GB+ VRAM per GPU because the ViT-Large encoder is memory intensive. Training supports fp32 or bf16 on Ampere SM80+ hardware with optional activation checkpointing for larger inputs.

Which TAO CLI actions does the depth skill cover?

tao-train-depth-anything-v2 documents PyT depth_net actions for train, evaluate, inference, export, and quantize, plus TAO Deploy gen_trt_engine for TensorRT engine generation. The mono and stereo families share the depth_net CLI selected via model.model_type in the YAML spec.

Data Science & MLpipelinesanalytics

About

Tao Train Depth Anything V2 by the numbers

Add your badge

How do you fine-tune Depth Anything V2 in NVIDIA TAO?

Who is it for?

When should I use this skill?

What you get

By the numbers

Files

Depth Net Mono

Train Action Policy

Workflow

Prerequisites — data accessibility

Step 1 — Annotation file

Step 2 — Pair model_type and dataset_name based on your data

Step 3 — Write spec yaml from Typical Spec Overrides

Step 4 — Run

Step 5 — Verify

Training Requirements

Per-Action Dataset Requirements

Typical Spec Overrides

Eval Dataset

Important Parameters

Finetuning Recipes

Multi-GPU / Multi-Node

Export / TRT Defaults

Hardware

Error Patterns

Spec Param / Parent Model Inference

Deployment

Evaluation Report

Evaluation Summary

Agents Used

Metrics Used

Test Tasks

Results

Tier 1: Static Validation Summary

Tier 2: Deduplication Summary

Publication Recommendation

Finetuning Recipes

Relative Variant Finetuning Recipe

Metric Variant Finetuning Recipe

Important Parameters

Pretrained checkpoint loading — use case matrix

Typical Spec Overrides

Spec Param / Parent Model Inference

DepthNet Mono Deploy

Quick Start

Generate TensorRT Engine

Evaluate TensorRT Engine

TensorRT Inference

Deploy Workflow

Required Inputs

Spec Templates

Relative variant (default)

Metric variant

Spec filename invariant

Job Chain Mapping

Outputs

Common errors

Error Patterns

Description: <br>

Owner

License/Terms of Use: <br>

Use Case: <br>

Deployment Geography for Use: <br>

Known Risks and Mitigations: <br>

Reference(s): <br>

Skill Output: <br>

Evaluation Agents Used: <br>

Evaluation Tasks: <br>

Evaluation Metrics Used: <br>

Evaluation Results: <br>

Skill Version(s): <br>

Ethical Considerations: <br>

Related skills

How it compares

FAQ

What depth model types does tao-train-depth-anything-v2 support?

What GPU setup does TAO depth training require?

Step 2 — Pair `model_type` and `dataset_name` based on your data