VSS Deploy Detection & Tracking — 3D (RTVI-CV-3D)

Bring up the RTVI-CV-3D stack from the warehouse blueprint: per-camera DeepStream perception (

vss-rtvi-cv-mv3dt

) + BEV Fusion (

vss-rtvi-cv-bev-fusion

) + mosquitto MQTT bus + broker + VST sensor stack — without the agent / LLM / VLM stack that comes with the full warehouse blueprint.

The actual compose machinery lives in

deploy/docker/industry-profiles/warehouse-operations/warehouse-mv3dt-app/

. This skill drives the env overrides, calibration chain, and verification.

Routing

Ask the user at most four questions, then dispatch.

Q0 — Profile size (overlays or not)

Default to extended unless the user explicitly asks for minimal. Extended deploys ELK +

vss-video-analytics-api-mv3dt

vss-kibana-init-mv3dt

vss-import-calibration-output-mv3dt

on top of MV3DT core — these are what the VST video wall needs to render bounding-box overlays. Without them, the video wall works but shows raw streams without overlays.

User answer	`MINIMAL_PROFILE`	What you get	When to choose
extended (default)	`""`	MV3DT core + ELK + analytics API + Kibana. Overlays work in VST video wall. Recommended for a complete e2e experience.	"I want the full e2e experience", "I want to see bounding boxes", or no preference stated
minimal	`"true"`	MV3DT core only. ~5 fewer containers. No overlays in VST. Metadata still on Kafka/Redis.	"I only need the data", "edge / Thor host", "minimum footprint"

Note on selective ELK: there's no "minimal + ELK only" middle path in the current compose. Every
${MINIMAL_PROFILE:+_extended}
-gated service comes up together (ES, Logstash, Kibana, video-analytics-api, kibana-init, import-calibration).
bash
's
:+
parameter expansion produces the
_extended
suffix when
MINIMAL_PROFILE
is set; extended switches the gating string back to plain
bp_wh_kafka_mv3dt
which the active compose profile already matches. Either you accept the full extended bundle or you stay minimal.

Q1 — Data source

Ask this unless the source is explicit in the user's first message. A bare request like "deploy rtvi-cv-3d" does not imply

sample

sample — the bundled 4-camera synthetic dataset (
```
warehouse-4cams-20mx20m-synthetic
```
). Calibration ships in-tree; no AMC run needed.
videos — the user has local video files (any
```
*.mp4
```
named after their cameras). Standalone AMC (
```
auto_calib
```
profile) will run if calibration is missing.
rtsp — the user has live RTSP URLs. Calibration via VIOS-driven AMC.

Q2 — Calibration coverage (skip for

sample

)

For

videos

and

rtsp

, check whether calibration is already on disk at the mount path the perception container expects:

bash

DATASET="${SAMPLE_VIDEO_DATASET:?}"          # the user's dataset slug; see Q3
CAL_DIR="${VSS_APPS_DIR}/industry-profiles/warehouse-operations/warehouse-mv3dt-app/calibration/sample-data/${DATASET}"

# Look for ANY of: calibration.json, plus camInfo/*.yml or *.yaml with either
# 'cam_*' or 'Camera*' naming (the shipped sample uses Camera*.yml, AMC may
# produce cam_*.yaml — broaden accordingly)
test -f "${CAL_DIR}/calibration.json" \
  && ls "${CAL_DIR}/camInfo/"*.{yml,yaml} 2>/dev/null

If the user supplied a calibration path themselves, validate that path instead — don't recompute. See

configure-cameras.md

for the authoritative camera-count discovery (parses

calibration.json

Q3 — Detector + dataset slug (only when Q2 triggers AMC)

```
resnet
```
(default, fast) or
```
transformer
```
(slower, better under occlusion) — passed to the AMC
```
/v1/calibrate/<id>
```
API at Step B (see
```
vss-generate-video-calibration/SKILL.md:48-62
```
).
A short kebab-case dataset slug used as
```
SAMPLE_VIDEO_DATASET
```
(e.g.
```
customer-aisle-4cams
```
). This drives the calibration mount path and gets persisted in
```
.env
```
.

Routing table

Q1	Q2 result	Path
`sample`	(cal ships in-tree)	`references/deploy-rtvi-cv-3d-stack.md` directly
`videos`	cal present	`references/deploy-rtvi-cv-3d-stack.md` directly
`videos`	cal missing	`references/calibration-workflow.md` (videos mode) → `references/configure-cameras.md` → `references/deploy-rtvi-cv-3d-stack.md`
`rtsp`	cal present	`references/deploy-rtvi-cv-3d-stack.md` directly
`rtsp`	cal missing	`references/calibration-workflow.md` (rtsp mode) → `references/configure-cameras.md` → `references/deploy-rtvi-cv-3d-stack.md`

Every path converges on

references/verify-and-view.md

once

up -d

completes.

references/troubleshooting.md

and

references/teardown.md

are linked but off the happy path.

Disambiguation rule. If the user mentions "warehouse" without "mv3dt" / "3D tracking" / "multi-view", consider routing to

../vss-deploy-profile/references/warehouse.md

instead — that's the full warehouse blueprint (2D / 3D / MV3DT + agents). This skill is for MV3DT only without the agent stack / LLM / VLM.

Prerequisites

1. Repo path

Locate

video-search-and-summarization/

on disk. All compose commands run from

<repo>/deploy/docker/

. If unknown, ask the user.

2. NGC CLI + key

$NGC_CLI_API_KEY

must be set. Both

nvidia/vss-core/*

and

nvstaging/vss-core/*

are valid orgs depending on which the user's key resolves to. See

vss-deploy-profile/references/ngc.md

for setup if missing.

If the user previously ran

ngc config set

but

$NGC_CLI_API_KEY

isn't exported in this shell, the key is already on disk:

bash

NGC_CLI_API_KEY=$(awk -F'= ' '/^apikey/{print $2}' ~/.ngc/config 2>/dev/null)
test -n "${NGC_CLI_API_KEY}" && echo "key sourced from ~/.ngc/config"

Make sure the key value also lands in

industry-profiles/warehouse-operations/.env:164

(

NGC_CLI_API_KEY=...

) — compose only reads it from there at

up

time, not from your shell env.

HARDWARE_PROFILE

slug

The canonical
HARDWARE_PROFILE
keys live in
industry-profiles/warehouse-operations/blueprint-configurator/blueprint_config.yml
(lines 592–642). Use the slug from the table below — e.g. on an RTX A6000 (Ampere) host the value is
RTXA6000
.

Pick from

nvidia-smi --query-gpu=name --format=csv,noheader

GPU name	`HARDWARE_PROFILE`	MV3DT `max_streams_supported`
RTX PRO 6000 Blackwell	`RTXPRO6000BW`	18
H100 (NVL, SXM HBM3)	`H100`	13
RTX A6000 Ada Generation	`RTXA6000ADA`	6
L40S	`L40S`	7
L4	`L4`	2
RTX A6000 (Ampere)	`RTXA6000`	2
IGX Thor	`IGX-THOR`	7
DGX Spark	`DGX-SPARK`	4

The per-GPU MV3DT cap is enforced at deploy time.

vss-configurator-mv3dt

computes

final_stream_count = min(NUM_STREAMS, max_streams_supported)

and applies a

keep_count

file-management op against

${VSS_DATA_DIR}/videos/${SAMPLE_VIDEO_DATASET}/

so only

final_stream_count

.mp4

files remain (sorted lexicographically, last N kept). If your GPU's

mv3dt

cap (above table) is below your camera count, perception /

mdx-raw

mdx-bev

run with the cap's worth of streams. Either pick a GPU with a higher cap or surface the cap explicitly to the user so they're aware which streams will be processed.

4. App data on disk

VSS_DATA_DIR

must point at the extracted
vss-warehouse-app-data
directory (separate from the repo). Pointing it at the repo's

deploy/docker/

causes the deploy to stall: the configurator can't find the dataset, redis can't open its log file, and perception stays in

Created

. Verify the path before deploy.

Pre-flight check before deploy:

bash

DATA_DIR="${VSS_DATA_DIR:?VSS_DATA_DIR not set in .env}"
DATASET="${SAMPLE_VIDEO_DATASET:-warehouse-4cams-20mx20m-synthetic}"

for sub in videos models data_log; do
  test -d "${DATA_DIR}/${sub}" || { echo "ERROR: ${DATA_DIR}/${sub} missing"; exit 1; }
done

# For sample / videos modes — videos directory must exist
test -d "${DATA_DIR}/videos/${DATASET}" \
  || { echo "ERROR: ${DATA_DIR}/videos/${DATASET} missing — wrong slug or app-data not extracted"; exit 1; }

# Sanity: video count should match calibration count.
# Some published app-data tarballs are known to ship the sample dataset with
# fewer videos than the dataset name implies — verify and source any missing
# cams separately if your GPU's mv3dt cap is high enough to use them all.
ls "${DATA_DIR}/videos/${DATASET}/"*.mp4 2>/dev/null | wc -l

# Ensure every per-service subdir under data_log/ exists, then open perms.
# kafka / elasticsearch / redis run as different non-root UIDs against
# bind-mounted host paths — without 777 the daemons exit with
# "Permission denied" (kafka cluster_id), "AccessDeniedException" (ES),
# or "Can't open the log file" (redis). chmod 777 is the documented fix;
# do NOT recursive-chown — see data-directory.md for the per-UID rationale.
mkdir -p \
  "${DATA_DIR}/data_log/analytics_cache" \
  "${DATA_DIR}/data_log/calibration_toolkit" \
  "${DATA_DIR}/data_log/elastic/data" \
  "${DATA_DIR}/data_log/elastic/logs" \
  "${DATA_DIR}/data_log/kafka" \
  "${DATA_DIR}/data_log/redis/data" \
  "${DATA_DIR}/data_log/redis/log"
chmod -R 777 "${DATA_DIR}/data_log"

Easy miss, hard to recover from. The
mkdir -p
+
chmod -R 777
step on
${DATA_DIR}/data_log
is required, not optional. Newly extracted
vss-warehouse-app-data
trees are owned by the extracting user (whoever ran
tar -xvf
) and container UIDs won't match. The deeper per-container UID table lives in
../vss-deploy-profile/references/data-directory.md
; the same doc explains why recursive chown is the wrong fix.

If app-data isn't extracted yet: download via

ngc registry resource download-version "nvidia/vss-warehouse/vss-warehouse-app-data:<version>"

and

tar -xvf

(see

references/deploy-rtvi-cv-3d-stack.md

for tag discovery and full steps).

5. Pre-flight (system)

nvidia-smi

, NVIDIA Docker runtime visible (

docker info | grep -i runtimes

), and

docker run --rm --gpus all ubuntu:24.04 nvidia-smi

all green. Full driver / kernel / sysctl checks live in

vss-deploy-profile/references/prerequisites.md

If any check fails, fix before continuing — don't proceed to deploy.

6. Browser reachability (cloud / corp-VPN hosts only)

If the user will view the VST video wall through a browser on a different network than the deploy host (cloud VM, corp VPN, ssh-tunnelled session), upstream firewall rules may block VST WebRTC (STUN to

stun.l.google.com:19302

, plus random UDP for media). See

references/verify-and-view.md#browser-reachability

for symptoms and workarounds. Also: some hosts block the AMC microservice's default port (TCP/8010); if the user reports the AMC UI on

:5000

works but its data calls fail, retry with a different

VSS_AUTO_CALIBRATION_PORT

How it fits together

SKILL.md (this file — Q0/Q1/Q2/Q3 routing)
  └─ if cal missing ─> calibration-workflow.md
  │                     └─ chains to vss-generate-video-calibration (deploy + drive API)
  │                     └─ fetches /v1/result/{project_id}/mv3dt_result?result_type=amc
  │                     └─ lands calibration files at warehouse-mv3dt-app/calibration/sample-data/<slug>/
  ├─> configure-cameras.md (NUM_STREAMS sync, VST sensor trim)
  └─> deploy-rtvi-cv-3d-stack.md (compose up with bp_wh_kafka_mv3dt + extended/minimal)
        └─> verify-and-view.md (FPS, fusion_ready, mdx-bev, VST video wall + WebRTC checks)

Related Skills

```
vss-generate-video-calibration
```
— the AMC skill. Owns the
```
auto_calib
```
compose profile, calibration API, and the
```
/v1/result/.../mv3dt_result
```
export hook this skill consumes.
```
calibration-workflow.md
```
chains into it.
```
vss-deploy-profile
```
— cross-profile umbrella. Use that instead when the user wants the full warehouse blueprint (with agents / LLM / VLM), not just MV3DT.
```
vss-manage-video-io-storage
```
— VIOS / VST API skill. Useful for the VST video wall (overlay viz) and for sensor management referenced in
```
configure-cameras.md
```
.

The repo's authoritative warehouse-blueprint reference at

../vss-deploy-profile/references/warehouse.md

covers 2D / 3D / MV3DT inside the full warehouse stack — this skill is the MV3DT-only companion that trims the agent / LLM / VLM layer.

vss-deploy-detection-tracking-3d

NPX Install

Tags

SKILL.md Content

VSS Deploy Detection & Tracking — 3D (RTVI-CV-3D)

Routing

Q0 — Profile size (overlays or not)

Q1 — Data source

Q2 — Calibration coverage (skip for
`sample`
)

Q3 — Detector + dataset slug (only when Q2 triggers AMC)

Routing table

Prerequisites

1. Repo path

2. NGC CLI + key

3.
`HARDWARE_PROFILE`
slug

4. App data on disk

5. Pre-flight (system)

6. Browser reachability (cloud / corp-VPN hosts only)

How it fits together

Related Skills

vss-deploy-detection-tracking-3d

NPX Install

Tags

SKILL.md Content

VSS Deploy Detection & Tracking — 3D (RTVI-CV-3D)

Routing

Q0 — Profile size (overlays or not)

Q1 — Data source

Q2 — Calibration coverage (skip for sample)

Q3 — Detector + dataset slug (only when Q2 triggers AMC)

Routing table

Prerequisites

1. Repo path

2. NGC CLI + key

3. HARDWARE_PROFILE slug

4. App data on disk

5. Pre-flight (system)

6. Browser reachability (cloud / corp-VPN hosts only)

How it fits together

Related Skills

Q2 — Calibration coverage (skip for
`sample`
)

3.
`HARDWARE_PROFILE`
slug