deepstream-run-mv3dt
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseSkill: Run DeepStream MV3DT
技能:运行DeepStream MV3DT
When to Use This Skill
何时使用本技能
Activate this skill when the user wants to set up, run, verify, or debug the DeepStream Multi-View 3D Tracking reference app. Typical prompts:
- "set up MV3DT DeepStream"
- "run the 4-camera MV3DT sample"
- "run the 12-camera MV3DT sample"
- "run MV3DT on my synchronized MP4s"
- "I have videos but no calibration; calibrate and run MV3DT"
- "show the BEV visualizer or Kafka metadata"
- "stop MV3DT" / "clean up MV3DT" / "tear down MV3DT"
Do not use this skill for single-view 3D tracking, generic DeepStream app development, or live-stream onboarding unless the user explicitly maps that work to this repo's MV3DT pipeline.
当用户想要设置、运行、验证或调试DeepStream多视图3D追踪参考应用时,激活本技能。典型请求示例:
- "设置MV3DT DeepStream"
- "运行4相机MV3DT示例"
- "运行12相机MV3DT示例"
- "在我的同步MP4文件上运行MV3DT"
- "我有视频但没有校准信息;完成校准并运行MV3DT"
- "展示BEV可视化工具或Kafka元数据"
- "停止MV3DT" / "清理MV3DT" / "关闭MV3DT"
请勿将本技能用于单视图3D追踪、通用DeepStream应用开发或直播入门操作,除非用户明确将相关工作映射到本仓库的MV3DT流水线。
Examples
示例场景
- "Deploy the DeepStream MV3DT 4-camera sample and show the OSD and BEV windows."
- "Run the MV3DT 12-camera sample headlessly with RTDETR and save videos."
- "Run MV3DT on synchronized MP4s under /data/mv3dt-demo using PeopleNetTransformer."
- "My custom MV3DT videos do not have calibration; use AutoMagicCalib, then run MV3DT."
- "部署DeepStream MV3DT 4相机示例并展示OSD和BEV窗口。"
- "以无头模式运行MV3DT 12相机示例,使用RTDETR并保存视频。"
- "在/data/mv3dt-demo下的同步MP4文件上运行MV3DT,使用PeopleNetTransformer。"
- "我的自定义MV3DT视频没有校准信息;使用AutoMagicCalib完成校准后运行MV3DT。"
Overview
概述
Operate the Multi-View 3D Tracking reference app in using the DeepStream Container path. The skill supports setup, shipped sample runs, custom synchronized MP4 datasets, calibration handoff to AutoMagicCalib, display/headless execution, OSD/BEV outputs, and Kafka metadata inspection.
DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-view通过DeepStream容器路径,操作位于中的多视图3D追踪参考应用。本技能支持环境设置、官方示例运行、自定义同步MP4数据集处理、将校准任务委托给AutoMagicCalib、显示/无头模式执行、OSD/BEV输出以及Kafka元数据查看。
DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-viewPrerequisites
前置条件
- MV3DT reference app directory on disk under ; if it is absent, ask before cloning the public DeepStream repo
DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-view - Docker with NVIDIA GPU support
- DeepStream container image access
- MV3DT sample datasets, models, custom parser libraries, Kafka, Mosquitto, and prepared by the repo setup script
mv3dt_venv - Working X11/VNC display for live OSD/BEV windows, or the saved-output headless path for tiled DeepStream MP4 plus Kafka-derived BEV MP4
- 磁盘上存在MV3DT参考应用目录,路径为;若目录不存在,需先询问用户是否同意克隆公开的DeepStream仓库
DeepStream/src/apps/reference_apps/deepstream-tracker-3d-multi-view - 支持NVIDIA GPU的Docker环境
- 可访问DeepStream容器镜像
- 仓库设置脚本已准备好MV3DT示例数据集、模型、自定义解析库、Kafka、Mosquitto和虚拟环境
mv3dt_venv - 可用的X11/VNC显示环境(用于实时OSD/BEV窗口),或采用无头模式保存输出(生成拼接后的DeepStream MP4文件以及基于Kafka的BEV MP4文件)
Instructions
操作步骤
Step 0: Resolve MV3DT App Checkout
步骤0:确认MV3DT应用目录
The skill can be installed outside the DeepStream repo. Resolve to the MV3DT app directory, not necessarily the Git top-level directory.
REPO_ROOTbash
MV3DT_APP_SUBDIR="src/apps/reference_apps/deepstream-tracker-3d-multi-view"
is_mv3dt_app_dir() {
test -f "$1/README.md" || return 1
test -d "$1/config_templates" || return 1
test -d "$1/scripts" || return 1
grep -q "Multi-View 3D Tracking" "$1/README.md"
}
GIT_TOP="$(git rev-parse --show-toplevel 2>/dev/null || true)"
CANDIDATES=()
if [ -n "${MV3DT_REPO_ROOT:-}" ]; then CANDIDATES+=("${MV3DT_REPO_ROOT}"); fi
if [ -n "${DEEPSTREAM_REPO_ROOT:-}" ]; then CANDIDATES+=("${DEEPSTREAM_REPO_ROOT}/${MV3DT_APP_SUBDIR}"); fi
CANDIDATES+=("${PWD}")
if [ -n "${GIT_TOP}" ]; then
CANDIDATES+=("${GIT_TOP}" "${GIT_TOP}/${MV3DT_APP_SUBDIR}")
fi
CANDIDATES+=("${HOME}/DeepStream/${MV3DT_APP_SUBDIR}" "${HOME}/deepstream/${MV3DT_APP_SUBDIR}")
REPO_ROOT=""
for candidate in "${CANDIDATES[@]}"; do
if [ -n "$candidate" ] && is_mv3dt_app_dir "$candidate"; then
REPO_ROOT="$(cd "$candidate" && pwd)"
break
fi
done
if [ -z "${REPO_ROOT}" ]; then
cat <<'EOF'
ERROR: MV3DT reference app directory was not found.
Set MV3DT_REPO_ROOT to an existing deepstream-tracker-3d-multi-view app directory, or ask the user to approve cloning the public DeepStream repo and then run:
DEEPSTREAM_REPO_ROOT="${DEEPSTREAM_REPO_ROOT:-$HOME/DeepStream}"
git clone https://github.com/NVIDIA/DeepStream.git "$DEEPSTREAM_REPO_ROOT"
export MV3DT_REPO_ROOT="$DEEPSTREAM_REPO_ROOT/src/apps/reference_apps/deepstream-tracker-3d-multi-view"
Do not clone silently.
EOF
exit 1
fi
cd "${REPO_ROOT}"
export REPO_ROOT MV3DT_REPO_ROOT="${REPO_ROOT}"If the app directory cannot be resolved, ask the user for an existing checkout path or for approval to clone . Do not clone silently.
https://github.com/NVIDIA/DeepStream本技能可安装在DeepStream仓库外部。需将解析为MV3DT应用目录,而非Git顶层目录。
REPO_ROOTbash
MV3DT_APP_SUBDIR="src/apps/reference_apps/deepstream-tracker-3d-multi-view"
is_mv3dt_app_dir() {
test -f "$1/README.md" || return 1
test -d "$1/config_templates" || return 1
test -d "$1/scripts" || return 1
grep -q "Multi-View 3D Tracking" "$1/README.md"
}
GIT_TOP="$(git rev-parse --show-toplevel 2>/dev/null || true)"
CANDIDATES=()
if [ -n "${MV3DT_REPO_ROOT:-}" ]; then CANDIDATES+=("${MV3DT_REPO_ROOT}"); fi
if [ -n "${DEEPSTREAM_REPO_ROOT:-}" ]; then CANDIDATES+=("${DEEPSTREAM_REPO_ROOT}/${MV3DT_APP_SUBDIR}"); fi
CANDIDATES+=("${PWD}")
if [ -n "${GIT_TOP}" ]; then
CANDIDATES+=("${GIT_TOP}" "${GIT_TOP}/${MV3DT_APP_SUBDIR}")
fi
CANDIDATES+=("${HOME}/DeepStream/${MV3DT_APP_SUBDIR}" "${HOME}/deepstream/${MV3DT_APP_SUBDIR}")
REPO_ROOT=""
for candidate in "${CANDIDATES[@]}"; do
if [ -n "$candidate" ] && is_mv3dt_app_dir "$candidate"; then
REPO_ROOT="$(cd "$candidate" && pwd)"
break
fi
done
if [ -z "${REPO_ROOT}" ]; then
cat <<'EOF'
ERROR: MV3DT reference app directory was not found.
Set MV3DT_REPO_ROOT to an existing deepstream-tracker-3d-multi-view app directory, or ask the user to approve cloning the public DeepStream repo and then run:
DEEPSTREAM_REPO_ROOT="${DEEPSTREAM_REPO_ROOT:-$HOME/DeepStream}"
git clone https://github.com/NVIDIA/DeepStream.git "$DEEPSTREAM_REPO_ROOT"
export MV3DT_REPO_ROOT="$DEEPSTREAM_REPO_ROOT/src/apps/reference_apps/deepstream-tracker-3d-multi-view"
Do not clone silently.
EOF
exit 1
fi
cd "${REPO_ROOT}"
export REPO_ROOT MV3DT_REPO_ROOT="${REPO_ROOT}"若无法解析应用目录,请询问用户现有目录路径或是否同意克隆仓库。请勿静默克隆。
https://github.com/NVIDIA/DeepStreamStep 1: Select The Primary Workflow
步骤1:选择主工作流
Load exactly one primary reference for the user's current request:
| User intent | Reference |
|---|---|
| Install, prepare, or verify prerequisites | |
| Run bundled 4-camera or 12-camera sample | |
| Run custom synchronized MP4s | |
| Missing custom calibration | |
| View OSD, BEV, screenshots, recordings, or Kafka metadata | |
| Stop a run, clean generated artifacts, or stop prerequisite services | |
If setup, datasets, models, Kafka, Mosquitto, Docker GPU runtime, or the Python venv are missing, load before continuing to the user's original workflow.
references/setup.mdWhen or needs to regenerate DeepStream configs, load as the canonical shared config-generation reference rather than duplicating the shell logic.
sample-run.mdcustom-dataset.mdreferences/generate-configs.md根据用户当前请求,加载恰好一个主参考文档:
| 用户意图 | 参考文档 |
|---|---|
| 安装、准备或验证前置条件 | |
| 运行内置的4相机或12相机示例 | |
| 运行自定义同步MP4文件 | |
| 缺少自定义校准信息 | |
| 查看OSD、BEV、截图、录制文件或Kafka元数据 | |
| 停止运行、清理生成的工件或停止前置服务 | |
若缺少环境设置、数据集、模型、Kafka、Mosquitto、Docker GPU运行时或Python虚拟环境,需先加载,再继续执行用户原本的工作流。
references/setup.md当或需要重新生成DeepStream配置时,加载作为标准的共享配置生成参考文档,而非重复编写Shell逻辑。
sample-run.mdcustom-dataset.mdreferences/generate-configs.mdStep 2: Follow The Run Stages
步骤2:遵循运行阶段
For every run, use this stage order:
| Stage | Action |
|---|---|
| Validate | Check app directory, prerequisites, dataset shape, display/headless mode, Docker GPU support, and output-directory writability. |
| Prepare | State the selected sample/custom dataset, detector, run mode, output directory, expected output surfaces, and the Docker security decision for |
| Execute | Generate configs, start offline BEV capture first in headless mode or when saved BEV MP4 is explicitly requested, then run DeepStream or delegate missing calibration to AutoMagicCalib. |
| Verify | Confirm functional readiness with |
| Report | Summarize selected options, generated files, artifact paths, file sizes/counts, and any skipped or failed output surface; after a default 4-camera sample run, mention that the 12-camera sample is also available as a follow-up. |
每次运行需按照以下阶段顺序执行:
| 阶段 | 操作 |
|---|---|
| 验证 | 检查应用目录、前置条件、数据集格式、显示/无头模式、Docker GPU支持以及输出目录的可写性。 |
| 准备 | 在启动前,说明所选的示例/自定义数据集、检测器、运行模式、输出目录、预期输出界面,以及Docker使用 |
| 执行 | 生成配置文件;若为无头模式或明确要求保存BEV MP4文件,先启动离线BEV捕获,再运行DeepStream或将缺失的校准任务委托给AutoMagicCalib。 |
| 验证 | 通过 |
| 报告 | 总结所选选项、生成的文件、工件路径、文件大小/数量,以及任何跳过或失败的输出界面;默认4相机示例运行完成后,提及12相机示例也可作为后续操作。 |
Step 3: Apply Defaults Explicitly
步骤3:明确应用默认设置
- Runtime path: DeepStream Container.
- Display path: if a working X11/VNC display is available, use the repo quick-start path with OSD and BEV windows. The OSD window does not save MP4 by default; when the user explicitly asks to save output in display mode, regenerate configs with both and
--enable-osdplus--enable-file-output.--enable-msg-broker - Headless path: if no working display is available, use saved outputs by default. Generate configs with and
--enable-file-output; start offline BEV capture before DeepStream so the run produces both the tiled DeepStream MP4 and the Kafka-derived BEV MP4.--enable-msg-broker - Sample dataset: support both shipped 4-camera and 12-camera datasets. If the user does not specify a sample, run the 4-camera sample first, then mention in the final report that the 12-camera sample is also available and can be run next.
- Detector: unless the user asks for
PeopleNetTransformerorRTDETR; carry the selectedPeopleNet2.6.3through config generation and AMCDETECTOR_MODELmodelInfo normalization. For custom calibration handoff, ask the user to choose the AutoMagicCalib detector instead of silently defaulting.camInfo - Custom data source: synchronized MP4 files. Live-stream handling is outside this first-release skill.
- Calibration handoff: use standalone AutoMagicCalib skills instead of duplicating their setup or API workflow.
- 运行路径:DeepStream容器。
- 显示模式:若有可用的X11/VNC显示环境,使用仓库快速启动路径,显示OSD和BEV窗口。默认情况下OSD窗口不保存MP4文件;当用户明确要求在显示模式下保存输出时,需重新生成配置,同时启用、
--enable-osd和--enable-file-output。--enable-msg-broker - 无头模式:若无可用显示环境,默认使用保存输出模式。生成配置时启用和
--enable-file-output;在运行DeepStream前启动离线BEV捕获,确保运行同时生成拼接后的DeepStream MP4文件和基于Kafka的BEV MP4文件。--enable-msg-broker - 示例数据集:支持内置的4相机和12相机数据集。若用户未指定示例,先运行4相机示例,然后在最终报告中提及12相机示例也可用,可后续运行。
- 检测器:默认使用,除非用户要求使用
PeopleNetTransformer或RTDETR;将所选的PeopleNet2.6.3贯穿配置生成和AMCDETECTOR_MODEL模型信息标准化流程。对于自定义校准委托,需询问用户选择AutoMagicCalib检测器,而非静默使用默认值。camInfo - 自定义数据源:同步MP4文件。直播流处理不在本初始版本技能范围内。
- 校准委托:使用独立的AutoMagicCalib技能,而非重复其设置或API工作流。
Step 4: Preserve Idempotency And User Data
步骤4:保持幂等性并保护用户数据
- Readiness checks are safe to rerun.
- Setup may install packages, pull containers, download models, and start services; ask first.
- Config generation and DeepStream runs update generated files under ; record
EXPERIMENT_DIRand do not report old artifacts as current-run success.RUN_STARTED_AT - Normal teardown stops only current run processes and leaves Kafka, Mosquitto, models, datasets, and generated artifacts in place unless the user explicitly asks to stop services or delete files.
- Custom datasets are user data. Copy missing calibration-format variants by default, and ask before renaming, overwriting, clearing, or deleting dataset files.
- 就绪检查可安全重复执行。
- 环境设置可能涉及安装包、拉取容器、下载模型和启动服务;执行前需先询问用户。
- 配置生成和DeepStream运行会更新下的生成文件;记录
EXPERIMENT_DIR,勿将旧工件报告为当前运行的成功结果。RUN_STARTED_AT - 正常关闭仅停止当前运行进程,保留Kafka、Mosquitto、模型、数据集和生成的工件,除非用户明确要求停止服务或删除文件。
- 自定义数据集属于用户数据。默认复制缺失的校准格式变体,重命名、覆盖、清除或删除数据集文件前需先询问用户。
Success Criteria
成功标准
- Prerequisite checks pass or the missing prerequisite is reported with a narrow next step.
- DeepStream run eventually prints .
App run successful - Display mode shows the DeepStream OSD grid and live BEV visualizer by default.
- Headless mode produces a fresh and attempts BEV MP4 capture by default.
${EXPERIMENT_DIR}/outVideos/tiled_display_raw.mp4 - Kafka topic receives current-run protobuf metadata when message broker output is enabled.
mv3dt - BEV MP4 is reported as successful only when the separate BEV capture process produced nonzero messages and frames.
- 前置条件检查通过,或报告缺失的前置条件并给出明确的下一步操作。
- DeepStream运行最终输出。
App run successful - 显示模式默认显示DeepStream OSD网格和实时BEV可视化工具。
- 无头模式默认生成新的文件,并尝试捕获BEV MP4文件。
${EXPERIMENT_DIR}/outVideos/tiled_display_raw.mp4 - 启用消息代理输出时,Kafka主题接收当前运行的protobuf元数据。
mv3dt - 仅当独立的BEV捕获进程生成了非零数量的消息和帧时,才报告BEV MP4生成成功。
Key Output
关键输出
- Generated configs: ,
${EXPERIMENT_DIR}/config_deepstream.txt,${EXPERIMENT_DIR}/config_tracker.yml${EXPERIMENT_DIR}/config_msgconv.txt - Saved DeepStream tiled MP4 when file output is enabled:
${EXPERIMENT_DIR}/outVideos/tiled_display_raw.mp4 - Saved BEV MP4 when offline capture is used:
${EXPERIMENT_DIR}/bev_outputs/trajectory_video_<timestamp>.mp4 - Kafka topic:
mv3dt - Sample output roots: and
${REPO_ROOT}/experiments/deepstream/4cam${REPO_ROOT}/experiments/deepstream/12cam
- 生成的配置文件:、
${EXPERIMENT_DIR}/config_deepstream.txt、${EXPERIMENT_DIR}/config_tracker.yml${EXPERIMENT_DIR}/config_msgconv.txt - 启用文件输出时保存的DeepStream拼接MP4文件:
${EXPERIMENT_DIR}/outVideos/tiled_display_raw.mp4 - 使用离线捕获时保存的BEV MP4文件:
${EXPERIMENT_DIR}/bev_outputs/trajectory_video_<timestamp>.mp4 - Kafka主题:
mv3dt - 示例输出根目录:和
${REPO_ROOT}/experiments/deepstream/4cam${REPO_ROOT}/experiments/deepstream/12cam
Troubleshooting
故障排除
| Issue | First action |
|---|---|
| Setup prerequisites missing | Load |
| Docker cannot access GPU | Fix NVIDIA Container Toolkit or Docker runtime before launching samples. |
| Display window missing | Check |
| DeepStream MP4 missing | Confirm configs were generated with |
| BEV MP4 missing or zero messages | Start offline BEV capture with |
| Kafka client shows no messages | Regenerate configs with |
| Custom dataset lacks calibration | Load |
| Generated files are root-owned | Report the ownership issue and ask before applying a narrow generated-directory permission fix. |
| 问题 | 首要操作 |
|---|---|
| 缺少前置条件 | 加载 |
| Docker无法访问GPU | 在启动示例前修复NVIDIA Container Toolkit或Docker运行时。 |
| 显示窗口缺失 | 检查 |
| DeepStream MP4文件缺失 | 确认生成配置时启用了 |
| BEV MP4文件缺失或消息数量为零 | 在运行DeepStream前使用 |
| Kafka客户端无消息显示 | 重新生成配置并启用 |
| 自定义数据集缺少校准信息 | 加载 |
| 生成文件为root权限所有 | 报告权限问题,在应用针对性的生成目录权限修复前需先询问用户。 |
Safety Notes
安全注意事项
- Ask before commands that use , install packages, pull containers, download models, start or stop services, change host display access, overwrite dataset files, or clear generated state.
sudo - Treat deleting Kafka, Mosquitto, models, datasets, or experiment outputs as destructive cleanup. Show the exact targets and get explicit confirmation before removing anything.
- Before running , explicitly state that the container gets broad host, device, network, and mounted-repo access, then get user approval.
docker run --privileged --net=host - Do not silently clone repositories, change host permissions outside the repo, rename or delete user datasets, or report old artifacts as current-run success.
- Treat custom videos, calibration, saved visualizations, and tracking metadata as potentially sensitive local data. Keep outputs local unless the user explicitly asks to move or share them.
- If permission fixes are needed for generated outputs, propose the narrowest generated-directory-only fix and ask first; never recommend broad world-writable recursive permission changes.
- 执行使用、安装包、拉取容器、下载模型、启动或停止服务、更改主机显示权限、覆盖数据集文件或清除生成状态的命令前,需先询问用户。
sudo - 将删除Kafka、Mosquitto、模型、数据集或实验输出视为破坏性清理操作。执行前需显示明确的目标并获得用户的明确确认。
- 运行前,需明确说明容器将获得广泛的主机、设备、网络和挂载仓库访问权限,然后获得用户批准。
docker run --privileged --net=host - 请勿静默克隆仓库、更改仓库外的主机权限、重命名或删除用户数据集,或将旧工件报告为当前运行的成功结果。
- 将自定义视频、校准信息、保存的可视化结果和追踪元数据视为潜在敏感的本地数据。除非用户明确要求移动或共享,否则保持输出本地化。
- 若需修复生成输出的权限问题,建议仅针对生成目录进行最小范围的修复,并先询问用户;绝不推荐广泛的全局可写递归权限更改。
Related Skills
相关技能
- - Launch the standalone AutoMagicCalib stack when calibration is needed.
amc-setup-calibration-stack - - Generate calibration from synchronized local MP4s before returning to MV3DT.
amc-run-video-calibration
- - 当需要校准时,启动独立的AutoMagicCalib栈。
amc-setup-calibration-stack - - 从同步本地MP4文件生成校准信息,之后返回MV3DT操作。
amc-run-video-calibration