Back to Details

vss-generate-video-calibration

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Purpose

用途

Run AutoMagicCalib end-to-end on local files, RTSP streams, or the bundled sample dataset and (when needed) deploy the AMC microservice.
在本地文件、RTSP流或捆绑的样本数据集上端到端运行AutoMagicCalib，并在需要时部署AMC微服务。

Instructions

说明

Follow the routing tables and step-by-step workflows below. Each section that ends in workflow, quick start, or flow is intended to be executed top-to-bottom. Detailed reference material lives in 
references/
and helper scripts live in 
scripts/
— call them via 
run_script
when the skill points to a script by name.
遵循以下路由表和分步工作流程。每个以workflow、quick start或flow结尾的部分都需要从上到下执行。详细参考资料位于
references/
目录，辅助脚本位于
scripts/
目录——当技能指向某个脚本名称时，通过
run_script
调用它们。

Examples

示例

Worked end-to-end examples are kept under 
evals/
(each 
*.json
manifest contains a runnable scenario) and inline in the per-workflow 
curl
blocks below. Run a Tier-3 evaluation with 
nv-base validate <this-skill-dir> --agent-eval
to replay them.
端到端的实操示例保存在
evals/
目录下（每个
*.json
清单包含一个可运行的场景），并内嵌在下方每个工作流程的
curl
块中。运行
nv-base validate <此技能目录> --agent-eval
即可重放这些示例，完成Tier-3评估。

Limitations

限制

  • Requires the matching VSS profile / microservice to be deployed and reachable from the caller.
  • NGC-hosted models and NIMs may be subject to rate-limits, GPU memory requirements, and license restrictions.
  • Concurrency, GPU memory, and storage limits depend on the host hardware and the profile's compose file.
  • 需要已部署匹配的VSS配置文件/微服务，且调用方可访问该服务。
  • NGC托管的模型和NIM可能受速率限制、GPU内存要求和许可证限制。
  • 并发能力、GPU内存和存储限制取决于主机硬件和配置文件的compose文件。

Troubleshooting

故障排查

  • Error: REST call returns connection refused. Cause: target microservice not running. Solution: probe 
    /docs
    or 
    /health
    ; redeploy via 
    vss-deploy-profile
    or the matching 
    vss-deploy-*
    skill.
  • Error: HTTP 401/403 from NGC pulls. Cause: missing/expired 
    NGC_CLI_API_KEY
    . Solution: 
    docker login nvcr.io
    and re-export the key before retrying.
  • Error: container OOM or model fails to load. Cause: insufficient GPU memory for the selected profile. Solution: switch to a smaller variant or free GPUs via 
    docker compose down
    .
  • 错误：REST调用返回连接拒绝。原因：目标微服务未运行。解决方案：探测
    /docs
    或
    /health
    接口；通过
    vss-deploy-profile
    或匹配的
    vss-deploy-*
    技能重新部署。
  • 错误：从NGC拉取时出现HTTP 401/403。原因：
    NGC_CLI_API_KEY
    缺失或过期。解决方案：执行
    docker login nvcr.io
    并重新导出密钥后重试。
  • 错误：容器内存不足（OOM）或模型加载失败。原因：所选配置文件所需的GPU内存不足。解决方案：切换到更小的变体，或通过
    docker compose down
    释放GPU资源。

VSS Generate Video Calibration

VSS 生成视频校准

Run AutoMagicCalib over one of three input sources and drive the calibration through the microservice REST API. The input-resolution work differs per source; everything from 
verify_project
onward is identical and lives in this file. Pick the right input-mode reference and pair it with the Shared Calibration Tail below.
在三种输入源之一上运行AutoMagicCalib，并通过微服务REST API驱动校准流程。不同输入源的输入分辨率处理工作有所不同；从
verify_project
开始的所有步骤都是相同的，且都记录在本文件中。选择合适的输入模式参考文档，并与下方的共享校准收尾流程配合使用。

Input Routing

输入路由

Match the user's request to a mode, then load that mode's reference for input collection, mode-specific API calls, and the full Python script.
User says / hasModeReference
"launch AMC" / "deploy auto-calibration" / "set up auto-magic-calib" / "start AMC microservice"
deploy
references/deploy-auto-calibration-service.md
"calibrate my videos" / "calibrate from video files" / local 
cam_*.mp4
files
videos
references/videos.md
"calibrate RTSP streams" / "calibrate from live cameras" / live RTSP URLs
rtsp
references/rtsp.md
"test sample dataset" / "verify AMC install" / "launch and test"
sample-dataset
references/sample-dataset.md
Disambiguation rule: if the user is asking to launch / deploy / set up AMC (no calibration verb) → 
deploy
. If they provide RTSP URLs → 
rtsp
. If they mention local files / a videos directory → 
videos
. If they ask to verify install or test the bundled sample → 
sample-dataset
. Combined intents (e.g. "launch AMC and calibrate my videos") → walk 
deploy
first, then the calibration mode. When ambiguous, ask via 
AskUserQuestion
.
将用户的请求匹配到对应模式，然后加载该模式的参考文档以进行输入收集、模式特定的API调用，以及获取完整的Python脚本。
用户表述/拥有的资源模式参考文档
"启动AMC" / "部署自动校准" / "设置auto-magic-calib" / "启动AMC微服务"
deploy
references/deploy-auto-calibration-service.md
"校准我的视频" / "从视频文件校准" / 本地
cam_*.mp4
文件
videos
references/videos.md
"校准RTSP流" / "从实时摄像头校准" / 实时RTSP地址
rtsp
references/rtsp.md
"测试样本数据集" / "验证AMC安装" / "启动并测试"
sample-dataset
references/sample-dataset.md
歧义消除规则：如果用户要求启动/部署/设置AMC（无校准动词）→ 使用
deploy
模式。如果用户提供RTSP地址→ 使用
rtsp
模式。如果用户提到本地文件/视频目录→ 使用
videos
模式。如果用户要求验证安装或测试捆绑样本→ 使用
sample-dataset
模式。混合意图（例如“启动AMC并校准我的视频”）→ 先执行
deploy
模式，再执行校准模式。存在歧义时，通过
AskUserQuestion
询问用户。

Prerequisites (shared across calibration modes)

先决条件（所有校准模式共享）

  • AMC microservice + UI running. If not, walk 
    references/deploy-auto-calibration-service.md
     first.
  • Microservice reachable at 
    http://<HOST_IP>:${VSS_AUTO_CALIBRATION_PORT:-8010}/v1/ready
    → 
    {"code":0,...}
    .
  • Python 3 with 
    requests
    installed (each input-mode reference includes a self-healing venv fallback for direct runs).
Mode-specific prerequisites (VIOS for 
rtsp
, sample zip for 
sample-dataset
) live in the respective references.
  • AMC微服务和UI正在运行。如果未运行，请先执行
    references/deploy-auto-calibration-service.md
    中的步骤。
  • 微服务可通过
    http://<HOST_IP>:${VSS_AUTO_CALIBRATION_PORT:-8010}/v1/ready
    访问，且返回
    {"code":0,...}
    。
  • 安装了
    requests
    库的Python 3环境（每个输入模式的参考文档都包含一个用于直接运行的自修复venv备选方案）。
模式特定的先决条件（
rtsp
模式需要VIOS，
sample-dataset
模式需要样本压缩包）位于各自的参考文档中。

Shared Calibration Tail

共享校准收尾流程

The verify → calibrate → poll → results sequence is identical regardless of input mode. After the mode-specific reference has uploaded videos / ingested RTSP clips / uploaded the bundled sample, run this tail.
无论输入模式如何，验证→校准→轮询→获取结果的流程都是相同的。在完成模式特定参考文档中的视频上传/RTSP片段采集/捆绑样本上传后，执行以下收尾流程。

Step A — Verify Project

步骤A — 验证项目

POST /v1/verify_project/<project_id>
Response: 
{"project_state": "READY"}
— must be 
READY
before calibrating. If not READY, re-check that videos + alignment + layout are present (either via API or via UI manual alignment).
POST /v1/verify_project/<project_id>
响应：
{"project_state": "READY"}
— 必须处于
READY
状态才能开始校准。如果未处于READY状态，请重新检查视频、对齐数据和布局文件是否存在（可通过API或UI手动对齐）。

Step B — Start Calibration

步骤B — 启动校准

POST /v1/calibrate/<project_id>
Content-Type: application/json

{"detector_type": "resnet"}   # or "transformer"
detector_type
is a separate 
/calibrate
parameter — not consumed by 
/v1/config/<id>
. If the user provided a calibration settings file, parse it for 
"detector"
/ 
"detector_type"
and use that value. If no settings file, ask the user via 
AskUserQuestion
:
  • resnet
    — default, fast.
  • transformer
    — slower, better under heavy occlusion.
UI Step 3 (Parameters) does NOT cover detector choice; never assume the user picked one in the UI.
POST /v1/calibrate/<project_id>
Content-Type: application/json

{"detector_type": "resnet"}   # 或 "transformer"
detector_type
是
/calibrate
接口的独立参数——不会被
/v1/config/<id>
接口读取。如果用户提供了校准设置文件，请解析文件中的
"detector"
/ 
"detector_type"
字段并使用对应值。如果没有设置文件，请通过
AskUserQuestion
询问用户：
  • resnet
    — 默认选项，速度快。
  • transformer
    — 速度较慢，但在严重遮挡情况下表现更好。
UI的步骤3（参数设置）不包含检测器选择；请勿假设用户已在UI中选择了检测器。

Step C — Poll for Completion

步骤C — 轮询完成状态

GET /v1/get_project_info/<project_id>
Poll every 10 s. 
project_info.project_state
:
StateMeaning
RUNNING
Calibration in progress
COMPLETED
Finished
ERROR
Failed — pull log via 
GET /v1/amc/calibrate/<id>/log
Typical time: 10–60 min (your-own videos), 10–30 min (bundled sample).
GET /v1/get_project_info/<project_id>
每10秒轮询一次。
project_info.project_state
的含义：
状态含义
RUNNING
校准进行中
COMPLETED
已完成
ERROR
失败 — 通过
GET /v1/amc/calibrate/<id>/log
获取日志
典型耗时：10–60分钟（用户自有视频），10–30分钟（捆绑样本）。

Step D — Results

步骤D — 获取结果

GET /v1/get_project_info/<project_id>                    # project state
GET /v1/result/<project_id>/evaluation_statistics        # only if GT uploaded
GET /v1/amc/calibrate/<project_id>/log                   # calibration log
Evaluation response includes 
Average L2 distance(m)
and 
Average reprojection error 0(px)
.
GET /v1/get_project_info/<project_id>                    # 项目状态
GET /v1/result/<project_id>/evaluation_statistics        # 仅当上传了GT数据时可用
GET /v1/amc/calibrate/<project_id>/log                   # 校准日志
评估响应包含
Average L2 distance(m)
和
Average reprojection error 0(px)
。

Step E — (Optional) VGGT Refinement

步骤E — （可选）VGGT优化

Only if 
vggt_state == "READY"
in project info (VGGT model must be staged — see 
references/deploy-auto-calibration-service.md
 Step 2):
POST /v1/vggt/calibrate/<project_id>
GET  /v1/get_project_info/<project_id>                    # poll vggt_state
GET  /v1/vggt_results/<project_id>/evaluation_statistics  # VGGT metrics
仅当项目信息中的
vggt_state == "READY"
时可用（必须已部署VGGT模型——请参阅
references/deploy-auto-calibration-service.md
的步骤2）：
POST /v1/vggt/calibrate/<project_id>
GET  /v1/get_project_info/<project_id>                    # 轮询vggt_state
GET  /v1/vggt_results/<project_id>/evaluation_statistics  # VGGT指标

Settings File + Detector Pattern

设置文件与检测器模式

Optional across all three modes. When the user provides a JSON settings file (typically exported from UI Step 3 Download), POST it verbatim:
POST /v1/config/<project_id>
Content-Type: application/json

<file contents, posted as-is>
The file replaces what the user would otherwise tune in UI Step 3 (rectification, bundle-adjustment, evaluation knobs, detector, …). After a successful POST, also parse the file for 
"detector"
/ 
"detector_type"
— if it's 
"resnet"
or 
"transformer"
, use that value for the 
/calibrate
call in Step B (detector is a separate API parameter, not consumed by 
/config
).
Non-2xx is surfaced — do not silently fall back. Skip this call entirely if the user chose the UI-fallback path.
所有三种模式均支持此可选操作。当用户提供JSON设置文件（通常从UI步骤3的下载功能导出）时，直接按原样POST：
POST /v1/config/<project_id>
Content-Type: application/json

<文件内容，直接提交>
该文件会替代用户原本在UI步骤3中调整的设置（校正、光束平差、评估参数、检测器等）。POST成功后，还需解析文件中的
"detector"
/ 
"detector_type"
字段——如果值为
"resnet"
或
"transformer"
，请将其用于步骤B中的
/calibrate
调用（检测器是独立的API参数，不会被
/config
接口读取）。
如果返回非2xx状态码，请明确提示用户——不要静默回退。如果用户选择UI回退路径，请完全跳过此调用。

UI Fallback Pattern

UI回退模式

When alignment / layout files aren't on disk, direct the user to the appropriate AMC UI step:
  • Settings missing → "Open UI project 
    <project_id>
    , go to Step 3: Parameters, tune via the settings dialog (or accept defaults), click Save." Also: before the 
    /calibrate
    call, ask the user via 
    AskUserQuestion
    whether to use the 
    resnet
    or 
    transformer
    detector — Step 3 doesn't cover detector choice.
  • Layout missing → "Open UI project 
    <project_id>
    , go to Step 2: Video Configuration, upload 
    layout.png
    only (do NOT re-upload videos — they're already attached via API/RTSP), click Save."
  • Alignment missing → "Open UI project 
    <project_id>
    , go to Step 4: Alignment, either upload 
    alignment_data.json
    or mark correspondence points on the layout, click Save."
Wait for user confirmation. For alignment/layout, verify on disk before continuing:
bash
undefined
当磁盘上没有对齐/布局文件时，请引导用户进入对应的AMC UI步骤：
  • 设置缺失 → "打开UI项目
    <project_id>
    ，进入步骤3：参数设置，通过设置对话框调整参数（或接受默认值），点击保存。" 补充说明：在调用
    /calibrate
    之前，请通过
    AskUserQuestion
    询问用户是否使用
    resnet
    或
    transformer
    检测器——步骤3不包含检测器选择。
  • 布局缺失 → "打开UI项目
    <project_id>
    ，进入步骤2：视频配置，仅上传
    layout.png
    （请勿重新上传视频——已通过API/RTSP关联），点击保存。"
  • 对齐数据缺失 → "打开UI项目
    <project_id>
    ，进入步骤4：对齐，上传
    alignment_data.json
    或在布局上标记对应点，点击保存。"
等待用户确认。对于对齐/布局文件，继续操作前请在磁盘上验证：
bash
undefined

Project state lives under $VSS_APPS_DIR/services/auto-calibration/projects

项目状态存储在 $VSS_APPS_DIR/services/auto-calibration/projects 目录下

(the path bind-mounted into the MS container in

该路径在 deploy/docker/services/auto-calibration/ms/compose.yml 中被绑定挂载到微服务容器内。

deploy/docker/services/auto-calibration/ms/compose.yml).

—
HOST_PROJECTS="${VSS_APPS_DIR}/services/auto-calibration/projects"
ls "$HOST_PROJECTS/project_<project_id>/manual_adjustment/"
HOST_PROJECTS="${VSS_APPS_DIR}/services/auto-calibration/projects"
ls "$HOST_PROJECTS/project_<project_id>/manual_adjustment/"

Expected: alignment_data.json, layout.png

预期文件：alignment_data.json, layout.png

undefined
undefined

Success Criteria

成功标准

  • project_state == "COMPLETED"
    after polling.
  • If manual alignment was used: 
    ${VSS_APPS_DIR}/services/auto-calibration/projects/project_<id>/manual_adjustment/
    contains 
    alignment_data.json
    + 
    layout.png
    .
  • If GT was uploaded: evaluation returns typical thresholds (
    Average L2 distance(m)
    < 1.5, 
    Average reprojection error 0(px)
    < 5 for your data; < 10 for the bundled sample).
  • No 
    ERROR
    state.
  • 轮询后
    project_state == "COMPLETED"
    。
  • 如果使用了手动对齐：
    ${VSS_APPS_DIR}/services/auto-calibration/projects/project_<id>/manual_adjustment/
    目录下包含
    alignment_data.json
    和
    layout.png
    。
  • 如果上传了GT数据：评估结果符合典型阈值（自有数据的
    Average L2 distance(m)
    < 1.5，
    Average reprojection error 0(px)
    < 5；捆绑样本的对应值 < 10）。
  • 无
    ERROR
    状态。

Key Output Files

关键输出文件

Under 
${VSS_APPS_DIR}/services/auto-calibration/projects/project_<project_id>/
:
project_<project_id>/
├── manual_adjustment/
│   ├── alignment_data.json
│   └── layout.png
├── output/
│   ├── single_view_results/cam_XX/
│   │   ├── camInfo_hyper_XX.yaml
│   │   └── trajDump_Stream_0_3d.txt
│   └── multi_view_results/BA_output/results_ba/
│       ├── initial/camInfo_XX.yaml
│       └── refined/camInfo_XX.yaml          # ← final calibration
└── calibration.log
位于
${VSS_APPS_DIR}/services/auto-calibration/projects/project_<project_id>/
目录下：
project_<project_id>/
├── manual_adjustment/
│   ├── alignment_data.json
│   └── layout.png
├── output/
│   ├── single_view_results/cam_XX/
│   │   ├── camInfo_hyper_XX.yaml
│   │   └── trajDump_Stream_0_3d.txt
│   └── multi_view_results/BA_output/results_ba/
│       ├── initial/camInfo_XX.yaml
│       └── refined/camInfo_XX.yaml          # ← 最终校准结果
└── calibration.log

Cross-cutting Troubleshooting

通用故障排查

Mode-specific issues live in each reference's own troubleshooting table.
IssueFix
verify_project
state not 
READY
Confirm videos uploaded/ingested and alignment + layout are present (either via API or via UI manual alignment). Mode-specific upload steps in the reference.
Manual alignment files missing after UI stepUser didn't click Save; also verify 
${VSS_APPS_DIR}/services/auto-calibration/projects/project_<id>/manual_adjustment/
exists.
Calibration stuck 
RUNNING
> 90 min
GET /v1/amc/calibrate/<id>/log
— usually insufficient tracklets (scene too static). See "Custom Dataset" guidelines in root 
README.md
.
Immediate 
ERROR
state		Check video naming: must be 
cam_00.mp4
, 
cam_01.mp4
, … contiguous (videos mode) / camera_name labels (RTSP mode).
Low L2 but high reprojectionProvide explicit 
focal_length
override during input upload (see videos / rtsp references).
VGGT 
INIT
, never 
READY
VGGT model not loaded — see 
references/deploy-auto-calibration-service.md
 Step 2.
Upload timeoutLarge videos — bump 
timeout=300
to e.g. 
600
in the per-mode Python script.
Port scan finds no backendBackend not running — walk 
references/deploy-auto-calibration-service.md
 first.
模式特定问题请查看各参考文档中的故障排查表。
问题解决方法
verify_project
状态不为
READY
确认视频已上传/采集，且对齐数据和布局文件存在（可通过API或UI手动对齐）。参考文档中包含模式特定的上传步骤。
执行UI步骤后手动对齐文件缺失用户未点击保存；同时验证
${VSS_APPS_DIR}/services/auto-calibration/projects/project_<id>/manual_adjustment/
目录是否存在。
校准停留在
RUNNING
状态超过90分钟		调用
GET /v1/amc/calibrate/<id>/log
查看日志——通常是因为跟踪轨迹不足（场景过于静态）。请查看根目录
README.md
中的“自定义数据集”指南。
立即进入
ERROR
状态		检查视频命名：必须为
cam_00.mp4
、
cam_01.mp4
……连续命名（videos模式）/符合camera_name标签（RTSP模式）。
L2距离低但重投影误差高在输入上传期间提供明确的
focal_length
覆盖值（请参阅videos/rtsp参考文档）。
VGGT处于
INIT
状态，从未变为
READY
VGGT模型未加载——请参阅
references/deploy-auto-calibration-service.md
的步骤2。
上传超时视频文件过大——在对应模式的Python脚本中将
timeout=300
调整为例如
600
。
端口扫描未找到后端服务后端服务未运行——先执行
references/deploy-auto-calibration-service.md
中的步骤。

For Downstream Skills — MV3DT Export

面向下游技能——MV3DT导出

Downstream consumers (e.g. a Multi-View 3D Tracking skill owned by another team) fetch the MV3DT-format calibration output directly from the microservice. This skill returns the 
project_id
; the downstream skill calls:
GET /v1/result/{project_id}/mv3dt_result?result_type=amc
下游消费者（例如其他团队拥有的多视图3D跟踪技能）直接从微服务获取MV3DT格式的校准输出。本技能返回
project_id
；下游技能调用：
GET /v1/result/{project_id}/mv3dt_result?result_type=amc

Response: application/zip — mv3dt_output.zip containing transforms.yml

响应：application/zip — 包含transforms.yml的mv3dt_output.zip


For VGGT-refined output (only available if VGGT ran to `COMPLETED`, see Step E):
GET /v1/result/{project_id}/mv3dt_result?result_type=vggt

对于VGGT优化后的输出（仅当VGGT运行至`COMPLETED`状态时可用，请参阅步骤E）：
GET /v1/result/{project_id}/mv3dt_result?result_type=vggt

Response: application/zip — vggt_mv3dt_output.zip

响应：application/zip — vggt_mv3dt_output.zip


Downstream skill flow:
1. Call this skill with the user's inputs; capture the printed `project_id`.
2. Wait for the skill to return (it polls until `COMPLETED` internally).
3. `GET /v1/result/{project_id}/mv3dt_result?result_type=amc` — save the ZIP locally.
4. If VGGT also ran, optionally fetch `?result_type=vggt` for the refined MV3DT.

下游技能流程：
1. 使用用户输入调用本技能；捕获输出的`project_id`。
2. 等待本技能返回（内部会轮询直到状态变为`COMPLETED`）。
3. 调用`GET /v1/result/{project_id}/mv3dt_result?result_type=amc` — 将ZIP文件保存到本地。
4. 如果VGGT也已运行，可选择获取`?result_type=vggt`以获取优化后的MV3DT结果。

Related Skills

相关技能

  • vss-manage-video-io-storage
     — VIOS API skill; only the 
    rtsp
    calibration mode depends on VIOS being reachable.
Root 
README.md
"Custom Dataset" and "Calibration Workflow (UI)" sections document input-video guidelines and the UI-driven alternative to this API flow.
bump:1
  • vss-manage-video-io-storage
     — VIOS API技能；仅
    rtsp
    校准模式依赖可访问的VIOS。
根目录
README.md
中的“自定义数据集”和“校准工作流程（UI）”部分记录了输入视频指南，以及本API流程的UI驱动替代方案。
bump:1