Back to Details

alerts

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

VSS Alert Management

VSS 告警管理

The alerts profile is deployed in one of two modes at a time. The mode is chosen at 
deploy -p alerts -m {verification,real-time}
and is static until you tear down and redeploy. Which mode is running determines which flow to use — this skill does not route per-request.
告警配置文件每次仅以两种模式之一部署。模式通过 
deploy -p alerts -m {verification,real-time}
选择，且在您拆除并重新部署前保持静态。当前运行的模式决定了要使用的流程——本技能不会按请求路由。

When to Use

使用场景

  • Start or stop a real-time alert on a sensor ("Start real-time alert for boxes dropped on sensor warehouse_sample")
  • List or query detected incidents / alerts
  • Add a new camera to the alerts pipeline
  • Customize the VLM-verifier prompts (CV mode)
  • Check verdicts (confirmed / rejected / unverified)
  • 在传感器上启动或停止实时告警（"在传感器warehouse_sample上启动针对箱子掉落的实时告警"）
  • 列出或查询已检测到的事件/告警
  • 向告警流水线添加新摄像头
  • 自定义VLM验证器提示（CV模式）
  • 查看判定结果（已确认/已驳回/未验证）

Deployment prerequisite

部署前提

This skill requires the VSS alerts profile running on the host at 
$HOST_IP
, in either 
verification
or 
real-time
mode. Before any request:
  1. Probe the stack:
    bash
    # Either perception-alerts (CV mode) OR rtvi-vlm (VLM mode) must be present.
curl -sf --max-time 5 "http://${HOST_IP}:8000/docs" >/dev/null \
  && docker ps --format '{{.Names}}' \
     | grep -qE '^(perception-alerts|rtvi-vlm)$'
  2. If the probe fails, ask the user:
    "The VSS 
    alerts
    profile isn't running on 
    $HOST_IP
    . Which mode should I deploy — 
    verification
    (CV) or 
    real-time
    (VLM)?"
    • Answer → hand off to the 
      /deploy
      skill with 
      -p alerts -m <mode>
      . Return here once it succeeds.
    • If the user declines → stop. Do not run this skill against a missing stack.
    (If your caller has granted explicit pre-authorization to deploy autonomously — e.g. the request says "pre-authorized to deploy prerequisites", or you are running in a non-interactive evaluation harness with that permission — skip the confirmation and invoke 
    /deploy
    directly. Default the mode to 
    verification
    unless the request specifies otherwise.)
  3. If the probe passes, detect the mode per § Step 1 below.
本技能要求VSS alerts配置文件在主机
$HOST_IP
上运行，模式为
verification
（CV）或
real-time
（VLM）。处理任何请求前：
  1. 探测堆栈：
    bash
    # 必须存在perception-alerts（CV模式）或rtvi-vlm（VLM模式）之一。
curl -sf --max-time 5 "http://${HOST_IP}:8000/docs" >/dev/null \
  && docker ps --format '{{.Names}}' \
     | grep -qE '^(perception-alerts|rtvi-vlm)$'
  2. 如果探测失败，询问用户：
    "VSS 
    alerts
    配置文件未在
    $HOST_IP
    上运行。我应该部署哪种模式——
    verification
    （CV）还是
    real-time
    （VLM）？"
    • 用户答复后 → 移交至
      /deploy
      技能，参数为
      -p alerts -m <mode>
      。部署成功后返回此处。
    • 如果用户拒绝 → 停止操作。不要在缺失堆栈的情况下运行本技能。
    （如果调用者已授予自主部署的明确预授权——例如请求中提到"预授权部署前置条件"，或您在具有该权限的非交互式评估环境中运行——则跳过确认步骤，直接调用
    /deploy
    。除非请求中指定其他模式，否则默认使用
    verification
    模式。）
  3. 如果探测通过，按照下文§步骤1检测模式。

The Two Modes (Deploy-Time Choice)

两种模式（部署时选择）

ModeDeploy flagEnv (
.env
)		What runsHow alerts are created
CV (verification)
-m verification
MODE=2d_cv
RT-CV (Grounding DINO) + Behavior Analytics + 
alert-bridge
VLM verifier		Static: any RTSP flowing through VIOS is auto-processed; Behavior Analytics emits candidates; VLM verifies each clip per 
alert_type_config.json
VLM (real-time)
-m real-time
MODE=2d_vlm
rtvi-vlm
continuous inference		Dynamic: user asks the VSS Agent to start monitoring a sensor with a natural-language detection prompt
Mode is static. Switching requires 
deploy down
+ 
deploy up
with the other 
-m
flag — see the 
deploy
skill. Never assume both flows are available at once.
模式部署标识环境变量（
.env
）		运行组件告警创建方式
CV（验证模式）
-m verification
MODE=2d_cv
RT-CV（Grounding DINO）+ 行为分析 + 
alert-bridge
VLM验证器		静态：所有流经VIOS的RTSP流会被自动处理；行为分析模块生成候选告警；VLM根据
alert_type_config.json
验证每个片段
VLM（实时模式）
-m real-time
MODE=2d_vlm
rtvi-vlm
持续推理		动态：用户通过自然语言检测提示请求VSS Agent开始监控传感器
模式是静态的。切换模式需要执行
deploy down
+ 
deploy up
并使用另一个
-m
标识——请查看
deploy
技能。切勿假设两种流程可同时使用。

Step 1 — Detect the Currently Deployed Mode

步骤1 — 检测当前部署的模式

Before running any alert workflow, check which mode is live. Use container names as the signal:
bash
undefined
在运行任何告警工作流前，检查当前运行的模式。使用容器名称作为判断信号：
bash
undefined

VLM real-time mode

VLM实时模式

docker ps --format '{{.Names}}' | grep -qx rtvi-vlm && echo "mode=VLM"
docker ps --format '{{.Names}}' | grep -qx rtvi-vlm && echo "mode=VLM"

CV verification mode (behavior analytics is CV-only; alert-bridge is the VLM verifier)

CV验证模式（行为分析是CV模式专属；alert-bridge是VLM验证器）

docker ps --format '{{.Names}}' | grep -qx vss-behavior-analytics-alerts && echo "mode=CV"

Exactly one of these should match on a healthy alerts deployment. If neither matches, the alerts profile is not deployed — direct the user to the `deploy` skill.

Alternative signal (if `docker ps` isn't available in the current context): check the profile's `.env`:

```bash
grep -E '^MODE=' deployments/developer-workflow/dev-profile-alerts/.env
docker ps --format '{{.Names}}' | grep -qx vss-behavior-analytics-alerts && echo "mode=CV"

在健康的告警部署中，上述命令应恰好匹配其中一个。如果都不匹配，说明告警配置文件未部署——引导用户使用`deploy`技能。

备选判断信号（如果当前环境中无法使用`docker ps`）：检查配置文件的`.env`文件：

```bash
grep -E '^MODE=' deployments/developer-workflow/dev-profile-alerts/.env

MODE=2d_cv → CV mode

MODE=2d_cv → CV模式

MODE=2d_vlm → VLM real-time mode

MODE=2d_vlm → VLM实时模式


---

---

Step 2 — Route by Deployed Mode

步骤2 — 根据部署模式路由

Deployed modeUser asks about…Action
CVany alert requestRun Workflow A (CV) — onboard RTSP via 
vios
skill, then query alerts. No per-request create call.
CVspecifically a VLM real-time alert ("start alert for boxes dropped…")Redeployment required. Confirm with the user first, then point to the 
deploy
skill to tear down and redeploy with 
-m real-time
. Do not attempt to run the VLM flow on a CV deployment — the agent's 
rtvi_vlm_alert
tool will fail because 
rtvi-vlm
is not running.
VLMany alert requestRun Workflow B (VLM) — call the VSS Agent with a detection prompt.
VLMspecifically a CV / behavior-analytics / PPE-rule alertRedeployment required. Confirm, then point to 
deploy
skill for 
-m verification
. The CV pipeline (RT-CV, Behavior Analytics, 
alert-bridge
) is not running on a VLM deployment.
Always confirm before triggering a redeploy. A mode switch stops all currently-running monitoring and restarts services.
已部署模式用户询问内容操作
CV任何告警请求运行工作流A（CV）——通过
vios
技能接入RTSP流，然后查询告警。无需按请求调用创建接口。
CV明确请求VLM实时告警（"启动针对箱子掉落的告警…"）需要重新部署。先与用户确认，然后引导至
deploy
技能，拆除现有部署并使用
-m real-time
重新部署。不要尝试在CV部署上运行VLM流程——因为
rtvi-vlm
未运行，Agent的
rtvi_vlm_alert
工具会执行失败。
VLM任何告警请求运行工作流B（VLM）——向VSS Agent发送检测提示。
VLM明确请求CV/行为分析/PPE规则告警需要重新部署。先确认，然后引导至
deploy
技能使用
-m verification
模式。CV流水线（RT-CV、行为分析、
alert-bridge
）未在VLM部署中运行。
触发重新部署前务必确认。模式切换会停止所有当前运行的监控并重启服务。

Prereq for Either Mode: Sensor Must Be in VIOS

两种模式的共同前提：传感器必须已在VIOS中注册

Both modes require the camera to be registered in VIOS first.
  • If the user hands you only an RTSP URL (or an IP camera) — defer to the 
    vios
    skill     to add it via 
    POST /sensor/add
    (see 
    vios
    skill Section 6). Record the returned 
    sensorId
    / name.
  • If the user names an existing sensor — confirm it is listed by 
    GET /sensor/list
    via the 
    vios
    skill before proceeding.
  • If the user asks to use a local/sample MP4 for a VLM real-time alert, do not upload the MP4 directly to VIOS storage (
    PUT /storage/file/...
    ). VIOS file uploads create 
    sensor_file
    entries whose stream URL is a local file path; 
    rtvi-vlm
    requires a live 
    rtsp://...
    URL.
  • For a local/sample MP4 in VLM real-time mode, first add the video to NVStreamer (or another RTSP restreamer) and obtain its RTSP live URL, then add that RTSP URL to VIOS via 
    POST /sensor/add
    with the desired sensor name. Only proceed once 
    GET /sensor/<sensorId>/streams
    shows a stream URL starting with 
    rtsp://
    .
On a CV deployment, adding the RTSP is the entire onboarding step — the pipeline picks up the stream automatically once it is in VIOS. On a VLM deployment, adding the RTSP is a prerequisite to Workflow B.
两种模式都要求摄像头先在VIOS中注册。
  • 如果用户仅提供RTSP URL（或IP摄像头）——移交至
    vios
    技能    ，通过
    POST /sensor/add
    添加（查看
    vios
    技能第6节）。记录返回的
    sensorId
    /名称。
  • 如果用户指定现有传感器——在继续操作前，通过
    vios
    技能调用
    GET /sensor/list
    确认该传感器已列出。
  • 如果用户要求为VLM实时告警使用本地/示例MP4文件，请不要直接将MP4上传至VIOS存储（
    PUT /storage/file/...
    ）。VIOS文件上传会创建
    sensor_file
    条目，其流URL为本地文件路径；
    rtvi-vlm
    需要实时
    rtsp://...
    URL。
  • 对于VLM实时模式下的本地/示例MP4文件，先将视频添加至NVStreamer（或其他RTSP转流器）并获取其实时RTSP URL，然后通过
    POST /sensor/add
    将该RTSP URL添加至VIOS并设置所需的传感器名称。只有当
    GET /sensor/<sensorId>/streams
    显示以
    rtsp://
    开头的流URL时，才可继续操作。
在CV部署中，添加RTSP流是完整的接入步骤——一旦流在VIOS中注册，流水线会自动处理该流。在VLM部署中，添加RTSP流是工作流B的前置条件。

The Agent 
/generate
Endpoint

Agent 
/generate
端点

All VLM-flow actions and all query actions go through the VSS Agent's natural-language endpoint:
bash
AGENT="http://<AGENT_ENDPOINT>"   # default http://localhost:8000 on the alerts profile

curl -s -X POST "$AGENT/generate" \
  -H "Content-Type: application/json" \
  -d '{"input_message": "<natural-language request>"}' | jq .
Endpoint resolution: use the agent endpoint from the active VSS deployment context. If unavailable, ask the user. Do not discover via filesystem.
Availability check: 
curl -sf --connect-timeout 5 "$AGENT/docs"
.
Do not call the 
rtvi-vlm
microservice endpoints directly — always go through the agent. The agent internally dispatches to 
rtvi_vlm_alert
, 
rtvi_prompt_gen
, and 
video_analytics_mcp.get_incidents
.
所有VLM流程操作和查询操作均通过VSS Agent的自然语言端点执行：
bash
AGENT="http://<AGENT_ENDPOINT>"   # 告警配置文件的默认地址为http://localhost:8000

curl -s -X POST "$AGENT/generate" \
  -H "Content-Type: application/json" \
  -d '{"input_message": "<自然语言请求>"}' | jq .
端点解析：使用活动VSS部署上下文中的Agent端点。如果不可用，询问用户。不要通过文件系统发现。
可用性检查：
curl -sf --connect-timeout 5 "$AGENT/docs"
。
请勿直接调用
rtvi-vlm
微服务端点——始终通过Agent调用。Agent会内部调度至
rtvi_vlm_alert
、
rtvi_prompt_gen
和
video_analytics_mcp.get_incidents
。

Workflow A — CV Mode (deployment is 
-m verification
/ 
MODE=2d_cv
)

工作流A — CV模式（部署参数为
-m verification
/ 
MODE=2d_cv
）

On a CV deployment, alerts are deployment-driven, not request-driven. There is no agent call to "create" an alert.
  1. Onboard the camera — add the RTSP to VIOS via the 
    vios
    skill (
    POST /sensor/add
    ). Once registered and online, the CV pipeline picks it up automatically.
  2. Confirm the sensor is online:
    bash
    curl -s "http://<VST_ENDPOINT>/vst/api/v1/sensor/<sensorId>/status" | jq .
  3. Wait for alerts to land in Elasticsearch. Behavior Analytics emits candidates that match configured rules; 
    alert-bridge
    calls the VLM to confirm/reject each candidate per 
    alert_type_config.json
    . Use Workflow C to query results.
If the user asks you to "start a real-time alert" on a CV deployment, that is a mode mismatch — see the routing table above.
在CV部署中，告警是部署驱动的，而非请求驱动的。没有Agent接口可用于"创建"告警。
  1. 接入摄像头——通过
    vios
    技能将RTSP流添加至VIOS（
    POST /sensor/add
    ）。注册并上线后，CV流水线会自动处理该流。
  2. 确认传感器已上线：
    bash
    curl -s "http://<VST_ENDPOINT>/vst/api/v1/sensor/<sensorId>/status" | jq .
  3. 等待告警进入Elasticsearch。行为分析模块会生成符合配置规则的候选告警；
    alert-bridge
    会调用VLM根据
    alert_type_config.json
    确认/驳回每个候选告警。使用工作流C查询结果。
如果用户要求在CV部署上"启动实时告警"，这属于模式不匹配——请查看上述路由表。

Workflow B — VLM Mode (deployment is 
-m real-time
/ 
MODE=2d_vlm
)

工作流B — VLM模式（部署参数为
-m real-time
/ 
MODE=2d_vlm
）

On a VLM deployment, the user drives alert creation via natural-language requests to the VSS Agent. The agent calls 
rtvi_prompt_gen
to turn the description into a Yes/No detection question, then 
rtvi_vlm_alert
with 
action="start"
to register the stream with 
rtvi-vlm
and begin continuous monitoring.
Before calling the agent, verify the target sensor is RTSP-backed:
bash
curl -s "http://<VST_ENDPOINT>/vst/api/v1/sensor/<sensorId>/streams" | jq .
At least one stream for the sensor must have a URL beginning with 
rtsp://
. If the stream URL is a local file path such as 
/home/vst/.../video.mp4
, the sensor was uploaded as a VIOS file and real-time alert start will fail. For sample videos, add the video to NVStreamer first, register the returned RTSP URL in VIOS, then start the alert against that RTSP-backed VIOS sensor.
Sample-video onboarding for real-time alerts:
  1. Add the MP4 to NVStreamer and get the RTSP live URL for the new stream.
  2. Register that RTSP URL in VIOS:
    bash
    curl -s -X POST "http://<VST_ENDPOINT>/vst/api/v1/sensor/add" \
  -H "Content-Type: application/json" \
  -d '{
    "sensorUrl": "rtsp://<nvstreamer-host>:<port>/<path>",
    "name": "warehouse_sample"
  }' | jq .
  3. Confirm 
    GET /sensor/warehouse_sample/streams
    returns the RTSP URL, then call the VSS Agent as shown below.
Canonical sample request:
bash
curl -s -X POST "$AGENT/generate" \
  -H "Content-Type: application/json" \
  -d '{"input_message": "Start real-time alert for boxes dropped on sensor warehouse_sample"}' | jq .
More examples:
bash
undefined
在VLM部署中，用户通过向VSS Agent发送自然语言请求来创建告警。Agent会调用
rtvi_prompt_gen
将描述转换为是/否检测问题，然后调用
rtvi_vlm_alert
并设置
action="start"
，以在
rtvi-vlm
中注册流并开始持续监控。
调用Agent前，验证目标传感器是否基于RTSP：
bash
curl -s "http://<VST_ENDPOINT>/vst/api/v1/sensor/<sensorId>/streams" | jq .
该传感器至少有一个流的URL必须以
rtsp://
开头。如果流URL是本地文件路径（如
/home/vst/.../video.mp4
），说明该传感器是作为VIOS文件上传的，启动实时告警会失败。对于示例视频，先将视频添加至NVStreamer，然后在VIOS中注册返回的RTSP URL，再针对该基于RTSP的VIOS传感器启动告警。
实时告警的示例视频接入流程：
  1. 将MP4添加至NVStreamer并获取新流的实时RTSP URL。
  2. 在VIOS中注册该RTSP URL：
    bash
    curl -s -X POST "http://<VST_ENDPOINT>/vst/api/v1/sensor/add" \
  -H "Content-Type: application/json" \
  -d '{
    "sensorUrl": "rtsp://<nvstreamer-host>:<port>/<path>",
    "name": "warehouse_sample"
  }' | jq .
  3. 确认
    GET /sensor/warehouse_sample/streams
    返回该RTSP URL，然后按如下方式调用VSS Agent。
标准示例请求：
bash
curl -s -X POST "$AGENT/generate" \
  -H "Content-Type: application/json" \
  -d '{"input_message": "Start real-time alert for boxes dropped on sensor warehouse_sample"}' | jq .
更多示例：
bash
undefined

Vehicle collisions on a street cam

街道摄像头的车辆碰撞检测

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json"
-d '{"input_message": "Start real-time alert for vehicle collisions on sensor Camera_02"}' | jq .
curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json"
-d '{"input_message": "Start real-time alert for vehicle collisions on sensor Camera_02"}' | jq .

Forklift-pedestrian proximity

叉车与行人距离监测

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json"
-d '{"input_message": "Monitor Warehouse_Dock_3 for a forklift passing within 1 meter of a pedestrian"}' | jq .
curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json"
-d '{"input_message": "Monitor Warehouse_Dock_3 for a forklift passing within 1 meter of a pedestrian"}' | jq .

Generic start (no specific target — uses default prompt)

通用启动（无特定目标——使用默认提示）

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json"
-d '{"input_message": "Start real-time alert for sensor warehouse_sample"}' | jq .

**What the agent does under the hood:**
1. `rtvi_prompt_gen` — converts "boxes dropped" → `prompt: "Detect for a box being dropped. Answer in Yes or No"`, `system_prompt: "You are a helpful assistant."`.
2. `rtvi_vlm_alert action="start"` — looks up the sensor in VIOS live streams, calls `POST /v1/streams/add` and `POST /v1/generate_captions_alerts` (`stream=true`) on `rtvi-vlm`. Returns `stream_id`.

**Alert semantics:** every chunk is captioned; a chunk whose VLM response contains **`"yes"` or `"true"`** (case-insensitive) triggers an incident published to the Kafka incident topic (`mdx-vlm-incidents` on the alerts profile). That is why prompts must force a Yes/No answer.

**Stop monitoring:**

```bash
curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "Stop real-time alert for sensor warehouse_sample"}' | jq .
If the user asks you to flag a scenario that matches a CV 
alert_type
(e.g. "ladder PPE violation") on a VLM deployment, that is a mode mismatch — see the routing table above.
curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json"
-d '{"input_message": "Start real-time alert for sensor warehouse_sample"}' | jq .

**Agent的底层操作**：
1. `rtvi_prompt_gen`——将"boxes dropped"转换为`prompt: "Detect for a box being dropped. Answer in Yes or No"`，`system_prompt: "You are a helpful assistant."`。
2. `rtvi_vlm_alert action="start"`——在VIOS实时流中查找传感器，调用`rtvi-vlm`的`POST /v1/streams/add`和`POST /v1/generate_captions_alerts`（`stream=true`）。返回`stream_id`。

**告警规则**：每个片段都会生成字幕；如果VLM响应包含**`"yes"`或`"true"`**（不区分大小写），则触发事件并发布至Kafka事件主题（告警配置文件中的`mdx-vlm-incidents`）。这就是为什么提示必须强制是/否回答的原因。

**停止监控**：

```bash
curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "Stop real-time alert for sensor warehouse_sample"}' | jq .
如果用户要求标记与CV 
alert_type
匹配的场景（如"梯子PPE违规"）而当前是VLM部署，这属于模式不匹配——请查看上述路由表。

Workflow C — Query / List Alerts (works on either mode)

工作流C — 查询/列出告警（两种模式均适用）

Both CV- and VLM-generated alerts land in Elasticsearch and are queryable via the agent's 
video_analytics_mcp.get_incidents
tool.
bash
curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "Show me recent alerts for sensor warehouse_sample"}' | jq .

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "List confirmed alerts from the last hour"}' | jq .

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "Were there any PPE violations today on Camera_02?"}' | jq .

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "Show collision incidents from Camera_02 between 2026-04-23T00:00:00.000Z and 2026-04-23T23:59:59.000Z"}' | jq .
For richer / non-natural-language filtering (sensor-level, time-series, counts): use the 
video-analytics
skill (VA-MCP on port 9901).
CV和VLM生成的告警都会存入Elasticsearch，可通过Agent的
video_analytics_mcp.get_incidents
工具查询。
bash
curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "Show me recent alerts for sensor warehouse_sample"}' | jq .

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "List confirmed alerts from the last hour"}' | jq .

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "Were there any PPE violations today on Camera_02?"}' | jq .

curl -s -X POST "$AGENT/generate" -H "Content-Type: application/json" \
  -d '{"input_message": "Show collision incidents from Camera_02 between 2026-04-23T00:00:00.000Z and 2026-04-23T23:59:59.000Z"}' | jq .
如需更丰富的非自然语言过滤（传感器级别、时间序列、统计）：使用**
video-analytics
技能**（VA-MCP，端口9901）。

Verdict interpretation (CV mode only)

判定结果解读（仅CV模式）

Verified alerts carry an extended 
info
block:
verdict
Meaning
confirmed
VLM determined the alert is real
rejected
VLM determined it is a false positive
unverified
Verification could not complete (error)
Check 
verification_response_code
(200 = success) and 
reasoning
for the VLM's explanation. VLM-mode incidents are always "confirmed" at source (the trigger itself is a Yes/No VLM answer), so there is no separate verdict field.
已验证的告警包含扩展的
info
块：
verdict
含义
confirmed
VLM判定告警真实有效
rejected
VLM判定为误报
unverified
验证未完成（错误）
查看
verification_response_code
（200=成功）和
reasoning
字段获取VLM的解释。VLM模式的事件在源头上始终是"已确认"的（触发条件本身就是VLM的是/否回答），因此没有单独的判定结果字段。

Customize CV Verifier Prompts (CV mode only)

自定义CV验证器提示（仅CV模式）

CV-path verifier prompts live in:
deployments/developer-workflow/dev-profile-alerts/vlm-as-verifier/configs/alert_type_config.json
Each entry maps a CV 
alert_type
(the 
category
field emitted by Behavior Analytics) to the VLM prompts used for verification:
json
{
  "version": "1.0",
  "alerts": [
    {
      "alert_type": "FOV Count Violation",
      "output_category": "Ladder PPE Violation",
      "prompts": {
        "system": "You are a helpful assistant.",
        "user": "Is anyone on the ladder without a hardhat and safety vest? Answer yes or no.",
        "enrichment": "Describe the PPE violation in detail..."
      }
    }
  ]
}
  • alert_type
     must match the 
    category
    emitted by Behavior Analytics.
  • output_category
     is the display name in Elasticsearch / UI.
  • enrichment
     (optional) triggers a second VLM call for a richer description; requires 
    alert_agent.enrichment.enabled: true
    .
  • Changes require a restart of the 
    alert-bridge
    (vlm-as-verifier) container.
VLM real-time prompts are not configured in a file — they are per-request, shaped by 
rtvi_prompt_gen
from the user's natural-language detection description.
CV流程的验证器提示位于：
deployments/developer-workflow/dev-profile-alerts/vlm-as-verifier/configs/alert_type_config.json
每个条目将CV 
alert_type
（行为分析模块输出的
category
字段）映射至用于验证的VLM提示：
json
{
  "version": "1.0",
  "alerts": [
    {
      "alert_type": "FOV Count Violation",
      "output_category": "Ladder PPE Violation",
      "prompts": {
        "system": "You are a helpful assistant.",
        "user": "Is anyone on the ladder without a hardhat and safety vest? Answer yes or no.",
        "enrichment": "Describe the PPE violation in detail..."
      }
    }
  ]
}
  • **
    alert_type
    **必须与行为分析模块输出的
    category
    匹配。
  • **
    output_category
    **是Elasticsearch/UI中的显示名称。
  • enrichment
    （可选）会触发第二次VLM调用以生成更详细的描述；需要设置
    alert_agent.enrichment.enabled: true
    。
  • 修改后需要重启
    alert-bridge
    （vlm-as-verifier）容器。
VLM实时模式的提示不通过文件配置——它们是按请求生成的，由
rtvi_prompt_gen
根据用户的自然语言检测描述生成。

Cross-Skill Links

跨技能链接

TaskSkill
Deploy, redeploy, or switch alert mode
deploy
 skill — 
deploy -p alerts -m {verification,real-time}
Add an RTSP / IP camera to VIOS
vios
 skill — Section 6 (Add Sensor / Stream)
List sensors, take a snapshot, download a clip
vios
 skill
Time-range incident / occupancy / PPE metrics from Elasticsearch
video-analytics
 skill (VA-MCP :9901)
Generate a detailed incident report from an alert
incident-report
 skill
任务技能
部署、重新部署或切换告警模式**
deploy
**技能 — 
deploy -p alerts -m {verification,real-time}
向VIOS添加RTSP/IP摄像头**
vios
**技能 — 第6节（添加传感器/流）
列出传感器、拍摄快照、下载片段**
vios
**技能
从Elasticsearch获取时间范围内的事件/占用率/PPE指标**
video-analytics
**技能（VA-MCP :9901）
根据告警生成详细事件报告**
incident-report
**技能

Gotchas

注意事项

  • Mode is static. Do not attempt to run the VLM flow on a CV deployment or vice versa — required services won't be running. Confirm with the user, then route to the 
    deploy
    skill for redeployment.
  • A mode switch tears down the current deployment. Any running VLM monitoring streams and any CV alert state not already in Elasticsearch will be lost.
  • Don't call the 
    rtvi-vlm
    microservice directly     from this skill. Always go through 
    $AGENT/generate
    . The agent handles sensor→RTSP lookup, stream registration, and teardown.
  • Sensor must already be in VIOS for either mode. If the user hands you only an RTSP URL, use the 
    vios
    skill first.
  • VLM alert trigger is a 
    "yes"
    / 
    "true"
    token match     on the VLM response (case-insensitive). 
    rtvi_prompt_gen
    enforces the Yes/No pattern — don't hand-craft prompts that break it.
  • Stopping a VLM alert is one agent call ("Stop real-time alert…"); the agent handles both the caption-stream and the stream-registration teardown.
  • Prompt changes to 
    alert_type_config.json
    need an 
    alert-bridge
    restart.     
    alert_agent.enrichment.enabled: true
    is required for the 
    enrichment
    prompt to fire.
  • 模式是静态的。不要尝试在CV部署上运行VLM流程，反之亦然——所需服务未运行。先与用户确认，然后引导至
    deploy
    技能进行重新部署。
  • 模式切换会拆除当前部署。所有正在运行的VLM监控流以及未存入Elasticsearch的CV告警状态都会丢失。
  • 请勿直接调用
    rtvi-vlm
    微服务    。始终通过
    $AGENT/generate
    调用。Agent会处理传感器→RTSP查找、流注册和拆除操作。
  • 传感器必须已在VIOS中注册才能使用任何模式。如果用户仅提供RTSP URL，先使用
    vios
    技能。
  • VLM告警触发条件是VLM响应中匹配
    "yes"
    /
    "true"
    令牌    （不区分大小写）。
    rtvi_prompt_gen
    会强制是/否模式——不要手动编写破坏该模式的提示。
  • 停止VLM告警只需一次Agent调用（"Stop real-time alert…"）；Agent会处理字幕流和流注册的拆除操作。
  • 修改
    alert_type_config.json
    中的提示后需要重启
    alert-bridge
    。使用
    enrichment
    提示需要设置
    alert_agent.enrichment.enabled: true
    。