deepstream-profile-pipeline

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

DeepStream Profiling Skill

DeepStream性能分析技能

Profile-driven pipeline creation. When the user indicates they want an efficient DeepStream pipeline, this skill replaces guesswork with two measured numbers — inference plateau batch and HW ceiling — and derives every other config from them. Then it profiles the E2E pipeline with Nsight Systems and reports per-plugin NVTX timings.

Model- and pipeline-agnostic. The skill assumes only that the inference element is

nvinfer

nvinferserver

(so model dims, precision, and batch knobs are settable through the standard config). It works for detection (with or without tracker), classification, segmentation, VLM, and embedding pipelines. Source can be file, RTSP, USB camera, or any mix. The skill reads the user's actual config to discover model dims / target FPS / source properties — it does NOT assume any particular model, codec, or resolution.

Constraint. Terminal only. Use
nsys profile
to capture and
nsys stats
to extract. Do not depend on Nsight Lens or any GUI.

基于性能分析的管道创建。当用户表示需要高效的DeepStream管道时，该技能会用两个实测数值——推理平台期批量和硬件上限——替代猜测，并据此推导所有其他配置。随后它会使用Nsight Systems对端到端（E2E）管道进行性能分析，并报告每个插件的NVTX时序数据。

与模型和管道无关。该技能仅假设推理元素为

nvinfer

或

nvinferserver

（因此模型维度、精度和批量参数可通过标准配置设置）。它适用于检测（带或不带跟踪器）、分类、分割、VLM和嵌入管道。数据源可以是文件、RTSP、USB摄像头或任意组合。该技能会读取用户的实际配置以发现模型维度/目标FPS/源属性——不假设任何特定模型、编解码器或分辨率。

约束条件：仅支持终端环境。使用
nsys profile
捕获数据，
nsys stats
提取数据。不依赖Nsight Lens或任何GUI。

When to trigger

触发时机

Activate this skill at pipeline creation time when the user's ask carries efficiency intent. Concrete triggers:

"build an efficient / fast / performant / optimized pipeline"
"give me a pipeline that runs well on this GPU"
"benchmark / profile / measure / tune / optimize this pipeline"
"I want to run N streams at M FPS"
"how many streams can this GPU handle"
user explicitly asks for
```
nsys
```
or Nsight

For plain "build a pipeline" / "display this video" / "save this stream" with no perf intent, hand off to the

deepstream-generate-pipeline

skill instead.

当用户的需求带有性能优化意图时，在管道创建阶段激活此技能。具体触发场景：

"构建一个高效/快速/高性能/优化后的管道"
"给我一个能在这块GPU上流畅运行的管道"
"对这个管道进行基准测试/性能分析/测量/调优/优化"
"我想要以M FPS的速率运行N路流"
"这块GPU能处理多少路流"
用户明确要求使用
```
nsys
```
或Nsight

对于普通的"构建一个管道"/"显示这段视频"/"保存这个流"且无性能优化意图的请求，转交给

deepstream-generate-pipeline

技能处理。

The 6-stage flow

六阶段流程

Run the stages in order. Stage 0 fires before the pipeline is generated, so the user starts from a perf-tuned skeleton. Stages 1–5 measure and verify.

按顺序执行各阶段。阶段0在管道生成之前触发，因此用户从一个已调优的性能骨架开始。阶段1-5用于测量和验证。

Stage 0 — Preset-apply (at pipeline-creation time)

阶段0 — 预设应用（管道创建阶段）

Trigger: any time the coding agent is about to generate a new DS pipeline AND the user's prompt carries efficiency intent (see "When to trigger" above).

Action: pre-apply these defaults without prompting. The user does not need to know any of them; they just get a pipeline that's already in the right shape.

Knob	Default value	Skip when
`nvinfer.network-mode`	`1` (INT8) if a calibration file is present at `int8-calib-file=<path>` , else `2` (FP16). Never FP32.	Model has no INT8 calibration AND the user explicitly says "FP32".
`nvinfer.model-engine-file`	Pre-built `.engine` path	Always set. Force a one-shot prebuild before measurement.
`nvinfer.infer-dims`	`3;<H>;<W>` matching the model's native input	Always set, even for static-shape ONNX (harmless).
`nvstreammux.batch-size`	`min(N_streams, 16)` until microbench refines it	—
`nvstreammux.width / height`	model's native input dims (read from the nvinfer config's `infer-dims=3;H;W` )	User explicitly asks for native source resolution at the muxer.
`nvstreammux.batched-push-timeout`	`1e6 / source_fps` µs (33333 for 30 fps)	—
`nvstreammux.nvbuf-memory-type`	`0` (NVMM)	—
Decoder `num-extra-surfaces`	`min(batch_size, 5)`	—
Decoder `cudadec-memtype`	`0` (NVMM)	—
Sink	`fakesink sync=False` for the benchmark variant	User asked for on-screen display or on-disk recording (then keep OSD/tiler/encoder/sink and produce TWO variants).
OSD + tiler	omit	User asked for visible output.
Tracker `ll-config-file`	`config_tracker_NvDCF_max_perf.yml` (perf-tuned NvDCF preset shipped with DS 9.0)	Tracker not present.
Tracker `tracker-width / height`	480 / 288	—
Tracker `enable-batch-process` (in linked YAML)	`1`	—
Queue between source and pgie	`max-size-buffers = batch_size × 4`	No queue requested (rare).
Kafka/message queue	`max-size-buffers=2, leaky=2`	No Kafka.
Decode-side `PerfMonitor`	attach (in addition to pgie-side)	Pipeline is `nvurisrcbin → pgie` direct without intermediate queue.

Why Stage 0 exists: without it, every newly generated pipeline starts from display-first defaults and Stages 1–5 spend cycles fixing avoidable issues. Stage 0 is the "don't write a bad pipeline in the first place" gate.

The student / API user never sees these knobs. The skill's response back to the user is in plain English (FPS, stream count, observed bottleneck), not knob names.

触发条件：当代码Agent即将生成新的DeepStream管道，且用户的提示带有性能优化意图（见上文"触发时机"）。

操作：无需提示，直接应用以下默认配置。用户无需了解这些配置细节；他们只需获得一个已处于最优状态的管道。

配置项	默认值	跳过条件
`nvinfer.network-mode`	如果 `int8-calib-file=<路径>` 存在校准文件，则为 `1` （INT8），否则为 `2` （FP16）。绝不使用FP32。	模型无INT8校准文件且用户明确要求使用"FP32"。
`nvinfer.model-engine-file`	预构建的 `.engine` 文件路径	始终设置。强制在测量前一次性预构建引擎。
`nvinfer.infer-dims`	`3;<H>;<W>` ，与模型原生输入维度匹配	始终设置，即使对于静态形状的ONNX模型（无负面影响）。
`nvstreammux.batch-size`	`min(N_streams, 16)` ，后续由微基准测试优化	—
`nvstreammux.width / height`	模型的原生输入维度（从nvinfer配置的 `infer-dims=3;H;W` 读取）	用户明确要求在多路复用器处使用源原生分辨率。
`nvstreammux.batched-push-timeout`	`1e6 / source_fps` 微秒（30 fps对应33333）	—
`nvstreammux.nvbuf-memory-type`	`0` （NVMM）	—
解码器 `num-extra-surfaces`	`min(batch_size, 5)`	—
解码器 `cudadec-memtype`	`0` （NVMM）	—
输出端	基准测试变体使用 `fakesink sync=False`	用户要求屏幕显示或磁盘录制（则保留OSD/分屏器/编码器/输出端，并生成两种变体）。
OSD + 分屏器	省略	用户要求可见输出。
跟踪器 `ll-config-file`	`config_tracker_NvDCF_max_perf.yml` （DeepStream 9.0附带的性能调优NvDCF预设）	管道中无跟踪器。
跟踪器 `tracker-width / height`	480 / 288	—
跟踪器 `enable-batch-process` （关联YAML文件中）	`1`	—
源与pgie之间的队列	`max-size-buffers = batch_size × 4`	未请求队列（罕见情况）。
Kafka/消息队列	`max-size-buffers=2, leaky=2`	无Kafka。
解码端 `PerfMonitor`	附加到管道（除pgie端外）	管道为 `nvurisrcbin → pgie` 直接连接，无中间队列。

阶段0存在的原因：如果没有这个阶段，每个新生成的管道都会从以显示为优先的默认配置开始，阶段1-5会花费时间修复可避免的问题。阶段0是"从一开始就不编写糟糕管道"的关卡。

学习者/API用户永远不会看到这些配置项。技能返回给用户的内容是通俗易懂的英文（FPS、流数量、观测到的瓶颈），而非配置项名称。

The verification flow (Stages 1–5)

验证流程（阶段1-5）

Run the stages in order. Do not skip a stage — later stages depend on earlier ones' outputs.

按顺序执行各阶段。不要跳过任何阶段——后续阶段依赖于前序阶段的输出。

Stage 1 — NVTX coverage check

阶段1 — NVTX覆盖检查

DeepStream plugins emit NVTX ranges natively; custom plugins and plain GStreamer-core elements (

queue

tee

h264parse

, etc.) do not. Before profiling, list the elements the pipeline uses and classify each.

Read the pipeline definition (gst-launch string or
```
pipeline.py
```
).
For each element, look it up in references/nvtx-coverage.md.
Classify COVERED (emits NVTX in this DS / image / nsys combo) or UNINSTRUMENTED.
MVP rule: the skill prefers per-plugin NVTX as confirmation but does not require it. Decode-bound diagnosis works from microbench shape +
```
nvidia-smi dmon
```
; compute-bound from CUDA kernel mix; memcpy from
```
cuda_gpu_mem_time_sum
```
. NVTX is a bonus.
For UNINSTRUMENTED elements, the skill reports "not directly measurable in this build" and still applies the closed-form R1–R6 knobs (which are derived from inputs, not from per-plugin profile data).
Auto-injecting NVTX for uninstrumented elements is out of scope for this version — flag it as follow-up in the final report.

Output of Stage 1: a short coverage table, e.g.

text

nvurisrcbin       COVERED
nvstreammux       COVERED
nvinfer           COVERED
nvtracker         COVERED
queue_src         UNINSTRUMENTED — not re-tuned
fakesink          UNINSTRUMENTED — not re-tuned

DeepStream插件会原生输出NVTX范围；自定义插件和普通GStreamer核心元素（

queue

、

tee

、

h264parse

等）则不会。在性能分析前，列出管道使用的所有元素并进行分类。

读取管道定义（gst-launch字符串或
```
pipeline.py
```
）。
对每个元素，在references/nvtx-coverage.md中查找。
分类为已覆盖（在此DeepStream/镜像/nsys组合下输出NVTX）或未插桩。
MVP规则：技能优先使用每个插件的NVTX数据作为确认，但不强制要求。解码瓶颈诊断可通过微基准测试形状+
```
nvidia-smi dmon
```
完成；计算瓶颈可通过CUDA内核组合完成；内存拷贝可通过
```
cuda_gpu_mem_time_sum
```
完成。NVTX是额外补充。
对于未插桩元素，技能会报告"在此构建版本中无法直接测量"，但仍会应用封闭式R1-R6配置项（这些配置项由输入推导而来，而非来自每个插件的性能分析数据）。
本版本不支持为未插桩元素自动注入NVTX——在最终报告中将其标记为后续工作。

阶段1输出：简短的覆盖表，例如：

text

nvurisrcbin       已覆盖
nvstreammux       已覆盖
nvinfer           已覆盖
nvtracker         已覆盖
queue_src         未插桩 — 未重新调优
fakesink          未插桩 — 未重新调优

Stage 2 — HW discovery

阶段2 — 硬件发现

Run

nvidia-smi

and derive theoretical ceilings for the host GPU. Minimum queries:

bash

undefined

运行

nvidia-smi

并推导主机GPU的理论上限。至少执行以下查询：

bash

undefined

Identity + memory + compute

标识 + 内存 + 计算

nvidia-smi --query-gpu=name,compute_cap,memory.total,memory.free,
clocks.max.sm,clocks.max.memory,utilization.gpu
--format=csv,noheader,nounits

NVDEC / NVENC utilization (per-engine)

NVDEC / NVENC 利用率（按引擎）

nvidia-smi --query-gpu=utilization.decoder,utilization.encoder
--format=csv,noheader,nounits

PCIe link width/gen (for H2D memcpy ceiling)

PCIe链路宽度/代次（用于H2D内存拷贝上限）

nvidia-smi --query-gpu=pcie.link.gen.current,pcie.link.width.current
--format=csv,noheader,nounits


Derive from those numbers:

- **Decode ceiling (fps)**: NVDEC_count × per-unit H265/H264 fps for the source resolution
  (table in [references/hw-ceiling-formulas.md](references/hw-ceiling-formulas.md)).
- **Compute ceiling (TOPS)**: SM count × clock × ops-per-clock at the target precision. Gives
  an upper bound — real models hit 30–60% of this.
- **Memory-bandwidth ceiling (GB/s)**: memory clock × bus width. Model weight reads +
  activations should fit well under this.
- **Memcpy ceiling (GB/s)**: PCIe gen × width × 0.8 practical. Only relevant if NVMM is
  broken and H2D/D2H transfers appear in Stage 5.

Store the derived ceilings — they drive the Stage 5 "actual vs. theoretical" section.

Full formulas and the per-codec NVDEC throughput table:
[references/hw-ceiling-formulas.md](references/hw-ceiling-formulas.md).

nvidia-smi --query-gpu=pcie.link.gen.current,pcie.link.width.current
--format=csv,noheader,nounits


从这些数值推导：

- **解码上限（fps）**：NVDEC数量 × 对应源分辨率下每单元H265/H264的fps（表格见[references/hw-ceiling-formulas.md](references/hw-ceiling-formulas.md)）。
- **计算上限（TOPS）**：SM数量 × 时钟频率 × 目标精度下每时钟周期操作数。给出一个上限——实际模型能达到此值的30-60%。
- **内存带宽上限（GB/s）**：内存时钟 × 总线宽度。模型权重读取+激活数据应远低于此值。
- **内存拷贝上限（GB/s）**：PCIe代次 × 宽度 × 0.8（实际值）。仅当NVMM失效且阶段5中出现H2D/D2H传输时才相关。

存储推导得到的上限——它们将驱动阶段5的"实际值vs理论值"部分。

完整公式和每编解码器NVDEC吞吐量表格：
[references/hw-ceiling-formulas.md](references/hw-ceiling-formulas.md)。

Stage 3 — Inference-only micro-benchmark

阶段3 — 仅推理微基准测试

Run only the inference stage (source → streammux → nvinfer → fakesink), sweeping

batch-size

to find the plateau. This isolates the model's true peak FPS from everything else, and answers "how many streams fit into a single batch without FPS dropping?".

Sweep:

batch-size ∈ {1, 2, 4, 8, 16, 32}

(cap at

N_streams

and at GPU memory).

For each batch size:

Set

nvstreammux.batch-size = nvinfer.batch-size = B

Set
```
nvstreammux.width/height
```
= the model's native
```
infer-dims
```
(read from the nvinfer config).
```
fakesink sync=False
```
as the only branch.
Run 30 s; measure FPS from
```
measure_fps_probe
```
(console) or DS
```
PerfMonitor
```
.
Record
```
(B, fps)
```
.

Plateau batch = the smallest B where increasing to 2×B yields < 5% FPS gain. That is the target batch for the full pipeline.

If the user's N_streams ≤ plateau batch, set final batch = N_streams. Otherwise set final batch = plateau batch and note that the pipeline will process streams in multiple batches per tick.

运行仅包含推理阶段的管道（源→多路复用器→nvinfer→fakesink），遍历

batch-size

以找到平台期。这将隔离模型的真实峰值FPS，排除其他因素的影响，并回答"单个批量中可容纳多少路流而不会导致FPS下降？"。

遍历范围：

batch-size ∈ {1, 2, 4, 8, 16, 32}

（上限为

N_streams

和GPU内存）。

对于每个批量大小：

设置

nvstreammux.batch-size = nvinfer.batch-size = B

。

设置
```
nvstreammux.width/height
```
= 模型的原生
```
infer-dims
```
（从nvinfer配置读取）。
使用
```
fakesink sync=False
```
作为唯一分支。
运行30秒；通过
```
measure_fps_probe
```
（控制台）或DeepStream
```
PerfMonitor
```
测量FPS。
记录
```
(B, fps)
```
。

平台期批量 = 当批量增加到2×B时FPS提升<5%的最小B值。这是完整管道的目标批量值。

如果用户的N_streams ≤ 平台期批量，则设置最终批量 = N_streams。否则设置最终批量 = 平台期批量，并注明管道将在每个时钟周期内分多个批量处理流。

Stage 4 — Derive configs

阶段4 — 推导配置

From

(plateau_batch, HW_ceilings, N_streams, source_res, source_fps)

, set every tunable knob at once. Do not tune one knob at a time — the derivation rules are closed-form.

Knobs to set, in order:

Streammux:

batch-size = final_batch

width/height = min(source_res, infer_dims)

batched-push-timeout = 1e6 / source_fps

µs,

nvbuf-memory-type = 0

Inference:

batch-size = final_batch

network-mode = 1 (INT8) if calib file exists else 2 (FP16)

interval = 0

infer-dims = model's native dims

model-engine-file = pre-built .engine path

Decoder (on

nvurisrcbin

nvmultiurisrcbin

nvv4l2decoder

num-extra-surfaces = min(final_batch, 5)

cudadec-memtype = 0

nvbuf-memory-type = 0

Tracker (if present):

enable-batch-process = 1

, tracker res 480×288, point

ll-config-file

config_tracker_NvDCF_max_perf.yml

Queues (if present between decoder and streammux, or streammux and nvinfer):
```
max-size-buffers = final_batch × 2
```
. Kafka/message branches:
```
leaky=2, max-size-buffers=2
```
.

Full derivation table with each formula and a one-line "why": references/config-derivation-rules.md.

Write the derived values into the user's config files (

pgie_config.yml

tracker_config.yml

pipeline.py

source properties, any

deepstream-app

.txt

). Always

Read

before

Edit

. Keep edits surgical — do not reformat unrelated lines.

根据

(platform_batch, HW_ceilings, N_streams, source_res, source_fps)

，一次性设置所有可调配置项。不要逐个调优配置项——推导规则为封闭式。

按顺序设置的配置项：

多路复用器：

batch-size = final_batch

，

width/height = min(source_res, infer_dims)

，

batched-push-timeout = 1e6 / source_fps

微秒，

nvbuf-memory-type = 0

。

推理：

batch-size = final_batch

，

network-mode = 1 (INT8)

（如果存在校准文件）否则

2 (FP16)

，

interval = 0

，

infer-dims = 模型原生维度

，

model-engine-file = 预构建的.engine文件路径

。

解码器（位于

nvurisrcbin

nvmultiurisrcbin

nvv4l2decoder

）：

num-extra-surfaces = min(final_batch, 5)

，

cudadec-memtype = 0

，

nvbuf-memory-type = 0

。

跟踪器（如果存在）：

enable-batch-process = 1

，跟踪器分辨率480×288，

ll-config-file

指向

config_tracker_NvDCF_max_perf.yml

。

队列（如果存在于解码器与多路复用器之间，或多路复用器与nvinfer之间）：
```
max-size-buffers = final_batch × 2
```
。Kafka/消息分支：
```
leaky=2, max-size-buffers=2
```
。

包含每个公式和单行说明的完整推导表格： references/config-derivation-rules.md。

将推导得到的值写入用户的配置文件（

pgie_config.yml

、

tracker_config.yml

、

pipeline.py

源属性、任何

deepstream-app

的

.txt

文件）。始终先读取再编辑。保持编辑精准——不要重新格式化无关行。

Stage 5 — E2E profile + report

阶段5 — 端到端性能分析 + 报告

Run the E2E pipeline under

nsys profile

and extract per-plugin timings via

nsys stats

Capture:

bash

TS=$(date +%Y%m%d_%H%M%S)
OUT=/tmp/ds_profile_${TS}
nsys profile \
  --trace=cuda,nvtx,osrt \
  --gpu-metrics-devices=all \
  --cuda-memory-usage=true \
  --force-overwrite=true \
  --duration=30 \
  --output=${OUT} \
  <your-pipeline-launch-command>

Extract:

bash

undefined

在

nsys profile

下运行端到端管道，并通过

nsys stats

提取每个插件的时序数据。

捕获命令：

bash

TS=$(date +%Y%m%d_%H%M%S)
OUT=/tmp/ds_profile_${TS}
nsys profile \
  --trace=cuda,nvtx,osrt \
  --gpu-metrics-devices=all \
  --cuda-memory-usage=true \
  --force-overwrite=true \
  --duration=30 \
  --output=${OUT} \
  <你的管道启动命令>

提取命令：

bash

undefined

Per-kernel GPU time (top 10)

按内核统计GPU时间（前10个）

nsys stats --report cuda_gpu_kern_sum --format csv ${OUT}.nsys-rep | head -20

Per-NVTX-range time (top 10) — this is the DS per-plugin breakdown

按NVTX范围统计时间（前10个）——这是DeepStream每个插件的细分数据

nsys stats --report nvtx_sum --format csv ${OUT}.nsys-rep | head -20

Memcpy totals

内存拷贝总计

nsys stats --report cuda_gpu_mem_time_sum --format csv ${OUT}.nsys-rep

GPU metrics (SM activity, DRAM throughput) — requires --gpu-metrics-devices

GPU指标（SM活动、DRAM吞吐量）——需要--gpu-metrics-devices参数

nsys stats --report gpu_metric_gpu_util_sum --format csv ${OUT}.nsys-rep


Full command reference: [references/nsys-cli-recipes.md](references/nsys-cli-recipes.md).

Report (Markdown, to stdout — no external UI):

```markdown

nsys stats --report gpu_metric_gpu_util_sum --format csv ${OUT}.nsys-rep


完整命令参考：[references/nsys-cli-recipes.md](references/nsys-cli-recipes.md)。

报告（Markdown格式，输出到标准输出——无外部UI）：

```markdown

Profile summary

性能分析摘要

Hardware: <name>, <mem_total> GB, SM x<sm>, NVDEC x<nvdec>, PCIe Gen<g> x<w> Ceilings: decode <X> fps, compute ~<Y> TOPS @ INT8, memory <Z> GB/s

Inference plateau: batch=<B>, peak=<F> fps per batch → <F × B> fps aggregate

E2E measured: <actual> fps (=<pct>% of inference plateau)

硬件：<名称>, <总内存> GB, SM ×<数量>, NVDEC ×<数量>, PCIe Gen<代次> ×<宽度> 上限：解码 <X> fps, 计算 ~<Y> TOPS @ INT8, 内存 <Z> GB/s

推理平台期：批量=<B>, 峰值=<F> fps/批量 → <F × B> fps 总吞吐量

端到端实测：<实际值> fps (=<百分比>% 的推理平台期值)

Per-plugin time (from NVTX) — only for plugins emitting NVTX in this build

插件耗时（来自NVTX）——仅适用于此构建版本中输出NVTX的插件

Plugin	Share of wall time	GPU / CPU	Notes
nvinfer	<pct>%	GPU	(always emitted; if absent, NVTX injection is broken)
nvdsosd	<pct>%	GPU	(when in pipeline)
...	...	...	(other plugins as the verification probe shows)

(Numbers above are illustrative — fill in from

nsys stats --report nvtx_sum

. Plugins that don't emit NVTX in your DS / image combo simply don't appear; that's not a bug, it's the limit of what NVTX captures here. See

references/nvtx-coverage.md

插件	占总运行时间比例	GPU / CPU	说明
nvinfer	<百分比>%	GPU	（始终输出；如果缺失，说明NVTX注入失效）
nvdsosd	<百分比>%	GPU	（当管道中包含时）
...	...	...	（其他插件根据验证探针显示）

（以上数值为示例——从

nsys stats --report nvtx_sum

填充数据。在此DeepStream/镜像组合中不输出NVTX的插件不会显示；这不是bug，而是NVTX在此场景下的捕获限制。详见

references/nvtx-coverage.md

。）

Applied configs (sample shape; values come from R1–R6 + Stage 3 measurements)

应用的配置（示例格式；值来自R1-R6 + 阶段3测量结果）

nvstreammux.batch-size = <plateau_batch>

```
nvinfer.network-mode = 1 (INT8)
```
if calibration available, else
```
2 (FP16)
```

decoder

num-extra-surfaces = min(plateau_batch, 5)

queue between source and pgie,
```
max-size-buffers = plateau_batch × 4
```
... (full list per the user's pipeline shape)

nvstreammux.batch-size = <平台期批量>

```
nvinfer.network-mode = 1 (INT8)
```
（如果有校准文件），否则
```
2 (FP16)
```

解码器

num-extra-surfaces = min(平台期批量, 5)

源与pgie之间的队列，
```
max-size-buffers = 平台期批量 × 4
```
...（根据用户管道形状列出完整列表）

Uninstrumented (skipped re-tune)

未插桩元素（跳过重新调优）

List the elements that didn't emit NVTX in this build (typically the closed-source binary plugins — see

references/nvtx-coverage.md

) plus plain GStreamer-core helpers. Report them so the user knows what wasn't directly measurable.


Keep the summary terse. Raw `nsys stats` CSV goes into the temp file, not the response.

列出在此构建版本中不输出NVTX的元素（通常为闭源二进制插件——详见

references/nvtx-coverage.md

）以及普通GStreamer核心辅助元素。报告这些元素以便用户了解哪些部分无法直接测量。


保持摘要简洁。原始`nsys stats` CSV文件存入临时文件，不包含在响应中。

Reference documents

参考文档

Document	Use when
references/nvtx-coverage.md	Stage 1 — classifying each pipeline element as COVERED or UNINSTRUMENTED.
references/hw-ceiling-formulas.md	Stage 2 — turning `nvidia-smi` output into decode / compute / memory ceilings.
references/config-derivation-rules.md	Stage 4 — per-knob formula keyed to `(plateau_batch, HW, N_streams, source_res, source_fps)` .
references/nsys-cli-recipes.md	Stages 3 & 5 — exact `nsys profile` / `nsys stats` invocations.

文档	使用场景
references/nvtx-coverage.md	阶段1——将每个管道元素分类为已覆盖或未插桩。
references/hw-ceiling-formulas.md	阶段2——将 `nvidia-smi` 输出转换为解码/计算/内存上限。
references/config-derivation-rules.md	阶段4——基于 `(platform_batch, HW, N_streams, source_res, source_fps)` 的每个配置项公式。
references/nsys-cli-recipes.md	阶段3 & 5——精确的 `nsys profile` / `nsys stats` 调用命令。

Non-goals (this version)

非目标（本版本）

No Nsight Lens / no GUI. Terminal only.
No NVTX auto-injection for uninstrumented plugins. MVP skips their knobs. Future work.
No iterative tune-measure-tune loop. Stage 4 derives configs once from closed-form rules; Stage 5 measures and reports. If the user wants to keep tuning, they can re-invoke the skill with updated inputs.

无Nsight Lens / 无GUI。仅支持终端环境。
不为未插桩插件自动注入NVTX。MVP版本跳过这些插件的配置项。后续版本再实现。
无迭代调优-测量-调优循环。阶段4根据封闭式规则推导一次配置；阶段5进行测量并报告。如果用户需要继续调优，可使用更新后的输入重新调用该技能。

Related skills

Notes

注意事项

Lives in
```
skills/deepstream-profile-pipeline/
```
alongside the other DS skills, per the repo convention in
```
CLAUDE.md
```
.
For ground-truth on any plugin's properties (types, defaults, ranges) and pad caps, query the loaded binary inside the DS container:
bash
```
gst-inspect-1.0 nvinfer
gst-inspect-1.0 nvstreammux
gst-inspect-1.0 nvurisrcbin   # works on closed-source binary plugins too
gst-inspect-1.0 | grep ^nv    # list every NVIDIA-specific element this build ships
```
Plugin naming convention: any element prefixed
```
nv*
```
is NVIDIA DeepStream-specific (NVMM-capable, may emit NVTX); everything else is upstream GStreamer-core (no NVMM, never emits DS NVTX). Use this prefix as the first-pass classifier when triaging an unfamiliar pipeline.
The open-source subset of plugin code lives under
```
/opt/nvidia/deepstream/deepstream/sources/gst-plugins/
```
if you need to read the implementation (only some plugins are open — closed ones must be inspected via
```
gst-inspect-1.0
```
and behaviour observed at runtime).

按照
```
CLAUDE.md
```
中的仓库约定，存放在
```
skills/deepstream-profile-pipeline/
```
，与其他DeepStream技能放在一起。
如需获取任何插件属性（类型、默认值、范围）和pad caps的真实信息，在DeepStream容器内查询已加载的二进制文件：
bash
```
gst-inspect-1.0 nvinfer
gst-inspect-1.0 nvstreammux
gst-inspect-1.0 nvurisrcbin   # 对闭源二进制插件也有效
gst-inspect-1.0 | grep ^nv    # 列出此构建版本中所有NVIDIA专属元素
```
插件命名约定：任何前缀为
```
nv*
```
的元素都是NVIDIA DeepStream专属（支持NVMM，可能输出NVTX）；其他所有元素都是上游GStreamer核心元素（不支持NVMM，从不输出DeepStream NVTX）。在分析不熟悉的管道时，使用此前缀作为初步分类依据。
插件代码的开源子集位于
```
/opt/nvidia/deepstream/deepstream/sources/gst-plugins/
```
（仅部分插件开源——闭源插件必须通过
```
gst-inspect-1.0
```
检查并在运行时观察其行为）。

deepstream-profile-pipeline

Original

Translation

DeepStream Profiling Skill

DeepStream性能分析技能

When to trigger

触发时机

The 6-stage flow

六阶段流程

Stage 0 — Preset-apply (at pipeline-creation time)

阶段0 — 预设应用（管道创建阶段）

The verification flow (Stages 1–5)

验证流程（阶段1-5）

Stage 1 — NVTX coverage check

阶段1 — NVTX覆盖检查

Stage 2 — HW discovery

阶段2 — 硬件发现

Identity + memory + compute

标识 + 内存 + 计算

NVDEC / NVENC utilization (per-engine)

NVDEC / NVENC 利用率（按引擎）

PCIe link width/gen (for H2D memcpy ceiling)

PCIe链路宽度/代次（用于H2D内存拷贝上限）

Stage 3 — Inference-only micro-benchmark

阶段3 — 仅推理微基准测试

Stage 4 — Derive configs

阶段4 — 推导配置

Stage 5 — E2E profile + report

阶段5 — 端到端性能分析 + 报告

Per-kernel GPU time (top 10)

按内核统计GPU时间（前10个）

Per-NVTX-range time (top 10) — this is the DS per-plugin breakdown

按NVTX范围统计时间（前10个）——这是DeepStream每个插件的细分数据

Memcpy totals

内存拷贝总计

GPU metrics (SM activity, DRAM throughput) — requires --gpu-metrics-devices

GPU指标（SM活动、DRAM吞吐量）——需要--gpu-metrics-devices参数

Profile summary

性能分析摘要

Per-plugin time (from NVTX) — only for plugins emitting NVTX in this build

插件耗时（来自NVTX）——仅适用于此构建版本中输出NVTX的插件

Applied configs (sample shape; values come from R1–R6 + Stage 3 measurements)

应用的配置（示例格式；值来自R1-R6 + 阶段3测量结果）

Uninstrumented (skipped re-tune)

未插桩元素（跳过重新调优）

Reference documents

参考文档

Non-goals (this version)

非目标（本版本）

Related skills

相关技能

Notes

注意事项