rocprof-compute-pc-sampling

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

PC sampling

PC sampling

PC sampling answers a question counters cannot: which instruction inside the kernel is hot, and what the wavefront was waiting on. Use it after counters have already identified the kernel worth this level of detail.
PC sampling is experimental. Every command below needs
--experimental
, and option names can change between releases, so run
rocprof-compute profile --experimental --help
before choosing flags.
PC sampling 能回答 counters 无法回答的问题:kernel 内部哪条指令是热点,以及 wavefront 当时在等待什么。请先通过 counters 找出值得进行这种细粒度分析的 kernel,然后再使用 PC sampling。
PC sampling 仍处于实验阶段。下面的每条命令都需要
--experimental
,并且选项名称可能会在版本之间发生变化,因此请先运行
rocprof-compute profile --experimental --help
,再选择参数。

Before you start

开始之前

If
command -v rocprof-compute
finds the tool and
rocprof-compute analyze --debug-verify-deps
passes, skip this section. On ROCm 10.2, use
--verify-deps
instead of
--debug-verify-deps
.
Otherwise, check that ROCm and a GPU are present:
bash
rocminfo
amd-smi list
If either command fails, ROCm is not installed or no GPU is visible. Stop and point the user to "Install the ROCm Core SDK" in the install guide. Installing rocprof-compute alone does not help.
Check the GPU against compatible accelerators. The table lists product families, so match the
Marketing Name
that
rocminfo
prints, and use the
gfx
name to confirm. If the GPU is not supported, stop and tell the user.
Then follow "Install the analyze mode dependencies" in the install guide, which installs rocprof-compute in a virtual environment of its own. Use the nightly index so the version matches these skills. Activate that environment with
source <venv>/bin/activate
before you run rocprof-compute, and confirm with
rocprof-compute analyze --debug-verify-deps
. If the application is a Python script, note the full path of its Python before you activate, and run the script with that path.
如果
command -v rocprof-compute
能找到该工具,并且
rocprof-compute analyze --debug-verify-deps
检查通过,则可跳过本节。 在 ROCm 10.2 上,请使用
--verify-deps
代替
--debug-verify-deps
。
否则,检查 ROCm 和 GPU 是否存在:
bash
rocminfo
amd-smi list
如果任一命令执行失败,说明 ROCm 未安装或没有检测到 GPU。请停止操作,并引导用户查看 安装指南中的 "Install the ROCm Core SDK"。仅安装 rocprof-compute 没有帮助。
请根据 兼容的加速器列表检查 GPU。该表列出产品系列,因此应匹配
rocminfo
输出的
Marketing Name
,并使用
gfx
名称进行确认。如果 GPU 不受支持,请停止操作并告知用户。
然后按照安装指南中的 "Install the analyze mode dependencies" 操作,该步骤会将 rocprof-compute 安装到其独立的虚拟环境中。请使用 nightly 索引,使版本与这些 skill 匹配。运行 rocprof-compute 前,使用
source <venv>/bin/activate
激活该环境,并通过
rocprof-compute analyze --debug-verify-deps
进行确认。如果应用程序是 Python 脚本,请在激活环境前记下其 Python 的完整路径,并使用该路径运行脚本。

1. Check what the hardware supports

1. 检查硬件支持

bash
rocprofv3-avail info --pc-sampling
This reports the sampling methods the installed device accepts and the valid interval range. Two methods exist:
  • stochastic
    samples in hardware and is the only method that records a stall reason. It requires MI300 or newer.
  • host_trap
    works more widely but records no stall reason, and its samples can land a little past the instruction responsible.
--pc-sampling-method
defaults to
stochastic
. Fall back to
host_trap
only when the device rejects stochastic.
Start with the default interval: leave out
--pc-sampling-interval
. Change it only when the analysis shows too few or too many samples, and keep it within the range the command above prints. Stochastic intervals are counted in cycles and must be a power of two. Host-trap intervals are in microseconds.
bash
rocprofv3-avail info --pc-sampling
该命令报告已安装的设备支持哪些采样方法,以及有效的时间间隔范围。共有两种方法:
  • stochastic
    由硬件执行采样,是唯一会记录 stall reason 的方法。它要求 MI300 或更新型号。
  • host_trap
    的适用范围更广,但不会记录 stall reason,且其采样点可能略微落在导致停顿的指令之后。
--pc-sampling-method
默认为
stochastic
。仅当设备拒绝 stochastic 时,才改用
host_trap
。
先使用默认间隔:不要指定
--pc-sampling-interval
。仅当分析结果显示样本过少或过多时才更改该值,并确保其处于上述命令输出的有效范围内。stochastic 间隔以 cycles 为单位,且必须是 2 的幂。host-trap 间隔以微秒为单位。

2. Build for source attribution

2. 构建可归因到源代码的程序

Samples always map to the kernel's assembly. Mapping assembly back to a source line requires debug info in the application, for example
hipcc -g
. Without it, the
Source
column in CSV output is empty. Keep optimization flags on; profiling a
-O0
build measures the wrong program.
Ask about the build before collecting, because rebuilding afterwards means re-profiling.
样本始终会映射到 kernel 的汇编代码。将汇编代码映射回源代码行需要应用程序包含调试信息,例如使用
hipcc -g
构建。如果没有调试信息,CSV 输出中的
Source
列将为空。 请保留优化编译参数;对
-O0
构建进行性能分析,测得并不是目标程序。
收集前询问构建方式,因为事后重新构建就需要重新分析。

3. Collect

3. 采集

bash
rocprof-compute profile \
    --experimental \
    --pc-sampling \
    --pc-sampling-method stochastic \
    --name <workload_name> \
    -- <application> <args>
PC sampling does not collect counters. A PC-sampling-only profile cannot be combined with the framework tracing options, and it will not populate the counter panels. When the user needs both, collect two workloads.
bash
rocprof-compute profile \
    --experimental \
    --pc-sampling \
    --pc-sampling-method stochastic \
    --name <workload_name> \
    -- <application> <args>
PC sampling 不收集 counters。仅执行 PC sampling 的 profile 不能与 framework tracing 选项结合使用,也不会填充 counter 面板。当用户同时需要两者时,请分别采集两个 workload。

4. Analyze

4. 分析

bash
rocprof-compute analyze --path ./workloads/<name>/<gpu_model>
The PC sampling table shows one row per kernel, hottest first, so it picks the kernel to look at.
--pc-sampling-rows
defaults to 10 kernels; pass
0
for every kernel.
Read the table as:
ColumnMeaning
Sample counthow often execution was in the kernel
Stall reasonwhat the kernel's wavefronts were waiting on, stochastic sampling only
For the hot instructions of that kernel, export the annotated disassembly (step 5) and open the kernel's ISA CSV.
A high sample count marks a hot instruction, not a defective one. A loop body is supposed to be hot. What matters is a hot instruction whose stall reason says it is waiting: a memory stall on a load means the kernel is latency-bound there, and the fix is usually coalescing, prefetching, or more independent work between the load and its use.
Stall reasons and what each one implies are listed in pc_sampling.rst. Do not interpret a stall reason from host-trap data; it does not record one.
bash
rocprof-compute analyze --path ./workloads/<name>/<gpu_model>
PC sampling 表格为每个 kernel 显示一行,并按热点程度从高到低排序,因此可据此选定要分析的 kernel。
--pc-sampling-rows
默认为 10 个 kernel;传入
0
可显示所有 kernel。
表格含义如下:
列含义
Sample count执行处于该 kernel 中的频率
Stall reasonkernel 的 wavefront 当时在等待什么;仅 stochastic sampling 会记录此项
要查看该 kernel 的热点指令,请导出带注释的反汇编(步骤 5),然后打开该 kernel 的 ISA CSV。
较高的 sample count 表明指令是热点,并不表示它有缺陷。循环体本来就应该是热点。真正值得关注的是:某个热点指令的 stall reason 表明它正在等待。例如,load 上的 memory stall 意味着 kernel 在此处受延迟限制(latency-bound),通常可通过 coalescing、prefetching,或在 load 与其使用位置之间安排更多独立工作来修复。
各 stall reason 及其含义列于 pc_sampling.rst。不要根据 host-trap 数据解读 stall reason;这种数据不会记录 stall reason。

5. Export annotated disassembly

5. 导出带注释的反汇编

bash
rocprof-compute analyze --path ./workloads/<name>/<gpu_model> \
    --output-format csv --output-name pc_report
CSV output writes per-kernel annotated disassembly into a
per_kernel_pc_sampling/
directory, with sample counts and source lines beside each instruction.
--output-format db
stores the same content in the analysis database. Both disable terminal output.
CSV output also writes
<file>_pc_sampling.csv
beside each captured source file, with the sample counts of every source line and the ISA rows each line compiled to. Start there to find the hottest source line.
bash
rocprof-compute analyze --path ./workloads/<name>/<gpu_model> \
    --output-format csv --output-name pc_report
CSV 输出会将每个 kernel 的带注释反汇编写入
per_kernel_pc_sampling/
目录,并在每条指令旁显示 sample count 和源代码行。
--output-format db
会将相同内容存储在分析数据库中。两种模式都会禁用终端输出。
CSV 输出还会在每个被捕获的源文件旁写入
<file>_pc_sampling.csv
,其中包含每个源代码行的 sample count,以及该行编译生成的 ISA 行。可以先从该文件着手查找最热点的源代码行。

6. When there are too few samples

6. 样本过少时

A short kernel may not accumulate enough samples to be meaningful. Increase the work the kernel does, or shorten the interval within the range the device accepts, rather than drawing conclusions from a handful of samples.
For multi-process workloads, the PC sampling table combines each kernel's samples from all processes, so use it only to pick the kernel. The per-kernel ISA files keep each process and code object separate; read the instructions there. See the multi-process sections of pc_sampling.rst.
运行时间较短的 kernel 可能无法累积足够的样本,导致结果没有统计意义。应增加 kernel 执行的工作量,或在设备接受的有效范围内缩短间隔,而不要仅凭少量样本得出结论。
对于多进程 workload,PC sampling 表格会合并来自所有进程的每个 kernel 的样本,因此只能使用该表选定 kernel。每个 kernel 的 ISA 文件会将各进程和 code object 分开保存;请在这些文件中读取相关指令。有关多进程的更多信息,请参阅 pc_sampling.rst的多进程相关章节。