Choosing a Profiling Preset
Recommends one of rocprofiler-systems' built-in runtime profiling presets for a
described workload, and returns the exact command to run it.
Prerequisites
None to make a first recommendation from the table below. To confirm what a preset
sets (see "Confirm with
"), and to run the command,
or
must be installed or built.
Version check
Presets need ROCm Systems Profiler
1.6.0 or newer (ROCm 7.13.0). Check with
rocprof-sys-run --version
, whose first line reads
rocprof-sys-run v<major>.<minor>.<patch> ...
.
| Installed version | What to tell the user |
|---|
| 1.6.0 or newer | , and all work. |
| 1.5.x (ROCm 7.12.0) | Presets exist only as flags (for example ), with no , or . Give that form and suggest upgrading. |
| Older than 1.5.0 | No presets. Suggest upgrading, or configure settings by hand. |
A newer preset name may be missing from an older build, so
is the list
of what the installed version supports. If the version cannot be checked, give the
command and say it needs 1.6.0 or newer.
Terminology
"Trace" and "tracing" mean two different things here. Keep them apart in every answer.
| Term | Meaning | Setting |
|---|
| Perfetto trace output | Writes a timeline file in Perfetto format | |
| rocPD output | Writes a rocPD database | |
| Profile output | Writes timemory profile files (flat or call-stack based) | |
| Call-stack sampling | Samples the CPU call stack at a fixed frequency | |
| API tracing | Intercepts and records the calls of one API | see below |
API tracing is chosen per API: ROCm (
), MPI
(
), RCCL (
), OpenMP (
)
and Kokkos (
). ROCm API tracing is on in every built-in preset,
by default for the HIP runtime API, ROCTx markers, kernel dispatches, memory copies and
scratch memory. A preset may narrow or widen that set, and
overrides it.
API tracing is separate from output selection. A preset's
(or
) turns Perfetto trace output off, not API tracing. The collected
data goes to whichever outputs are on: Perfetto, rocPD or profile files. Never say a
preset "has tracing off" without saying which of these you mean.
Preset reference
A routing aid only: it says which preset fits which need, not exactly what each one sets.
The installed binary is the source of truth for that (see "Confirm with
").
| Preset | Category | Use case |
|---|
| general | Recommended default, moderate overhead |
| general | Lowest overhead flat profile (production) |
| general | Maximum depth, high overhead |
| gpu | GPU activity timeline (kernel dispatch, memory copies) |
| gpu | GPU hardware counters (occupancy, VALU utilization) |
| gpu | AI/ML training, long-running GPU/HPC workloads |
| hpc | Compute-intensive MPI/OpenMP/Kokkos/RCCL apps, with CPU counters (PAPI) |
| hpc | OpenMP GPU target-offload apps |
| hpc | MPI communication latency, flat profile, no Perfetto trace |
| tracing | Debugging runtime-layer interactions, widest ROCm API tracing (HIP, HSA, ROCTx, RCCL) |
| tracing | Same as without HSA/compiler APIs |
How to choose
Resolve these in order; the first question that narrows to a single preset wins. Do not
require the user to name a runtime such as MPI before a step applies.
-
API tracing or debugging. Questions about HIP, HSA or runtime-layer behavior (e.g.
"why is my HIP call behaving oddly", "what's HSA doing under the hood") →
(full ROCm API tracing), or
when HSA and compiler API
noise should be left out.
-
One named measurement.
- GPU hardware counters such as occupancy or VALU utilization → .
- MPI communication latency is the only concern, no Perfetto trace file needed →
.
- OpenMP GPU target-offload kernels → .
-
Long-running GPU work. AI/ML training, or a long-running GPU-accelerated HPC job
that needs MPI and RCCL API tracing and a large Perfetto trace buffer →
. A training job qualifies even if it never mentions MPI, and it wins
over step 4 when the prompt also names MPI or OpenMP.
-
HPC runtimes. A compute-intensive MPI, OpenMP, Kokkos or RCCL application →
. It writes a Perfetto trace, traces those runtimes' APIs and sets CPU
hardware-counter events (PAPI). If the user wants neither the trace file nor the
counters,
or
write no Perfetto trace.
-
GPU activity. Kernel dispatch or device activity (a GPU activity timeline), with
none of the goals above stated →
.
-
General-purpose profiling. Pick by overhead tolerance and depth:
- Lowest overhead, quick flat profile (e.g. production) → ; it turns
off Perfetto trace output, rocPD output and call-stack sampling, but ROCm API calls
and kernels still appear in the flat profile.
- Maximum depth, willing to accept high overhead → .
- Anything else, or a good default for general profiling (profile output, rocPD
output, call-stack sampling and GPU metrics, no Perfetto trace) → .
Confirm with
The presets are compiled into the installed binary, so
matches the user's
build exactly and the table above can lag behind it. Once you have a candidate:
- Run
rocprof-sys-run --explain=<name>
(or rocprof-sys-sample --explain=<name>
) as a
single command, with no check or chain in front of it. Use its
list as the facts about the preset. Quote the settings that matter to
the user's goal, and if they contradict the table or your first choice, follow
.
- If the command is not found or cannot run, use the table and say that the settings
are unconfirmed.
Output
Return the exact invocation. Default to
:
bash
rocprof-sys-run --preset=<name> -- ./myapp
Both tools take the same
, and a preset that sets
overrides
either one:
samples at 50 Hz on both, and
leaves call-stack
sampling off on both. The tools differ only when the preset says nothing about sampling
(for example
):
turns call-stack sampling on and
leaves it off. So use
when the user wants call-stack
sampling and the preset does not already enable it:
bash
rocprof-sys-sample --preset=<name> -- ./myapp
State in one line what the recommended preset turns on and what it leaves off, using the
terms above, so the user knows what they gain and give up. For example,
writes a flat profile with no Perfetto trace file, no rocPD database and no call-stack
sampling, while ROCm API calls and kernels still appear in the profile.
Mention, if relevant:
- Domain flags (, , , ) can layer on top of
any preset to add/override specific metrics, e.g.
rocprof-sys-run --preset=balanced --gpu=temp,power -- ./myapp
. Use this when the
user's need doesn't fit a preset neatly: start from and extend it.
- and let the user verify the choice themselves.
--export-config=<file>.json
freezes the resolved preset+overrides into a
reusable JSON config — this is the supported way to customize a preset, not
hand-editing the shipped JSON files.
- The file format for custom presets is the JSON schema installed at
<install-prefix>/share/rocprofiler-systems/presets/schema.json
.
Common Mistakes
| Mistake | Fix |
|---|
Confusing this with the CMake --preset debug|release|ci
used to build rocprofiler-systems | Build presets configure compilation; profiling presets configure a profiling run. This skill only covers the latter. |
| Looking for the built-in preset JSON files to edit | The built-in presets are compiled into the binary and are not installed as files. Use to inspect one and to produce an editable copy. |
| Picking by default for "just get me some data" | Default to unless the user explicitly wants maximum depth and can tolerate higher overhead. |