moat-cuda-to-rocm

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

CUDA to ROCm/HIP

CUDA 到 ROCm/HIP

Accumulated across many real ports. This file is the index and the decision-critical parts; everything else is in
references/
, loaded only when you need it.
Most porting bugs are not in symbol names. Hipify handles the renaming. The bugs are semantic differences where AMD is strict and CUDA is lenient, and most surface at runtime on a build that compiled cleanly, though some, like a 32-bit
*_sync
mask, fail to compile even after hipify runs clean. Read the fault-class index below before concluding a port works.
这是多次真实移植中积累下来的经验。本文件是索引,也是决策关键部分;其余内容都在
references/
中,只在需要时加载。
大多数移植 bug 并不在符号名称中。 Hipify 会处理重命名。真正的 bug 是语义差异:AMD 更严格,而 CUDA 更宽松。大多数问题会在编译干净的构建中于运行时暴露,不过有些问题,例如 32 位
*_sync
掩码,即使 hipify 成功运行后也会编译失败。在断定移植成功之前,先阅读下面的故障类别索引。

Naming: ROCm or HIP?

命名:ROCm 还是 HIP?

They name different things and reviewers notice.
  • HIP is the programming model: the kernel dialect plus the
    hipXxx
    runtime API. The analogue of CUDA C++ and the CUDA runtime.
  • ROCm is the platform: compiler, runtime, driver, and the roc*/hip* libraries. The analogue of the CUDA Toolkit.
So the CODE port is "to HIP" (hipify, the compat header, runtime symbols), while the TARGET, build flag and libraries are "ROCm" (
USE_HIP
/
USE_ROCM
, "ROCm 7.2.1", cuFFT -> hipFFT). A language+runtime port is precisely "a HIP port targeting ROCm". Never call the platform "HIP" or the kernel dialect "ROCm". Details:
references/naming.md
.
二者指的是不同事物,审阅者会注意到这一点。
  • HIP 是编程模型:包括 kernel 方言以及
    hipXxx
    runtime API,相当于 CUDA C++ 和 CUDA runtime。
  • ROCm 是平台:包括编译器、runtime、driver 以及 roc*/hip* 库,相当于 CUDA Toolkit。
因此,代码移植是“移植到 HIP”(hipify、兼容头文件、runtime 符号),而目标平台、构建标志和库则属于“ROCm”(
USE_HIP
/
USE_ROCM
、“ROCm 7.2.1”、cuFFT -> hipFFT)。语言和 runtime 的移植准确表述是“以 ROCm 为目标的 HIP 移植”。绝不要把平台称为“HIP”,也不要把 kernel 方言称为“ROCm”。详情参见
references/naming.md
。

Which strategy

选择哪种策略

Classify the build first -- implementing the wrong strategy correctly is still wrong.
buildstrategyshape
pure CMakeA (preferred)one
cuda_to_hip.h
compat header,
enable_language(HIP)
,
set_source_files_properties(... LANGUAGE HIP)
; sources keep CUDA spelling
pytorch extensionBrely on torch's build-time hipify; fix only what it cannot
anything elseneitherdriver-API, runtime PTX, Go/cgo, qmake and codegen builds exist -- see the runtime-PTX fault class
How to tell: look for
find_package(Torch)
,
torch.utils.cpp_extension
,
CUDAExtension
, or a torch dependency in
setup.py
/
pyproject.toml
. If any is present it is a pytorch extension -- Strategy B. Otherwise treat it as a pure CMake (or Makefile, or meson) project -- Strategy A. The build system does not pick the strategy; what the sources look like does. lc0 is meson and ports Strategy-A-shaped -- it just means writing by hand what
enable_language(HIP)
would have given you, and
references/strategy-a-cmake.md
says what.
Strategy A is the minimal-footprint model: the compat header is a no-op on NVIDIA, so the CUDA build is untouched. Where the project includes CUDA headers by name, prefer the shim-header method -- shadow headers on the HIP include path only -- which leaves the source tree byte-identical to upstream and produces a far more mergeable diff. Full recipes:
references/strategy-a-cmake.md
,
references/strategy-b-torch.md
.
首先对构建进行分类——正确实现错误的策略,结果仍然是错误的。
构建类型策略形态
纯 CMakeA(首选)一个
cuda_to_hip.h
兼容头文件、
enable_language(HIP)
、
set_source_files_properties(... LANGUAGE HIP)
;源代码保留 CUDA 写法
PyTorch extensionB依赖 torch 在构建时执行 hipify;只修复它无法处理的部分
其他类型两者均不适用存在 driver-API、runtime PTX、Go/cgo、qmake 和 codegen 构建——参见 runtime-PTX 故障类别
**判断方法:**查找
find_package(Torch)
、
torch.utils.cpp_extension
、
CUDAExtension
,或者
setup.py
/
pyproject.toml
中对 torch 的依赖。如果存在其中任何一项,它就是 PyTorch extension,即 Strategy B。否则,将其视为纯 CMake(或 Makefile、meson)项目,即 Strategy A。决定策略的不是构建系统,而是源码的形态。lc0 使用 meson,其移植采用 Strategy A 的形态——只不过需要手工实现
enable_language(HIP)
本可提供的结构;
references/strategy-a-cmake.md
中对此有说明。
Strategy A 是 footprint 最小化的模型:兼容头文件在 NVIDIA 上为空操作,因此不会影响 CUDA 构建。如果项目按名称包含 CUDA 头文件,首选 shim-header 方法——仅在 HIP include path 上使用 shadow header——这样可以让源码树与上游保持逐字节一致,并生成更易于合并的 diff。完整方案参见
references/strategy-a-cmake.md
、
references/strategy-b-torch.md
。

Before porting

开始移植之前

Check whether an AMD port already exists, and whether it is authoritative. The deciding axis is authoritativeness, not existence -- an AMD-official work-in-progress shifts the value to validating and improving it, while a one-off community fork is a hint, never code to inherit. Cheapest high-signal check first: grep the upstream's own docs for
amd|rocm|hip|gfx[0-9]
, because reference repos routinely link platform forks.
references/assess-existing-support.md
.
Some CUDA has no HIP analogue to translate INTO, so hipify succeeds at doing nothing and the port looks small. CUTLASS reimplements against Composable Kernel; CuTe has a real structural correspondence in AMD's FlyDSL, whose layout algebra is modelled on it. That is a reimplementation with a defined target, not the same verdict as "no target exists" -- and both get written off as "not portable" unless you say which.
references/no-hip-equivalent.md
.
检查是否已经存在 AMD 移植,以及它是否具有权威性。判断依据是权威性,而不是是否存在——AMD 官方正在进行的工作会提高该移植的价值,使工作重点转向验证和改进;一次性的社区 fork 只能作为线索,绝不能直接继承其代码。首先进行成本最低、信号最强的检查:在上游自己的文档中 grep
amd|rocm|hip|gfx[0-9]
,因为参考仓库经常链接平台 fork。参见
references/assess-existing-support.md
。
有些 CUDA 实现没有可翻译为目标的 HIP 对应物,因此 hipify 会成功完成转换,却实际上什么也没转换,移植看起来也很小。CUTLASS 是基于 Composable Kernel 重新实现的;CuTe 在 AMD 的 FlyDSL 中存在真实的结构对应关系,后者的 layout algebra 以其为模型。这属于面向明确目标的重新实现,与“没有目标存在”不是同一个结论——如果不说明具体是哪一种情况,这两者往往都会被归结为“不可移植”。参见
references/no-hip-equivalent.md
。

Fault classes -- the index

故障类别——索引

Scan this list against what you are doing. If any line could apply, open
references/fault-classes.md
for the full entry with evidence and fix.
Wavefront and warp semantics
  • Warp size: NVIDIA is always 32; AMD is 64 on CDNA (gfx9xx) and 32 on RDNA. Never hardcode.
  • Warp-derived array sizing: quantities scaling WITH
    warpSize
    need the 64 upper bound, quantities scaling as
    blockDim/warpSize
    need the 32 LOWER bound. Getting it backwards writes out of bounds.
  • HIP's
    *_sync
    warp functions reject a 32-bit mask at compile time; CUDA's
    0xffffffff
    needs a 64-bit spelling on HIP.
  • A compat
    __ballot_sync
    that casts
    __ballot()
    to
    uint32_t
    returns the wrong 32 lanes on wave64.
  • An over-wide
    __shfl*
    width silently clamps to the physical wavefront instead of erroring; the symptom is a metric shift, not a crash.
  • A full 64-bit
    *_sync
    mask in 32-lane group code traps in a HIP debug build on wave64 when the halves diverge; pass the group mask.
  • Intra-wave barrier divergence: a per-row early return before
    __syncthreads()
    is benign on CUDA, faults on wave64.
  • cub
    /
    hipCUB
    block-collective
    TempStorage
    reuse races on a 64-thread block without an explicit
    __syncthreads
    .
  • __smid()
    can EXCEED
    multiProcessorCount
    on AMD, unlike NVIDIA.
  • HIP's device
    warpSize
    is not an
    int
    ; cast it at any
    printf
    vararg.
Memory and lifetime
  • Out-of-bounds reads: CUDA often tolerates one element past an allocation; AMD faults.
  • Fresh device allocations are NOT zeroed on ROCm, where CUDA's allocator often hands back zeroed pages. Whenever you port or review code, trace every allocation whose first use is a read-modify-write -- an
    atomicAdd
    counter, an accumulator, histogram bins, a
    +=
    output -- and confirm a memset or init kernel runs first on the same stream. Report a missing one even when the rest of the review is about something else.
  • cudaMemcpy*Async
    from a soon-freed pageable host buffer: CUDA stages synchronously, HIP does not. The same hides an unsynchronized upload on one stream feeding a kernel on another.
  • Bitwise/min/max atomics on
    hipMallocManaged
    memory are silently dropped on discrete xnack- cards (PCIe AtomicOps); whether a build is hit depends on optimization level, so check the app's ISA.
  • Rule-of-five on resource handles: CUDA tolerates double-destroy of a texture/stream/event; AMD does not.
  • hipEventQuery
    /
    hipEventSynchronize
    dereference the event's recording stream: querying an event after that stream is destroyed (legal on CUDA) hangs or crashes intermittently. Keep destroyed streams alive: route stream create/destroy through a sync-park-reuse pool in the compat header, not per-holder sweeps.
  • A functor returning a reference to a by-value parameter (lifetime UB) is tolerated by nvcc, not by clang/HIP.
  • hipLaunchHostFunc
    callback threads must not call ANY runtime API.
  • A
    hipLaunchHostFunc
    callback must not block either: ROCr runs it on the async-events thread, so a callback waiting on a host gate stalls every later event on the device (a multi-stream test hung on gfx1100).
  • HIP after
    fork()
    crashes in the child (segfault in hipEventQuery); a test that forks and then touches the GPU fails on ROCm.
  • A query HIP reports as unsupported (
    cudaLimitPrintfFifoSize
    -> hipErrorUnsupportedLimit) leaves a sticky last error that the NEXT launch check reports; clear it under the HIP branch.
Textures
  • Texture pitch alignment: AMD requires 256-byte row pitch for pitched 2D binds (32 on NVIDIA).
  • Layered
    cudaArray
    collapses across kernel launches -- use a non-layered 3D array (confirmed gfx90a bug).
  • Hardware linear-filter texture over an element-read float array is arch-specific, not a blanket AMD limitation.
  • Gate hardware-vs-software texture paths on a VERIFIED runtime self-test, never on creation success or
    _WIN32
    .
Floating point
  • __fsqrt_rn
    is not always correctly rounded on gfx90a or gfx1100 (1 ULP high); CUDA's is IEEE-correct. Map it to
    sqrtf
    when a test compares against host sqrt.
  • Without
    __HIP_DEFINE_EXTENDED_HOST_MIN_MAX__
    HIP's HOST
    min
    /
    max
    take int, so
    min(float, float)
    in
    __host__ __device__
    code silently truncates on the host (whole-pixel camera errors); define it for the HIP build.
  • clang(HIP) defaults to
    -ffp-contract=fast
    and forms FMAs ACROSS expressions; nvcc contracts expression-only.
  • Exact float-equality branches fed by
    __fdividef
    produce out-of-range indices.
  • HIP device
    cuda::min
    /
    max
    NaN-selection can differ from host
    std::min
    /
    max
    .
  • HIP host
    min
    /
    max
    in
    .cu
    files exist only for
    (int, int)
    : float arguments silently truncate. Fix the call sites with
    fmin
    /
    fmax
    ; the macro is ROCm >= 7.2 only and defaults off.
Headers, includes and build
  • A shared compat header must be host-includable: gate device-only includes behind
    __CUDACC__
    /
    __HIPCC__
    , or CUB leaks into host TUs. Hit independently by two projects.
  • __HIP_PLATFORM_AMD__
    is undefined until
    hip_runtime.h
    is included; a gate placed before it silently picks the CUDA branch.
  • Missing includes in a HIP port are usually pre-existing upstream omissions unmasked by the narrower include graph.
  • A
    -include
    d compat header creates no dependency edge: wipe objects after editing it or you validate stale code.
  • clang type-checks kernel bodies in the HIP host pass, where
    __CUDA_ARCH__
    is unset: a
    __CUDA_ARCH__
    -split debug macro calling host code fails only in Debug.
  • The CUDA Samples
    helper_math.h
    operators tie with
    HIP_vector_type
    's own: guard its arithmetic block off on HIP, wholesale.
Types, dispatch and platform limits
  • Never name the pointee struct of an opaque handle (
    CUstream_st
    ); use
    std::remove_pointer_t<cudaStream_t>
    .
  • CUPTI has no ROCm analogue; ck_tile fmha ships headers but no instance library (generate the instances; MIOpen's MHA has no FP16 path).
  • CUTLASS-style GEMMs map to CK device ops (XDL on CDNA, WMMA on RDNA, DL for RDNA FP32 with padded copies).
  • cuDF/cuVS/RMM: hipDF, hipVS, hipMM ship as C++ wheels on AMD's package index, with a few API gaps.
  • hipSPARSE breaks HIP graph capture where cuSPARSE does not (destroying a CSR SpMat syncs, default-algorithm analysis, size-0 buffers).
  • Pick a library's NVIDIA or AMD build per device, never by starting every backend; a HIP port that keeps the soname (hipVS, hipDF) allows one per process.
  • NVML -> AMD SMI, keyed by PCI address, not device index.
  • clock64()
    is a 20-bit wrapping counter on RDNA3; cycle-bounded spins and delays end early. Use
    wall_clock64()
    .
  • char
    vs
    signed char
    vector base types differ between HIP and CUDA.
  • AMD compute-capability values COLLIDE with NVIDIA arch numbers, so CC-keyed dispatch picks the wrong path.
  • Library swaps: cuBLAS -> hipBLAS, cuFFT -> hipFFT, cuRAND -> hipRAND, cuSPARSE -> hipSPARSE, cuDNN -> MIOpen, Thrust/CUB -> rocThrust/hipCUB. Handle types and v2-enum signatures differ.
  • nvJPEG/nvImageCodec -> rocJPEG for JPEG decode only (VCN; baseline SOF0 only: no SOF1 or progressive, nothing under 64x64, no encoder, no JPEG 2000); the rest needs a named CPU path, chosen from the frame header, not a catch-all. A JPEG with no scan hangs the VCN engine: validate markers on the host first. NVDEC -> FFmpeg demux + rocDecode's parser and decoder (no FFmpeg hwaccel); gate on profile as well as
    rocDecGetDecoderCaps
    , which has no profile and lets unsupported H.264 profiles decode wrong, and whose size limits are coded sizes (round the display size up to the block size); a success status can still carry wrong pixels on some GPU and VA driver pairs, so decode a reference clip per codec once per process before trusting it; the CPU path needs libdav1d for AV1. NVENC has no ROCm equivalent: name the software encoder path. See fault-classes "Vulkan interop and image codecs".
  • Runtime PTX + CUDA Driver API is a third build class beyond A and B.
  • MSVC-only upstreams accept code gcc/clang reject; the HIP build is a stricter compiler.
  • No graphics pipeline on compute-only CDNA: OpenGL/Vulkan-interop apps build but cannot run on gfx90a.
  • Vulkan timeline semaphores do not import into HIP on either OS, and the Windows OpaqueWin32 import can silently miss a wait: order on the host.
  • An arch-specific fix keyed on OS or ROCm version can BREAK an already-validated arch.
Two diagnostic methods in
references/validation.md
are worth knowing before you escalate a suspected AMD fault: a per-tile "later data corrupts earlier" signature usually means the reproducer's input dtype is wrong, and SP-vs-DP triangulation distinguishes FP reassociation from a genuine wavefront bug. Both were learned the expensive way.
对照下面的列表检查正在处理的工作。如果其中任何一项可能适用,请打开
references/fault-classes.md
查看包含证据和修复方案的完整条目。
Wavefront 和 warp 语义
  • Warp size:NVIDIA 始终为 32;AMD 在 CDNA(gfx9xx)上为 64,在 RDNA 上为 32。绝不能硬编码。
  • 根据 warp 推导的数组大小:随
    warpSize
    缩放的数量需要使用 64 作为上界,按
    blockDim/warpSize
    缩放的数量需要使用 32 作为下界。弄反会导致越界写入。
  • HIP 的
    *_sync
    warp 函数在编译期拒绝 32 位掩码;CUDA 的
    0xffffffff
    在 HIP 上需要使用 64 位写法。
  • 如果兼容层中的
    __ballot_sync
    将
    __ballot()
    强制转换为
    uint32_t
    ,那么在 wave64 上会返回错误的 32 个 lane。
  • 过宽的
    __shfl*
    width 会静默截断到物理 wavefront,而不是报错;其症状是指标偏移,而不是崩溃。
  • 在 32-lane group 代码中使用完整的 64 位
    *_sync
    掩码,当两半分歧时会在 wave64 的 HIP debug build 中触发 trap;应传入 group mask。
  • Wave 内 barrier 分歧:在
    __syncthreads()
    之前按行提前 return,在 CUDA 上通常无害,但在 wave64 上会触发故障。
  • 在 64 线程 block 中复用
    cub
    /
    hipCUB
    block-collective 的
    TempStorage
    时,如果没有显式的
    __syncthreads()
    ,会产生竞态。
  • 与 NVIDIA 不同,AMD 上的
    __smid()
    可能超过
    multiProcessorCount
    。
  • HIP 设备端的
    warpSize
    不是
    int
    ;在任意
    printf
    可变参数中都要显式转换。
内存和生命周期
  • 越界读取:CUDA 经常容忍读取分配区域外一个元素的操作;AMD 会触发故障。
  • ROCm 上新分配的设备内存不会自动清零,而 CUDA allocator 经常直接返回已经清零的页面。移植或审查代码时,应追踪每一次分配,只要其首次使用是 read-modify-write——例如
    atomicAdd
    计数器、累加器、直方图 bin、
    +=
    输出——就应确认同一 stream 上事先执行了 memset 或初始化 kernel。即使审查的重点在其他方面,也应报告缺失的初始化。
  • 使用即将释放的 pageable host buffer 调用
    cudaMemcpy*Async
    :CUDA 会同步暂存数据,HIP 不会。相同问题还会隐藏一个 stream 上未同步的上传操作向另一个 stream 上的 kernel 提供数据。
  • 在 discrete xnack- 卡上,对
    hipMallocManaged
    内存执行按位/min/max 原子操作时,操作会被静默丢弃(PCIe AtomicOps);是否受影响取决于应用的优化级别,因此应检查应用的 ISA。
  • 资源 handle 遵循 rule-of-five:CUDA 容忍重复销毁 texture/stream/event;AMD 不会。
  • hipEventQuery
    /
    hipEventSynchronize
    会解引用 event 的 recording stream:在该 stream 已销毁后查询 event,在 CUDA 上是合法的,但在 HIP 上可能间歇性挂起或崩溃。应让已销毁的 stream 继续存活:在兼容头文件中统一通过同步—停放—复用池管理 stream 的创建和销毁,而不是分别在各持有者中进行清理。
  • functor 返回按值参数的引用会造成生命周期 UB;nvcc 会容忍这种情况,而 clang/HIP 不会。
  • hipLaunchHostFunc
    的 callback 线程不得调用任何 runtime API。
  • hipLaunchHostFunc
    的 callback 也不得阻塞:ROCr 会在 async-events 线程上运行它,因此等待 host gate 的 callback 会阻塞设备上的所有后续 event(某个 multi-stream 测试曾在 gfx1100 上挂起)。
  • HIP 在
    fork()
    之后会令子进程崩溃(在
    hipEventQuery
    中发生段错误);fork 后再访问 GPU 的测试在 ROCm 上会失败。
  • 如果 HIP 报告某个查询不受支持(
    cudaLimitPrintfFifoSize
    ->
    hipErrorUnsupportedLimit
    ),它会留下一个 sticky last error,并在下一次 launch 检查时被报告;应在 HIP 分支中将其清除。
纹理
  • 纹理 pitch 对齐:AMD 的 pitched 2D 绑定要求 256 字节的行 pitch,而 NVIDIA 为 32 字节。
  • 分层
    cudaArray
    会在多次 kernel launch 之间发生折叠——应使用非分层的 3D array(已在 gfx90a 上确认存在此 bug)。
  • 在以元素方式读取 float 数组时使用硬件线性过滤纹理,其行为取决于具体架构,不能笼统地视为 AMD 限制。
  • 应通过经过验证的运行时自检来选择硬件或软件纹理路径,绝不能依据创建是否成功或
    _WIN32
    。
浮点数
  • 在 gfx90a 或 gfx1100 上,
    __fsqrt_rn
    并不总能正确舍入(可能高出 1 ULP);CUDA 的实现符合 IEEE 规范。当测试与宿主机的 sqrt 比较时,应将其映射为
    sqrtf
    。
  • 如果未定义
    __HIP_DEFINE_EXTENDED_HOST_MIN_MAX__
    ,HIP 的 HOST
    min
    /
    max
    会接受 int,因此
    __host__ __device__
    代码中的
    min(float, float)
    会在主机端静默截断(导致相机结果出现整像素误差);应为 HIP build 定义该宏。
  • clang(HIP) 默认启用
    -ffp-contract=fast
    ,并跨表达式形成 FMA;nvcc 只会对单个表达式进行合并。
  • 使用
    __fdividef
    结果作为输入的精确浮点相等分支会产生越界索引。
  • HIP 设备端
    cuda::min
    /
    max
    对 NaN 的选择行为可能不同于主机端
    std::min
    /
    max
    。
  • .cu
    文件中的 HIP host
    min
    /
    max
    仅支持
    (int, int)
    :float 参数会被静默截断。应使用
    fmin
    /
    fmax
    修复调用点;该宏仅适用于 ROCm >= 7.2,并且默认关闭。
头文件、include 和构建
  • 共享兼容头文件必须能够被主机端 include:应使用
    __CUDACC__
    /
    __HIPCC__
    守卫仅限设备端的 include,否则 CUB 会泄漏到 host TU 中。两个项目都曾独立遇到此问题。
  • 在包含
    hip_runtime.h
    之前,
    __HIP_PLATFORM_AMD__
    尚未定义;如果守卫放在它之前,就会静默选择 CUDA 分支。
  • HIP 移植中缺少 include,通常是上游原有的遗漏,只是由于更窄的 include 依赖关系而被暴露出来。
  • 通过
    -include
    引入的兼容头文件不会创建依赖关系:修改它后应清理目标文件,否则验证的是过期代码。
  • clang 会在 HIP host pass 中对 kernel body 进行类型检查,此时
    __CUDA_ARCH__
    尚未设置:使用
    __CUDA_ARCH__
    分隔的调试宏如果调用 host code,就只会在 Debug 构建中失败。
  • CUDA Samples 的
    helper_math.h
    运算符会与
    HIP_vector_type
    自带的运算符产生冲突:在 HIP 上应整体禁用其算术代码块。
类型、分派和平台限制
  • 绝不能命名 opaque handle 所指向的结构体(
    CUstream_st
    );应使用
    std::remove_pointer_t<cudaStream_t>
    。
  • CUPTI 没有 ROCm 对应物;ck_tile fmha 提供了头文件,但没有 instance library(需要自行生成 instances;MIOpen 的 MHA 没有 FP16 路径)。
  • CUTLASS 风格的 GEMM 应映射到 CK device ops(CDNA 上使用 XDL,RDNA 上使用 WMMA,RDNA FP32 使用 DL 并配合 padded copies)。
  • cuDF/cuVS/RMM:hipDF、hipVS、hipMM 以 C++ wheel 的形式发布在 AMD package index 中,但存在少量 API 缺口。
  • 在 cuSPARSE 不会破坏 HIP graph capture 的场景中,hipSPARSE 可能会破坏它(销毁 CSR SpMat 会触发同步、默认算法分析和 size-0 buffer)。
  • 应按设备选择库的 NVIDIA 或 AMD 构建版本,绝不能通过同时启动所有后端来实现;保留 soname(hipVS、hipDF)的 HIP 移植允许每个进程只使用一个版本。
  • NVML -> AMD SMI,应以 PCI address 为键,而不是 device index。
  • clock64()
    在 RDNA3 上是 20 位回绕计数器;以周期数为界的自旋和延迟会提前结束。应使用
    wall_clock64()
    。
  • HIP 与 CUDA 的 vector base type 在
    char
    与
    signed char
    之间存在差异。
  • AMD compute-capability 数值与 NVIDIA 架构编号会冲突,因此基于 CC 的分派可能选择错误路径。
  • 库替换:cuBLAS -> hipBLAS,cuFFT -> hipFFT,cuRAND -> hipRAND,cuSPARSE -> hipSPARSE,cuDNN -> MIOpen,Thrust/CUB -> rocThrust/hipCUB。handle 类型和 v2 enum 签名存在差异。
  • nvJPEG/nvImageCodec -> rocJPEG 仅支持 JPEG 解码(VCN;仅支持 baseline SOF0:不支持 SOF1 或 progressive、不支持小于 64x64 的图像、不支持 encoder、不支持 JPEG 2000);其余格式需要根据 frame header 选择明确的 CPU 路径,而不能使用兜底方案。没有 scan 的 JPEG 会使 VCN engine 挂起:应先在主机端验证 marker。NVDEC -> FFmpeg demux + rocDecode 的 parser 和 decoder(不使用 FFmpeg hwaccel);除
    rocDecGetDecoderCaps
    外还应按 profile 进行门控,因为 caps 不包含 profile,可能让不受支持的 H.264 profile 被错误解码,而且其尺寸限制针对的是 coded size(应将 display size 向上取整到 block size);在某些 GPU 与 VA driver 组合上,即使返回成功状态也可能产生错误像素,因此每个进程、每种 codec 都应先解码一次参考 clip,再信任该路径;CPU 路径处理 AV1 时需要 libdav1d。NVENC 没有 ROCm 等价实现:应明确指定软件 encoder 路径。参见故障类别中的“Vulkan interop 和 image codecs”。
  • Runtime PTX + CUDA Driver API 是 A 和 B 之外的第三种构建类别。
  • 仅支持 MSVC 的上游代码可能接受 gcc/clang 拒绝的写法;HIP build 使用的是更严格的编译器。
  • compute-only CDNA 没有 graphics pipeline:OpenGL/Vulkan-interop 应用可以构建,但无法在 gfx90a 上运行。
  • Vulkan timeline semaphores 在任一 OS 上都无法导入 HIP;Windows 的 OpaqueWin32 import 还可能静默漏掉一次等待,需要在主机端进行排序。
  • 根据 OS 或 ROCm version 进行分支的架构专用修复,可能破坏已经验证通过的架构。
在升级处理疑似 AMD 故障之前,值得了解
references/validation.md
中的两种诊断方法:按 tile 出现的“后续数据会破坏更早数据”特征,通常意味着复现代码的输入 dtype 有误;SP 与 DP 对比三角测量可以区分 FP reassociation 与真正的 wavefront bug。这些经验都是通过付出高昂代价换来的。

Portability rule for shared code

共享代码的可移植性规则

Any fix to code that is not arch-guarded must be correct on both wavefront sizes. A per-arch hack that fixes wave32 and regresses wave64 makes the archs ping-pong, and each "fix" then costs a full re-test on every architecture. Prefer a runtime
warpSize
query on the host and a compile-time constant in device code.
对未受架构守卫的代码所做的任何修复,都必须同时适用于 两种 wavefront size。只修复 wave32 却让 wave64 退化的架构专用补丁,会使各架构之间反复打补丁;每增加一个“修复”,都意味着要在每个架构上重新进行完整测试。优先在 host 端运行时查询
warpSize
,并在 device code 中使用编译期常量。

Validation bar

验证标准

A real-GPU test run is the only evidence that counts. Lint is not validation, and a CPU-only build proves compilation and nothing else. Cover both wavefront widths and each OS the project supports: at least one wave64 (CDNA) GPU, one wave32 (RDNA) GPU, and Windows when the project builds there. Also compile the CUDA path with nvcc as a no-regression gate: an additive ROCm port must leave the CUDA build a pure passthrough.
references/validation.md
.
只有真实 GPU 上的测试运行才算有效证据。Lint 不属于验证,仅使用 CPU 构建只能证明代码能够编译。必须覆盖两种 wavefront width,以及项目支持的每一种 OS:至少包含一台 wave64(CDNA)GPU、一台 wave32(RDNA)GPU;如果项目支持 Windows,也必须覆盖 Windows。还应使用 nvcc 编译 CUDA 路径,将其作为无回归门禁:增量式 ROCm 移植必须使 CUDA 构建保持纯直通,不改变原有行为。参见
references/validation.md
。

Keeping an open PR mergeable

保持开放的 PR 可合并

Upstream moves while a PR is in review. Sync with a MERGE of upstream's default branch, never a rebase on a published branch, and treat the clean auto-merge as the risk: upstream writes new code against the surfaces the port multiplexed (per-vendor backends, platform predicates), so it lands the NVIDIA half only.
references/upstream-sync.md
.
PR 在审阅期间,上游仍会继续变化。应使用 MERGE 同步上游默认分支,绝不要对已发布分支执行 rebase;同时应将干净的 auto-merge 视为风险:上游可能基于移植所复用的接口编写新代码,例如按厂商划分的后端和平台谓词,从而只合入 NVIDIA 那一半。参见
references/upstream-sync.md
。

Writing it up

编写说明

Commit messages, PR bodies and code comments go to upstream maintainers who do not know your team's vocabulary. Before pushing, check the whole branch -- not just the commit you wrote last -- for in-house terms: internal stage or state names, tool names, and this skill's own labels such as "Strategy A" or "fault class". A term list with a replacement for each is worth keeping and running mechanically.
A term list does not catch a comment that explains a hazard by naming ANOTHER ported project as shorthand for it. arrayfire carried eleven such comments; the six saying "fault class" were caught by a term check and five saying "the amgcl void*-aliasing fix" or "the gsplat target-attribute rule" were not. To an upstream reader both are dangling references to a codebase they have never seen. State the hazard itself -- "hipSPARSE typedefs several descriptor types to the same void*, so a wrapper keyed on the handle type cannot separate them" -- and before pushing, grep new comments for the names of other projects you have ported.
Commit message、PR body 和代码注释是写给不了解团队术语的上游维护者的。推送前应检查整个分支,而不只是最后一条 commit,查找内部术语:内部 stage 或 state 名称、工具名称,以及本 skill 自身的标签,例如“Strategy A”或“fault class”。维护一份术语表,并为每个术语提供替换说法,然后机械地运行检查。
术语表无法发现另一种注释问题:使用另一个已移植项目的名称作为某个 hazard 的简称。arrayfire 中有十一条这类注释;其中提到“fault class”的六条被术语检查捕获,而提到“the amgcl void*-aliasing fix”或“the gsplat target-attribute rule”的五条则没有。对上游读者而言,这两种写法都像是在引用一个他们从未见过的代码库。应直接说明 hazard 本身——“hipSPARSE 将多种描述符类型 typedef 为同一个 void*,因此按 handle 类型区分的 wrapper 无法将它们分开”——并且在推送前,grep 新注释中是否出现了其他已移植项目的名称。