Loading...
Loading...
Compare original and translation side by side
bash scripts/lock-gpu-clock.sh # before profiling
bash scripts/reset-gpu-clock.sh # after donebash scripts/lock-gpu-clock.sh # 分析前执行
bash scripts/reset-gpu-clock.sh # 分析完成后执行params.jsonsrc/mla_var3/conf/devices.jsondocs/devices/ridge_point = peak_tflops / peak_gbpsparams.jsonsrc/mla_var3/conf/devices.jsondocs/devices/ridge_point = peak_tflops / peak_gbps| Mode | | When to use | Output root |
|---|---|---|---|
| Annotation | | Default first pass. Roofline + NCU metrics + comparison tables | |
| Event | | Quick iteration timing (carries Python overhead) | |
| NCU | | Deep investigation: full NCU sections, source annotations, optimization suggestions | |
| NSYS | | Pipeline overlap, stream concurrency, launch ordering | |
annotationeventncunsysConcurrentKernels| 模式 | | 使用场景 | 输出根目录 |
|---|---|---|---|
| Annotation | | 默认首次分析。Roofline模型 + NCU指标 + 对比表格 | |
| Event | | 快速迭代计时(带有Python开销) | |
| NCU | | 深度调研:完整NCU章节、源码注解、优化建议 | |
| NSYS | | 流水线重叠、流并发、启动顺序分析 | |
annotationeventncuConcurrentKernelsnsyspython -m mla_var3.kernel <kernel_package> [<version>] \
--b=32 --s=16 --t=4096 --prof_type=<mode>python -m mla_var3.kernel <kernel_package> [<version>] \
--b=32 --s=16 --t=4096 --prof_type=<mode>undefinedundefinedundefinedundefinedNotice that version is optional, and allows you to specify a custom version. This retrieves the right `KernelPlan` of the specified version, and uses that, NOT the `plan` passed to the function. This is just a convenience to automatize version profiling.
Alternatively, you can directly build the plan object using the class of the version to be profiled, and ignore the version argument.
**Important**: sometimes, after auto-tuning, there might be some errors. In this case, you can simply re-run the command, the best autotune config is already cached and everything should work.注意,版本参数是可选的,允许您指定自定义版本。这会检索指定版本对应的`KernelPlan`并使用它,而非传入函数的`plan`对象。这只是为了自动化版本分析提供的便利。
或者,您可以直接使用待分析版本的类构建plan对象,并忽略version参数。
**重要提示**:有时自动调优后可能会出现一些错误。这种情况下,您只需重新运行命令即可,最佳自动调优配置已被缓存,一切都能正常工作。tests/benchmark/python -m tests.benchmark.bench_mla_var6_plus # base version
python -m tests.benchmark.bench_mla_var6_plus_v4 # specific version
python -m tests.benchmark.bench_all_mla # compare all kernels (slow)tests/benchmark/python -m tests.benchmark.bench_mla_var6_plus # 基础版本
python -m tests.benchmark.bench_mla_var6_plus_v4 # 指定版本
python -m tests.benchmark.bench_all_mla # 对比所有内核(速度较慢)out/profiles/<mode>/<kernel>/<params>/<timestamp>/out/profiles/<mode>/<kernel>/<params>/<timestamp>/params.jsonreport.mdtiling/<stage>.jsontiling/<stage>/autotuning.json.cache/kernel-autotune/...compiled/<stage>/params.jsonreport.mdtiling/<stage>.jsontiling/<stage>/autotuning.json.cache/kernel-autotune/...compiled/<stage>/| Metric | Healthy | If unhealthy → Issue | Optimization hint |
|---|---|---|---|
| TC Util | >60% | Memory or latency bound | Check DRAM%, occupancy |
| DRAM Throughput | >70% | Compute or latency bound | Check TC%, occupancy |
| Achieved Occupancy | >25% | Register/smem pressure | Reduce tile size, occupancy hint |
| L2 Hit Rate | >80% | Poor data reuse | Swizzle, larger tiles, data layout |
| Local Spilling | 0 bytes | Register overflow | Smaller tiles, fewer accumulators |
| Waves/SM | >1.0 | Underfilled GPU | More blocks, reduce per-block resources |
| 指标 | 健康状态 | 异常时→问题 | 优化建议 |
|---|---|---|---|
| TC利用率 | >60% | 内存或延迟受限 | 检查DRAM占比、 occupancy |
| DRAM吞吐量 | >70% | 计算或延迟受限 | 检查TC占比、 occupancy |
| 实际Occupancy | >25% | 寄存器/共享内存压力 | 减小分片尺寸、 occupancy优化提示 |
| L2命中率 | >80% | 数据复用性差 | 内存混洗、更大分片、数据布局调整 |
| 本地溢出 | 0字节 | 寄存器溢出 | 更小分片、减少累加器数量 |
| 每SM波数 | >1.0 | GPU未被充分利用 | 更多块、减少每块资源占用 |
| Pattern | Classification | Focus |
|---|---|---|
| DRAM% high + TC% low | Memory-bound | Data reuse, TMA hints, head grouping |
| TC% high + DRAM% low | Compute-bound | Kernel efficiency, tile sizes |
| Both low | Latency-bound | Occupancy, reduce spilling, more blocks |
| L2 hit < 80% | Locality issue | Swizzle scheduling, tile size adjustment |
| 模式 | 分类 | 优化重点 |
|---|---|---|
| DRAM占比高 + TC占比低 | 内存受限 | 数据复用、TMA提示、头部分组 |
| TC占比高 + DRAM占比低 | 计算受限 | 内核效率、分片尺寸 |
| 两者都低 | 延迟受限 | Occupancy、减少溢出、更多块 |
| L2命中率 < 80% | 局部性问题 | 混洗调度、分片尺寸调整 |
params.jsonundefinedparams.jsonundefined| b | s | t | dtype |
|---|---|---|---|
| X | X | X | bfloat16 |
| b | s | t | 数据类型 |
|---|---|---|---|
| X | X | X | bfloat16 |
| Stage | Duration (us) | TC% | DRAM% | Occ% | Bottleneck | Key Issue |
|---|
| 阶段 | 耗时(微秒) | TC% | DRAM% | Occ% | 瓶颈类型 | 核心问题 |
|---|
| Metric | Previous | Current | Change |
|---|
undefined| 指标 | 之前值 | 当前值 | 变化 |
|---|
undefineddocs/kernels/mla-var6-plus.md**Performance** (<device>, locked clocks if applicable, bfloat16, b=X, s=X, t=X):
| Metric | Value | vs Previous |
| ----------------- | ------- | ----------- |
| Duration | X.XX μs | Y% faster |
| Achieved TFLOPs/s | X.XX | +Z% |
| Achieved GB/s | X.XX | +Z% |
| Occupancy | XX% | -- |
| TC Util | XX% | -- |
**Bottleneck**: [Memory-bound / Compute-bound / Latency-bound]
**Issues**:
- [Remaining problems]
**Insights**:
- [Key lessons — why optimization worked or didn't]
- [Guidance for next iteration]docs/kernels/mla-var6-plus.md**性能**(<设备名称>,如已锁定时钟,bfloat16,b=X,s=X,t=X):
| 指标 | 数值 | 与之前版本对比 |
| ----------------- | ------- | ----------- |
| 耗时 | X.XX μs | 快Y% |
| 实际TFLOPs/s | X.XX | +Z% |
| 实际GB/s | X.XX | +Z% |
| Occupancy | XX% | -- |
| TC利用率 | XX% | -- |
**瓶颈**: [内存受限 / 计算受限 / 延迟受限]
**现存问题**:
- [剩余问题]
**洞察**:
- [关键经验——优化生效或失效的原因]
- [下一迭代的指导建议]