Module 0 — Toolchain and benchmark harness
1 week (~5 hours) · Part of Kernel Ladder · Next: module-1-hip-model
What this module is for
Every later number in this course comes out of the harness you build here. A harness that forgets to warm up, times a cold cache, or skips the correctness check will produce confident, wrong numbers for twelve weeks. This module is short on purpose, but nothing after it is trustworthy until it is done.
0.1 Install ROCm on both cards
Goal
rocminfolists gfx1010 (RX 5700 XT) and gfx1201 (RX 9070 XT).
- Install a Linux release listed on the current ROCm system requirements page and install ROCm for the 9070 XT from the official instructions.
- For the 5700 XT, use your distribution’s ROCm packages (Debian 13 / Ubuntu 23.10+ build the math libraries for gfx1010) or the community build scripts. See hardware-specs for both routes.
- Add yourself to the
renderandvideogroups and reboot. - Run
rocminfo | grep gfxand confirm both targets appear.
Done when
rocminfolists both gfx targets, and a trivial HIP kernel compiled with--offload-arch=gfx1010,gfx1201runs on each card.
0.2 Create the repo
Create a public repository, for example rocm-kernels-from-scratch:
bench/ harness, reference implementations, tolerance table
m1-hip/ one folder per module
...
RESULTS.md every number, with GPU, gfx target, ROCm version, commit hashEvery row in RESULTS.md records the GPU, gfx target, ROCm version and commit. A number without those is not reproducible, even by you next month.
0.3 Build the harness
Goal
One Python entry point that checks correctness, then times a kernel properly.
The harness must:
- Check the output against a PyTorch or NumPy reference with explicit
atol/rtolper dtype (for example fp321e-5, bf161e-2, FP8 a cosine-similarity floor). Fail loudly; never time an incorrect kernel. - Warm up (at least 10 launches), then time 100 launches with
hipEventpairs and report the median and the spread. - Flush the L2 cache and Infinity Cache between timed runs by touching a buffer larger than both. Otherwise small tensors stay resident and the numbers flatter you.
- Report achieved GB/s and TFLOP/s next to the card’s peak, so every result is a percentage.
0.4 Build tooling
Write a Makefile (or CMake) target that:
- builds for several targets at once:
--offload-arch=gfx1010,gfx1201,gfx942,gfx950 - keeps the intermediate ISA with
--save-tempsso you can read the instructions the compiler emitted - prints register and LDS usage with
-Rpass-analysis=kernel-resource-usage - runs
rocprofv3on a kernel and saves the trace next to the source
Done when
One command builds and runs vector add on both Radeon cards and prints achieved GB/s as a percentage of each card’s peak memory bandwidth. Record both numbers in
RESULTS.md.
It's not working
hipErrorNoBinaryForGpu: you did not build for that card’s gfx target. Check--offload-arch.- Vector add reaches well under half of peak: check you are timing the kernel, not the host-to-device copy, and that you warmed up.
- The 5700 XT disappears after a ROCm upgrade: an official package update can replace the gfx1010-enabled distribution libraries. Pin the packages you rely on.
Check your understanding
- Why does flushing the cache matter more for a 1 MB tensor than a 1 GB one?
- Why report the median rather than the mean?