Module 7 — ROCm kernel libraries

1 week · Part of Kernel Ladder · Prev: module-6-quantization-moe · Next: module-8-shipping

What this module is for

Production teams rarely hand-write every kernel. Knowing which library to reach for, and how to extend it with a custom epilogue or a new shape, is most of the day-to-day job. You’ll rebuild earlier kernels in two libraries and compare them, with numbers, against your hand-written HIP.

LibraryWhat it isWhen you reach for it
Composable Kernel (CK / CK-Tile)C++ templates for GEMM, attention and fused operatorsProduction GEMM and attention variants with custom epilogues
Triton on ROCmBlock-level Python DSL compiled to AMDGPUFast iteration on fused kernels
AITERA library of tuned inference kernels (assembly, CK and Triton) used by serving enginesReading state-of-the-art Instinct kernels; contribution target
hipBLASLt / rocBLASVendor GEMM librariesThe baseline for every GEMM benchmark
hipCUB / rocPRIMBlock, warp and device-wide primitivesReductions, scans and sorts inside your own kernels
rocWMMAC++ wrapper over WMMA and MFMAMatrix-core code portable across RDNA and CDNA

Labs

  1. Rebuild your module-4-gemm kernel in CK-Tile for MI300X with a custom epilogue (bias + GELU + FP8 quantize), and compare it with your hand-written step 9.
  2. Write fused RMSNorm and a GEMM in Triton on the 9070 XT, with autotuning. Compare lines of code and speed against your HIP versions.
  3. Read one AITER kernel end to end — an MoE or attention kernel. Write one page explaining its tiling, pipelining and instruction scheduling.
  4. Replace the reduction in your softmax with hipcub::BlockReduce and confirm no regression.

Done when

Your repo has a decision guide — hand-written HIP vs CK vs Triton vs rocWMMA — where every recommendation is backed by one of your own benchmark numbers.

Check your understanding

  • When would you choose Triton over CK for a new fused operator, and when would that choice hurt you?