The GPU MODE track

Runs alongside every module · Part of Kernel Ladder

What this track is for

GPU MODE is a practice arena, an independent benchmark referee and a peer group in one place. Every module ends with one of its problems as well as your own lab. A leaderboard time is a public, independently measured number, which is harder to dismiss than a self-reported benchmark.


How it works

You submit kernels to a Discord bot, or with the popcorn CLI, and it runs and times them on cloud GPUs, so you don’t need your own hardware. Rankings are kept separately per algorithm and per device, and submitting is free. Past AMD rounds ran LLM inference kernels on MI300s provided free through the Discord.

Whenever an AMD leaderboard is open, use it for timed Instinct runs and keep your rented hours for interactive profiling, where you need a shell.

The workflow for every problem

  1. Develop and check correctness on the 9070 XT or 5700 XT.
  2. Profile locally.
  3. Submit for the official time.
  4. After the deadline, read the top solutions and write down one technique you missed.

Problem for each module

ModuleProblemFolder in reference-kernels
module-0-setupVector add, to learn the submission flowpmpp_v2
module-1-hip-model, module-2-memory-boundGrayscale, histogram, prefix sum, sort, vector add (the standard practice set)pmpp_v2
module-4-gemmFP8 GEMM on MI300amd
module-5-attentionMLA decodeamd_202602
module-6-quantization-moeMoE on MI300; MXFP4 GEMM and MXFP4 MoE on MI355Xamd, amd_202602
module-8-shippingAll-to-all and other multi-GPU kernelsamd_distributed

The amd_202602 problems are the qualifier kernels of the AMD × GPU MODE E2E Model Speedrun (MXFP4 MoE, MLA decode, MXFP4 GEMM). Closed competitions may not accept new leaderboard entries. If so, run the problem’s reference harness yourself on rented MI300X time and compare against the published winning times.

Weekly habits

  • Watch one GPU MODE lecture that matches the current module. The archive is in gpu-mode/lectures.
  • Post your module write-up in the relevant Discord channel and ask for one round of feedback.
  • Study one past solution. The kernelbot-data dataset holds real submissions, including AMD MI300 ones.
  • In week 12, find two mock-interview partners and pitch a short community talk on your capstone.

Three rules

  1. Read each competition’s terms before entering. Sponsored rounds can exclude the sponsor’s employees or require approval. Practising on closed problems is always fine.
  2. Win honestly. A public audit found evaluator bypasses accepted on several leaderboards, including MI355X ones. Anyone reviewing your work will read the code, so a gamed rank costs more than it earns.
  3. Use agents, but own the result. Competitors now drive the popcorn CLI with AI agent loops; one placed 12th of 183 after more than 1,500 Codex-driven submissions. That is a legitimate tool, as long as you can explain every line you submit.