The GPU MODE track
Runs alongside every module · Part of Kernel Ladder
What this track is for
GPU MODE is a practice arena, an independent benchmark referee and a peer group in one place. Every module ends with one of its problems as well as your own lab. A leaderboard time is a public, independently measured number, which is harder to dismiss than a self-reported benchmark.
How it works
You submit kernels to a Discord bot, or with the popcorn CLI, and it runs and times them on cloud GPUs, so you don’t need your own hardware. Rankings are kept separately per algorithm and per device, and submitting is free. Past AMD rounds ran LLM inference kernels on MI300s provided free through the Discord.
Whenever an AMD leaderboard is open, use it for timed Instinct runs and keep your rented hours for interactive profiling, where you need a shell.
The workflow for every problem
- Develop and check correctness on the 9070 XT or 5700 XT.
- Profile locally.
- Submit for the official time.
- After the deadline, read the top solutions and write down one technique you missed.
Problem for each module
| Module | Problem | Folder in reference-kernels |
|---|---|---|
| module-0-setup | Vector add, to learn the submission flow | pmpp_v2 |
| module-1-hip-model, module-2-memory-bound | Grayscale, histogram, prefix sum, sort, vector add (the standard practice set) | pmpp_v2 |
| module-4-gemm | FP8 GEMM on MI300 | amd |
| module-5-attention | MLA decode | amd_202602 |
| module-6-quantization-moe | MoE on MI300; MXFP4 GEMM and MXFP4 MoE on MI355X | amd, amd_202602 |
| module-8-shipping | All-to-all and other multi-GPU kernels | amd_distributed |
The amd_202602 problems are the qualifier kernels of the AMD × GPU MODE E2E Model Speedrun (MXFP4 MoE, MLA decode, MXFP4 GEMM). Closed competitions may not accept new leaderboard entries. If so, run the problem’s reference harness yourself on rented MI300X time and compare against the published winning times.
Weekly habits
- Watch one GPU MODE lecture that matches the current module. The archive is in gpu-mode/lectures.
- Post your module write-up in the relevant Discord channel and ask for one round of feedback.
- Study one past solution. The kernelbot-data dataset holds real submissions, including AMD MI300 ones.
- In week 12, find two mock-interview partners and pitch a short community talk on your capstone.
Three rules
- Read each competition’s terms before entering. Sponsored rounds can exclude the sponsor’s employees or require approval. Practising on closed problems is always fine.
- Win honestly. A public audit found evaluator bypasses accepted on several leaderboards, including MI355X ones. Anyone reviewing your work will read the code, so a gamed rank costs more than it earns.
- Use agents, but own the result. Competitors now drive the popcorn CLI with AI agent loops; one placed 12th of 183 after more than 1,500 Codex-driven submissions. That is a legitimate tool, as long as you can explain every line you submit.