P135/4 The post on X
P135 Bubbling Under, week 7
#35Bubbling Under
on SlopHits
on SlopHits
Chasing the Roofline
“claude opus 5.5 just one-shot a music video on how to optimize CUDA kernels”
- By
- @elliotarledge on X ↗Elliot Arledge
- Views
- 585K views on X
- Length
- 3:10
- Posted
- Sep 27, 2026
- Duels
- 7 won, 8 lost
- Move
- Down 9 since last week
P135/2 Lyrics & meaning
What it’s about
A tutorial song about making GPU code fast, sung as a pop chorus. Each verse covers one CUDA idea: thousands of parallel threads, how a thread finds its index, warps of 32 threads and branch divergence, reusing data from fast shared memory, tensor cores, FlashAttention, low-precision numbers and splitting work across GPUs. The chorus is the roofline model: speed is capped by memory bandwidth (the slope) or by raw math (the flat line), so the goal is more math per byte moved. The post says Claude Opus 5.5 made the video in one go. The words appear on screen as a lyric video with diagrams.
Names and terms
- Roofline
- The roofline model plots a kernel's speed against how much math it does per byte of memory traffic. A sloped line (memory limit) meets a flat line (compute limit). The song's chorus is about climbing toward that ceiling.
- Block times size plus thread
- The usual CUDA formula for a thread's global index: blockIdx.x * blockDim.x + threadIdx.x. It lets each thread pick its own element of the data.
- Warp
- A group of 32 GPU threads that execute the same instruction together. If threads in one warp take different sides of an if, the two paths run one after the other (divergence), so some threads wait.
- Load a tile to shared
- Threads in a block copy a tile of data from slow global memory into fast on-chip shared memory once and reuse it. The synchronisation step (__syncthreads in CUDA) makes sure the whole tile has arrived.
- Tensor cores
- Units on NVIDIA GPUs that multiply small matrix blocks in one operation. The song's "sixteen by sixteen" refers to the 16 by 16 tile size that CUDA's WMMA interface exposes.
- Flash attention
- FlashAttention (Tri Dao and co-authors, 2022) computes transformer attention in tiles so the full sequence-by-sequence score matrix is never written to GPU main memory. That is the song's "never writes the square".
- 32 bits down to 4
- Quantization stores numbers in fewer bits. Going from 32-bit to 4-bit values means eight times fewer bytes to move, which helps when memory bandwidth is the limit.
- CUDA for Deep Learning
- Title on the video's closing card, with the name Elliot Arledge. The poster's X bio says he made a 12-hour CUDA course and links to his books.
Chasing the Roofline: lyrics
Verse 1A CPU thinks with a few big cores.
A GPU just throws ten thousand more.
One thread for every number you've got.
Block times size plus thread finds its spot.
Ten thousand threads all launch at once.
So tell me, how fast did it run?
ChorusI'm chasing the roofline.
Memory's the slope and the math's the line.
Squeeze more math from every byte,
Till I'm hitting the ceiling tonight.
Verse 2Thirty-two threads in a warp, in line.
One instruction marching in time.
One little if and the warp splits in two.
Half of them wait while the others go through.
Pre-ChorusSame old code, but it's crawling slow.
Tell me, where'd all my speed go?
ChorusI'm chasing the roofline.
Memory's the slope and the math's the line.
Squeeze more math from every byte,
Till I'm hitting the ceiling tonight.
Verse 3Neighbors read their neighbors side by side.
One trip to memory, nice and wide.
Load a tile to shared and keep it near.
Sync the block and reuse it here.
Fetch it once and use it more.
That's what the roofline's for.
ChorusI'm chasing the roofline.
Memory's the slope and the math's the line.
Squeeze more math from every byte,
Till I'm hitting the ceiling tonight.
Verse 4Tensor cores eat a matrix whole,
Sixteen by sixteen in a single go.
Flash attention never writes the square,
Streams it through in tiles from here to there.
Thirty-two bits down to four and it's fine,
Eight times less to move down the line.
When one GPU is not enough,
Split it eight ways, then sum it up.
ChorusI'm chasing the roofline.
Memory's the slope and the math's the line.
Squeeze more math from every byte,
Till I'm hitting the ceiling tonight.
OutroI'm chasing the roofline.
Memory's the slope and the math's the line.
Transcribed from the video and checked by ear. Words we could not make out are marked [unclear]. Made this song and want the text off this page? Tell us and it goes.
P135/3 More from the chart
Around #35of 894 in the running
- 33
Untitled“send this to anyone who says you make nothing”by @antic - 34
life of coderabbit's rabbitby @IndraVahan - 35
Chasing the Rooflineby @elliotarledgeThis clip - 36
Back to the Shireby @marksoares - 37
I'm Upping My P(Doom)Coverby @_mexicat