A Coding Agent Hit 232x on a GPU Kernel, Then Stalled

A developer spent 14 days and 1,500-plus submissions driving GPT-5.5 through Codex, with Claude Pro as an advisor, to optimize a batched Householder QR kernel on a B200. Runtime dropped from about 419ms to about 1.8ms, a 232x speedup good for 12th place out of 183 on the leaderboard, per the writeup at sankalp.bearblog.dev. The setup cost $200 for the ChatGPT Pro plan plus $20 for Claude Pro.

The path went through 10 structural rewrites: cuSOLVER to custom Triton and CUDA, fused panel assembly, grouped WY updates, CUDA graph replay. The operational method is what's worth copying. He ran a beam search over multiple candidate ideas, pruned by measured results. That beat one conversation thread committing to its first idea and defending it for three hours. He also used /goal prompts with quantitative targets, so the agent optimized against a number instead of a vibe.

Then the honest part. The final stretch, from 3,000 microseconds down to 1,805, needed sharply increased human steering. Brute-force search got 99% of the way there. The last bit needed someone who understood Householder QR. Compute plateaued exactly where domain knowledge became the bottleneck.

A paper submitted August 13 makes the same point from a different direction. CLI-based coding agents ported CReSS, a 250,000-plus line legacy Fortran weather simulation, to GPU. They produced validated implementations for 162 target kernels and a 5.1x application-level speedup, per arXiv 2608.13122. Five kernels showed numerical discrepancies from floating-point differences and branch divergence. They were caught only because validation ran inside the loop, not bolted on at the end.

Five out of 162 is a 3% silent-failure rate. Nothing crashed. The code compiled, ran, and produced wrong numbers in a weather model. Point agents at a large migration and design the harness around that 3% rate, not the 0% you wish for.