One Transformer Layer May Rival Full RL Tuning

A new paper argues RL post-training needs one transformer layer, not the whole network, to match full-parameter fine-tuning, per the arXiv preprint circulating on Hacker News.

If it holds, alignment training gets far cheaper for labs and independent researchers who can't afford full-parameter RL runs today. A single-layer approach would put post-training within reach of far smaller budgets.

Reinforcement learning post-training is the RLHF and RLVR pass that turns a raw model into something you'd trust with instructions or a tool call. It normally touches every parameter in the network. Updating one layer instead cuts the compute bill for that step.

The available summary doesn't say how the result holds up on harder RLVR tasks or bigger base models. Single-layer results have a habit of looking great on the benchmark they were built for, then falling apart elsewhere.

It surfaced on Hacker News. It's the latest sign of a pattern I keep seeing: RL post-training touches less of a model than people assumed going in.

My bet is this doesn't generalize past the setup in the paper, since RL usually spreads updates across many layers to handle different tasks. Other researchers reproducing the result on a bigger model or a harder task would settle it. Until then, it's not the default recipe for cheap alignment, per the arXiv preprint.