S0 Tuning: Single State Matrix Per Layer Outperforms LoRA by +10.8pp on HumanEval with Zero Inference Overhead

arXiv·high signal

S0 tuning optimizes one initial state matrix per recurrent layer in hybrid recurrent-attention models while freezing all weights, achieving zero inference overhead. Using only ~48 HumanEval training solutions, it outperforms LoRA by +10.8pp (p<0.001). On Qwen3.5-4B (GatedDeltaNet hybrid), S0 improves greedy pass@1 by +23.6pp. On FalconH1-7B (Mamba-2 hybrid), reaches 71.8%. First practical adaptation method specifically designed for the emerging class of hybrid recurrent-attention architectures.