Manifold-Constrained Hyper-Connections: PEFT That Adapts Residual Streams, Not Weights

arXiv·medium signal

Valentijn Oldenburg, Floris de Kam and Bente Zuijdam (arXiv 2607.18130, cs.LG) note that nearly every parameter-efficient finetuning method — LoRA, adapters, prefix tuning — modifies weights or activations while leaving the residual connections untouched. Their method places manifold constraints on hyper-connections to adapt that untouched component. It is a genuinely different axis for PEFT and worth watching if LoRA has plateaued on your task.