TRSP Retains 83% Accuracy on NoLiMa at 8× Training Length via a Topologically Regularized Side-Path

arXiv 2607.20484·medium signal

Tao, Cheng, Lu, Liu, and Chen diagnose long-context degradation as two opposite spectral pathologies — homogenization collapse (attention sinks driving rank deficiency) and isolation collapse — and identify a structural tension between mixing efficiency and information retention that standard attention cannot resolve. TRSP is a non-invasive side-path combining a parameter-free triangular mechanism with a lightweight context-length-aware scaling term to rebalance the spectrum without changing the base architecture. On NoLiMa at eight times the training length it retains 83% accuracy, substantially above competing long-context interventions.