'Attention Amnesia': Chain-of-Thought Fine-Tuning Can Break Long-Range Recall in Hybrid LLMs
arXiv (via HuggingFace Daily Papers)·medium signal
'Attention Amnesia in Hybrid LLMs' (arXiv 2606.11052) finds that chain-of-thought fine-tuning can degrade long-range recall in hybrid attention+state-space models, with proposed mitigations. It's a concrete warning for teams fine-tuning long-context hybrid models for reasoning — the CoT training that boosts step-by-step performance may silently cost you long-context retrieval.