A Continual-Learning Method's 5.0-Point Win Becomes an 11.6-Point Loss When the Baseline Gets a Bigger LoRA Rank

arXiv 2609.03900·medium signalXBlueskyLinkedInCopy link

Continual knowledge-updating methods are usually declared superior from one final checkpoint at one conventional adapter rank. Comparing a periodic hierarchy against cumulative replay over a 24-month Wikidata stream while varying evaluation month, replay LoRA rank and query formulation, the winner flips: on Qwen2.5-1.5B the hierarchy's 5.0-point advantage over rank-8 replay becomes an 11.6-point deficit against rank-72 replay, and at high ranks a consolidation-aligned endpoint suggests a tie while time-averaged replay leads by 9-13 points. The same rank-conditioned reversal appears on Llama-3.2-1B and held-out paraphrases, so the honest reading is that the hierarchy is a lower-update-cost operating point, not a quality winner.