A one-third-cost upgrade policy for fine-tuned specialists gave up 0.37pp of quality with zero behavioral regressions

arXiv 2608.20918·medium signalXBlueskyLinkedInCopy link

UpgradeBench asks the question every team with a fine-tuned model faces when a new base version ships: keep, port the adapter, or retrain. Across four Qwen releases and OLMo checkpoints, specialist durability ranged from under one release interval to over fourteen months, and direct adapter copying degraded with continued-pretraining distance from near-perfect retention to zero, independent of architecture or model family. Applying one consistent policy across 33 episodes cost 0.37pp mean quality regret at a third of the compute and label budget of full retraining, and a CKA probe over just 256 prompts predicted adapter portability at Spearman 0.74.