Elastic Weight Consolidation Keeps General Skills After Translation Fine-Tuning but Still Loses Formality and Gender Control
arXiv·low signal
arXiv 2609.28395 (23 Sep) fine-tunes Llama 3.1 8B Instruct on Arabic-English and Spanish-English data. EWC held the general-benchmark drop to 1.7 points against 11.0 for standard fine-tuning, but formality and grammatical-gender instruction following degraded about as much as with no mitigation. Only mixing in control-task examples preserved those controls, and that did not transfer to unseen prompts, so retention on general benchmarks is a poor proxy for task-specific instruction following.