DoRA (Weight-Decomposed LoRA): 2026's Preferred Fine-Tuning Technique — Decomposes Updates Into Magnitude and Direction for Better Convergence

DasRoot·low signal

DoRA improves on standard LoRA by decomposing weight updates into magnitude and direction components, resulting in better convergence on complex tasks without additional VRAM overhead. Available in PEFT and Unsloth with a single flag (use_dora=True). The 2026 consensus: use target_modules='all-linear' as your starting point (not just q_proj/v_proj), and combine DoRA with RS-LoRA's √r scaling fix for best results. LoRA now recovers 90-95% of full fine-tune performance.