Sequence model optimization as Prompt → RAG → Fine-tune → Distill, with DPO as the default
BigData Boutique·low signal
The 2026 consensus ordering is Prompt → RAG → Fine-tune → Distill: exhaust prompting and retrieval before touching weights, and when you do fine-tune, use a thin LoRA/QLoRA adapter on a strong base paired with retrieval rather than replacing it — full fine-tuning is rarely the right call. Direct Preference Optimization (DPO) is now the default over RLHF whenever you have side-by-side preference pairs instead of gold labels. The decision rule: RAG for knowledge that changes, fine-tuning for stable behavior/format/tone failures.