Follow the Prompt to RAG to Fine-tune to Distill ladder before reaching for a custom model

BigData Boutique·medium signal

2026 fine-tuning guidance is to escalate in order — prompt, then RAG, then a thin QLoRA adapter (rank ~16, 4-bit base, one command via Unsloth on a single GPU), and only then distillation — rather than jumping straight to training. A distilled 8B from a 70B teacher routinely keeps 90–95% of quality at ~10% of inference cost, so distillation is the lever for latency and cost, while LoRA-on-a-strong-base plus retrieval covers most customization. Match the rung to the actual gap instead of over-investing.