For sub-34B customization, default to QLoRA on a single GPU — ~95% of full-fine-tune quality at ~10% of the cost
Spheron·medium signal
In 2026 the cost-effective move for anything under 34B parameters is QLoRA on a single RTX 4090/H100, where a small run lands around 8–12 hours and $10–16, and LoRA generally captures ~95% of full fine-tuning quality for ~10% of the cost. If you need a fast/cheap deployable model, distillation is the complement: a distilled 8B from a 70B teacher keeps 90–95% of teacher quality at ~10% of inference cost. Budget for the hidden costs — data prep, storage, and maintenance can double the headline number.