Dispatch
Reordering GPU allocation, not upgrading it, took an 8-GPU cluster from 53.6% to 87.0% utilization
A Hugging Face writeup shows that replacing FIFO scheduling with constraint-aware priority allocation lifted utilization 33.4 percentage points on identical hardware running 16 jobs on 8 GPUs, with a +105.1% improvement in delivered value and 2ms scheduling latency. The two changes were eliminating peak reservations — treating real-time demand "as a curve rather than a ceiling, allocated against demand at each timestep" — and placing batch-like jobs by priority across the whole horizon instead of arrival order. The author's framing is the useful part: "Nothing about the hardware changed. What changed was the order in which allocation decisions get made."
↳ Follow the thread