The Pragmatic Engineer Deep Dive: 'What Is Inference Engineering?' — New Discipline Emerges Between ML and Systems

The Pragmatic Engineer·medium signal

Gergely Orosz at The Pragmatic Engineer published a deep dive on inference engineering, featuring Philip Kiely's new book 'Inference Engineering.' The piece maps out inference engineering as an emerging discipline distinct from ML engineering, focused on the challenges of serving models at production scale — latency optimization, batching strategies, hardware-model co-design, and cost management. Many engineers use inference daily but the engineering discipline behind it remains obscure and rich with interesting challenges.