A production RTX 5090 bake-off pits NInfer and vLLM NVFP4 against llama.cpp GGUF on a six-tier custom eval
A builder serving Qwen3.8-27B for a content-intelligence pipeline moved off llama.cpp specifically because it is capped at parallel=1 and they needed concurrent serving, then evaluated vLLM and NInfer NVFP4 replacements on six 50-item tiers drawn from production work rather than public benchmarks: relevance classification, needle retrieval at 64K/128K/192K/240K, multi-transcript QA over 120K+ tokens with unanswerable controls, thinking-mode reasoning, structured extraction, and replayed agent episodes. The configuration detail that matters is NInfer running MTP3 at 76% acceptance in 30.5 GB and llama.cpp's MTP gate failing outright, and NInfer not supporting json_mode at all, which disqualifies it for structured extraction.
↳ Follow the thread