Tools
llama.cpp's context auto-fit now sizes unified KV by model length times parallel slots
PR #28849, merged 2026-09-16, changes auto-fitting to try up to model context length multiplied by parallel slots for unified KV, matching how the non-unified-KV path already behaved, and giving more headroom for concurrent requests. It remeasures memory at the larger size and shrinks context if needed, leaving minimum limits, alignment and explicit nonzero --ctx-size values unchanged. Anyone running llama-server with -np above 1 and no explicit context size gets a different, larger allocation from this build onward.
Source
↳ Follow the thread