llama.cpp's multi-model server hung on concurrent requests for the same model until a scheduler tick was added

GitHub·medium signalXBlueskyLinkedInCopy link

PR #28539, merged 2026-09-07T13:50Z, fixes an LRU hang in llama-server when several requests target the same model, closing #28443. The fix adds a tick() to the scheduler to make it more coroutine-like rather than restructuring the threading. The maintainer notes in the PR that server_models is very IO-bound and hard to maintain as a thread model, and that converting it to coroutines is under consideration, so anyone running llama-server in multi-model mode should expect further churn there.