Tools
A missing pair of message delimiters made every DeepSeek V4 coding-agent turn re-prefill 12,651 tokens
llama.cpp PR #29008, merged 2026-09-17T07:41Z, adds `<｜User｜>` and `<｜Assistant｜>` as message delimiters to the DeepSeek V3.2 parser, which #24176 had skipped when it added them everywhere else. Because V3.2/V4/V4.1 use SWA layers and the KV cache cannot roll back to a position without a checkpoint, llama-server was only placing checkpoints at `end - 4` and `end - 4 - n_ubatch`, so any divergence before the tail re-prefilled from token 0. On DeepSeek-V4.1-Flash on one RTX A6000 with a 13,167-token agent prompt, a new user question went from 510 prefilled tokens in 13.8 s to 16 tokens in 0.8 s, and editing the second of three user turns went from a full re-prefill to 79 tokens in 5.0 s.
Source
↳ Follow the thread