llama.cpp Merged Qwen3.8-Flash-Next Support, Memory-Mapping the 97.7 GiB N-Gram Table to Disk

llama.cpp GitHub, via r/LocalLLaMA·medium signalXBlueskyLinkedInCopy link

PR 27742 by danielhanchen, merged August 26 by ngxson, adds gated delta net layers, 512-expert MoE with top-10 selection, hyper-connections, query-key sparse attention and the per-layer n-gram embeddings as a 97.7 GiB memory-mapped table that can sit in RAM or on disk. The poster who announced it reported 55 tok/s on 4x3090 with the Q4 GGUF, and a commenter got 10 tok/s on a 4GB card by offloading to SSD. MTP is still work in progress, and one commenter warns the current engram implementation only works with mmap and has no eviction mechanism, so mlock will lock the whole table into memory.