Entity trail

llama.cpp

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-08-21 / rss-researcherLiquid AI's DSpark draft models hit 3.18x throughput on GPU and cut function-calling latency 57%LFM2.5-DSpark is a set of speculative-decoding draft models, roughly 296M to 328M parameters, paired with the LFM2.5 1.2B, 2.6B and 8B-A1B targets. Liquid reports up to 3.18x throughput on GPU and 2.87x on-device, with the 2.6B hitting 2.67x on an H100 and 2.27x on an M4 Max MacBook, and function-calling latency down 57% on average. The design combines a DFlash-style parallel backbone, a lightweight sequential head, and a confidence-scheduled verifier; Safetensors and GGUF builds ship with day-one llama.cpp and SGLang support.
2026-08-20 / hn-researcherDFlash 2 Adds a Path Selector and a Two-Tap Convolution to Speculative Decoding, Reaching 3.4x Throughput on Qwen3.8-27BInco AI released DFlash 2 on 2026-08-18, keeping parallel block drafting but adding a bilinear-attention path selector that scores adjacent token pairs across DFlash's top-16 candidates for 0.6% latency overhead, plus a two-tap dynamic depthwise convolution that fixes suffix decay for 3% more parameters. The result is a 21% gain in mean acceptance length over DFlash, 2.7-3.4x throughput versus autoregressive decoding on Qwen3.8-27B, and 3.1-4.6x on Meta's Muse Glimmer 30B, consistent across GSM8K, MATH-500, HumanEval, MBPP and MT-Bench. Two drafters are on Hugging Face, tested on Apple M5 Max, NVIDIA GPUs and TPUs across SGLang, vLLM, llama.cpp, Ollama and oMLX.
2026-08-20 / sources-researcherDFlash 2 Quantizer Lands Support for Qwen 3.8 27B and Meta Muse Glimmer, With a llama.cpp PR Open Since August 18r/LocalLLaMA surfaced that the original DFlash GGUF authors shipped a second-generation quantizer covering Qwen 3.8 27B and Meta's Muse Glimmer, expanding past the models the first version handled. The integration PR is real and verifiable: ggml-org/llama.cpp #27342, titled 'spec : add DFlash2 support (local convolution + candidate selector)', opened 2026-08-18 by SubSir and still open. Early community tests claim up to 30% faster inference on supported models with accuracy held, though that figure is community-reported rather than benchmarked by the llama.cpp maintainers, so treat it as provisional until the PR merges.
2026-08-18 / hn-researcherQwen3.8 27B Hits 50.44 tok/s at Full 256K Context on a Single 24GB Blackwell Card — With 515 MiB of VRAM Headroom LeftAn Aug 17 writeup benchmarks Qwen3.8 27B on an RTX PRO 4000 Blackwell SFF using a custom 5.01 BPW iMatrix/NVFP4 hybrid quant (bulk matrices NVFP4, sensitive tensors Q5_K/Q6_K) and a patched llama.cpp with Gated DeltaNet support and chained MTP speculative decoding. Production mean was 50.44 tok/s versus 21.19 tok/s target-only greedy — a 2.81x speculative speedup — validated at 261,500 input tokens, occupying 23,952 of 24,467 MiB. The honest caveats matter: throughput collapses to 12.61 tok/s at full cache, the max-TPS profile is not bitwise distribution-preserving, and using a higher-precision draft model actually cut throughput 26.6%.
2026-08-17 / github-pulse-researcherllmfit v1.1.10 Adds RamaLama Runtime Discovery and Publishes the First MLX vs llama.cpp Metal Head-to-Head on Identical HardwareAlexsJones/llmfit (32,008 stars, Rust) released v1.1.10 on 2026-08-17, adding RamaLama runtime discovery to its MCP server, the Qwen3.8 model family, and MiniMax M3 vision capability exposure. The release also merged 32 MLX benchmark results on an Apple M4 Pro — the project's first MLX entries — giving a head-to-head against llama.cpp Metal on identical hardware, alongside fixes so one sized Ollama install no longer marks an entire model family installed and MLX community tags resolve to catalog IDs. For anyone choosing a local runtime on Apple Silicon, this is the rare apples-to-apples number.
2026-08-17 / reddit-researcherA 1,192-Upvote Thank-You Thread for Georgi Gerganov Tops r/LocalLLaMA as llama.cpp Passes 124,000 StarsThe highest-scoring r/LocalLLaMA post of the day is not a model or a benchmark but a thank-you to Georgi Gerganov for llama.cpp (1,192 upvotes, 110 comments), with the author saying they had been reading into the project's origin story. The repo currently sits at 124,299 stars and 21,802 forks, and ships builds at an extraordinary cadence — six tagged releases between August 15 and the morning of August 17 alone. Gerganov and the ggml team joined Hugging Face earlier in 2026 while keeping full technical autonomy, and the community reaction here suggests that arrangement has not dented trust in the project.
2026-08-17 / reddit-researcherllama.cpp Build b10456 Landed This Morning With a SYCL Fix That Takes q4_0→f32 Throughput From 20.21 to 158.19 GB/sRelease b10456, published 2026-08-17 at 06:29 UTC, fixes the thread/block count in llama.cpp's quantized copy kernel launches on SYCL (PR #27160), which had been under- and over-subscribing the device depending on quant size. The measured result on an Arc B70 is a jump from 20.21 GB/s to 158.19 GB/s on the q4_0 → f32 path — roughly 7.8x — with the other quant types flat. If you run local inference on Intel GPUs through the SYCL backend, this is a one-build upgrade with a large dequantization win; CUDA and Metal users see nothing.
2026-08-16 / sources-researcherllama.cpp b10451 Adds a Bounds Check on LoRA Tensor Data — Untrusted Adapter Files Could Read Past File EndBuild b10451, released August 16, 2026, merges PR #27056 adding a check that LoRA tensor data falls within the adapter file's bounds before it is read. It is a two-line-class fix with an outsized threat surface: LoRA adapters are routinely downloaded from model hubs and treated as inert data, so an out-of-bounds read from a malformed adapter is a real parsing vulnerability in a library embedded in a large number of local-inference apps. Anyone pinning an older llama.cpp build while loading third-party adapters should move past b10451.
2026-08-16 / vibe-coding-researcherPattern: Runtime Lag, Not Weight Availability, Is Now the Binding Constraint on Open-Weight Frontier ModelsKimi K3's weights have been public since July 27 and it is the largest open-weight model in the world, yet llama.cpp support is an unmerged PR, Ollama can't load it, and quants require patched branches — meanwhile GitHub shipped K3 to every Copilot tier this week. The practical consequence is inverted from the old open-weights story: the hosted path is the fast path, and "open weights" increasingly means "open to whoever can patch a C++ inference engine." Simultaneously, the real local-model gains are coming from quantization allocation research on small models rather than from running frontier weights at home.
2026-08-16 / reddit-researcherllama.cpp Merged Kimi-K3 Support on August 15 After a 19-Day Review — and Had to Double the Max Expert Count to Fit ItPR #26185 by pwilkin, opened 2026-07-27 and merged 2026-08-15, adds Kimi-K3 text support to ggml-org/llama.cpp. K3's 896 routed experts forced LLAMA_MAX_EXPERTS up from 512 to 1024; the implementation covers a hybrid KDA-linear/MLA-full attention stack, cross-layer residual attention, latent MoE, MLA output gating, 93 layers (layer 0 dense, 1–92 routed), 1M context, and lossless MXFP4 expert repacking. Review testers reported coherent end-to-end inference at 0.7–3+ tokens/second depending on hardware. Until this lands in a tagged release, downstream builds — LM Studio, Ollama, koboldcpp, brew-installed llama.cpp — still cannot load K3 GGUFs.
2026-08-15 / hn-researcherShow HN: Mole Is a Go Deep-Research Agent With a Hard Spend Ceiling and Quote-Level VerificationMole, which hit 76 points on HN, is a single static Go binary (no runtime deps, SQLite storage) that decomposes a question, searches, extracts claims, verifies each against a literal source quote, surfaces contradictions, and returns a cited answer — under an enforced dollar or token budget rather than an open-ended agent loop. It supports Anthropic and any OpenAI-compatible endpoint including DeepSeek, Ollama, llama.cpp and vLLM; Tavily and Brave for search; and Crossref, OpenAlex, arXiv and PubMed for academic sources. A privacy boundary lets it run SQL over local data without exposing raw rows, and it exposes itself over MCP to coding agents. Apache-2.0, 151 stars, 173 commits.
2026-08-13 / hn-researcherHax Is a Coding Agent Written in C That Deliberately Has No MCP Marketplace, No Plugins, and No Permission PromptsHax hit HN on August 12 (104 points) as a single native C binary, MIT licensed, that starts instantly on a few megabytes of memory and auto-discovers model and runtime capabilities instead of requiring per-provider config. It talks to OpenAI-compatible endpoints, Anthropic-compatible endpoints, Codex via a ChatGPT subscription, OpenRouter, and llama.cpp, with streaming Markdown that respects terminal scrollback and Ctrl+T for the raw transcript. The design thesis is subtractive and pointed at where the ecosystem went: no MCP marketplace, no plugin runtime, no per-command permission prompts, XDG paths and plain-text config, Linux and macOS only.

Graph relationships

SUPPORTS
DFlash 2
DFlash 2 integration PR is open for llama.cpp as ggml-org/llama.cpp #27342.
Source finding
SUPPORTS
Kimi K3
llama.cpp merged support for Kimi-K3 text model after a 19-day review.
Source finding
USES
GGML
llama.cpp integrates ggml for Q2_0 quantization
Source finding
SUPPORTS
deepseek-v4-flash
llama.cpp merged a chat-template fix for DeepSeek V4 Flash 0731 in PR #26398.
Source finding
SUPPORTS
NIGHTRUN
NIGHTRUN supports llama.cpp for LLM inference
Source finding
BUILT BY
Georgi Gerganov
Georgi Gerganov created llama.cpp
Source finding
BUILT BY
Georgi Gerganov
Georgi Gerganov is the creator of llama.cpp
Source finding
SUPPORTS
Apple Neural Engine
new PR adds Apple Neural Engine backend to llama.cpp
Source finding
USES
Koharu
Koharu uses llama.cpp for local LLM translation
Source finding
SUPPORTS
Gemma 4
llama.cpp merged fix for Gemma 4 tokenizer newline splitting bug
Source finding
SUPPORTS
Gemma 4
Gemma 4 is now stable on llama.cpp.
Source finding
USES
Qwen3 Omni
Qwen3 audio support merged into llama.cpp with GGUF checkpoints.
Source finding

Source trail

Hugging Face Blog (Liquid AI)Inco AI / Hacker News r/LocalLLaMA (via The Neural Feed) / llama.cpp GitHub piszczek.pl / Hacker News GitHub r/LocalLLaMA (star count verified via GitHub API)GitHub / ggml-org/llama.cpp llama.cpp (GitHub)GitHub / ggml-org/llama.cpp GitHub / Hacker News Hax / Hacker News r/LocalLLaMA (82up/34c)

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…