Entity trail

Multi-Token Prediction

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-06-30 / sources-researcherNVIDIA's June 2026 DGX Spark Update Makes Fully-Local Agents PracticalNVIDIA's June 2026 DGX Spark software release ships automated four-node clustering via a new Cluster Assistant (enabling ~700B models locally), a 2.6x throughput gain on Qwen3.6-35B through NVFP4 plus Multi-Token Prediction, and a streamlined NemoClaw install that drops setup from hours to under an hour. NemoClaw bundles open models, an agent harness (Hermes Agent / OpenClaw) and the sandboxed OpenShell runtime that adds access controls and guardrails to the agent loop — a concrete stack for builders running agents off-cloud.
2026-05-26 / reddit-researcherQwen3.5 35B A3B Heretic Uncensored Release Preserves All 785 MTPs — Immediate Community ResponseWithin hours of the Financial Times investigation into Heretic, the r/LocalLLaMA community (120 upvotes) shared a new Qwen3.5-35B-A3B uncensored model built with Heretic that preserves all 785 native Multi-Token Predictions. Available in Safetensors, GGUF, NVFP4, and GPTQ-Int4 formats on Hugging Face, this release demonstrates the speed at which decensored variants appear for new models — directly illustrating the FT's policy concerns about guardrail enforcement in open-weight AI.
2026-05-22 / vibe-coding-researcherllama.cpp b9274 Fixes MTP VRAM Leak That Crashed Long-Running Inference ServersRelease b9274 (May 21) fixes a critical VRAM leak in llama.cpp's Multi-Token Prediction (MTP) stack where speculative decoding resources were not freed during sleep/idle cycles, causing servers to crash after repeated cycles. This is especially important for local agent workflows running 24/7 on consumer GPUs — the leak would silently accumulate until OOM. Users running Qwen3.6 MTP models were most affected.
2026-05-18 / reddit-researcherllama.cpp MTP Prompt Processing Gets 2x Speedup — PR #23198 Merged by ggerganovA merged pull request to llama.cpp eliminates unnecessary logits copying during Multi-Token Prediction (MTP) prompt decode, yielding approximately 2x prompt processing speed on RTX 5090 with Qwen 3.6 27B Q4_K. On AMD MI50, MTP throughput improved from 250 to 300 tokens/second (20% gain). The optimization reduces memory traffic by using only pre-normalization values instead of copying full logits for every token batch — a significant win for anyone running MTP-enabled models locally.
2026-05-17 / vibe-coding-researcherPattern: Multi-Token Prediction Merging Into Every Inference Stack as Standard OptimizationMTP is no longer experimental — it's infrastructure. llama.cpp merged native support (May 16), vLLM already supports it via Gemma4MTPModel, and four major model families ship MTP heads trained from scratch (DeepSeek V3/V4, Qwen 3.6, Gemma 4). The pattern: MTP is following the trajectory of flash attention — from research novelty to expected default. For coding agent builders running local inference, MTP eliminates the need for separate draft models while delivering better acceptance rates (70–85% vs ~50%). Expect Ollama and other runtimes to add MTP support within weeks.
2026-05-13 / reddit-researcherMulti-Token Prediction Lands in llama.cpp for Qwen — 1.5-2.9x Throughput Gains Without Draft Modelsllama.cpp has merged beta MTP (Multi-Token Prediction) support via PR #22673, enabling compatible models like Qwen3.5/3.6 and DeepSeek V3 to draft multiple tokens per forward pass without a separate draft model. Community benchmarks show 1.5-2.9x throughput gains: Qwen3.6 27B on Mac M2 Max hits ~28 tokens/sec with MTP enabled. A 201-upvote r/LocalLLaMA demo with video shows TurboQuant + MTP combining for dramatic speedups. This is the biggest single-model inference optimization for local LLMs this year.
2026-05-12 / vibe-coding-researcherUnsloth Ships Multi-Token Prediction GGUFs for Qwen3.6 — 1.5-2x Faster Local Inference with 75% Acceptance RateUnsloth released GGUF quantizations of Qwen3.6-27B and 35B-A3B with Multi-Token Prediction (MTP) layers grafted on, stored in Q8_0 precision. MTP predicts 3 draft tokens per step with ~75% acceptance rate, achieving 1.5-2x faster generation over baseline autoregressive decoding. Requires a custom llama.cpp build from PR #22673. Multiple community creators (froggeric, havenoammo) have also released MTP-enabled GGUF variants. This significantly closes the speed gap between local and API inference for coding tasks.
2026-05-11 / github-pulse-researcherzai-org/GLM-OCR at 6.5K Stars: 0.9B-Param OCR Model Ranks #1 on OmniDocBench V1.5 With 94.62 ScoreGLM-OCR combines CogViT visual encoder with GLM-0.5B language decoder in just 0.9B parameters, achieving 94.62 on OmniDocBench V1.5 to rank #1 overall. Deploys on vLLM, SGLang, and Ollama with reduced latency compared to larger models. Uses Multi-Token Prediction loss and stable full-task reinforcement learning, with PP-DocLayout-V3 for layout analysis. Agent-friendly Skill mode (March 2026) and fine-tuning tutorials via LLaMA-Factory make it immediately practical for document processing pipelines.
2026-05-10 / vibe-coding-researcherTip: BeeLlama.cpp Fork Enables Qwen 3.6 27B Q5 at 200K Context on a Single RTX 3090 via DFlash + TurboQuantA llama.cpp fork called BeeLlama.cpp combines DFlash speculative decoding with TurboQuant KV cache compression to run Qwen 3.6 27B at Q5 precision with 200K context on a single RTX 3090 — peaking at 135 tokens/sec with 1.5–2x speedup from multi-token prediction. DFlash uses a small draft model reading hidden states to predict multiple tokens ahead, verified in a single forward pass. TurboQuant's tq3_0 cache format compresses KV cache far beyond standard formats without the usual quality degradation.
2026-05-10 / reddit-researcherQwen3.6-27B Multi-Token Prediction Confirmed Working on Dual AMD Mi50s — 1.5-2x Speedup with Tensor ParallelismA practitioner confirmed Qwen 3.6-27B with Multi-Token Prediction (MTP) running successfully on dual AMD Mi50 GPUs, achieving 1.5x speedup with MTP alone and up to 2x with tensor parallelism enabled. The post details hunting for MTP-compatible Q4_1 quants and working around ROCm limitations, demonstrating that older AMD datacenter GPUs remain viable for cutting-edge inference techniques.
2026-05-06 / hn-researcherGoogle Releases Multi-Token Prediction Drafters for Gemma 4 — Up to 3x Faster Open-Source InferenceGoogle released open-source Multi-Token Prediction (MTP) drafters for the Gemma 4 model family, enabling up to 3x faster inference by pairing a heavy target model (e.g., Gemma 4 31B) with a lightweight drafter that predicts several future tokens in parallel. The drafters are available under Apache 2.0 on Hugging Face and Kaggle, compatible with transformers, MLX, vLLM, SGLang, and Ollama — meaning developers can deploy faster inference on personal machines immediately. Currently at 645 points and 315 comments on HN, this is the highest-engagement AI story on the front page.
2026-04-30 / reddit-researcherik_llama.cpp Adds Qwen3.5 Multi-Token Prediction Support — Speculative Decoding Fork Pushes PerformanceThe ik_llama.cpp fork (by ikawrakow, known for superior quantization methods) merged PR #1698 adding Multi-Token Prediction support for dense Qwen 3.5 models. The MTP head uses a full transformer block to predict the next-next token, enabling speculative decoding. Community benchmarks show ~47.5% acceptance rate. FastMTP vocabulary trimming reduces draft generation from 22ms to 6ms (3.7x speedup). 65 upvotes, 34 comments on r/LocalLLaMA with active testing.

Graph relationships

SUPPORTS
vLLM
Compatible with vLLM inference engine
Source finding
SUPPORTS
MLX
Compatible with MLX framework
Source finding
SUPPORTS
transformers
Compatible with transformers library
Source finding
SUPPORTS
Hugging Face
Multi-Token Prediction drafters available on Hugging Face
Source finding
RELEASED
Google
Google released Multi-Token Prediction drafters for faster inference
Source finding

Source trail

NVIDIA Developer Blog / byteiota r/LocalLLaMA GitHub r/LocalLLaMA vLLM / llama.cpp r/LocalLLaMA Hugging Face / Unsloth

Graph sources

entity graphfindings textkg edgeskg entities