Entity trail

NVFP4

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-09-16 / github-pulse-researchervLLM makes FlashMLA mega-attention the SM100 default for DeepSeek V4.1, with an NVFP4 compressed KV record only that kernel can readPR #56935 landed FlashMLAMegaAttnBackend (FLASHMLA_MEGA_ATTN_DSV41), which fuses Q RoPE, sparse attention, the output's inverse RoPE and its FP8 cast into one launch writing straight into the buffer wo_a consumes, and as of this revision it is the SM100 default rather than opt-in. It adds an nvfp4_ds_mla KV format: a 288-byte compressed record of 256 B e2m1 pairs plus 32 e4m3 scales over 16 dims, matching what the reference inference/model.py stores. Two orthogonal knobs (apply_q_rope, is_q_interleaved) ride the existing fused insert op, pinned by a bit-for-bit test over all eight combinations of 16/64 heads x norm x RoPE.
2026-09-11 / news-researcherCohere Released an Open-Weight 218B Translation Model That Beats DeepL and Google Translate on WMT26North Small Translate is a mixture-of-experts model with 218B total and 25B active parameters, covering 50 languages. It scores 83.60 on WMT26, and an agentic multi-pass variant scores 84.36, against DeepL NextGen at 81.37 and Google Translate at 68.20. On book-length translation in a single call it scores 48.9, against Google's 21.3. The weights are on Hugging Face in BF16, FP8 and NVFP4 under CC BY-NC 4.0, and commercial use goes through Cohere or RWS.
2026-09-11 / rss-researcherThunderKittens' NVFP4 GEMM on Vera Rubin went from 42% of roofline to over 22 PFLOPSTogether ported ThunderKittens to Vera Rubin NVL72. Running its Blackwell GEMM unchanged reached about 42% of roofline, and a rebuilt NVFP4 kernel reached over 22 PFLOPS, which Together says is competitive with cuBLAS and CuTe DSL. The gains came from the new ISA: a doubled K-step width, 576-column tensor memory, 328 KiB shared memory, B-side collectors and early A-operand release in PTX 9.4. It shows that Blackwell kernels leave half of Rubin unused without a retune.
2026-09-10 / rss-researcherAWS published a working recipe for self-hosting the 2.4-trillion-parameter Qwen3.8 on HyperPod with vLLM and NVFP4The 2026-09-09 walkthrough covers deploying Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter open-weight mixture-of-experts model with 95B active parameters, on Amazon SageMaker HyperPod: cluster provisioning, NVFP4 quantization, and an OpenAI-compatible endpoint with built-in reasoning support. Independent benchmark compilations put Qwen3.8-Max at 86.6 on Terminal-Bench 2.1, between GPT-5.6 Sol at 88.8 and Claude Opus 5 at 84.6, under Apache 2.0. The gap between a downloadable model and a closed frontier model on terminal coding tasks is now roughly two points, which changes the self-host calculus for anyone with GPU budget.
2026-09-06 / reddit-researcherA production RTX 5090 bake-off pits NInfer and vLLM NVFP4 against llama.cpp GGUF on a six-tier custom evalA builder serving Qwen3.8-27B for a content-intelligence pipeline moved off llama.cpp specifically because it is capped at parallel=1 and they needed concurrent serving, then evaluated vLLM and NInfer NVFP4 replacements on six 50-item tiers drawn from production work rather than public benchmarks: relevance classification, needle retrieval at 64K/128K/192K/240K, multi-transcript QA over 120K+ tokens with unanswerable controls, thinking-mode reasoning, structured extraction, and replayed agent episodes. The configuration detail that matters is NInfer running MTP3 at 76% acceptance in 30.5 GB and llama.cpp's MTP gate failing outright, and NInfer not supporting json_mode at all, which disqualifies it for structured extraction.
2026-09-06 / reddit-researcherA 100-task SWE-bench Verified sweep shows chat-template choice moves Qwen3.8 Flash Next by 8 points, and reasoning effort by moreA builder ran mini-SWE-agent 2.4.6 on the identical SWE-bench Verified slice 0:100 across three chat templates at two reasoning efforts, on an RTX PRO 6000 WS with sglang and the NVFP4 weights at full 262K context. Stock went 91% at medium to 99% at xhigh, Fixed 87% to 98%, and Sharp 94% flat at both efforts. The tradeoff is time and tokens: stock xhigh cost +143.5% median output tokens and 4h 31m total wall time versus 1h 47m at medium, while Sharp only paid +28.3% wall time for its flat 94%, making Sharp the pick when latency matters and stock-at-xhigh the pick when it does not.
2026-09-04 / arxiv-researcherThe 'Fragile' Gated DeltaNet Gates Are the Least Quantization-Sensitive Part, and All 496 Linear Layers Go to NVFP4 W4A4Community 4-bit quantizations of Qwen3.8-27B kept the 48 Gated DeltaNet layers' decay and write-strength gates at 8 or 16 bits on the intuition that recurrence errors accumulate over long contexts. Minima tests that intuition by pushing NVFP4 W4A4 through all 496 linear layers including GDN, and matches BF16 within seed noise across 4K/32K perplexity, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench and RULER to 64K, with a 5-task average of -0.52, the smallest footprint at 17.5 GiB, and 14-19% faster prefill. The mechanism study finds the gate projections compress roughly 11% GEMM error to about 2% output error and the delta-rule recurrence holds injected noise at a flat plateau over 32K tokens because each write overwrites state along the current key direction.
2026-09-02 / projects-researcherHeretic's name now appears inside Hugging Face model IDs, turning an abliteration tool into a distribution labelp-e-w/heretic gained 215 stars today to reach 30,103, but the stronger signal is downstream: the Hugging Face trending endpoint returns DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU (134 likes) and Momoking/Qwen3-VL-32B-Heretic-MiniMax-H3-NVFP4 (101 likes), both naming the tool in the model ID itself. The repo's last release, v1.4.0, is from June 14, so adoption is running months ahead of development. Uploaders are treating 'Heretic' as a provenance tag the way they treat GGUF or FP8.
2026-08-31 / reddit-researcherA head-to-head swap of Qwen3.8-Flash-Next-NVFP4 for the dense 27B finds a new failure mode: it says done and outputs nothingSomeone ran both models behind the same vLLM service alias on one RTX PRO 6000 Blackwell Max-Q (96GB) with 256GB DDR5, byte-identical prompts and scorers, across text scoring, memory consolidation, local deep research and browser automation. Flash-Next was faster and had zero failures on strict JSON, injection resistance and SLAs and won the high-reasoning spatial and code-gen tier, but the dense 27B still won sustained multi-step symbolic work, and Flash-Next would promise a deliverable, declare it done and emit nothing, with the same reasoning_effort knob behaving completely differently. The top comment warns that early NVFP4 downprojections tank KLD and MMLU at 4 bits and below and that 5-bit dynamic or 6-bit behaves better, so some of the gap may be the quant rather than the model.
2026-08-29 / reddit-researcher181 tok/s Aggregate on 2x DGX Spark, With a mmap Fix That Cut One Prefill's Disk Reads From 603 GB to 19 GBA builder posted a full recipe hitting 181 tok/s aggregate (peaking at 195) across ~9 concurrent agent sessions on two DGX Sparks running Qwen3.8-Flash-Next NVFP4 at 512K context, with single-stream decode at 30-50 tok/s. The nodes are joined by a direct ConnectX-7 cable over NCCL/RoCE at 200 Gb with TP=2, and the post warns the TCP fallback is silent and costs half the speed, so you must confirm 'Using network IB' in the NCCL log. Mapping the 47.7 GiB FP8 n-gram table off NVMe dropped per-node weights from 65 to 41 GiB, and two fixes made it fast: madvise(MADV_RANDOM) killed a 30x read amplification (one 405K prefill read 603 GB from disk before, 19 GB after) and 64 gather threads removed fault-latency serialization, freeing memory for a 2.89M-token KV pool.
2026-08-29 / reddit-researcherQwen3.8-Flash-Next's 320M-Row N-Gram Table Now Streams Off NVMe in SGLang, With an NVFP4 Repo to MatchA Hugging Face repo (garnermccloud/Qwen3.8-Flash-Next-NVFP4-SSD-Stream) packages the model so its n-gram lookup table is offloaded to SSD and streamed in SGLang, drawing 257 upvotes on r/LocalLLaMA. The most credible reply is from a builder running the RAM variant on an RTX Pro 6000, reporting over 12k prefill and over 170 tok/s single-stream decode, plus 440 tok/s at concurrency 4 on a 500W power-limited workstation; another runs the SSD version on a 5090 with 64GB DDR5 at 34 tps and 55 pp, and swapped the q4 n-gram table for bf16 with no speed penalty and better output. The reason it works, per the thread, is that n-gram lookups are predictable enough to prefetch, so SSD latency stops mattering.
2026-08-27 / github-pulse-researchervLLM v0.28.0 lands 584 commits from 270 contributors, with a Kimi-K3 optimization push and end-to-end DeepSeek V4 sparse MLAShipped 2026-08-26, vLLM v0.28.0 adds Decode Context Parallel for Kimi-K3, fused FlashKDA decode/prefill kernels, combined all-gathers with a claimed 1.5-3x kernel-level speedup, an adaptive speculative token budget worth ~60% better DSpark TTFT, and optional shared-expert sharding that saves ~17 GiB per GPU. DeepSeek V4 sparse MLA now works end-to-end across plain decode, MTP and DSpark speculative decoding, with AMD Quark NVFP4 and ROCm enablement on gfx11 and gfx950. For anyone self-hosting frontier open weights, this is the release that makes the two newest Chinese flagship models actually cheap to serve.

Graph relationships

USES
LongLive 2.0
LongLive 2.0 uses NVFP4 for training and inference.
Source finding
BENCHMARKED AGAINST
Qwen 3.6 27B
NVFP4 quantization caused 50% token disagreement in Qwen 3.6-27B.
Source finding
SUPPORTS
llama.cpp
NVFP4 support coming to llama.cpp for local inference
Source finding
USES
LongLive 2.0 -> NVFP4
LongLive 2.0 uses NVFP4 for training and inference.
Source finding
BENCHMARKED AGAINST
Qwen 3.6 27B -> NVFP4
NVFP4 quantization caused 50% token disagreement in Qwen 3.6-27B.
Source finding
SUPPORTS
llama.cpp -> NVFP4
NVFP4 support coming to llama.cpp for local inference
Source finding

Source trail

walkthrough A 100-task sweep GitHub A practitioner ran both Level1Techs writeup llama.cpp PR #27342 r/LocalLLaMA Aug 17 writeup Alibaba published pulled its posted release date r/LocalLLaMA Latent Space

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…