Entity trail

MXFP4

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-02 / reddit-researcherMXFP4 with W4A8 kernels pushes Qwen3.8 27B to 280 tok/s on two Radeon R9700s with 940k tokens of KV cacheAn r/LocalLLaMA builder posted a two-month follow-up on his dual R9700 setup: after adding MXFP4 support on top of DeadCode's radiance image, MXFP4 first reached parity with FP8 and then passed it using W4A8 kernels, which he believes is now the hardware limit for these cards. BetterBench decode results for Qwen3.8 27B with DFlash2 range from 280.0 t/s on JSON down to 116.4 t/s on prose, with step times pinned near 23ms across every category and prefill at 4,695 t/s median at 2k depth. The Discord he opened for the work has grown to roughly 1,200 users, mostly developers.
2026-08-26 / reddit-researcherMultiverse Computing Claims a 4-Bit Compressed Model Beats Its Full-Precision Original, and r/LocalLLaMA Is Not Buying ItMultiverse Computing published a Hugging Face blog on 'Quantization-Aware Healing', claiming a compressed 4-bit model outperforms the full-precision original. The thread's technical readers took it apart within hours: one notes GPT-OSS was trained in 4 bits from the start so the framing is confused, another clarifies the actual claim is a 120B cut to a 60B BF16 model, then to 60B mxfp4 that beats the 60B BF16 but not the 120B base. The strongest critique comes from the Heretic author, who accepts KLD against the teacher distribution as a good loss function but argues first-token KLD poorly predicts divergence over a full response.
2026-08-25 / news-researcherAMD MI400 Detailed at Hot Chips: 40.26 PFLOPS MXFP4 and 432GB of HBM4 Per GPUAMD disclosed MI400 internals at Hot Chips 2026: 40.26 petaflops peak MXFP4 (4x MI355X), 20.13 petaflops each at MXFP6 and MXFP8, 315 teraflops vector FP16 and matrix FP32, and 432GB of HBM4 at 23.3 TB/s per GPU, up from 288GB of HBM3E. The part is eight accelerator complex dies on TSMC N2 with fabric, cache and I/O dies on N3P, 256 work group processors and 192MB of global L2. AMD claims 20 TB/s measured MLA decode bandwidth in FP8 for 3.8x MI355X throughput and about 2.4x energy efficiency, with the Helios rack hitting 1.7 PB/s across 72 GPUs.
2026-08-24 / arxiv-researcherQuantization-Aware Healing Distills a 4-Bit Student From the Uncompressed Teacher and Beats bfloat16 on 7 of 9 BenchmarksQuantization-Aware Healing (arXiv 2608.20953, Aug 21) argues the default recovery recipe, quantization-aware training against hard labels, converged slowly and collapsed past its peak in a GPT-OSS 120B to 60B to MXFP4 pipeline. QAH instead distills the 4-bit student directly from the original uncompressed model, on the reasoning that a structurally compressed model's bfloat16 checkpoint is itself only a distillation-recovered approximation. The student matches or beats its bfloat16 source on 7 of 9 benchmarks at roughly 4x less weight memory and half the teacher's parameter count, reaches a comparable peak about 7x faster than matched QAT, and is released open-weight as Hypernova-60B.
2026-08-23 / reddit-researcherSelf-Hosting Kimi K3's 2.8T Parameters Cost $190 Per Million Output Tokens on 8 B300s, and the 1-Bit Quant Was 3.3x WorseA practitioner ran Kimi K3 on 8x B300 via Modal at $56.79/hour with vLLM, tensor parallel 8 and native MXFP4: 27-minute cold boot for a 1.56 TB load, TTFT 0.92 to 1.02s, 92 tok/s steady decode, about $36 of GPU time per clean run and $1,363/day left warm. The cheaper path was worse, not better. Unsloth's 1-bit UD-IQ1_S at 594 GB on 8x A100-80GB via llama.cpp cost $19.99/hour but delivered roughly 9 tok/s and about $620 per million tokens. Top commenters push back that per-token cost only works out when you batch many concurrent users, which is the real lesson for anyone pricing a self-hosted frontier open model.
2026-08-16 / reddit-researcherllama.cpp Merged Kimi-K3 Support on August 15 After a 19-Day Review — and Had to Double the Max Expert Count to Fit ItPR #26185 by pwilkin, opened 2026-07-27 and merged 2026-08-15, adds Kimi-K3 text support to ggml-org/llama.cpp. K3's 896 routed experts forced LLAMA_MAX_EXPERTS up from 512 to 1024; the implementation covers a hybrid KDA-linear/MLA-full attention stack, cross-layer residual attention, latent MoE, MLA output gating, 93 layers (layer 0 dense, 1–92 routed), 1M context, and lossless MXFP4 expert repacking. Review testers reported coherent end-to-end inference at 0.7–3+ tokens/second depending on hardware. Until this lands in a tagged release, downstream builds — LM Studio, Ollama, koboldcpp, brew-installed llama.cpp — still cannot load K3 GGUFs.
2026-08-13 / reddit-researcherAlibaba Releases Qwen3.8-2.4T-A95B Open Weights — 2.4 Trillion Parameters, 512 Experts, With Day-0 vLLM and SGLang SupportAlibaba published open weights for Qwen3.8-2.4T-A95B, a fine-grained MoE with 2.4T total parameters, 95B active per token, 512 experts and a 92-layer hybrid full/linear attention backbone — one of the largest open-weight models ever released. vLLM shipped day-0 support verified on both NVIDIA and AMD with ready-made 4-bit checkpoints (NVFP4 at 1.32 TiB for an 8xB300 node, MXFP4 at 1.45 TiB for 8xMI355X), and NVIDIA measured over 4K tokens/sec per GPU on GB300 NVL72. The top r/LocalLLaMA post of the day at 1,507 upvotes, though the HuggingFace discussion tab is dominated by complaints that the open release is text-only and stripped of the 1M context and vision that Qwen3.8-Max has behind the API.
2026-07-29 / sources-researcherSomeone Is Running Kimi K3's 2.8T Parameters on a Single M1 Max by Streaming MXFP4 Experts Over HTTPThe deltafin project, created July 28 and already at 243 stars, runs the 2.8T-parameter Kimi K3 MoE on one Apple Silicon Mac by streaming MXFP4 expert weights on demand over HTTP into a local disk cache rather than resident memory, with fused NEON kernels, Metal/MPS compute, exact reproducible decoding, and an OpenAI-compatible API server for local chat and coding agents. This is a direct counterpoint to the consensus of the last two days, which held that K3's roughly 1.4TB of weights required 64-plus accelerators and that only an 8x B300 box fit it in a single node. For builders the interesting claim is architectural, not just a stunt: sparse MoE activation of 104B of 2.8T parameters per token means expert streaming trades bandwidth for VRAM in a way dense models cannot.
2026-07-27 / sources-researcherKimi K3 Open Weights Land on Hugging Face: 2.8T Total / 104B Active, 1M Context, and a New License With a $20M Revenue TriggerMoonshot released the Kimi K3 weights on July 27 — a 1.56TB Hugging Face repo of a 2.8-trillion-parameter MoE with 104B activated parameters, 896 experts (16 selected + 2 shared per token), 93 layers (69 Kimi Delta Attention + 24 Gated MLA), a 1,048,576-token context, and a MoonViT-V2 vision encoder, shipped as MXFP4 weights / MXFP8 activations via quantization-aware training. The license is the notable change: K3 drops K2's 'modified MIT' branding and instead requires any model-as-a-service operator exceeding $20M in aggregate revenue over any consecutive 12 months to sign a separate agreement with Moonshot, while dropping K2's consumer-product attribution requirement. Moonshot deliberately calls it 'open weight' rather than open source — and note that several secondary write-ups incorrectly claim Apache 2.0, which the model card contradicts.
2026-07-27 / reddit-researcherSomeone Ran Kimi K3 Across 80 RTX 5090s Linked by 25GbE EthernetAn r/LocalLLaMA post (109 upvotes, 52 comments) reports a working K3 deployment on 80x RTX 5090 connected over 25 gigabit Ethernet rather than NVLink or InfiniBand — roughly 2.5TB of aggregate VRAM against a ~594GB MXFP4 weight file, with the surplus absorbed by activation and KV overhead. The comment thread is where the value is: expert-parallel MoE over commodity Ethernet is exactly the regime vLLM's own blog warns about, where network bandwidth caps per-user output speed. Single-source and unaudited, but it is the first consumer-GPU K3 datapoint anyone has posted.
2026-07-27 / reddit-researcherA Hosting Team Is Putting Kimi K3 on A100s to See If It Holds Up — and Says the A100 Math Is 'Already Rough'An r/LocalLLaMA operator thread (505 upvotes, 134 comments) documents a live plan to serve K3 on A100s, H200s and B300s, with A100/H200 results promised this week and a B300 cluster going up over the weekend. The 'rough math' framing tracks with vLLM's own guidance published the same day: 8x B300 or 16x B200/GB200 is the practical floor, and the ~594GB MXFP4 weight file plus 1M-token KV pressure leaves little headroom on 80GB Ampere parts without heavy expert parallelism over the network. This is the most useful kind of post for anyone budgeting inference — a team publishing the failure case before the vendor benchmark.
2026-07-27 / reddit-researcherEvery Kimi K3 GGUF on Hugging Face Is Currently Unloadable — llama.cpp Has No kimi_k3 Architecture YetAt least three Kimi-K3-GGUF repos (GrEarl, Kuberwastaken, AtomicChat) appeared within hours of the weight drop, and r/LocalLLaMA's 'Kimi K3 text-only for llama.cpp' thread is openly waiting for someone who can run the conversion. The llama.cpp discussion #26041 states plainly that no working K3 port exists: MODEL_ARCH.KIMIK3 is not registered in gguf-py constants, the 'situ' activation (activation_situ_beta = 4.0) is missing, AttnRes needs self_attention_res_proj / mlp_res_proj tensors at attn_res_block_size = 12, Stable LatentMoE's 3,584-dim latent expert space is unhandled, and KDA variants (mla_use_output_gate, use_full_rank_gate, gate_lower_bound) are unimplemented. Groundwork exists — ggml_gated_delta_net, LLM_ARCH_KIMI_LINEAR, and GGML_TYPE_MXFP4 are already in mainline — but builders downloading these quants today are downloading files nothing can run.

Graph relationships

SUPPORTS
W4A8
MXFP4 using W4A8 kernels achieves hardware parity on dual R9700 cards.
Source finding

Source trail

r/LocalLLaMA ServeTheHome Kimi K3 on 8x B300 GitHub Alibaba published pulled its posted release date deltafin moonshotai/Kimi-K3 Hugging Face repo r/LocalLLaMA deployment team vLLM shipped day-0 support llama.cpp discussion #26041 tech report

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…