Entity trail

ROCm

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-07 / saas-disruption-researcherAMD Bought Taalas to Etch Model Weights Into Metal Layers, Weeks After Its Cerebras DealAMD announced a definitive agreement on August 6, 2026 to acquire Taalas, a Toronto startup founded in 2023 that builds 'Hardcore Models' — processors physically specialized to one model's weights by finalizing only two of a chip's 100-plus metal layers, with a claimed tape-out time of roughly two months. Taalas' HC1 served Llama 3.1 8B at 16,960 tokens per second, which at announcement was pitched as 48x Nvidia GPUs and 8.5x Cerebras. Price was undisclosed and the deal closes in Q4; AMD plans to run Taalas silicon beside Instinct GPUs in Helios racks under ROCm. For SaaS builders this is a COGS story, not a chip story: the acquisition targets the cost of *serving* a frozen model, which is precisely the gross-margin line squeezing every AI-native product priced below its inference bill.
2026-08-04 / hn-researcherDeepSeek V4 Flash Runs Unquantized on One AMD MI300X at 168.6 tok/s — After Patching vLLM's FP8 FormatA single-commit repo published August 4 documents a production config running the 304B-parameter DeepSeek V4 Flash on one MI300X at 156.67 GB in memory with no quantization: 168.6 tok/s single-stream decode, 7.9–8.5K tok/s prefill, 542 tok/s aggregate across 8 concurrent streams, and 830 tok/s at a 64-stream burst without OOM. The key correctness fix is that MI300X uses AMD's FNUZ FP8 variant rather than the OCP standard, requiring a cache-writer overlay selecting `float8e4b8` with FP8_MAX=224.0 and shuffled write offsets on ROCm. Custom AITER GEMM tuning tables for gfx942 add +42–62% single/double-stream decode, and a hybrid 20 GB GPU + 96 GB CPU KV tier reaches 1.93M-token capacity. Only 18 stars — early, single-author, Apache-2.0.
2026-08-02 / vibe-coding-researcherFresh llama.cpp Bugs Cut DeepSeek V4 Flash Throughput 6.4x on ROCm and Corrupt Quantized KV CacheTwo eval bugs filed in the last 48 hours limit DeepSeek V4 Flash as a local coding-agent backend. Issue #26399 (2026-08-01) reports GGML_OP_TOP_K falling back to CPU on HIP/ROCm above roughly 3-4K context, costing a 6.4x token-generation loss exactly where agent contexts live. Issue #26423 (2026-08-02) reports quantized KV cache still producing garbage on master even with #25202, and points at the arch-scoped rotation disable from #25391 as the fix. Both are open.
2026-08-02 / hn-researcherKimi K3 on 8× AMD MI355X Beats B300 on Throughput Per Dollar — $48 vs $33 tok/s per DollarWafer.ai's July 31 writeup runs the 2.8T-parameter Kimi K3 at TP8 on a single MI355X node, hitting 952 tok/s aggregate, 118 tok/s single-stream decode and roughly 13k tok/s steady-state prefill after optimization work on missing PyTorch sampling functions and AITER MLA prefill kernels via SGLang and ROCm. Against a two-node TP16 B200 deployment that's 3.8× the aggregate throughput per node and over 1.3× single-stream decode; B300 delivers 1.65× higher aggregate throughput but costs 2.4× more per GPU, landing MI355X at $48 tok/s/$ versus B300's $33 at $2.50/GPU-hr. The caveat worth flagging: the numbers come from a vendor-adjacent blog, not an independent benchmark house.
2026-08-02 / projects-researcherantirez's ds4 Adds DeepSeek V4 PRO Support and CUDA/ROCm Backends — 19,834 Stars, +150 Today, 26 tok/s on a 128GB M3 MaxSalvatore Sanfilippo's single-file C inference engine for DeepSeek V4 Flash has widened from Metal-only to Metal, CUDA and ROCm, and now covers V4 PRO as well as Flash, with a last push on 2026-08-01. It is deliberately narrow — not a GGUF runner, not a wrapper — shipping DS4-specific model loading, KV cache handling, prompt rendering and an OpenAI-compatible server. Third-party reviews report the 284B MoE running at 26 tokens/sec at 50W peak draw on a MacBook Pro M3 Max with 128GB under antirez's own Q2 quantization.
2026-07-28 / github-pulse-researchervoicebox Reaches 47,142 Stars With Seven Local TTS Engines and LuxTTS Running 150× Realtime on CPUjamiepine/voicebox added 3,059 stars this week to hit 47,142, bundling Qwen3-TTS, Qwen CustomVoice, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo (a 350M model), HumeAI TADA and Kokoro into one MIT-licensed local voice studio. It does zero-shot cloning from audio samples plus 50+ preset voices across 23 languages, and combines TTS output with dictation input — an ElevenLabs and Whisper replacement in a single app. Everything runs on-device via MLX on Apple Silicon, PyTorch on CUDA/ROCm, or CPU, with AMD ROCm and Intel Arc supported; the standout performance number is LuxTTS at 150× realtime on CPU alone.
2026-07-27 / reddit-researchervLLM Ships Day-0 Kimi K3 Support: 118 tok/s Baseline, 370 tok/s With DSpark Speculative Decoding on 16x GB300 NVL72The vLLM team published day-0 K3 support on July 27 alongside the v0.26.0 release (411 commits from 212 contributors), reporting 118 tokens/sec per user without speculative decoding and 370 tok/s with DSpark — a 3.14x improvement — on 16 NVIDIA GB300 NVL72 GPUs. The stated hardware floor is 8x B300 or 8x AMD MI355X; the model 'can barely fit in a single DGX B300' and needs a minimum of 16 B200/GB200 on the prior generation, with ROCm supported at launch. Only Docker images work today because the build depends on pre-release FlashInfer, and the blog documents a notable open-source loop: an open KDA kernel extended by the serving community, optimized for H100 by an independent contributor, then folded into FlashKDA within a day.
2026-07-24 / reddit-researcherAMD May Put Up to $5 Billion Into Anthropic as Anthropic Commits to 2 Gigawatts of Instinct MI450Reuters reported July 23 that AMD may invest up to $5B in Anthropic, released against deployment milestones, with Anthropic deploying up to two gigawatts of Instinct MI450 accelerators beginning in 2027. It is the mirror-image structure of the Nvidia/OpenAI and Microsoft/OpenAI arrangements — compute vendor capital flowing to the lab that commits to buy the silicon — and it gives AMD its first frontier-lab anchor tenant at gigawatt scale. For anyone tracking whether CUDA's moat erodes, a two-gigawatt Anthropic commitment is a far stronger ROCm signal than any benchmark.
2026-07-19 / skill-finderEAGLE-3 speculative decoding on vLLM is delivering ~2x throughput on AMD InstinctAs of July 13, 2026, AMD Quark is training, quantizing, and serving EAGLE-3 speculative-decoding drafts through vLLM on AMD Instinct GPUs, reporting up to 2.00x throughput for Kimi-K2.5 and 1.79x for MiniMax-M2.5. Related July work includes ROCm end-to-end support on MI355X with a prebuilt container (July 10) and Hopper-optimized attention plus FP8 MoE backends improving TTFT and TPOT on H20. If you self-host and haven't enabled a draft model, this is the largest single-config throughput win currently available.
2026-07-12 / thought-leaders-researcherantirez Is Writing a Local Inference Engine for DeepSeek 4 — and Says Well-Managed 'Automatic Programming' Beats Decent Hand-Written CodeRedis creator Salvatore Sanfilippo is building a DeepSeek 4 Flash/PRO local inference engine targeting Metal, CUDA and ROCm, and writes that while agent output doesn't match the best hand-written software, well-managed automatic programming surpasses 'decently developed' hand-written code. He describes using Opus plus GPT-5.x to draft specs and system-programming code even on high-stakes C. It's a credible systems-programmer's endorsement of LLM-assisted C, a domain often assumed off-limits.
2026-07-08 / hn-researcherZML Releases LLMD Alpha — a Zig-Based, Chip-Agnostic Inference Server Across 5 ArchitecturesFrench startup ZML released LLMD (alpha) on 2026-07-08 (covered by TechCrunch), a self-contained inference server running Llama, Gemma, Qwen, and Mistral transparently across NVIDIA CUDA, AMD ROCm, Google TPU, Intel oneAPI, and Apple Metal. Instead of wrapping Python around CUDA kernels like vLLM/Ollama, ZML uses Zig + MLIR + OpenXLA to compile model graphs into standalone native binaries with zero Python deps. It supports continuous batching, paged attention, tensor-parallel sharding, and prefix caching, but is explicitly not production-ready yet.
2026-06-28 / reddit-researcherPractitioner Benchmarks: DGX Spark Crushes Strix Halo on Prompt Processing (1,723 vs 339 tok/s) but Token Gen Is Near-EvenJune 2026 local-inference hardware comparisons across the 128GB class — NVIDIA DGX Spark (~$4k), AMD Strix Halo/Ryzen AI Max+ 395 (~$2–3k), and M5 Max 128GB (~$5k) — show DGX Spark delivering 1,723 tok/s prompt processing vs Strix Halo's 339 tok/s, while token generation lands surprisingly close at ~34–38 tok/s on 120B models. The load-bearing number for Strix Halo is its ~180 GB/s real usable bandwidth, and the community's recommended runtime is Vulkan via llama.cpp, not ROCm. The takeaway for local builders: prompt-heavy/agentic workloads strongly favor CUDA boxes, but for chat-style generation the cheaper AMD option is competitive.

Graph relationships

USES
Kimi K3
Kimi K3 inference on MI355X relies on ROCm for GPU support
Source finding
RELEASED
vLLM
vLLM announced ROCm as an experimental AMD backend with self-contained bundle including PyTorch ROCm.
Source finding
SUPPORTS
vLLM
vLLM supports AMD ROCm acceleration.
Source finding
USES
Kimi K3 -> ROCm
Kimi K3 inference on MI355X relies on ROCm for GPU support
Source finding
RELEASED
vLLM -> ROCm
vLLM announced ROCm as an experimental AMD backend with self-contained bundle including PyTorch ROCm.
Source finding
SUPPORTS
vLLM -> ROCm
vLLM supports AMD ROCm acceleration.
Source finding

Source trail

Announced August 6 GitHub DeepSeek-V4-Flash-0731 CostPerPrompt #2 on Product Hunt August 1 llama.cpp PR #26398 #26399 #26423 r/LocalLLaMA member ds4 Wafer.ai GitHub

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…