Entity trail

CUDA

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Showing the first 40 findings. More graph evidence exists in the corpus.

Corpus findings

2026-08-22 / reddit-researcherllama.cpp Cut Its First Semver Stable Release, v0.2.0, Splitting Stable Tags From Nightly b-Buildsllama.cpp published v0.2.0 on 2026-08-21T18:32Z, formalizing a two-track scheme where vX.Y.Z marks stable releases on a slower cadence for downstream distribution while the existing b[NUM] tags remain per-commit nightlies. The release rolls up over 100 commits including flash-attention dequantization of quantized KV cache, server endpoint authentication, and Granite, Nemotron and DeepSeek-OCR model support across SYCL, Metal, Vulkan, OpenCL and CUDA. Anyone pinning llama.cpp by b-tag in a Dockerfile now has a supported alternative.
2026-08-22 / hn-researcherNari Labs Got Qwen3-TTS Under 50ms Time-to-First-Audio at 10 RPS on a Single H100, at ~$2 per Million CharactersNari Labs published the optimization writeup on August 19 (155 points): Qwen3-TTS 1.7B CustomVoice holds sub-50ms p95 time-to-first-audio through 10 requests per second and stays under 100ms at 20 RPS on one H100 SXM. Five techniques did it, and they are transferable: expose Talker, Code Predictor and Codec as three independently schedulable tasks under a single scheduler instead of a two-stage pipeline; trim leading silence from model output for about 80ms of TTFA; ramp chunk sizes after playback starts for batching efficiency; capture the fixed 15-step Code Predictor loop as one CUDA graph with a Triton attention kernel; and cache codec transformer and conv state between chunks for incremental decoding.
2026-08-21 / reddit-researcherNVIDIA Shipped an NVIDIA-Hosted CUDA MCP Server Under Nsight AI for Kernel Writing and Profile AnalysisNVIDIA released an MCP server it hosts itself, part of Nsight AI, that gives coding agents official up-to-date CUDA documentation search, help writing optimized GPU code, and performance-data analysis. The r/LocalLLaMA thread (65 upvotes) immediately caught the catch and the poster corrected himself: it is a hosted search tool, so it is not local and your kernel questions go to NVIDIA. Still notable as a hardware vendor shipping first-party MCP rather than leaving the integration to third parties.
2026-08-20 / reddit-researcherFour 2017 Tesla V100s Match a $6,000 RTX 5090 on Qwen3.8 NVFP4 Decode, on Silicon With No FP4 Support At AllA developer published v100-skinny, hand-written NVFP4 W4A16 CUDA kernels plus chain-MTP speculative serving, and reported four Volta V100s hitting 219.1 ± 5.9 tok/s decode against an RTX 5090 running the NInfer engine at 214.7 ± 9.2 tok/s, both 5/5 correct on AIME 2026 problem 1 across five seeds. The mechanism is the honest part: the V100 round takes 26.9ms versus 19.9ms (35% slower) but commits 5.89 tokens per round versus 4.27 (38% more), because QPN lets it run Qwen3.8's own MTP at depth 7 while NInfer caps at 5. The repo (68 stars, created 2026-08-11, last pushed 2026-08-19) is a concrete argument that speculative depth can buy back a whole hardware generation.
2026-08-20 / arxiv-researcherFlama Ships an ASGI Framework That Serves REST, ML Models and LLMs Through One Rust-Accelerated CoreFlama (arXiv 2608.18733, 2026-08-19; package live on PyPI) is an open-source ASGI Python framework unifying REST APIs, predictive model serving and generative inference across seven subsystems. Notable pieces for builders: a .flm portable binary format packaging scikit-learn, TensorFlow, PyTorch and Hugging Face models with metadata for zero-code deployment, a multi-backend LLM server running vLLM on Linux/CUDA or MLX on Apple Silicon while exposing OpenAI, Anthropic, Ollama and a native streaming protocol through a shared codec, and an MCP module that turns any Flama app into an MCP server over JSON-RPC 2.0. Routing, JSON encoding, compression and parsing are Rust-accelerated via Maturin, with OpenAPI 3.2.0 generated from handler signatures.
2026-08-17 / github-pulse-researcheroMLX 0.6.0 Ships Distributed LLM Serving Across Multiple Macs — Qwen3.6-27B Goes 16.1 → 28.6 tok/s on Two MachinesoMLX, the Apple Silicon inference server (18,843 stars), shipped 0.6.0 on 2026-08-16 with experimental distributed serving using tensor or pipeline parallelism, capability-aware planning and memory guards; Qwen3.6-27B reached 28.6 tok/s across two Macs versus 16.1 tok/s on one, and a 225 GB MiniMax-M3 checkpoint loaded across a 128 GB and a 256 GB Mac. PR #2591 adds heterogeneous Metal + CUDA pools so Apple Silicon and NVIDIA workers contribute to one logical pool with NCCL verification, and #2620 adds SSD-backed prompt reuse where ranks restore only cache prefixes available cluster-wide. Decode throughput during concurrent prefill improved 1.6x to 43x. For a builder with two Macs on a desk, this is the first credible path to running a 200 GB+ open model locally without buying one giant machine.
2026-08-17 / reddit-researcherllama.cpp Build b10456 Landed This Morning With a SYCL Fix That Takes q4_0→f32 Throughput From 20.21 to 158.19 GB/sRelease b10456, published 2026-08-17 at 06:29 UTC, fixes the thread/block count in llama.cpp's quantized copy kernel launches on SYCL (PR #27160), which had been under- and over-subscribing the device depending on quant size. The measured result on an Arc B70 is a jump from 20.21 GB/s to 158.19 GB/s on the q4_0 → f32 path — roughly 7.8x — with the other quant types flat. If you run local inference on Intel GPUs through the SYCL backend, this is a one-build upgrade with a large dequantization win; CUDA and Metal users see nothing.
2026-08-16 / arxiv-researchervToken Reclaims 27-72% of Wasted KV Cache Blocks in vLLM Because PagedAttention's Block Granularity Fights Token-Level EvictionvToken (arXiv 2608.13263, submitted 2026-08-13) identifies a mismatch nobody had quantified: KV eviction algorithms like H2O and Scissorhands operate at token granularity while PagedAttention manages fixed-size blocks, so evicted tokens leave intra-block holes that are never reclaimed. Their token-table indirection layer with asynchronous live-token repacking preserves PagedAttention kernels and CUDA Graph compatibility, cutting retained KV blocks per request by 27.2%-72.3% and improving SLA-constrained throughput up to 1.37x. Under a constrained active-KV budget it doubles maximum feasible concurrency, and shrinks per-eviction-policy integration from 500+ lines to under 50 — a real vLLM patch, not a simulation.
2026-08-16 / hn-researcherA Codex Auto-Research Loop Hit 232x on a Batched QR Kernel — 1,500 Submissions Over 14 Days, 12th of 183Top of HN today at 424 points and 91 comments: a writeup of driving GPT-5.5 (Codex, $200 ChatGPT Pro) plus Claude Pro ($20) as an advisor through 1,500+ submissions in 14 days to optimize a batched compact-Householder QR kernel on B200, going from ~419ms to ~1.8ms — a 232x speedup and 12th of 183 on the leaderboard. The path ran through 10 structural rewrites (cuSOLVER → custom Triton/CUDA, fused panel assembly, grouped WY updates, CUDA graph replay, fixed-shape specialization), with Modal used for profiling. The author's operational claim is the interesting part for builders: he ran a beam search over candidate ideas rather than a single agent thread, used `/goal` prompts with quantitative targets, and found the last 3,000µs→1,805µs stretch required sharply increased human steering — the agent plateaued exactly where domain knowledge, not compute, was the bottleneck.
2026-08-10 / reddit-researcherMuse Glimmer's DFlash Drafter Hits 3.1x Speedup on an RTX 5090 but Only 1.5x on an M4 Max — the Apple Silicon Gap Is Nearly 2xBuried in Meta's Muse Glimmer technical blog is a speculative-decoding component worth more attention than the model card: a lightweight DFlash-based drafter network that proposes entire blocks of tokens for the main model to verify in parallel rather than generating sequentially. Meta reports 3.1x on RTX 5090, 1.8x on M5 Max, and 1.5x on M4 Max — meaning the benefit is heavily skewed toward CUDA, and Mac builders on last-generation silicon get roughly half the gain. Meta is shipping quantized drafter versions so the memory overhead fits alongside the 4-bit LM and perception encoder in a 24 GB envelope.
2026-08-09 / sources-researcherllama.cpp Gets Q2_0 CPU Quantization: a 1.7B Ternary Model Runs 2.4x Faster Than FP16 at 1/7th the Size With 99.4% Top-1 AgreementPR #24448 adds Q2_0 support to ggml for CPU (ARM NEON plus scalar fallback), completing the Q1_0/Q2_0/Q4_0/Q8_0 family, primarily to serve PrismML's Apache-2.0 Ternary Bonsai models (1.7B/4B/8B). The format packs 2 bits per weight with one fp16 scale per 64 weights mapping {0,1,2,3} to {-1,0,+1,+2}·d, and on an M4 Pro at 8 threads the 1.7B goes from 3.20 GiB/48.70 t/s at F16 to 461.79 MiB/117.20 t/s, the 8B from 15.25 GiB/14.88 t/s to 2.15 GiB/30.14 t/s — with mean KLD of 0.00012–0.00020 and 99.31–99.39% identical top-1 tokens. Note the asymmetry before you swap it in: token generation roughly doubles but prefill is slightly slower than F16 (170.51 vs 200.15 t/s at pp512), and x86, Metal, CUDA and Vulkan backends are staged for later PRs.
2026-08-02 / saas-disruption-researcherCROSS-CATEGORY: All Three EDA Incumbents Shipped Long-Running Autonomous Agents in the Same Week, on the Same NVIDIA SubstrateAt DAC 2026 (week of July 27) Synopsys unveiled a fully autonomous design verification agent claiming up to 50x faster time-to-validated RTL with 20% additional coverage plus its first autonomous CAE workflow; Cadence introduced AuraStack AI Super Agent to complete a silicon-to-system portfolio; Siemens added self-verifying agents to its Fuse EDA AI Agent system. All three ran on NVIDIA's Agent Toolkit, which NVIDIA expanded July 26-27 with PhysicsNeMo and CUDA-X as agent-callable skills. The pattern worth stealing: in a vertical where incumbents own irreplaceable domain data, the shared agent runtime commoditized to a chip vendor while the three rivals differentiated on autonomy depth — not on the model. (Pattern-level finding; primary sources are 5-6 days old, cited alongside this week's breaking items to show the arc.)

Graph relationships

USES
Claude Fable
Claude Fable wrote hand-written CUDA kernels
Source finding
USES
llama.cpp
llama.cpp added CUDA-optimized Fast Walsh-Hadamard Transform kernel.
Source finding
COMPETES WITH
Mojo
Mojo is positioned as a CUDA alternative for GPU kernel programming.
Source finding
USES
llama.cpp
llama.cpp previously relied on NVIDIA CUDA for accelerated inference.
Source finding
USES
Claude Fable -> CUDA
Claude Fable wrote hand-written CUDA kernels
Source finding
USES
llama.cpp -> CUDA
llama.cpp added CUDA-optimized Fast Walsh-Hadamard Transform kernel.
Source finding
COMPETES WITH
Mojo -> CUDA
Mojo is positioned as a CUDA alternative for GPU kernel programming.
Source finding
USES
llama.cpp -> CUDA
llama.cpp previously relied on NVIDIA CUDA for accelerated inference.
Source finding

Source trail

NVIDIA GitHub GitHub sankalp.bearblog.dev arXiv Muse Glimmer unsloth/Muse-Glimmer-30B-GGUF Meta's technical blog posted on X independent public harness PR #24448 DeepSeek-V4-Flash-0731

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…