Fetching from the wire…
01
02
03
04
05
06
07
08
Infra2026-09-16 · source-backed
Simulating HBM, DRAM and SSD tiers against a random-forest execution-time predictor across chat, agent and document QA workloads, tiering supported 73.02x more concurrent sessions per GPU at 62.04x lower cost per session. The authors attribute the gains to tier capacities of 1 + 8 + 64, not to placement policy. Because decode was compute-bound at batch size one in their setup, policy mainly changed PCIe migration traffic and time to first token, with recency producing 2.30x less migration traffic than alternatives.
Each link below shares sources, entities, or timing with this story.
SOL-ExecBench measures AI-generated GPU kernels against theoretical hardware speed-of-light limits rather than relative rankings. Current agentic systems achieve 40–70% of theoretical hardware efficiency, with clear headroom. As agents increasingly generate and optimize GPU co...
Multi-turn serving hurts because the KV cache grows linearly with conversation length, choking GPU memory and bandwidth. Tangram spends memory unevenly across the cache instead of treating all tokens equally, cutting the footprint of long sessions. Source: arXiv Directly usefu...
Computable GPU Index took #2 with 194 upvotes, publishing USD-per-GPU-hour for H100, H200, B200 and B300 from 28 providers every 15 minutes, with both collection code and calculation methodology on GitHub so anyone can reproduce a published value (Product Hunt). It uses an int...
New batching algorithms enable ~7x, up to 12x+, longer-context GRPO training with no accuracy or speed penalty versus optimized FA3 and chunked-loss setups (Unsloth Docs). Qwen3-8B GRPO reaches 110K context on one 80GB H100 via vLLM plus QLoRA. For solo builders doing reasonin...
Paritok-4B (arXiv 2608.24188) is a LoRA on Qwen3-4B distilled from a gpt-4.1-mini teacher over 67,074 real OpenHands trajectories. It's extractive rather than paraphrasing, with 96.0% of emitted identifiers, paths and numbers already present in its input, and intent-conditione...
Measuring expert time on two datacenter GPU generations shows it is linear in neither token count (EPLB, LPLB, UltraEP) nor activated-expert count (METRO). Below roughly 156 to 168 tokens, HBM weight streaming dominates so cost attaches to activated replicas. Above it, grouped...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.