Fetching from the wire…
01
02
03
04
05
06
07
08
Infra2026-09-19 · source-backed
PR #44890 adds the method, a POST /release_kv_cache_memory endpoint and async variants, discarding only kv_cache-tagged allocations while keeping weights resident and pausing the scheduler until wake_up(). The old workaround was sleep(level=2) then wake_up(tags=['weights']), which threw away the weights too. Equivalence was validated on Qwen3-0.6B on an H200 with identical token sequences after release and restore. vLLM
Each link below shares sources, entities, or timing with this story.
PR #19378 landed in llama.cpp this week, and I think most people are underselling what it means. Backend-agnostic tensor parallelism via --split-mode tensor makes multi-GPU inference work across AMD, Intel, and Apple Silicon. Not just CUDA. Everything. For context: llama.cpp h...
The abliteration tool gained 215 stars to reach 30,103, but the stronger signal is downstream: the HF trending endpoint returns DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU and Momoking/Qwen3-VL-32B-Heretic-MiniMax-H3-NVFP4, both naming the too...
PR #21038 adds graph-level Hadamard rotation of Q/K/V before caching, doing attention in the rotated space then rotating back, which makes standard quant types far more accurate in the KV cache. The benchmarks are not subtle: Qwen3 0.6B q5_1 KV perplexity drops from ~61.7 to ~...
Quesma ran the model across GPQA Diamond, IFBench and Terminal-Bench 2.1 (89 agentic coding tasks) on L40S, H100 and H200 via Modal. Q4_K_M at 17 GB matched BF16 at 55 GB within a point on all three. UD-Q2_K_XL at 10.7 GB held instruction-following but dropped Terminal-Bench f...
The authors define goal-directed execution as four repeated behaviors: selecting goals, constructing task-relevant state, maintaining fidelity to higher-level objectives, verifying completion against the environment. Post-training Qwen3.5-122B-A10B on 363 long-horizon multi-to...
langchain-typesafe 0.0.1a1, merged September 17, implements the POST /v1/systemone contract directly against httpx2 rather than wrapping typesafe-sdk, reversing LangChain's standing policy. The PR body's stated reason: wrapping was a response to limited maintainer attention, a...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.