Fetching from the wire…
01
02
03
04
05
06
07
08
Public story · 2026-08-17 · high
Qwen3.6-27B throughput rose from 16.1 to 28.6 tokens per second, and a 225GB checkpoint loaded across two Macs at once.
Why now: oMLX shipped 0.6.0 on August 16, with the CUDA-pooling and cache-reuse pull requests merged alongside it.
A 225GB MiniMax-M3 checkpoint loaded across a 128GB Mac and a 256GB Mac on August 16, per oMLX's 0.6.0 release. Neither Mac held the model alone, so the update splits it by tensor or by layer stage across the pair. That opens 200GB-plus checkpoints to anyone with two machines instead of one giant one.
Qwen3.6-27B throughput rose from 16.1 to 28.6 tokens per second running split across the two machines. Decode speed during concurrent prefill improved 1.6x to 43x, per the release notes.
The Apple Silicon inference server has 18,843 GitHub stars. Two pull requests extend the same idea. PR #2591 adds heterogeneous pooling, letting Apple Silicon and Nvidia GPUs join one logical worker pool, with NCCL verification. PR #2620 adds SSD-backed prompt caching, so each worker restores only the prompt prefixes the whole cluster already shares.
The release also adds capability-aware planning and memory guards, placing model layers based on what each machine has free rather than splitting evenly. None of this needs a rack or a subscription. It's just two Macs and a checkpoint too big for either one alone.
Whether the CUDA-pooling code in #2591 gets used outside all-Mac setups is the next thing to check.
Each link below shares sources, entities, or timing with this story.
NVIDIA uses Claude Code / Shared entities / Same source domain / Earlier coverage
Linked by a graph relationship (NVIDIA uses Claude Code); both cover GitHub, Metal, NVIDIA; reported by the same outlet (github.com).
NVIDIA invested in OpenAI / Shared entities / Same source domain / Earlier coverage
Linked by a graph relationship (NVIDIA invested in OpenAI); both cover Apple Silicon, LLM, Qwen3; reported by the same outlet (github.com).
NVIDIA released Blackwell / Shared entities / Same source domain / Earlier coverage / Downstream implication
Linked by a graph relationship (NVIDIA released Blackwell); both cover Apple Silicon, CUDA, NVIDIA, Qwen3; reported by the same outlet (github.com).
Alibaba partners with NVIDIA / Shared entities / Earlier coverage / Tension
Linked by a graph relationship (Alibaba partners with NVIDIA); both cover GitHub, Macs, Qwen3; earlier GitHub coverage from 2026-04-23.
NVIDIA invested in OpenAI / Shared entities / Same source domain / Earlier coverage / Tension
Linked by a graph relationship (NVIDIA invested in OpenAI); both cover GitHub, LLM; reported by the same outlet (github.com).
NVIDIA invested in OpenAI / Shared entities / Earlier coverage
Linked by a graph relationship (NVIDIA invested in OpenAI); both cover CUDA, NVIDIA, Qwen3; earlier CUDA coverage from 2026-08-10.
Linked by a graph relationship (NVIDIA invested in OpenAI); both cover CUDA, GitHub, NVIDIA; earlier CUDA coverage from 2026-07-27.
NVIDIA uses Claude Code / Shared entities / Same source domain / Earlier coverage / Tension
Linked by a graph relationship (NVIDIA uses Claude Code); both cover GitHub, LLM; reported by the same outlet (github.com).