A 30-Line Metal Capability Shim Makes llama.cpp 16x Faster Inside macOS VMs on Apple Silicon
The trycua/cua team published measurements on GPU passthrough for macOS guests using a process-scoped Metal capability shim that lies about two values — reporting Apple GPU family 9 instead of 5, and 64 KB instead of 32 KB max threadgroup memory — which lets llama.cpp select SIMD-group matrix and bfloat16 kernels while staying on Apple's existing paravirtualized GPU path. On an M1 Ultra host (macOS 26.6.1) with a Tahoe guest (macOS 26.5.2) under Lume 0.5.1, TinyLlama 1.1B token generation went from 12.63 to 206.60 tok/s (16.36x) on 2026-08-09, Gemma 4 12B from 3.41 to 49.67 tok/s (14.54x) on 08-10, and Meta Muse Glimmer 30B from 2.38 to 21.08 tok/s (8.87x) on 08-11. 297 points, 43 comments — this makes sandboxed local-model agents on Mac hardware actually usable.
↳ Follow the thread