MTPLX triples Qwen inference speed on Apple Silicon

MTPLX gets roughly 3x speedups running Qwen 3.8 27B on Apple Silicon, according to its GitHub repository.

For anyone running Qwen locally on a Mac, that's the gap between speculative decoding being worth the memory cost and not. Unified memory is already the constraint that decides whether local inference is fast enough to use.

Speculative decoding normally needs a second, smaller draft model to guess ahead of the main model, which then checks the guesses. On a Mac with unified memory, that draft model competes with the main model for RAM. Tuning the two models to agree with each other is fiddly too. MTPLX skips the separate model. It uses Qwen's own multi-token prediction head to generate draft tokens instead. There's no second model to load and no alignment tuning between mismatched models.

The project has 1,585 stars since going up in May, and its last push was August 22. It's a smaller effort than mainline MLX tooling, built by one developer, youssofal, rather than a maintained framework team.

Worth watching whether this gets folded into general MLX tooling. Or whether it stays a per-model trick that has to be rebuilt for every architecture that ships its own prediction head. The repo doesn't say.