Orthrus-Qwen3-8B Achieves 7.8x Tokens/Forward Pass via Dual-View Diffusion — Provably Identical Output Distribution
r/LocalLLaMA·high signal
Orthrus introduces a frozen-backbone approach that generates up to 7.8x tokens per forward pass on Qwen3-8B with provably identical output distribution and <1% GPU memory overhead (~4.5 MiB constant KV cache). Unlike speculative decoding with a separate draft model, Orthrus conditions a diffusion head directly on the AR head's causal cache, achieving ~6x speedup over baseline with strictly lossless performance. Paper on arxiv (2605.12825), code on GitHub.