Orthrus Speculative Decoding Is Only Lossless in FP32: BF16 Matches the Reference Trajectory 45% of the Time

arXiv 2609.15504·medium signalXBlueskyLinkedInCopy link

An independent reproduction tests Orthrus's central claim that its intra-model consensus mechanism produces the same output sequence as the autoregressive backbone. Under BF16 inference, exact trajectory matching occurs in 45% of cases for the authors' checkpoint and 43% for an independently trained model across 1,190 prompts from 12 domains, and match probability tracks the reference model's response-conditional perplexity. Repeating the same evaluation in FP32 yields exact matching on every prompt, and lm-eval-harness scores do not degrade under BF16, so trajectory equivalence and downstream task performance have to be measured separately.