SPADE Cuts Cloud Model Calls 76% With Zero Accuracy Loss by Running the Draft Model on the Edge

arXiv 2608.13076·medium signalXBlueskyLinkedInCopy link

SPADE (arXiv 2608.13076, 2026-08-13) splits speculative decoding across the edge/cloud boundary rather than within one machine: a compact draft model on the edge proposes tokens, a large cloud verifier validates them in parallel, and only rejections trigger a cloud correction. The design is plug-and-play with no retraining and shifts the bulk of compute to the edge. On SpecBench and CNN/DailyMail across multiple NLP tasks it reduces cloud model calls by 76% with zero accuracy loss versus the full model — a direct cost lever for anyone paying per cloud token in a local-first product.