MIST Prunes Chain-of-Thought Using the Model's Own Residual Stream Instead of an External Scorer

arXiv 2608.31066·medium signalXBlueskyLinkedInCopy link

Token-level chain-of-thought compression hinges on picking which reasoning tokens to keep, and existing methods lean on external scorers or heuristics only loosely connected to how the model actually computes its answer. MIST scores each token from inside the model, treating the ripple it leaves in the residual stream as its contribution, and combines two axes: necessity, the drop in answer likelihood when that internal contribution is removed, and sufficiency, the gain when it alone is supplied. Across four reasoning benchmarks and four models it consistently beats the baseline selection methods, supporting the view that model-internal saliency is a better proxy for token importance than anything measured from outside.