SelectInfer Loads Only the Neurons That Matter, Cutting On-Device LLM Memory Without Retraining

arXiv·low signal

arXiv:2607.18081 (July 20, Kabakibo, Schniedermeyer, Burchanow and Wang) attacks edge deployment at the neuron level with two mechanisms: selective loading, which uses an offline profiler to identify task-specific and general-purpose neurons and loads only the critical ones, and selective computation, which evaluates only relevant neurons at runtime. The approach requires no retraining or fine-tuning, and reports significant memory and compute reductions while preserving task performance. The abstract does not publish headline percentage figures, so treat the magnitude as unverified — but the no-finetune constraint makes it unusually easy to test against an existing local quantized model.