Layer pruning as an Ising problem beats block-influence heuristics by 23 MMLU points.

Multiverse Computing turns "which transformer blocks do I delete" into constrained binary optimization on an Ising glass, using a second-order Taylor expansion so the Hessian's off-diagonal entries become pairwise couplings between blocks, which is the interaction heuristics ignore. Removing 40 of 80 blocks from Llama-3.3-70B-Instruct, the correlated method holds 76.9 MMLU against 82.2 for the full model, while block-influence pruning falls to 54.0. Small cases solve by GPU brute force over up to ~29 billion configurations, larger ones by an open-source tabu solver in seconds. (Hugging Face)