ATLAS Automates Per-Layer FHE Approximation for Transformers, Searching a 10^225 Configuration Space for LLaMA3

arXiv 2607.23478·medium signal

Deploying transformers under fully homomorphic encryption is bottlenecked by polynomial approximations of softmax, normalization, and activation, whose multiplicative depth dominates inference cost — and every prior framework applies one manually tuned configuration uniformly across all layers. ATLAS (arXiv 2607.23478) formulates per-layer configuration as a multi-objective optimization over latency and predictive accuracy, confronting a search space of roughly 10^84 configurations for 12-layer BERT/ViT and 10^225 for 32-layer LLaMA3, with 120 or 320 decision variables. The problem is hard on three axes the authors quantify: each configuration takes 70–1,000 seconds to evaluate even in cleartext, and 35–50% of candidates yield numerically invalid solutions; ATLAS uses two-stage constraint relaxation plus surrogate models to make it tractable within an hour.