GELO: Good-Enough LLM Obfuscation for Shared Accelerators

arXiv 2603.05035·high signalXBlueskyLinkedInCopy link

Addresses a real deployment problem: adversaries with read access to shared GPU memory can observe KV caches and hidden states, compromising prompt privacy. GELO uses per-batch invertible mixing (random matrix A transforms hidden states before offloading, A-inverse applied on return). On Llama-2 7B, preserves float32 accuracy with ~20-30% latency overhead, resisting ICA/BSS and anchor-based attacks. Practical for anyone deploying on shared inference infrastructure where full FHE/MPC is too expen