Do Sparse Autoencoders Actually Capture Concept Manifolds? Evidence Says Not Always

arXiv·medium signal

Challenges a core assumption in mechanistic interpretability: that concepts in neural networks correspond to independent linear directions extractable by SAEs. Growing evidence shows concepts live on manifolds, not lines, and SAEs may systematically miss or fragment these nonlinear structures. Directly relevant to the Qwen-Scope release and anyone building SAE-based steering or interpretability tools — the linear features you extract may be incomplete projections of richer concept geometry.