Do VLMs Need Vision Transformers? State Space Models Say Maybe Not

Researchers systematically evaluate whether Mamba-class state space models can replace ViT encoders in large VLMs, finding competitive performance with linear-time processing versus ViT's quadratic attention. Meaningful memory savings on high-resolution or long-context vision tasks. Opens a practical architecture alternative for memory-constrained vision workloads.