Tight Sample Complexity of Transformers

arXiv·medium signal

Tightly characterizes the VC dimension of depth-L Transformers with W total parameters mapping length-T sequences to a single output, proving an upper bound of O(L·W·log(TW)) and a near-matching lower bound. Closes a gap in the theory of how much data Transformers fundamentally need to learn. Foundational theory result — matters for researchers reasoning about generalization and the data-efficiency limits of attention-based models.