'Attention Is All You Need' Co-Author Argues It's Time to Move Past Transformers — 251-Upvote Debate With 111 Comments
r/singularity·high signal
A post on r/singularity (251 upvotes, 111 comments) highlights one of the original authors of the 2017 'Attention Is All You Need' paper arguing that the field should move beyond the transformer architecture. The discussion references Pathway's Post-Transformer debate, where the co-author laid out limitations of the current paradigm and directions for successor architectures. The 111-comment thread shows deep technical engagement with practitioners debating whether transformers have peaked.