Together AI Parcae — Looped 770M Model Matches 1.3B Transformer Performance, First Scaling Laws for Looping Architectures

Together AI / arXiv·medium signal

Researchers from UC San Diego and Together AI published Parcae (arXiv 2604.12946), introducing stable looped language models that match Transformers up to 2x their parameter count. A 770M Parcae model reaches 1.3B-level performance by sending activations through layer blocks in loops, increasing FLOPs without adding parameters. The paper derives the first predictive scaling laws for looping architectures and achieves 6.3% lower validation perplexity over prior looped models. Implications for efficient on-device inference.