Stanford's Transformer Transformer Designs a Robot From a Demonstration Video in Under a Minute vs 3+ Hours for CMA-ES
Huy Ha, Karen Liu, and Shuran Song at Stanford released Transformer Transformer, a single model that takes a manipulation demonstration and emits a complete robot design — every link, joint, motor, and inertial property — then controls the resulting robot to validate it, collapsing generator, critic, and controller into one architecture. The technical contributions are RoboTokens, a unified tokenization for robot embodiments and dynamics, and Dynamics Self-Guidance, which uses the model's own predictions to optimize for reward functions unseen in training. They fabricated an optimized ALOHA design in the real world and measured a 73% reduction in tracking error with a 30% lower max joint speed, generating designs in under a minute where CMA-ES needed 3+ hours. Code is at real-stanford/transformer-transformer.
↳ Follow the thread