Jim Fan's DreamDojo: Open-Weight Neural Simulator for Robot Policy Evaluation, +17% Real-World Success

LinkedIn·high signal

NVIDIA's Jim Fan released DreamDojo, an open-weight neural simulator built on NVIDIA Cosmos that enables live VR teleoperation inside dream simulations. The system demonstrated +17% real-world success on a fruit packing task through model-based planning — sampling multiple action proposals and simulating them in parallel. Running on a single RTX 5090 with a PICO headset and Unitree G1, the team open-sourced everything: weights, code, post-training dataset, eval set, and whitepaper.