FASTER Cuts RL Test-Time Compute Cost via Value-Guided Sampling Instead of Exhaustive Candidate Generation

arXiv·medium signal

Dong, Swerdlow, and Sadigh propose FASTER, a method that replaces expensive test-time scaling (sampling many action candidates and selecting the best) with value-guided sampling that directs generation toward high-value regions of the action space. This addresses the prohibitive compute cost of top-performing RL algorithms that rely on candidate overgeneration. Directly useful for practitioners deploying RL agents where inference budget matters.