FlashRT has agents manage latency across five models

Agarwal, Chen and Qin published FlashRT, a framework that has agents wire together ASR, LLM, TTS and video generation models into one real-time chain instead of running a single model.

The pitch: for voice agents and interactive video, the hard part was never any one model's output quality. It's that five models now share one latency ceiling, and a slow stage anywhere in the chain burns budget meant for every other stage, per the paper.

FlashRT treats the agent as the engineer who has to negotiate that budget across stages, not just as a wrapper that calls a model and waits. That's a different job description than most agent frameworks assume.

The paper doesn't say what latency numbers the framework hits in practice, or whether anyone's run it against a production voice stack. Without that, it reads as a framing contribution more than a benchmark win.

Still, the framing holds up without the numbers. Teams building voice or video agents tend to tune the LLM and call it done, while ASR and TTS quietly eat the rest of the latency budget. FlashRT's bet is that managing the interaction between stages matters more than optimizing any single one, and that's a claim worth testing against whatever benchmarks show up next.