NemotronLabs VoiceChat puts tool calls on a parallel output stream inside a full-duplex speech-to-speech model

arXiv·medium signalXBlueskyLinkedInCopy link

arXiv 2609.21967 (18 Sep) releases an open full-duplex speech-to-speech model with native tool calling, combining a streaming speech encoder and decoder-only LM with parallel specialized output streams for agent text and structured function calls, an auxiliary RNN-T branch for incremental user transcription, and a streaming TTS decoder. On Full-Duplex-Bench 1.0 it has the lowest pause-handling takeover rates among evaluated open-weight systems, 100% takeover after user interruptions and a 4.33/5 post-interruption response quality score, and resumes after user backchannels in 93% of cases on FDB 1.5. It scores 55.1 normalized on VoiceBench and 82.5% tool-selection F1 on FDB 3.0, with the authors conceding that argument accuracy and end-to-end tool execution are still weak — the honest gap for anyone shipping a voice agent on it.