StepAudio 3 Gen goes back to discrete autoregressive audio and skips the diffusion Transformer

arXiv / HuggingFace Daily Papers·low signalXBlueskyLinkedInCopy link

arXiv 2609.12945, submitted 2026-09-11, is a unified audio generation model covering zero-shot TTS, voice design, vocal generation, sound effects, music and mixtures, built as a discrete autoregressive generator over residual vector quantization tokens rather than the diffusion Transformer approach most recent general audio models use. Its tokenizer runs at 12.5 Hz in a shared 16x2048 residual code space, jointly quantizing semantic and waveform-level acoustic features so each code layer carries both. The backbone predicts codebook one along time; a lightweight causal Transformer fills the remaining fifteen along the codebook axis.