A technical report posted to arXiv on September 11 describes StepAudio 3 Gen, a single model meant to cover music, sound effects, singing, designed voices and zero-shot text-to-speech at once.
The notable choice is architectural. Instead of the diffusion transformer approach common to recent general-audio models, StepAudio 3 Gen predicts discrete residual vector quantization tokens autoregressively, an audio counterpart to how large language models predict text tokens. Its tokenizer encodes audio at 12.5 Hz across a shared 16 by 2048 code space, combining semantic and acoustic detail in one representation.
The authors describe three design choices: pretraining that adds audio skills without degrading the base language model’s text ability, an adaptor for multiple codebooks of acoustic information, and autoregressive modeling shared across speech, music and effects. After progressive pretraining and supervised fine-tuning, they report state-of-the-art results on text-to-speech and voice design. Those benchmarks are the authors’ own; the report does not describe independent evaluation.
The wager is that the token-prediction approach behind large language models, not diffusion, is what scales cleanly to audio. Teams testing voice or audio stacks should treat these numbers as a starting claim, not a verdict, pending independent runs against the released samples.
Per a technical report posted to arXiv on September 11, 2026.
