Speaker Separation via Audio Language Modeling

We propose LlaSep, an autoregressive speaker separation model operating entirely in the discrete token domain: conditioned on a tokenized mixture and semantic features from a pretrained speech encoder, a causal language model generates per-speaker codec token streams in a single decoding pass. Trained via supervised fine-tuning on a 15k-hour synthetic multilingual dataset spanning seven languages, LlaSep matches prior baselines on LibriCSS and CallHome while delivering substantially higher audio quality.