SURF: Separation via Unsupervised Remixing Flow

The goal of single-channel source separation is to reconstruct $K$ sources given their mixture. In supervised settings where vast amounts of clean source data are available, this challenging, ill-posed problem has been addressed successfully by generative diffusion and flow-based prior models. However, access to such clean source samples is often limited, and even when available, supervised models are vulnerable to domain shifts. To bridge this gap, we present Separation via Unsupervised Remixing Flow (\textbf{SURF}), an unsupervised flow matching approach for source separation that learns directly from observed mixtures. This method relies on a novel combination of state-of-the-art supervised flow matching and regression-based self-supervised techniques. At a high level, starting from a teacher model, we utilize a ``remixing'' step to bootstrap the learning of a student flow model from the teacher's estimates. We provide insights into the objectives optimized by this approach and draw a novel connection to the Wake-Sleep algorithm. Empirical evaluations on image and audio benchmarks demonstrate that \textbf{SURF} establishes a new state-of-the-art, significantly outperforming existing unsupervised methods. See our demo page for examples. https://google.github.io/df-conformer/surf/

Imagine trying to isolate a single conversation at a loud party. Computers are good at this task, known as source separation, but they usually require millions of examples of perfectly clean data to learn the characteristics of each individual sound. To fix this, researchers developed SURF, a method that learns to separate individual sounds (or images) by using only the messy, mixed-up data itself. It works using a feedback loop where an initial model makes a rough guess at separating a mixture, those guesses are remixed into new, never seen, sound mixtures, and a student model uses that new blend to teach itself how to untangle the original signals. By eliminating the need for pristine training data, SURF significantly outperformed existing unsupervised methods, making it easier to build high-quality separation tools for the real world.