SAM Audio: Segment Anything in Audio

General audio source separation is a key capability for multimodal AI systems that can perceive and reason about sound. Despite substantial progress in recent years, existing separation models are either domain-specific, designed for fixed categories such as speech or music, or limited in controllability, supporting only a single prompting modality such as text. In this work, we present SAM AUDIO, a foundation model for general audio separation that unifies text, visual, and temporal span prompting within a single framework. Built on a diffusion transformer architecture, SAM AUDIO is trained with flow matching on large-scale audio data spanning speech, music, and general sounds, and can flexibly separate target sources described by language, visual masks, or temporal spans. The model achieves state-of-the-art performance across a diverse suite of benchmarks, including general sound, speech, music, and musical instrument separation in both in-the-wild and professionally produced audios, substantially outperforming prior general-purpose and specialized systems. Furthermore, we introduce a new real-world separation benchmark with human-labeled multimodal prompts and a reference-free evaluation model that correlates strongly with human judgment.

Isolating a specific sound from a complex audio recording has long been challenging for computers. Existing AI systems are typically narrow and highly specialized: one model may separate speech, while another is designed specifically to extract instruments such as bass or drums. Moreover, these systems generally operate on fixed categories and provide little control over what users actually want to isolate. To address this limitation, we developed SAM Audio, a unified and flexible system capable of extracting virtually any sound from real-world audio. Instead of relying on predefined classes, SAM Audio allows users to specify their target interactively: by typing a text description, clicking on a sounding object in a video, or selecting a temporal region of interest. Trained on a large and diverse collection of real-world audio, SAM Audio generalizes across speech, music, and everyday sounds, consistently outperforming prior specialized systems. By enabling intuitive, user-guided sound extraction, SAM Audio makes high-quality audio editing significantly more accessible and versatile.