Poster
in
Workshop: Structured Probabilistic Inference and Generative Modeling
Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion
Hila Manor · Tomer Michaeli
Keywords: [ Diffusion Models ] [ Unsupervised editing ] [ Text-based editing ] [ Prompt-based editing ] [ Music Editing ] [ Audio Editing ] [ Text-guided editing ]
Editing signals using large pre-trained models, in a zero-shot manner, has recently seen rapid advancements in the image domain. However, this wave has yet to reach the audio domain. In this paper, we explore two zero-shot editing techniques for audio signals, which use DDPM inversion with pre-trained diffusion models.The first, which we coin ZEro-shot Text-based Audio (ZETA) editing, is adopted from the image domain. The second, named ZEro-shot UnSupervized (ZEUS) editing, is a novel approach for discovering semantically meaningful editing directions without supervision. When applied to music signals, this method exposes a range of musically interesting modifications, from controlling the participation of specific instruments to improvisations on the melody.examples page. Anonymized code can be found here.The full version of this paper was accepted for the main conference.