cs.SDcs.AI
Spot, Separate, and Enhance: Fully Generative Approach for Audio Mixing
📅
👥 5 authors
arXiv:2609.29169v1
Authors
Ilpo ViertolaGiulio CengarleGouthaman KVDaniel ArteagaLie Lu
Abstract
We introduce Spot, Separate, and Enhance (SSE), the first multimodal, user-guided generative model for audio remixing and enhancement. SSE enhances video content by rebalancing the audio, removing unwanted audio sources, and reducing reverberation, guided by both video and textual descriptions. To support its training and evaluation, we propose DegradedMix, a new dataset built on the audio remixing benchmark MuddyMix. We also adopt evaluation metrics from generative modeling, which better capture the creative nature of remixing than standard reconstruction-based metrics. SSE outperforms existing baselines in both controllability and remixing quality, as shown by extensive experiments. Project page: https://sse-ai.notion.site
Paper Information
- arXiv ID:
- 2609.29169v1
- Published:
Related Categories
cs.SDcs.AI