MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models
Authors:
- Yixiao ZhangYZYixiao Zhang
- Simon DixonSDSimon Dixon
Preprint
Abstract
Recent advances in text-to-music generation models have opened new avenues in musical creativity. However, music generation usually involves iterative refinements, and how to edit the generated music remains a significant challenge. This paper introduces a novel approach to the editing of music generated by such models, enabling the modification of specific attributes, such as genre, mood and instrument, while maintaining other aspects unchanged. Our method transforms text editing to \textit{latent space manipulation} while adding an extra constraint to enforce consistency. It seamlessly integrates with existing pretrained text-to-music diffusion models without requiring additional training. Experimental results demonstrate superior performance over both zero-shot and certain supervised baselines in style and timbre transfer evaluations. Additionally, we showcase the practical applicability of our approach in real-world music editing scenarios.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models2024
- Instruct-Imagen: Image Generation with Multi-modal Instruction2024
- Music ControlNet: Multiple Time-Varying Controls for Music GenerationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024
- AudioLDM 2: Learning Holistic Audio Generation With Self-Supervised PretrainingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024
- Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioning2024
- Efficient Parallel Audio Generation Using Group Masked Language ModelingIEEE Signal Processing Letters, 2024
- Mustango: Toward Controllable Text-to-Music Generation2024
- Transplayer: Timbre Style Transfer with Flexible Timbre Control2023
- Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation2023
- Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation2023