Simple and Controllable Music Generation
Authors:
- Jade CopetJCJade Copet
- Felix KreukFKFelix Kreuk
- Itai GatIGItai Gat
- Tal RemezTRTal Remez
- David KantDKDavid Kant
- Gabriel SynnaeveGSGabriel Synnaeve
- Yossi AdiYAYossi Adi
- Alexandre DéfossezADAlexandre Défossez
Preprint
Abstract
We tackle the task of conditional music generation. We introduce MusicGen, a single Language Model (LM) that operates over several streams of compressed discrete music representation, i.e., tokens. Unlike prior work, MusicGen is comprised of a single-stage transformer LM together with efficient token interleaving patterns, which eliminates the need for cascading several models, e.g., hierarchically or upsampling. Following this approach, we demonstrate how MusicGen can generate high-quality samples, both mono and stereo, while being conditioned on textual description or melodic features, allowing better controls over the generated output. We conduct extensive empirical evaluation, considering both automatic and human studies, showing the proposed approach is superior to the evaluated baselines on a standard text-to-music benchmark. Through ablation studies, we shed light over the importance of each of the components comprising MusicGen. Music samples, code, and models are available at https://github.com/facebookresearch/audiocraftKeywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Neural Codec Language Models are Zero-Shot Text to Speech SynthesizersIEEE Transactions on Audio, Speech and Language Processing, 2025
- Diffsound: Discrete Diffusion Model for Text-to-Sound GenerationIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
- Unsupervised Symbolic Music Segmentation using Ensemble Temporal Prediction Errors2022
- Efficient Training of Audio Transformers with Patchout2022
- SoundStream: An End-to-End Neural Audio CodecIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021
- Hierarchical Neural Story Generation2018
- Sensitivity to musical structure in the human brainJournal of Neurophysiology, 2012