Meta open sources an AI-powered music generator

Not to be outdone by Google, Meta has released its own AI-powered music generator — and, unlike Google, open-sourced it.

Called MusicGen, Meta’s music-generating tool, a demo of which can be found here, can turn a text description (e.g. “An ’80s driving pop song with heavy drums and synth pads in the background”) into about 12 seconds of audio, give or take. MusicGen can optionally be “steered” with reference audio, like an existing song, in which case it’ll try to follow both the description and melody.  

We present MusicGen: A simple and controllable music generation model. MusicGen can be prompted by both text and melody.
We release code (MIT) and models (CC-BY NC) for open research, reproducibility, and for the music community: https://t.co/OkYjL4xDN7 pic.twitter.com/h1l4LGzYgf

— Felix Kreuk (@FelixKreuk) June 9, 2023

Meta says that MusicGen was trained on 20,000 hours of music, including 10,000 “high-quality” licensed music tracks and 390,000 instrument-only tracks from ShutterStock and Pond5, a large stock media library. The company hasn’t provided the code it used to train the model, but it has made available pre-trained models that anyone with the right hardware — chiefly a GPU with around 16GB of memory — can run.

So how does MusicGen perform? Well, I’d say — though certainly not well enough to put human musicians out of a job. Its songs are reasonably melodic, at least for basic prompts like “ambient chiptunes music,” and — to my ears — on par (if not slightly better) with the results from Google’s AI music generator, MusicLM. But they won’t win any awards.

Here’s the output from MusicGen for “jazzy elevator music”: