Using Deep Learning to generate pop music!

You can also experiment through the web app - musicautobot.com

Recent advances in NLP have produced amazing results in generating text. Transformer architecture is a big reason behind this.

This project aims to leverage these powerful language models and apply them to music. It's built on top of the fast.ai library

MusicTransformer - This basic model uses Transformer-XL to take a sequence of music notes and predict the next note.

MultitaskTransformer - Built on top of MusicTransformer, this model is trained on multiple tasks.

• Next Note Prediction (same as MusicTransformer)
• BERT Token Masking
• Sequence To Sequence Translation - Using chords to predict melody and vice versa.

Training on multiple tasks means we can generate some really cool predictions (Check out this Notebook):

1. Harmonization - generate accompanying chords
2. Melody - new melody from existing chord progression
3. Remix tune - new song in the rhythm of a reference song
4. Remix beat - same tune, different rhythm

1. MusicTransformer

• Train - End to end example on how to create a dataset from midi files and train a model from scratch
• Generate - Loads a pretrained model and shows how to generate/predict new notes

2. MultitaskTransformer

• Train - End to end example on creating a seq2seq and masked dataset for multitask training.
• Generate - Loads a pretrained model and shows how to harmonize, generate new melodies, and remix existing songs.

3. Data Encoding

• Midi2Tensor - Shows how the libary internally encodes midi files to tensors for training.
• MusicItem - MusicItem is a wrapper that makes it easy to manipulate midi data. Convert midi to tensor, apply data transformations, even play music or display the notes within browser.

• musicautobot/
• numpy_encode.py - submodule for encoding midi to tensor
• music_transformer.py - Submodule structure similar to fastai's library.
  • Learner, Model, Transform - MusicItem, Dataloader
• multitask_transformer.py - Submodule structure similar to fastai's library.
  • Learner, Model, Transform - MusicItem, Dataloader

CLI scripts for training models:
run_multitask.py - multitask training

python run_multitask.py --epochs 14 --save multitask_model --batch_size=16 --bptt=512 --lamb --data_parallel --lr 1e-4

run_music_transformer.py - music model training

python run_multitask.py --epochs 14 --save music_model --batch_size=16 --bptt=512 --lr 1e-4

run_ddp.sh - Helper method to train with mulitple GPUs (DistributedDataParallel). Only works with run_music_transformer.py

SCRIPT=run_multitask.py bash run_ddp.sh --epochs 14 --save music_model --batch_size=16 --bptt=512 --lr 1e-4

1. Install anaconda: https://www.anaconda.com/distribution/

2. Run:

git clone https://github.com/bearpelican/musicautobot.git

cd musicautobot

conda env update -f environment.yml

source activate musicautobot

3. Install Musescore - to view sheet music within a jupyter notebook

   Ubuntu:

   sudo apt-get install musescore

   MacOS - download

Installation:

cd serve

conda env update -f environment.yml

You need to setup an s3 bucket to save your predictions. After you've created a bucket, update the config api/api.cfg with the new bucket name.

Development:

python run.py

Production:

gunicorn -b 127.0.0.1:5000 run_guni:app  --timeout 180 --workers 8

Unfortunately I cannot provide the dataset used for training the model.

Here's some suggestions:

• Classical Archives - incredible catalog of high quality classical midi
• HookTheory - great data for sequence to sequence predictions. Need to manually copy files into hookpad
• Reddit - 130k files
• Lakh - great research dataset

This project is built on top of fast.ai's deep learning library and music21's incredible musicology library.

Special thanks to SPC and PalapaVC