keunwoochoi / dl4mir Goto Github PK

View Code? Open in Web Editor NEW

236.0 10.0 37.0 1.97 MB

Deep learning for MIR

Python 14.85% Jupyter Notebook 82.16% HTML 2.98%

deep-learning mir

dl4mir's Introduction

dl4mir: A Tutorial on Deep Learning for MIR

by Keunwoo Choi ([email protected])

This is a repo for my tutorial paper; A Tutorial on Deep Learning for Music Information Retrieval.

Tutorials

Prerequisites

$ pip install -r requirements.txt
$ git clone https://github.com/keunwoochoi/kapre.git
$ cd kapre
$ python setup.py install

to install

Librosa, Keras, Numpy, Matplotlib, Future
kapre

Notes

Datasets is removed from Kapre and the codes are directly imported into here.

Datasets

Dataset management

GTZan: (30s, 10 genres, 1,000 mp3)
MagnaTagATune: (29s, 188 tags, 25,880 mp3) for tagging and triplet similarity
MusicNet: (full length 330 classicals music, note-wise annotations)
FMA: small/medium/large/full collections, up to 100+K songs from free music archieve, for genre classification. With genre hierarchy, pre-computed features, splits, etc.
Jamendo: 61/16/24 songs for vocal activity detection

Some links

Repo
- DL_MIR_TUTORIAL: Another DL for MIR tutorial repo by Thomas Lidy
- Awesome deep learning music: A long list of deep learning x music papers + etc.
Slides
- Deep Neural Networks in MIR: A tutorial focusing on feature learning, beat/rhythm analysis, structure analysis. Also a nice literature overview including publications by year, conference, task, network types, input representations, frame work, etc. By Meinard Muller et al.
- DL in music informatics: ISMIR 2014 tutorial.
Documents, books
- Deep learning book: The first deep learning textbook. by Ian Goodfellow and Yoshua Bengio and Aaron Courville.
Online
- cs213n: Perhaps the best lecture on convnets. From Stanford university.
- librosa tutorial: If you're interested in learning a bit more of MIR and its implementations.
- MIRDL: state-of-the-art: A wikipedia on MIR and DL by Jordi Pons.
- MIR datasets: An awesome list of MIR datasets
- /r/musicir: A subreddit for MIR

Cite?

@article{choi2017tutorial,
  title={A Tutorial on Deep Learning for Music Information Retrieval},
  author={Choi, Keunwoo and Fazekas, Gy{\"o}rgy and Cho, Kyunghyun and Sandler, Mark},
  journal={arXiv:1709.04396},
  year={2017}
}

Or visit the paper page on Google scholar for potential updates.

dl4mir's People

Contributors

Stargazers

Watchers

dl4mir's Issues

Cannot download FMA dataset.

When running python main_preprocess.py fma, the following error occurs:

FileNotFoundError: File b'dataset_download/fma/fma_metadata/tracks.csv' does not exist

Following is the full error message.

Traceback (most recent call last):
  File "main_preprocess.py", line 95, in <module>
    main(sys.argv[1])
  File "main_preprocess.py", line 77, in main
    prep_fma_small()
  File "main_preprocess.py", line 46, in prep_fma_small
    tracks = pd.read_csv(os.path.join(DIR_FMA_CSV, 'tracks.csv'), index_col=0, header=[0, 1])
  File "/home/kehops/dl4mir/venv/lib/python3.5/site-packages/pandas/io/parsers.py", line 705, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/home/kehops/dl4mir/venv/lib/python3.5/site-packages/pandas/io/parsers.py", line 445, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/kehops/dl4mir/venv/lib/python3.5/site-packages/pandas/io/parsers.py", line 814, in __init__
    self._make_engine(self.engine)
  File "/home/kehops/dl4mir/venv/lib/python3.5/site-packages/pandas/io/parsers.py", line 1045, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/home/kehops/dl4mir/venv/lib/python3.5/site-packages/pandas/io/parsers.py", line 1684, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 391, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 710, in pandas._libs.parsers.TextReader._setup_parser_source
  FileNotFoundError: File b'dataset_download/fma/fma_metadata/tracks.csv' does not exist

Should I download tracks.csv from separate source?

Example_5.1 - Predicting an entire song & Data generator details

While trying to visualize the predictions on this example, I am having difficulties understanding the data_gen() function and y_sample_to_frame().

From what I read, a data generator on image processing has the purpose of providing smaller chunks of the data. Is there any recommended way to predict an entire song?

1.1. On a test data, of one song.
n_hop = 256
nsp_y = 5637632
I end up receiving 20 chunks of len 22022, which is not equivalent to the entire song.
Shouldn't I need 256 (5637632// 256) of those?

1.2 Using predict_generator returns only 22022 predictions... which leads me back to question 1

On y_sample_to_frame().

n_hop = N_HOP
nsp_y = len(y)
ret = np.array([np.round(np.mean(y[max(0, (i - 1) * n_hop): min(nsp_y, (i + 1) * n_hop)])) \
                    for i in range(nsp_y // n_hop)], dtype=np.int)

Could you provide some comments on line 3?

In fact, I am trying to modify your example and see how it performs on SALAMI dataset. But it seems that the understanding of this two functions is fundamental. I have found relatively less information about the pre-processing of data for music structure analysis.
Sorry if my questions are not very clearly formulated, any extra information or source would be helpful.
Thanks in advance

ModuleNotFoundError: No module named 'models_MLP'

On Example_5-1 there is:

import models_MLP

Not sure if there is a file missing from the repo.

Thanks in advance

Typos and fix suggestions in Example 1 notebook.

Thx for the awesome tutorial 👍
I have run it on my own, and found some typos and python compatibility issues.

this is probably 12 output nodes?

432 = 36 x 12, 36 x 12 + 12

next() in iters as been renamed to __next__() in python 3. If proceeded with this, keras model would give DataGen is not iterable error.

I have tested this with Keras==2.0.9

Similarly, calling next iteration have been changed in python3.

All datagen.next() had to be modified to next(datagen) in this cell.

AttributeError: module 'kapre' has no attribute 'datasets'

Hi,

I downloaded and placed jamendo dataset in dataset_download/jamendo.
When I run

python main_preprocess.py jamendo

I am getting the following error

Traceback (most recent call last):
File "main_preprocess.py", line 95, in
main(sys.argv[1])
File "main_preprocess.py", line 79, in main
prep_jamendo()
File "main_preprocess.py", line 62, in prep_jamendo
srcs_sets, ys_sets = kapre.datasets.load_jamendo(save_path=DIR_JAMENDO_DOWNLOAD,
AttributeError: module 'kapre' has no attribute 'datasets'

Error : No module named 'models_MLP' in Example_5-1.py

When I run Example_5-1.py, error is like this:

the ExImportError: No module named 'models_MLP'

so how to import models_MLP?
thanks