justinsalamon / audio_to_midi_melodia Goto Github PK
View Code? Open in Web Editor NEWExtract the melody from an audio file and export to MIDI
Extract the melody from an audio file and export to MIDI
I'm sorry I don't really know where you're supposed to ask questions like this. Is it possible to export only the WAV file? I have been searching as hard as I could to find a way to convert the MELODIA output directly to an audio file, but this is the closest thing I've seen to that.
How do I use melodia in Jupyter notebook?
Finding this software was a life saver! I'm just struggling to get it to work currently. I'm trying to use it for a research project where I need a large data set of midi files of melodies of songs so this functionality is pretty perfect.
I encountered a type error when executing the command below in my terminal.
$ python3 audio_to_midi_melodia_master/audio_to_midi_melodia.py watchwhathappens.wav watchwhathappens.mid 172
Loading audio...
Extracting melody f0 with MELODIA...
Converting Hz to MIDI notes...
audio_to_midi_melodia_master/audio_to_midi_melodia.py:142: RuntimeWarning: divide by zero encountered in log2
midi = 69 + 12*np.log2(hz_nonneg/440.)
Traceback (most recent call last):
File "audio_to_midi_melodia_master/audio_to_midi_melodia.py", line 216, in <module>
savejams=args.jams)
File "audio_to_midi_melodia_master/audio_to_midi_melodia.py", line 182, in audio_to_midi_melodia
notes = midi_to_notes(midi_pitch, fs, hop, smooth, minduration)
File "audio_to_midi_melodia_master/audio_to_midi_melodia.py", line 114, in midi_to_notes
if p_prev > 0:
TypeError: '>' not supported between instances of 'NoneType' and 'int'
I traced it back to the following code in the midi_to_notes
function:
notes = []
p_prev = None
duration = 0
onset = 0
for n, p in enumerate(midi_filt):
if p == p_prev:
duration += 1
else:
# treat 0 as silence
if p_prev > 0:
# add note
duration_sec = duration * hop / float(fs)
# only add notes that are long enough
if duration_sec >= minduration:
onset_sec = onset * hop / float(fs)
notes.append((onset_sec, duration_sec, p_prev))
# start new note
onset = n
duration = 1
p_prev = p
In the for loop, the else
part of the if condition evaluates p_prev
before it's been set to the current p
, which gives an error in the first iteration of the loop when p_prev = None
from initialisation.
I think this can be solved by changing p_prev
to p
in the lines if p_prev > 0
and notes.append((onset_sec, duration_sec, p_prev))
.
I tried creating a new script and amending this and then I received the following error in my terminal:
$ python3 audio_to_midi_melodia_master/new_script.py watchwhathappens.wav watchwhathappens.mid 172
Loading audio...
Extracting melody f0 with MELODIA...
Converting Hz to MIDI notes...
audio_to_midi_melodia_master/new_script.py:144: RuntimeWarning: divide by zero encountered in log2
midi = 69 + 12*np.log2(hz_nonneg/440.)
Saving MIDI to disk...
Traceback (most recent call last):
File "audio_to_midi_melodia_master/new_script.py", line 220, in <module>
savejams=args.jams)
File "audio_to_midi_melodia_master/new_script.py", line 190, in audio_to_midi_melodia
save_midi(outfile, notes, bpm)
File "audio_to_midi_melodia_master/new_script.py", line 88, in save_midi
midifile.writeFile(binfile)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/midiutil/MidiFile.py", line 1637, in writeFile
self.close()
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/midiutil/MidiFile.py", line 1699, in close
self.tracks[i].writeMIDIStream()
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/midiutil/MidiFile.py", line 835, in writeMIDIStream
self.writeEventsToStream()
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/midiutil/MidiFile.py", line 852, in writeEventsToStream
self.MIDIdata += event.serialize(previous_event_tick)
File "/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/midiutil/MidiFile.py", line 124, in serialize
midibytes += struct.pack('>B', self.pitch)
struct.error: required argument is not an integer
I realised that the resulting midi
array returned from hz2midi
has float elements, but I tried converting them to integers and I still get the same error and cannot get around it.
Am I doing something really wrong? I can't seem to figure out how to solve this.
Thanks in advance!
xxx
I have hardly compiled and installed it ...
"python audio_to_midi_melodia.py a.wav a.mid 60 --smooth 0.25 --minduration 0.1 --jams"
"python audio_to_midi_melodia.py a.mp3 a.mid 60 --smooth 0.25 --minduration 0.1 --jams"
but the converted mid file play just ding... dang dong.. ding...
I can not recognize what the original audio/music is.
In the code, the vamp plugin generates the pitch for the audio. I was wondering if there is a way to get the corresponding timestamps for these pitch values?
Code:
print("Extracting melody f0 with MELODIA...")
melody = vamp.collect(data, sr, "mtg-melodia:melodia",
parameters={"voicing": 0.2})
pitch = melody['vector'][1]
Fixed.
The plugin fails to load in python but works in sonic visualizer.
System particulars-
Windows10, Python 3.6
@justinsalamon
I used this plugin in SV (windows),and found that the monophonic worked better than polyphonic.
if i want to set the program in python,which parameters should i send.
thank you.
Hi
I have some problem using vamp plug in.
I've read the related issues, however they didn't fix my problem.
I'm using windows 10 64bit and python 3.6.6
It says
Backend TkAgg is interactive backend. Turning interactive mode on.
Vamp::HostExt: Unable to load library "C:\vamp plugins\mtg-melodia.dll"
Vamp::HostExt::PluginLoader: No library found in Vamp path for plugin "mtg-melodia:melodia"
Traceback (most recent call last):
File "C:\Users\Alex\AppData\Local\Programs\Python\Python36\lib\site-packages\IPython\core\interactiveshell.py", line 2961, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-c735dc41cd7b>", line 1, in <module>
runfile('C:/Users/Alex/Desktop/audio_to_midi_melodia-master/test.py', wdir='C:/Users/Alex/Desktop/audio_to_midi_melodia-master')
File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.5\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.5\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Alex/Desktop/audio_to_midi_melodia-master/test.py", line 15, in <module>
data = vamp.collect(audio, sr, "mtg-melodia:melodia")
File "C:\Users\Alex\AppData\Local\Programs\Python\Python36\lib\site-packages\vamp\collect.py", line 166, in collect
plugin, step_size, block_size = vamp.load.load_and_configure(data, sample_rate, plugin_key, parameters, **kwargs)
File "C:\Users\Alex\AppData\Local\Programs\Python\Python36\lib\site-packages\vamp\load.py", line 83, in load_and_configure
vampyhost.ADAPT_CHANNEL_COUNT)
TypeError: Failed to load plugin: mtg-melodia:melodia
I can see the melodia plugin in Sonic Visualiser but it doesn't work in python. Anything else I should try? Thanks.
Vamp::HostExt::PluginLoader: No library found in Vamp path for plugin "mtg-melodia:melodia"
Traceback (most recent call last):
File ".../audio_to_midi_melodia.py", line 220, in
audio_to_midi_melodia("test.wav", "test.mid", 60)
File ".../audio_to_midi_melodia.py", line 166, in audio_to_midi_melodia
parameters={"voicing": 0.2})
File "...\audio_to_midi_melodia\venv\lib\site-packages\vamp\collect.py", line 166, in collect
plugin, step_size, block_size = vamp.load.load_and_configure(data, sample_rate, plugin_key, parameters, **kwargs)
File "...\audio_to_midi_melodia\venv\lib\site-packages\vamp\load.py", line 83, in load_and_configure
vampyhost.ADAPT_CHANNEL_COUNT)
TypeError: Failed to load plugin: mtg-melodia:melodia
Can anyone help me with this issue? I'm not sure why the plugin failed to load. Could it because I am working in a virtual environment (venv) within pycharm?
Regardless of the source audio, I get the following warning:
audio_to_midi_melodia.py:142: RuntimeWarning: divide by zero encountered in log2
midi = 69 + 12*np.log2(hz_nonneg/440.)
In this code block:
def hz2midi(hz):
# convert from Hz to midi note
hz_nonneg = hz.copy()
hz_nonneg[hz <= 0] = 0
midi = 69 + 12*np.log2(hz_nonneg/440.)
midi[midi <= 0] = 0
# round
midi = np.round(midi)
return midi
To silence the warning, I edited that to initialize the other entries to a very small value instead of 0, changing that line above the warning to be:
hz_nonneg[hz <= 0] = 1.e-15;
I don't know if that's the preferred fix or not, as I haven't fully checked to see how it affects the remaining code, but it seems to work OK to get rid of that warning. And the resulting midi files, on the ones that I checked, were identical with or without the fix.
Otherwise, this script and the Melodia plugin work fantastically! Great job with it!
As a proxy for note loudness
ERROR: Failed building wheel for vamp
Running setup.py clean for vamp
Failed to build vamp
Installing collected packages: vamp
Running setup.py install for vamp ... error
ERROR: Command errored out with exit status 1:
command: /Users/wudi/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/08/mzdfs5294rq4rvhtwh21xrk80000gn/T/pip-install-5ugefj3b/vamp/setup.py'"'"'; file='"'"'/private/var/folders/08/mzdfs5294rq4rvhtwh21xrk80000gn/T/pip-install-5ugefj3b/vamp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/08/mzdfs5294rq4rvhtwh21xrk80000gn/T/pip-record-l27rp7lh/install-record.txt --single-version-externally-managed --compile
cwd: /private/var/folders/08/mzdfs5294rq4rvhtwh21xrk80000gn/T/pip-install-5ugefj3b/vamp/
Complete output (28 lines):
running install
running build
running build_py
creating build
creating build/lib.macosx-10.7-x86_64-3.7
creating build/lib.macosx-10.7-x86_64-3.7/vamp
copying vamp/collect.py -> build/lib.macosx-10.7-x86_64-3.7/vamp
copying vamp/init.py -> build/lib.macosx-10.7-x86_64-3.7/vamp
copying vamp/frames.py -> build/lib.macosx-10.7-x86_64-3.7/vamp
copying vamp/process.py -> build/lib.macosx-10.7-x86_64-3.7/vamp
copying vamp/load.py -> build/lib.macosx-10.7-x86_64-3.7/vamp
running build_ext
building 'vampyhost' extension
creating build/temp.macosx-10.7-x86_64-3.7
creating build/temp.macosx-10.7-x86_64-3.7/vamp-plugin-sdk
creating build/temp.macosx-10.7-x86_64-3.7/vamp-plugin-sdk/src
creating build/temp.macosx-10.7-x86_64-3.7/vamp-plugin-sdk/src/vamp-hostsdk
creating build/temp.macosx-10.7-x86_64-3.7/native
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/wudi/anaconda3/include -arch x86_64 -I/Users/wudi/anaconda3/include -arch x86_64 -D_USE_MATH_DEFINES=1 -Ivamp-plugin-sdk -I/Users/wudi/anaconda3/lib/python3.7/site-packages/numpy/core/include -I/Users/wudi/anaconda3/include/python3.7m -c vamp-plugin-sdk/src/vamp-hostsdk/Files.cpp -o build/temp.macosx-10.7-x86_64-3.7/vamp-plugin-sdk/src/vamp-hostsdk/Files.o
warning: include path for stdlibc++ headers not found; pass '-stdlib=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
In file included from vamp-plugin-sdk/src/vamp-hostsdk/Files.cpp:37:
In file included from vamp-plugin-sdk/vamp-hostsdk/PluginHostAdapter.h:41:
In file included from vamp-plugin-sdk/vamp-hostsdk/Plugin.h:45:
vamp-plugin-sdk/vamp-sdk/Plugin.h:40:10: fatal error: 'string' file not found
#include
^~~~~~~~
1 warning and 1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/wudi/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/08/mzdfs5294rq4rvhtwh21xrk80000gn/T/pip-install-5ugefj3b/vamp/setup.py'"'"'; file='"'"'/private/var/folders/08/mzdfs5294rq4rvhtwh21xrk80000gn/T/pip-install-5ugefj3b/vamp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/08/mzdfs5294rq4rvhtwh21xrk80000gn/T/pip-record-l27rp7lh/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
I didn't see this posted so I hope I'm not repeating a known issue.
I'm using a new Mac Studio with an M1, which I believe is the issue. Here's what I'm getting:
Vamp::HostExt: Unable to load library "/Library/Audio/Plug-Ins/Vamp/mtg-melodia.dylib": dlopen(/Library/Audio/Plug-Ins/Vamp/mtg-melodia.dylib, 0x0005): tried: '/Library/Audio/Plug-Ins/Vamp/mtg-melodia.dylib' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Library/Audio/Plug-Ins/Vamp/mtg-melodia.dylib' (no such file), '/Library/Audio/Plug-Ins/Vamp/mtg-melodia.dylib' (fat file, but missing compatible architecture (have 'i386,x86_64', need 'arm64'))
Thanks for you help.
Hello, I'm trying to get this up and running but cannot seem to download the Melodia plugin from the linked site: http://mtg.upf.edu/technologies/melodia
It seems the site has undergone some changes as the above link resolves to: https://www.upf.edu/web/mtg/melodia instead.
Then, the DOWNLOAD FORM link on that page (which should go to http://mtg.upf.edu/descargas/node/1/) redirects back to the homepage (https://www.upf.edu/web/mtg/) presumably because the download form has been moved or deleted.
Where can I download this plugin?
It's a great tool, but the note segmentation module seems to be the bottleneck, does anyone know better algorithm for note segmentation/quantization ?
Hi is there any specific song requirements to get the best result of melody extraction using this melodia? i.e. song with .wav format / song with 48000 Hz?
What this supossed to mean? It's failed during launch...
D:\audio2midi>python audio_to_midi_melodia.py ~/test.wav ~/test.mid 60 --smooth
0.25 --minduration 0.1 --jams
Traceback (most recent call last):
File "audio_to_midi_melodia.py", line 3, in
import librosa
File "D:\Python27\lib\site-packages\librosa_init_.py", line 11, in
from . import cache
File "D:\Python27\lib\site-packages\librosa\cache.py", line 7, in
from joblib import Memory
File "D:\Python27\lib\site-packages\joblib_init_.py", line 119, in
from .parallel import Parallel
File "D:\Python27\lib\site-packages\joblib\parallel.py", line 28, in
from .parallel_backends import (FallbackToBackend, MultiprocessingBackend,
File "D:\Python27\lib\site-packages\joblib_parallel_backends.py", line 22, in
from .executor import get_memmapping_executor
File "D:\Python27\lib\site-packages\joblib\executor.py", line 14, in
from .externals.loky.reusable_executor import get_reusable_executor
File "D:\Python27\lib\site-packages\joblib\externals\loky_init.py", line 1
1, in
from .backend.context import cpu_count
File "D:\Python27\lib\site-packages\joblib\externals\loky\backend_init_.py"
, line 4, in
from .context import get_context
File "D:\Python27\lib\site-packages\joblib\externals\loky\backend\context.py",
line 21, in
from .process import LokyProcess, LokyInitMainProcess
File "D:\Python27\lib\site-packages\joblib\externals\loky\backend\process.py",
line 11, in
from .compat import BaseProcess
File "D:\Python27\lib\site-packages\joblib\externals\loky\backend\compat.py",
line 23, in
from .compat_win32 import wait
File "D:\Python27\lib\site-packages\joblib\externals\loky\backend\compat_win32
.py", line 17, in
from .win_wait import wait
File "D:\Python27\lib\site-packages\joblib\externals\loky\backend_win_wait.py
", line 26, in
GetTickCount64 = ctypes.windll.kernel32.GetTickCount64
File "D:\Python27\lib\ctypes_init.py", line 366, in getattr
func = self.getitem(name)
File "D:\Python27\lib\ctypes_init_.py", line 371, in getitem
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'GetTickCount64' not found
hello :D
when i run this code from cmd i always get this error
Loading audio...
Extracting melody f0 with MELODIA...
Vamp::HostExt::PluginLoader: No library found in Vamp path for plugin "mtg-melodia:melodia"
Traceback (most recent call last):
File "audio_to_midi_melodia.py", line 225, in
savejams=args.jams)
File "audio_to_midi_melodia.py", line 174, in audio_to_midi_melodia
parameters={"voicing": 0.2})
File "C:\Python27\lib\site-packages\vamp\collect.py", line 166, in collect
plugin, step_size, block_size = vamp.load.load_and_configure(data, sample_rate, plugin_key, parameters, **kwargs)
File "C:\Python27\lib\site-packages\vamp\load.py", line 83, in load_and_configure
vampyhost.ADAPT_CHANNEL_COUNT)
TypeError: Failed to load plugin: mtg-melodia:melodia
i installed vamp and melodia , i work by python 2.7 And visual studio code and installed sonic to try the plugin and its work but when it comes to python it say no library found in vamp
could u help me please solve this issue , i'm using this code in my graduation project and the deadline is soon :(
The link of "several years of research" is broken
I tried running the code using the sample command, in order to open a .wav file for execution and ran into this error. I'm running on a Linux system with Python 2.7 and I have all the dependencies installed.
I even have ffmpeg
and other dependencies that work around the audioread error. Should I be using other ways to open the audio file, and note Librosa.load()
Loading audio...
Traceback (most recent call last):
File "audio_to_midi_melodia.py", line 210, in
savejams=args.jams)
File "audio_to_midi_melodia.py", line 154, in audio_to_midi_melodia
data, sr = librosa.load(infile, sr=fs, mono=True)
File "/usr/local/lib/python2.7/site-packages/librosa/core/audio.py", line 119, in load
with audioread.audio_open(os.path.realpath(path)) as input_file:
File "/usr/local/lib/python2.7/site-packages/audioread/init.py", line 116, in audio_open
raise NoBackendError()
audioread.NoBackendError
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.