Offline `get` method about meyda HOT 12 CLOSED

Until this is fixed, are there any workarounds to implement meyda completely serverside?

from meyda.

@Number-42 yes, I've done it, it's a bit hacky but it's possible with v2.0. Here's how to do it so you can also use spectral features:

• you first include the meyda and jsfft stuff separately

var jsfft = require("./node_modules/meyda/node_modules/jsfft/")
var complex_array = require("./node_modules/meyda/node_modules/jsfft/lib/complex_array.js")
var extractors = require("./node_modules/meyda/dist/node/featureExtractors.js")
var utils = require("./utils.js")
var meyda_utils = require("meyda").utils

• then you need to use jsfft to extract your amplitude spectrum

function getSpectrum(_d) {

    var windowedSignal = meyda_utils.applyWindow(_d, 'hanning')
    // create complexarray to hold the spectrum
    var data = new complex_array.ComplexArray(_d.length)
    // map time domain
    data.map(function(value, i, n) {
        value.real = windowedSignal[i]
    });
    // transform
    var spec = data.FFT();
    // assign to meyda
    var ampSpectrum = new Float32Array(_d.length/2)
    for (var i = 0; i < _d.length/2; i++) {
        ampSpectrum[i] = Math.sqrt(Math.pow(spec.real[i],2) + Math.pow(spec.imag[i],2))
    }

    return ampSpectrum
}

• then, you can extract features

var mfccs = extractors.mfcc({
    ampSpectrum: getSpectrum(signal),
    bufferSize: bufSize,
    sampleRate: 44100
});

(obviously, signal.length === bufSize)

Good news is, v3.0.0 which will have an easy offline API and even a CLI for wav files, is almost finished ✨

from meyda.

Thank you. Any callback functions get an array of feature values extracted in the order their names were fed into meyda.start() as arguments, right?

from meyda.

@hughrawlinson correct me if I'm wrong, but I think in v2.0.0 the callback receives an object, with feature names as keys. It is in the same order though.

Of course, meyda.start() won't run without Web Audio.

from meyda.

I'm assuming that callbacks are defined inside mfccs. Does extraction automatically start, or does mfccs.get() have to be run first? I see that get() returns output once (the example in the documentation has it used to define a single variable), which I assume to contain all feature values extracted throughout the input file. Is the output a JSON containing all feature extracted feature sets in chronological order, or an array of arrays?

Does this workaround work with time-domain features?

What is _d supposed to be, and how can that property be extracted from a local file with a known path? Is the buffer size defined as a number of samples or bytes?

from meyda.

Not sure what you mean there – this workaround works if you have a chunk of audio data (should be a Float32Array btw), and you want to extract a feature (could be spectral or temporal) on that one chunk, once. The get and start methods are not applicable here, we will have something like that in v3.

Because there's no Web Audio in node.js (I'm assuming that's what you're using), the internal ScriptProcessor doesn't run, and therefore Meyda can't analyze an audio stream on its own. You have to create that functionality yourself, by splitting your audio into bufferSize-length chunks, and feeding each chunk into the extractor function – in the example above, extractors.mfcc().

For temporal features, you pass signal: your_signal instead of ampSpectrum.

The output in this case is either a Number or Array, depending on the feature. See wiki for info on which features are represented by a single value and which by an array (I know, the wiki could be more descriptive).

_d is the time-domain signal you pass to our getSpectrum function, which then returns the amplitude spectrum of that signal.

Let's keep this thread for discussion on the issue, I'm happy to help out if you have further questions though. Feel free to tweet at @fiala__ or send an email to the address on my profile.

Thanks,

jakub

from meyda.

Yes, all of what Jakub said 😄

I was nearly done writing a big long comment to cover all of that, and then Jakub's comment covered it all, and more! 😄

from meyda.

It's the Meyda vibe! 💃

from meyda.

Sorry to have been a bother.

Also, I assume it's been tried, but node-web-audio-api seems like it'd help with the offline implementation.

from meyda.

No problem, you haven't! As I said, if you need help, feel free to get in touch.

RE: node-web-audio, yes, it certainly would, Sébastien is doing a great job on that one. However, only a handful of nodes are implemented, and I think live audio in node is still not widespread enough.
Then again, SPN is already implemented, so theoretically, Meyda should already run on it!

from meyda.

Actually, it probably won't run on it, but a tiny code modification like adding require('web-audio-api'). where the SPN is created should do the job

from meyda.

I've put the above hack for v2 in this gist if anyone finds it more convenient.

from meyda.