web-mfcc's Introduction

Web-MFCC

Calculate Mel-frequency cepstral coefficients (MFCCs) in the browser from prepared audio or receive live audio input from the microphone using Javascript Web Audio API.

Implement and accelerate Tensorflow 'AudioSpectrogram' and 'Mfcc' operators by compiling the TensorFlow/lite/kernels use emscripten. So it uses the same parameters as TensorFlow.

Compile the code

Download and install emscripten follow the instructions.
Compile each .c/.cc file to bitcode:
```
emcc -O3 test.cc -o test.bc
```
Compile all the .bc file to tf_mfcc.bc:
```
emcc *.bc -o tf_mfcc.bc
```

Compile tf_mfcc.bc to WASM:

mkdir wasm

emcc -O3 -s WASM=1 -s "EXTRA_EXPORTED_RUNTIME_METHODS=['ccall', 'cwrap']" tf_mfcc.bc -o ./wasm/mfcc.js

After compile you will get the following files int the wasm folder:

wasm
├── mfcc.js
└── mfcc.wasm

Run the example

Download release package and unzip it.
```
tar xzvf web-mfcc-version.tar.gz
```

Put it in to the sample folder:

example
├── favicon.ico
├── index.html
├── main.js
├── wasm
│   ├── mfcc.js
│   └── mfcc.wasm
└── yes.wav

Start an http server in the example folder. You can install http-server via:
```
npm install http-server -g

http-server
```
Open up the browser and access this URL:

http://localhost:8080/
Click on Play button to see results from console.

web-mfcc's People

Stargazers

Watchers

web-mfcc's Issues

Strange results - more details about 'getAudioPCMData' and a reference graph?

Hi @miaobin

this is very interesting work 👍 and I've been planning to integrate it into an audio library for SEPIA I'm working on as a replacement for Meyda MFCC to gain some WASM speed 😀 ... but the results seem a bit confusing 😅

Here is a screenshot comparing Web-MFCC (1, 3) to Meyda MFCC (2):

In graph 1 and 2 I've used the same chunked buffer to create the MFCC step-by-step. 1 is Web-MFCC and 2 is Meyda. In graph 3 I've used Web-MFCC on the same buffer but as one big step and not in chunks. In graph 3 I've also removed the first and every 13th entry of the result array, which leads me to my first question:

What exactly is the result of getAudioPCMData as seen in your example? Looking at my results it almost seems like the first entry of every block (1 block = number of MFCCCoefficients = 13) is the sum of the signal instead of a coefficient.
Did you check the results against any reference? Comparing Meyda (2) and the cleaned-up Web-MFCC (3) the Web-MFCC result looks scrambled somehow. Now the parameters might not be exactly the same (I've tried to get as close as possible) but I'm pretty sure there is something else going on.

Btw I've used a recording of "yes ... no ... yes" for my graphs.

Any comment or help to get the results fixed would be appreciated 🙂

Ty,
Florian

Recommend Projects

miaobin / web-mfcc Goto Github PK

web-mfcc's Introduction

Web-MFCC

Compile the code

Run the example

web-mfcc's People

Stargazers

Watchers

Forkers

web-mfcc's Issues

Strange results - more details about 'getAudioPCMData' and a reference graph?

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent