Comments (13)
This is high on our wishlist. The issue we haven't solved yet is that the pre-processing of the audio data to the format the network expects hasn't been written for python yet. As I understand it, fft processing of audio is handled differently natively in javascript and python, making it tricky.
The model that is used for audio training is https://github.com/tensorflow/tfjs-models/tree/master/speech-commands, and does unfortunately not yet have a python counter part.
Contributions to this is very welcomed!
from teachablemachine-community.
Thanks for the reply @HalfdanJ, does the speech commands package work on node.js? I tried running it in a non-browser environment and it didnt seem to work.
from teachablemachine-community.
I don't believe so
from teachablemachine-community.
@d0minikt Can you say a little more about your use case? The audio model is tied to WebAudio's frequency analyzer (FFT). This means that in order to use the model in Python, you'll find a way to replicate the audio input parameters and frequency analysis.
from teachablemachine-community.
@caisq if that's hard to do, is it easier to port the speech-commands
package so that it also runs on node.js
? I'm sure I'm not the only one with a headless use case.
from teachablemachine-community.
There are ops for DSP in tensorflow directly[1], but I guess it's hard to maintain these for different platforms like TFLite and TFjs.
Also, most likely you rely on optimized FFT of browser.
from teachablemachine-community.
Also, any plans to support exporting saved model? Currently I only see export to Tensorflow.js
from teachablemachine-community.
As I understand it, fft processing of audio is handled differently natively in javascript and python, making it tricky.
The model that is used for audio training is ...... and does unfortunately not yet have a python counter part.
In case anyone hasn't noticed, the Coral example project Keyphrase detector seems to have the pre-processing code necessary. Not sure it's equivalent to those in Speech Commands, but at least they both compute Mel spectrogram.
I am just saying this, in case it may be helpful to someone.
from teachablemachine-community.
@nickoala To be clear, I'm pretty sure the preprocessing steps in the Coral example doesn't fit TF.js Speech Commands, because Speech Commands is based on the browser's WebAudio FFT, which is a linear-frequency spectrum, not a Mel one.
from teachablemachine-community.
@caisq, but there is a SOFT_FFT
option to speechCommands.create()
right? This file does seem to compute Mel spectrogram.
from teachablemachine-community.
@nickoala My apologies: The document is not very clear and some of the code is obsolete. The SOFT_FFT mode does use Mel spectrum. But the default mode of Speech Commands (BROWSER_FFT) uses linear spectrum from WebAudio.
from teachablemachine-community.
I see. Thank you for clarification.
from teachablemachine-community.
Hey guys, I had the same problem trying to run an audio model in a headless device with python. I could make it work but with node.js, but the trick could work also with python. The little trick was to launch a headless chromium with puppeteer where the javascript run the model and inside the node.js script the predictions are parsed and then you are free to go and do whatever with the predictions.
I made it to turn off/on my room's light. If you want to check out the code and how to do it go to: https://github.com/charlielito/teachable-machines-audio-demo
Any feedback is welcome!
from teachablemachine-community.
Related Issues (20)
- Teachable Machine Project Help HOT 5
- How to run when there is no network in the local browser? HOT 1
- TeachableMachine.Calculate() always returns "Value cannot be null. (Parameter 's')" using C# .NET API
- [BUG]: TeachableMachine on local machine
- Is there an API that could obtain a trained .tflite model?
- Compilation error: tensorflow/lite/version.h: No such file or directory HOT 1
- linker_script.ld:138 cannot move location counter backwards (from 200532b0 to 2003fc00)
- Unable to build docker image
- [BUG]: 'keras.md' code refers erroneously to downloaded model file
- Error when doing inference in loading my model HOT 2
- Unable to Compile Zip File for Arduino Nano
- [FEATURE REQUEST]: Ability to get timestamps of detected audio classes while recording
- Teachable machine with tensorflow.js model - jpg files for clasification instead of video stream from webcam
- Substantial performance degrade in image recognition from TM web results
- [BUG]: Preview Input photo upload not working
- [BUG]: Can't start a project on iPad devices HOT 1
- Pose Project not finding points
- Where are the datasets stored for the model to trained and licensing for commercial usage of the trained model
- [BUG]: Unable to train model for audio classification
- [BUG]: Unable to "open an exisiting project from file" or "from Drive"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from teachablemachine-community.