lingjzhu / mtracker.github.io Goto Github PK

I used your CNN to make something that could potentially show a live tongue shape from an ultrasound video stream. My wife, @heatherkoff, is an SLP researching ultrasound at NYU and attended UltraFest IX. Your talk was fortunately timed as we were in the car and, since I had been suggesting she try CNNs, she put your talk on the speakers. I am a software developer, I know nothing about the speech field.

It was relatively easy to get the CNN working in a docker container. On my machine I quickly ran into versioning issues that you seemed to hint at in your talk, so I went right for docker.

The system is a tiny website that allows you to select a window containing an ultrasound image/video, like you would share a window in a video chat. Frames are sent to a small web server that saves the image, runs your python, and sends the result back. Currently it's too slow to be useful, but if it could keep your python running it wouldn't have to pay the price of initializing the CNN for every image.

https://github.com/stutrek/live-tongue-shape

Hope you like it!