Git Product home page Git Product logo

lipreadinggui's Introduction

LipReadingGUI

This is an implementation of LipNet for CSCE 629 - Neural Networks at Texas A&M University. Reference - https://github.com/rizkiarm/LipNet Research Paper - https://arxiv.org/abs/1611.01599

The repository contains a small subset of GRID CORPUS to see how code for preprocessing, training and prediction works.

Dataset Preprocessing

Run mouth_extract.py in the MouthExtract directory. The code loads videos from the Video folder and saves each video as 75 mouth-crop frames.

Network Training

Run train.py in the Training directory. The code loads training data which is preprocessed.

Prediction

Run GUI.py in the PredictGUI directory.

lipreadinggui's People

Contributors

abishalini avatar

Stargazers

 avatar

Watchers

James Cloos avatar  avatar paper2code - bot avatar

lipreadinggui's Issues

a bytes-like object is required, not str

Hello author, when I run train.py, it appears
File "LipReadingGUI-master/Training/lipnet/callbacks.py", line 86, in on_epoch_end
"{0:.5f}".format(stats['bleu'][0]), "{0:.5f}".format(stats['bleu'][1])])
TypeError: a bytes-like object is required, not 'str'
I am using python 3.6, how to solve this situation?

Installation issues

There are a lot of errors and issues while installing all the dependencies as many of them are outdated

So can you please provide requirements.txt for this project?

And if you have a container image of this project can you please share it on dockerhub

Thank you

ValueError: Input tensors to a Functional must come from `tf.keras.Input`

Hello. I get this error while running the GUI.py. Kindly let me know how to solve this. Thankyou

Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\pedne\anaconda3\lib\tkinter_init_.py", line 1883, in call
return self.func(*args)
File "C:\Users\pedne\Desktop\onkar\Upwork\LipReadingGUI-master\PredictGUI\GUI.py", line 50, in caption
video, result = predict(WEIGHTS_PATH, VIDEO_PATH)
File "C:\Users\pedne\Desktop\onkar\Upwork\LipReadingGUI-master\PredictGUI\predict.py", line 59, in predict
y_pred = lipnet.predict(X_data)
File "C:\Users\pedne\Desktop\onkar\Upwork\LipReadingGUI-master\PredictGUI..\Training\lipnet\model.py", line 74, in predict
return self.test_function([input_batch])[0] # the first 0 indicates test
File "C:\Users\pedne\Desktop\onkar\Upwork\LipReadingGUI-master\PredictGUI..\Training\lipnet\model.py", line 79, in test_function
return K.function([self.input_data, K.learning_phase()], [self.y_pred, K.learning_phase()])
File "C:\Users\pedne\anaconda3\lib\site-packages\keras\backend.py", line 4062, in function
model = models.Model(inputs=inputs, outputs=outputs)
File "C:\Users\pedne\anaconda3\lib\site-packages\tensorflow\python\training\tracking\base.py", line 522, in _method_wrapper
result = method(self, *args, **kwargs)
File "C:\Users\pedne\anaconda3\lib\site-packages\keras\engine\functional.py", line 110, in init
self._init_graph_network(inputs, outputs)
File "C:\Users\pedne\anaconda3\lib\site-packages\tensorflow\python\training\tracking\base.py", line 522, in _method_wrapper
result = method(self, *args, **kwargs)
File "C:\Users\pedne\anaconda3\lib\site-packages\keras\engine\functional.py", line 147, in _init_graph_network
self._validate_graph_inputs_and_outputs()
File "C:\Users\pedne\anaconda3\lib\site-packages\keras\engine\functional.py", line 683, in _validate_graph_inputs_and_outputs
raise ValueError('Input tensors to a ' + cls_name + ' ' +
ValueError: Input tensors to a Functional must come from tf.keras.Input. Received: 0 (missing previous layer metadata).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.