sjtuwxz / loconet_asd Goto Github PK

Hi,

I am really interested in your model. However, I do not know how to apply your model to a new fresh video data. Can you help me answer several answers?

1. I am currently having a mp4 file. I have extracted into frames and mel-spectrogram data. However, the time stamp for video and audio does not match, do I need to match those?
2. For the face crop, which model can I use and do they need to have the same dimension after cropping because from your paper, they need to be H x W?
3. After having the face crop and audio data, Do I need to do any preprocessing? If yes, is there any note how to preprocess the 2 data.
4. In the inference step, If I am not wrong, I need to load the loconet model from loconet.py with weight and run similar to evaluate_network function right?

Hope you can answer my question or even better if you can create new new pyinb file with an example on how to run the model on new video.

May I kindly request if you could provide the preprocessing and testing codes for the Talkies dataset, please? I would really appreciate it if that's possible.