soujanyaporia / mustard Goto Github PK

Hi,
I was trying to play around with the data set. But I was confused regarding the shape of the dataset.
I understand that the shape of the feature-vectors for context is
(#samples x #sentences x feature-vector-size)
But on the other hand this was true only for Text and Visual. I was not sure for audio.
Can you please clarify regarding the same.?

Also I would like to know is there anyway I could get the features word wise?
Thank you.

Is it possible to see the audio extraction python script to fully analyze how it works in detail?

As well, how were you able to reduce the laugh track as per your paper "Then we remove background noise from the signal
by applying a heuristic vocal-extraction method."

Hi,

Please convert the code into a Google Collab notebook and drop a link in the README. Love to play with it but it's not clear what the sizes of dataset and model would be and I have a tiny MBP so would be great to run on Collab.

Hi
Thank you for your work and code
Could I get the context_final data by following the Visual Feature Extraction steps?
It seems that I could only get the features/utterances_final hdf5 data.
Did I miss anything in the process?
Thanks.

Hi, I am working with a project that uses your methods for feature extraction on facial features. I am wondering how to extract the 512 resnet features using your model. When I extract features using the process described in the visual folder I get 2048 values.

Kind regards

It seems that there is no validation set for model optimization.
Though I can find the argument of val_split = 0.1 in the config.py file, I cannot find where it is called to form the validation set in other files.
For the Speaker-dependent setting, I can get the idea of Cross Validation. But for Speaker-independent setting, how would you select the model?

Below is the error i get when i try to run the file train_svm.py

_pickle.UnpicklingError: the STRING opcode argument must be quoted

Can yu please suggest ?

Why is there no sound in some videos in the data? This means that some data is missing audio.

Can

Hi
Thank you for your work and code
I tried to reproduce the results shown in th paper but noticed large degradations of performance among all configs.

For example, I got

 weighted avg      0.574     0.584     0.573       356

for independent T+A

weighted avg      0.602     0.587     0.589       356

for independent T+V

Weighted Precision: 0.483  Weighted Recall: 0.472  Weighted F score: 0.472

for dependent T

Weighted Precision: 0.629 Weighted Recall: 0.626 Weighted F score: 0.626
for dependent T+V

Did I miss anything or could you suggest some training tricks?

Thanks