Training own dataset about lipnet HOT 7 CLOSED

The GRID Corpus which was the dataset I used has the following structure to their alignment files:

0 23750 sil
23750 29500 bin
29500 34000 blue
34000 35500 at
35500 41000 f
41000 47250 two
47250 53000 now
53000 74500 sil

The first number is the timestamp of the beginning of the word (or silence period, denoted with sil) and the second number the timestamp for the end.

You could manually go over your dataset and write the timestamps and transcriptions for each video if it's small enough. Or you could use any of the available audio speech recognition libraries. You can try with pocketsphinx.

My particular implementation of lipnet does not use the timestamps, just the text. But you might want to experiment with the times too.

Hope this helped a bit!

from lipnet.

Thanks for the suggestion. What do you mean by just the text, is your align files different from GRID corpus align files?

from lipnet.

No, they are the same aligns as in the GRID Corpus but I just extract the words and ignore the numbers before handing that data to the model.

from lipnet.

OK thanks for the help. :)

from lipnet.

from lipnet.

from lipnet.

from lipnet.