How do I change loss function to CTC?? about avsr-tf1 HOT 2 CLOSED

Hello @rlawoqls7
I am not sure if using the CTC loss on the decoder outputs could work well.

We previously used the CTC loss with the encoder, complementing the decoder's cross-entropy loss, as on this diagram. It was a different task though, lipreading.

To add the CTC loss, you could first project the encoder outputs to a dimension equal to the vocabulary size plus one, and call tf.nn.ctc_loss with the projected encoder outputs as your inputs. Additionally, you may have to convert the labels tensor from the current format (a sequence of integers) to the tf.SparseTensor representation.

from avsr-tf1.

Please let us know in case you are still facing difficulties with the implementation.
Christian @saamc may be able to assist, as he has recently explored some ideas based on the CTC loss.

from avsr-tf1.