Any plans to release a new recipe for Chinese corpus? about icefall HOT 11 CLOSED

@tz301 Thanks, will update the documents.

@pkufool I have reproduce the results, excellent!

Some advice is that valid loss is missed, since the data per epoch is so small and valid loss computation will be skiped. During my reproduction, I have change valid log interval to 1000 and we can see valid loss on tensorboard below. CER is nearly the same (4.24%), compared with master (4.26%).

from icefall.

There is an ongoing PR for the AISHELL dataset. Will be ready soon, I think. @pkufool

from icefall.

there is a PR #123

from icefall.

there is a PR #123

thanks~

from icefall.

There is an ongoing PR for the AISHELL dataset. Will be ready soon, I think. @pkufool

thanks~

from icefall.

There is an ongoing PR for the AISHELL dataset. Will be ready soon, I think. @pkufool

thanks~

from icefall.

It has been merged. You can find it at https://github.com/k2-fsa/icefall/tree/master/egs

from icefall.

That's very cool work.

When I see the aishell document and reproduce it, I find my att loss has obvious mismatch (much higher) with that in the tutorial docs (ctc loss is ok). I found that was caused by the change of LabelSmoothing algorithm in #109. Maybe @pkufool can help to update that? I'm a little confusing at first glance. So maybe also for other new users.

Also, if there will be some scripts to deal with both Chinese and English, that will be very great. Maybe combine bpe for English and char for Chinese?. Chinese with English (code-switch) is so common today!

from icefall.

When I see the aishell document and reproduce it, I find my att loss has obvious mismatch (much higher) with that in the tutorial docs (ctc loss is ok)

Please see #107

Before the change, the label smoothing loss has an additional term

q * log(q)

which is negative as q is in the range (0, 1); therefore the loss before the change is smaller, but it should not affect your final WER.

from icefall.

@tz301 Thanks, will update the documents.

from icefall.

Also, if there will be some scripts to deal with both Chinese and English, that will be very great. Maybe combine bpe for English and char for Chinese?. Chinese with English (code-switch) is so common today!

Thanks, it is a good idea, will consider adding code-switch recipe. You are very welcome to contribute this recipe as well, if you have time.

from icefall.