Using a BTC/OTC in the training Zipformer instead of Conformer. about icefall HOT 10 CLOSED

from icefall.

@DongjiGao any interest?

from icefall.

Thanks, Dr. Daniel Povey, JinZr,
Hello @DongjiGao,

1- Is there any script to create lang OTC based on Phone lexicon instead of BPE (k2-fsa/icefall/tree/master/egs/librispeech/WSASR/local/prepare_otc_lang_bpe.py), since the paper showed a better performance in the Phone Based Lexicon over (Bypass Temporal Classification)?
2- Could this Method be used to clean up the errors present in the training transcripts, something similar to original Kaldi (/egs/wsj/s5/steps/cleanup/clean_and_segment_data.sh or clean_and_segment_data_nnet3.sh ) ?
3- Is there any plan to extend that to be used in ONNX (decoder with int8) ?

Thanks in advance,
Kerolos

from icefall.

Thanks, Dr. Daniel Povey, JinZr, Hello @DongjiGao,

1- Is there any script to create lang OTC based on Phone lexicon instead of BPE (k2-fsa/icefall/tree/master/egs/librispeech/WSASR/local/prepare_otc_lang_bpe.py), since the paper showed a better performance in the Phone Based Lexicon over (Bypass Temporal Classification)? 2- Could this Method be used to clean up the errors present in the training transcripts, something similar to original Kaldi (/egs/wsj/s5/steps/cleanup/clean_and_segment_data.sh or clean_and_segment_data_nnet3.sh ) ? 3- Is there any plan to extend that to be used in ONNX (decoder with int8) ?

Thanks in advance, Kerolos

Thank you for your interest.

1. I can check the phone-based script if needed. We modified how we model the star token in OTC (as the average probability of all non-blank tokens instead of an individual token), so I would guess the performance would be similar to using BPE.
2. Yes, please refer to this script for more details. It does "flexible alignment" by replacing suspicious tokens or wrongly inserted tokens with a star, and by placing a star where a word is missing.
3. We do not have such a plan in the near term. Do you have any suggestions?

Dongji

from icefall.

Thanks for the quick response, @DongjiGao
1- It would be great if you could check the file in prepare_otc_lang.py (using the phone based lexicon ) and any related files that required to complete the training. I would really appreciate that.
2-It seems quite helpful. I'll definitely give it a try.
3- Perhaps my focusing on utilizing Zipformer could be a primary consideration, and then exporting it to ONNX.
BR, Kerolos

from icefall.