How to use an external RNN-LM (mono-lingual) with a bilingual ASR? about icefall HOT 3 CLOSED

I think it's possible as long as the German bpe and English bpe are distinguishable.

And you also need to make sure which language you are decoding, otherwise you might end up rescoring the German utterance with English RNNLM.

from icefall.

but wouldnt different vocab size of the BPE model for ASR and RNN-LM create an issue in the first place.

When the loading the RNN LM

            model = RnnLmModel(
                vocab_size=params.vocab_size,
                embedding_dim=params.rnn_lm_embedding_dim,
                hidden_dim=params.rnn_lm_hidden_dim,
                num_layers=params.rnn_lm_num_layers,
                tie_weights=params.rnn_lm_tie_weights,
            )

params.vocab_size is the size of the sentence piece tokenizer from ASR (1000 in my case), which is different from the actual RNN LM vocab size (500 in my case). How can I overcome this?

from icefall.

You need to change the code, I only mean that it's theoretically possible to use a mono-lingual RNNLM to rescore multi-lingual ASR model.

from icefall.