How can I use external LM and fusion LM? about neural_sp HOT 4 CLOSED

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉| 370180/370189 [3:41:03<00:01,  5.61it/s]Traceback (most recent call last):
  File "/home/jupyter-nakachi-s/neural_sp/examples/csj/s5/../../../neural_sp/bin/asr/train.py", line 529, in <module>
    save_path = pr.runcall(main)
  File "/opt/tljh/user/envs/ncr_env/lib/python3.7/cProfile.py", line 121, in runcall
    return func(*args, **kw)
  File "/home/jupyter-nakachi-s/neural_sp/examples/csj/s5/../../../neural_sp/bin/asr/train.py", line 424, in main
    optimizer.n_epochs + 1, logger)
  File "/home/jupyter-nakachi-s/neural_sp/examples/csj/s5/../../../neural_sp/bin/asr/train.py", line 481, in evaluate
    metric, cer = eval_wordpiece(models, dataset, recog_params, epoch=epoch)
  File "/home/jupyter-nakachi-s/neural_sp/neural_sp/evaluators/wordpiece.py", line 82, in eval_wordpiece
    ensemble_models=models[1:] if len(models) > 1 else [])
  File "/home/jupyter-nakachi-s/neural_sp/neural_sp/models/seq2seq/speech2text.py", line 707, in decode
    exclude_eos, refs_id, utt_ids, speakers)
  File "/home/jupyter-nakachi-s/neural_sp/neural_sp/models/seq2seq/decoders/las.py", line 911, in greedy
    lmout, lmstate = self.lm.decode(self.lm(y), lmstate)
  File "/home/jupyter-nakachi-s/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/jupyter-nakachi-s/neural_sp/neural_sp/models/lm/lm_base.py", line 62, in forward
    loss, state, observation = self._forward(ys, state)
  File "/home/jupyter-nakachi-s/neural_sp/neural_sp/models/lm/lm_base.py", line 66, in _forward
    ys = [np2tensor(y, self.device_id) for y in ys]  # <eos> is included
  File "/home/jupyter-nakachi-s/neural_sp/neural_sp/models/lm/lm_base.py", line 66, in <listcomp>
    ys = [np2tensor(y, self.device_id) for y in ys]  # <eos> is included
  File "/home/jupyter-nakachi-s/neural_sp/neural_sp/models/torch_utils.py", line 49, in np2tensor
    tensor = torch.from_numpy(array)
TypeError: expected np.ndarray (got Tensor)