jctian98 / e2e_lfmmi Goto Github PK
View Code? Open in Web Editor NEWE2E system with LF-MMI; word N-gram for Mandarin
E2E system with LF-MMI; word N-gram for Mandarin
When I kick off training, it met error.
File "/home/Data/jing.lu/project/e2e_lfmmi/egs/aishell1.bk/espnet/asr/pytorch_backend/asr.py", line 694, in train
updater = CustomUpdater(
File "/home/Data/jing.lu/project/e2e_lfmmi/egs/aishell1.bk/espnet/asr/pytorch_backend/asr.py", line 186, in init
self.device = device
AttributeError: can't set attribute
Traceback (most recent call last):
File "/home/Data/jing.lu/project/e2e_lfmmi/egs/aishell1.bk/../..//bin/asr_train.py", line 699, in
main(sys.argv[1:])
File "/home/Data/jing.lu/project/e2e_lfmmi/egs/aishell1.bk/../..//bin/asr_train.py", line 685, in main
train(args)
File "/home/Data/jing.lu/project/e2e_lfmmi/egs/aishell1.bk/espnet/asr/pytorch_backend/asr.py", line 694, in train
updater = CustomUpdater(
File "/home/Data/jing.lu/project/e2e_lfmmi/egs/aishell1.bk/espnet/asr/pytorch_backend/asr.py", line 186, in init
self.device = device
AttributeError: can't set attribute
/home/Data/jing.lu/tools/wenet/wenet/miniconda3/envs/K2/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects --local_rank
argument to be set, please
change it to read from os.environ['LOCAL_RANK']
instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions
Hello, please ask the following this contains all the code related to e2e lfmmi, which feels missing when executing (e.g., lfmmi criterion)
Hi,
Thanks for publishing your work this is very interesting.
Is there any chance that you can upload the librispeech egs folder so others can reproduce it?
Hi, when I want to training model with aishell1, I meet the problem that connect() timeout . Can you help me?
Thx again.
非常有用的code-switch的工作!我有一个小小的疑问:
ys_pad, cls_ids = ys_pad[:, 1:], ys_pad[:, 0].squeeze(0)
这里的cls_ids是用的全局的language id作为label吗?分三类:中文、英文、中英混?
Hi!
I met this error while running lm-mmi decoding. If I set nj=188
,10 jobs won't have this issue and have a common decoing result. But if I set nj=50
, all of my jobs is crashed.
number of phones 218
Found parameter lm_scores with shape torch.Size([47960])
Found parameter lo.1.weight with shape torch.Size([219, 512])
Found parameter lo.1.bias with shape torch.Size([219])
Using MMI scorer type: frame
MMI Scorer Module: <class 'espnet.nets.scorers.mmi_rnnt_scorer.MMIRNNTScorer'>
Traceback (most recent call last):
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/../..//bin/asr_recog.py", line 456, in
main(sys.argv[1:])
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/../..//bin/asr_recog.py", line 433, in main
recog(args)
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/asr/pytorch_backend/asr.py", line 1197, in recog
word_ngram_scorer = word_ngram_scorer(
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/nets/scorers/word_ngram.py", line 179, in init
self.WordNgram = WordNgram(lang, device)
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/nets/scorers/word_ngram.py", line 51, in init
self.load_G()
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/nets/scorers/word_ngram.py", line 66, in load_G
fcntl.flock(f, fcntl.LOCK_EX) # lock
OSError: [Errno 38] Function not implemented
Any idea how to solve this issue?
How to use your trained model directly, can you give an example? thank you very much
Hi, I tried running nets/scorers/word_ngram.py
, but the result doesn't seem to be as expected
if __name__ == "__main__":
device = torch.device("cuda:0")
lang = sys.argv[1]
word_ngram = WordNgram(lang, device)
texts = ["甚至出现交易停滞的情况", "甚至出现交易停滞的情形", "欲擒故纵", "欲擒故放"]
for i in range(1):
scores = word_ngram.score_texts(texts, log_semiring=True)
print(scores)
result
tensor([-inf, -inf, -inf, -inf], device='cuda:0', dtype=torch.float64)
G.fst.txt:
ngram-count -order 3 -lm aishell.arpa -kndiscount -interpolate -text text
python3 -m kaldilm \
--read-symbol-table="words.txt" \
--disambig-symbol='#0' \
--max-order=3 \
aishell.arpa > G.fst.txt
The words.txt
file has 137079 lines, and nothing abnormal was found.
thanks.
Thanks for this great framework for unified e2e lfmmi training and inference!
I noticed that snowfall has updated the MmiTrainingGraphCompiler
in snowfall/training/mmi_graph.py:135 from
ctc_topo_P_vec = k2.create_fsa_vec([ctc_topo_P.detach()])
ctc_topo_P_vec = k2.create_fsa_vec([self.ctc_topo_P])
Since in this repo the denominator FSA vec is detached, It seems that bigram LM FSA parameters can only get supervision (gradient) from numerator FSA.
I'm not sure if I missed something or the detach operation could be a problem.
Any help or explanation would be appreciated. Thanks!
Could you please tell me the version of your lhotse?
I have problem running the decoding benchmark because librosa need numpy<1.22, while lhotse is complied with numpy==1.22, the error are as follws
2022-02-22 17:47:07,153 (asr_init:162) WARNING: reading model parameters from exp/train_sp_pytorch_8v100_ddp_rnnt_mmi/results_0/model.last91_100.avg.best
RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe
Traceback (most recent call last):
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/../..//bin/asr_recog.py", line 456, in
main(sys.argv[1:])
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/../..//bin/asr_recog.py", line 433, in main
recog(args)
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/asr/pytorch_backend/asr.py", line 1063, in recog
model, train_args = load_trained_model(args.model, training=False)
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/asr/pytorch_backend/asr_init.py", line 172, in load_trained_model
model_class = dynamic_import(model_module)
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/utils/dynamic_import.py", line 22, in dynamic_import
m = importlib.import_module(module_name)
File "/mypath/install_dir/anaconda3/envs/k2/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 843, in exec_module
File "", line 219, in _call_with_frames_removed
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/nets/pytorch_backend/e2e_asr_transducer.py", line 52, in
from espnet.snowfall.warpper.warpper_mmi import K2MMI
File "/mypath/work/k2/E2E-ASR-Framework/egs/aishell1/espnet/snowfall/warpper/warpper_mmi.py", line 19, in
from lhotse.utils import nullcontext
File "/mypath/install_dir/anaconda3/envs/k2/lib/python3.8/site-packages/lhotse/init.py", line 4, in
from .cut import CutSet, MonoCut
File "/mypath/install_dir/anaconda3/envs/k2/lib/python3.8/site-packages/lhotse/cut.py", line 33, in
from lhotse.features import (
File "/mypath/install_dir/anaconda3/envs/k2/lib/python3.8/site-packages/lhotse/features/init.py", line 1, in
from .base import (
File "/mypath/install_dir/anaconda3/envs/k2/lib/python3.8/site-packages/lhotse/features/base.py", line 19, in
from lhotse.features.io import FeaturesWriter, get_reader
File "/mypath/install_dir/anaconda3/envs/k2/lib/python3.8/site-packages/lhotse/features/io.py", line 7, in
import lilcom
File "/mypath/install_dir/anaconda3/envs/k2/lib/python3.8/site-packages/lilcom/init.py", line 2, in
from .lilcom_interface import compress, decompress, get_shape
File "/mypath/install_dir/anaconda3/envs/k2/lib/python3.8/site-packages/lilcom/lilcom_interface.py", line 3, in
from . import lilcom_extension
Thank you very much.
hello jctian98
Impressive work!
Very happy to see that aishell's sota has been refreshed again.
I have some doubts, hope you can help me figure it out,
Why the aishell-1 result on paper with code leaderboard is 4.18%
https://paperswithcode.com/sota/speech-recognition-on-aishell-1
but the result on aishell-1 in you paper is, 4.10%,
Is there any different between two result?
Is the dev set used during training?
Sincerely hope to get your reply.
I have tried to remove the options about DDP,after this,can i run this repo on one mechaine?
Hello,I am trying to train locally with multiple gpus and I am getting some errors. There are always four on the local machine. I want to set two cards for training. I have set the DDP parameters as follows, but some errors have occurred. Please, how do I set them?
export HOST_GPU_NUM=2
export HOST_NUM=1
export NODE_NUM=1
export INDEX=0
One gpu can be used for normal training, but multiple gpus cannot be executed normally
error log:
2022-08-30 17:59:56,822 (ctc:138) INFO: CTC input lengths: tensor([140, 140, 140, 140, 140, 140, 140, 140, 140, 140, 140, 140, 140, 140, 140, 140], device='cuda:0')
2022-08-30 17:59:56,823 (ctc:143) INFO: CTC output lengths: tensor([23, 18, 22, 23, 20, 22, 22, 22, 22, 21, 21, 24, 22, 21, 17, 23], device='cuda:0')
2022-08-30 17:59:56,823 (ctc:154) INFO: ctc loss:1071.3641357421875
2022-08-30 17:59:57,021 (e2e_asr_transducer:92) INFO: loss:1988.887451171875
2022-08-30 17:59:57,528 (asr:250) INFO: on device cuda:0 grad norm=7541.00927734375
[E ProcessGroupNCCL.cpp:587] [Rank 0] Watchdog caught collective operation timeout: WorkNCCL(OpType=ALLREDUCE, Timeout(ms)=1800000) ran for 1801883 milliseconds before timing out.
[E ProcessGroupNCCL.cpp:341] Some NCCL operations have failed or timed out. Due to the asynchronous nature of CUDA kernels, subsequent GPU operations might run on corrupted/incomplete data. To avoid this inconsistency, we are taking the entire process down.
2022-08-30 18:30:01,813 (ctc:138) INFO: CTC input lengths: tensor([123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, 123], device='cuda:1')
2022-08-30 18:30:01,813 (ctc:143) INFO: CTC output lengths: tensor([22, 21, 22, 23, 19, 21, 20, 20, 21, 26, 20, 21, 22, 24, 14, 24], device='cuda:1')
2022-08-30 18:30:01,814 (ctc:154) INFO: ctc loss:936.232421875
2022-08-30 18:30:02,099 (e2e_asr_transducer:92) INFO: loss:1777.2901611328125
2022-08-30 18:30:02,719 (asr:250) INFO: on device cuda:1 grad norm=6572.220703125
Tue Aug 30 18:30:02 2022 | rank: 1 | | iteration: 0 | gradient applied
2022-08-30 18:30:02,909 (ctc:138) INFO: CTC input lengths: tensor([34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34, 34], device='cuda:1')
2022-08-30 18:30:02,910 (ctc:143) INFO: CTC output lengths: tensor([4, 9, 9, 6, 6, 9, 7, 6, 4, 4, 6, 8, 6, 9, 6, 5], device='cuda:1')
2022-08-30 18:30:02,910 (ctc:154) INFO: ctc loss:262.8006286621094
2022-08-30 18:30:02,972 (e2e_asr_transducer:92) INFO: loss:503.36956787109375
2022-08-30 18:30:03,115 (asr:250) INFO: on device cuda:1 grad norm=1751.3741455078125
/home/miniconda3/envs/lfmmi/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects --local_rank
argument to be set, please
change it to read from os.environ['LOCAL_RANK']
instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions
warnings.warn(
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 147095 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -6) local_rank: 0 (pid: 147080) of binary: /home/miniconda3/envs/lfmmi/bin/python
Traceback (most recent call last):
File "/home/miniconda3/envs/lfmmi/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/miniconda3/envs/lfmmi/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/miniconda3/envs/lfmmi/lib/python3.8/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in
main()
File "/home/miniconda3/envs/lfmmi/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/home/miniconda3/envs/lfmmi/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/home/miniconda3/envs/lfmmi/lib/python3.8/site-packages/torch/distributed/run.py", line 710, in run
elastic_launch(
File "/home/miniconda3/envs/lfmmi/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/miniconda3/envs/lfmmi/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 259, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
I followed the instructions, but there is always a version conflict. How can I improve it?
[root@b394acbc6baf tools]# make TH_VERSION=1.7.1 CUDA_VERSION=10.1
CUDA_VERSION=10.1
PYTHON=/anaconda/envs/lfmmi/bin/python3
PYTHON_VERSION=Python 3.9.15
USE_CONDA=1
TH_VERSION=1.7.1
WITH_OMP=ON
. ./activate_python.sh && ./installers/install_torch.sh "true" "1.7.1" "10.1"
2022-12-07T05:51:42 (install_torch.sh:149:main) [INFO] python_version=3.9.15
2022-12-07T05:51:42 (install_torch.sh:150:main) [INFO] torch_version=1.7.1
2022-12-07T05:51:42 (install_torch.sh:151:main) [INFO] cuda_version=10.1
2022-12-07T05:51:43 (install_torch.sh:97:install_torch) conda install -y pytorch=1.7.1 torchaudio=0.7.2 cudatoolkit=10.1 -c pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package pytorch conflicts for:
torchaudio=0.7.2 -> pytorch==1.7.1
pytorch=1.7.1
Package _openmp_mutex conflicts for:
pytorch=1.7.1 -> libgcc-ng[version='>=7.3.0'] -> _openmp_mutex[version='>=4.5']
python=3.9 -> libgcc-ng[version='>=11.2.0'] -> _openmp_mutex[version='>=4.5']
Package cudatoolkit conflicts for:
torchaudio=0.7.2 -> pytorch==1.7.1 -> cudatoolkit[version='>=10.1,<10.2|>=11.0,<11.1|>=10.2,<10.3|>=9.2,<9.3']
cudatoolkit=10.1The following specifications were found to be incompatible with your system:
Your installed version is: 2.17
make: *** [Makefile:102: pytorch.done] Error 1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.