jinserk / pytorch-asr Goto Github PK
View Code? Open in Web Editor NEWASR with PyTorch
License: GNU General Public License v3.0
ASR with PyTorch
License: GNU General Public License v3.0
Hi, Could you please offer some demo for this project?
it seems like that set_logfile is deprecated.
still many scripts are trying to import set_logfile from utils.logger
hi,jinserk, where can i download the train dataset(aspire+swbd)?
thanks very much!
Hi @jinserk ,
Have you ever encountered the following error? My system is Ubuntu 16.04.5 LTS with Python 3.6.8.
python train.py deepspeech_ctc --use-cuda
Traceback (most recent call last):
File "train.py", line 21, in <module>
m = importlib.import_module(f"asr.models.{model}")
File "/home/kwon/.pyenv/versions/3.6.8/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/media/kwon/DISK2/DEV/pytorch-asr/asr/models/deepspeech_ctc/__init__.py", line 1, in <module>
from .train import batch_train, train, test
File "/media/kwon/DISK2/DEV/pytorch-asr/asr/models/deepspeech_ctc/train.py", line 14, in <module>
from asr.kaldi.latgen import LatGenCTCDecoder
File "/media/kwon/DISK2/DEV/pytorch-asr/asr/kaldi/latgen/__init__.py", line 1, in <module>
from ._latgen import Labeler, LatGenDecoder, LatGenCTCDecoder
File "/media/kwon/DISK2/DEV/pytorch-asr/asr/kaldi/latgen/_latgen.py", line 8, in <module>
import torch_asr._latgen_lib as latgen_lib
ImportError: /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages/torch_asr-0.0.1-py3.6-linux-x86_64.egg/torch_asr/_latgen_lib.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZN5kaldi12ParseOptions16RegisterStandardIbEEvRKSsPT_S3_
It worked well until the above error came up.
cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
python -V
Python 3.6.8
2019-02-06 21:44:30,423 [DEBUG] pytorch version: 1.0.0
2019-02-06 21:44:30,476 [DEBUG] pytorch-asr version: 08c28aad71a28810672884ef24d0d325d319bd83 (2019-02-06 21:33:21-08:00)
2019-02-06 21:44:30,477 [INFO ] target data path : /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data
2019-02-06 21:44:30,477 [INFO ] processing "train" ...
2019-02-06 21:44:30,500 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/train/segments file ...
100%|█████████████████████████████████████████████████████| 92973/92973 [00:00<00:00, 631059.69it/s]
2019-02-06 21:44:30,708 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/train/wav.scp file ...
100%|███████████████████████████████████████████████████████████| 1495/1495 [11:29<00:00, 1.99it/s]
2019-02-06 21:56:00,396 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/train/text file ...
100%|████████████████████████████████████████████████████████| 92973/92973 [01:42<00:00, 905.67it/s]
2019-02-06 21:57:43,361 [INFO ] generating manifest to "train.csv" ...
100%|█████████████████████████████████████████████████████| 92973/92973 [00:00<00:00, 407827.62it/s]
2019-02-06 21:57:43,663 [INFO ] total 92967 entries listed in the manifest file.
2019-02-06 21:57:43,663 [INFO ] min: 0.60 sec max: 41.02 sec
2019-02-06 21:57:43,663 [INFO ] <5 secs: 10.53 % <10 secs: 26.68 % <15 secs: 43.37 % <20 secs: 68.16 % <25 secs: 83.52 % <30 secs: 93.06 %
2019-02-06 21:57:43,678 [INFO ] processing "dev" ...
2019-02-06 21:57:43,678 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/dev/segments file ...
100%|█████████████████████████████████████████████████████████| 507/507 [00:00<00:00, 606016.57it/s]
2019-02-06 21:57:43,816 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/dev/wav.scp file ...
100%|█████████████████████████████████████████████████████████████████| 8/8 [00:03<00:00, 1.85it/s]
2019-02-06 21:57:47,146 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/dev/text file ...
100%|████████████████████████████████████████████████████████████| 507/507 [00:00<00:00, 718.37it/s]
2019-02-06 21:57:48,007 [INFO ] generating manifest to "dev.csv" ...
100%|█████████████████████████████████████████████████████████| 507/507 [00:00<00:00, 415091.18it/s]
2019-02-06 21:57:48,009 [INFO ] total 507 entries listed in the manifest file.
2019-02-06 21:57:48,009 [INFO ] min: 0.94 sec max: 79.82 sec
2019-02-06 21:57:48,009 [INFO ] <5 secs: 4.14 % <10 secs: 10.85 % <15 secs: 25.44 % <20 secs: 44.18 % <25 secs: 62.33 % <30 secs: 78.30 %
2019-02-06 21:57:48,009 [INFO ] processing "test" ...
2019-02-06 21:57:48,009 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/test/segments file ...
100%|███████████████████████████████████████████████████████| 1155/1155 [00:00<00:00, 686470.33it/s]
2019-02-06 21:57:48,025 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/test/wav.scp file ...
100%|███████████████████████████████████████████████████████████████| 11/11 [00:05<00:00, 2.32it/s]
2019-02-06 21:57:53,896 [INFO ] processing /media/kwon/DISK2/DEV/kaldi_b9d4b76/egs/tedlium/s5_r2/data/test/text file ...
100%|██████████████████████████████████████████████████████████| 1155/1155 [00:01<00:00, 893.51it/s]
2019-02-06 21:57:55,276 [INFO ] generating manifest to "test.csv" ...
100%|███████████████████████████████████████████████████████| 1155/1155 [00:00<00:00, 381600.72it/s]
2019-02-06 21:57:55,279 [INFO ] total 1155 entries listed in the manifest file.
2019-02-06 21:57:55,280 [INFO ] min: 0.70 sec max: 65.10 sec
2019-02-06 21:57:55,280 [INFO ] <5 secs: 9.26 % <10 secs: 27.19 % <15 secs: 44.16 % <20 secs: 67.88 % <25 secs: 84.16 % <30 secs: 95.84 %
2019-02-06 21:57:55,280 [INFO ] data preparation finished.
python setup.py install
## download & prepare model...
downloading pretrained ASpIRE model
0001_aspire_chain_model.tar.gz 100%[============================================================================>] 431.50M 10.2MB/s in 45s
Composing decoding graph T.fst succeeded
fstminimizeencoded
fstpushspecial
fstdeterminizestar --use-log=true
fsttablecompose /media/kwon/DISK2/DEV/pytorch-asr/asr/kaldi/scripts/aspire/data/lang_pp_test/L_disambig.fst /media/kwon/DISK2/DEV/pytorch-asr/asr/kaldi/scripts/aspire/data/lang_pp_test/G.fst
fstisstochastic /media/kwon/DISK2/DEV/pytorch-asr/asr/kaldi/scripts/../graph/LG.fst
-0.167291 -0.16797
[info]: /media/kwon/DISK2/DEV/pytorch-asr/asr/kaldi/scripts/../graph/LG.fst is not stochastic
Composing decoding graph LG.fst succeeded
fsttablecompose /media/kwon/DISK2/DEV/pytorch-asr/asr/kaldi/scripts/../graph/T.fst /media/kwon/DISK2/DEV/pytorch-asr/asr/kaldi/scripts/../graph/LG.fst
Composing decoding graph TLG.fst succeeded
## building latgen module...
running install
running bdist_egg
running egg_info
creating torch_asr.egg-info
writing torch_asr.egg-info/PKG-INFO
writing dependency_links to torch_asr.egg-info/dependency_links.txt
writing top-level names to torch_asr.egg-info/top_level.txt
writing manifest file 'torch_asr.egg-info/SOURCES.txt'
reading manifest file 'torch_asr.egg-info/SOURCES.txt'
writing manifest file 'torch_asr.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'torch_asr._latgen_lib' extension
creating build
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/src
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/media/kwon/DISK2/DEV/kaldi_b9d4b76/src -I/media/kwon/DISK2/DEV/kaldi_b9d4b76/tools/openfst/include -I/home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages/torch/lib/include -I/home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages/torch/lib/include/torch/csrc/api/include -I/home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages/torch/lib/include/TH -I/home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages/torch/lib/include/THC -I/home/kwon/.pyenv/versions/3.6.8/include/python3.6m -c src/latgen_lib.cc -o build/temp.linux-x86_64-3.6/src/latgen_lib.o -std=c++11 -w -fPIC -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_latgen_lib -D_GLIBCXX_USE_CXX11_ABI=0
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/torch_asr
g++ -pthread -shared -L/home/kwon/.pyenv/versions/3.6.8/lib -L/home/kwon/.pyenv/versions/3.6.8/lib build/temp.linux-x86_64-3.6/src/latgen_lib.o -L/media/kwon/DISK2/DEV/kaldi_b9d4b76/src/lib -L/media/kwon/DISK2/DEV/kaldi_b9d4b76/tools/openfst/lib -Wl,--enable-new-dtags,-R/media/kwon/DISK2/DEV/kaldi_b9d4b76/src/lib -Wl,--enable-new-dtags,-R/media/kwon/DISK2/DEV/kaldi_b9d4b76/tools/openfst/lib -lkaldi-nnet3 -lkaldi-decoder -lkaldi-sgmm2 -lkaldi-lm -lkaldi-ivector -lkaldi-chain -lkaldi-nnet2 -lkaldi-online2 -lkaldi-hmm -lkaldi-fstext -lkaldi-lat -lkaldi-tree -lkaldi-base -lkaldi-transform -lkaldi-rnnlm -lkaldi-gmm -lkaldi-kws -lkaldi-cudamatrix -lkaldi-nnet -lkaldi-matrix -lkaldi-feat -lkaldi-util -lfst -o build/lib.linux-x86_64-3.6/torch_asr/_latgen_lib.cpython-36m-x86_64-linux-gnu.so
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/torch_asr
copying build/lib.linux-x86_64-3.6/torch_asr/_latgen_lib.cpython-36m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg/torch_asr
creating stub loader for torch_asr/_latgen_lib.cpython-36m-x86_64-linux-gnu.so
byte-compiling build/bdist.linux-x86_64/egg/torch_asr/_latgen_lib.py to _latgen_lib.cpython-36.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying torch_asr.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch_asr.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch_asr.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying torch_asr.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
torch_asr.__pycache__._latgen_lib.cpython-36: module references __file__
creating dist
creating 'dist/torch_asr-0.0.1-py3.6-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing torch_asr-0.0.1-py3.6-linux-x86_64.egg
removing '/home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages/torch_asr-0.0.1-py3.6-linux-x86_64.egg' (and everything under it)
creating /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages/torch_asr-0.0.1-py3.6-linux-x86_64.egg
Extracting torch_asr-0.0.1-py3.6-linux-x86_64.egg to /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages
torch-asr 0.0.1 is already the active version in easy-install.pth
Installed /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages/torch_asr-0.0.1-py3.6-linux-x86_64.egg
Processing dependencies for torch-asr==0.0.1
Finished processing dependencies for torch-asr==0.0.1
pip install -r requirements.txt
Collecting git+https://github.com/pytorch/audio (from -r requirements.txt (line 5))
Cloning https://github.com/pytorch/audio to /tmp/pip-req-build-9x9pogbv
Requirement already satisfied (use --upgrade to upgrade): torchaudio==0.2 from git+https://github.com/pytorch/audio in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 5))
Collecting git+https://github.com/pytorch/text (from -r requirements.txt (line 8))
Cloning https://github.com/pytorch/text to /tmp/pip-req-build-yctuqyf6
Requirement already satisfied (use --upgrade to upgrade): torchtext==0.4.0 from git+https://github.com/pytorch/text in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 8))
Collecting git+https://github.com/pytorch/tnt (from -r requirements.txt (line 9))
Cloning https://github.com/pytorch/tnt to /tmp/pip-req-build-sokakvai
Requirement already satisfied (use --upgrade to upgrade): torchnet==0.0.4 from git+https://github.com/pytorch/tnt in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 9))
Collecting git+https://github.com/NVIDIA/apex (from -r requirements.txt (line 10))
Cloning https://github.com/NVIDIA/apex to /tmp/pip-req-build-r877_gbe
Requirement already satisfied (use --upgrade to upgrade): apex==0.1 from git+https://github.com/NVIDIA/apex in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 10))
Requirement already satisfied: numpy in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 1)) (1.16.1)
Requirement already satisfied: scipy in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 2)) (1.2.0)
Requirement already satisfied: torch in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 4)) (1.0.0)
Requirement already satisfied: torchvision in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 7)) (0.2.1)
Requirement already satisfied: pyro-ppl in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 11)) (0.3.0)
Requirement already satisfied: cffi in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 12)) (1.11.5)
Requirement already satisfied: pyyaml in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 13)) (3.13)
Requirement already satisfied: wheel in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 14)) (0.32.3)
Requirement already satisfied: scikit-learn in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 15)) (0.20.2)
Requirement already satisfied: matplotlib in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 16)) (3.0.2)
Requirement already satisfied: colorlover in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 17)) (0.3.0)
Requirement already satisfied: t_sne_bhcuda in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 18)) (0.2.1)
Requirement already satisfied: tqdm in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 19)) (4.30.0)
Requirement already satisfied: pysndfx in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 20)) (0.3.6)
Requirement already satisfied: python-Levenshtein in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 21)) (0.12.0)
Requirement already satisfied: visdom in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 22)) (0.1.8.8)
Requirement already satisfied: tensorboardx in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 23)) (1.6)
Requirement already satisfied: tensorboard in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 24)) (1.12.2)
Requirement already satisfied: slackclient in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 25)) (1.3.0)
Requirement already satisfied: gitpython in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from -r requirements.txt (line 26)) (2.1.11)
Requirement already satisfied: requests in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from torchtext==0.4.0->-r requirements.txt (line 8)) (2.21.0)
Requirement already satisfied: six in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from torchtext==0.4.0->-r requirements.txt (line 8)) (1.12.0)
Requirement already satisfied: pillow>=4.1.1 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from torchvision->-r requirements.txt (line 7)) (5.4.1)
Requirement already satisfied: contextlib2 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from pyro-ppl->-r requirements.txt (line 11)) (0.5.5)
Requirement already satisfied: graphviz>=0.8 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from pyro-ppl->-r requirements.txt (line 11)) (0.10.1)
Requirement already satisfied: networkx>=2.2 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from pyro-ppl->-r requirements.txt (line 11)) (2.2)
Requirement already satisfied: opt_einsum>=2.3.0 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from pyro-ppl->-r requirements.txt (line 11)) (2.3.2)
Requirement already satisfied: pycparser in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from cffi->-r requirements.txt (line 12)) (2.19)
Requirement already satisfied: kiwisolver>=1.0.1 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from matplotlib->-r requirements.txt (line 16)) (1.0.1)
Requirement already satisfied: cycler>=0.10 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from matplotlib->-r requirements.txt (line 16)) (0.10.0)
Requirement already satisfied: python-dateutil>=2.1 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from matplotlib->-r requirements.txt (line 16)) (2.7.5)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from matplotlib->-r requirements.txt (line 16)) (2.3.1)
Requirement already satisfied: setuptools in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from python-Levenshtein->-r requirements.txt (line 21)) (40.6.2)
Requirement already satisfied: tornado in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from visdom->-r requirements.txt (line 22)) (5.1.1)
Requirement already satisfied: pyzmq in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from visdom->-r requirements.txt (line 22)) (17.1.2)
Requirement already satisfied: torchfile in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from visdom->-r requirements.txt (line 22)) (0.1.0)
Requirement already satisfied: websocket-client in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from visdom->-r requirements.txt (line 22)) (0.54.0)
Requirement already satisfied: protobuf>=3.2.0 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from tensorboardx->-r requirements.txt (line 23)) (3.6.1)
Requirement already satisfied: werkzeug>=0.11.10 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from tensorboard->-r requirements.txt (line 24)) (0.14.1)
Requirement already satisfied: markdown>=2.6.8 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from tensorboard->-r requirements.txt (line 24)) (3.0.1)
Requirement already satisfied: grpcio>=1.6.3 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from tensorboard->-r requirements.txt (line 24)) (1.18.0)
Requirement already satisfied: gitdb2>=2.0.0 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from gitpython->-r requirements.txt (line 26)) (2.0.5)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from requests->torchtext==0.4.0->-r requirements.txt (line 8)) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from requests->torchtext==0.4.0->-r requirements.txt (line 8)) (2.8)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from requests->torchtext==0.4.0->-r requirements.txt (line 8)) (1.24.1)
Requirement already satisfied: certifi>=2017.4.17 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from requests->torchtext==0.4.0->-r requirements.txt (line 8)) (2018.11.29)
Requirement already satisfied: decorator>=4.3.0 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from networkx>=2.2->pyro-ppl->-r requirements.txt (line 11)) (4.3.2)
Requirement already satisfied: smmap2>=2.0.0 in /home/kwon/.pyenv/versions/3.6.8/lib/python3.6/site-packages (from gitdb2>=2.0.0->gitpython->-r requirements.txt (line 26)) (2.0.5)
Building wheels for collected packages: torchaudio, torchtext, torchnet, apex
Running setup.py bdist_wheel for torchaudio ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-7_odfrih/wheels/6b/7e/7a/ee4ed533517964e33dc9c0ec29435f8492c976ea85c795527b
Running setup.py bdist_wheel for torchtext ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-7_odfrih/wheels/73/14/71/ed033fd999ae4933e17df3e91be2014e61c2f312a88a164ff5
Running setup.py bdist_wheel for torchnet ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-7_odfrih/wheels/f0/bd/21/f8c3bb0ffcc30ae390971b7fa99242c2733b0b0c03b0d0f84c
Running setup.py bdist_wheel for apex ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-7_odfrih/wheels/91/1e/dc/41a5ba86547c578bd19be9cb9bdfd90b4e797acc58377e343b
Successfully built torchaudio torchtext torchnet apex
You are using pip version 18.1, however version 19.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Hi Jinserk,
I encountered the importing model error and tried to find what root cause is with some logs. But it has off the top of my head. Any suggestions?
...
try:
print('models: ' + " ".join(models))
print('input model: ' + model)
print('trying importlib.import_module(f"asr.models.{model}")')
m = importlib.import_module(f"asr.models.{model}")
m.train(argv)
except:
raise
...
python -V
Python 3.6.3 :: Anaconda custom (64-bit)
ls /home/kwon/EXP/ted_pytorch -alF
total 20940
drwxr-xr-x 5 kwon domain users 202 Jan 7 13:42 ./
drwxr-xr-x 3 kwon domain users 33 Jan 7 13:38 ../
drwxr-xr-x 10 kwon domain users 241 Jan 7 13:42 dev/
-rw-r--r-- 1 kwon domain users 2022 Jan 7 13:42 dev_convert.txt
-rw-r--r-- 1 kwon domain users 109533 Jan 7 13:42 dev.csv
drwxr-xr-x 13 kwon domain users 320 Jan 7 13:42 test/
-rw-r--r-- 1 kwon domain users 2093 Jan 7 13:42 test_convert.txt
-rw-r--r-- 1 kwon domain users 246191 Jan 7 13:42 test.csv
drwxr-xr-x 1497 kwon domain users 53248 Jan 7 13:40 train/
-rw-r--r-- 1 kwon domain users 372408 Jan 7 13:42 train_convert.txt
ls -alF
total 88
drwxr-xr-x 4 kwon domain users 317 Jan 7 15:20 ./
drwxr-xr-x 18 kwon domain users 4096 Jan 7 10:09 ../
drwxr-xr-x 7 kwon domain users 128 Jan 7 11:47 asr/
-rw-r--r-- 1 kwon domain users 455 Jan 7 10:10 batch_train.py
drwxr-xr-x 8 kwon domain users 211 Jan 7 10:10 .git/
-rw-r--r-- 1 kwon domain users 1339 Jan 7 10:10 .gitignore
-rw-r--r-- 1 kwon domain users 35147 Jan 7 10:10 LICENSE
-rw-r--r-- 1 kwon domain users 451 Jan 7 10:10 predict.py
-rw-r--r-- 1 kwon domain users 473 Jan 7 10:10 prepare.py
-rw-r--r-- 1 kwon domain users 5381 Jan 7 10:10 README.md
-rw-r--r-- 1 kwon domain users 547 Jan 7 10:10 requirements.txt
-rw-r--r-- 1 kwon domain users 448 Jan 7 10:10 test.py
-rwxr-xr-x 1 kwon domain users 670 Jan 7 10:10 train_deepspeech.sh*
-rwxr-xr-x 1 kwon domain users 737 Jan 7 10:10 train_las.sh*
-rw-r--r-- 1 kwon domain users 592 Jan 7 15:20 train.py
python train.py deepspeech_ctc --data-path /home/kwon/EXP/ted_pytorch
models: densenet deepspeech_ce deepspeech_var resnet_ce resnet_ctc resnet_split convnet ssvae capsule1 deepspeech_ctc capsule2 resnet_split_ce las densenet_ctc
input model: deepspeech_ctc
trying importlib.import_module(f"asr.models.{model}")
Segmentation fault (core dumped)
python train.py las --data-path /home/kwon/EXP/ted_pytorch
models: deepspeech_var las ssvae capsule2 convnet densenet densenet_ctc resnet_split resnet_split_ce deepspeech_ctc capsule1 resnet_ctc resnet_ce deepspeech_ce
input model: las
trying importlib.import_module(f"asr.models.{model}")
Segmentation fault (core dumped)
[kwon@ssi-dnn-slave-001 pytorch-asr]$ python train.py deepspeech_var --data-path /home/kwon/EXP/ted_pytorch
models: las convnet resnet_split densenet resnet_split_ce capsule2 ssvae deepspeech_ce deepspeech_ctc densenet_ctc resnet_ctc deepspeech_var resnet_ce capsule1
input model: deepspeech_var
trying importlib.import_module(f"asr.models.{model}")
Segmentation fault (core dumped)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.