First of all, thank you for putting in so much work into this wonderful solution. It makes life easier being able to use Python to interface with Kaldi instead of having to string together commands all the time - this is nice!
Debugging checklist
[X --> v0.6.0] Have you updated to latest Kalpy version?
Describe the issue
During an attempt to make a simple function to align text to audio without any temp files (using GmmAligner.align_utterance), I encountered kaldi::KaldiFatalError during alignment, and couldn't trace the issue further.
For Reproducing your issue
All files involved in my testing are in the Drive folder here:
- https://drive.google.com/drive/folders/148-z_-i_7U09QY8HgpUoeAE5_84U0zex?usp=sharing
- The folder
kalpy_testing/gh_raw_files
is a copy paste from this repository's testing/data
folder.
- The Folder
kalpy_testing/acoustic_model
is the extracted acoustic model files for English (US) ARPA acoustic model v2.0.0
- The Folder
kalpy_testing/acoustic_model_mfa
is the extracted acoustic model files for English MFA acoustic model v2.2.1
- The
test.py
file is the python script I wrote, which you can run to reproduce the errors.
Extract the entire folder somewhere and make sure the file structure is the same as in the Drive folder.
Environment:
Run conda install -c conda-forge kalpy
if you don't already have. (I used v0.6.0, since it is the latest version on conda-forge).
Run python test.py
This should reproduce the outputs:
Running custom forced alignment with no temp files...
Initializing lexicon...
Initializing aligner...
Initializing feature generator...
Performing forced alignment...
Computing features...
Compiling training graph...
Performing alignment...
Traceback (most recent call last):
File "/app/test.py", line 242, in <module>
align_one(input_wav, input_align_text, feature_generator, training_graph_compiler, aligner)
File "/app/test.py", line 155, in align_one
alignment = aligner.align_utterance(
File "/env/lib/python3.10/site-packages/kalpy/gmm/align.py", line 72, in align_utterance
) = gmm_align_compiled(
RuntimeError: kaldi::KaldiFatalError
Additional context
I ran all of this in a GPU enabled Docker container on Ubuntu with Python v3.10.
I was ALMOST able to run the test in the repository test_align_sat_first_pass
using the data in the kalpy_testing/gh_raw_files/kaldi
folder corresponding to tests/data/kaldi
in the repo (this is in the python file in the Drive folder I provided). The alignment was generated, but LexiconCompiler.phones_to_pronunciations
failed to run, dumping the following error:
ERROR: SymbolStringToLabels: Symbol "" is not mapped to any integer label in symbol table <unspecified>
ERROR: Failed to compile string ` be be `, with token_type: symbol
I was NOT able to run the test in the repository test_align
, and got the same RuntimeError: kaldi::KaldiFatalError
as in my custom function.
I tried with two different dictionary+acoustic combinations:
- English MFA dictionary v2.2.1 + English MFA acoustic model v2.2.1
- English (US) ARPA dictionary v2.0.0 + English (US) ARPA acoustic model v2.0.0
Again, thank you for your work. I have used MFA for the past several months in projects, and am excited to get kalpy to work for even better interfacing with kaldi!