Tacotron2 + LPCNET for complete End-to-End TTS System

Thanks to https://github.com/MlWoo/ for most of changes ( Check this issue Also MlWoo/LPCNet#4)

librosa , tqdm , matplotlib, lws , unidecode , inflect, falcon, scipy , numpy, keras

sudo apt-get install python-pyaudio sudo apt-get install portaudio19-dev

Checkout the LPCTron , and run tts.sh it converts text_file ( Tacotron2/text.txt ) to test.wav

• Checkout Tacatron2 (https://github.com/alokprasad/LPCTron/blob/master/Tacotron-2/Tacotron2-lpcnet_changes.diff)

python3 preprocess.py --base_dir /media/alok/ws/sandbox/lpc_tacatron2/dataset --dataset LJSpeech-1.1

this will generate /dataset/training_data folder

├── training_data (1)
│   ├── audio (*)
│   ├── linear
│   └── mels
    └── train.txt

*all three folder contain npy array, train.txt contains text for each audio.

• this is replaced by script for training to be used by LPCNET

Since Tacatron2 is trained with LPSpeech its good idea to train with Same data. LPCNET uses a single Merged PCM for training. So you need to remove wav headers from each file and merge them for training.

===============================================================================

sudo apt-get install python3-tk

pip3 install lws --user

python -m pip uninstall pip && sudo apt install python3-pip --reinstall

python3 train.py --input_dir ../dataset/training_data --tacotron_input ../dataset/training_data/train.txt --model='Tacotron'

python3 synthesize.py --model='Tacotron' --mode='eval'

sudo apt-get install python3-tk

pip3 install lws --user python -m pip uninstall pip && sudo apt install python3-pip --reinstall

Download https://jmvalin.ca/misc_stuff/lpcnet_models/lpcnet15_384_10_G16_100.h5

apt-get install -y google-perftools LD_PRELOAD = /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4

python3 train.py --input_dir ../dataset/training_data --tacotron_input ../dataset/training_data/train.txt --model='Tacotron'

python3 synthesize.py --model='Tacotron' --mode='eval'

./test_lpcnet f32_for_lpcnet.f32 test.s16

ffmpeg -f s16le -ar 16k -ac 1 -i test.s16 test-out.wav