Now, we are preparing the paper "Speech Enhancement Using a Two Step Network", so this project will change frequently (If you want to use it, please change the path in prj_dir/lib/config.py)

The speech enhancement (SE) removes the noise signal from the noisy speech signal.

Now, the SE in this toolkit is based on the deep neural network (DNN). And the proposed model will be uploaded.

We hope that this toolkit will contribute as the baselines for SE research area.

This toolkit provides as follows:

• The data generator script for building the noisy training and test dataset from the speech and noise dataset. (MATLAB)

• The training and test script. (python3)

• Python3
• Tensorflow 1.7
• TensorboardX
• librosa
• Matlab 2017b

1. Install aformentioned prerequistes.

2. Open the MATLAB and add the directories ./SE and ./Datamake including their sub-directories.

3. Install matlab.engine

cd "matlabroot/extern/engines/python"
python3 setup.py install

1. Prepare the speech and noise data. In general, the TIMIT corpus is used for the speech data. And, the noise data can be found in Hu's corpus, USTC's corpus and NOISEX-92.

2. project_directory(prj_dir)/Datamake/make_train_noisy.m will make the training set from your data. This code sequentially load the clean speech and synthesize the noisy speech with randomly selected SNR. Here, the type of noise is randomly selected from your training noise dataset. To reduce the file number, this code concatenate all generated noisy speech. Therefore, if your RAM is not enough, you should modify the code. All generated data will be written in '.raw' format with 'int16' datatype.

3. project_directory(prj_dir)/Datamake/make_test_noisy.m will make the test set from your data. This code sequentially load the clean speech and synthesize the noisy speech with desired SNR. Here, the code use all types of noises in the test noise dataset when synthesize the noisy speech. All generated data will be written in '.raw' format with 'int16' datatype.

Before run the code, move your training speech and noise dataset by referring the below code.

% prj_dir/Datamake/make_train_noisy.m
timit_list = dirPlus('./speech/TIMIT/TRAIN', 'FileFilter', '\.(wav|WAV)$');

hu_list = dirPlus('./noise/Nonspeech', 'FileFilter', '\.(wav|WAV)$');
ad_list = dirPlus('./noise/noise-15', 'FileFilter', '\.(wav|WAV)$');

• You can set the SNRs for noisy speech by adjusting snr_list.
• You can make more fluent data by adjusiting aug.

The generated dataset will be saved in prj_dir/SE/data/train/noisy and prj_dir/SE/data/train/clean.

Before run the code, move your test speech and noise dataset by referring the below code.

% prj_dir/Datamake/make_test_noisy.m
timit_list = dirPlus('./speech/timit_coretest', 'FileFilter', '\.(wav|WAV)$');
noise_list = dirPlus('./noise/NOISEX-92_16000');

• You can set the SNRs for noisy speech by adjusting snr_list.

The generated dataset will be saved in prj_dir/SE/data/test/noisy and prj_dir/SE/data/test/clean.

To run the code, the validation set is needed. I used to randomly select about 50 noisy utterances with corresponding clean utterances from test set then, move these to prj_dir/SE/data/valid/noisy and prj_dir/SE/data/valid/clean.

This code conduct Z-score normalization to the input features, so that some normalization factor from training dataset is needed.

To get the normalization factor, just run the prj_dir/SE/get_norm.py

• You can use the multiple core by adjusting distribution_num.

The generated normalization factor will be saved in prj_dir/SE/data/train/norm.

Just run the prj_dir/SE/main.py

You can check the training model in prj_dir/SE/lib/trnmodel.py

You can check the training configuration in prj_dir/SE/lib/config.py

While training, you can use the tensorboard for monitoring the training procedure.

tensorboard --logdir='prj_dir/SE/logs_dir/your log directory'

This toolkit supports followings:

• PESQ, STOI, LSD, SSNR (Objective measure).

• Clean, noisy, and enhanced spectrogram.

• Clean, noisy and enhanced wavs.

• Configuration

[1] Xu, Yong, et al. "A regression approach to speech enhancement based on deep neural networks." IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP) 23.1 (2015): 7-19.

[2] Brookes, Mike. (2011). Voicebox: Speech Processing Toolbox for Matlab.

[3] Jacob, SoundZone_Tools, (2017), GitHub repository, https://github.com/JacobD10/SoundZone_Tools

[4] Loizou, P.C.: "Speech enhancement: theory and practice", (CRC press, 2013), pp. 83−84