This project partially reflects the best practices in speaker verification technology up till 2020.

For more details, please refer to the link: The VoxSRC Workshop 2020.

• Training data are prepared following Kaldi voxceleb recipe: VoxCeleb2_dev and its four augmented versions are generated using RIRS_NOISES and MUSAN datasets.

• 80-dimensional (or 40-dimensional) FBANKs are extracted.

• Speaker augmentation is not applied.

• Online data augmentation is not applied.

• 1D-conv model: 512-d TDNN model following Kaldi voxceleb recipe.

• 2D-conv models: Res2Net model and DPN (Dual Path Networks) model.

• Margin based (AAM-softmax, CM-softmax) loss functions.

• Scheduled learning rate and margin adjustments.

• Large margin finetuning (LMFT).

• Mixed precision training and distributed training.

• Cosine similarity scoring.

• Adaptive symmetric normalization (asnorm) on cosine scores.

• Quality measure function (QMF) or score calibration is not applied.

• Data preparation takes a long time. It is recommended to run data preparation script on a server. With a D48s_v5 Virtual Machine on Azure (48 vCPUs, 192 GB memory), it takes about 10 hours to complete the whole process.

• For the smallest Res2Net model, res2net50_w8_s6_c16, it takes about 52 + 5 hours to train the model on a single NVIDIA Geforce RTX 4090 GPU.

• For the largest Res2Net model, res2net200_w8_s6_c16, it takes about 76 + 8 hours to train the model on eight NVIDIA Geforce RTX 3090 GPUs.

• Install TensorFlow 1.x and Kaldi toolkit manually

• or setup a docker container with Dockerfile (recommended):

  docker build -t xx205/tensorflow:23.03-tf1-py3-kaldi -f Dockerfile .

• or pull the container directly (recommended):

  docker pull xx205/tensorflow:23.03-tf1-py3-kaldi

• Run docker container:

  docker run --rm -it --gpus device=all -v `pwd`:`pwd` --ipc=host --ulimit memlock=-1 xx205/tensorflow:23.03-tf1-py3-kaldi

• Clone this repository:

  git clone https://github.com/xx205/voxsrc2020_speaker_verification

• Run data preparation script (note, 80-dimensional FBANKs are extracted by default. FBANK dimension can be set in global_config.sh):

  cd voxsrc2020_speaker_verification
  
  bash prepare_data.sh

• Go to working directory:

  cd tensorflow

• Train a Res2Net model with JTBD's VoxSRC2020 setup on VoxCeleb2_dev_aug training set:

  bash run_res2net_local_vox2_dev_aug.sh models.res2net_model res2net50_w24_s4_c32

• Export trained checkpoint to pb file for inference:

  bash export_inference_model.sh \
      models.res2net_model \
      res2net50_w24_s4_c32 \
      exp/voxceleb2_dev_aug/res2net50_w24_s4_c32_sc_cm_linear_frames200_scale32_margin0.2_8GPUs \
      3 \
      sc_cm_linear/kernel

• Evaluate trained Res2Net model performance on VoxCeleb1 Test/Extended/Hard trials:

  export saved_model=exp/voxceleb2_dev_aug/res2net50_w24_s4_c32_sc_cm_linear_frames200_scale32_margin0.2_8GPUs_122636.pb
  
  bash eval_inference_model.sh ${saved_model} 3
  
  for partition in T E H; do
      for approach in cosine snorm; do
          python3 eer_minDCF.py \
              --trial ../data/voxceleb1_trials/list_test_${partition}.txt \
              --score ${saved_model%.pb}_embeddings/voxceleb1/${approach}_${partition}.txt
      done
  done

• Finetune the Res2Net model with LMFT on VoxCeleb2_dev training set:

  bash run_res2net_finetune_local_vox2_dev.sh models.res2net_model res2net50_w24_s4_c32

• Export finetuned checkpoint to pb file for inference:

  bash export_inference_model.sh \
      models.res2net_model \
      res2net50_w24_s4_c32 \
      exp/voxceleb2_dev/res2net50_w24_s4_c32_sc_cm_linear_frames600_scale32_margin0.4_8GPUs \
      3 \
      sc_cm_linear/kernel

• Evaluate finetuned Res2Net model performance on VoxCeleb1 Test/Extended/Hard trials:

  export saved_model=exp/voxceleb2_dev/res2net50_w24_s4_c32_sc_cm_linear_frames600_scale32_margin0.4_8GPUs_127968.pb
  
  bash eval_inference_model.sh ${saved_model} 3
  
  for partition in T E H; do
      for approach in cosine snorm; do
          python3 eer_minDCF.py \
              --trial ../data/voxceleb1_trials/list_test_${partition}.txt \
              --score ${saved_model%.pb}_embeddings/voxceleb1/${approach}_${partition}.txt
      done
  done