This is the official implementation of the work RespireNet.

* Python3
* Pytorch (torch, torchvision and other dependencies for Pytorch)
* Numpy
* Librosa (0.8.0)
* nlpaug (0.0.14)
* OpenCV (4.2.0)
* Pandas (0.22.0)
* scikit-learn (0.23.1)
* tqdm (4.48.0)
* cudnn (CUDA for training on GPU)

These are all easily installable via, e.g., pip install numpy. Any reasonably recent version of these packages shold work. It is recommended to use a python virtual environment to setup the dependencies and the code.

• The dataset used is the ICBHI Respiratory Challenge 2017 dataset.
• To find out more details about the dataset please visit the official ICBHI Challenge webpage.
• Download from above and place the dataset in the data folder.

• We follow both the official 60-40 train-test split as well as the 80-20 split.
• For training we employ a 2-stage training protocol.
• Stage 1: The model is trained end-to-end on train data from all the 4 devices.
• Stage 2: The model is fine-tuned (with a lower learning rate 1e-4) on only the data from a single device. We do this separately for each device (device specific fine-tuning).

Stage 1

python train.py --data_dir ../data/icbhi_dataset/audio_text_data/ --folds_file ../data/patient_list_foldwise.txt --model_path models_out --lr 1e-3 --batch_size 64 --num_worker 4 --start_epochs 0 --epochs 200 --test_fold 4

Stage 2 (Device specific fine-tuning)

python train.py --data_dir ../data/icbhi_dataset/audio_text_data/ --folds_file ../data/patient_list_foldwise.txt --model_path models_out --lr 1e-4 --batch_size 64 --num_worker 4 --start_epochs 0 --epochs 50 --test_fold 4 --checkpoint models/ckpt_best.pkl --stetho_id 0

replace the stetho_id as 0 or 1 or 2 or 3 for devices 0-3

Please go through our paper for more details.

Evaluation script

python eval.py --data_dir ../data/icbhi_dataset/audio_text_data/ --folds_file ../data/patient_list_foldwise.txt --batch_size 64 --num_worker 4 --test_fold 4 --checkpoint models/ckpt_best.pkl

Performance comparison of the proposed model with the state-of-the-art systems following random splits. We see significant improvements from our proposed techniques: concatenation-based augmentation (CBA), blank region clipping (BRC) and device specific fine-tuning (FT).

We also tried experiments with Mixup augmentations, using pre-trained VGGish features and a novel Hybrid CNN + Transformer architecture, however they did not prove to be very useful. 
However transformers with appropriate pretraining have found to be useful in many applications (especially NLP tasks) and may prove to be useful in the future.

VGGish link

• train.py: Main trainer code.
• image_dataloader.py: Dataloader module.
• utils.py: Contains a lot of utility functions for audio file loading, feature extraction, augmentations, etc.
• eval.py: Evaluation source files for trained model.
• scripts: Directory which contains the runner scripts.
• nets: Contains the different network modules.
• data: Training-Testing split and should contain the ICBHI data.
• models: Contains the trained checkpoint for our proposed framework.

@misc{gairola2020respirenet,
      title={RespireNet: A Deep Neural Network for Accurately Detecting Abnormal Lung Sounds in Limited Data Setting}, 
      author={Siddhartha Gairola and Francis Tom and Nipun Kwatra and Mohit Jain},
      year={2020},
      eprint={2011.00196},
      archivePrefix={arXiv},
      primaryClass={cs.SD}
}