q-hu / connear_cochlea Goto Github PK

The supporting paper can be found at https://arxiv.org/abs/2004.14832.

This work was funded with support from the EU Horizon 2020 programme under grant agreement No 678120 (RobSpear).

This repository contains two notebooks made for running and testing the CoNNear model. The full version connear_notebook.ipynb holds both the CoNNear model and the reference transmission line (TL) model (Verhulst et al.), the latter can be used as a validation tool. The connear_notebook_light.ipynb only runs the CoNNear model, hence decreases significantly the computational time of the full notebook. Both notebooks consist of different blocks corresponding to different sections of the paper. Each block can be adapted by the reader to run variations on the simulations that were described in the paper.

Besides the notebooks (both in .html and .ipynb format) the repository contains the trained model (in the connear folder), a helper_ops.py file, this README.md document, a license file, a speech fragment (dutch_sentence.wav), and the folder that contains the reference TL model (tlmodel).

1. If it's the first time you run the full notebook, you'll have to compile the cochlea_utils.c file that is used for solving the transmission line (TL) model of the cochlea. This requires some C++ compiler which should be installed beforehand. Go the tlmodel-folder from the terminal and run:

   gcc -shared -fpic -O3 -ffast-math -o tridiag.so cochlea_utils.c
   

If running on google colab: add the following as a code block and run it to compile cochlea_utils.c in the runtime machine.

!gcc -shared -fpic -O3 -ffast-math -o tridiag.so cochlea_utils.c

2. Install numpy, scipy, keras and tensorflow in a code environment. We opted for Anaconda v2020.02 and used the following versions:

   • Python 3.6
   • Numpy v1.18.1
   • Scipy v1.4.1
   • Keras v2.3.1
   • Tensorflow v1.13.2

3. Run the code blocks from the "Import required python packages and functions" section of the notebook. All other code blocks are independent and can be run in an order of your choice.

4. Run the desired code blocks and/or adapt the various parameters to look at the performance for similar tasks. All of the code blocks do hold extra comments to clarify the steps or to define the choice of a parameter value.

The CoNNear model is an 8 layer, tanh, 64 filter length - deep convolutional neural network model, trained on a training set containing 2310 speech sentences of the TIMIT speech dataset. It predicts the basilar membrane displacements for 201 cochlear channels, resembling a frequency range from 100Hz (channel 0) to 12kHz (channel 200) based on the Greenwood map.

During the CoNNear simulations, 256 context samples are added at both sides to account for possible loss of information due to the slicing of full speech fragments.

The CoNNear model can take a stimulus with a variable sample lengths as an input, however, due to the convolutional character of CoNNear, this sample length has to be a multiple of 16.

The system was tested on a MacBook Pro 2015 (macOS Sierra v10.12.6) with 3.1 GHz Intel Corei7, 16 GB RAM, and on a MacBook Air 2017 (macOS Catalina v10.15.3) with 1.8 GHz Dual-Core Intel Core i5, 8 GB RAM.

The installation time for the Anaconda environment and dependencies is approximately 20 min. The run-time of the first code-blocks in the connear_notebooks is very fast, only the DPOAE simulation code-block is slow due to the specific nature of how to present the stimuli and analyse the results for the DPOAE analysis. This code-block will automatically calculate the ETA (can be up to 25 mins).

For questions, please reach out to one of the corresponding authors

• Deepak Baby: [email protected]
• Arthur Van Den Broucke: [email protected]
• Sarah Verhulst: [email protected]