Git Product home page Git Product logo

connear_cochlea's Introduction

CoNNear: A convolutional neural-network model of human cochlear mechanics and filter tuning for real-time applications.

The supporting paper can be found at https://arxiv.org/abs/2004.14832.

This work was funded with support from the EU Horizon 2020 programme under grant agreement No 678120 (RobSpear).

This repository contains two notebooks made for running and testing the CoNNear model. The full version connear_notebook.ipynb holds both the CoNNear model and the reference transmission line (TL) model (Verhulst et al.), the latter can be used as a validation tool. The connear_notebook_light.ipynb only runs the CoNNear model, hence decreases significantly the computational time of the full notebook. Both notebooks consist of different blocks corresponding to different sections of the paper. Each block can be adapted by the reader to run variations on the simulations that were described in the paper.

Besides the notebooks (both in .html and .ipynb format) the repository contains the trained model (in the connear folder), a helper_ops.py file, this README.md document, a license file, a speech fragment (dutch_sentence.wav), and the folder that contains the reference TL model (tlmodel).

How to test the CoNNear model

  1. If it's the first time you run the full notebook, you'll have to compile the cochlea_utils.c file that is used for solving the transmission line (TL) model of the cochlea. This requires some C++ compiler which should be installed beforehand. Go the tlmodel-folder from the terminal and run:
    gcc -shared -fpic -O3 -ffast-math -o tridiag.so cochlea_utils.c
    

If running on google colab: add the following as a code block and run it to compile cochlea_utils.c in the runtime machine.

!gcc -shared -fpic -O3 -ffast-math -o tridiag.so cochlea_utils.c

  1. Install numpy, scipy, keras and tensorflow in a code environment. We opted for Anaconda v2020.02 and used the following versions:

    • Python 3.6
    • Numpy v1.18.1
    • Scipy v1.4.1
    • Keras v2.3.1
    • Tensorflow v1.13.2
  2. Run the code blocks from the "Import required python packages and functions" section of the notebook. All other code blocks are independent and can be run in an order of your choice.

  3. Run the desired code blocks and/or adapt the various parameters to look at the performance for similar tasks. All of the code blocks do hold extra comments to clarify the steps or to define the choice of a parameter value.

CoNNear model specifications

The CoNNear model is an 8 layer, tanh, 64 filter length - deep convolutional neural network model, trained on a training set containing 2310 speech sentences of the TIMIT speech dataset. It predicts the basilar membrane displacements for 201 cochlear channels, resembling a frequency range from 100Hz (channel 0) to 12kHz (channel 200) based on the Greenwood map.

During the CoNNear simulations, 256 context samples are added at both sides to account for possible loss of information due to the slicing of full speech fragments.

The CoNNear model can take a stimulus with a variable sample lengths as an input, however, due to the convolutional character of CoNNear, this sample length has to be a multiple of 16.

System test

The system was tested on a MacBook Pro 2015 (macOS Sierra v10.12.6) with 3.1 GHz Intel Corei7, 16 GB RAM, and on a MacBook Air 2017 (macOS Catalina v10.15.3) with 1.8 GHz Dual-Core Intel Core i5, 8 GB RAM.

CoNNear model specifications

The installation time for the Anaconda environment and dependencies is approximately 20 min. The run-time of the first code-blocks in the connear_notebooks is very fast, only the DPOAE simulation code-block is slow due to the specific nature of how to present the stimuli and analyse the results for the DPOAE analysis. This code-block will automatically calculate the ETA (can be up to 25 mins).


For questions, please reach out to one of the corresponding authors

connear_cochlea's People

Contributors

deepakbaby avatar gitstaralfur avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.