yinruiqing / change_detection Goto Github PK

View Code? Open in Web Editor NEW

61.0 8.0 15.0 374 KB

Code for Speaker Change Detection in Broadcast TV using Bidirectional Long Short-Term Memory Networks

License: MIT License

change_detection's Introduction

⚠️ This repository is no longer maintained: it has been integrated into pyannote.audio.

Speaker Change Detection using Bi-LSTM

Code for Speaker Change Detection in Broadcast TV using Bidirectional Long Short-Term Memory Networks

Citation

@inproceedings{Yin2017,
  Author = {Ruiqing Yin and Herv\'e Bredin and Claude Barras},
  Title = {{Speaker Change Detection in Broadcast TV using Bidirectional Long Short-Term Memory Networks}},
  Booktitle = {{18th Annual Conference of the International Speech Communication Association, Interspeech 2017}},
  Year = {2017},
  Month = {August},
  Address = {Stockholm, Sweden},
  Url = {https://github.com/yinruiqing/change_detection}
}

Installation

Foreword: The code is based on pyannote. You can also find a similar Readme file for TristouNet .

$ conda create --name change-detection python=2.7 anaconda
$ source activate change-detection
$ conda install gcc
$ conda install -c yaafe yaafe=0.65
$ pip install "pyannote.audio==0.2.1"
$ pip install pyannote.db.etape

What did I just install?

keras (and its theano backend) is used for all things deep. If you want to use GPU, please downgrade Keras version to 1.2.0
yaafe is used for MFCC feature extraction in pyannote.audio.
pyannote.audio is the core for this project. (Model architecture, optimizer, sequence generator)
pyannote.db.etape is the ETAPE plugin for pyannote.database, a common API for multimedia databases and experimental protocols (e.g. train/dev/test sets definition).

Then, edit ~/.keras/keras.json to configure keras with theano backend.

$ cat ~/.keras/keras.json
{
    "image_dim_ordering": "th",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "theano"
}

About the ETAPE database

To reproduce the experiment, you obviously need to have access to the ETAPE corpus.
It can be obtained from ELRA catalogue.

However, if you own another corpus with "who speaks when" annotations, you can fork pyannote.db.etape and adapt the code to your own database.

Training and evaluation

You can use:

$ pyannote-change-detection -h

to find the usage information.

change detection

Usage:
    pyannote-change-detection train [--database=<db.yml> --subset=<subset>] <experiment_dir> <database.task.protocol>
    pyannote-change-detection evaluate [--database=<db.yml> --subset=<subset> --epoch=<epoch> --min_duration=<min_duration>] <train_dir> <database.task.protocol>
    pyannote-change-detection apply  [--database=<db.yml> --subset=<subset> --threshold=<threshold> --epoch=<epoch>  --min_duration=<min_duration>] <train_dir> <database.task.protocol>
    pyannote-change-detection -h | --help
    pyannote-change-detection --version
...

Example of config file can be found in change_detection/config/. Before doing the training and evaluation, you can clone this project to local directory.

git clone https://github.com/yinruiqing/change_detection.git

Training

$ pyannote-change-detection train --database change_detection/config/db.yml --subset train change_detection/config Etape.SpeakerDiarization.TV

This is the expected output:

Epoch 1/100
62464/62464 [==============================] - 171s - loss: 0.1543 - acc: 0.9669   
Epoch 2/100
62464/62464 [==============================] - 117s - loss: 0.1375 - acc: 0.9692     
Epoch 3/100
62464/62464 [==============================] - 115s - loss: 0.1376 - acc: 0.9691     
...
Epoch 50/100
62464/62464 [==============================] - 112s - loss: 0.0903 - acc: 0.9724  
...
Epoch 98/100
62464/62464 [==============================] - 115s - loss: 0.0837 - acc: 0.9732     
Epoch 99/100
62464/62464 [==============================] - 112s - loss: 0.0839 - acc: 0.9732     
Epoch 100/100
62464/62464 [==============================] - 112s - loss: 0.0840 - acc: 0.9731

Evaluation

$ pyannote-change-detection evaluate --database change_detection/config/db.yml --subset development change_detection/config/train/Etape.SpeakerDiarization.TV Etape.SpeakerDiarization.TV

This is the expected output:

0         95.720% 36.603%
0.0526316 95.526% 49.018%
0.105263  95.213% 57.660%
...
0.526316  92.396% 83.756%
...
0.894737  88.155% 91.139%
0.947368  87.468% 92.046%
1         86.929% 92.672%

The first column is threshold, the second column is purity and the third column is coverage.

change_detection's People

Contributors

Stargazers

Watchers

Forkers

hbredin codeashu entn-at 1215thebqtic ishine aitorbajo sheenchi twistedmove wuqiangch mrcoderk njho thumimku sohailkhanmarwat xggyh

change_detection's Issues

Final version of the paper

Could you please upload the final version of the paper into the repository?
And link it to it directly instead of linking to your old cloud version?

subpackage version conflict with python2.7

Following the instructions at "Installation" section of README,
various sub-packages of pyannote would be installed using latest version.
Some would conflict with python2.7

downgrade some packages can make pyannote-change-detection -h works

Note: I'm doing it at 2018/2/5, Beijing

pip install --force-reinstall pyannote.database==0.11.2
pip install --force-reinstall pyannote.generators==0.13

about install

I got an error when I do pip install "pyannote.audio==0.2.1"

Running setup.py bdist_wheel for pysndfile ... error
Complete output from command /home/user1/anaconda3/envs/py36/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-fjxlgynr/pysndfile/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/tmpv8qoj5jqpip-wheel- --python-tag cp36:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/pysndfile
copying pysndfile/init.py -> build/lib.linux-x86_64-3.6/pysndfile
copying pysndfile/Faiff.py -> build/lib.linux-x86_64-3.6/pysndfile
copying pysndfile/sndio.py -> build/lib.linux-x86_64-3.6/pysndfile
running build_ext
check for clang compiler ... no
building '_pysndfile' extension
creating build/temp.linux-x86_64-3.6
gcc -pthread -B /home/user1/anaconda3/envs/py36/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/user1/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/core/include -I/home/user1/anaconda3/envs/py36/include/python3.6m -c _pysndfile.cpp -o build/temp.linux-x86_64-3.6/_pysndfile.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
In file included from /home/user1/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1816:0,
from /home/user1/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:18,
from /home/user1/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from _pysndfile.cpp:469:
/home/user1/anaconda3/envs/py36/lib/python3.6/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^
g++ -pthread -shared -B /home/user1/anaconda3/envs/py36/compiler_compat -L/home/user1/anaconda3/envs/py36/lib -Wl,-rpath=/home/user1/anaconda3/envs/py36/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/_pysndfile.o -lsndfile -o build/lib.linux-x86_64-3.6/pysndfile/_pysndfile.cpython-36m-x86_64-linux-gnu.so
/home/user1/anaconda3/envs/py36/compiler_compat/ld: cannot find -lm
/home/user1/anaconda3/envs/py36/compiler_compat/ld: cannot find -lpthread
/home/user1/anaconda3/envs/py36/compiler_compat/ld: cannot find -lc
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1

Failed building wheel for pysndfile

citation is incorrect

Title != Conference

@inproceedings{Yin2017,
  Author = {Ruiqing Yin and Herv\'e Bredin and Claude Barras},
  Title = {{Interspeech 2017, 18th Annual Conference of the International Speech Communication Association}},
  Year = {2017},
  Month = {August},
  Address = {Stockholm, Sweden},
  url = {https://github.com/yinruiqing/change_detection},
}

cite

hi, how can I cite this work and the paper? Is it published somewhere?