[ICASSP19] An Interaction-aware Attention Network for Speech Emotion Recognition in Spoken Dialogs

Python 96.05% Dockerfile 2.99% Shell 0.96%

speech-emotion-recognition icassp-2019 icassp emotion-recognition tensorflow

interaction-aware-attention-network's Introduction

Interaction-aware Attention Network

A tensorflow implementation of Interaction-aware Attention Network in An Interaction-aware Attention Network for Speech Emotion Recognition in Spoken Dialogs.

Model Overview

Data:

Data disciptions of IEMOCAP please refer to here.

Requirements

Some required libraries:

python                   >=3.6   
tensorflow-gpu           1.11.0
joblib   		 0.13.0
pandas                   0.22.0
scikit-learn             0.19.1
numpy			 1.15.3

Code:

codes	descriptions
data.py	Includes batch generator & data generator.
model.py	main codes.
hyparams.py	hyperparameters
script_train.py	testing scripts
script_test.py	training scripts

To evaluate under realistic scenarios of our model, we adopt leave-one-session-out cross validation.

Run:

For feature pooling to reduce computational cost:

python3 pool_feats.py --input_file INPUT_FILE --output_file OUTPUT_FILE --feat_dim FEAT_DIM --step STEP --max_size MAX_SIZE

For training:

python3 script_train.py --seq_dim SEQ_DIM \
                        --atten_size ATTEN_SIZE \
                        --batch_size BATCH_SIZE \
                        --model_dir MODEL_DIR \
                        --record_file outputs/RECORD_FILE.json \
                        --feat_dir data/XXX.pkl

For testing

python3 script_test.py  --result_file outputs/RECORD_FILE.json --feat_dir data/XXX.pkl

I include whole process and hyperparameters in a script:

sh run.sh

Citation

@inproceedings{yeh2019interaction,
  title={An interaction-aware attention network for speech emotion recognition in spoken dialogs},
  author={Yeh, Sung-Lin and Lin, Yun-Shao and Lee, Chi-Chun},
  booktitle={ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={6685--6689},
  year={2019},
  organization={IEEE}
}

interaction-aware-attention-network's People

Contributors

Stargazers

Watchers

Forkers

qiongxiao hadryan qianfan1996 ammade47 jocelyn1981 czfuchason nilofarmoradifarisar xiaoheng-zhang99 deyh2020

interaction-aware-attention-network's Issues

script arguments outdated

When running
python3 script_train.py -lr 0.0001 -batch_size 64 -keep_proba 0.9 -seq_dim 512 -save_path ./model/iaan/ > out.txt

I get
script_train.py: error: unrecognized arguments: -save_path ./model/iaan/

Is there a new version of the script that is not commited?

Thanks

Performance comparison with original paper

Thanks very much for the implementation @30stomercury ! It is great!

Am I right that currently there is no evaluation on a validation set during training, thus why no early stopping is used for training? There is a fixed training steps (e.g. 2000 steps).

Have you managed to replicate the results reported in the original paper? I have only been able to get 55.4 in UAR, which is far lower than what they have reported:

[[115  26  26   3]
 [ 74 149 172  47]
 [ 27  31 255  71]
 [ 15  36  49 145]]
Ave test acc: 0.535, Ave test uar: 0.567
ave uar over all data: 0.5547344560992651

Thanks again!

In the model.py file, I find a problem.It is the code:
"paddings = (mask_att)*(-1e8)
vu = tf.where(tf.equal(mask_att, 0), paddings, vu)"
I think the "paddings" is no useful,in other words,it doesn't work.Because "mask_att" variable is a matrix about "0" and "1", "paddings" variable is 0 in where "mask_att" is 0, -1e8 where "mask_att" is 1.Then variable "vu" equal to
"vu = tf.where(tf.equal(mask_att, 0), mask_att, vu)"

I've seen that you name data.py to have a function to generate those, but I don't know how to use it.

Is there a missing script? Or am I missing something?

Thanks a lot

30stomercury / interaction-aware-attention-network Goto Github PK