- Explainable multi-modality neural network using
- PID (Partial Information Decomposition in information theory)
- Prepare dataset: see Multibench for more detail
- Training model and extract features:
!python3 examples/affect/train_affect_mult.py --data_path /content/mosi_data.pkl --save_path /content/ --train_modal 0 1 --num_epochs 20 --extract_features
- Compute PID with use output from step 2
from utils.compute_pid import compute_pid
dataset_name = 'mosi'
measure = compute_pid('/content/vision_audio', dataset_name=dataset_name)
print(measure)
See Example: examples/Training - Extract feaure - Compute PID.ipynb