zwcolin / eeg-transformer Goto Github PK

Hi
I just wonder Which dataset from BCI Competition IV are you using?

Hello 王哥
I wrote a simple ViT model to decode MI-EEG signals.
The overall model is much the same as original ViT, and the code is at here.
I used bci competition IV 2a dataset, which the input data shape is [1, 22, 1125].
I directly used a patch size = [22, 25], so the patch num is 1125/25 = 45.
With this patch setting, I soon face a problem as you mentioned in readme, that the model is overfitted on training set.
The results show that the generalize ability is worse that EEGNet.

Moreover, I applied a dropout layer in patch embedding, which performed to drop some patches.
With this dropout, the results get much better, but it took muuuuuch more epochs to converge(still worse than EEGNet).

Recently, I found other versions of transformer in MI-EEG, the ShallowMirrorTransformer and Conformer.
Sadly, both methods didn't performed as good as some CNN or LSTM-based ones.

I hope to know if there are some tricks when training ViT and see your experiment results.
Thanks a lot!