Replicating results from paper with dropout about swa_gaussian HOT 4 CLOSED

Hi,

What command did you attempt to run and with which model?

I believe that we simply turned the dropout layers back on at test time for test MC Dropout.

Please see https://github.com/wjmaddox/swa_gaussian/blob/master/experiments/uncertainty/README.md for an example command.

from swa_gaussian.

Hi Wesley,
Thanks for responding. That is the instruction I was following. My question was more for training the dropout models. I tried training vgg dropout model with the following command

python experiments/train/run_swag.py --dir=./results/vgg_dropout_cifar10 --dataset=CIFAR10 --data_path=./datasets/ --model=VGG16Dropout --epochs=300 --lr_init=0.05

This model never converged, the losses started increasing, and the accuracy started dropping after about 50 epochs or so.

I think this is because I am not training dropout models properly, and wanted to check with you regarding your commandline parameters.

from swa_gaussian.

Looks like you forgot to set the weight decay properly wd=5e-4 It doesn't seem to be diverging for me.

from swa_gaussian.

Thanks a lot for the quick response. The network is converging now.

from swa_gaussian.