davidstutz / confidence-calibrated-adversarial-training Goto Github PK

Hi,

In the README, under the feature section, there is a link to out-of-distribution examples and it links to the following address:
https://github.com/davidstutz/confidence-calibrated-adversarial-training/blob/master/common/eval/out_of_distribution_evaluation.py

But it seems that this file does not exist.
Does this part of experiment made into the paper?

Thanks.

Hi, I tried to replicate the experiments from training.

I got an error when I run the following command which is copied from the main readme.md

<Command>

python3 train.py config.svhn confidence_calibrated_adversarial_training_ce_f7p_i40_random_momentum_backtrack_power2_10 set_linf_white

<Error>
Traceback (most recent call last):
File "train.py", line 66, in
program.main()
File "train.py", line 55, in main
module = importlib.import_module(self.args.config_module)
File "/home/byunggill/anaconda3/envs/ccat/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named ` 'confidence_calibrated_adversarial_training_ce_f7p_i40_random_momentum_backtrack_power2_10'

I guess the module file 'confidence_calibrated_adversarial_training_ce_f7p_i40_random_momentum_backtrack_power2_10' is omitted from the current commit. Can you give me any hints for this?

<What-I-have-done list>

1. Set paths in common/path.py

2. Download data with examples/readme/download_data.py
   python download_dataset.py svhn

3. Download model with examples/readme/download_model.py and extract the models in BASE_EXPERIMENTS
   python download_model.py svhn ccat

4. Try to train a model.
   python3 train.py config.svhn confidence_calibrated_adversarial_training_ce_f7p_i40_random_momentum_backtrack_power2_10 set_linf_white

Thanks in advance!

Hi, David. it is an excellent work with a detailed guide on the codes.
I have met some problem with the svhn_at.zip. I try to test it on clean images and get only 0.4% accuracy. I get svhn data from its website in a .mat format, replace label 10 with 0, scale data into [0,1]. So I wonder is there any difference with your test data?

A good work, but the implementation seems too complex (I don't like object-oriented programming, it is NOT easy to read).

Hi~
The lr_scheduler:
def get_exponential_scheduler(optimizer, batches_per_epoch, gamma=0.97):
return torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=[lambda epoch: gamma ** math.floor(epoch/batches_per_epoch)])
may alawys return init_lr * 1 (0.97**0=1), since batches_per_epoch=len(self.trainloader) = 500 and epochs = 200.
Is there something wrong?