ResNet50 get 0% (not ~2% as reported in the paper) about natural-adv-examples HOT 7 CLOSED

torchvision's ResNet-50 gets 0%. A different ResNet-50 trained from scratch gets ~2%.

from natural-adv-examples.

torchvision's ResNet-50 gets 0%. A different ResNet-50 trained from scratch gets ~2%.

Can this be explained by the fact that ResNet-50 was used for selecting images?

Thereafter we delete the images that fixed ResNet-50 [24] classifiers correctly predict.

If it is the exact same model and all correctly classified examples have been removed, it is no surprise that it has 0% accuracy.

from natural-adv-examples.

Other models' results in Table 1 ImageNet-A are reproducible but not ResNet50. @hendrycks Can you check it?

from natural-adv-examples.

@hendrycks Hello. Thanks for your kind reply. Could you point me to the ResNet-50 trained from scratches? In your code, you still use torchvision's ResNet50.

from natural-adv-examples.

We did not upload a ResNet-50 trained from scratch. https://github.com/clovaai/CutMix-PyTorch has a ResNet-50 (though I think it was trained for 300 epochs).

from natural-adv-examples.

Thank you @hendrycks

from natural-adv-examples.

Did anyone try if it works with the pretrained ResNet-50 from https://github.com/clovaai/CutMix-PyTorch?

from natural-adv-examples.