madrylab / robust_representations Goto Github PK

Hi What version of python does this repo require? It doesn't load under python3.7

Thanks!

Hello,
Thank you for sharing this interesting work, I use a custom dataset of RGB images with size 224*224 saved in the 5 label names from 0 to 4 in train/images/0..4 folders and no validation folder and in the training phase. I use the predefined resnet50 architecture + the below parameters

train_kwargs = {
'out_dir': "./train_out",
'adv_train': 1,
'constraint': '2',
'eps': 0.05,
'attack_lr': 1.5,
'attack_steps': 10,
'epochs': 10,
'log_iters':5,
'lr':0.001,
'momentum':0.9,
'weight_decay':1e-3,
'use_best': True,
'random_restarts': 0,
'save_ckpt_iters':-1
}
train.train_model(train_args, model, (train_loader, val_loader), store=out_store)

Then it generates some .pt model files that I used the best version of it for the test phase

# Load model
model_kwargs = {
    'arch': 'resnet50',
    'dataset': ds,
    'resume_path' : './train_out/4bce8667-bc86-4776-aa1d-1489eacda01f/checkpoint.pt.best'
}

model, _ = model_utils.make_and_restore_model(**model_kwargs)
model.eval()
pass

BATCH_SIZE = 32
NUM_WORKERS = 2
_, test_loader = ds.make_loaders(workers=NUM_WORKERS,
                                      batch_size=BATCH_SIZE)

# PGD Parameters
kwargs = {
    #'criterion': ch.nn.CrossEntropyLoss(),
    'custom_loss': activation_loss,
    'constraint':'2',
    'eps': 50,
    'step_size': 0.5,
    'iterations': 200,
    'do_tqdm': True,
    'targeted': True,
}

Then implemented the remained cells of maximizing_inputs notebook but the output representation is completely noisy and the model does not sort the 5 top images based on their labels. Could you please help me with this issue?
Any comments would be appreciated

Hi, your paper is really interesting! I am reproducing your experiments with both robustness package and my own codes. Can you provide more details about how to adversarially training the robust model you provided? Specifically, I want to know eps, steps, step size, attack-lr and adversarial algorithm you used to train the model so I can compare the inversion performance between my own codes and your robustness package. Thx a lot in advance.

Edit user_constants.py to point to PyTorch-formatted versions of the CIFAR and ImageNet datasets.
What is the right way to point to the dataset(s)?

Good day! You have a wonderful and very useful project, but I would like to get a more specific description of the learning process of the neural network robust. In the context of Your source code. Can you write instructions on how to train an arbitrary neural network on an arbitrary dataset? For example VGG16 or VGG19 on Places365?

Hi,

Thanks for this awesome work. I just notice that in the inversion_loss function for representation inversion,

def inversion_loss(model, inp, targ):
    _, rep = model(inp, with_latent=True, fake_relu=True)
    loss = ch.div(ch.norm(rep - targ, dim=1), ch.norm(targ, dim=1))
    return loss, None

the fake_relu is set to True. According to the source code of robustness library, this would make the gradients pass straight through during the backprop.

Then my question is as follows:

1. My best guess for using this fake_relu is that it can enable a better inversion process. But I have no idea how much it matters and I do not see discussions/instructions on this detail in your paper. Is it a common practice when doing representation inversion?

2. I also do not see why this fake_relu is only applied to the last "layer" in the ResNet architecture.

    def forward(self, x, with_latent=False, fake_relu=False, no_relu=False):
        assert (not no_relu),  \
            "no_relu not yet supported for this architecture"
        out = F.relu(self.bn1(self.conv1(x)))
        out = self.layer1(out)
        out = self.layer2(out)
        out = self.layer3(out)
        out = self.layer4(out, fake_relu=fake_relu)

Any clarifications or explanations are appreciated!
Thanks

Hi, thanks for the great work!

I wanted to test the model on some of my custom images, and tried to understand what preprocessing steps were applied. By examining outputs of the dataset function, I found that all of them have min=0 and max=1. Also I found TEST_TRANSFORMS_224 list and that ImageNet mean and std are present in the dataset function, but I wonder if they are used. Does the following way to process input image reflect all the transformations I need for ImageNet model?

from PIL import Image
import torch as ch
from torchvision import transforms

TEST_TRANSFORMS_224 = transforms.Compose([
        transforms.ToPILImage(),
        transforms.Resize((224,224)),
        transforms.ToTensor(), # This converts the image to range [0,1]
        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
    ])

def normalize(vector):
    min_v = ch.min(vector)
    range_v = ch.max(vector) - min_v
    if range_v > 0:
        normalised = (vector - min_v) / range_v
    else:
        normalised = ch.zeros(vector.size())
    return normalised


def get_patch_embedding(np_img):
   im_1 =  TEST_TRANSFORMS_224(ch.from_numpy(np_img)).unsqueeze(0)
   im_1 = normalize(im_1.cuda())
   (_, rep_1), _  = model(im_1, with_latent=True)
   return rep_1.detach().cpu().numpy()

im_1 = np.array(Image.open('9_256_384.png').convert('RGB'))

emb = get_patch_embedding(im_1)

Another question is how correct would be to compare embeddings emb1, emb2 with l2, cosine, etc?

Hi,

As mentioned in table 3, it says to use 7 step PGD with eps 3.5 and step size=0.1 for adv l2 training.

But with 7 steps and step size=0.1, the adv_image will be 0.7 l2 distance apart from the original image.

Can you also comment on the image pixel range (ie is it from [-1,1] or [0,1] and the config for the optimizer?

Hello !
Thanks for the great work !
I have been running into a Not JSON Error regarding the "maximizing_inputs.ipynb" file.
Could you please let me know how to fix the error ?
Thanks,

Hello,
Thank you for this great work. I tried the maximizing_input note book with CIFAR and worked fine however it gives this error on make_loaders when I try restrictedimagenet and imagenet with tiny-imagenet-200 dataset :"Test data must be stored in dataset/test or robust_representations/tiny-imagenet/tiny-imagenet-200/test" the test directory is exactly the same as robust_representations/tiny-imagenet/tiny-imagenet-200/test !!

In addition I plan to use a public dataset that is not in the defaults of robustness through the below way and the dataset 's path is to a directory in my google drive but it does not understand it. how should I fix it?

from robustness.datasets import DataSet
from robustness import cifar_models

class MyDataset(DataSet):
def init(self, data_path='/content/drive/My Drive/MyDataset/train_images', **kwargs):
ds_kwargs = {
'num_classes': 5,
'mean': ch.tensor([0., 0., 0.]),
'std': ch.tensor([1., 1., 1.]),
'custom_class': datasets.MyDataset,
'label_mapping': None,
'transform_train': ..., # TODO
'transform_test': ... # TODO
}
ds_kwargs = self.override_args(ds_kwargs, kwargs)
super(MyDataset, self).init('mydataset', data_path, **ds_kwargs)

def get_model(self, arch, pretrained):
    if pretrained:
        raise ValueError('CIFAR does not support pytorch_pretrained=True')
    return cifar_models.__dict__[arch](num_classes=self.num_classes)

from robustness.datasets import MyDataset>>>> does not recognize MyDataset

Thank you in advance

Hey,

Thanks for providing this robust model. I was using it for my experiments when I noticed that the top-1 accuracy on the validation set is around 3% (1555/50000 to be precise). But in the paper, it is mentioned that the accuracy is around 45%

It would be great if you could help me verify it.

P.S. - I am using the ImageNet model provided in this repository.

Thanks,
Naman