aks2203 / poisoning-benchmark Goto Github PK

A cuda problem occurred when running the script python test_model.py --model mobilenet_v2 --model_path "pretrained_models/MobileNetV2_Tinyimagenet_first.pth" although I have installed the compatible 10.1 version. Can you answer this question?

Hi, I like to thank you for providing a benchmark for the fair analysis. However, I would like to know more about black-box settings.
You mentioned in the paper that we craft poison using the known model and tested it on the two unknown models, averaging the results.

I do not clearly understand this setting. It would be nice if you clear it.

Is the dataset is known to attacker?

Many thanks

I believe you need to also pass 'args.dataset' to un_normalize_data, or else the target may be unnormalized from a different base distribution (the default CIFAR10 one) than what the poisons and all the rest of the data belongs to

Hi all, I have some doubts on the poison_test.py for all attack methods. I think HTBD should be evaluated on the images from source category added with a trigger. Could you please help check this?

It's a great project. It seems that only ResNet18_CIFAR10_adv.pth is provided in pretrained-models. Would you mind providing PGD adversarial training code in CLBD? Thank you in advance!

Hi,

I'm not sure if I can locate the poison examples that are used in all benchmarks (reported in the paper).

In the google drive folder that you have shared here, I can only find the poison examples generated by different attacks in CIFAR-10 benchmarks in the linear transfer learning scenario. Is it possible for you to share the poison examples for the CIFAR-10 benchmarks in the training-from-scratch scenario? What about TinyImageNet benchmarks as well (including both linear transfer learning and training-from-scratch scenarios?

I know you have put a lot of resources into that. I hope I do not ask here for too much. Your help would significantly help me with the experiments, as I'm including your benchmark in my current work.

Thanks a lot.

The files poison_test.py/benchmark_test.py contains evaluation code for only targeted attacks. For evaluation of untargeted attacks, I couldn't find any code in the repo or details in the paper. Is the evaluation methodology for target and untargeted attacks the same or do you use the evaluation methodology mentioned in CLBD paper to evaluate untargeted attacks?
Can you please clarify?

Thanks!

Hello, Thanks so much for your open source benchmark. I appreciate your work.
I want to clarify something. How do you calculate the success rate? How did you get those success rate percentages?

Another related question is, I understood that the training set contains 25 poisoned samples. How about the test set? How many poisoned examples are they tested during the evaluation of the attack?

Thanks a lot

I am trying to run poison_test file based on the provided poison example. However, I am getting error while creating a dataloader of the poison dataset.

Hi,

First of all, I want to appreciate your amazing work on benchmarking this field of research. That's indeed very valuable.

I have some doubts, which might seem naive, but I wanted to be 100% sure before proceeding with your benchmark.

In the transfer learning settings, does white-box mean that the victim uses the exact same model that is used by the attacker? Or the architecture is only the same (i.e., the parameters of the model is different).

A somehow related question, in learning_module.py you set the address model_paths['cifar10']['whitebox'] as ResNet18_CIFAR100_A.pth, but in the pertained models that you have shared with us, there is no model named that. In particular, we have a model named ResNet18_CIFAR100.pth. Is that the model you are referring in learning_module.py? I.e., is that the model that you have used to craft poison samples? If yes, is that the same model that is used by the victim in the white-box setting?

Thanks a lot in advance for answering my questions.