zoky-2020 / sga Goto Github PK

thanks for sharing your great work.

I am confused about this step
"we select the most matching caption pairs from the dataset of each image v to form an augmented caption set t ={t1, t2, ..., tM }"

because the dataset only gives single image-text pair, how could you find multiple matched texts for an single image

老师，读过您的论文之后，我有一个困惑，我在我爱计算机视觉这个公众号上，看到对您的方法描述是“在迭代优化对抗图像和对抗文本的过程，该策略逐步拉远图像和文本在特征空间中的距离，从而破坏跨模态交互，达到攻击效果。”但是这个描述，我没有在您的论文中找到，可以问下您在论文的哪个部分有提到这样的描述嘛？

Hi,
Thank you for great work.

Could please provide script for ALBEF--to--CLIP_CNN attack.

To check transferability from ALBEF to CLIP_CNN, I replaced target_model from ViT-B/16 to RN101 in python script eval_albef2clip-vit_flickr.py , and i got following scores which are different from the ones reported in paper (Table 2, last column for SGA). Could you please clarify the anomaly or I missed something.

ALBEF to CLIP-CNN

Hi @Zoky-2020
Thanks for responding to my previous issues.

I need a few clarifications regarding Table-5 results.

1. Upon inspection of Refcoco+ dataset, I found out that refcoco+_test.json and refcoco+_val.json contain paths of images from train set of MSCOCO. I created a json file consisting of paths of these train images (along with captions) and then generated adversarial images by attacking ALBEF model.
2. Afterwards, I performed evaluation using Grounding.py. I ensured that dataset class loads adversarial images during evaluation by modifying image paths in __getitem__ of grounding.dataset.py.

I obtained following results which are not close to the ones reported in paper. Could you please comment if I missed something while reproducing Tabe-5.

Hi, thanks for the great work.
Could you please provide script (and instructions) for cross task transferability (ITR-to-IC and ITR-VG) to reproduce the results of Table-4 and Table-5 of arxive paper.
Thanks!

你好，我想问一下代码是在什么设备上跑的，单卡能跑起来吗，我试了一下跑albef2clip-vit爆显存

Hi，Zhiqiang @Zoky-2020
I am encountering some issues while replicating Sep-Attack and Co-Attack for transferability validation. Could you please share the relevant code? Thank you very much.

I think this parameter will affect the final experimental results.

Excuse me, the existing code only gives the code about ALBEF, TCL, CLIP(VIT), I did not find CLIP(CNN), what should I do if I want to use it?

thanks for your great work
but, I have a question for the specific method for generate corresponding adversarial captions, is used bert-attack???
and the last step to generate corresponding adversarial captions