researchmm / soho Goto Github PK

"wget https://sohose.s3.ap-southeast-1.amazonaws.com/checkpoint/soho_res18_fp16_40-9441cdd3.pth"

I can not download this pth file even if i am using a VPN.

It throw a "ERROR 403: Forbidden" error.

Can you fix it? thanks.

Hi, what is your mvm accuracy of pretrained model? I only got about 30% when pretraining and wanted to know if that is normal?

In the paper, there is a Visual Dictionary(VD) to remodel the image of query, but the class of SOHO_direct_VD(SOHO/models/necks/utils.py) only operate the image by torch.agrmax in the code, which is not matched with what you described in paper. Please tell me where I can find the VD which is the same as described in the paper. Thank you.

vd呢。。

wget https://sohose.s3.ap-southeast-1.amazonaws.com/data/pretraining/coco_cap_train_pre.json
wget https://sohose.s3.ap-southeast-1.amazonaws.com/data/pretraining/coco_cap_val_pre.json
wget https://sohose.s3.ap-southeast-1.amazonaws.com/data/pretraining/vg_cap_pre.json

Hi;
I conduct the pretraining with resent18+3 layer transformer by using indomain data. (without MVM loss)

I can get a similar result on VQA downstream tasks, around 66.5 accuracy.
But the performance on visual entailments is relatively lower than reported in the paper, I can just get 74 accuracy (~82% reported in paper)
I am wondering why the resnet18+3 layer outperforms the Uniter Base?
Are there any training strategies specialized for this downstream task?

Thanks

Can you share your slide of your oral presentation?

Hi,
many thanks for your sharing SOHO.
In Readme.MD, i can only find how to pretrain and train a VQA model. However, there is no instruction to train or evaluate an image/text retrieval model. Could you release the retrieval codebase?

hi, would you release a tool for visualizing the visual dictionary

Thanks for your great codes. This is an impressive work that may inspire many ones to follow it.
Do you plan to release the training configurations and scripts of the pre-training?

Where is the link regarding the pre-trained model of Soho based on Resnet101?