starmemda / camoe Goto Github PK

hi，I have read your paper, but I have some questions.

According to the code you left, I found that your formula is inconsistent with the code.

Which one should I choose?

thanks for your great work, it looks very strong baseline to explore. a simple Question about DSL, in your expriment,whta's the temperature ? I can't find anything in your paper....

It seems that caption and video must be one by one pairing diagonally .

I am trying to evaluate the DSL on MSRVTT full split (2990 videos and 2990*20 captions), but the DSL didn't work. Howerver, on MSRVTT 1k split (1000 videos and 1000 captions), it works well (49.0% V2T-R@1 and 47.8% T2V-R@1). My model is CLIP4CLIP.

Therefore, video and text matching information needs to be known in advance. Could you report the random shuffle comparative experiments on evaluation? If the random shuffle invalidate DSL, I am suspicious of data leak.

Hi, I'm trying to evaluate the effectiveness of DSL loss. I've tested it with CLIP4Clip, but the improvement was not consistent with you paper. I got 44.1%@V2T-R@1 for original loss and 44.6%@V2T-R@1 for DSL loss. I revised simlarity matrix both on training and infernece time. Can you help me ?

Thanks for your work ! I am unable to reproduce the results using DSL... Are you using DSL in both training and evaluation stage , or using it only in training stage? Thanks.

Hello,

Open-source project
We intented to publish the dual softmax loss firstly, the entire version will be available before the end of this year.

I'm looking forward to the full version!

By the way, are you going to clarify any open-source license of this project?

Thank you for sharing the key code of DSL, which has been of great help to me.
However, during the testing phase of the MSVD dataset, text and video are not in a one-on-one relationship. I would like to know how to modify the code in this situation.

您好，您发布的DSL损失中，sim_matrix这个是什么呢，能否具体的发布一下这个的用法呢？谢谢

Hi, thanks for your work. I read the paper and the boost of DSL is substantial so it is worthy to find this. However, my main criticism when using this in practice would be:
a) at inference requires all text to be queried together, in order to use the prior
b) the prior that there is a one-to-one mapping between test set queries and videos is not always true in the real world. You could do this with classification tasks if you know all classes have equal frequency -- however in practice this is not the case. So I think this is an unrealistic setup for text-to-video retrieval, you can have a user spam the text query "boy running" 100 times and this would cause catastrophic results for DSL.

Do you have results when this is used during training but not testing? If it helps in that case it would be good to know