Contrastive loss implementation discrepancy between the paper and codebase about mdetr HOT 1 CLOSED

Hi,
It looks like you're confusing the contrastive_align_loss with the contrastive_loss.
In our paper and published results, we do not use the contrastive loss (which is akin to an image-text matching loss from other vision+language pre-training papers). We only left it in the code for completeness since it is something we tried at some point, and thought it would be useful if other users of our code base were interested in experimenting with it. For the two losses that we do use, read the following:

1. Contrastive align loss, which is calculated between the predictions of the decoder and the embedded representations of the text and the output of the cross encoder. Relevant lines in the code:

   mdetr/models/mdetr.py

   Line 81 in fdee8c5

   if contrastive_align_loss:

   ,

   mdetr/models/mdetr.py

   Line 203 in fdee8c5

   if self.contrastive_align_loss:

   ,

   mdetr/models/mdetr.py

   Line 496 in fdee8c5

   def loss_contrastive_align(self, outputs, targets, positive_map, indices, num_boxes):

2. Contrastive alignment -> loss_contrastive_align that we just discussed above. Soft token prediction is loss_labels

   mdetr/models/mdetr.py

   Line 464 in fdee8c5

   def loss_labels(self, outputs, targets, positive_map, indices, num_boxes):

Hope this makes it more clear! :)

from mdetr.