yongliu20 / unilseg Goto Github PK

Is there anything I need to adjust?
Why is the result I get different form yours so much?

Thank you for your outstanding work! The model seems to be more tailored towards Referring Image Segmentation, and I'm still somewhat confused about testing for Semantic Segmentation (SS) and Open-Vocabulary Segmentation (OVS). Although the paper mentions that "Semantic segmentation and open-vocabulary segmentation can be reformulated as language-guided paradigm by replacing output layers with computing the similarity between visual and linguistic embeddings," the process still appears unclear to me.

From what I understand, the model seems to output a mask by calculating the similarity between the activated visual features and content-aware linguistic embedding. However, I'm unsure how this is evaluated in SS or OVS. Here's my guess:

For example, in Open-Vocabulary Segmentation, for a given image, we need to identify which categories are present (say, M categories). Then, for each category, the similarity calculation is performed between the activated visual features and content-aware linguistic embedding, ultimately outputting M masks. These masks are then merged to create the final semantic segmentation map.

Could you please confirm if this understanding is correct? If not, could you provide more details on how the model operates for these tasks?

Thank you for your assistance!

Hi authors, thanks for your good work.
Whether the training code be released?

  if not self.cfg.aux_loss:
      pred = torch.bmm(query_output, pixel_output.flatten(2)) 
      pred = rearrange(pred, 'b l (h w) -> b l h w', h=h, w=w)   
  else:
      for l, q in enumerate(query_output):
          final_output = []
          pred = torch.bmm(query_output[l], pixel_output.flatten(2))
          pred = rearrange(pred, 'b l (h w) -> b l h w', h=h, w=w)
          final_output.append(pred)
  return pred.detach()

请问这里为什么会输出最后pred，其他5个pred得作用是什么呢?
麻烦您解决我的困惑，非常感谢！！！！

would you release the UniLSeg-100 model weights
thanks!

Looking forward to it!

Hi @workforai et al.,

thx for ur cvpr'24 work. for part segmentation, may i ask if the conventional ap metric (apart from iou in the paper) could be reported as well? looking fwd to the code & ckpt. thx & best,

注意到在bpe_simple_vocab_16e6.txt.gz文件中没有发现object、all objects等提示词，想分割图像中的所有物体 提示词写什么