Train on custom dataset for panoptic segmentation about mask2former HOT 5 CLOSED

The overlap is calculated with mask instead of box.

from mask2former.

The ZeroDivisionError: division by zero is because n (number of classes) is zero in your defined dataset. Please check your data is in the COCO panoptic segmentation format: https://cocodataset.org/#format-data

from mask2former.

Thanks for replying.

I find out what happened, but am still confused.
My data is in the COCO panoptic segmentation format. And training schedule seems alright.
The problem lies in that n in panopticapi.evaluation.pq_average is not increased due to an empty prediction.
(where all of catagories skipping n += 1 because of tp + fp + fn == 0: continue). I will report this to panopticapi.

But I still wonder how to prevent emtpy predition from the mask2former model.
It looks like following. I am still confused about

{       
        "images": [...],
        "annotations": [
                {
                    "image_id": "0012",
                    "file_name": "0012.png",
                    "segments_info": []
                },...
        ],
        "categories": [...]
}

def pq_average(self, categories, isthing):
    pq, sq, rq, n = 0, 0, 0, 0
    per_class_results = {}
    for label, label_info in categories.items():
        if isthing is not None:
            cat_isthing = label_info['isthing'] == 1
            if isthing != cat_isthing:
                continue
        iou = self.pq_per_cat[label].iou
        tp = self.pq_per_cat[label].tp
        fp = self.pq_per_cat[label].fp
        fn = self.pq_per_cat[label].fn
>>        if tp + fp + fn == 0:
>>            per_class_results[label] = {'pq': 0.0, 'sq': 0.0, 'rq': 0.0}
>>            continue
>>        n += 1
        pq_class = iou / (tp + 0.5 * fp + 0.5 * fn)
        sq_class = iou / tp if tp != 0 else 0
        rq_class = tp / (tp + 0.5 * fp + 0.5 * fn)
        per_class_results[label] = {'pq': pq_class, 'sq': sq_class, 'rq': rq_class}
        pq += pq_class
        sq += sq_class
        rq += rq_class

    return {'pq': pq / n, 'sq': sq / n, 'rq': rq / n, 'n': n}, per_class_results

from mask2former.

I'm also curious about the following configuration in config/*.yaml
Does it mean that, when two bbox overlap to a certain extend, smaller one will be suppress ?

For my own dataset and task, stuff (isthing=0) category will always occupy most of the picture, with thing(instance) ones drenched in it. While some instances will also overlap with each other.

    TEST:
      SEMANTIC_ON: False
      INSTANCE_ON: False
      PANOPTIC_ON: True
      OVERLAP_THRESHOLD: 0.8
      OBJECT_MASK_THRESHOLD: 0.8

Best regards ~

from mask2former.

@JasonRichard Hi! Did you resolve this problem?

from mask2former.