Reintroducing the crosswalk class back into MSeg about mseg-api HOT 2 CLOSED

Hi @ionicaML, thanks for your interest in our work. There are a few different ways to go about it:

Regarding (1), mseg-mturk is designed for re-assigning categories to instances, not for re-drawing polygon boundaries. So you would need to use a different polygon-drawing tool (Amazon MTurk has example code for this, and there's an active area of research about building tools for human-in-the-loop segmentation annotation).

Regarding (2), this is feasible. You would set "road" in 5 datasets (ADE20K, COCO, IDD, BDD, and Cityscapes) to "unlabeled", and then add "crosswalk" to the universal taxonomy as a new class, and route Mapillary's crosswalk marking to it, instead of to "road", as we had it previously. This would hurt generalization for the "road" class, but if you need crosswalk predictions urgently enough, maybe the tradeoff is worth it for you.

Regarding (3), this is another reasonable and possible approach. Allowing for this inconsistency clashes with the principles/decision tree we designed in Figure 3 of our paper, and we haven't evaluated the effect on lane marking or crosswalk labels. But it's feasible.

A fourth option is to use the equivalent of a logical OR gate in your loss function, like SwiftNet did, summing over multiple logits using multi-label ground truth (See Equation 4 of their paper).

In short, this is still an open research area and we welcome your feedback and seeing what you discover.

from mseg-api.

Hi, @johnwlambert I want to make something similar with the dataset, but also add curb and curb cut. So for this until now i figured out, that I should modify the tsv files of dataset you mentioned above. But I dont know if this is complete, so I have two questions:

1. When you said add 'crosswalk' to universal taxonomy you refer to add a new line in MSeg_master.tsv for this class? I should do this also for, curb (curb, curb cut)? are there other files, i should change?

2. I want to keep only 64 classes, so this means I only remove them (their lines) from MSeg_master.tsv and add to final line to unlabeled, and also modify the tsv file for every dataset (ex. ade20k-151_to_ade20k-150.tsv and ade20k-151_to_ade20k-150-relabeled.tsv).

Waiting for you response.

from mseg-api.