Possible bug in the instance segmentation evaluation script about scannet HOT 9 CLOSED

Fixed in a3c66b4

from scannet.

When I run this script, I met the following error:

File "evaluate_semantic_instance.py", line 252, in assign_instances_for_scan
gt_ids = util_3d.load_ids(gt_file)
File "/home/liang/ScanNet/BenchmarkScripts/util_3d.py", line 41, in load_ids
ids = np.array(ids, dtype=np.int64)
ValueError: invalid literal for int() with base 10: 'pred_mask/scene0086_00_1.txt 19 1.000000'

The gt_file should be a file like this format:
pred_mask/scene0086_00_1.txt 19 1.000000

So why can it be the input of "load_ids" function?

As charlesq34 said, gt_ids should be label_id*1000+inst_id.

The ground truth of instance task is generated by export_train_mesh_for_evaluation.py

from scannet.

I do not understand why gt_ids == label_id*1000+inst_id. I do not find the function to define this.

from scannet.

So maybe I understand. I misunderstood the format of gt_file.

gt_file should be a txt with one integer each line. And The integer should be equal to label_id*1000+inst_id.

Do I understand rightly? So if we want to evaluate, we need to generate the gt_file by ourselves?

from scannet.

@LZDSJTU right. the format of the GT file for instance segmentation is different from the PREDICTION files. You have to generate the GT files by yourself.

from scannet.

@LZDSJTU right. the format of the GT file for instance segmentation is different from the PREDICTION files. You have to generate the GT files by yourself.

Yep, and I have generated the GT file by myself. But I have a question that how you know that gt_file should be like "label_id*1000+inst_id". I do not find this in the scannet scripts.

Is this your guess according to the evaluate scripts?

from scannet.

@LZDSJTU you can see in the evaluation script the GT_FILE is loaded by the load_ids function and then fed into util_3d.get_instances which then calls the Instance class initialization which assumes the id to be label_id*1000+inst_id.

from scannet.

Hello, I use this command to generate the instance gt-file "python export_train_mesh_for_evaluation.py --scan_path /home/data/scannetv2/scans/scene0000_00 --output_file /home/data/scannetv2/output.txt --label_map_file /home/data/scannetv2/scannetv2-labels.combined.tsv --type instance",
and I get the gt file like this:
pred_mask/output_1.txt 9 1.000000
pred_mask/output_2.txt 9 1.000000
pred_mask/output_3.txt 7 1.000000
pred_mask/output_4.txt 12 1.000000
pred_mask/output_5.txt 38 1.000000
pred_mask/output_6.txt 16 1.000000
pred_mask/output_7.txt 16 1.000000
pred_mask/output_8.txt 14 1.000000
pred_mask/output_9.txt 3 1.000000

I am not sure whether this is correct, waiting for your reply

from scannet.

@LZDSJTU hello, the gt file should be gt_ids == label_id*1000+inst_id, what does the label_id and ins_id mean? As you mentioned above, gt_file should be a txt with one integer each line, but the ins gt file I generate like "pred_mask/output_1.txt 9 1.000000", can you tell me how to generate instance ground truth file? I really hope to get your help! thanks!

from scannet.