BOX_FILTER usage vs POINT_CLOUD_RANGE about st3d HOT 4 CLOSED

Hello, good question.

1. This BOX_FILTER filters predicted boxes in inference stage, while mask_points_and_boxes_outside_range works on data prepare stage for ground truths.

   ST3D/pcdet/datasets/kitti/kitti_dataset.py

   Line 285 in 7b3ad3a

   if self.dataset_cfg.get('TEST', None) and self.dataset_cfg.TEST.BOX_FILTER['FOV_FILTER']:

2. Since KITTI dataset only annotates fov objects while other datasets annotate ring view objects, this filter is only applyed on KITTI.
3. FOV_POINTS_ONLY can filter the points outside fov. However, since both source labeled data and pseudo labeled target data have full ring view points, I am not sure whether testing model on only fov points would cause some negative transfer. In this regard, this BOX_FILTER works when we set FOV_POINTS_ONLY to False in KITTI. Actually, we found that in some situations, set FOV_POINTS_ONLY to False could even result in better performance (It seems that the provided checkpoint of Waymo -> KITTI has this characteristic).

from st3d.

See following codes, the mask gts will not works during inference.

Since mask gts already solve situations during training, you just need to consider inference time.
In this regard, you should modify following two place:

1. Filter predictions outside range similar to our BOX_FILTER configs.
   

   ST3D/pcdet/datasets/kitti/kitti_dataset.py

   Line 284 in 7b3ad3a

   # BOX FILTER

2. Filter gts outside range in following place.
   

   ST3D/pcdet/datasets/kitti/kitti_dataset.py

   Line 341 in 7b3ad3a

   eval_gt_annos = [copy.deepcopy(info['annos']) for info in self.kitti_infos]

You can also refer to here since I already implement this function in Lyft dataset.

from st3d.

Thanks for the detailed response. That makes a lot of sense.

Following on from that - in my understanding, when we set mask_points_and_boxes_outside_range and REMOVE_OUTSIDE_BOXES=True, it'll automatically remove the gt_boxes that are outside of the specific POINT_CLOUD_RANGE. E.g. if I set max x = 20m, it'll only keep any ground truth annotations that are less than 20m. My aim is to train on nuscenes and test on kitti within a certain pointcloud range and so I assume that it'll remove any gt_boxes outside this range.

However, when I exported some kitti pcds from the getitem function during evaluation, it was not the case. I show the exported pcds below. The pcd below is with fov=True and POINT_CLOUD_RANGE: [-20, -20, -2, 20, 20, 4]. You can see that the pointcloud was succesfully filtered however the ground truth boxes at around 60m are still present.

For reference, this is what the full fov pointcloud with POINT_CLOUD_RANGE: [-75.2 -75.2, -2, 75.2, 75.2, 4] looks like.

How do I specify the cfg settings for the kitti dataset in test/train to filter the ground truth annotation boxes such that my detection results are only evaluated on gt_boxes that lie within the POINT_CLOUD_RANGE I've defined?

from st3d.

Amazing. Thanks for your prompt response and for giving such helpful direction.

from st3d.