More questions for nuScenes about clocs HOT 5 OPEN

Sorry to bother you again! I have questions when transfering CLOCs on nuScenes dataset.

1. Could you please ask if you implement CLOCs in SECOND1.6? or is it feasible?
2. You said "we simply use 10 separate CLOCs networks for 10 different classes.". Do you mean that each network only focuses on one category, and the other nine are set to "Don't Care"?
3. For nuScenes, there are six cameras and a 360-degree LiDAR. Does that mean six 2D detections correspond to one 3D detections? Could you please explain more detail for me about how the correspondence is set up here as it is not quite the same as KITTI?

from clocs.

Hello @JessieW0806 , please don't say sorry, I'll help you as much as I can.

1. I have not implemented CLOCs in SECOND 1.6, but I would say it is feasible, I use SECOND-1.5 just because when I developed CLOCs around 2 or 3 years ago, SECOND-1.6 has not been released. Just be careful with the representations of the 3D bounding boxes in SECOND-1.6, check the orders of the 7 parameters (x,y,z,h,w,l,r), and check the center location (xyz) is on the center of the bounding box or at the bottom of the bounding box.
2. Currenly one CLOCs network can only do fusion on one class. Therefore, for 10 classes, we use 10 CLOCs networks.
3. Yes, LiDAR is 360 degrees field of view. There are 6 cameras in nuScenes dataset facing six different directions, each one of them covers around 90 degrees field of view (I am not sure about the exact number, but I am sure each camera can only covers a limited field of view), and each one has its own field of view. There could be some objects that are visible from two cameras, for this situation, I just simply pick the one with the higher score. For each camera, we only care about the same view from LiDAR.

from clocs.

Thanks a lot！
1）So you are modifying the results of nuScenes based on CLOCs（Second 1.5）, right？
2） Just to be sure, the 10 CLOCs networks are trained individually just like KITTI dataset?

from clocs.

3）Could you share which code you ran to get the 2D detection results？：）

from clocs.

Hi @JessieW0806 ,
(1) I used CenterPoint as the codebase for nuScenes, because SECOND has limited performance, but you could use SECOND if your prefer, I would recommend CenterPoint because it is better tuned for nuScenes dataset.
(2) The 10 CLOCs networks can be trained individually or jointly, if individually, just follow the KITTI dataset style. If you want to train them jointly, simply build 10 loss functions, one for each class, and you are good to go.
(3) I am sorry the code for 2D detections that we use for nuScenes is a custom designed network and it is under review, we will release it in the future. But I think there are other available 2D detection networks available.

from clocs.