Source Only and SN training about st3d HOT 13 CLOSED

I have also encountered the same issue, not only with source only and sn but also with ROS. My configuration is 1 * RTX4090, and I have found

# source only
bev  AP:10.0899, 8.0913, 9.1928
3d   AP:1.1020, 0.8122, 0.9163
# ros
bev  AP:41.8606, 29.9103, 29.7847
3d   AP:19.3675, 14.3215, 13.9450

from st3d.

I haven't meet this problem before. Can you eval the provided pretrained model normally?

from st3d.

I haven't meet this problem before. Can you eval the provided pretrained model normally?

Yes, the provided pretrained models are no problem. Even fine-tuning the ROS model by self-training using the provided pretrained model works fine. However, I encountered the aforementioned issue when training ROS and source only models from scratch.

from st3d.

Would you like to try to increase the batch size in the pretraining stage as our default setting?

from st3d.

Would you like to try to increase the batch size in the pretraining stage as our default setting?

Thank you for your enthusiastic reply. Due to the limited GPU memory, my current batchsize can only be set to a maximum of 8. If possible, I will try a larger batchsize in the future. Thank you so much!

from st3d.

I increased the batch_size and, by selecting the best model, ultimately obtained similar results. Thank you very much for your patient explanation.

from st3d.

I increased the batch_size and, by selecting the best model, ultimately obtained similar results. Thank you very much for your patient explanation.

Excuse me, how many batchsize did you use to get the above result.

from st3d.

I increased the batch_size and, by selecting the best model, ultimately obtained similar results. Thank you very much for your patient explanation.

Excuse me, how many batchsize did you use to get the above result.

the default value, 16

from st3d.

Great! I will close this issue. Feel free to re-open if you have further question.

from st3d.

Great! I will close this issue. Feel free to re-open if you have further question.

Hello, although I obtained similar results, it's still slightly off. Regarding this, I have some further questions to ask. In the GETTING_STARTED documentation, I found the following sentence: "Notice that you need to select the best model as your Pre-train model because the performance of the adapted model is really unstable when the target domain is KITTI." How should I choose the best model? Is it manually selected among the 30 evaluated models? When selecting, do I only consider AP_3d? Because sometimes the $AP_{3d}$ value is biggest, but the $AP_{BEV}$ value is very small.

Additionally, for the self-training models of ST3D, should I choose the last model? It seems that the results of the self-training models also have significant fluctuations.

from st3d.

Yes. I select with evaluating all 30 ckpts on $AP_{3D}$.
In ST3D, you also need to do so especially for target dataset is KITTI, but not need for other target dataset.
In ST3D++, this issue is largely mitigated, so you can also just choose the last ckpt.

from st3d.

I tried the nuscence >> kitti with PV-RCNN, the datasets for nu is v1.0-mini, I followed the sugguestions and used default batch_size 16, 50 epochs, trained on A6000. However, the results are:

(pvrcnn_old_anchor_sn.yaml)
bev AP:10.9448, 12.4171, 12.5784
3d AP:4.5455, 4.5455, 9.0909

I am wondering if the version v1.0-mini of nuscence datasets affact the final results?

from st3d.

The results of v1.0 mini doesn't have any relationship with the performance on v1.0-trainval.

from st3d.