Trouble on pre-training models on Cityscapes about sc-sfmlearner-release HOT 16 CLOSED

1. Use "https://github.com/Huangying-Zhan/kitti-odom-eval" for VO evaluation to avoid wrong scale calculation.
2. If results are still bad, please check whether you model is correctly trained. e.g., You can evaluate the depth model in KITTI test images, and see whether it is reasonable.
3. If depth model is bad, you may check the training and validation loss

from sc-sfmlearner-release.

@JiawangBian Thanks for your reply! The results got better in the case where I trained models with gt. But I find there is a big gap between w/t gt and w/o gt, keeping the same other parameters. For example, I evaluate the depth model:

without gt:
abs_rel, sq_rel, rms, log_rms, d1_all, a1, a2, a3
0.4429, 4.7569, 12.0832, 0.5876, 0.0000, 0.3033, 0.5608, 0.7662

with gt:
abs_rel, sq_rel, rms, log_rms, d1_all, a1, a2, a3
0.1571, 1.1660, 5.8016, 0.2358, 0.0000, 0.7859, 0.9281, 0.9724

I see in other issues that you said your methods could work well when w/o gt for validation. Is there anything wrong I did? Or when I want to train models w/o gt, I need to carefully tune the parameters?

from sc-sfmlearner-release.

It seems that you meet a bug when training w/o gt。you may check the training loss and validation loss。Or you can just train it again。

from sc-sfmlearner-release.

@JiawangBian I visualize outputs of dispnet with different training settings. And I find that DispResNet can only predict a trivial result w/o gt for validation, while this bug won't occur in the case of training with gt or using VGG as backbone. It seems that DispResNet w/o gt is struggling in a local minimum. Have you ever meet this bug? And maybe it's similar to #2 that in my case (DispResNet w/o gt) the smoothness loss will soon drop to zero due to the trivial result even if its weight is 0.

from sc-sfmlearner-release.

1. This bug is regarless of using gt for validation, because gt is not used for training and not contributing any graident for avoiding terrible local minimum.

2. It appears ramdonly. It may work well when you train that again without changing anything.

from sc-sfmlearner-release.

Yes, my statement is not accurate. But this phenomenon is not random in my tests. Every time I train my models on Cityscapes with ResNet as the backbone, I can only get naive results, so the results are terrible. However, it is much better when VGG is used as the backbone. And another confusing thing is that even when the outputs (depth and relative pose) are very bad, the warped image looks correct, which may be caused by jointly training. But what I didn't think clearly is whether this is an inherent problem with this joint training method, or is it just a bug in the code. Sorry to bother you many times. Thanks!

from sc-sfmlearner-release.

Yes, my statement is not accurate. But this phenomenon is not random in my tests. Every time I train my models on Cityscapes with ResNet as the backbone, I can only get naive results, so the results are terrible. However, it is much better when VGG is used as the backbone. And another confusing thing is that even when the outputs (depth and relative pose) are very bad, the warped image looks correct, which may be caused by jointly training. But what I didn't think clearly is whether this is an inherent problem with this joint training method, or is it just a bug in the code. Sorry to bother you many times. Thanks!

Hi Feng @UltronAI ,

Is your depth visualization while training written by yourself? I haven't seen it in original code. Another question, have you test the posenet model trained by yourself? I test the model trained by myself, but it looks terrible.

regards

Yu

from sc-sfmlearner-release.

@TopGun666 Hi Yu. You can see codes for visualization in https://github.com/ClementPinard/SfmLearner-Pytorch. And the following picture shows the best results in my tests which are worse than those in the original paper.

You can visualize your estimates and other information during training and see if it is also the case I mentioned.

from sc-sfmlearner-release.

Thanks for your reply.

from sc-sfmlearner-release.

Where to download the pre-training model on CityScapes? Thanks!

from sc-sfmlearner-release.

@CongFang you can find the download link in README. In my case, I want to pre-train models on Cityscapes by myself.

from sc-sfmlearner-release.

@CongFang you can find the download link in README. In my case, I want to pre-train models on Cityscapes by myself.

I want the download link of the pre-training model on CityScapes. But, in README only have k_depth.tar and cs+k_depth.tar.

from sc-sfmlearner-release.

@CongFang you can find the download link in README. In my case, I want to pre-train models on Cityscapes by myself.

I want the download link of the pre-training model on CityScapes. But, in README only have k_depth.tar and cs+k_depth.tar.

Added.

from sc-sfmlearner-release.

@CongFang you can find the download link in README. In my case, I want to pre-train models on Cityscapes by myself.

I want the download link of the pre-training model on CityScapes. But, in README only have k_depth.tar and cs+k_depth.tar.

Added.

Think you for reply. But, I think you uploaded the wrong model. Can you share the depth model pre-training model on CityScapes? As you mentioned in your paper: 'Also, we pre-train the network on CityScapes and fine-tune on KITTI, each for 200 epochs.'

from sc-sfmlearner-release.

@CongFang you can find the download link in README. In my case, I want to pre-train models on Cityscapes by myself.

I want the download link of the pre-training model on CityScapes. But, in README only have k_depth.tar and cs+k_depth.tar.

Added.

Think you for reply. But, I think you uploaded the wrong model. Can you share the depth model pre-training model on CityScapes? As you mentioned in your paper: 'Also, we pre-train the network on CityScapes and fine-tune on KITTI, each for 200 epochs.'

Updated

from sc-sfmlearner-release.

@CongFang you can find the download link in README. In my case, I want to pre-train models on Cityscapes by myself.

I want the download link of the pre-training model on CityScapes. But, in README only have k_depth.tar and cs+k_depth.tar.

Added.

Think you for reply. But, I think you uploaded the wrong model. Can you share the depth model pre-training model on CityScapes? As you mentioned in your paper: 'Also, we pre-train the network on CityScapes and fine-tune on KITTI, each for 200 epochs.'

Updated

Thinks again!

from sc-sfmlearner-release.