layumi / seg-uncertainty Goto Github PK

Could you please provide VGG-based pre-trained model?

Hi, thanks for the nice code~
I have some questions about the pseudo-labels training part of the code,

this is the pseudo labeling loss of the source data as Eq.11 in paper:
loss_seg1 = self.update_variance(labels, pred1, pred2), in which the labels come from the source domain.

the target domain does not use the pseudo-labels but the entropy minimization:
loss_kl = ( self.kl_loss(self.log_sm(pred_target2) , mean_pred) + self.kl_loss(self.log_sm(pred_target1) , mean_pred))/(nhw)

Hi, thanks for the code!
I was wondering why in stage 2 the loss weight lambda-kl-target is set to 0.
From my understanding, the rectification is done by using this term.
Thanks in advance.

Sorry to bother you, when I downloaded the SYNTHIA dataset and decompressed it. I found there was no folder with "ground truth"(GT). I want to know if someone else is in this situation.

Hello, thank for sharing the great work. I have a quick question. I find that ''random scale'' and ''mirror'' are used for preprocessing data, which is a little different from previous works.

• So, for what reason did you adopt these strategies?

• Have you conducted experiments to verify how much performance improvement they bring?

Thanks!

Hi, thanks for your great job!
Recently I've been reading your code and I have a question about stage 2(rectifying). You set lambda_adv_target1 and lambda_adv_target2 to 0 which means there is no adv training in stage 2(Right?), but you keep training generator with false instruction from discriminator(the weight of discriminators is not loaded in stage 3), you annotated here which means you keep training G, but here you never updated D, is this the right behavior or maybe I misunderstood sth?

Hi, thank you for your great work on domain adaptation!
The origin input image is 1024512, but now I want to train on higher resolution, like 19201080.
How should I change the class balance parameters like max_value and 6464n.
In my opinion, it will be better to use ratio threshold instead of a hard assigned number like 64 like this:
for i in range(self.num_classes):
count[i] = torch.sum(labels==i)
if count[i]/(nhw) < 1/128.0: #small classes, original train size is 1024512
weight[i] = self.max_value
and 6464/(5121024) = 1/128
But I get worse results than default settings trained with input size (1024512).
Is there anything that I missed?
Any help will be appreciated.

Hi, can you share the SYNTHIA to Cityscapes pre-trained model to replicate the results of the paper? Thank you for the great work!

Nice work! I am a novice in the domain adaptation field. I have a question about stage2.
In my understanding about your paper, uncertainty estimation is used for target domain images.
But in your code trainer_ms_variance.py, it seems to be calculated on the image of the source domain.

    def gen_update(self, images, images_t, labels, labels_t, i_iter):
            self.gen_opt.zero_grad()

            pred1, pred2 = self.G(images)
            pred1 = self.interp(pred1)
            pred2 = self.interp(pred2)

            if self.class_balance:            
                self.seg_loss = self.update_class_criterion(labels)

            loss_seg1 = self.update_variance(labels, pred1, pred2) 
            loss_seg2 = self.update_variance(labels, pred2, pred1)
 
            loss = loss_seg2 + self.lambda_seg * loss_seg1

why you train your model in stage 1 and 3 with 100000 iterations, but choose 25000th iteration for finetunning and final evaluation?

I just run same setting as your code, but i only got 43.32 miou, compared with your 45.46.

Fig 3

Hello~As we know, SYNTHIA dataset only share 16 classes with Cityscapes. So when training the model on the Synthia dataset, do I need to change the model classification layer to 16 channels? Does this matter?

Thank you for sharing this wonderful code first!
And I have a small question in discriminator.I find the adversarial loss of AdaptSegNet is very unstable because of the global alignment in the segmentation output.I add non-local attention in the discriminator,but the performance drops dramatically.Then your discriminator,you say 'we follow the PatchGAN and deploy the multi-scale discrimimator model'.So for what consideration you utilize this strategy,and do you do an experiment to see how much performance improvement does this approach bring?

If I want to run the code without rectifying the pseudo label, I am confused that why in stage 2 the loss weight lambda-kl-target is set to 0? (From README)
From Eq.9 of the IJCAI2020 paper, I understand the KL-divergence loss is should be used.
Thanks

How much performance will decrease if batchsize=1 is used in the first stage? thanks.

Hello,

could you maybe explain the difference between prediction variance and prediction Confidence? In the paper, Figure 5 visualizes both, but I'm not sure how do you calculate them. I assume the KL term divergence is used as variance, then how to get the confidence?

Thanks in advance.

I use two titanx gpus.Can you tell me the type of GPU you use？

When conducting experiments of stage 1 training, state dicts of self.G model is inconsistent with the pretrained model in 'http://vllab.ucmerced.edu/ytsai/CVPR18/DeepLab_resnet_pretrained_init-f81d91e8.pth'. For example, key of state dict of self.G is 'layer5.conv2d_list.0.0.weight' and 'layer6.bottleneck.0.se.0.weight', but key of the pretrained model is 'layer5.conv2d_list.0.weight' and there is no module 'layer6.bottleneck.0.se.0.weight'. Should I set the strict as False in load_state_dict()?

Hi, I have some problems
1.for your release model:
stage 1 model (SE_GN_batchsize2_1024x512_pp_ms_me0_classbalance7_kl0.1_lr2_drop0.1_seg0.5), I test this model using your code, the mIoU is only 38.07, much lower than 45.5(stage 1 miou in MRNet paper). stage2 model(1280x640_restore_ft_GN_batchsize9_512x256_pp_ms_me0_classbalance7_kl0_lr1_drop0.2_seg0.5_BN_80_255_0.8_Noaug) is 50.34 , which is same as the report result in second paper.
2. train result
I use your released stage 1 model to generate pseudo label, then train stage 2, However, the performance is low: 43.89 for 25000.pth, 42.69 for 50000.pth, 41.98 for 100000.pth, much lower than the result in your paper. And I do not change any code except variable DATA_DIRECTORY

Thank you for sharing this code!
I have a small question, when I run the train_ms_robot. py file, I always get an error, AttributeError: 'Namespace' object has no attribute 'sync_bn'.

Hi @layumi , I noticed that in your paper, the uncertainty is obtained by computing both MC-Dropout and difference between two classifiers, but I cannot find the code used for MC-Dropout, could you please kindly tell me?

Hi,
thanks for sharing your implementation. I have two questions about it:

1. Does it also work on tabular data?
2. Is it possible to identify the noisy instances (return the noisy IDs or the clean set)?

Thanks!