cc-ai / munit Goto Github PK

code https://github.com/GauthierGidel/Variational-Inequality-GAN/blob/master/optim/extragradient.py

+

def opt_step(opt, step):
    """
    Run an optimizing step ; if using ExtraAdam, there needs to be an extrapolation step every other step
    """
    if step % 2 == 0:
        opt.extrapolation()
    else:
        opt.step()

Though may not be needed with LSGan loss ? to investigate

Learning rates? D vs G updates? etc.

use nn.DataParallel

TODO to improve the code architecture :
train.py

• Function for the display images (train.py l123)
• Switch logger and output folders to utils (train.py l137)
  trainer.py
• Switch the output classifier to the segmentation head (trainer.py)
• Factorize the encoding in the trainer (trainer 253, 292 etc..)
  -Optimize compute_semantic_seg_loss()
  -clean the sample() function and add synthetic data in the comet
  -add a resume to the segmentation head

Some things we are missing, looking through the Mapillary data:

• Having non-flooded images with some water (fountains or sea or ponds or whatnot)
• Sidewalk/road orthogonal, facing buildings + houses
• Parking lots

Things to do:
Check if mapillary test is taken from train
Check segmentation on img with fences and plants around them, or hills: im5eLSxcGN

To be able to compare the range of these losses and understand why we'd need semantic matching if we have L1 matching (weakened by masks)

Use the one test-time guiding image at train-time too

Data sources:

LabelBox (VICC team)
KeyMaker
Flood Water Roadways
Semantic Segmentation model

In cyclegan there's this comment

        Forward cycle loss:  lambda_A * ||G_B(G_A(A)) - A|| (Eqn. (2) in the paper)
        Backward cycle loss: lambda_B * ||G_A(G_B(B)) - B|| (Eqn. (2) in the paper)

we should try having lambda_A > lambda_B