Questions regarding training and distillation about gan-compression HOT 2 CLOSED

Yes, you're correct. The once-for-all network training will take no more than 2 days on a single 1080Ti.

from gan-compression.

I have read the Appendix 6.1 on the paper, and the authors have provided the complete pipeline of the training.

1. train a MobileNet style network from scratch (teacher network)
2. distill a smaller student network using the network train in the previous stage
3. initialized by the student network, train ta "once-for-all" network
4. evaluate all sub-network under a certain computation budget
5. choose the best-performing sub-network within the previous step and fine-tune the compressed model for the final model.

For the second stage, I am guessing the author has transferred the weight of the teacher to the student network using load_networks.

Finally, the authors provided Table 5. for training details such as training epochs (for training from scratch, distillation and fine-tuning) and once-for-all network training. For pix2pix and cycle gan they have doubled the epoch for once-for-all network compared to the training/distillation/fine-tuning.

I guess this answers all the questions I had.
Please correct me if I got something wrong.

from gan-compression.