if I change the teacher model to a much smaller one like mobilenet, will the distill works too? about gan-compression HOT 6 CLOSED

thx for your reply!!
btw, I should run distill.py then train_supernet.py , or I just run train_supernet.py after training the teacher model. And what is the difference between the two process.

I think the distillation will work, but you may need to add some codes to specify the distillation layers for the mobilenet or unet architecture.

If you need to reproduce our results, you may need to run distill.py then train_supernet.py. The distill.py can make the model compact before the "once-for-all" network training.
As a matter of fact, you could directly run train_supernet.py after training a teacher model if you change the candidate subnet set in resnet_configs. We will release a lite pipeline later which will skip the distillation process.

from gan-compression.

Sorry, I do not quite understand your question. Currently, our codebase only supports ResNet generator because the generators in pix2pix and CycleGAN are in ResNet architecture (on pix2pix, we observe that the ResNet generator is better than the UNet generator, refer to
Network architecture for pix2pix in Appendix 6.2). We do not modify the backbone of the original generator.

from gan-compression.

Sorry, I do not quite understand your question. Currently, our codebase only supports ResNet generator because the generators in pix2pix and CycleGAN are in ResNet architecture (on pix2pix, we observe that the ResNet generator is better than the UNet generator, refer to
Network architecture for pix2pix in Appendix 6.2). We do not modify the backbone of the original generator.

thanks for your reply!
I want to know your code base is compatible with different architecture. That is if I train my teacher model on mobilenet or unet instead of resnet, would the distill work.

from gan-compression.

I think the distillation will work, but you may need to add some codes to specify the distillation layers for the mobilenet or unet architecture.

from gan-compression.

I think the mapping_layers defines the distillation layers (see this line). We should try to make it as a flag and make it also applicable to other architectures.

from gan-compression.

thx for your reply!!
btw, I should run distill.py then train_supernet.py , or I just run train_supernet.py after training the teacher model. And what is the difference between the two process.

I think the distillation will work, but you may need to add some codes to specify the distillation layers for the mobilenet or unet architecture.

from gan-compression.