Comments (3)
Hi, @twmht
Our supernet does not contain dynamic gate, you can use Autoslim algorithm to find a most suitable subnetwork in the supernet. Please note that our routing space (or search space) is small (only 14 sub-networks) as we need to save BN statics for every sub-network. If you want to perform NAS (eg. Autoslim), you could enlarge the search space.
During our supernet training, we use in-place bootstrapping, which outperform the in-place distillation used in original Autoslim paper (by around 1~2%). So we expect searching in our supernet can lead to better result than Autoslim.
from ds-net.
Or can we just remove the dynamic gate after training, and then run the geedy autoslim, and found a better result?
from ds-net.
I am not sure if the performance can be boosted if I use dynamic gate for training, but inference without them.
For example, without the distillation, Have you ever compared the following two examples,
-
training with dynamic gate, and remove the dynamic gate, and then searching from the supernet, get a subnet and get the accruacy1.
-
training without dynamic gate, like normal slimmable network, and get a similar flops subnet like example1 and get the accuracy2.
I am curious if accuracy1 is better than accuracy2? if it is, then I can conclude that gate training is helpful for boost the performance with slimmable network.
By the way, talking about the distillation for object detection, I am trying to train for feature map distilling, but it's not good. Maybe the feature map distilling dose not make sense for slimmable network, since the weights are shared for all subnet and supernet. So I am wondering how you do distilling for object detection?
from ds-net.
Related Issues (18)
- The Approximate Date for Stage II training code HOT 6
- The usage of gumbel softmax in DS-Net HOT 7
- Actual acceleration on Resnet HOT 2
- Dynamic path for DS-mobilenet HOT 1
- 运行问题 HOT 1
- DS-Net for object detection HOT 8
- MAdds of Pretrained Supernet HOT 2
- why not set ensemble_ib to True? HOT 2
- Softmax twice for SGS loss?
- Question about calculating MAdds of dynamic network in the paper HOT 3
- Pretrained models HOT 2
- Object Detection HOT 2
- project environment HOT 1
- Why the num_choice in different yml is different? HOT 2
- Some issues about the gradients of slimNet HOT 6
- UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum. HOT 3
- Error of change the num_choice in mobilenetv1_bn_uniform_reset_bn.yml HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ds-net.