Comments (3)
Hi, I haven't conducted experiments on CDAN.It seems that target bn won't update properly. Here are the tips to check.
First, use pretrained model(e.g. ImageNet, ...) and initialize each bn with the same value.
Second, check the magnitude of the gradients of target bn parameter. If the conditional adversarial loss is too small or too large compared to source bn parameter, the bn parameter won't update properly. In most case, use low learning rate on backbone with SGD will help. Also, you can consider to change discriminator output of sigmoid to softmax with 2 classes.
Third, adjust balance hyper-parameter λ (adjust source classifier and discriminator) or scheduling parameters if you use other datasets.
Good luck with your project :)
from dsbn.
Hi, thanks for your reply. In fact, I have used some tips you mentioned.
First, I used pretrained model on ImageNet, and used the dbsn.py and resnetdbsn.py which are from your respository to take place of the network.py in CDAN.
Second, I used the conditional adversarial loss in default setting same as CDAN, and the corresponding hyper-parameter λ is setted to 1 in default. Also, I used the default low learning rate (0.001) in target bn with SGD same as other layers (i.e., convolutional layers).
In short, I just utilized the dsbn layer to take place of the bn layer in CDAN without any other unnecessary change.
Just as you said, I also think the refine parameters in target bn cannot be updated properly. Dose it mean the model need a other loss function to help the target bn update properly?
from dsbn.
I think you need to check the magnitude of the gradients of bn parameters. If the target bn gradients are low, try to change domain discriminator output of sigmoid to softmax with 2 classes, which may provide stable gradients.
Also you can add additional loss. I think psuedo-label based recent algorithms work well with DSBN.
from dsbn.
Related Issues (10)
- dsbn HOT 2
- Different domains in one batch HOT 1
- Can not reproduce the result HOT 1
- t-sne issue HOT 2
- Difference between DSBN and AdaBN HOT 1
- have you tried regression tasks HOT 1
- run script in other exp setting HOT 1
- reproduce HOT 4
- One problem HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dsbn.