Comments (7)
Ok. Thank you very much for your support. I guess we can close this issue.
Thank you again for this very nice framework.
from compressai.
Hi Navid, yes we had to get rid of this to add support for DataParallel (and we dropped nn.ParametersList also)
I just updated the changelog at https://github.com/InterDigitalInc/CompressAI/blob/master/NEWS.md with pointers. Here's the main change:
def configure_optimizers(net, args):
"""Separate parameters for the main optimizer and the auxiliary optimizer.
Return two optimizers"""
parameters = set(p for n, p in net.named_parameters() if not n.endswith(".quantiles"))
aux_parameters = set(p for n, p in net.named_parameters() if n.endswith(".quantiles"))
optimizer = optim.Adam((p for p in parameters if p.requires_grad), lr=1e-4)
aux_optimizer = optim.Adam((p for p in aux_parameters if p.requires_grad), lr=1e-5)
return optimizer, aux_optimizer
(You can also just define a single optimizer and per group options, see the pytorch doc here, but it's less flexible)
If you have some trained models you can update the state_dict to a compatible version with load_pretrained
.
Please let me know if you have any issues.
from compressai.
Thank you Jean,
I think you have already corrected also examples/CompressAI Inference Demo.ipynb
demo.
Thank you again.
from compressai.
Hello again Jean,
Before closing this issue, I found something strange that I wanted to share with you.
Since I have saved everything with the previous structure, I was able to double check something. I want to know if I am doing something wrong or you have the same results.
Let's imagine I am using ScaleHyperprior with quality of 5. For the number of parameters, with the previous structure I was getting:
# model parameters______________________: 5068035
# entropy bottleneck(s) parameters______: 7808
But with the new structure I get:
# model parameters______________________: 5075459
# entropy bottleneck(s) parameters______: 384
The sum of the two versions are equal (5068035+7808 == 5075459+384), but as you can see the number of parameters for entropy bottleneck and model parameters are very different. I am using the following code in the new structure to find the number of parameters:
parameters = set(p for n, p in net.named_parameters() if not n.endswith(".quantiles"))
aux_parameters = set(p for n, p in net.named_parameters() if n.endswith(".quantiles"))
optimizer = torch.optim.Adam((p for p in parameters if p.requires_grad), lr=args.learning_rate)
optimizer_aux = torch.optim.Adam((p for p in aux_parameters if p.requires_grad), lr=args.learning_rate_aux)
n_parameters_dict = dict()
n_parameters_dict["# model parameters"] = sum(p.numel() for p in parameters if p.requires_grad)
n_parameters_dict["# entropy bottleneck(s) parameters"] = sum(p.numel() for p in aux_parameters if p.requires_grad)
print(n_parameters_dict)
Have you seen the same issue? I am asking because it might be a bug in my code because I am not exactly using your code.
best,
Navid
from compressai.
Hi Navid, that's correct.
Previously the aux_parameters
group was containing all the parameters of the entropy bottlenecks, which is not 100% correct since only the quantiles should be updated by the auxiliary optimizer. However it was not impacting the convergence or the performance (the only difference was the learning rate for the entropy bottlenecks parameters).
See this other issue also: #16.
The current implementation in the example should be cleaner and asserts that there's no overlap (or omission) between parameters groups.
from compressai.
Thank you Jean. That's very interesting.
Just the last question. Don't you think this has an effect on backward compatibility? For those parameters that are moved from aux to model parameters, all models learned with the previous version were using another learning rate for aux parameters which is less than the learning rate for the model parameters. I think this has an impact. Have you tested to see the difference in the performance?
from compressai.
Hi Navid, I've trained multiple models with both versions and haven't noticed any performance difference. But please let me know if you observe any issue.
from compressai.
Related Issues (20)
- Adding new codec into CompressAI HOT 4
- Lambda values missing for neural codecs HOT 1
- The reconstructed image using codec.py is inconsistent with the original image HOT 2
- How can I finetune the large bitrate model (λ=0.05) to obtain the low bitrate model (i.e. λ=0.013) HOT 2
- training SSF2020 model with quality 8 HOT 1
- The Variable-Rate model training script HOT 1
- from compressai.ans import BufferedRansEncoder, RansDecoder HOT 1
- 'utf-8' UnicodeDecodeError: codec can't decode byte oxd6 in position 1: invalid continuation byte When export to ONNX HOT 1
- quality level argument missing in train.py? HOT 1
- Issue Regarding the Use of GaussianMixtureConditional HOT 6
- seems something worng with "python -m compressai.utils.eval_model --help " HOT 2
- Issue with Using CheckerboardMaskedConv2d from layer.py HOT 1
- sensetime and google HOT 8
- When I wanted to migrate that program to sequence signal data, I found that @register_model("mbt2018") The compress and decompress methods of class JointAutoregressiveHierarchicalPriors are very complex HOT 1
- On the Compatibility of GAN HOT 2
- Request ONNX export support for Cheng2020 model HOT 1
- error when trying to use entropy bottleneck of compressai in my code HOT 1
- Module Not Found Error: 'compressai' HOT 1
- How to convert compress and decompress into two ONNXs? Normally, compress and decompress are placed on two devices
- Intermittent Process Crash During Bit Loss Testing with RansDecoder (Process finished with exit code -1073741819 (0xC0000005))
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from compressai.