Comments (4)
I could think of one way to utilize VAE which is using it as a regularization of the latent codes. Semantic codes become samples from a normal distribution. We have tried this. It was hard to strike a balance between the sample-ability (strong regularization) and expressiveness (weak regularization) of the latent code. From the quality of sample perspective, it turned out be better to learn another DDIM on top of the learned (and frozen) latent codes.
from diffae.
They are not trained at the same time.
You can train the autoencoder alone with only images (and you will only get the autoencoder).
Since it is still an autoencoder, you CANNOT sample novel images from it yet.
To generate new images, you need to be able to sample the "semantic code".
In order to do this, you need a generative model, which is called latent DPM in this case, trained on a pool of semantic codes (to get this you need a trained autoencoder).
from diffae.
Thank you for your response! I finally understand your points after re-checking section 4 in your paper. Btw, have you experimented with a sampled latent from VAE for the generative process of DPM model? What I mean is that you stack a VAE on top of the diffusion model to get latent vector z for the decoding process.
from diffae.
what affects the sample-ability and expressiveness? I am a beginner on neural network.
from diffae.
Related Issues (20)
- Why use zero_module? HOT 2
- Configuration of the experiment -- attribute manipulation on real images HOT 4
- what is the input of conditional DDIM decoder? HOT 6
- how to visualize the reconstruction result
- Inquiry about using Guided-Diffusion parameters HOT 2
- How to determine cond_fn in condition_mean HOT 4
- I got the issue about lmdb: lmdb.Error: ffhq256.lmdb: No such file or directory HOT 7
- Regarding the error I met when I try to run the run_bedroom128.py HOT 4
- Difference in Model Weights HOT 1
- I cannot access to URL for converting the datasets to LMDB format
- Extensive GPU Usage for Manipulation HOT 1
- Issues with Conditional Sampling HOT 3
- about the partition of training and validation sets HOT 1
- It looks like z-sem is not being trained HOT 9
- an error occurred during evaluation. HOT 3
- the setting of use_inverted_noise
- Retraining for getting higher resolution Image
- Checkpoint
- log_sample after the batch training HOT 1
- How is the autoencoding happening in the code during inference?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diffae.