photrek / coupled-vae Goto Github PK
View Code? Open in Web Editor NEWMachine learning algorithms for complex systems leveraging the methods of nonlinear statistical coupling
License: GNU General Public License v3.0
Machine learning algorithms for complex systems leveraging the methods of nonlinear statistical coupling
License: GNU General Public License v3.0
Use the latest VAE from the following directory, integrate the nsc lib at the following parts:
This will then become Coupled VAE v4.
This is to check and ensure that the latest VAE code is working properly.
Use the neural networks mentioned in Kingma's as encoder/decoder.
use calculate_robustness as a starting point. Generalize this by making -2/3 a variable called riskBias. Call this calculate_generalizedmean
Then compute a particular riskBias using the formula riskBias = - 2 coupling_loss/(1 + dim * coupling_loss). Compute the generalized mean with this particular riskBias and call this calculate_coupling_loss_bias
I have enhanced the VAE model in vae_v4.1b_wip.ipynb. v4.1b is a branch of v4 and is independent from v4.1a. The additional features that you can see here are the following:
Please do the following:
Play around with the BATCH_SIZE
, n_epoch
, n_sample
, and z_dim
hyperparameters. See how when you change them can affect ELBO/loss value per epoch as well as the runtime length. One thing to note is that n_sample is bounded by the BATCH_SIZE so n_sample cannot be greater than the batch.
Store the kl_div
and neg_ll
values into the self.metrics
default_dict, just like what I did for loss
and elbo
. You may need to find a way to output these values together in the compute_loss()
, for example, concatenating these values as a Tensor vector.
Plot the kl_div
and neg_ll
values at the end of the notebook, just like what is done for loss
and elbo
.
I have marked the above 3 points in the vae_v4.1b_wip for easy following.
Running the following code:
datasets, datasets_info = tfds.load(name='cifar10_corrupted',
with_info=True,
as_supervised=False
)
print(datasets)
It only contains the test datasets, no train.
{'test': <PrefetchDataset shapes: {image: (32, 32, 3), label: ()}, types: {image: tf.uint8, label: tf.int64}>}
Other datasets, such as mnist_corrupted, would have contained by train and test.
datasets, datasets_info = tfds.load(name='mnist_corrupted',
with_info=True,
as_supervised=False
)
print(datasets)
{'test': <PrefetchDataset shapes: {image: (28, 28, 1), label: ()}, types: {image: tf.uint8, label: tf.int64}>,
'train': <PrefetchDataset shapes: {image: (28, 28, 1), label: ()}, types: {image: tf.uint8, label: tf.int64}>}
Review the following VAE metrics:
Solution for 1-D case:
https://mi-cs.slack.com/files/U019H5B0PED/F02CF02G23T/image.png
Nelson needs to extend this to multidimensions than it code be added to the Python code base.
Run the VAE and compare the -ELBO score between beta-VAE with beta > 1 and a regular VAE with Sigma prior > 1. See whether there are any empirical relationships between those two.
There is a need to expand the analysis shown in Fig 5 & 8 of the Coupled VAE paper. The Coupled VAE paper provided a graphic of the reconstruction loss by plotting a histogram of the loss per image and overlaying this with the Accuracy, Robustness, and Decisiveness geometric mean metrics. The analysis needs to add a plot for the latent layer divergence and to combine the divergence & reconstruction into the probabilities which represent ELBO metric.
The design for this analysis is being developed in the Mathematica file Generalized_ELBO.nb. There are three components:
I have looked at the VAE architecture from Keras here, and propose to make the following changes to VAE v.4.1:
train_step()
compute_loss()
log_normal_pdf()
generate_and_save_images()
.fit()
to train VAEModel instead of .train()
.The benefits of this approach is simplication of the code by not having use a wrapper class. This also aligns with how Keras model usually trains the model by using .fit()
.
If this is the way to go, I have created a new VAE version vae_v4.1a_wip.ipynb, but has encounter a handful of errors:
latent_dim = 2
encoder_inputs = keras.Input(shape=(28, 28, 1))
x = layers.Conv2D(32, 3, activation="relu", strides=2, padding="same")(encoder_inputs)
x = layers.Conv2D(64, 3, activation="relu", strides=2, padding="same")(x)
x = layers.Flatten()(x)
x = layers.Dense(16, activation="relu")(x)
z_mean = layers.Dense(latent_dim, name="z_mean")(x)
z_log_var = layers.Dense(latent_dim, name="z_log_var")(x)
z = Sampling()([z_mean, z_log_var])
encoder = keras.Model(encoder_inputs, [z_mean, z_log_var, z], name="encoder")
encoder.summary()
tensorflow_datasets
object types into VAEModel to fit. For some reason, in the Keras tutorial, they had to concatenate x_train
and x_test
into mnist_digits
and then fit just mnist_digits
. They can do that because x_train
and x_test
are numpy arrays. In our case, we currently pass in fm_train_dataset
and fm_train_dataset
separately. Do we have to mimic what keras is doing here. Would tensorflow.python.data.ops.dataset_ops.ShuffleDataset
even work here?(x_train, _), (x_test, _) = keras.datasets.mnist.load_data()
mnist_digits = np.concatenate([x_train, x_test], axis=0)
mnist_digits = np.expand_dims(mnist_digits, -1).astype("float32") / 255
vae = VAE(encoder, decoder)
vae.compile(optimizer=keras.optimizers.Adam())
vae.fit(mnist_digits, epochs=30, batch_size=128)
Here is the link to keras Model class.
Ability for it to train without the .train()
from the wrapper class. Do we simply move those contents into the .train_step()
function in VAEModel? How about the test_step()
or fit()
function? Do we need to override them in from tfk.Model?
Where to put the generate_and_save_images()
? Put that into the .train_step()
so that there can be plots for every train step? Also, generate_and_save_images()
inputs only test images while self.compute_loss()
inputs only train images. So it may be problematic if we have the concatenate train and test images together as mentioned in 2.
Can you take a look at the following?
Thanks!
Hi Kevin,
Can you confirm the outputs (the ones shown at least) are consistent with the outputs for the original v3 file?
Thanks,
John
Dependency conflicts can arise when running the command pip install -i https://test.pypi.org/simple/ coupledvae==0.0.15
. scikit-learn
seems to be the major culprit.
INFO: pip is looking at multiple versions of coupledvae to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install coupledvae because these package versions have conflicting dependencies.
scikit-learn 1.2.1 depends on threadpoolctl>=2.0.0
scikit-learn 1.2.0 depends on threadpoolctl>=2.0.0
scikit-learn 1.1.3 depends on threadpoolctl>=2.0.0
scikit-learn 1.1.2 depends on threadpoolctl>=2.0.0
scikit-learn 1.1.1 depends on threadpoolctl>=2.0.0
scikit-learn 1.1.0 depends on threadpoolctl>=2.0.0
scikit-learn 1.0.2 depends on threadpoolctl>=2.0.0
scikit-learn 1.0.1 depends on threadpoolctl>=2.0.0
scikit-learn 1.0 depends on threadpoolctl>=2.0.0
scikit-learn 0.24.2 depends on joblib>=0.11
scikit-learn 0.24.1 depends on joblib>=0.11
scikit-learn 0.24.0 depends on joblib>=0.11
scikit-learn 0.23.2 depends on joblib>=0.11
scikit-learn 0.23.1 depends on joblib>=0.11
scikit-learn 0.23.0 depends on joblib>=0.11
scikit-learn 0.22 depends on joblib>=0.11
scikit-learn 0.21.0 depends on scipy>=0.17.0
scikit-learn 0.20.3 depends on scipy>=0.13.3```
This can be circumvented by installing `nsc` first, but I'm raising this as an issue because we'll need to come up with a better solution in the long term.
Example methods include dimensionality reduction such as T-SNE.
The n_epoch_display argument for the train_VAEs function in the notebook doesn't actually do anything. While it gets passed to other functions, it looks like the VAEMNIST and VAECIFAR modules don't actually use the variable.
Hi Kevin,
Can you ensure the outputs for this file look correct for the MNIST data?
https://colab.research.google.com/drive/138BvRpUsZpB0ofl5OOQIAYVdiSBxCriW#scrollTo=8soTkUGEt0xN
coupledvae_v3.1: Comment the notebook using terms from PDL - TF2 Coursera course.
coupledvae_v4.x: Rearchitect existing Coupled VAE model with the contents from the PDL - TF2 Coursera course. Including:
coupledvae_v5.x: Integrate nsc lib.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.