Git Product home page Git Product logo

hands-on-image-generation-with-tensorflow-2.0's Introduction

Viewing the Notebooks

The Jupyter notebooks contain large images therefore they may not load properly on Github page. Therefore, it is best to Git clone to view in your local machine.

Hands-On Image Generation with TensorFlow

Hands-On Image Generation with TensorFlow

This is the code repository for Hands-On Image Generation with TensorFlow : A practical guide to generating images and videos using deep learning ,published by Packt.

as recommended by Francois Chollet, Google AI, Creator of Keras

"All TensorFlow/Keras, with very readable code examples. Includes a section on StyleGAN, which will come in handy ... it's well explained."

What is this book about?

The emerging field of Generative Adversarial Networks (GANs) has made it possible to generate indistinguishable images from existing datasets. With this hands-on book, you’ll not only develop image generation skills but also gain a solid understanding of the underlying principles.

Starting with an introduction to the fundamentals of image generation using TensorFlow, this book covers Variational Autoencoders (VAEs) and GANs. You’ll discover how to build models for different applications as you get to grips with performing face swaps using deepfakes, neural style transfer, image-to-image translation, turning simple images into photorealistic images, and much more. You’ll also understand how and why to construct state-of-the-art deep neural networks using advanced techniques such as spectral normalization and self-attention layer before working with advanced models for face generation and editing. You'll also be introduced to photo restoration, text-to-image synthesis, video retargeting, and neural rendering. Throughout the book, you’ll learn to implement models from scratch in TensorFlow 2.x, including PixelCNN, VAE, DCGAN, WGAN, pix2pix, CycleGAN, StyleGAN, GauGAN, and BigGAN.

By the end of this book, you'll be well versed in TensorFlow and be able to implement image generative technologies confidently.

This book covers the following exciting features:

  • Train on face datasets and use them to explore latent spaces for editing new faces
  • Get to grips with swapping faces with deepfakes
  • Perform style transfer to convert a photo into a painting
  • Build and train pix2pix, CycleGAN, and BicycleGAN for image-to-image translation
  • Use iGAN to understand manifold interpolation and GauGAN to turn simple images into photorealistic images
  • Become well versed in attention generative models such as SAGAN and BigGAN
  • Generate high-resolution photos with Progressive GAN and StyleGAN

You can also checkout on my blog for detailed book overview

The images above were generated using StyleGAN code in the book. You too can do it using the Jupyter notebook and pretrained model provided.

If you feel this book is for you, get your copy today!

https://www.packtpub.com/

Citation

Please use this bibtex for citation

@book{cheong2020hands,
  title={Hands-On Image Generation with TensorFlow: A practical guide to generating images and videos using deep learning},
  author={Cheong, S.Y.},
  isbn={9781838826789},
  url={https://books.google.co.uk/books?id=tGcREAAAQBAJ},
  year={2020},
  publisher={Packt Publishing}
}

Table of Content

  1. Getting started with Image generation with TensorFlow
  2. Variational Autoencoder
  3. Generative Adversarial Network
  4. Image-to-Image Translation
  5. Style Transfer
  6. AI Painter
  7. High Fidelity Face Generation
  8. Self-Attention for Image Generation
  9. Video Synthesis 10 .Road Ahead

Instructions and Navigations

The code in this book uses tensorflow_gpu==2.2.0, and is compatitible with version up to 2.3.0

The code will look like the following:

content_image = self.preprocess(content_image_input)
style_image = self.preprocess(style_image_input)
self.content_target = self.encoder(content_image)
self.style_target = self.encoder(style_image)
adain_output = AdaIN()([self.content_target[-1], self.style_target[-1]])
self.stylized_image = self.postprocess(self.decoder(adain_output))
self.stn = Model([content_image_input, style_image_input], self.stylized_image)

Create virtual environment

python3 -m venv ./venv/imgentf2

Source virtual environment

source ./venv/imgentf2/bin/activate

Install dependencies

pip3 install --upgrade pip
pip3 install -r requirements.txt

Add virtual environment into Jupyter Notebook

python -m ipykernel install --user --name=imgentf2

Enable Jupyter Notebook extension

jupyter nbextension enable --py widgetsnbextension

Following is what you need for this book: This book is a step-by-step guide to show you how to implement generative models in TensorFlow 2.x from scratch. You’ll get to grips with the image generative technology by covering autoencoders, style transfer, and GANs as well as fundamental and state-of-the-art models. You should have basic knowledge of deep learning training pipelines, such as training convolutional neural networks for image classification. This book will mainly use highlevel Keras APIs in TensorFlow 2, which is easy to learn. Should you need to refresh or learn TensorFlow 2, there are many free tutorials available online, such as the one on the official TensorFlow website, https://www.tensorflow.org/tutorials/keras/classification.

With the following software and hardware list you can run all code files present in the book (Chapter 1-10).

Software and Hardware List

Chapter Software required OS required
1 - 10 TensorFlow 2.2, GPU with minimum 4 GB memory Windows, Mac OS X, and Linux (Any)

Training deep neural networks is computationally intensive. You can train the first few simple models using the CPU only. However, as we progress to more complex models and datasets in later chapters, the model training could take a few days before you start to see satisfactory results. To get the most out of this book, you should have access to the GPU to accelerate the model training time. There are also free cloud services, such as Google's Colab, that provide GPUs on which you can upload and run the code.

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. Click here to download it.

Related products

Get to Know the Author

Soon Yau Cheong is an AI consultant and the founder of Sooner.ai Ltd. With a history of being associated with industry giants such as NVIDIA and Qualcomm, he provides consultation in the various domains of AI, such as deep learning, computer vision, natural language processing, and big data analytics. He was awarded a full scholarship to study for his PhD at the University of Bristol while working as a teaching assistant. He is also a mentor for AI courses with Udacity.

Note from the Author

I hope you will enjoy reading my book. If you do, I would be most grateful if you could write a book review on Amazon.

Please feel free to connect with me on www.linkedin.com/in/soonyau. Please add a note saying that you've read my book and I will gladly accept your connection request.

Download a free PDF

If you have already purchased a print or Kindle version of this book, you can get a DRM-free PDF version at no cost.
Simply click on the link to claim your free PDF.

https://packt.link/free-ebook/9781838826789

hands-on-image-generation-with-tensorflow-2.0's People

Contributors

manikandankurup-packt avatar packt-itservice avatar packtutkarshr avatar roshank10 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hands-on-image-generation-with-tensorflow-2.0's Issues

Stylegan: InvalidArgumentError

When you train the Stylegan with a Resolution of 1024, you get an error when the model expands to 1024:

`InvalidArgumentError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs, op_def)
1653 try:
-> 1654 c_op = pywrap_tf_session.TF_FinishOperation(op_desc)
1655 except errors.InvalidArgumentError as e:

InvalidArgumentError: slice index 8 of dimension 1 out of bounds. for '{{node strided_slice_8}} = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=1, ellipsis_mask=0, end_mask=1, new_axis_mask=0, shrink_axis_mask=2](input_2, strided_slice_8/begin, strided_slice_8/end, strided_slice_8/strides)' with input shapes: [?,8,512], [2], [2], [2] and with computed input tensors: input[1] = <0 8>, input[2] = <0 9>, input[3] = <1 1>.

in grow_generator(self, log2_res)
268
269 x = self.generator_blocks[log2_res]([x,
--> 270 w[:,log2_res-2],
271 self.noise_inputs[log2_res]])
272 print(f"grow_generator end, log2={log2_res} w={log2_res-2}, noise={log2_res}", )

`

Should you do log2_res - 1 instead of - 2?

Error in cyclegan identity loss!

From the paper:
" regularize the generator to be near an identity mapping when real samples
of the target domain are provided as the input to the generator:
i.e., Lidentity(G,F) = Ey∼pdata (y)[‖G(y) − y‖1] + Ex∼pdata (x)[‖F(x) −x‖1] "

The generator_AB is supposed to be fed with the target image - image_B, and the output is compared with image_B.
also, generator_BA is supposed to be fed with image_A.

Chapter 1 - ch1_generate_first_image - Loading images

When I attempt to run the line - ds_train, ds_info = tfds.load('celeb_a', split='test', shuffle_files=False, with_info=True)

I get the following error:

"NonMatchingChecksumError Traceback (most recent call last)

NonMatchingChecksumError: Artifact https://drive.google.com/uc?export=download&id=0B7EVK8r0v71pZjFTYXZWM3FlRnM, downloaded to C:...\ucexport_download_id_0B7EVK8r0v71pZjFTYXZWM3FlDDaXUAQO8EGH_a7VqGNLRtW52mva1LzDrb-V723OQN8.tmp.521a68dd559d4cd49b06809e3e944c38\uc, has wrong checksum. Expected: UrlInfo(size=1.34 GiB, checksum='46fb89443c578308acf364d7d379fe1b9efb793042c0af734b6112e4fd3a8c74', filename='img_align_celeba.zip'). Got: UrlInfo(size=3.02 KiB, checksum='7871bf45999c9dfc0e6c0ebc60f44e77e6add3cdc9eee3072489754587b2a6f6', filename='uc').To debug, see: https://www.tensorflow.org/datasets/overview#fixing_nonmatchingchecksumerror"

Celeb-A dataset not working in tensorflow datasets

You may already know about this and have updated some code snippets, in which case I am very sorry.
The Celeb-A dataset used in a couple chapters no longer works. The problem on stack overflow and GitHub don't have any solutions to this problem, and say that the dataset has to be downloaded manually. The following are the links to those problems respectively
stack overflow
github
The following is the result in google colab:
Screenshot (106)
Do you know any simple solutions to this issue?

Error in loss calculation in pix2pix example!

When calculating the MAE loss of the generator, the loss is supposed to be between the target image and the generated image,
But in your example, the MAE is between the output of the discriminator and real_labels (tf.ones)

Im very sorry for all the issues, found a great solution

replace the download of the celeb-a datast with:

import os
import tempfile
import urllib
gcs_base_dir = "gs://celeb_a_dataset/"
celeb_a_builder = tfds.builder("celeb_a", data_dir=gcs_base_dir, version='2.0.0')

celeb_a_builder.download_and_prepare()

num_test_shards_dict = {'0.3.0': 4, '2.0.0': 2} # Used because we download the test dataset separately
version = str(celeb_a_builder.info.version)
print('Celeb_A dataset version: %s' % version)

local_root = tempfile.mkdtemp(prefix='test-data')
def local_test_filename_base():
return local_root

def local_test_file_full_prefix():
return os.path.join(local_test_filename_base(), "celeb_a-test.tfrecord")

def copy_test_files_to_local():
filename_base = local_test_file_full_prefix()
num_test_shards = num_test_shards_dict[version]
for shard in range(num_test_shards):
url = "https://storage.googleapis.com/celeb_a_dataset/celeb_a/%s/celeb_a-test.tfrecord-0000%s-of-0000%s" % (version, shard, num_test_shards)
filename = "%s-0000%s-of-0000%s" % (filename_base, shard, num_test_shards)
res = urllib.request.urlretrieve(url, filename)

celeb_a_test_data = celeb_a_builder.as_dataset(split='test')

Copy test data locally to be able to read it into tfma

copy_test_files_to_local()

dropout at test time in pix2pix example

In the pix2pix paper they mention using dropout to add randomness and use using the normalization using the statistics of
the test batch, rather than aggregated statistics of the train-set.
But in your code you use
self.generator.predict
dropout is turned off, and normalization uses aggregated stasitics

Error in cyclegan train_discriminator

In train_discriminator, there is a confusion with the input.
For example:
fake_images = generator.predict(real_images_A)
The generator always gets real_images_A even if its a generator_B which is supposed to get real_images_B
Also later:
pred_real = discriminator(real_images_B)
but if a discriminator_A its supposed to get real_images_A

Also, like in #20 the generator is supposed to be called with training=True

Restart training Attribute Error

If you want to retrain the StyleGAN model with saved weights like this

gan = StyleGAN(resolution=IMAGE_RESOLUTION, load_path='./checkpoints/stylegan-29042021', start_log2_res=10)

You get following output:

Growing model to 1024x1024
grow_generator mid, log2=3 w=1, noise=3
grow_generator mid, log2=4 w=2, noise=4
grow_generator mid, log2=5 w=3, noise=5
grow_generator mid, log2=6 w=4, noise=6
grow_generator mid, log2=7 w=5, noise=7
grow_generator mid, log2=8 w=6, noise=8
grow_generator mid, log2=9 w=7, noise=9
grow_generator end, log2=10 w=8, noise=10


AttributeError Traceback (most recent call last)
in
----> 1 gan = StyleGAN(resolution=IMAGE_RESOLUTION, load_path='./checkpoints/stylegan-29042021', start_log2_res=10)

in init(self, z_dim, resolution, load_path, start_log2_res)
28
29 if load_path:
---> 30 self.load_checkpoint(load_path)
31
32 # initialize generator with the base

in load_checkpoint(self, ckpt_path)
77 def load_checkpoint(self, ckpt_path):
78 self.grow_model(self.start_log2_res)
---> 79 self.ckpt.restore(ckpt_path)
80
81

AttributeError: 'StyleGAN' object has no attribute 'ckpt'

Pix2pix

Hello - when I try to run pix2pix, it gives error " ValueError: Layer model_1 expects 1 input(s), but it received 2 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(1, 256, 256, 3) dtype=float32>, <tf.Tensor 'IteratorGetNext:1' shape=(1, 256, 256, 3) dtype=float32>]"

Error in StyleGAN and Faster Stylegan

Hi! I'm learning stylegans and I have been investigating different books and codes implementing this type of GAN. I tried to replicate your code (from your book and this repository), but both scripts have different errors:

  • StyleGAN: Data cardinality is ambiguous:
    x sizes: 16, 16, 1, 16, 16, 16, 16, 16, 16, 16
    y sizes: 16
    Make sure all arrays contain the same number of samples
    -> This issue runs the first set of images, but in the next training it crushes with this error. Tried to checked the sizes, but haven't found anything.

  • Faster StyleGAN: len is not well defined for symbolic Tensors
    -> This code doesn't even begin the training. Different searches show that the solution is downgrading tensorflow to 1.5 (currently i'm working 2.5 in google colab), but it doesn't fix the problem.

I would like to know if you had this issues as well.
Thanks for your great work.

[ch4_pix2pix] Layer model_1 expects 1 input(s)

`ValueError Traceback (most recent call last)
/tmp/ipykernel_676362/329867214.py in
----> 1 pix2pix.train(iter(train_dataset), iter(test_dataset), BATCH_SIZE, 20000, 1000)

/tmp/ipykernel_676362/2401627389.py in train(self, data_generator, test_data_generator, batch_size, steps, interval)
146
147 # train generator
--> 148 g_loss = self.model.train_on_batch([real_images_A, real_images_B], real_labels)
149 if i%interval == 0:
150 msg = "Step {}: discriminator_loss {:.4f} g_loss {:.4f}"\

~/anaconda3/envs/gan/lib/python3.8/site-packages/keras/engine/training.py in train_on_batch(self, x, y, sample_weight, class_weight, reset_metrics, return_dict)
1854 class_weight)
1855 self.train_function = self.make_train_function()
-> 1856 logs = self.train_function(iterator)
1857
1858 if reset_metrics:

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in call(self, *args, **kwds)
883
884 with OptionalXlaContext(self._jit_compile):
--> 885 result = self._call(*args, **kwds)
886
887 new_tracing_count = self.experimental_get_tracing_count()

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
931 # This is the first call of call, so we have to initialize.
932 initializers = []
--> 933 self._initialize(args, kwds, add_initializers_to=initializers)
934 finally:
935 # At this point we know that the initialization is complete (or less

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
757 self._graph_deleter = FunctionDeleter(self._lifted_initializer_graph)
758 self._concrete_stateful_fn = (
--> 759 self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
760 *args, **kwds))
761

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
3064 args, kwargs = None, None
3065 with self._lock:
-> 3066 graph_function, _ = self._maybe_define_function(args, kwargs)
3067 return graph_function
3068

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _maybe_define_function(self, args, kwargs)
3461
3462 self._function_cache.missed.add(call_context_key)
-> 3463 graph_function = self._create_graph_function(args, kwargs)
3464 self._function_cache.primary[cache_key] = graph_function
3465

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
3296 arg_names = base_arg_names + missing_arg_names
3297 graph_function = ConcreteFunction(
-> 3298 func_graph_module.func_graph_from_py_func(
3299 self._name,
3300 self._python_function,

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes, acd_record_initial_resource_uses)
1005 _, original_func = tf_decorator.unwrap(python_func)
1006
-> 1007 func_outputs = python_func(*func_args, **func_kwargs)
1008
1009 # invariant: func_outputs contains only Tensors, CompositeTensors,

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in wrapped_fn(*args, **kwds)
666 # the function a weak reference to itself to avoid a reference cycle.
667 with OptionalXlaContext(compile_with_xla):
--> 668 out = weak_wrapped_fn().wrapped(*args, **kwds)
669 return out
670

~/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
992 except Exception as e: # pylint:disable=broad-except
993 if hasattr(e, "ag_error_metadata"):
--> 994 raise e.ag_error_metadata.to_exception(e)
995 else:
996 raise

ValueError: in user code:

/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/keras/engine/training.py:853 train_function  *
    return step_function(self, iterator)
/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/keras/engine/training.py:842 step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:1286 run
    return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2849 call_for_each_replica
    return self._call_for_each_replica(fn, args, kwargs)
/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:3632 _call_for_each_replica
    return fn(*args, **kwargs)
/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/keras/engine/training.py:835 run_step  **
    outputs = model.train_step(data)
/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/keras/engine/training.py:787 train_step
    y_pred = self(x, training=True)
/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/keras/engine/base_layer.py:1020 __call__
    input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
/home/roastedkernel/anaconda3/envs/gan/lib/python3.8/site-packages/keras/engine/input_spec.py:199 assert_input_compatibility
    raise ValueError('Layer ' + layer_name + ' expects ' +

ValueError: Layer model_1 expects 1 input(s), but it received 2 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(4, 256, 256, 3) dtype=float32>, <tf.Tensor 'IteratorGetNext:1' shape=(4, 256, 256, 3) dtype=float32>]`

Code should specify which version of gdown

As of March 2021, the following problems occur:
When running the code in the first notebook: ch1_generate_first_image.ipynb, the code throws a FileNotFound exception. (It was run in google colab). Here is an image:
Screenshot (107)

For readers reading after that make sure the gdown version is 3.12.2. In google colab it defaults to an older version. This can be run in jupyter notebook with the following command:
!pip install gdown==3.12.2

Multi GPU strategy

Is there are reason why you never used a strategy (like MirrorStrategy, etc.) despite of having 4 GPUs?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.