Stable Diffusion in Tensorflow / Keras

Now supports Older Tensorflow Versions (< 2.8)

Original repo by Divam Gupta

Backward compatibility fixes by Yam Peleg

A Keras / Tensorflow implementation of Stable Diffusion Models for Image Generation. This is a fork of the original repo, that now supports past versions of Tensorflow and Keras. This implementation is easy to understand and have a minimal code footprint. The weights had been converted from the original Pytorch implementation, and are available for download. (Automaticly downloaded when running the code)

Usage

Command Line

python text2image.py --prompt = "An astronaut riding a horse."

Python

from stable_diffusion_tf.stable_diffusion import get_model, text2image
text_encoder, diffusion_model, decoder = get_model(512, 512, download_weights=True)

img = text2image("An astronaut riding a hourse" , 
	img_height=512, 
	img_width=512,  
	text_encoder=text_encoder, 
	diffusion_model=diffusion_model, 
	decoder=decoder
)

cv2.imwrite("/tmp/a.png" , img[0][... , ::-1])

Example Outputs

The following outputs have been generated using the Tensorflow/Keras of stable diffusion:

1. An epic and beautiful rococo werewolf drinking coffee, in a burning coffee shop. ultra-detailed. anime, pixiv, uhd 8k cryengine, octane render

2. Spider-Gwen Gwen-Stacy Skyscraper Pink White Pink-White Spiderman Photo-realistic 4K

3. A vision of paradise, Unreal Engine

References:

• https://github.com/divamgupta/stable-diffusion-tensorflow
• https://github.com/CompVis/stable-diffusion
• https://github.com/geohot/tinygrad/blob/master/examples/stable_diffusion.py