xingqian2018 / sd-webui-prompt-free-diffusion Goto Github PK

View Code? Open in Web Editor NEW

50.0 3.0 0.0 3.31 MB

Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models

Home Page: https://arxiv.org/abs/2305.16223

License: MIT License

Python 100.00%

sd-webui-prompt-free-diffusion's Introduction

Prompt-Free Diffusion for Stable Diffusion WebUI

This is the WebUI extension to inject Prompt-Free Diffusion into Stable Diffusion WebUI.

News

[2023.06.20]: Repo created

Instruction

This extension is for AUTOMATIC1111's Stable Diffusion web UI, allows the Web UI to add Prompt-Free Diffusion to the original Stable Diffusion model to generate images. No training required.

This repo also borrows UI designs and code structures from SDWebUI ContorlNet.

Installation

Open "Extensions" tab.
Open "Install from URL" tab in the tab.
Enter https://github.com/xingqian2018/sd-webui-prompt-free-diffusion.git to "URL for extension's git repository".
Press "Install" button.
Wait for 5 seconds, and you will see the message "Installed into stable-diffusion-webui\extensions\sd-webui-prompt-free-diffusion. Use Installed tab to restart".
Go to "Installed" tab, click "Check for updates", and then click "Apply and restart UI". (The next time you can also use these buttons to update Prompt-Free Diffusion.)
Completely restart A1111 webui including your terminal.
Download models.
After you put models in the correct folder, you may need to refresh to see the models. The refresh button is right to your "SeeCoder" dropdown.

Download Models

Download the models from Prompt-Free Diffusion mode repo: https://huggingface.co/shi-labs/prompt-free-diffusion/tree/main/pretrained/pfd/seecoder. Put it under extentions/sd-webui-prompt-free-diffusion/models/. You only need to download SeeCoder.

sd-webui-prompt-free-diffusion's People

Contributors

Stargazers

Watchers

sd-webui-prompt-free-diffusion's Issues

Error generating images

Traceback (most recent call last):
  File "E:\stable-diffusion-webui\modules\call_queue.py", line 55, in f
    res = list(func(*args, **kwargs))
  File "E:\stable-diffusion-webui\modules\call_queue.py", line 35, in f
    res = func(*args, **kwargs)
  File "E:\stable-diffusion-webui\modules\txt2img.py", line 57, in txt2img
    processed = processing.process_images(p)
  File "E:\stable-diffusion-webui\modules\processing.py", line 620, in process_images
    res = process_images_inner(p)
  File "E:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
    return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  File "E:\stable-diffusion-webui\modules\processing.py", line 729, in process_images_inner
    p.setup_conds()
  File "E:\stable-diffusion-webui\extensions\sd-webui-prompt-free-diffusion\scripts\pfd.py", line 371, in setup_conds
    self.uc = self.get_conds_with_caching(
TypeError: StableDiffusionProcessing.get_conds_with_caching() missing 1 required positional argument: 'extra_network_data'

Windows, latest A1111, model placed in folder, settings as below

The extension does not seem to work well

Hi, I ran some tests with the extension after downloading all 3 seecoder models. I am unable to get any good results with any of them, no matter what setting I use. The webui is only able to generate something intelligible when the extension dropdown is disabled.

Example settings

Input image

When I restart the webui and load a previously generated image from the PNG Info tab, then press generate, I get a different scrambled mess every time for the first generation. Then, all follow-up generations (without restarting the webui) from the 2nd generation onward are all the same:

Sampling result

(note that this image is not the same as the first generation that you can find in the settings image above. the extension consistently generates this image after the first generation)

Logs

Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Version: v1.0.0-pre-1578-g394ffa7b
Commit hash: 394ffa7b0a7fff3ec484bcd084e673a8b301ccc8
Installing requirements
Launching Web UI with arguments: --api --api-log --allow-code --ckpt-dir E:\sd\models\Stable-diffusion --embeddings-dir E:\sd\models\embeddings --vae-dir E:\sd\models\VAE --lora-dir E:\sd\models\Lora --esrgan-models-path E:\sd\models\ESRGAN --realesrgan-models-path E:\sd\models\RealESRGAN --controlnet-dir E:\sd\models\ControlNet --controlnet-annotator-models-path E:\sd\models
D:\src\stable-diffusion-webui\venv\lib\site-packages\pkg_resources\__init__.py:123: PkgResourcesDeprecationWarning: rch is an invalid version and will not be supported in a future release
  warnings.warn(
No module 'xformers'. Proceeding without it.
2023-07-02 22:21:16,929 - PFD - INFO - Prompt-Free-Diffusion v1.0.1
2023-07-02 22:21:17,605 - PFD - INFO - Prompt-Free-Diffusion v1.0.1
Loading weights [de329f31ad] from E:\sd\models\Stable-diffusion\silica\models\original\Silicon29-dark.safetensors
Running on local URL:  http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
Startup time: 8.6s (import torch: 2.1s, import gradio: 1.2s, import ldm: 0.6s, other imports: 1.0s, setup codeformer: 0.1s, list SD models: 0.5s, load scripts: 2.0s, create ui: 0.6s, gradio launch: 0.3s, add APIs: 0.2s).
Creating model from config: D:\src\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading VAE weights specified in settings: E:\sd\models\VAE\kl-f8-anime2.vae.pt
Applying attention optimization: Doggettx... done.
Model loaded in 6.8s (load weights from disk: 1.1s, create model: 0.7s, apply weights to model: 0.6s, apply half(): 0.9s, load VAE: 0.6s, move model to device: 0.9s, load textual inversion embeddings: 1.9s, calculate empty prompt: 0.1s).
2023-07-02 22:21:28,903 - PFD - INFO - Loading model: seecoder-v1-0 [936e345d]
2023-07-02 22:21:28,956 - PFD - INFO - Loaded state_dict from [E:\sd\models\SeeCoder\seecoder-v1-0.safetensors]
2023-07-02 22:21:28,956 - PFD - INFO - Loading config: D:\src\stable-diffusion-webui\extensions\sd-webui-prompt-free-diffusion\models\seecoder-v1-0.yaml
preload_extensions_git_metadata for 86 extensions took 9.69s
2023-07-02 22:21:30,781 - PFD - INFO - seecoder-v1-0 [936e345d] loaded.
  3%|â–Ž         | 1/30 [00:00<00:21,  1.35it/s]
Total progress:   0%|          | 0/30 [00:00<?, ?it/s]
 13%|â–ˆâ–Ž        | 4/30 [00:01<00:06,  4.25it/s]
 20%|â–ˆâ–ˆ        | 6/30 [00:01<00:04,  5.88it/s]
Total progress:  20%|â–ˆâ–ˆ        | 6/30 [00:00<00:02,  8.00it/s]
 27%|â–ˆâ–ˆâ–‹       | 8/30 [00:01<00:03,  6.70it/s]
Total progress:  27%|â–ˆâ–ˆâ–‹       | 8/30 [00:00<00:02,  7.80it/s]
 33%|â–ˆâ–ˆâ–ˆâ–Ž      | 10/30 [00:01<00:02,  7.21it/s]
Total progress:  33%|â–ˆâ–ˆâ–ˆâ–Ž      | 10/30 [00:01<00:02,  7.78it/s]
 40%|â–ˆâ–ˆâ–ˆâ–ˆ      | 12/30 [00:02<00:02,  7.23it/s]
Total progress:  40%|â–ˆâ–ˆâ–ˆâ–ˆ      | 12/30 [00:01<00:02,  7.52it/s]
 47%|â–ˆâ–ˆâ–ˆâ–ˆâ–‹     | 14/30 [00:02<00:02,  7.67it/s]
Total progress:  47%|â–ˆâ–ˆâ–ˆâ–ˆâ–‹     | 14/30 [00:01<00:02,  7.80it/s]
 53%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–Ž    | 16/30 [00:02<00:01,  7.59it/s]
Total progress:  53%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–Ž    | 16/30 [00:02<00:01,  7.68it/s]
 60%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ    | 18/30 [00:03<00:01,  7.57it/s]
Total progress:  60%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ    | 18/30 [00:02<00:01,  7.59it/s]
 67%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–‹   | 20/30 [00:03<00:01,  7.14it/s]
Total progress:  67%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–‹   | 20/30 [00:02<00:01,  7.17it/s]
 73%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–Ž  | 22/30 [00:03<00:01,  7.37it/s]
Total progress:  73%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–Ž  | 22/30 [00:02<00:01,  7.38it/s]
 80%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ  | 24/30 [00:03<00:00,  7.36it/s]
Total progress:  80%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ  | 24/30 [00:03<00:00,  7.35it/s]
 87%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–‹ | 26/30 [00:04<00:00,  7.28it/s]
Total progress:  87%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–‹ | 26/30 [00:03<00:00,  7.29it/s]
 93%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–Ž| 28/30 [00:04<00:00,  7.00it/s]
Total progress:  93%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–Ž| 28/30 [00:03<00:00,  7.00it/s]
100%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ| 30/30 [00:04<00:00,  6.40it/s]
Total progress: 100%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ| 30/30 [00:03<00:00,  7.43it/s]
Total progress: 100%|â–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆâ–ˆ| 30/30 [00:04<00:00,  6.94it/s]
{"prompt": "", "all_prompts": [""], "negative_prompt": "", "all_negative_prompts": [""], "seed": 1860717067, "all_seeds": [1860717067], "subseed": 409547144, "all_subseeds": [409547144], "subseed_strength": 0, "width": 512, "height": 512, "sampler_name": "DPM++ 2M Karras", "cfg_scale": 2, "steps": 30, "batch_size": 1, "restore_faces": false, "face_restoration_model": null, "sd_model_hash": "de329f31ad", "seed_resize_from_w": 0, "seed_resize_from_h": 0, "denoising_strength": null, "extra_generation_params": {"ControlNet": "model: seecoder-v1-0 [936e345d], weight: 1"}, "index_of_first_image": 0, "infotexts": ["Steps: 30, Sampler: DPM++ 2M Karras, CFG scale: 2, Seed: 1860717067, Size: 512x512, Model hash: de329f31ad, Model: silica_models_original_Silicon29-dark, ControlNet: \"model: seecoder-v1-0 [936e345d], weight: 1\", Version: v1.0.0-pre-1578-g394ffa7b"], "styles": [], "job_timestamp": "20230702222128", "clip_skip": 1, "is_using_inpainting_conditioning": false}

Is there any other piece of information I can provide? I want to believe this is a problem with the extension code, and not with the technique described in the paper.

Note that despite my logs saying I am on Version: v1.0.0-pre-1578-g394ffa7b, I pulled the latest version of the webui (v1.4.0) before running these tests. I have disabled every other extension (apart from the builtin ones) and completely restarted the webui.

No effect on output beyond slight noise.

It doesn't seem to actually do anything.

Enabled, with no text prompt, two negative embeddings, and a controlnet pose:

Not enabled, with no text prompt, same two negative embeddings, same controlnet pose:

AttributeError: 'StableDiffusionProcessingTxt2Img' object has no attribute 'setup_conds'

could you please tell me how to fix it ?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.