12GB VRAM not enough? RTX 3060 - Linux Mint 21.1 about instruct-pix2pix HOT 27 CLOSED

More numbers for reference

1024x1024 = 11.6 GB

1280x1024 = 16.9 GB

1280x1280 = >23 GB (starts offloading to RAM at this point)

from instruct-pix2pix.

Hey, I checked the link you posted and realized that it is an install for NMKD SD GUI. Actually I tried that yesterday itself with their version. It does work. But I was actually looking for the dedicated GUI only for this instruct pix2pix.

Oh I see. So I guess all the people reporting to me on Reddit that it can work with less than 18GB of vram are using this other version, which is not quite what you and all of us here were expecting/hoping for. I guess we'll just have to be patient and wait for the optimizations.

Cheers!

from instruct-pix2pix.

@holynski

512x512 takes up 4.6 GB VRAM on my Windows 10 machine.

Using a very basic python script utilizing the Diffusers pipeline.
You can kinda reproduce this by dumping the Colab code into a py file (or running the notebook locally if you have Jupyter).

from instruct-pix2pix.

I think I know the answer to our problems. I chatted with a reddit user who got this working just fine on very tame hardware. He claimed he never installed the github version, but rather this specific version of it from here:

https://nmkd.itch.io/t2i-gui/devlog/480628/sd-gui-190-now-with-instructpix2pix

I hope it's OK I'm posting a link. Anyhow, I guess the nmkd version is slightly different than this Github version in some very important ways.

Thank you man. I will definitely check that version.

from instruct-pix2pix.

I think I know the answer to our problems. I chatted with a reddit user who got this working just fine on very tame hardware. He claimed he never installed the github version, but rather this specific version of it from here:

https://nmkd.itch.io/t2i-gui/devlog/480628/sd-gui-190-now-with-instructpix2pix

I hope it's OK I'm posting a link. Anyhow, I guess the nmkd version is slightly different than this Github version in some very important ways.

Yeah that's my GUI.

This is the code it uses: https://github.com/n00mkrad/stable-diffusion-cust/blob/eaeaf3e79dfac6e1b31748fa661a1175705278c2/sd_ip2p/ip2p_batch.py

It's really basic, it just loads a command/prompt list from JSON and then calls the Diffusers implementation in a loop.

from instruct-pix2pix.

Sorry for the rant but it feels like I'm talking to a wall here.

I barely understand what your entire post means (total noob). I don't know if that code you posted should be added to a python script and executed with python, if that code is meant to replace something else in the ip2p root, can it be run from my local machine? You know what I mean? To someone who doesn't understand how any of this magic works it's very difficult to know what to do with the advice.

What are you hoping for that my (aka literally the original) code does not do?

I can't speak for everyone else here, but what I was hoping for was to be able to follow the quickstart instructions, run python edit_app.py, go to localhost:7860 and use the webGUI to change around some images.

from instruct-pix2pix.

Sorry for the rant but it feels like I'm talking to a wall here.

I barely understand what your entire post means (total noob). I don't know if that code you posted should be added to a python script and executed with python, if that code is meant to replace something else in the ip2p root, can it be run from my local machine? You know what I mean? To someone who doesn't understand how any of this magic works it's very difficult to know what to do with the advice.

What are you hoping for that my (aka literally the original) code does not do?

I can't speak for everyone else here, but what I was hoping for was to be able to follow the quickstart instructions, run python edit_app.py, go to localhost:7860 and use the webGUI to change around some images.

"Isn't this place where we ask for help regarding an issue related to this particular repo rather" Everything you pointed out here is perfectly put. I totally relate to that. As an automatic1111 user I would love to see this implemented there or else run this as a standalone GUI. Hope people who rant understand this.

from instruct-pix2pix.

No it's my bad actually... I'm definitely too tired... I was not loading the ip2p model in A1111's UI but I was going with the default SD 1.5 😓 ... but in my defense I can say that since I had previously used this repo "download_checkpoints.sh" to download the model weights, I gave for granted they were downloaded to the usual "~/.cache/huggingface/" dir, so when I later switched to try A1111 extension and it didn't complain about missing the model during the installation or even after it I just automatically assumed it was picking it up from that dir... and then it went as it went... 😑🐸

Anyway, now that the mystery is solved I'll play around with it and see how fun it will get... 😎

from instruct-pix2pix.

I am getting the same error

from instruct-pix2pix.

I am getting the same error

What OS/GPU do you have?

from instruct-pix2pix.

So relieved to find that I am not the only one scratching my head over the exact error from 2-3hrs. Wish somebody had a solution.
My Build
Win10, 16GB RAM, GTX 970(4Gb)_

from instruct-pix2pix.

I am getting the same error

What OS/GPU do you have?

Windows 10/ GTX 3050 4GB

from instruct-pix2pix.

Same error on an 8GB card, Ubuntu.

from instruct-pix2pix.

Hey, we added this to the readme recently, but our current version of the code requires 18+ GB of GPU VRAM. If you want to run it on a smaller card, you'll likely have to use one of the alternative methods shown at the bottom of the page here: https://github.com/timothybrooks/instruct-pix2pix#other-ways-of-using-instructpix2pix

Closing for now, but feel free to comment further.

from instruct-pix2pix.

Hey, we added this to the readme recently, but our current version of the code requires 18+ GB of GPU VRAM. If you want to run it on a smaller card, you'll likely have to use one of the alternative methods shown at the bottom of the page here: https://github.com/timothybrooks/instruct-pix2pix#other-ways-of-using-instructpix2pix

Closing for now, but feel free to comment further.

Thank you for the clarification. I wasn't sure it actually required 18GB+.

from instruct-pix2pix.

There are probably some optimizations that could be made to our code to reduce memory usage, and we'd welcome anyone to submit a PR for that. In the meantime, the other links should work for smaller GPUs (and even CPUs!).

from instruct-pix2pix.

I am getting replies on reddit that people are able to use ip2p with much less vram than I have. I think there is something wrong with our installs. I am doing some tests now. Will report back with my findings.

from instruct-pix2pix.

512x512 takes up 4.6 GB VRAM on my Windows 10 machine.

Using a very basic python script utilizing the Diffusers pipeline.

Yup, definitely something wrong with our installs. I wonder if it's clashing with my other AI installations or we just straight up installed it incorrectly.

from instruct-pix2pix.

I am not sure what the issue is.

I re-installed Anaconda and all of ip2p on a different drive (same computer though) and I got the same problem. Seems like lots of people have got this working just fine. I am asking people how they went about the install. Hopefully we can figure out what all of us have in common that makes it not work for us.

from instruct-pix2pix.

Yes, I have the same issue when running in windows 10 on my RTX A4000 .
I tried for a few hours to solve the problem, but I am far from an expert. If someone could figure this problem out, I would be very appreciative :)

from instruct-pix2pix.

I think I know the answer to our problems. I chatted with a reddit user who got this working just fine on very tame hardware. He claimed he never installed the github version, but rather this specific version of it from here:

https://nmkd.itch.io/t2i-gui/devlog/480628/sd-gui-190-now-with-instructpix2pix

I hope it's OK I'm posting a link. Anyhow, I guess the nmkd version is slightly different than this Github version in some very important ways.

from instruct-pix2pix.

I think I know the answer to our problems. I chatted with a reddit user who got this working just fine on very tame hardware. He claimed he never installed the github version, but rather this specific version of it from here:
https://nmkd.itch.io/t2i-gui/devlog/480628/sd-gui-190-now-with-instructpix2pix
I hope it's OK I'm posting a link. Anyhow, I guess the nmkd version is slightly different than this Github version in some very important ways.

Thank you man. I will definitely check that version.

Report back, let me know if it works for you on a 4GB 970!

from instruct-pix2pix.

I think I know the answer to our problems. I chatted with a reddit user who got this working just fine on very tame hardware. He claimed he never installed the github version, but rather this specific version of it from here:
https://nmkd.itch.io/t2i-gui/devlog/480628/sd-gui-190-now-with-instructpix2pix
I hope it's OK I'm posting a link. Anyhow, I guess the nmkd version is slightly different than this Github version in some very important ways.

Thank you man. I will definitely check that version.

Report back, let me know if it works for you on a 4GB 970!

Hey, I checked the link you posted and realized that it is an install for NMKD SD GUI. Actually I tried that yesterday itself with their version. It does work. But I was actually looking for the dedicated GUI only for this instruct pix2pix.

from instruct-pix2pix.

Hey, I checked the link you posted and realized that it is an install for NMKD SD GUI. Actually I tried that yesterday itself with their version. It does work. But I was actually looking for the dedicated GUI only for this instruct pix2pix.

Oh I see. So I guess all the people reporting to me on Reddit that it can work with less than 18GB of vram are using this other version, which is not quite what you and all of us here were expecting/hoping for. I guess we'll just have to be patient and wait for the optimizations.

Cheers!

My god, will no one listen to me?

I am not using some funky customized version. My GUI's code is based off of the original author's Colab notebook for InstructPix2Pix. There are no hacks, no workarounds, zero architectural changes. I literally just call the Diffusers pipeline.

These lines are all you need to reproduce it:

pipe = StableDiffusionInstructPix2PixPipeline.from_pretrained("timbrooks/instruct-pix2pix", torch_dtype=torch.float16, revision="fp16", safety_checker=None)
pipe.to("cuda")
pipe.enable_attention_slicing()

image = PIL.Image.open("./example.png")
image = PIL.ImageOps.exif_transpose(image)
image = image.convert("RGB")
image

image = pipe("turn it into a picasso painting", image=image).images[0]
image.save("./example-out.png")

this other version, which is not quite what you and all of us here were expecting/hoping for.

Again I do NOT understand what you're trying to imply here. What are you hoping for that my (aka literally the original) code does not do?

Sorry for the rant but it feels like I'm talking to a wall here.

from instruct-pix2pix.

Should be mentioned that my VRAM tests were done with huggingface/diffusers@f4dddaf

from instruct-pix2pix.

I have the same problem I am on a 12 GB GPU and I can't run the code I installed by cloning this repository and setting up the conda environment.
I tried scaling the example image first to 384, then to 256, then 128 up to 64x64 ... to see if there was a point at which it would begin to start working... but it didn't... same old CUDA Memory Error problem.

So I took a look at these other implementation you are talking about including A1111's one... which I just installed.
I don't know why but for some reason the results it produces are nowhere near close to the original ip2p demo I tried on Hugging Face...
Not sure what is actually going on here... maybe I am just tired...

from instruct-pix2pix.

I don't know why but for some reason the results it produces are nowhere near close to the original ip2p demo I tried on Hugging Face...

I'm guessing the actual ip2p implementation uses some primo CLIP interrogator (perhaps this is where the huge VRAM requirement goes) to better understand what the image actually is.

Fwiw, tryout n00mkrad implementation (windows only it seems), people on Reddit seem to be having good fun with it, even on lower spec hardware. https://nmkd.itch.io/t2i-gui/devlog/480628/sd-gui-190-now-with-instructpix2pix

from instruct-pix2pix.