[bug]: ROCm Out of Memory Errors - Excessive VRAM Allocation about invokeai HOT 6 OPEN

That is super excessive VRAM usage. Looks like it is running really, really slowly too. The ESRGAN upscale is ridiculously slow.

Unfortunately, I don't have an AMD GPU to test on. Not sure if any regular contributors do. May need community help to figure this out...

From a brief search, it looks like the env var PYTORCH_HIP_ALLOC_CONF may be useful. This old post for A1111 has a value to try: AUTOMATIC1111/stable-diffusion-webui#6460 (comment)

If fp32 is being used instead of fp16, you'll use ~2x VRAM - try setting precision: float16 in invokeai.yaml.

from invokeai.

Setting both PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.9,max_split_size_mb:512 and precision: float16 doesn't seem to fix it. :v
(But then again, I noticed that adding log_format: color didn't actually add colored output either, so maybe the yaml just isn't being read? I'm loading the environment variables directly via the script though, so I know that's at least being read)
This is weird.

from invokeai.

If you suspect the YAML file isn't being read, an easy way to test is add use_memory_db: true. You should see a line like this on startup:

[2024-05-03 13:30:13,686]::[InvokeAI]::INFO --> Initializing in-memory database

Sorry, I'm not sure where to go from here with the performance issue :/

from invokeai.

Ah, that does appear in the terminal output; so I guess it is reading the file but just... not outputting color for some reason? I guess that would otherwise be somewhat reasonably safe to assume that precision: float16 is being passed but whatever the root problem is simply doesn't care.

from invokeai.

We expect colored logs for warnings and errors only (and debug, if you set the log level accordingly). This works for me.

I'm confident the precision is set correctly in normal circumstances, but who knows with the issues you are facing.

from invokeai.

Having A similar issue, Though my system is not allocating massive amounts of VRAM for txt2img, It has a staggeringly low cap. So I can generate images fine but the moment I move to inpainting I cant do anything, and get this error:

OutOfMemoryError: HIP out of memory. Tried to allocate 13.91 GiB. GPU 0 has a total capacity of 19.98 GiB of which 3.43 GiB is free. Of the allocated memory 16.06 GiB is allocated by PyTorch, and 41.54 MiB is reserved by PyTorch but unallocated.

So, for some reason, on my RX 7900 XT with 20GB VRAM, only a measly 3.43GB are available.. It doesn't make sense

Edit: I am on the same system, Ubuntu Linux using AMD ROCm for processing

from invokeai.