joeyballentine / esrgan Goto Github PK

First off, huge thanks for the effort in putting this together. It's been a pleasure working with the code.

I am chaining a number of different pre-trained models that all have x4 enhancement but I don't need to upscale the image size that much. The final aim is to enhance the image at 4x in total but to get the effects of the different models in a chained manner. Is there a way to set --outscale to 1 as is possible in REAL-ESRGAN?

Things of note:

• Running on M1 Mac via --cpu
• python 3.10.2 via homebrew

Here's the command I used:
python3 upscale.py --cpu "1x_ReFocus_V3_140000_G.pth>2xBS_Wolly_36000.pth"

I also tried:
python3 upscale.py -c "1x_ReFocus_V3_140000_G.pth>2xBS_Wolly_36000.pth"

Traceback (most recent call last): File "/Users/*/ESRGAN-master/upscale.py", line 571, in <module> app() File "/opt/homebrew/lib/python3.10/site-packages/typer/main.py", line 214, in __call__ return get_command(self)(*args, **kwargs) File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/opt/homebrew/lib/python3.10/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/opt/homebrew/lib/python3.10/site-packages/typer/main.py", line 500, in wrapper return callback(**use_params) # type: ignore File "/Users/*/ESRGAN-master/upscale.py", line 567, in main upscale.run() File "/Users/*/ESRGAN-master/upscale.py", line 220, in run self.load_model(model_path) File "/Users/*/ESRGAN-master/upscale.py", line 306, in load_model state_dict = torch.load(model_path) File "/opt/homebrew/lib/python3.10/site-packages/torch/serialization.py", line 713, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/opt/homebrew/lib/python3.10/site-packages/torch/serialization.py", line 920, in _legacy_load magic_number = pickle_module.load(f, **pic

I've also encountered "killed" despite trying several models that previously worked before. No other info; just the word "killed".

Like this:
'python3 upscale.py --cpu 4x-UltraSharp.pth
Model: "models/4x-UltraSharp.pth"
Upscaling ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% -:--:--zsh: killed python3 upscale.py --cpu 4x-UltraSharp.pth'

Is it something with the image I'm using? The model? Something I am overlooking? I've really no idea where to even start.

I installed required packages on arch linux
yay -S python-pytorch-opt python-torchvision opencv

It returns error.

python upscale.py 1x_Saiyajin_DeJPEG_300000_G.pth                                                                                         
Model: 1x_Saiyajin_DeJPEG_300000_G
Upscaling...
1 low
Traceback (most recent call last):
  File "/home/arun/ESRGAN-master/upscale.py", line 349, in <module>
    load_model(model_path)
  File "/home/arun/ESRGAN-master/upscale.py", line 215, in load_model
    model = model.to(device)
  File "/usr/lib/python3.9/site-packages/torch/nn/modules/module.py", line 612, in to
    return self._apply(convert)
  File "/usr/lib/python3.9/site-packages/torch/nn/modules/module.py", line 359, in _apply
    module._apply(fn)
  File "/usr/lib/python3.9/site-packages/torch/nn/modules/module.py", line 359, in _apply
    module._apply(fn)
  File "/usr/lib/python3.9/site-packages/torch/nn/modules/module.py", line 381, in _apply
    param_applied = fn(param)
  File "/usr/lib/python3.9/site-packages/torch/nn/modules/module.py", line 610, in convert
    return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
  File "/usr/lib/python3.9/site-packages/torch/cuda/__init__.py", line 166, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

Interpolation doesn't seem to work with ERSGAN+ models. It throws the following error:

upscale.py, line 303, in load_model
state_dict[k] = (int(interps[0].split("@")[1]) / 100) * v_1 + (
TypeError: unsupported operand type(s) for *: 'float' and 'collections.OrderedDict'

Looks like v_1 is an ordered dictionary for ERSGAN+ models rather than a tensor.

This code is pretty clearly assuming CUDA:

When doing CPU inference (at least on most Linux systems), an out-of-memory error won't result in a CUDA exception, it will just result in a process (probably Python, but maybe some random other process on the system) being killed. On Windows, it's even worse; the entire system is likely to lock up and require a hard restart.

I see two possible approaches to fix it:

1. Allow the user to choose to explicitly provide a tile size, as upstream Real-ESRGAN's inference code does.
2. Use some heuristic to detect high (but not critical) RAM usage, e.g. checking the RAM size of the machine and comparing it to system RAM+swap usage, and ramp up the tile size until the RAM usage gets too high.

Option 2 seems very messy to me, and I suspect it will not yield optimal results.

I've observed my colab was broken today and them I realized I had to checkout one commit later (9000b43) here for it to be fixed.

That's the error it gives me

Traceback (most recent call last):
File "upscale.py", line 20, in
from utils.architecture.RRDB import RRDBNet as ESRGAN
File "/content/Colab-ESRGAN/utils/architecture/RRDB.py", line 15, in
STATE_T = OrderedDict[str, Tensor]
TypeError: 'type' object is not subscriptable

And thats the command:

!cd "/content/Colab-ESRGAN" && python upscale.py 1x_JPEG_00-20.pth --input /content/current_movie/frames --output /content/current_movie/sr

If I want to try this without CUDA, is it possible to try with CPU and what should I add to the line?

python upscale.py 4xBox.pth --seamless tile

I first reported this on n00mkrad/cupscale#91 but I have tested this script alone and the same issue occurs

On my computer (Nvidia RTX 3080 - 10GB VRAM), if I upscale images that are too big for the VRAM, the script will exit out with a CUDA out of memory exception

However, on my laptop (Nvidia RTX 3080 - 16GB VRAM) I have no issue with any images I attempt to upscale no matter the source image size and model

If I was to guess, it's something to do with the 10GB of VRAM, sadly I don't have more Nvidia cards to throw at this to test it further.