kiteretsu77 / apisr Goto Github PK

View Code? Open in Web Editor NEW

800.0 800.0 58.0 11.38 MB

APISR: Anime Production Inspired Real-World Anime Super-Resolution (CVPR 2024)

License: GNU General Public License v3.0

Python 99.79% Shell 0.21%

apisr's People

Contributors

Stargazers

Watchers

apisr's Issues

AssertionError

Hi brother, I'm running with this error: when I run the train.py file, there are multiple input folders and output folders in the tmp folder, but it reminds me that only one folder can exist.

This is strange
Process Process-1:
Traceback (most recent call last):
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/root/autodl-tmp/APISR/scripts/generate_lr_esr.py", line 100, in single_process
    obj_img.degradate_process(out, opt, store_path, process_id, verbose = False)
  File "/root/miniconda3/envs/APISR/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/autodl-tmp/APISR/degradation/degradation_esr.py", line 90, in degradate_process
    self.H264_instance.compress_and_store(np_frame, store_path, process_id)
  File "/root/autodl-tmp/APISR/degradation/video_compression/h264.py", line 52, in compress_and_store
    assert(len(os.listdir(temp_store_path)) == 1)
AssertionError
Process Process-4:
Traceback (most recent call last):
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/root/autodl-tmp/APISR/scripts/generate_lr_esr.py", line 100, in single_process
    obj_img.degradate_process(out, opt, store_path, process_id, verbose = False)
  File "/root/miniconda3/envs/APISR/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/autodl-tmp/APISR/degradation/degradation_esr.py", line 96, in degradate_process
    self.MPEG2_instance.compress_and_store(np_frame, store_path, process_id)
  File "/root/autodl-tmp/APISR/degradation/video_compression/mpeg2.py", line 50, in compress_and_store
    assert(len(os.listdir(temp_store_path)) == 1)
AssertionError
This is strange
Process Process-6:
Traceback (most recent call last):
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/root/autodl-tmp/APISR/scripts/generate_lr_esr.py", line 100, in single_process
    obj_img.degradate_process(out, opt, store_path, process_id, verbose = False)
  File "/root/miniconda3/envs/APISR/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/autodl-tmp/APISR/degradation/degradation_esr.py", line 90, in degradate_process
    self.H264_instance.compress_and_store(np_frame, store_path, process_id)
  File "/root/autodl-tmp/APISR/degradation/video_compression/h264.py", line 52, in compress_and_store
    assert(len(os.listdir(temp_store_path)) == 1)
AssertionError
Process Process-5:
Traceback (most recent call last):
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/root/autodl-tmp/APISR/scripts/generate_lr_esr.py", line 100, in single_process
    obj_img.degradate_process(out, opt, store_path, process_id, verbose = False)
  File "/root/miniconda3/envs/APISR/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/autodl-tmp/APISR/degradation/degradation_esr.py", line 93, in degradate_process
    self.H265_instance.compress_and_store(np_frame, store_path, process_id)
  File "/root/autodl-tmp/APISR/degradation/video_compression/h265.py", line 50, in compress_and_store
    assert(len(os.listdir(temp_store_path)) == 1)
AssertionError
This is strange
Process Process-3:
Traceback (most recent call last):
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/root/autodl-tmp/APISR/scripts/generate_lr_esr.py", line 100, in single_process
    obj_img.degradate_process(out, opt, store_path, process_id, verbose = False)
  File "/root/miniconda3/envs/APISR/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/autodl-tmp/APISR/degradation/degradation_esr.py", line 90, in degradate_process
    self.H264_instance.compress_and_store(np_frame, store_path, process_id)
  File "/root/autodl-tmp/APISR/degradation/video_compression/h264.py", line 52, in compress_and_store
    assert(len(os.listdir(temp_store_path)) == 1)
AssertionError
This is strange
Process Process-2:
Traceback (most recent call last):
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/root/miniconda3/envs/APISR/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/root/autodl-tmp/APISR/scripts/generate_lr_esr.py", line 100, in single_process
    obj_img.degradate_process(out, opt, store_path, process_id, verbose = False)
  File "/root/miniconda3/envs/APISR/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/autodl-tmp/APISR/degradation/degradation_esr.py", line 90, in degradate_process
    self.H264_instance.compress_and_store(np_frame, store_path, process_id)
  File "/root/autodl-tmp/APISR/degradation/video_compression/h264.py", line 52, in compress_and_store
    assert(len(os.listdir(temp_store_path)) == 1)
AssertionError

something went wrong

I tried to install, but this is the response cmd gave me

Hello, I've had the pleasure of reading your work, and I must say, I found it deeply inspiring. Your efforts are truly commendable!
Would you kindly consider releasing the API Dataset sooner in the future? I'm eagerly looking forward to it.
Thank you very much!

有没有通用的，风景，真人，二次元？

关于训练

感谢您的分享，请问您在什么硬件上进行的训练，一共花费了多久的时间？

about GPU

Hello, thank you very much for sharing.
Can Sheet 4080 complete the training?

datasets

Can you send me the dataset you used, or provide a download link，Thanks a lot

请教

请问，本人在预览界面中觉得dat的模型效果更好，但是本人相关知识与水平有限，本地使用时会有报错：

D:\Anaconda\envs\apisr\lib\site-packages\timm\models\layers\__init__.py:49: DeprecationWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", DeprecationWarning)
Traceback (most recent call last):
  File "D:\Project\Local\APISR-0.3.0\test_code\inference.py", line 117, in <module>
    generator = load_grl(weight_path, scale=scale)  # GRL for Real-World SR only support 4x upscaling
  File "D:\Project\Local\APISR-0.3.0\test_code\test_utils.py", line 163, in load_grl
    generator.load_state_dict(weight)
  File "D:\Anaconda\envs\apisr\lib\site-packages\torch\nn\modules\module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for GRL:

简单看代码似乎只能使用GRL和RRDB的模型?应该是需要有改动，还请帮忙解答，谢谢

A Proposal for Inferencing High-Resolution Images with limited gpu vram less than 6GB.

We can split the high-resolution image into multiple fixed size patches without overlap, then do inference on each patch, and finally merge the upscaled patches to obtain the full high-resolution image. I have already implemented this, and it is indeed feasible for enabling low VRAM GPUs like RTX3060 Laptop with 6GB VRM to upscale 1080P images. Notably, it seems to have no apparent negative effect on the quality of the upscaled image.
The motivation from vision transformer and your paper, in vision transformer the image is split into multiple patches for tokenization, and in your paper actually train proportion of high resolution image instead of the whole image.
Moreover, I suppose this apporach can also work for accelerating inference with multiple GPUs.

4x quality not good

This Upscaler does changes OG image for the worse
4x on the right. lips, eyes and flower details got worse

Require a paired dataset

Could you provide a paired dataset(include low-resolution images and high resolutions)?
Google drive link or BaiduDisk link both ok.
Also you can sent it to [email protected] .
Thanks a lot!

Good perfermance!

Nice work ,I just want to say!

Leave your comment on what kinds of upscler and model architecture you want.

Hello! I really want to know what kinds of upscale factors (2x, 3x, 4x) and architecture models (GRL, RRDB, HAT, etc) that users need. Though I cannot train all of them, I still can train the most popular option and release the weight if the visual is good enough. Thanks!

Batch mode has VRAM leakage issues

When i want to transform 1 mintue video, it used more VRAM and not clear cache....

I made windows version

Thank you for great work!
I tested it works well, so i made autoinstall windows version.
https://github.com/sdbds/APISR-for-windows

Hope to add video transform with batch mode.

The result is cool

Thanks to this great job!

About gpu inference

Why am I inferring this model through rtx4090? Its inference speed is very slow. The model used is DAT. Inference speed takes about five seconds to infer a picture. If you check the gpu, you can confirm that it is the gpu that is being called.

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

hope support mac ...

About data curation

Dear author:

Please tell me the correct way to set prepare_dataset.sh.

Here I set data/processed which contains the I-frames from the previous step.
However, I got:

scripts/prepare_datasets.sh: line 4: ./data/processed/: Is a directory

Moreover, could you tell me the torch vision version you used?
since the following message shows it has version conflict.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchaudio 2.3.0+cu121 requires torch==2.3.0, but you have torch 2.3.1 which is incompatible.
Successfully installed torch-2.3.1 torchvision-0.18.1 triton-2.3.1

and

Traceback (most recent call last):
  File "/content/drive/MyDrive/GitClone/APISR/scripts/crop_images.py", line 16, in <module>
    from degradation.ESR.usm_sharp import USMSharp
  File "/content/drive/MyDrive/GitClone/APISR/degradation/ESR/usm_sharp.py", line 11, in <module>
    from degradation.ESR.utils import filter2D, np2tensor, tensor2np
  File "/content/drive/MyDrive/GitClone/APISR/degradation/ESR/utils.py", line 18, in <module>
    from degradation.ESR.degradations_functionality import *
  File "/content/drive/MyDrive/GitClone/APISR/degradation/ESR/degradations_functionality.py", line 10, in <module>
    from torchvision.transforms.functional_tensor import rgb_to_grayscale
ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'
Traceback (most recent call last):
  File "/content/drive/MyDrive/GitClone/APISR/scripts/crop_images.py", line 16, in <module>
    from degradation.ESR.usm_sharp import USMSharp
  File "/content/drive/MyDrive/GitClone/APISR/degradation/ESR/usm_sharp.py", line 11, in <module>
    from degradation.ESR.utils import filter2D, np2tensor, tensor2np
  File "/content/drive/MyDrive/GitClone/APISR/degradation/ESR/utils.py", line 18, in <module>
    from degradation.ESR.degradations_functionality import *
  File "/content/drive/MyDrive/GitClone/APISR/degradation/ESR/degradations_functionality.py", line 10, in <module>
    from torchvision.transforms.functional_tensor import rgb_to_grayscale
ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'

Multi GPU support ?

The DAT model can be very heavy, even on a 3090, when a lots of images needs to be upscalled. Is there any chance you could implements multi-gpu in order for a second card to be active ?

I have no clue how to use torch multi-gpu myself.

Thanks.

Help Needed with Missing `ck.pth` File in APISR Project

Hi APISR Team,

I've been diving into APISR and it's awesome.

Here's a little snag I've hit, though. While running dataset_curation_pipeline/collect.py, I bumped into this snag:

FileNotFoundError: [Errno 2] No such file or directory: 'pretrained/ck.pth

I don't seem to have found this CKPT in the repo. Can you provide it?

The code seems to be looking for it right here:

class video_scoring:
    def __init__(self) -> None:

        # Init the model
        self.scorer = ICNet()
        self.scorer.load_state_dict(torch.load('pretrained/ck.pth',map_location=torch.device('cpu')))
        self.scorer.eval().cuda()

Thanks a ton for the help! Can't wait to get back to exploring APISR.

Cheers!

kiteretsu77 / apisr Goto Github PK

apisr's People

Contributors

Stargazers

Watchers

Forkers

apisr's Issues

Recommend Projects

Recommend Topics

Recommend Org