Thanks for sharing the great project - there is no pass to the https://pan.baidu.com/s/1ZIq6T98Iv1oGk7rC46WACg Baidu link.
Please advise

Nice work!
However, based on the title, I could not find the corresponding paper on google scholar or arxiv. Where could I find the paper?

Hi, I want to know when the code of pami will be released.

Is it not possible to run on Windows?

When we test Vid4 dataset， we find the FTVSR support the input resolution is least 64. However， we get the solution of foliage and walk is 88x60. It's contradictory！！！ So grotesque...

RT

Hello, I tried to use your model to train on the general ×4 video super-resolution task. Vimeo90K is used for the training set, and the degraded model is BD. Vid4 is used for the validation set, and the degraded model is BD. The effect seems not to be very good, which is worse than BasicVSR. Is that true?

great work,

hope you can setup a colab notebook in which users can upload their .mp4 files and get a super-res output back.

When I was running the model, I found that there seems to be a problem with the weight of the DCT layer. Thanks you very much!

Hello,

MSU Graphics & Media Lab Video Group has recently launched two new Super-Resolution Benchmarks.

• Super-Resolution for Video Compression benchmark aims to test Super-Resolution methods on compressed videos and select the best model for each video codec standard.
• Video Upscalers Benchmark: Quality Enhancement determines the best upscaling methods for increasing video resolution and improving visual quality.

If you are interested in participating, you can add your algorithm following the submission steps:

• Submit for Super-Resolution for Video Compression benchmark
• Submit for Video Upscalers Benchmark: Quality Enhancement

We would be grateful for your feedback on our work!

When reproducing the REDS datasets, I reduced the num_input_frames=20 to fit the GPU memory, so the training is fine, 8G is used for every 16G GPU.

But when it's evaluating, the GPU consumed by the training iteration seems not released. So the evaluation encounters the OOM as
RuntimeError: CUDA out of memory. Tried to allocate 436.00 MiB (GPU 1; 15.90 GiB total capacity; 10.98 GiB already allocated; 189.81 MiB free; 14.78 GiB reserved in total by PyTorch)

The config file for the data is as below:
data = dict( workers_per_gpu=4, train_dataloader=dict(samples_per_gpu=1, drop_last=True), test_dataloader=dict(samples_per_gpu=1, workers_per_gpu=1),

BTW, similar problem was reported in mmdet but no feasible solutions was found.

Hello !
I tried to run inference to compare to other VSR networks, but I run out of memory where other networks worked.
What I do is: in a notebook prepare 20 frames of 512x384 and feed this to the model.
This crashes with OOM on GPU w 24GB CUDA memory. At home I don't have bigger GPU.
So does your model require more memory or I am not testing it like it should be done ?
AFAIK the more frames you feed at once, the better result can be expected, is this true ?

KeyError: 'TTVSR is not in the model registry'.
We cannot find the TTVSR model in the released code, we doubt the released code is incomplete so we cannot execute the test code successfully. If possible, please the author to review this released code. Thank you!!!

It seems that there is less frames in your Vid4 than the original Vid4 testing set (each sequence is 4 frame shorter than the original one).

把python分布式命令
python -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$PORT
$(dirname "$0")/train.py $CONFIG --launcher pytorch ${@:3}
改成
python ./tools/train.py ./configs/FTVSR_reds4.py
运行模型训练报错：KeyError: 'TTVSR is not in the model registry'

请问是什么原因呢？

For Vid4, according to your testing code, if I do not misunderstand the code, it seems that you average the metric results of 4x4=16 sub-sequences instead of calculating the average metric results of the 4 original sequences. This will cause inconsistency when comparing with other methods. Maybe you need to stitch the 2x2 results back to get the final results and then calculate the metrics.

How to draw the DCT-based spectral maps of Fig 1(b) and Amplitude-Frequency curves of Fig(c) in your ECCV2022 paper?

Thank you very much!

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/A9dCpjHPfE or add me on WeChat (ID: van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

Attention: please create a new virtual environment for OpenMMLab 2.0.

Nice work! And I'am trying to reproduce the training with Reds dataset on 4 GPU(each with 16G memory). The batchsize is set 1 as suggested in the config file. The total training time looks to be ~31 days. I'm wondering if this is case for your training or I'm wrong somewhere.

Thanks a lot.

In the readme file, 'Generate the compressed videos by ffmpeg with command "ffmpeg -i LR.mp4 -vcodec libx264 -crf CRFvalue LR_compressed.mp4".' to obtain the compressed videos dataset, but REDS and Vimeo90K are the sequences datasets not *.mp4, how to get the compressed video dataset from these sequences datasets specifically?

For example, the vid4 dataset name is the 'calendar_0_0', 'calendar_0_1', 'calendar_1_0', 'calendar_1_1', what dose the mean of the "_0_0" in "./data/vid4/BD_start_1_crf_25/calendar_0_0". Thank you!

In the paper, we got the training dataset split two parts: 50% uncompressed videos and 50% compressed videos, but we can't find how to split the training dataset. It's so confusing!

I use the code provided by your github. Thop shows the flops and params are 1.668T and 43.331M, so which model is 10.8M(in your paper)?

Hi,

I've tested single gpu test in two different machines, one with a GPU 8GB, another is with A10 24 GB. Both gave me oom error even with num_input_frames = 2. What am I missing?

First one with RTX said:

Tried to allocate 620.00 MiB (GPU 0; 7.80 GiB total capacity; 4.31 GiB already allocated; 412.31 MiB free; 6.33 GiB reserved in total by PyTorch)

while on another machine with A10 said:

Tried to allocate 5.50 GiB (GPU 0; 22.02 GiB total capacity; 10.77 GiB already allocated; 4.78 GiB free; 15.63 GiB reserved in total by PyTorch)

I changed REDS dataset to SRFolderMultipleGTDataset and I've a 2 video subset of REDS4_val which looks like

-REDS4_short/
|-- val_sharp/
|--|-- 000/
|--|--|-- %08d.png
|--|-- 001/
|--|--|-- %08d.png
|-- val_sharp_bicubic/
|--|-- X4/
|--|--|-- 000/
|--|--|--|-- %08d.png
|--|--|-- 001/
|--|--|--|-- %08d.png

And I've used the following command: tools/test.py <path/to/config> <path/to/redsModel>--crf 25 --startIdx 0 --test_frames 50

Let me know if you need any further info to help. Thanks!

If I need to test a sequence of pictures with a total length of 1000 and test 20 pictures each time, how should I modify the code? Because I found that modifying num_ input_ frame to 20, and the test image is only 20,but i want to test 1000 pictures

How to fix this error?

Looked at Google Drive link: https://drive.google.com/drive/folders/1ptezGuRhY6qJuaLSJNAbypgG9kDol172

Downloaded FTVSR_REDS.zip, but there are no .pth models in it?

I run your release code in this repo, the parameters of model is 42.11M, but the parameters of model in your paper is 10.8M. Could you release the correct models in your ECCV and TPAMI for fair comparison. Thank you!