RCAN X2 PSNR only 36.xx about cutblur HOT 7 CLOSED

Hi. @BarCodeReader
When I evaluate RCAN x2 (w/ MoA) with inference code on this repo using full DIV2K valid dataset, I can get 34.99 PSNR which is slightly better than the baseline (RCAN x2 w/o MoA); 34.96.
Of course, these models were trained with exactly the same settings and codes.

BTW, have you tried on training RCAN x2 w/o MoA as well? and could you elaborate more on why you think that 36.32 PSNR is lower than your thought?

from cutblur.

sorry i did not make the statement very clear.

RCAN x2 on DIV2K, without MoA is currenty under training so I am still waiting.
I trained RCAN x2 on DIV2K, with MoA, and evaluate on Set5 x2, I get PSNR 36.32, which is low compare to other models.
So i wonder, do I misunderstand anything or messed up something there?

maybe when I finish everything and have more data, then I update this thread again. Currently I just try to make sure I did not do anything wrong. :)

from cutblur.

hmm... it's strange. I got 38.xx on both w/ and w/o MoA models.

• Did you use the same eval code? (to grayscale using rgb2ycbcr function in utils.py and calc psnr)
• Were there any signs of PSNR degradation during training? I sometimes observed that RCAN suffers the huge PSNR fall at the early steps, which degrade the final performance.

from cutblur.

Thanks for the advices provided.
let me check through again on my side.
Thanks!

from cutblur.

@nmhkahn
hello, finally collected some updates here:

usnig all the code provided in this repo (train, inference, etc. all parameter keeps the same only modified the data file path), I get below results for RCAN x2:

------------------------------- without MoA-----------with MoA----------------RCAN paper
RCAN x2 Set5------------PSNR@38.15---------PSNR@38.15---------PSNR@38.27
RCAN x2 Set14----------PSNR@33.96---------PSNR@33.97---------PSNR@34.12
RCAN x2 Urban100-----PSNR@33.08---------PSNR@33.13---------PSNR@33.34

seems the MoA is working 😃 ...but I think the PSNR I get here is a bit low compare to RCAN paper...
Did you also counter this during your training and inference?

Really thanks for your reply and amazing work!

from cutblur.

For RCAN, we also observed some degradation compared to the original report, especially for X2.
I think that retrain again and again is the best solution...
And in our paper, we focus on the benchmarking model on the X4 scale, so we're not carefully tuning RCAN on X2. You may get very similar results when you train the model by X2->X4.

from cutblur.

Thanks! I also notice that your paper mainly focus on X4.
I might try to tune from X2 -> X4.

I closed the issue. Thanks for your reply.

from cutblur.