Comments (7)
Having tested lot of more images - I will close this issue.
Just a small visual difference, and it really depends on the input. But it's probably just down to the more fine texture detail compared to edsr
Thank you
from super-resolution.
I had the same issue with https://github.com/Tencent/Real-SR so I guess that color shift is not (yet) considered a real issue in SR methods but the community (only PSNR takes into account color shift, SSIM and LPIPS consider it a minor disturbance).
Hereβs how I remove the color-shift from super-resolved images: downscale the SR image, compute the difference with the LR image, Gaussian blur, upscale, add to the SR image. All computation is done in linear color space.
This gives this one-line G'MIC script (edited to work also with older versions of gmic):
gmic image.jpg image_4x.jpg -srgb2rgb [1] -resize[1] 25%,25%,[0],[0],2 -sub[0,1] -blur[0] 2 -resize[0] 400%,400%,[0],[0],3 -add -rgb2srgb -output image_4x_nocolorshift.jpg
from super-resolution.
@krasserm sorry no, I didn't measure that, as I was mainly interested by the visual quality, and the color difference between SR-then-downscaled and the original image in my case was too large to get unnoticed (I first noticed it on vegetation and on various textured textiles).
I noticed it on Tencent's Real-SR, which won a SR challenge this year, and then did some research for mentions of color-shift, and only found this github issue concerning SRGAN - which is not very far from Tencent's Real-SR.
Note that the fix posted above does a local average of the color shift and subtracts it from the SR image, which does not fix everything.
I've seen Real-SR adding a red tint to black-and-white textures, and fixing it means adding some cyan to the reddish white, but also adds a bit of cyan to the black. So in terms of PSNR it should be better, but there is still room for improvement, for example by modeling a color-dependent local color shift, rather than a locally constant color shift.
Why this happens still escapes me, and I haven't looked into details, but if the training data is generated by processing gamma-compressed (sRGB) pixel values rather than linear colors, that may be an explanation.
Any image resizing, blurring, convolution, etc should be done in linear color space (Eric Brasseur has the best explanation on the topic, and Helmut Dersch's explanation also helps). You will notice in my G'MIC command-line above that srgb2rgb is the first thing I do, and rgb2srgb is the last thing. All processing is done in floating-point arithmetic.
The deep network could "learn" to do the SR properly (in linear rather than gamma space), but it still needs to have the right training data to do so.
from super-resolution.
Can you please be more specific?
- how did you run inference?
- share sample images?
- ...
from super-resolution.
I've attached my result for srgan and edsr plus manual upres image to compare. You can best see a slight shift in the background behind the cat.
It's tricky to spot side by side, but if you flip between them you can see srgan is slightly more red.
Using the pretrained weights you have provided.
Code is below images.
manual resize:
edsr, overall same color values as above:
srgan, slightly more red (comparing background against edsr and manual resized above)
srgan:
from model.srgan import sr_resnet
from model.common import resolve
from utils import load_image
import tensorflow as tf
import numpy as np
import scipy.misc
model = sr_resnet()
model.load_weights('weights/srgan/gan_generator.h5')
lr = load_image('demo/0869x4-crop.png')
lr = lr[:,:,:3]
h, w, c = lr.shape
lr = lr.reshape(1, h, w, 3)
sr = resolve(model, lr)
sr = np.array(sr)
e, h, w, c = sr.shape
im = np.zeros((w, h, 3))
sr = sr.reshape((h, w, 3))
scipy.misc.imsave('result/srgan_test.png', sr)
edsr:
from model.edsr import edsr
from model.common import resolve
from utils import load_image
import tensorflow as tf
import numpy as np
import scipy.misc
model = edsr(num_res_blocks=16)
model.load_weights('weights/edsr-16-x4/weights.h5')
lr = load_image('demo/0869x4-crop.png')
lr = lr[:,:,:3]
h, w, c = lr.shape
lr = lr.reshape(1, h, w, 3)
sr = resolve(model, lr)
sr = np.array(sr)
e, h, w, c = sr.shape
im = np.zeros((w, h, 3))
sr = sr.reshape((h, w, 3))
scipy.misc.imsave('result/edsr_test.png', sr)
from super-resolution.
Honestly, I cannot see a color shift by plain visual inspection. I asked for some samples to make sure there are no gross distortions but from what you provided these do not exist. Can you quantify this red-shift and tell if it is significant?
from super-resolution.
Thanks for the useful comment @devernay. Wasn't aware of that. Can you quantify the impact of this correction has on PSNR?
from super-resolution.
Related Issues (20)
- Scale x2
- Progressive training
- The training time of per iteration changes when the batch_size is changed
- A tensorflow error in wdsr
- ValueError: The channel dimension of the inputs should be defined. Found `None`.
- Pre-train model for super-resolution by x2/x3
- PSNR doesn't change on custom dataset
- Transparency not supported
- EDSR implementation is buggy
- Not working on GPU HOT 1
- x1 model?
- AssertionError HOT 3
- ValueError: axes don't match array when loading WDSR pre-trained weights
- How can I run it using distributed trainning?
- How Should I customize data for Custom Dataset Training? HOT 2
- How should I create caches file? HOT 1
- How to plot train vs. val loss over epochs
- Weight normalization for tf versions that are incompatible with tfa HOT 2
- missing evaluation file
- weird ouput on pre-trained weights HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from super-resolution.