TensorFlow implementation of the CVPR 2018 spotlight paper, Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs. If you use any code or data from our work, please cite our paper.
I will release all the data and code ASAP. <(_ _)>
Results
Method
Description
Label
Retouched by photographer from MIT-Adobe 5K dataset [1]
Our (HDR)
Our model trained on our HDR dataset with unpaired data
Our (SL)
Our model trained on MIT-Adobe 5K dataset with paired data (supervised learning)
Our (UL)
Our model trained on MIT-Adobe 5K dataset with unpaired data
CycleGAN (HDR)
CycleGAN's model [2] trained on our HDR dataset with unpaired data
DPED_device
DPED's model [3] trained on a specified device with paired data (supervised learning)
CLHE
Heuristic method from [4]
NPEA
Heuristic method from [5]
FLLF
Heuristic method from [6]
Input
Label
Our (HDR)
Our (SL)
Our (UL)
CycleGAN (HDR)
DPED_iPhone6
DPED_iPhone7
DPED_Nexus5x
CLHE
NPEA
FLLF
Input (MIT-Adobe)
Our (HDR)
DPED_iPhone7
CLHE
Input (Internet)
Our (HDR)
DPED_iPhone7
CLHE
User study
Preference Matrix
(20 participants and 20 images using pairwise comparisons)
CycleGAN
DPED
NPEA
CLHE
Ours
Total
CycleGAN
-
32
27
23
11
93
DPED
368
-
141
119
29
657
NPEA
373
259
-
142
50
824
CLHE
377
281
258
-
77
993
Ours
389
371
350
323
-
1433
Our model trained on HDR images ranked the first and CLHE was the runner-up. When comparing our model with CLHE, 81% of users (323 among 400) preferred our results.
Other applications of global U-Net, A-WGAN and iBN
This paper proposes three improvements: global U-Net, adaptive WGAN (A-WGAN) and individual batch normalization (iBN). They generally improve results; and for some applications, the improvement is sufficient for crossing the bar and leading to success. We have applied them to some other applications.
Input
Ground truth
global U-Net
U-Net
For global U-Net, we applied it to trimap segmentation for pets using the Oxford-IIIT Pet dataset. The accuracies of U-Net and global U-Net are 0.8759 and 0.8905 respectively.
λ = 0.1
λ = 10
λ = 1000
WGAN-GP
A-WGAN
With different λ values, WGAN-GP could succeed or fail. The proposed A-WGAN is less dependent with λ and succeeded with all three λ values.
Male -> Female
Female -> Male
Input
with iBN
w/o iBN
Input
with iBN
w/o iBN
We applied the 2-way GAN to gender change of face images. As shown in the figure, the 2-way GAN failed on the task but succeeded after employing the proposed iBN.
Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs. Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition 2018 (CVPR 2018), to appear, June 2018, Salt Lake City, USA.
Citation
@INPROCEEDINGS{Chen:2018:DPE,
AUTHOR = {Yu-Sheng Chen and Yu-Ching Wang and Man-Hsin Kao and Yung-Yu Chuang},
TITLE = {Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs},
YEAR = {2018},
MONTH = {June},
BOOKTITLE = {Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2018)},
PAGES = {to appear},
LOCATION = {Salt Lake City},
}
Reference
Bychkovsky, V., Paris, S., Chan, E., Durand, F.: Learning photographic global tonal adjustment with a database of input/output image pairs. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. pp. 97-104. CVPR'11 (2011)
Zhu, J. Y., Park, T., Isola, P., Efros, A. A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the 2017 IEEE International Conference on Computer Vision. pp. 2242-2251. ICCV'17 (2017)
Ignatov, A., Kobyshev, N., Vanhoey, K., Timofte, R., Van Gool, L.: DSLR-quality photos on mobile devices with deep convolutional networks. In: Proceedings of the 2017 IEEE International Conference on Computer Vision. pp. 3277-3285. ICCV'17 (2017)
Wang, S., Cho, W., Jang, J., Abidi, M. A., Paik, J.: Contrast-dependent saturation adjustment for outdoor image enhancement. JOSA A. pp. 2532-2542. (2017)
Wang, S., Zheng, J., Hu, H. M., Li, B.: Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Transactions on Image Processing. pp. 3538-3548. TIP'13 (2013)
Aubry, M., Paris, S., Hasinoff, S. W., Kautz, J., Durand, F.: Fast local laplacian filters: Theory and applications. ACM Transactions on Graphics. Article 167. TOG'14 (2014)
Contact
Feel free to contact me if there is any questions (Yu-Sheng Chen [email protected]).