mahmoudnafifi / cie_xyz_net Goto Github PK

PyTorch & Matlab code for the paper: CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks (TPAMI 2021).

MATLAB 54.17% Python 45.83%

computational-photography computer-vision deep-learning deep-neural-networks deeplearning image-processing image-restoration low-level-vision dataset datasets

cie_xyz_net's Introduction

CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks

Mahmoud Afifi, Abdelrahman Abdelhamed, Abdullah Abuolaim, Abhijith Punnappurath, and Michael S. Brown

York University

Reference code for the paper CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks. Mahmoud Afifi, Abdelrahman Abdelhamed, Abdullah Abuolaim, Abhijith Punnappurath, and Michael S. Brown, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2021. If you use this code or our dataset, please cite our paper:

@article{CIEXYZNet,
  title={CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks},
  author={Afifi, Mahmoud and Abdelhamed, Abdelrahman and Abuolaim, Abdullah and Punnappurath, Abhijith and Brown, Michael S},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  pages={},
  year={2021}
}

Code (MIT License)

Prerequisite

Python 3.6
opencv-python
pytorch (tested with 1.5.0)
torchvision (tested with 0.6.0)
cudatoolkit
tensorboard (optional)
numpy
future
tqdm
matplotlib

The code may work with library versions other than the specified.

Get Started

Demos:

Run demo_single_image.py or demo_images.py to convert from sRGB to XYZ and back. You can change the task to run only one of the inverse or forward networks.
Run demo_single_image_with_operators.py or demo_images_with_operators.py to apply an operator(s) to the intermediate layers/images. The operator code should be located in the pp_code directory. You should change the code in pp_code/postprocessing.py with your operator code.

Training Code:

Run train.py to re-train our network. You will need to adjust the training/validation directories accordingly.

Note:

All experiments in the paper were reported using the Matlab version of CIE XYZ Net. The PyTorch code/model is provided to facilitate using our framework with PyTorch, but there is no guarantee that the Torch version gives exactly the same reconstruction/rendering results reported in the paper.

Prerequisite

Matlab 2019b or higher
Deep Learning Toolbox

Get Started

Run install_.m.

Demos:

Run demo_single_image.m or demo_images.m to convert from sRGB to XYZ and back. You can change the task to run only one of the inverse or forward networks.
Run demo_single_image_with_operators.m or demo_images_with_operators.m to apply an operator(s) to the intermediate layers/images. The operator code should be located in the pp_code directory. You should change the code in pp_code/postprocessing.m with your operator code.

Training Code:

Run training.m to re-train our network. You will need to adjust the training/validation directories accordingly.

sRGB2XYZ Dataset

Our sRGB2XYZ dataset contains ~1,200 pairs of camera-rendered sRGB and the corresponding scene-referred CIE XYZ images (971 training, 50 validation, and 244 testing images).

Validation set (570 MB): Part 0

Testing set (2.83 GB): Part 0 | Part 1

Dataset License:

As the dataset was originally rendered using raw images taken from the MIT-Adobe FiveK dataset, our sRGB2XYZ dataset follows the original license of the MIT-Adobe FiveK dataset.

cie_xyz_net's People

Contributors

Stargazers

Watchers

Forkers

killsking abhijithpunnappurath vinthony zachlewis fasladodo liuguoyou edmontdants cv-ip futureprecd barzels shir-barzel-healthy lizhihao6 haolyshiit hminle zfxu wyf0912 yinghappy

cie_xyz_net's Issues

About the GPU usage in the Pytorch code

Hi, Afifi,

Awesome work!

I have tried your code for a larger image. However, I find something that might cause unnecessary memory usage in your PyTorch evaluation code.
The core torch model should be wrapped in

with torch.no_grad():

to avoid the calculation of gradient when testing.

More details can be found here: https://datascience.stackexchange.com/questions/32651/what-is-the-use-of-torch-no-grad-in-pytorch

Best wishes,
Xiaodong Cun

Does the XYZ color space apply to some image retouching operations, such as Hue or Saturation adjustments?

Hi, thank you for your great work. I have been studying image retouching recently, which usually involves using rawRGB images. I am wondering if the XYZ color space is also suitable for some image retouching operations. For example, when I want to modify the Hue of an image, I would convert the image to the HSV color space and then adjust the H channel. However, I am not sure if the general RGB-to-HSV conversion formula is applicable in the XYZ color space.

PSNR calculation

Hi @mahmoudnafifi. Thanks for your great work.
I have some question about XYZ colorspace & PSNR calculation.

First, I searched for the range of the XYZ color space because the PSNR calculation requires the maximum value of the signal.
As you can see in Professor Michael S. Brown's slide, there is a range of XYZ values:

As far as I know, these three functions represent the XYZ mixing coefficients (XYZ values) for representing the color of light corresponding to a particular single wavelength.

In one article, the author says that the values of XYZ are [X-95.047, Y-100, Z-108.883].
I believe Michael's slide is correct, but is there another version of CIE XYZ that I'm not aware of?

Second, if so, how can I calculate the PSNR of a CIE XYZ image?
The channel ranges in CIE XYZ are all different. How did you calculate the PSNR calculated in table 1 of the CIE XYZnet paper?

the difference between the direct 3*3 sRGB2XYZ matrix

Hello，i hava some questions：
1.Why not using direct 33 sRGB2XYZ matrix directly，which is [[3.24..,-1.5,...]]?
2.The model -produced XYZ image is not the same with directly using the 33 sRGB2XYZ matrix

ImageProcessingToolbox, where can I download this toolbox？

ImageProcessingToolbox, where can I download this toolbox？ please tell me the URL

Question about sRGB-XYZ pair generation

Hi, @mahmoudnafifi.
I have a question about sRGB-XYZ image pair generation.
I am currently generating my own stylized sRGB - XYZ image dataset, using NUS-8 dataset.
The stylized sRGB images are generated using Adobe lightroom classic, using RAW files.
And I generated XYZ images with DCRAW.
[using "dcraw -r {1/wb_kernel[0]} 1.0 {1/wb_kernel[2]} 1.0 -o 5 -g 1 1 -6 -T {raw_file}"]

However, I noticed that the XYZ and stylized sRGB images have different sizes.
I think the lens correction matrix might be the reason, but I'm not sure why.
I'd like the software to generate both images to be the same, but I'm stuck between lightroom for generating stylized sRGB and DCRAW or rawpy for generating XYZ images.
Can you think of anything that might be causing the size of the two images to mismatch?

The size of stylized sRGB is 5616x3744.

The size of XYZ is 5640x3752.

Can the model handle sRGB images with incorrect white balance?

In addition, can the model process sRGB images taken by any camera? Does the performance of the model decrease when the camera used to capture the image is not existent in the training set? Many thanks!

Training code, dataset availability?

Hey Mahmoud,

Nice work! This is super inspiring.

Do you still have plans to share the training code and the dataset?

Question about XYZ to sRGB conversion details

Hi Mahmoud,

Thank your valuable work. I have downloaded the image pairs you provided and I want to further modify some parameters such as brightness, contrast, saturation, etc. to augment the number of traing image pairs.
I was wondering if you could provide more details on the XYZ to sRGB conversion process. Specifically, I'm interested in knowing which parameters in the Adobe Camera RAW software development kit you adjusted and how these parameters were adjusted.
Thank you very much for your time and attention.

mahmoudnafifi / cie_xyz_net Goto Github PK

cie_xyz_net's Introduction

CIE XYZ Net: Unprocessing Images for Low-Level Computer Vision Tasks

Code (MIT License)

Prerequisite

The code may work with library versions other than the specified.

Get Started

Demos:

Training Code:

Note:

Prerequisite

Get Started

Demos:

Training Code:

sRGB2XYZ Dataset

Dataset License:

cie_xyz_net's People

Contributors

Stargazers

Watchers

Forkers

cie_xyz_net's Issues

Recommend Projects

Recommend Topics

Recommend Org