The caffemodel needs BGR
When you write the datasets,you use cv2 and save the img as a mat ,so the mat is BGR ,and when you TEST a img,the img is read by cv2,so it is BGR, so I think you should not write
alex_images = (tf.pow(images, 1.0 / config_get_input_gamma()) *
255.0)[:, :, :, ::-1]
::-1 in fcn.py build_branches here ,because the img is alreadly BGR,if not ,correct me~~

img_corrected = tf.pow(
images[:, :, :, ::-1] / 65535 / illums_pooled[:, None, None, :] *
tf.reduce_mean(illums_pooled, axis=(1), keep_dims=True)[:, None, None, :],
1 / VIS_GAMMA)

Why need to multipy np.mean(illums_pooled)?

The tensorflow only support the python3.x in windows10,but your code uses python2.7....

Could you tell me the concrete version of python,tensorflow and cuda? Thanks a lot!

hi Mr. Hu:
I reimplement your work on mxnet, the same operation as yours in loss function. But I find that it's easy for 'dot' parameter out of range [-safe_v, safe_v], so that the loss will be the same for lots of images, and the loss will not decrease. However, without clip 'dot', the loss will be 'nan'. I change the loss to be l2 loss, the loss decrease normally.
I wonder why you set 'safe_v' parameters here, and how to avoid the 'dot' parameter out of range to make loss decrease.
Thanks a lot!

Here is my loss function:

    safe_v = 0.999999
    illum_normalized = F.L2Normalization(pred)
    _ilum_normalized = F.L2Normalization(label)
    dot = F.sum(illum_normalized*_ilum_normalized)
    dot = F.clip(dot, -safe_v, safe_v)
    angle = F.arccos(dot)*(180 / math.pi)
    mean_angle = F.mean(angle)
    return mean_angle

Hello@yuanming:
when I train the dataset,it shows that function takes at most 1 positional arguments(2 given),and when I test the dataset use the pretrained model ,it also shows the error:function takes at most 1 positional arguments(2 given),the error lines in self.model = pkl.load(open(MODEL_PATH, 'r')), I don't know why ,Can you help me?thank you

Hello! I'm having a bit of difficulty with viewing the processed images. It's mentioned that a directory outputs_cc would be created with the images but nothing was created other than outputs with the .pkl files. Am I missing any step or is the code not updated?

Many thanks!

Hi, thank you for your excellent work!
I find that the released pretrained models were trained with SEPARATE_CONFIDENCE set to False.
Is it possible that you could also release the pretrained models with SEPARATE_CONFIDENCE set to True, as I was wanting to explicitly use the confidence map for other branches in my network. Thanks a lot!

Hi, I'm puzzled about the vis_confidence_color's compute method .Could you tell me about that?

1. black point of canon1D is 0, not 1, as showed in http://www.cs.sfu.ca/~colour/data/shi_gehler/
2. In NUS-8, It should be img = (raw - darkness_level) / (saturation_level - darkness_level), to normalization the img, but in datasets.py it is saturation_level without darkness_level.

It is reported by some user that when this option is set to True (explicit confidence weighting, the mean error goes to ~4 (in contrast to 1.63 when it is set to False, where implicit confidence weighting is used). This is not a reasonable behavior and worth figuring out what's wrong with the current code.

Hello Mr.Hu,
Step-g) How to make inference on images based on a trained model
Test on other images: (e.g. sample_inputs/a.png)
python2 fc4.py test pretrained/colorchecker_fold1and2.ckpt -1 sample_inputs/a.png

is not working with the above command. Observing the below error:

Loading image pack None
Traceback (most recent call last):
File "fc4.py", line 227, in
globals()mode
File "fc4.py", line 34, in test
data = load_data(image_pack_name.split(','))
File "/data/color_constancy/fc4/data_provider.py", line 23, in load_data
with open(fn) as f:
TypeError: coercing to Unicode: need string or buffer, NoneType found

Please help to resolve the issue. Tried to execute with sample_inputs/a.png

Thank you..

When we get the illuminant estimate using your experiment，How to correct it using the color we estimated and show the correct image? Can you share your code or method?

Hello ,still me....
I have checked your combine.py ,and find that it should be models = sys.argv[1:] in combine.py,or we will miss the final model

Is it better to use fixed learning rate? And why?

This is my understanding. If there are mistakes, I hope you can correct me. tks!!
https://www.cnblogs.com/wangyong/p/9141075.html

Hi Yuanming,

Thanks for the released code. I've been having fun playing around with your code.
Since I just recently studied about color constancy, I have a (probably stupid )problem about the visualization of corrected image.

The corrected image is rendered as follows:

img_corrected = tf.pow(
      images[:, :, :, ::-1] / 65535 / illums_pooled[:, None, None, :] * exposure_boost *
      tf.reduce_mean(illums_pooled, axis=(1), keep_dims=True)[:, None, None, :],
      1 / VIS_GAMMA)

Here what I do not understand about is that why do we need this * tf.reduce_mean(illums_pooled...)? I surveyed several papers and I couldn't find the white-balancing formula about this.

hello , In your data folder ,there are only "geheler" for Shi's datasets,and in this folder ,you have set a groundtruth.mat as label,and in the NUS-8 datasets web ,There are many formats for us to download,so can you please tell me which format should I download ,and how To pre-process NUS-8 Camera Dataset as the Shi's datasets,thank you!

I have finished implementing your model in tensorflow without adjustments before you open source. After you open sourse, I'm just referring to the necessary parts and made some changes because your code is complicated ,I have the following results:
25: 0.91 med: 1.75 tri: 1.44 mean: 2.28 75: 3.12 95: 5.79
It's surprising me that the result is higher than the one in your paper. I'm very puzzled. What do I need to do with data before training or anything else is wrong?

In paper,you said the structure of conv7 should be 1 × 1 × 4
In code ,I see the structure of conv7 is 113,and the param for per_patch_weight setted by zero
How to understand the effect of the weright

reverse gamma correction for sRGB

  img = (img / 255.0) ** 2.2 * 16384

as we all known,when gamma correction
y=(x/255) ** (1/2.2) * 255
so ,I think(if wrong ,correct me) x=(y/255) ** 2.2 * 255 if we want to reverse gamma correction
and your training data is 16bit while test image is 8 bit ,16384==2**14,Could you please tell me why 16384,did this number a scale to invert img from 8bit to 16 bit ,but,why....

hi, yuanming-hu:
It's great job and thank you for your sharing the code.

在跑nus-8实验的时候发现，test的结果恢复出来的图片与数据集中的raw文件、jpg文件差别很大，数据集中的图片颜色更鲜艳，实验跑出来的结果明显偏灰、偏暗，我猜测数据集中的raw和jpg文件是经过了后期处理的图像，但在网页上没找到相关的说明，能否告知测试结果和数据集中数据差异的原因。
另：能否分享Illuminant Estimation for Color Constancy: Why spatial domain methods work and the role of the color distribution这篇论文的全文。

谢谢！

Hi,
It's a great reading experience and a lot of fun to play with the code. One question.

In test() function under fc4.py you apply 16384 for 14 bit DSLR,
　img = (img / 255.0) ** 2.2 * 16384
but when you visualize the image you apply 65535 in get_visualization() under summary_utils.py
　img = tf.pow(images[:, :, :, ::-1] / 65535, 1 / VIS_GAMMA)
　img_corrected = tf.pow(images[:, :, :, ::-1] / 65535 / illums_pooled[:, None, None, :] *
tf.reduce_mean(illums_pooled, axis=(1), keep_dims=True)[:, None, None, :], 1 / VIS_GAMMA)

This makes the "input panel", i.e. upper-left of your 6 panel figure obviously darker than it should be,
particularly when you test external figures and compare the input png with the 6 panel output.
Can you explain why you do this different scaling?

in step b), when I run 'show_pathces.py', it would appear the following bug. I can't find the solution about this bug?
QObject:: moveToThread: Current thread (0x1e60d60) is not the object's thread (0x2e6dca0). Cannot move to target thread (0x1e60d60)

I also want to know, whether the 'show_patches.py' is for data argument? is it necessary to run this code? And are there any other considerations when I run your code in Python 3？

I posted my question here and I am really lost. I checked everything you wrote about FC4 but I couldn't figure out the answer.

Can you please help me out? I would really appreciate!
https://stackoverflow.com/questions/56396730/how-to-show-color-corrected-images-after-performing-illumination-estimation-by-d

Hi, since I'm trying to re-implement your code in Keras (python 3.6), I open this thread for some questions and advice.

1. You defined your input images as 512x512x3, but then your SqueezeNet take an input of 224x224x3, I'm confused, can you clarify this?
2. Since you used Adam optimizer, this part of the code which uses SGD as training optimizer is unnecessary, right?

  opt1 = tf.train.AdamOptimizer(self.learning_rate * FINE_TUNE_LR_RATIO)
  opt2 = tf.train.AdamOptimizer(self.learning_rate)
  grads = tf.gradients(self.total_loss, var_list1 + var_list2)
  grads1 = grads[:len(var_list1)]
  grads2 = grads[len(var_list1):]
  train_op1 = opt1.apply_gradients(zip(grads1, var_list1))
  train_op2 = opt2.apply_gradients(zip(grads2, var_list2))
  self.train_step_sgd = tf.group(train_op1, train_op2) 

3. Did you train your SqueezeNet from scratch or use the wieghts from the pretrained SqueezeNet model ?

4. When you perform the data augmentation:

• The main problem is white balancing, is it necessary to crop and rotate images?
• From your code, you augmented the ground truth illuminations by doing this:

color_aug[i, i] = 1 + random.random( ) * AUGMENTATION_COLOR - 0.5 * AUGMENTATION_COLOR

This will change the original ground truth illumination wthout retaining the von Kries ratio, so my question is why? What's your intuition behind this?

Bonus question: Is it necessary to mask out the MCCs? I see no reason behind this also

Hi,
I was trying to work on resized input images close to 256X256 rather than 50% of Raw camera image size (1020X680). Turns out the confidence matrix become quite small and hits the performance.

I was thinking about modifying squeeze_net structure. Your squeeze_net has 1 extra 2X2 max_pooling than squeeze_net v1.1. Did you retrain the squeeze_net model you provided on imagenet data?
I wonder if I can remove it such that the final confidence matrix resolution is larger?
Do I have to retrain squeezenet backbone on image_net data if I do so?
Any other suggestion? would you recommend doing upconvolution in FC1?

Many thanks,

The website of NUS dataset seems to be closed. Who can provide other download methods?
http://cvil.eecs.yorku.ca/projects/public_html/illuminant/illuminant.html

In your code fcn.py,you said that if not SEPERATE_CONFIDENCE ,you will use way 2 confidence-weighted pooling
2. Just output unnormalized R, G, B, and simply take the sum and normalize. Thus we are weighting using the length.
which is different from your paper(I thought the way you stress in your paper is way1,if not ,please correct me)
so ,forgive me ,I have two questions:
1.Did you trained colorchecker_fold1and2.ckpt with SEPERATE_CONFIDENCE = False?
2. Did the way 2 means the average pooling (c=1) in your paper?
By the way,you use three-fold validations to get the final score, does this mean that I can get fold 0's scores by training fold1,fold2, fold1's score ( trainng fold0,fold2),fold2(traing fold1,fold0) , and just calculate the arithmetic mean value of them ,or I have to calculate the geometrical mean to get the final scores?
So many questions,hh,forgive me,and wishing for your early reply~!

How can I get the "Shi's Re-processing of Gehler's Raw Dataset".Is there another way I can get these database.

When i click the "Canon 1D" on the webpage , it shows that "The requested URL /~colour/data2/shi_gehler/png_canon1d.zip ws not found on this server"

I want to understand the folds.mat .

Hi! I'm working on an open-source re-implementation of the FC4 method in PyTorch. The code is available at https://github.com/matteo-rizzo/pytorch-fc4. Everything seems to work quite smoothly, but I would much appreciate it if @yuanming-hu or anyone who is interested in the project could have a look.

• The architecture is defined at https://github.com/matteo-rizzo/pytorch-fc4/blob/main/classes/fc4/FC4.py
• Running the visualize.py script allows to generate this kind of plots:

Thank you!

Hi.
I recently reproduced your code with TF2 but the trained model seems wrong w.r.t. the original one.
I understood that the confidence map should be like clustered shape. But after I get the images
with my trained model, the image looks all red. Is this meaning that the variance of the distribution of the values
in illums_est are too small, or an evidence of overfitting? Numerical performance is not bad though.
I know the magnitude of these values are not that important because we are investigating the angle of the illumination
vector. But I really wanna know that this full of red color doesn't means that the training went wrong or not. Thanks.

Hello ,I am using tensorflow-gpu 1.2.0 to run your training code and meet this problem
Traceback (most recent call last):
File "fc4.py", line 240, in
globals()mode
File "fc4.py", line 172, in train
fcn = FCN(sess=sess, name=name, kwargs=kwargs)
File "/media/yt/fc4/fcn.py", line 54, in init
self.build()
File "/media/yt/fc4/fcn.py", line 174, in build
fc2 = self.build_branches(self.images, self.dropout)
File "/media/yt/fc4/fcn.py", line 107, in build_branches
fc1 = slim.dropout(fc1, dropout)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args
return func(*args, **current_args)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1216, in dropout
_scope=sc)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/core.py", line 247, in init
self.rate = min(1., max(0., rate))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 578, in nonzero
raise TypeError("Using a tf.Tensor as a Python bool is not allowed. "
TypeError: Using a tf.Tensor as a Python bool is not allowed. Use if t is not None: instead of if t: to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.

It said that I should Use if t is not None: instead of `if t: but I can't find a "if t " in your code,Could you please help me to fix this problem?

fyi
your instructions to create the dataset read python dataset.py
they should read python datasets.py

So I'm running the code remotely on a server with GPU, and the first thing I'm trying to do is download the pretrained models and look at how they perform at samples images. I've run the following line
python2 fc4.py test pretrained/colorchecker_fold1and2.ckpt -1 sample_inputs/37.jpg
(I have a 37.jpg image saved in sample_inputs folder)

I've run the code but it says Gtk-WARNING **: cannot open display:. Is this because I'm running on server only via command line? (no GUI/Desktop options). I don't really care about display as long as it saves the output somewhere. A little noob question but any help would be appreciated!

When I run

CUDA_VISIBLE_DEVICES=0 python2 fc4.py test pretrained/colorchecker_fold1and2.ckpt -1 sample_inputs/37.jpg

this is the output

FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
 from ._conv import register_converters as _register_converters
2018-10-17 13:32:48.510892: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-10-17 13:32:48.620582: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: Tesla K40c major: 3 minor: 5 memoryClockRate(GHz): 0.745
pciBusID: 0000:05:00.0
totalMemory: 11.17GiB freeMemory: 9.74GiB
2018-10-17 13:32:48.620621: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-10-17 13:32:48.883559: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-10-17 13:32:48.883614: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0
2018-10-17 13:32:48.883621: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N
2018-10-17 13:32:48.883903: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9433 MB memory) -> physical GPU (device: 0, name: Tesla K40c, pci bus id: 0000:05:00.0, compute capability: 3.5)
FOLD 0
['g0', 'g1']
['g2']
fire3/squeeze1x1_weights (16, 128, 1, 1)
fire4/expand1x1_weights (128, 32, 1, 1)
conv1_bias (64,)
fire7/squeeze1x1_weights (48, 384, 1, 1)
fire7/expand1x1_weights (192, 48, 1, 1)
fire5/expand3x3_weights (128, 32, 3, 3)
fire5/squeeze1x1_bias (32,)
fire9/squeeze1x1_weights (64, 512, 1, 1)
fire4/expand3x3_weights (128, 32, 3, 3)
fire4/expand3x3_bias (128,)
fire8/squeeze1x1_bias (64,)
fire3/expand3x3_weights (64, 16, 3, 3)
fire8/expand3x3_weights (256, 64, 3, 3)
fire2/expand3x3_weights (64, 16, 3, 3)
fire3/squeeze1x1_bias (16,)
fire6/expand3x3_weights (192, 48, 3, 3)
fire7/expand3x3_weights (192, 48, 3, 3)
fire5/expand1x1_weights (128, 32, 1, 1)
fire2/squeeze1x1_weights (16, 64, 1, 1)
fire5/expand3x3_bias (128,)
conv10_weights (1000, 512, 1, 1)
fire8/squeeze1x1_weights (64, 384, 1, 1)
fire4/squeeze1x1_bias (32,)
fire9/squeeze1x1_bias (64,)
fire6/squeeze1x1_bias (48,)
fire3/expand1x1_weights (64, 16, 1, 1)
fire2/expand1x1_weights (64, 16, 1, 1)
fire8/expand1x1_weights (256, 64, 1, 1)
fire4/squeeze1x1_weights (32, 128, 1, 1)
fire6/squeeze1x1_weights (48, 256, 1, 1)
fire6/expand3x3_bias (192,)
fire2/squeeze1x1_bias (16,)
fire4/expand1x1_bias (128,)
fire5/squeeze1x1_weights (32, 256, 1, 1)
fire9/expand3x3_weights (256, 64, 3, 3)
fire3/expand3x3_bias (64,)
fire8/expand3x3_bias (256,)
fire2/expand1x1_bias (64,)
fire7/expand1x1_bias (192,)
fire9/expand1x1_bias (256,)
fire8/expand1x1_bias (256,)
conv1_weights (64, 3, 3, 3)
fire6/expand1x1_bias (192,)
fire7/squeeze1x1_bias (48,)
fire9/expand3x3_bias (256,)
fire5/expand1x1_bias (128,)
fire9/expand1x1_weights (256, 64, 1, 1)
fire6/expand1x1_weights (192, 48, 1, 1)
fire3/expand1x1_bias (64,)
conv10_bias (1000,)
fire2/expand3x3_bias (64,)
fire7/expand3x3_bias (192,)
/home/USER/fc4-master/squeeze_net.py:112: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
 if init == 'variance':
/home/USER/fc4-master/squeeze_net.py:118: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
 elif init == 'xavier':
conv1_w (3, 3, 3, 64)
fire2_s1_weight (1, 1, 64, 16)
fire2_e1 (1, 1, 16, 64)
fire2_e3 (3, 3, 16, 64)
fire3_s1_weight (1, 1, 128, 16)
fire3_e1 (1, 1, 16, 64)
fire3_e3 (3, 3, 16, 64)
fire4_s1_weight (1, 1, 128, 32)
fire4_e1 (1, 1, 32, 128)
fire4_e3 (3, 3, 32, 128)
fire5_s1_weight (1, 1, 256, 32)
fire5_e1 (1, 1, 32, 128)
fire5_e3 (3, 3, 32, 128)
fire6_s1_weight (1, 1, 256, 48)
fire6_e1 (1, 1, 48, 192)
fire6_e3 (3, 3, 48, 192)
fire7_s1_weight (1, 1, 384, 48)
fire7_e1 (1, 1, 48, 192)
fire7_e3 (3, 3, 48, 192)
fire8_s1_weight (1, 1, 384, 64)
fire8_e1 (1, 1, 64, 256)
fire8_e3 (3, 3, 64, 256)
fire9_s1_weight (1, 1, 512, 64)
fire9_e1 (1, 1, 64, 256)
fire9_e3 (3, 3, 64, 256)
(?, 32, 32, 512)
conv10 (1, 1, 512, 1000)
(?, 32, 32, 1000)
(?, 32, 32, 1000)
(?, 19, 19, 1000)
Feed to FC shape (?, 15, 15, 512)
FC1 shape (?, 15, 15, 64)
FC2 shape (?, 15, 15, 3)
FC2 shape (?, 15, 15, 3)
WARNING:tensorflow:From /home/USER/fc4-master/summary_utils.py:121: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/USER/fc4-master/summary_utils.py:138: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
visualization shape (?, 1040, 1560, 3)
WARNING:tensorflow:From /home/USER/fc4-master/fcn.py:219: get_regularization_losses (from tensorflow.contrib.losses.python.losses.loss_ops) is deprecated and will be removed after 2016-12-30.
Instructions for updating:
Use tf.losses.get_regularization_losses instead.
list1 FCN/AlexNet/Wconv1_w:0
list1 FCN/AlexNet/bias_relu1_b:0
list1 FCN/AlexNet/Wfire2_s1_weight:0
list1 FCN/AlexNet/Wfire2_e1:0
list1 FCN/AlexNet/Wfire2_e3:0
list1 FCN/AlexNet/bias_fire2_fire_bias_s1:0
list1 FCN/AlexNet/bias_fire2_fire_bias_e1:0
list1 FCN/AlexNet/bias_fire2_fire_bias_e3:0
list1 FCN/AlexNet/Wfire3_s1_weight:0
list1 FCN/AlexNet/Wfire3_e1:0
list1 FCN/AlexNet/Wfire3_e3:0
list1 FCN/AlexNet/bias_fire3_fire_bias_s1:0
list1 FCN/AlexNet/bias_fire3_fire_bias_e1:0
list1 FCN/AlexNet/bias_fire3_fire_bias_e3:0
list1 FCN/AlexNet/Wfire4_s1_weight:0
list1 FCN/AlexNet/Wfire4_e1:0
list1 FCN/AlexNet/Wfire4_e3:0
list1 FCN/AlexNet/bias_fire4_fire_bias_s1:0
list1 FCN/AlexNet/bias_fire4_fire_bias_e1:0
list1 FCN/AlexNet/bias_fire4_fire_bias_e3:0
list1 FCN/AlexNet/Wfire5_s1_weight:0
list1 FCN/AlexNet/Wfire5_e1:0
list1 FCN/AlexNet/Wfire5_e3:0
list1 FCN/AlexNet/bias_fire5_fire_bias_s1:0
list1 FCN/AlexNet/bias_fire5_fire_bias_e1:0
list1 FCN/AlexNet/bias_fire5_fire_bias_e3:0
list1 FCN/AlexNet/Wfire6_s1_weight:0
list1 FCN/AlexNet/Wfire6_e1:0
list1 FCN/AlexNet/Wfire6_e3:0
list1 FCN/AlexNet/bias_fire6_fire_bias_s1:0
list1 FCN/AlexNet/bias_fire6_fire_bias_e1:0
list1 FCN/AlexNet/bias_fire6_fire_bias_e3:0
list1 FCN/AlexNet/Wfire7_s1_weight:0
list1 FCN/AlexNet/Wfire7_e1:0
list1 FCN/AlexNet/Wfire7_e3:0
list1 FCN/AlexNet/bias_fire7_fire_bias_s1:0
list1 FCN/AlexNet/bias_fire7_fire_bias_e1:0
list1 FCN/AlexNet/bias_fire7_fire_bias_e3:0
list1 FCN/AlexNet/Wfire8_s1_weight:0
list1 FCN/AlexNet/Wfire8_e1:0
list1 FCN/AlexNet/Wfire8_e3:0
list1 FCN/AlexNet/bias_fire8_fire_bias_s1:0
list1 FCN/AlexNet/bias_fire8_fire_bias_e1:0
list1 FCN/AlexNet/bias_fire8_fire_bias_e3:0
list1 FCN/AlexNet/Wfire9_s1_weight:0
list1 FCN/AlexNet/Wfire9_e1:0
list1 FCN/AlexNet/Wfire9_e3:0
list1 FCN/AlexNet/bias_fire9_fire_bias_s1:0
list1 FCN/AlexNet/bias_fire9_fire_bias_e1:0
list1 FCN/AlexNet/bias_fire9_fire_bias_e3:0
list1 FCN/AlexNet/Wconv10:0
list1 FCN/AlexNet/bias_relu10_b:0
list2 FCN/fc1/weights:0
list2 FCN/fc1/biases:0
list2 FCN/fc2/weights:0
list2 FCN/fc2/biases:0
Model pretrained/colorchecker_fold1and2.ckpt restored.
fire3/squeeze1x1_weights (16, 128, 1, 1)
fire4/expand1x1_weights (128, 32, 1, 1)
conv1_bias (64,)
fire7/squeeze1x1_weights (48, 384, 1, 1)
fire7/expand1x1_weights (192, 48, 1, 1)
fire5/expand3x3_weights (128, 32, 3, 3)
fire5/squeeze1x1_bias (32,)
fire9/squeeze1x1_weights (64, 512, 1, 1)
fire4/expand3x3_weights (128, 32, 3, 3)
fire4/expand3x3_bias (128,)
fire8/squeeze1x1_bias (64,)
fire3/expand3x3_weights (64, 16, 3, 3)
fire8/expand3x3_weights (256, 64, 3, 3)
fire2/expand3x3_weights (64, 16, 3, 3)
fire3/squeeze1x1_bias (16,)
fire6/expand3x3_weights (192, 48, 3, 3)
fire7/expand3x3_weights (192, 48, 3, 3)
fire5/expand1x1_weights (128, 32, 1, 1)
fire2/squeeze1x1_weights (16, 64, 1, 1)
fire5/expand3x3_bias (128,)
conv10_weights (1000, 512, 1, 1)
fire8/squeeze1x1_weights (64, 384, 1, 1)
fire4/squeeze1x1_bias (32,)
fire9/squeeze1x1_bias (64,)
fire6/squeeze1x1_bias (48,)
fire3/expand1x1_weights (64, 16, 1, 1)
fire2/expand1x1_weights (64, 16, 1, 1)
fire8/expand1x1_weights (256, 64, 1, 1)
fire4/squeeze1x1_weights (32, 128, 1, 1)
fire6/squeeze1x1_weights (48, 256, 1, 1)
fire6/expand3x3_bias (192,)
fire2/squeeze1x1_bias (16,)
fire4/expand1x1_bias (128,)
fire5/squeeze1x1_weights (32, 256, 1, 1)
fire9/expand3x3_weights (256, 64, 3, 3)
fire3/expand3x3_bias (64,)
fire8/expand3x3_bias (256,)
fire2/expand1x1_bias (64,)
fire7/expand1x1_bias (192,)
fire9/expand1x1_bias (256,)
fire8/expand1x1_bias (256,)
conv1_weights (64, 3, 3, 3)
fire6/expand1x1_bias (192,)
fire7/squeeze1x1_bias (48,)
fire9/expand3x3_bias (256,)
fire5/expand1x1_bias (128,)
fire9/expand1x1_weights (256, 64, 1, 1)
fire6/expand1x1_weights (192, 48, 1, 1)
fire3/expand1x1_bias (64,)
conv10_bias (1000,)
fire2/expand3x3_bias (64,)
fire7/expand3x3_bias (192,)
conv1_w (3, 3, 3, 64)
fire2_s1_weight (1, 1, 64, 16)
fire2_e1 (1, 1, 16, 64)
fire2_e3 (3, 3, 16, 64)
fire3_s1_weight (1, 1, 128, 16)
fire3_e1 (1, 1, 16, 64)
fire3_e3 (3, 3, 16, 64)
fire4_s1_weight (1, 1, 128, 32)
fire4_e1 (1, 1, 32, 128)
fire4_e3 (3, 3, 32, 128)
fire5_s1_weight (1, 1, 256, 32)
fire5_e1 (1, 1, 32, 128)
fire5_e3 (3, 3, 32, 128)
fire6_s1_weight (1, 1, 256, 48)
fire6_e1 (1, 1, 48, 192)
fire6_e3 (3, 3, 48, 192)
fire7_s1_weight (1, 1, 384, 48)
fire7_e1 (1, 1, 48, 192)
fire7_e3 (3, 3, 48, 192)
fire8_s1_weight (1, 1, 384, 64)
fire8_e1 (1, 1, 64, 256)
fire8_e3 (3, 3, 64, 256)
fire9_s1_weight (1, 1, 512, 64)
fire9_e1 (1, 1, 64, 256)
fire9_e3 (3, 3, 64, 256)
(?, 189, 252, 512)
conv10 (1, 1, 512, 1000)
(?, 189, 252, 1000)
(?, 189, 252, 1000)
(?, 176, 239, 1000)
Feed to FC shape (?, 94, 125, 512)
FC1 shape (?, 94, 125, 64)
FC2 shape (?, 94, 125, 3)
FC2 shape (?, 94, 125, 3)
visualization shape (?, 616, 1224, 3)
62.94697 1260.4567 333.5397

(Ret:20889): Gtk-WARNING **: cannot open display:
`

Note: The same error occurs even when I do
CUDA_VISIBLE_DEVICES=0 python2 fc4.py test pretrained/colorchecker_fold1and2.ckpt -1 sample_inputs/37.jpg sample_inputs/new.jpg

Thanks!

I use the python 3,and train the squeeze net by use caffe
so can you help me for parse the '.caffemodel' to 'pkl'...Tks!

Hi Mr. Hu,
When I try to run the inference on illuminated .PNG or .jpg images, observed the illumination correction is not as good as for the images from the mentioned datasets. Observing a pale pink tint in the corrected output image for various samples.

So does it mean the trained model works only for linear images and in order to make it work for non linear images (jpg, Png) does it need to be re-trained?

I have pre-processed the datasets using your code ,and When I run you test code,I meet this error,I can get result when I run
Test on other images: (e.g. sample_inputs/a.png)
python2 fc4.py test pretrained/colorchecker_fold1and2.ckpt -1 sample_inputs/a.png
can you please tell me how to fix this problem?
/usr/local/lib/python2.7/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
Traceback (most recent call last):
File "fc4.py", line 235, in
globals()mode
File "fc4.py", line 31, in test
external_image = image_pack_name.index('.') != -1
ValueError: substring not found

this is the error information of train：
E:\cp\light\fc4-master>python3 fc4.py train model
2019-03-19 16:34:36.920164: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
FOLD 0
['g0', 'g1']
['g2']
Traceback (most recent call last):
File "fc4.py", line 243, in
globals()mode
File "fc4.py", line 175, in train
fcn = FCN(sess=sess, name=name, kwargs=kwargs)
File "E:\cp\light\fc4-master\fcn.py", line 53, in init
self.build()
File "E:\cp\light\fc4-master\fcn.py", line 173, in build
fc2 = self.build_branches(self.images, self.dropout)
File "E:\cp\light\fc4-master\fcn.py", line 88, in build_branches
alex_outputs = create_convnet(alex_images)
File "E:\cp\light\fc4-master\squeeze_net.py", line 233, in create_convnet
sn = SqueezeNet(imgs)
File "E:\cp\light\fc4-master\squeeze_net.py", line 16, in init
self.build_model()
File "E:\cp\light\fc4-master\squeeze_net.py", line 22, in build_model
self.model = pkl.load(open(MODEL_PATH),'r')
TypeError: Function takes at most 1 positional arguments (2 given)

Just to verify, is the estimation in RGB order or BGR order when you apply to corrected image.
The code below shows how you apply the estimation to the image. Based on my understanding, you flip the channel order to RGB (img[:,:,::-1]/ 65535) and apply the estimation gain, then you flip back the channel order to BGR
corrected = np.power(img[:,:,::-1] / 65535 / est[None, None, :] * np.mean(est), 1/2.2)[:,:,::-1]

So the estimation is in RGB order.

Thank you for your answer

Hi Yuanming,

I noticed that I get different results when I use the loss function in train and in test. Can you tell me please what is the difference between these two loss functions? (angular_error function gives me smaller result when I use the same input)
get_angular_loss and angular_error

thanks in advance,
Sahar

I am interest about your experiments on color constancy, can you provide more code or useful tutorials for train and test the FC4 model.