Implementing grad-cam in models that use dropout about keras-grad-cam HOT 10 CLOSED

When calling gradient_function, the actual learning phase should be a scalar.
Please try changing the line
output, grads_val = gradient_function([image, K.learning_phase()])
To:
output, grads_val = gradient_function([image, 0])
Does that work?

from keras-grad-cam.

Traditionally, dropout do not used for convolutional and pooling layers. So it is not needed to use K.set_learning_phase in the code.

from keras-grad-cam.

Hi,

output, grads_val = gradient_function([image, 0])

returns the same error message:

Traceback (most recent call last):
File "grad-cam-debug.py", line 90, in <module>
cam  = grad_cam(model, preprocessed_input, predicted_class, "maxpooling2d_2")
File "grad-cam-debug.py", line 58, in grad_cam
output, grads_val = gradient_function([image, 0])
File "/Users/tuomo/.virtualenvs/keras-tf/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 1040, in __call__
updated = session.run(self.outputs + [self.updates_op], feed_dict=feed_dict)
File "/Users/tuomo/.virtualenvs/keras-tf/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 717, in run
run_metadata_ptr)
File "/Users/tuomo/.virtualenvs/keras-tf/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 872, in _run
+ e.args[0])
TypeError: Cannot interpret feed_dict key as Tensor: Can not convert a int into a Tensor.

I guess the full code would be more helpful here; I commit it to my repo (thanks for the suggestion by the way!) tomorrow.

from keras-grad-cam.

@thiippal You should check your code carefully. I have run this code, and it works well.

from keras-grad-cam.

@jf003320018 Which code are you referring to?

@jacobgil I've added the full code for grad-cam on this branch.

from keras-grad-cam.

I took a look at the branch above (only ran with dummy weights since there were no tensorflow format weights).

The problem seems to be the K.set_learning_phase(0) line. Drop that line and it should be ok.

Also in the branch you shared there are a few dimension problems, it seems that MiniVGGNet was called with wrong parameters- (150, 150, 3, 3) instead of (3, 150, 150, 3).
Also replace instances of 224 with 150.

from keras-grad-cam.

Hi @jacobgil, unfortunately the cam branch was behind a few commits, so the TensorFlow weights were missing. I also did not get far enough to resize the images from the canonical ImageNet size, but thanks for pointing that out!

Removing K.set_learning_phase(0) helped, and the script now works with my model and weights.

The output, however, is quite interesting: most images produce a class activation map with red colour only, such as the one below:

Another image, in turn, seems to be working well, as passing different values to the category_index variable produces different kinds of activation maps.

My up-to-date code may be found here ... thanks for your support & patience!

from keras-grad-cam.

Hi @thiippal,
The issue seems to be the preprocessing of the image.
Looks like your model was trained on pixels in the range [0, 1], but the images fed were in the range [0, 255].
Change load_image to something like this, and multiply by 255 when used for visualizing:

def load_image(path):

original = keras.preprocessing.image.load_img(path, target_size=(150, 150))  # Load image and resize to 150x150

array = img_to_array(original, dim_ordering='tf')  # Convert image to numpy array

normalized = array.astype("float") / 255.0  # Normalize the array into range 0...1

reshaped = np.expand_dims(normalized, axis=0)  # Reshape for input for CNN    

return reshaped

An example output i'm getting looks like this:

Btw what are the 3 categories in the output?

from keras-grad-cam.

Hi Jacob,

thanks – should have figured that out: I faced & solved the same issue when originally using your CAM implementation. Now the script returns completely (well perhaps always not so) sensible results. :-)

The three classes correspond to T-72 tanks, BMP APCs and 'other', i.e. street and countryside scenes without none of the aforementioned classes.

I think this issue can be closed now. Hope this will help someone else implementing Grad-CAM ... thanks for your support!

from keras-grad-cam.

Thanks a lot, @jacobgil . you answer:

When calling gradient_function, the actual learning phase should be a scalar.
Please try changing the line
output, grads_val = gradient_function([image, K.learning_phase()])
To:
output, grads_val = gradient_function([image, 0])
Does that work?

is ok, and helped me in my code:

iterate = K.function([input_img, K.learning_phase()], [loss, grads])
loss_value, grads_value = iterate([input_img_data, 1])

from keras-grad-cam.