jacobgil / keras-grad-cam Goto Github PK

fitst, thank for share the code, learn a lot.
but i has a question: can you explain why repace keras.activations.relu？
thank you!

`def modify_backprop(model, name):
g = tf.get_default_graph()
with g.gradient_override_map({'Relu': name}):

    # get layers that have an activation
    layer_dict = [layer for layer in model.layers[1:] if hasattr(layer, 'activation')]

    # replace relu activation
    for layer in layer_dict:
        if layer.activation == keras.activations.relu:
            layer.activation = tf.nn.relu

    # re-instanciate a new model
    new_model = VGG16(weights='imagenet')
return new_model`

Hi and thanks for this. I was wondering if you could give me a hint on why I receive the following error every time that I use the grad-cam.

Here is a traceback of the error:

Traceback (most recent call last):
  File "main.py", line 180, in <module>
    "conv2d_2")
  File "/Users/user/grad_cam.py", line 163, in run_gradcam
    layer_name)
  File "/Users/user/grad_cam.py", line 106, in grad_cam
    model.add(input_model)
  File "/Users/user/anaconda2/lib/python2.7/site-packages/keras/models.py", line 441, in add
    ' pre-existing inbound connections.')
ValueError: A layer added to a Sequential model must not already be connected somewhere else. Model received layer sequential_1 which has 4 pre-existing inbound connections.

Here is my model:

def cnn_model(img_rows=28, img_cols=28,
              channels=1, nb_filters=64, nb_classes=10):
    """
    :param img_rows: number of row in the image
    :param img_cols: number of columns in the image
    :param channels: number of color channels (e.g., 1 for MNIST)
    :param nb_filters: number of convolutional filters per layer
    :param nb_classes: the number of output classes
    :return:
    """

    # Define the layers successively
    if K.image_dim_ordering() == 'th':
        data_shape = (channels, img_rows, img_cols)
    else:
        data_shape = (img_rows, img_cols, channels)

    model = Sequential([
        Dropout(0.2, input_shape=data_shape),
        Convolution2D(nb_filters, 8, 8, subsample=(2, 2),
                      border_mode='same', activation='relu'),
        Convolution2D((nb_filters * 2), 6, 6, subsample=(2, 2),
                      border_mode='valid', activation='relu'),
        Convolution2D((nb_filters * 2), 5, 5, subsample=(1, 1),
                      border_mode='valid', activation='relu'),
        Dropout(0.5),
        Flatten(),
        Dense(nb_classes, activation='softmax')
    ])
    model.compile(optimizer='adam', loss='categorical_crossentropy',
                  metrics=['accuracy'])

    return model

Hi, thanks for such a great blogpost. love it.

I'm trying to understand the paper's approach deeper with your code implementation.
As far as I know, the paper suggests getting the gradient of the Softmax input w.r.t the target conv layer.

In your code I think it's referring to the output of the softmax layer
loss = K.sum(model.layers[-1].output)
I was wondering if this should be corrected as
loss = K.sum(model.layers[-1].output.op.inputs[0])
and get the gradient with K.gradient function.

Please correct me if I've misunderstood the concept or your approach.

Thank you!

Thank you for your great code! I'm trying to use it for other keras classifier models.
However, I got an error shown below with other models than VGG16 as it is.
How can I load model and weights? Is it possible to use "load_model"?

IndexError: list index out of range

saliency = saliency_fn([preprocessed_input, 0]) this gives me NaN for valid inputs. Below is my modified VGG16 network with BatchNorm.

def VGG16(input_shape=(224,224,3),classes=10):
input_img = Input(shape=input_shape)
# Block 1
x=Conv2D(64, (3, 3), activation='relu', padding='same', input_shape= input_shape, name='block1_conv1', kernel_initializer='glorot_normal')(input_img)
x=Conv2D(64, (3, 3), activation='relu', padding='same', name='block1_conv2', kernel_initializer='glorot_normal')(x)
x=MaxPooling2D((2, 2), strides=(2, 2), name='block1_pool')(x)
x=BatchNormalization()(x)
# Block 2
x=Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv1', kernel_initializer='glorot_normal')(x)
x=Conv2D(128, (3, 3), activation='relu', padding='same', name='block2_conv2', kernel_initializer='glorot_normal')(x)
x=MaxPooling2D((2, 2), strides=(2, 2), name='block2_pool')(x)
x=BatchNormalization()(x)
# Block 3
x=Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv1', kernel_initializer='glorot_normal')(x)
x=Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv2', kernel_initializer='glorot_normal')(x)
x=Conv2D(256, (3, 3), activation='relu', padding='same', name='block3_conv3', kernel_initializer='glorot_normal')(x)
x=MaxPooling2D((2, 2), strides=(2, 2), name='block3_pool')(x)
x=BatchNormalization()(x)
# Block 4
x=Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv1', kernel_initializer='glorot_normal')(x)
x=Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv2', kernel_initializer='glorot_normal')(x)
x=Conv2D(512, (3, 3), activation='relu', padding='same', name='block4_conv3', kernel_initializer='glorot_normal')(x)
x=MaxPooling2D((2, 2), strides=(2, 2), name='block4_pool')(x)
x=BatchNormalization()(x)
# Block 5
x=Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv1', kernel_initializer='glorot_normal')(x)
x=Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv2', kernel_initializer='glorot_normal')(x)
x=Conv2D(512, (3, 3), activation='relu', padding='same', name='block5_conv3', kernel_initializer='glorot_normal')(x)
x=MaxPooling2D((2, 2), strides=(2, 2), name='block5_pool')(x)
# Classification block
x=Flatten(name='flatten')(x)
x=Dense(256, activation='relu', name='fc1', kernel_initializer='glorot_normal')(x)
#model.add(Dropout(0.5))
x=Dense(128, activation='relu', name='fc2', kernel_initializer='glorot_normal')(x)
#model.add(Dropout(0.5))
x=Dense(classes, activation='softmax', name='predictions', kernel_initializer='glorot_normal')(x)
model = Model(input_img, x)
model.summary()
#model = Model(inputs, x, name='vgg16')
return model

when I run the code ,the following question appeared
AttributeError: Layer vgg16 has multiple inbound nodes, hence the notion of "layer input" is ill-defined. Use get_input_at(node_index) instead.
i change it as
gradient_function = K.function([model.layers[0].get_input_at(1)], [conv_output, grads])
but it also does not work
i do not understand what the function want? any help will be appreciated.

Originally, use the probability value of yc as the category to obtain the partial derivative of Aij in the feature layer. Why do you use loss to obtain the partial derivative of Aij in your code?

Line 47: weights = np.mean(grads_val, axis = (0, 1))

Hi, Jacob.

Why do you do average before multiplying weights to conv_outputs so that per conv filter only multiply to a single "averaged" weight?

If I understand the paper correctly, it would be more accurate we directly do element-wise multiplication between weights of every pixel of every conv filter output to the filter output, then average over number of filters to get gradients-activation-map. Am I understanding it wrong?

My code looks sth. like this:

` width, height, _ = original_img.shape

img_rgb = cv2.cvtColor(original_img, cv2.COLOR_BGR2RGB)

img = normalize_color(img_rgb[None,:,:,:])
img = img.reshape([1,66,200,3])

conv_outputs, grads_val, angle = gradients_function([img])
conv_outputs, grads_val = conv_outputs[0,:], grads_val[0,:,:,:]

# Evaluate the angle to determine the weights
class_weights = grad_cam_loss(grads_val, angle) # function to convert steering angle to weigts

#Create the class activation map.
cam = np.zeros(dtype = np.float32, shape = conv_outputs.shape)

# Element-wise muliplication
cam = class_weights*conv_outputs

cam = np.mean(cam, axis = (2)) # Average between all filters


cam /= np.max(cam)
cam = cv2.resize(cam, (height, width))
heatmap = cv2.applyColorMap(np.uint8(255*cam), cv2.COLORMAP_JET)

`

I am using this code with pretrained weight from the following link for cifar10 dataset
https://github.com/geifmany/cifar-vgg/blob/master/cifar10vgg.py

model = cifar10vgg()

creates a vgg model

Then, I added gradcam.py code from the forked version:
https://github.com/PowerOfCreation/keras-grad-cam/blob/master/grad-cam.py

I pass this model as input_model to grad_cam method. I keep getting error saying cifar10vgg object has no attribute input and output in the following lines:

x = Lambda(target_layer, output_shape = target_category_loss_output_shape) (input_model.output)
model = Model(inputs=input_model.input, outputs=x

Can you please help?

@jacobgil @twuilliam Hi thanks for the wonderful code , i had few queries

1. Can we use this source code for any other architecture besides vg16/ resnet/customized architecture
2. If we have to use this code then what are the modficiations to be made in the code
   Thnaks in advance

Hi all,
I am working with a feature extractor (Inception V3, VGG16, whatever) plus an LSTM for sequences classification (let's say 3 sec. each one). There's any way to use GradCam in order to obtain areas activation on the input sequences? Right now, I just classify one sequence without single frame classification, is still possible apply GradCam to this network?
I didn't find any example code about it, but just for GradCam applied to single feature extractor.
Thanks in advance for the help,
Gabriele

Hey,

I'm attempting to implement grad-cam into my project, which uses a different architecture than VGG16. The model, which has just two convolutional blocks (see here under MiniVGGNet), uses dropout to combat overfitting. Note that this model still uses Theano; I'm moving to TensorFlow now.

Now, if I've understood correctly, Keras must be told whether to operate in training (include dropout) or testing (exclude dropout) mode, as Francois Chollet describes here.

I've therefore included K.learning_phase (which has been set to zero, that is, testing mode) into the gradient function in grad-cam:

gradient_function = K.function([model.layers[0].input, K.learning_phase()], [conv_output, grads])
output, grads_val = gradient_function([image, K.learning_phase()])

This, however, returns the following error:

Traceback (most recent call last):
File "grad-cam-debug.py", line 90, in <module>
cam  = grad_cam(model, preprocessed_input, predicted_class, "maxpooling2d_2")
File "grad-cam-debug.py", line 58, in grad_cam
output, grads_val = gradient_function([image, K.learning_phase()])
File "/Users/tuomo/.virtualenvs/keras-tf/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 1040, in __call__
updated = session.run(self.outputs + [self.updates_op], feed_dict=feed_dict)
File "/Users/tuomo/.virtualenvs/keras-tf/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 717, in run
run_metadata_ptr)
File "/Users/tuomo/.virtualenvs/keras-tf/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 872, in _run
+ e.args[0])
TypeError: Cannot interpret feed_dict key as Tensor: Can not convert a int into a Tensor.

Any idea what might be the issue here?

Does this implementation only work with ImageNet like images or can it be tuned to your specific problem?

Hi
I have trained a model which has several convolution and batch normalization layers and 2 classifier layers. This model trained with 5000 medical images categorized to 2 classes. Know, I have high accuracy and ROC but my heatmap doesn't show the right area of each class in images. I used grad-cam. Anybody knows what is the problem?

Thanks in advanced

I want to output grad-cam of U-net architecture which does't has sigmoid layer at the end and no fully connected and softmax layer.

Hello,

I would like to use grad-CAM to add explainability to my model train on multiple multivariate timeseries.

The idea would be to have something like this :

found a couple of paper that do it (Explainable Deep Neural Networks for Multivariate Time Series Predictions, XCM: An Explainable Convolutional Neural Network for Multivariate Time Series Classification) but they dont explain how just that they use grad-CAM keras.

I have tried to adapt grad-CAM to my own model and adapt it for timeseries using [this issue from grad-cam github] (#23) but I am really strugelling to adapt it to my own network.

Do you have piece of code even for image cam so I could use grad-CAM on my own network.

Thanks in advance

Hi. Good job putting this together. I was wondering whether loss=K.sum(...) should be replaced with K.max(...) once we are interested on the output for the class with highest output value only. What do you think?

@jacobgil Thank you for this repository.
I am trying to implement this in my keras trained segmentation model (FCN8+VGG19).
I have referenced the Segmentation Notebook from your gradcam repo for pytroch, where you have created a wrapper, as pytorch model was returning a dictionary rather than a tensor. However, with Keras, I loaded my model, pre-processed and fed the input image, and ran model.predict(), I get a tensor which is basically of the shape of input image and has values depending on the category (in my case 0 and 1).

I'm having issues proceeding from here.
This repo does:

top_1 = decode_predictions(predictions)[0][0]
print('Predicted class:')
print('%s (%s) with probability %.2f' % (top_1[1], top_1[0], top_1[2]))

predicted_class = np.argmax(predictions)
cam, heatmap = grad_cam(model, preprocessed_input, predicted_class, "block5_conv3")

So it's basically picking up a category, and calculating gradients.
What should I do in my case?

Kindly help me out,
Thank you.

Is it good for 3D convolutional Neural Network? I mean the input shape is like this (nb_ex, nx, ny, nz, nb_channel). Many thanks.

I want to realize it but I failed. So really hope you realize the guided back propagation, thank you.

Hi. I just updated Keras (2.1.0) and TF (1.4.0) and the your code is no longer working. It seems that the function K.gradients(...) is now returning [None]. Any idea how to fix it without downgrading to older versions?

Full error report:

Traceback (most recent call last):
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
    preferred_dtype=default_dtype)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 926, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 229, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 208, in constant
    value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 371, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 524, in _apply_op_helper
    values, as_ref=input_arg.is_ref).dtype.name
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 926, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 229, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 208, in constant
    value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 371, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "grad-cam-or.py", line 138, in <module>
    cam, heatmap = grad_cam(model, preprocessed_input, predicted_class, "block5_conv3")
  File "grad-cam-or.py", line 100, in grad_cam
    grads = normalize(K.gradients(loss, conv_output)[0])
  File "grad-cam-or.py", line 22, in normalize
    return x / (K.sqrt(K.mean(K.square(x))) + 1e-5)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 1412, in square
    return tf.square(x)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py", line 449, in square
    return gen_math_ops.square(x, name=name)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4567, in square
    "Square", x=x, name=name)
  File "/Users/mviana/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 528, in _apply_op_helper
    (input_name, err))
ValueError: Tried to convert 'x' to a tensor and failed. Error: None values not supported.

I am using Keras with tensorflow backend and I have fine-tuned the last Conv layer and FC layer of my network based on VGG weights. Now I am using grad-CAM technique to visualize which parts of my image triggered the prediction and I get all zeros for mean intensity of the gradient over a specific feature map channel.

I have 4 classes, for my test sample these are the prediction:

preds_sample = model.predict(x)
output>> array([[1., 0., 0., 0.]], dtype=float32)

# This is the "sample image" entry in the prediction vector    
image_0 = model.output[:, 0]

last_conv_layer = model.get_layer('conv2d_13')
grads = K.gradients(toilet_w, last_conv_layer.output)[0]
grads.shape
output>> TensorShape([Dimension(None), Dimension(512), Dimension(14), Dimension(14)])

Since I am using theano image ordering - when I calculate the mean of grads my axis is (0,2,3)

from keras import backend as K
K.set_image_dim_ordering('th')

pooled_grads = K.mean(grads, axis=(0,2,3))
pooled_grads.shape
output>> TensorShape([Dimension(512)])

iterate = K.function([model.input], [pooled_grads, last_conv_layer.output[0]])
pooled_grads_value, conv_layer_output_value = iterate([x])
pooled_grads_value.shape, conv_layer_output_value.shape
output>> ((512,), (512, 14, 14))

pooled_grads_value is all zero

Reference: https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/5.4-visualizing-what-convnets-learn.ipynb

I tested the algorithm with more images and found out it works for some of the images. Then I noticed that I have a dropout layer after my last conv layer. I did more research #2 and modified the code as:

last_conv_layer = model.get_layer('conv2d_13')
grads = K.gradients(sample_output, last_conv_layer.output)[0]

# normalization trick: we normalize the gradient
#grads = normalize_grad(grads)

pooled_grads = K.mean(grads, axis=(0, 2, 3))
iterate = K.function([model.input, K.learning_phase()], [pooled_grads, last_conv_layer.output[0]])

pooled_grads_value, conv_layer_output_value = iterate([x, 0])

But Still for some of the images all pooled_grads are zeros.

Hi, @jacobgil

As a result of classifying with Resnet, Accuarcy is over 99%. If you hit map the object area with gradCAM with that model file, it does not match exactly. Why?

It seems to be a problem of GradCAM rather than Resnet classification learning. The objects to be hit-mapped are not as local or blob like dogs or cats, but close to a long straight line. In this case, GradCAM seems to miss the object area. Have you experienced this?

For a well-trainedd Resnet34 model, how do you optimize GradCAM?

Thanks, in advance.

from @bemoregt.

I've tried loading my own model (A fine tuned version of VGG16) using load_model but I get this error:
`AttributeError Traceback (most recent call last)
in ()
146
147 predicted_class = np.argmax(predictions)
--> 148 cam, heatmap = grad_cam(model, preprocessed_input, predicted_class, "block5_conv3")
149 cv2.imwrite("gradcam.jpg", cam)
150

in grad_cam(input_model, image, category_index, layer_name)
102
103 x = input_model.layers[-1].output
--> 104 x = Lambda(target_layer, output_shape=target_category_loss_output_shape)(x)
105 model = keras.models.Model(input_model.layers[0].input, x)
106

/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py in call(self, inputs, **kwargs)
604
605 # Handle mask propagation.
--> 606 previous_mask = _collect_previous_mask(inputs)
607 user_kwargs = copy.copy(kwargs)
608 if not _is_all_none(previous_mask):

/usr/local/lib/python3.6/dist-packages/keras/engine/topology.py in _collect_previous_mask(input_tensors)
2832 inbound_layer, node_index, tensor_index = x._keras_history
2833 node = inbound_layer._inbound_nodes[node_index]
-> 2834 mask = node.output_masks[tensor_index]
2835 masks.append(mask)
2836 else:

AttributeError: 'Node' object has no attribute 'output_masks'`

I am getting the following error when I am trying to run your code on one of the example images. Is this due to some missing package? Thanks!

Predicted class:
boxer (n02108089) with probability 0.42
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
    preferred_dtype=default_dtype)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1040, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py", line 235, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py", line 214, in constant
    value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_util.py", line 421, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 524, in _apply_op_helper
    values, as_ref=input_arg.is_ref).dtype.name
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1040, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py", line 235, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/constant_op.py", line 214, in constant
    value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_util.py", line 421, in make_tensor_proto
    raise ValueError("None values not supported.")
ValueError: None values not supported.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "grad-cam.py", line 135, in <module>
    cam, heatmap = grad_cam(model, preprocessed_input, predicted_class, "block5_conv3")
  File "grad-cam.py", line 99, in grad_cam
    grads = normalize(K.gradients(loss, conv_output)[0])
  File "grad-cam.py", line 22, in normalize
    return x / (K.sqrt(K.mean(K.square(x))) + 1e-5)
  File "/usr/local/lib/python3.5/dist-packages/keras/backend/tensorflow_backend.py", line 1435, in square
    return tf.square(x)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/math_ops.py", line 470, in square
    return gen_math_ops.square(x, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_math_ops.py", line 7813, in square
    "Square", x=x, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 528, in _apply_op_helper
    (input_name, err))
ValueError: Tried to convert 'x' to a tensor and failed. Error: None values not supported.

Hi, I notice your algorithm is using last conv gradient times the last conv output and go through a relu function to calculate the heatmap. I don't understand why you do this but after a quick scan of the paper, I feel using the gradient of last conv layer rule is more reasonable.
I changed: "last_conv_layer = model.get_layer("conv_pw_13")" to "last_conv_layer = model.get_layer("conv_pw_13_relu") "
and remove "heatmap = np.maximum(heatmap, 0)"

But doing this, the output is more accurate to me.

output, grads_val = gradient_function([image])

I get this error:

c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'input_1_1' with dtype float and shape [?,299,299,3]
   [[{{node input_1_1}}]]

I am running this on a different custom model

Why are you initializing the cam array with ones?

Running the script raises the following error:

File "grad-cam.py", line 35, in <lambda>
target_layer = lambda x: target_category_loss(x, category_index, nb_classes)
File "grad-cam.py", line 13, in target_category_loss
return tf.mul(x, K.one_hot([category_index], nb_classes))
File "theano_backend.py", line 726, in one_hot
input_shape = tuple((indices.shape[i] for i in range(indices.ndim)))
AttributeError: 'list' object has no attribute 'ndim'

Any ideas how to fix this?

能给下环境配置嘛，具体到tf的版本号，谢谢

Thank you for this immensely awesome demo. I think this technique opens many doors.

It took me a lot of fiddling to get reasonable output, since my network architecture is quite a bit different. All is good now. However I noticed that things are running on the CPU. In keras, my other scripts are working on the GPU with the tensorflow backend.

Would you happen to know offhand how to ensure the K.function and K.gradients calls in here run on the GPU. Im digging into the keras source to understand that they are just frontends for tensorflow functions, but I get a bit lost here.

Thanks!