1konny / gradcam_plus_plus-pytorch Goto Github PK

Hi, I've noticed there is no license associated to your repository. Could you, please, specify the license? Thank you

Hi,

Can you please tell me why do we need to upsampling to 224 in preprocess stage?

torch_img = F.upsample(torch_img, size=(224, 224), mode='bilinear', align_corners=False)

I used my own model and if I don't upsample to 224,224, the results seem not correct.

Thank you.

Hi I tried your code to visualize a trained densenet169 model.

But the code throws error

 heatmap, result = visualize_cam(mask, torch_img)
  File "/home/user/vizualize_Nets/utils.py", line 15, in visualize_cam
    heatmap = cv2.applyColorMap(np.uint8(255 * mask.squeeze()), cv2.COLORMAP_JET)
  File "/home/user/.local/lib/python3.6/site-packages/torch/tensor.py", line 460, in __array__
    return self.numpy().astype(dtype, copy=False)
TypeError: can't convert CUDA tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Simple converting to CUDA tensor or cpu().numpy() tensor does not solve the problem. Any idea how to solve it ?

Thanks in advance.

mask, _ = gradcam(inputs) File "/home/ashish18024/anaconda3/envs/dl/lib/python3.7/site-packages/gradcam/gradcam.py", line 88, in __call__ return self.forward(input, class_idx, retain_graph) File "/home/ashish18024/anaconda3/envs/dl/lib/python3.7/site-packages/gradcam/gradcam.py", line 70, in forward score.backward(retain_graph=retain_graph) File "/home/ashish18024/anaconda3/envs/dl/lib/python3.7/site-packages/torch/tensor.py", line 118, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/home/ashish18024/anaconda3/envs/dl/lib/python3.7/site-packages/torch/autograd/__init__.py", line 87, in backward grad_tensors = _make_grads(tensors, grad_tensors) File "/home/ashish18024/anaconda3/envs/dl/lib/python3.7/site-packages/torch/autograd/__init__.py", line 28, in _make_grads raise RuntimeError("grad can be implicitly created only for scalar outputs") RuntimeError: grad can be implicitly created only for scalar outputs

Hi! Thanks for your amazing work here. You've really made testing out gradcam a breeze.

I have a question regarding the usage of these modules. When I create a saliency map using a model I have fine-tuned as the architecture, are the resulting maps shown for the original torchvision pretrained network or my fine-tuned model?

Thanks in advance.

# model is fine-tuned on task-specific classification, initialized with pretrained resnet18 torchvision model weights

model.eval()
resnet_model_dict = dict(type='resnet', arch=model, layer_name='layer4',
                        input_size=(224,224))

resnet_gradcam = GradCAM(resnet_model_dict, True)
resnet_gradcampp = GradCAMpp(resnet_model_dict, True)

Dear @1Konny,

Thanks for your implementation!

I have detected that line 168 in gradcam.py:
alpha_denom = gradients.pow(2).mul(2) + \ activations.mul(gradients.pow(3)).view(b, k, u*v).sum(-1, keepdim=True).view(b, k, 1, 1)

should be:
global_sum = activations.view(b, k, u*v).sum(-1, keepdim=True).view(b, k, 1, 1) alpha_denom = gradients.pow(2).mul(2) + global_sum.mul(gradients.pow(3))

This is because of Eq. 19 in the paper [@adityac94]. If you pay attention, you need first to compute the sum over all {a, b} for each activation map k. Then you have a ponderation for each activation map k that you will use as a multiplier of all gradients {i, j} in the respective kernel k.

In your implementation, you first multiply each cell activation A{a,b,k} by its respective gradient {i, j, k} and then you sum over {i, j}, mixing the indices {a, b} and {i, j}, but they are independent.

I hope it's clear :)

PD: I have fixed this in gradcam.py and added a flag in example.ipynbto automatically detect if Cuda can be used, otherwise, use CPU. I will make you a "Pull request".

Thanks for your time!

Current versions of modules (January 2024).