How can I implement Grad-CAM in video action Recognition?? about pytorch-grad-cam HOT 5 CLOSED

Hi,
Currently there is no support for running on a batch of images - it accepts a single image of size 1, 3, H, W (3 is color channel)

You will have to get the cam for every member of the batch separately.
For example, if the batch size is B, T, 3, C, H, W,

# input tensor is x

# i goes over the batch dimension
for i in range(x.size(0)):
	# j goes over the T dimension
	for j in range(x.size(1)):
	cam_grayscale = cam(input_tensor=x[i, j, :, :].unsqueeze(0),
                                    method="gradcam",
                                    target_category=target_category)

from pytorch-grad-cam.

@jacobgil HI Jacob,
if the input of the model is a clip of images with T as number of images in one clip, which other parts of Grad-Cam should be modified? I noticed that in base_cam, img is resized according to input tensor shape, which means that resizing will not work under this situation.

from pytorch-grad-cam.

@chengengliu Have you figured this out?

from pytorch-grad-cam.

@chengengliu Have you figured this out?

I have same quetion about grad-cam used for video recognition.

from pytorch-grad-cam.

@chengengliu Have you figured this out?

I have same quetion about grad-cam used for video recognition.

Hi Leo, sorry for the late reply. I’m sorry I cannot remember the details about how to use it since I did the project two years ago; but yes it is doable, I managed to use it frame by frame, so each image is saved with its CAM concatenation. Maybe try to change the input shape or something. The main flow would be the same.

Hope it will help you.

from pytorch-grad-cam.