How to handle multi label multiclass? about flashtorch HOT 3 CLOSED

Okay thanks. This is pretty much what I was doing and just wanted to make sure I was on the right track and get your insight.

from flashtorch.

I think you can hack the script at backprob.py to generate the target vector you are looking for:

    def calculate_gradients(self,
                            input_,
                            target_class=None,
                            take_max=False,
                            guided=False,
                            use_gpu=False):

       (...some unrelated code...)

            target = torch.FloatTensor(1, output.shape[-1]).zero_()

      (...some unrelated code...)

            target[0][top_class] = 1

and change that to something like:

    def calculate_gradients(self,
                            input_,
                            target_vector=None, <--- your target vector goes here
                            take_max=False,
                            guided=False,
                            use_gpu=False):
            
          if isinstance(target_class, torch.tensor):
                  target = target_vector  # in this case, you have to generate the target-vector yourself. 
          else:
                 (continue as before)

Then you also have to change the respective parent functions that call this function such that you can pass your own target vector, but this should be easy.

In that way you can set a arbitrary target vector, which can also contain multiple "1"-entries. This might achieve what you are looking for. I am not deep enough in the math behind saliency maps though. i hope @MisaOgura can comment on this.

from flashtorch.

Hi @rrags, apologies for the late reply and thanks @dnns92 for jumping in.

The current behaviour is that if target_class and top_class are different, it defaults to visualise the gradients w.r.t the top_class (see here). The simplest hack to achieve what you want to do would be to change backprop.py#L117 from target[0][top_class] = 1 to target[0][target_class] = 1.

Allowing users to have more control over which class to visualise, is something I have been wanting to revisit - please expect an update on this.

@dnns92 The modification suggested would make sense when target_vector contains only one value of 1 (i.e. one target class). When there is more than one target class, .backward() would calculate gradients w.r.t multiple target classes at once, and you will only get one set of gradients out of the graph. This probably wouldn't be the desired behaviour when creating per-class saliency map, so multiple targets would need to be handled differently. Hope this helps.

from flashtorch.