Model explanations for regression problems about alibi HOT 8 CLOSED

I'd love this feature!

Is anyone working on this? What would be the scope of implementing this?

from alibi.

Thanks for your interest @frederikfaye ! Support for regression problems is quite an open-ended area and there are many ways to do this, so the scope can be quite broad (e.g. even doing it the obvious way of binning the continuous response into discrete bins can be done in various ways depending on the use case). Do you have any particular use case that you have in mind?

from alibi.

When you say

binning the continuous response into discrete bin

do you mean such that the problem effectively becomes a multiclass classification problem?

I wanted to explain a trained CNN, and I was originally looking at something like a modified GradCAM (e.g. here (see the "Third attempt" section) or here), but the more I look at it, I don't think it would be that informative for my particular problem, as I don't have a naturally defined "center" of my regression target (like angle=0 in the linked-to steering angle problems). I guess SHAP would maybe be a solution, but if you have any other suggestions, please do let me know!

However, I now realize that the thing I would be most interested in would actually be a confidence method for regression models (so sorry that I hijacked this thread, which is about model explanations!).

As far as I can see, trust scores only work for classification tasks. I've instead been looking at the following two papers, but their implementation is pretty model-dependent: Bayesian Uncertainty Estimation for Batch Normalized Deep Networks and A Scalable Laplace Approximation for Neural Networks.
Again, do you have any suggestions? Or maybe these are not in the scope of what Alibi seeks to cover?

Best,
Frederik

from alibi.

do you mean such that the problem effectively becomes a multiclass classification problem?

Yes, if you can find an appropriate way of binning your response variable then you could use all these techniques, e.g. you should be able to use trust scores by binning the response without changing your underlying model.

That being said, confidence methods for regression models are definitely in the roadmap for Alibi, but not the immediate focus. It is a harder problem than classification because if our regressor only returns scalar values there is not much to work with. We are also on the look-out for interesting model-specific (white-box?) methods, in particular the Laplace approx. paper you linked looks very interesting if you have access to the full model.

I think SHAP would be interesting to try, I don't think I've seen an example with regression and image data.

Last note, there is a cool toolbox (https://github.com/albermax/innvestigate) which brings a lot of the methods for inspecting CNNs together (mostly classification though). Some things seem to work better than others, in particular one should be wary of saliency maps for use in explanations (https://arxiv.org/abs/1810.03292) !

from alibi.

Thanks for the suggestions, much appreciated!

I'm curious, what about the Laplace approx. paper caught your interest over the other paper?

from alibi.

Purely because I saw that the other one is BatchNorm specific, so wouldn't generalize to non-image applications! In any case I will have a closer look, also feel free to share any other research you come across as we are on the lookout for good ideas!

from alibi.

Right, makes sense - and will do! :)

from alibi.

ALE, IntegratedGradients and SHAP versions all support regression use cases so closing this for now. Will open a new issue if focus on regression use cases specifically becomes a more strategic direction.

from alibi.