(Paper): Conditional Expectation Calculation & Inverse Function Questions about shap HOT 4 CLOSED

Good questions!

1. Equation 12 is a further simplification of Equation 11, and depending on what you pass as a background distribution dataset Kernel SHAP could use either approximation (at least as implemented here). Passing a background dataset with just a single sample does exactly what you guessed, it replaces all the features with a 0 in z' with the value from the passed background dataset sample, and leaves all the features with a 1 in z' unchanged. If however you pass many samples then it repeats the process for each background dataset sample and then averages the resulting model outputs to get the expected model output over the samples. In practice you can weight the samples and so pass a set of k weighted medians rather than an entire dataset to speed up the process.
2. Thanks!!! That's a typo from a last-minute notational change I made. It should not be inverted. Fortunately we are still in the NIPS update period so I'll fix that.

from shap.

Okay, that makes sense!

Out of curiosity, what do you think is the best way to approach the "superpixels" simplified features used in the original LIME? Since there doesn't seem to be an obvious definition of what it means to take the "expected" superpixel over the distribution of possible superpixels. And it seems like there are arguments for either setting all of those pixels to the global pixel average, or else the pixel average within that superpixel. My suspicion is that the practical difference isn't dramatic, but I am curious if your intuitions on this problem cause you to lean in one direction or the other.

from shap.

Super pixels are tricky. The ideal would be if you built a conditional generative model that could impute arbitrary parts of an image conditioned on the rest. If you had this then you could draw samples from it and integrate over them without the independence assumption of Eq 11 or the linearity assumption of Eq 12.

In practice that would be computationally challenging and I have not seen anyone do that (since I don't focus on images I have not tried either). So given that as the ideal, I would say do whatever approximation to it that you prefer. I imputed with the average of the surrounding super pixels for my experiments and took that as the single reference value (no integration).

from shap.

This issue has been inactive for two years, so it's been automatically marked as 'stale'.

We value your input! If this issue is still relevant, please leave a comment below. This will remove the 'stale' label and keep it open.

If there's no activity in the next 90 days the issue will be closed.

from shap.