Interpretation of Kernel SHAP and its hyperparameters about shap HOT 3 CLOSED

Good questions!

1. All the background examples are evaluated for each sample. In the case you describe there are n * k evaluations of the predictor (passed as a batch to the provider predictor function). I should also note that if k >= 2^d then the entire space is evaluated directly (rather than random sampling subsets from the kernel distribution) and this leads to an exact computation.

2. Very close except for step 2. As mentioned above, in step 2 we produce n copies with each copy using a different x_i from the background dataset. The expected value of the prediction is taken over these n copies.

3. The paper presents two approximations to conditional expectations that make them easier to compute. The first is feature independence, which allows us to integrate out each feature independent of the others, and the second is model linearity (as an approximation), which allows us to only provide a single reference value for the background. Tree SHAP does not make either of these assumptions (it just assumes the tree has captured any feature dependence), but Kernel SHAP as implemented here always assumes feature independence, and may assume linearity depending on how you use it. If you just pass one reference value for the background then you are assuming linearity, if you pass multiple samples then it will integrate over them to provide a better approximation.

I should make this clearer in the docs, but I recommend using either a single background data point, a small random subset of the true background, or for the best performance a set of k-medians (weighted by how many training points they each represent) designed to represent the background succinctly. Passing a large background dataset wastes lots of effort. Perhaps I'll add a warning about that to the code.

from shap.

I see. It is much clearer now. Thanks!

from shap.

I should also mention that the choice to use a single background reference or multiple samples depends on the type of input data. For images, integrating over a few instances is unlikely to be any better an approximation that just using a single reference. But for structured data using 20 weighted k-medians or a small random subsample might help.

from shap.