Credible Interval size about tfcausalimpact HOT 2 CLOSED

Hi @DavidJ005 ,

Usually the wide intervals are an expression of a local level component with a wide standard deviation. Local levels are essentially random walks which means the more they are fit to predict the data the less capable the covariates and the model as a whole were to explain it.

In order words, in order to decrease its interval range, best approach would be to find better covariates and better models that explain your observed data. This is heuristic based and you'll have to try several hypothesis to find what works best. Maybe adding seasonal components will help, maybe finding new covariates or new model components will reduce this interval.

Another thing you can try is setting the prior for the local level standard deviation equal to 0.01. This only makes sense if you are confident that the covariates you used are a good fit and a local level component shouldn't be used much to explain observed data.

You can do so by instantiating the ci object with something like:

ci = CausalImpact(data, pre_period, post_period, model_args={'prior_level_sd': 0.01})

It may help you on this case, but it should only be done if you are confident that the covariates are good explanatory variables of the observed data.

Hope that helps,

Best,

Will

from tfcausalimpact.

Thanks for the help !
To give some feed back, I did try to lower the 'prior_level_sd', although I had to push it to 0,001.
The results are quite good, it tends to reduce the CI size to 6% (ie, >20% => <6%). In general it push the analyses in the edge of the middle area to the positive or negative while keeping some in the middle.

In my case I think it's pretty acceptable to set the hypothesis that the covariates are very explanatory. For example several shops in the same city are good covariates of each other in matter of sales under the hypothesis that they are not close enough to steal each other customers. Plus, in retail there is a lot of one-time events that can create local spikes (eg, a very sunny day, holidays, etc...) without having clear link with "Y", so ignoring these spikes is a very good thing to do since that they are difficult to replicate.

Anyway, it was very helpful.

Cheers !

David

from tfcausalimpact.