Does DoWhy have a function to validate a causal graph? about dowhy HOT 11 CLOSED

Got it. Let me try to start the implementation with something simple and then maybe you can contribute.

from dowhy.

Yes, there is now a causal graph refutation functions that checks the independence constraints. This notebook includes an example usage (see Step 4):

The documentation needs to be improved to make this easier to find.

from dowhy.

You can also take a look at: https://github.com/py-why/dowhy/blob/main/dowhy/gcm/validation.py#L24 and see the very recent discussion in #926. We are actually working on a new method there, hopefully can add this soon.

from dowhy.

Hey @fredthedead that's a great question. No, currently does not support it.
But this would be a great addition as a new refutation method. So given a causal model m, we can refute/test whether the dataset follows constraints from the causal model.

The trick is to find all falsifiable constraints that are entailed by a causal model. One methodic way is to consider all pairs, triplets and so on. Here's a start:

1. Enumerate over all pairs (edges) and check for non-zero correlation with a statistical test.
2. Enumerate all triplets in the graph, identify its type (path structure, confounder structure, collider structure, etc.) and then check whether the data follows expected correlation and independence constraints.

Pairs and triplets can be extended to quartets and so on. That said, it is still a heuristic because we leave out longer paths or nodes connected by more than 1-2 edges.

What do you think?

from dowhy.

Thanks for your reply @amit-sharma, much appreciated. The approach you describe is what I had in mind and was hoping it might of been implemented. I've started looking at the code a bit to estimate how much time it'll take me to add it, might give it a go next time I have some availability...

from dowhy.

Quick update regarding the novel refutation method mentioned by @bloebp. In #930 we added a function validate_lmc which tests the implied (conditional) independencies (via local Markov condition [LMC]) on some data (very similar to what @amit-sharma described). Additionally, we added a function falsify_graph to compare the result of this test to a baseline of random node-permutations to find whether the graph is significantly better than random. Here you can find an example notebook that highlights the key ideas and features of this function. We would greatly appreciate any feedback or comments! Please let me know if you have any questions!

from dowhy.

@amit-sharma @fredthedead was this implemented?

from dowhy.

Not yet unfortunately. Would you like to contribute @mauriciozuardi ?

from dowhy.

@amit-sharma I'm still familiarizing with the concepts (of causal analysis/inference) and the code, but as soon as I get confident enough I'll give it a try.

from dowhy.

Hi all, this is something that I e been looking for as well, and it seems that it would be a commonly used part of the causal analysis process. Any developments on this implementation?

from dowhy.

Much appreciated! @eeulig

from dowhy.