azariagmt / causality Goto Github PK

Tasks to undertake to implement proper MLOps pipeline

• Integrate CML into runs
• Integrate DVC and have different versions of the data(with gdrive remote)
• Incorporate MLFlow

• Modeling a given problem as a causal graph
• Statistical Modelling and Inference Extraction
• Building model pipelines and orchestration
• Knowledge about casual graph and statistical learning
• Hypothesis Formulation and Testing
• Statistical Analysis

• Conduct an exploratory data analysis on the data & communicate useful insights.
• Ensure that you identify and treat all missing values and outliers in the dataset by using appropriate methods.
• Perform feature extraction and scaling

• Split data into training and hold-out set
• Create a causal graph using all training data and get the insights (this will be considered the ground truth)
• Create new causal graphs using increasing fractions of the data and compare with the ground truth graph
  The comparison can be done with a Jaccard Similarity Index, measuring the intersection and union of the graph edges
• After reaching a stable causal graph, select only variables that point directly to the target variable
• Train one model using all variables and another using only the variables selected by the graph
• Measure how much each of the models overfit the hold-out set created in step 1.

Notebooks & Github codes

• Quickstart Notebook for using Causalgraphicalmodels python module: used to describe and manipulate Causal Graphical Models and Structural Causal Models.
• Introduction to CasualgraphicModel

Key Papers & blogs

• If correlation doesn’t imply causation, then what does?
• DoWhy: An End-to-End Library for Causal Inference (arxiv) - DoWhy package paper
• Mini course on Causality, Cambridge MIT
• Controlling Confounding Bias
• Slides on Causality
• Causality lecture note
• Confounding Bias
• Analysis of Breast Cancer Detection Using Different Machine Learning Techniques | SpringerLink
• ANALYSIS OF FEATURE SELECTION WITH CLASSFICATION: BREAST CANCER DATASETS

Talks & Videos

• Tutorial Session B – Causes and Counterfactuals: Concepts, Principles and Tools (Microft, 2014)
• Plenary 2: The Mathematics of Causal Inference: with Reflections on Machine Learning

General

• Causality by Judea Pearls 2nd edition
• Causal inference in statistics: An overview (2019)
• Main reference

Wikipedia

• Causal Graphical Models
• Structural Causal Models
• ​​Rubin causal model - Wikipedia
• Instrumental variables estimation - Wikipedia