This repository contains the code for the paper "A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models".

Alessandro Stolfo*, Zhijing Jin*, Kumar Shridhar, Bernhard Schölkopf, Mrinmaya Sachan

• python==3.9
• pytorch==1.11
• pandas==1.4
• spacy==3.3.1
• transformers==4.20.0
• wandb

The intervention experiments can be run by setting the desired parameters in run_numeracy_exp.sh and then executing sh run_numeracy_exp.sh.

The parameters representing the different intervention/effect combinations are the following:

• 1b: DCE(N --> R)
• 2: TCE(N --> R)
• 3: DCE(S --> R)
• 4: TCE(T --> R)

The models on which this repo was tested are:

• distilgpt2
• gpt2
• gpt2-medium
• gpt2-large
• gpt2-xl
• EleutherAI/gpt-neo-1.3B
• EleutherAI/gpt-neo-2.7B
• EleutherAI/gpt-j-6B
• gpt3/text-davinci-002

For GPT-3, the organization key and secret key are necessary to access the OpenAI APIs. If you want to experiment with it, paste the keys as the first (organization) and second (secret) lines in the openai_keys.txt file.

All the other models are accessed through Huggingface Transformers.

GPT-Neo-2.7B requires a GPU with at least 24GB of memory, and GPT-J-6B requires a GPU with at least 32 GB of memory.

To obtain the data needed to plot the heatmaps reported in the paper, execute

python heatmap_experiments/heatmap_experiment.py [model] [device] [out_dir] not_random arabic [seed] statement [data_path] [max_n] disabled + [n_templates]"

This will store the average probabilities assigned to each possible groud-truth result in the range 0,...,max_n as a .csv file.

Part of this repo is based on the code for Finlayson et al. (2020)