Source code and data for The Magic of IF: Investigating Causal Reasoning Abilities in Large Language Models of Code (Findings of ACL 2023)
- Python>=3.7
- openai
- backoff
Code and data of the abductive reasoning task are in the abductive/
folder.
call_codex.py
: prompt generation and prediction with Codexcall_davinci.py
: prompt generation and prediction with Davincitest_cleanup_no_label.json
: input of the test datatest_cleanup_ref.json
: labels of the test data The data files are downloaded from https://github.com/XiangLi1999/Diffusion-LM/tree/main/datasets/ROCstory/anlg/anlg
Code and data of the counterfactual reasoning task are in the counterfactual/
folder.
call_codex.py
: prompt generation and prediction with Codexcall_davinci.py
: prompt generation and prediction with Davincitest_data_new.json
: test data from https://github.com/qkaren/Counterfactual-StoryRW
Generation results of Codex are in the codex_output/
folder.
Please cite our paper if this repository inspires your work.
@article{liu2023magic,
title={The Magic of IF: Investigating Causal Reasoning Abilities in Large Language Models of Code},
author={Liu, Xiao and Yin, Da and Zhang, Chen and Feng, Yansong and Zhao, Dongyan},
journal={arXiv preprint arXiv:2305.19213},
year={2023}
}
If you have any questions regarding the code, please create an issue or contact the owner of this repository.