Paper list for logical reasoning

-- A survey paper will be released soon.

Key Words: premise, conclusion, argument, reasoning, inference, proposition, forward chaining, backward chaining, critical thinking, syllogism

Maintained by Hanmeng Liu, Ruoxi Ning

Welcome to contribute!

1. Natural Language Inference A Dissertation Jun 2009. paper

2. A Survey of Paraphrasing and Textual Entailment Method May 2010. paper

3. An Overview of Natural Language Inference Data Collection: The Way Forward? Oct 2017. paper

4. Logical Formalization of Commonsense Reasoning: A Survey Aug 2017. paper

5. Recent Advanced in Natural Language Inferenece: A Survey of Benchmarks, Resources, and Approahces Apr 2019. paper

6. A Survey on Recognizing Textual Entailment as an NLP Evaluation Nov 2020. paper

7. Symbolic And Neural Approaches to Natural Language Inference Jun 2021. paper

8. Towards Reasoning in Large Language Models: A Survey July 2023. paper

9. Towards LOGIGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models Mar 2024. paper

10. A Survey of Reasoning with Foundation Models Dec 2023. paper

11. Logical Reasoning over Natural Language as Knowledge Representation: A Survey Mar 2023. paper

12. Natural Language Reasoning, A Survey May 2023. paper

1. FaiRR: Faithful and Robust Deductive Reasoning over Natural Language May 2022. paper

2. Deductive Additivity for Planning of Natural Language Proofs June 2023. paper

3. AnaLog: Testing Analytical and Deductive Logic Learnability in Language Models July 2022. paper

4. Can Pretrained Language Models (Yet) Reason Deductively? May 2023 paper

5. RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners Dec 2022. paper

6. Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction May 2022. paper

7. A Generation-based Deductive Method for Math Word Problems Dec 2023. paper

8. GeoDRL: A Self-Learning Framework for Geometry Problem Solving using Reinforcement Learning in Deductive Reasoning July 2023. paper

9. Inductive and deductive inferences in a Crowdsourced Lexical-Semantic Network Sep 2013. paper

10. Questions require an answer: A deductive perspective on questions and answers Nov 2006 paper

11. Natural Language Deduction with Incomplete Information Dec 2022. paper

12. Natural Language Deduction through Search over Statement Compositions Dec 2022. paper

13. Hence, Socrates is mortal: A Benchmark for Natural Language Syllogistic Reasoning July 2023. paper

14. Teaching Machine Comprehension with Compositional Explanations Nov 2020. paper

15. Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples 2023 paper

16. Deductive Verification of Chain-of-Thought Reasoning 2023 paper

17. Natural Language Deduction with Incomplete Information Dec 2022. paper

1. Language Models as Inductive Reasoners Mar 2024. paper

2. Inductive and deductive inferences in a Crowdsourced Lexical-Semantic Network Sep 2013. paper

3. CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text Nov 2019. paper

4. Contrastive Learning for Inference in Dialogue Dec 2023. paper

5. Flexible Generation of Natural Language Deductions Nov 2021. paper

6. A Comprehensive Evaluation of Inductive Reasoning Capabilities and Problem Solving in Large Language Models March 2024. paper

7. I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation July 2023 paper

8. Learning Explainable Linguistic Expressions with Neural Inductive Logic Programming for Sentence Classification Nov 2020. paper

9. Incorporating Linguistics Constraints into Inductive Logic Programming 2000. paper

10. Contrastive Learning with Generated Representations for Inductive Knowledge Graph Embedding July 2023. paper

11. Inductive Relation Prediction with Logical Reasoning Using Contrastive Representations Dec 2022. paper

12. Deep Inductive Logic Reasoning for Multi-Hop Reading Comprehension May 2022. paper

13. Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphs July 2023. paper

14. It is not True that Transformers are Inductive Learners: Probing NLI Models with External Negation Mar 2024. paper

15. Thinking Like a Skeptic: Defeasible Inference in Natural Language Nov 2020. paper

1. Abductive Explanation-based Learning Improves Parsing Accuracy and Efficiency paper

2. ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language Aug 2021. paper

3. Case-Based Abductive Natural Language Inference Oct 2022. paper

4. Self-Consistent Narrative Prompts on Abductive Natural Language Inference Nov 2023. paper

5. Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations July 2023. paper

6. Abductive Commonsense Reasoning Dec 2019. paper

7. Multi-modal Action Chain Abductive Reasoning July 2023. paper

8. True Detective: A Deep Abductive Reasoning Benchmark Undoable for GPT-3 and Challenging for GPT-4 July 2023. paper

9. How well do SOTA legal reasoning models support abductive reasoning? Apr 2023. paper

10. AbductionRules: Training Transformers to Explain Unexpected Inputs May 2022. paper

11. Learning as Abduction: Trainable Natural Logic Theorem Prover for Natural Language Inference Dec 2020 paper

12. Simple Augmentations of Logical Rules for Neuro-Symbolic Knowledge Graph Completion July 2023. paper

13. LAMBADA: Backward Chaining for Automated Reasoning in Natural Language Dec 2022. paper

14. Forward-Backward Reasoning in Large Language Models for Mathematical Verification Sep 2023. paper

15. Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond Jun 2023. paper

1. ANALOGICAL - A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models July 2023. paper

2. Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performance Dec 2023. paper

3. E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning May 2022. paper

4. StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding Dec 2023. paper

5. In-Context Analogical Reasoning with Pre-Trained Language Models July 2023. paper

6. Analogical Math Word Problems Solving with Enhanced Problem-Solution Association Dec 2022. paper

7. A Neural-Symbolic Approach to Natural Language Understanding Dec 2022. paper

8. ThinkSum: Probabilistic reasoning over sets using large language models July 2023 paper

9. Relevant or Random: Can LLMs Truly Perform Analogical Reasoning? Apr 2024. paper

1. Detecting Argumentative Fallacies in the Wild: Problems and Limitations of Large Language Models Dec 2023. paper

2. Multitask Instruction-based Prompting for Fallacy Recognition Dec 2022 paper

3. The Search for Agreement on Logical Fallacy Annotation of an Infodemic Jun 2022 paper

4. Logical Fallacy Detection Dec 2022 paper

5. Breaking Down the Invisible Wall of Informal Fallacies in Online Discussions Aug 2021 paper

6. Argument-based Detection and Classification of Fallacies in Political Debates Dec 2023 paper

1. TakeLab at SemEval-2018 Task12: Argument Reasoning Comprehension with Skip-Thought Vectors Jun 2018. paper

2. The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants Jun 2018 paper

3. Automatically Identifying Implicit Arguments to Improve Argument Linking and Coherence Modeling Jun 2013 paper

4. Uncovering Implicit Inferences for Improved Relational Argument Mining May 2023. paper

5. Implicit Premise Generation with Discourse-aware Commonsense Knowledge Models Nov 2021. paper

1. NaturalLI: Natural Logic Inference for Common Sense Reasoning Oct 2014. paper

2. Neural Natural Logic Inference for Interpretable Question Answering Nov 2021. paper

3. Natural Logic for Textual Inference Jun 2007. paper

4. Neuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference 2022 paper

5. NatLogAttack: A Framework for Attacking Natural Language Inference Models with Natural Logic July 2023. paper

6. Higher-order logical inference with compositional semantics Sep 2015 paper

7. NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning Aug 2021. paper

8. Monotonic Inference for Underspecified Episodic Logic Jun 2021. paper

9. A (Mostly) Symbolic System for Monotonic Inference with Unscoped Episodic Logical Forms Jun 2021. paper

10. Logical Inferences with Comparatives and Generalized Quantifiers July 2020. paper

11. LogicAttack: Adversarial Attacks for Evaluating Logical Consistency of Natural Language Inference Dec 2023. paper

12. Decomposing Natural Logic Inferences for Neural NLI Dec 2022. paper

13. Natural Language Inference with Monotonicity May 2019. paper

14. QA-NatVer: Question Answering for Natural Logic-based Fact Verification Dec 2023. paper

15. Combining Natural Logic and Shallow Reasoning for Question Answering Aug 2016. paper

16. Semantic and Logical Inference Model for Textual Entailment Jun 2007. paper

17. MonaLog: a Lightweight System for Natural Language Inference Based on Monotonicity 2020 paper

18. PLOG: Table-to-Logic Pretraining for Logical Table-to-Text Generation Dec 2022. paper

19. Pragmatic and Logical Inferences in NLI Systems: The Case of Conjunction Buttressing July 2022. paper

20. Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models Dec 2022. paper

21. Synthetic Dataset for Evaluating Complex Compositional Knowledge for Natural Language Inference Jun 2023. paper

22. A Tableau Prover for Natural Logic and Language Sep 2015. paper

23. ProoFVer: Natural Logic Theorem Proving for Fact Verification 2022. paper

24. Knowledge-Guided Linguistic Rewrites for Inference Rule Verification Jun 2016 paper

25. Augmenting Neural Networks with First-order Logic paper

26. Neural Unification for Logic Reasoning over Natural Language Nov 2021. paper

27. Recognising Textual Entailment with Logical Inference Oct 2005. paper

28. NLog-like Inference and Commonsense Reasoning 2014. paper

29. Conditional Natural Language Inference Dec 2023. paper

1. From legal to technical concept: Towards an automated classification of German political Twitter postings as criminal offenses June 2019. paper

2. Learning Logical Structures of Paragraphs in Legal Articles Nov 2011. paper

3. Can AMR Assist Legal and Logical Reasoning? Dec 2022. paper

4. Syllogistic Reasoning for Legal Judgment Analysis Dec 2023. paper

5. How well do SOTA legal reasoning models support abductive reasoning? Apr 2023. paper

1. Critical Thinking for Language Models June 2021. paper

1. LangPro: Natural Language Theorem Prover Sep 2017. paper

2. A Tableau Prover for Natural Logic and Language Sep 2015 paper

1. Using the Framework (FraCaS) 1996.

2. The PASCAL Recognizing Textual Entailment Challenge 2005. paper

3. The Second PASCAL Recognizing Textual Entailment Challenge 2006. paper

4. The Third PASCAL Recognizing Textual Entailment Challenge 2007. paper

5. Natural Logic for Textual Inference Jun 2007. paper

6. The Winograd Schema Challenge 2011. paper

7. The Fourth PASCAL Recognizing Textual Entailment Challenge 2008. paper

8. SemEval-2014 Task 1: Evaluzation of Compositional Distributional Semantic Models on Full Sentence through Semantic Relatedness and Textual Entailment (SICK) Aug 2014. paper paper

9. A Large Annotated Corpus for Learning Language Inference (SNLI) Sept 2015. paper

10. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference (MNLI) Jun 2018. paper

11. Lessons from Natural Language Inference in the Clinical Domain Aug 2018. paper

12. GLUE: A Multi-task Benchmark and Analysis Platform for Natural Language Understanding Feb 2019. paper

13. Inherent Disagreement in Human Textual Inferences May 2019. paper

14. Entailment as Few-Shot Learners Apr 2021. paper

15. CURRICULUM: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding July 2022. paper

16. Synthetic Dataset for Evaluating Complex Compositional Knowledge for Natural Language Inference Jun 2023. paper

1. BiQuAD: Towards QA based on deeper text understanding Aug 2021. paper

2. StrategyQA paper

1. Reading Comprehension as Natural Language Inference: A Semantic Analysis Dec 2012. paper

2. Transforming Question Answering Datasets Into Natural Language Inference Datasets Sep 2018. paper

1. Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summerization Jul 2022. paper

2. SUMMAC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization Aug 2021. paper

1. TABFACT: A Large-scale Dataset for Table-base Fact Verification Sep 2019. paper

2. INFOTABS: Inference on Tables as Semi-structured Data Jul 2020. paper

3. SemEval-2021 Task 9: Fact Verification and Evidence Finding for Tabular Data in Scientific Documents (SEM-TAB-FACTS) Aug 2021 . paper

4. Is My Model Using the Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning Sep 2021. paper

5. The Fact Extraction and VERification Over Unstructured and Structured information (FEVEROUS) Shared Task Nov 2021. paper

1. Fake News Challenge Dec 2016. webpage

2. FEVER: A Large-scale Dataset for Fact Extraction and VERification Jul 2018. paper

1. A Probabilistic Classification Approach for Lexical Textual Entailment 2005. paper

2. Uncertain Natural Language Inference Sep 2019. paper

1. Natural Language Inference in Context — Investigating Contextual Reasoning over Long Texts Nov 2011. paper

2. Evaluating the Factual Consistency of Abstractive Text Summarization (FactCC) Nov 2020. paper

3. DOCNLI: A Large-scale Dataset for Document-level Natural Language Inference Aug 2021. paper

4. ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts Nov 2021. paper

5. Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization Jul 2022. paper

6. BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examples Oct 2022. paper

7. Validity Assessment of Legal Will Statement as Natural Language Inference Oct 2022. paper

8. LawngNLI: A Multigranular, Long-premise NLI Benchmark for Evaluating Models’ In-domain Generalization from Short to Long Contexts Dec 2022. paper

1. Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation Oct 2018. paper

2. Can neural networks understand monotonicity reasoning? (MED) 2019. paper

3. HELP: A Dataset for Identifying Shortcomings of Neural Models in Monotonicity Reasoning Jun 2019. paper

4. Transformers as Soft Reasoners over Language Feb 2020. paper

5. Adversarial NLI: A New Benchmark for Natural Language Understanding Jul 2020. paper

6. Are Natural Language Inference Models Impressive? Learning Implicature and Presupposition Jul 2020. paper

7. Natural Language Inference in Context — Investigating Contextual Reasoning over Long Texts Nov 2020. paper

8. ConjNLI: Natural Language Inference Over Conjunctive Sentences Nov 2020. paper

9. TaxiNLI: Taking a ride up the NLU hill Nov 2020. paper

10. Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI Nov 2021. paper

11. Explaining Answers with Entailment Trees (ENTAILMENTBANK) May 2022. paper

12. FOLIO: Natural Language Reasoning with First-Order-Logic Sep 2022. paper

1. Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation Apr 2020. paper

2. An Analysis of Natural Language Inference Benchmarks through the Lens of Negation Nov 2020. paper

3. Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation Nov 2022. paper

1. The Winograd Schema Challenge 2011. paper

2. WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation Jan 2022. paper

1. Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (HANS) Feb 2019. paper

1. FraCaS: Temporal Analysis Dec 2020. paper

2. Explaining Answers with Entailment Trees (ENTAILMENT BANK) Nov 2021. paper

1. Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming July 2023. paper

2. Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning Dec 2023. paper

3. Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming July 2023. paper

4. Analytical, Symbolic and First-Order Reasoning within Neural Architectures July 2021. paper

5. LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers Dec 2023. paper

6. Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinement March 2024. paper

7. Investigating Transformer guided Chaining for Interpretable Natural Logic Reasoning July 2023 paper

8. Conversational Multi-Hop Reasoning with Neural Commonsense Knowledge and Symbolic Logic Rules. Sept. 2021. paper.

9. Are Pretrained Language Models Symbolic Reasoners over Knowledge?. 2020. paper.

10. Differentiable Reasoning on Large Knowledge Bases and Natural Language. Apr. 2020. paper.

11. Learning Symbolic Rules for Reasoning in Quasi-Natural Language. Nov. 2021. paper.

1. Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text May 2021. paper

2. MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning Mar 2022. paper

1. LogiQA2.0 - An Improved Dataset for Logic Reasoning in Question Answering and Textual Inference May 2023. paper dateset

2. ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning 2023. paper ReClor

3. MTR: A Dataset Fusing Inductive, Deductive, and Defeasible Reasoning July 2023. paper [dataset not available]

4. AR-LSAT: Investigating Analytical Reasoning of Text Apr 2021. paper AR-LSAT

5. CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text Sep 2019. paper CLUTRR

6. Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies 2021 paper StrategyQA

1. FOLIO: Natural Language Reasoning with First-Order Logic 2022. paper dataset

2. Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI 2021. paper LogicNLI

3. Transformers as Soft Reasoners over Language Feb 2020. paper ruletaker

1. GLoRE: Evaluating Logical Reasoning of Large Language Models. Oct 2023 paper dataset

2. Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models paper dataset

3. LogicBench: A Benchmark for Evaluation of Logical Reasoning. Dec 2023 paper

5. RobustLR: A Diagnostic Benchmark for Evaluating Logical Robustness of Deductive Reasoners Dec 2022. paper

6. Hence, Socrates is mortal: A Benchmark for Natural Language Syllogistic Reasoning July 2023. paper

7. RuleBERT: Teaching Soft Rules to Pre-Trained Language Models Nov 2021 paper

8. PuzzLing Machines: A Challenge on Learning From Small Data July 2020. paper

9. CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text Nov 2019. paper

10. True Detective: A Deep Abductive Reasoning Benchmark Undoable for GPT-3 and Challenging for GPT-4 July 2023. paper

11. AbductionRules: Training Transformers to Explain Unexpected Inputs May 2022. paper

12. E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning May 2022. paper

13. StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding Dec 2023. paper

14. How susceptible are LLMs to Logical Fallacies? Aug 2023. paper LOGICOM

1. LLaMA-7B-LogiCoT

2. LOGIPT paper

3. Symbol-LLM

1. Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond March 2021. paper

2. Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4 Apr 2023. paper

3. A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity Nov 2023. paper

1. LogiTorch: a PyTorch-based library for logical reasoning on natural language

2. GLoRE: a benchmark for evaluating the logical reasoning of LLMs

3. LogiGLUE

4. Logiformer: This is a model to tackle the logical reasoning task in the field of multiple-choice machine reading comprehension. The code of the decoder part is not the final version, but it is one of the alternatives. You can also implement it based on own design, which may further improve the experimental results.

5. Awesome Natural Language Reasoning Papers: A collection of research papers related to Knowledge Reasoning with Natural Language Models.

6. Awesome LLM Reasoning: Curated collection of papers and resources on how to unlock the reasoning ability of LLMs and MLLMs.

7. Deep-Reasoning-Papers: Recent Papers including Neural Symbolic Reasoning, Logical Reasoning, Visual Reasoning, natural language reasoning and any other topics connecting deep learning and reasoning.

8. Awesome deep logic: Must-Read Papers or Resources on how to integrate symbolic logic into deep neural nets.

9. Awesome-Reasoning-Foundation-Models

1. Workshop on Natural Language Reasoning and Structured Explanations (2023) link

1. ART: rule bAsed futuRe-inference deducTion Dec 2023. paper

2. Multimodal Fallacy Classification in Political Debates March 2024 paper

1. A Benchmark for Compositional Visual Reasoning 2022 paper

2. Emergent Communication for Rules Reasoning 2023 paper

3. Multimodal Logical Inference System for Visual-Textual Entailment July 2019. paper

4. The Abstraction and Reasoning Corpus (ARC) dataset