TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task [Paper]
| Path | Description |
|---|---|
| dataset/ | The experiment notebooks expect the patched TACRED dataset splits to be stored here. |
| notebooks/ | This directory contains the notebooks that we used to produce the results in the paper. |
| patch/ | This directory contains the patches for dev and test split of the original TACRED. |
| results/ | This directory contains the predictions of all models on dev (dev_results/) and test split (test_results/). |
| scripts/ | This directory contains scripts, e.g., to apply the patch to TACRED dev or test split. |
The code is tested with:
- Python 3.7
- Mysql Config 5.7 (required by Errudite)
# Minimal dependency install (Ubuntu) sudo apt install default-libmysqlclient-dev
git clone https://github.com/DFKI-NLP/tacrev
cd tacrev
pip install -r requirements.txt # only necessary for notebookspython scripts/apply_tacred_patch.py \
--dataset-file <TACRED DIR>/dev.json \
--patch-file ./patch/dev_patch.json \
--output-file ./dataset/dev_rev.jsonmd5 checksum of patched dev split: ce23ba10ca15bde94a3f733679bf1b05
python scripts/apply_tacred_patch.py \
--dataset-file <TACRED DIR>/test.json \
--patch-file ./patch/test_patch.json \
--output-file ./dataset/test_rev.jsonmd5 checksum of patched test split: dbcce82f5ab67fbfd1062db6cc6b66cd
- notebooks/tables.ipynb produces the tables presented in the paper.
- notebooks/error_groups.ipynb produces the error group bar chart from the paper.
If you find the code or dataset patch helpful, please cite the following paper:
@inproceedings{alt-etal-2020-tacrev,
title={TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task},
author={Christoph Alt and Aleksandra Gabryszak and Leonhard Hennig},
year={2020},
booktitle={Proceedings of ACL},
url={https://arxiv.org/abs/2004.14855}
}
The code is released under the under terms of the MIT License.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent