LIMIT-BERT: Linguistic Informed Multi-task BERT)
- Python 3.6 or higher.
- Cython 0.25.2 or any compatible version.
- PyTorch 1.0.0+.
- EVALB. Before starting, run
make
inside theEVALB/
directory to compile anevalb
executable. This will be called from Python for evaluation. - pytorch-transformers PyTorch 1.0.0+ or any compatible version.
The following pre-trained models are available for download from Google Drive:
LIMIT-BERT
: PyTorch version, same setting with BERT-Large-WWM,loading model with pytorch-transformers.
To train LIMIT-BERT, simply run:
sh run_limitbert.sh
To test after setting model path:
sh test_bert.sh
@article{zhou2019limit,
title={{LIMIT-BERT}: Linguistic informed multi-task {BERT}},
author={Zhou, Junru and Zhang, Zhuosheng and Zhao, Hai},
journal={arXiv preprint arXiv:1910.14296},
year={2019}
}