mrqa's Introduction

Domain-agnostic Question-Answering with Adversarial Training

Code for our paper "Domain-agnostic Question-Answering with Adversarial Training" which is accepted by EMNLP-IJCNLP 2019 MRQA Workshop.

Data Preparation

Download the original data

Download the data by running shell file.
Then run the code. Preprocessed train data will be created before training (It will takes quite a long time)

$ cd data
$ ./download_data.sh

(Optional) Download the pickled data (for fast data loading)

Download the pickled data from this link.
Unzip the zipfile on the root directory.

.
├── ...
├── pickled_data_bert-base-uncased_False
│   ├── HotpotQA.pkl
│   ├── NaturalQuestions.pkl
│   ├── NewsQA.pkl
│   ├── SQuAD.pkl
│   ├── SearchQA.pkl
│   └── TriviaQA.pkl
└── ...

Arguments should be same as below if you use pickled data. If you want to change one of these two arguments.

parser.add_argument("--bert_model", default="bert-base-uncased", type=str, help="Bert model")
parser.add_argument("--skip_no_ans", action='store_true', default=False, help="whether to exclude no answer example")

Requirements

Please install the following library requirements specified in the requirements.txt first.

torch==1.1.0
pytorch-pretrained-bert>=0.6.2
json-lines>=0.5.0

Model Training & Validation

$ python3 main.py \
         --epochs 2 \
         --batch_size 64 \
         --lr 3e-5 \
         --do_lower_case \
         --use_cuda \
         --do_valid \
         --adv \
         --dis_lambda 0.01

If you are using uncased bert model, give the option --do_lower_case.
If you want to do validation, give the option --do_valid.

Reference

@misc{lee2019domainagnostic,
    title={Domain-agnostic Question-Answering with Adversarial Training},
    author={Seanie Lee and Donggyu Kim and Jangwon Park},
    year={2019},
    eprint={1910.09342},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}