How do i use Selector for a custom document? about active-qa HOT 6 CLOSED

You need to:
1 - add the new document to data/bidaf/shared.json;
2 - start the environment;
3 - query the environment using the doc id of the newly added document:
environment_fn = environment_client.make_environment_reward_fn(
FLAGS.environment_server_address)
f1_scores, _, answers = environment_fn(["Why the sky is blue?"], ["new doc id"])

from active-qa.

But data/bidaf/shared.json contains:
char2idx, word2idx. It doesn't have any parapgraphs.

Do you mean data/squad/shared_dev.json / shared_train.json ?
which are generated by running the preprocess code.

from active-qa.

Sorry, that is right, shared_train.json and shared_dev.json are the ones you need to modify.

from active-qa.

I have added the document(paragraph), word tokens, character tokens of the words to the shared_train.json, shared_dev.json. Previously there are 12635 documents. Now there are 12636 documents. So i queried the bidaf environment with 12635 id. It is showing the following error.

from active-qa.

@rodrigonogueira4 I have run the reformulator with the given checkpoints and now I am also trying to get answers from my custom document. Please if you could explain in detail how to convert custom data into squad data set format and the following steps to get the answers.

from active-qa.

Hello,
@rodrigonogueira4 @graviraja could you please tell me how to run the reformulator? I was able to run the demo.py file, but it just gives the answers to questions using bidaf environment server.

Thanks

from active-qa.