wenhuchen / time-sensitive-qa Goto Github PK

Hello author. I was unable to reproduce the results in the paper.
I used the hyperparameters of provided in the github repository as well as the hyperparameters provided in the original article for several runs, and could not reproduce the results in the Easy part; In the hard part, I could obtain results similar to the paper.

In issue #5 Xinsu Also meet the same problem.
I was wondering if you would release the detail hyperparameters for training Easy part.
Thank you.

It seems that for the human-paraphrased sets, the easy and hard splits contain the same data. Is this just how it's constructed or is there a mistake in the data release?

Hello, It looks like the paragraphs field of the examples includes only the first 100 paragraphs. I wonder if I could get the dataset with full paragraphs. Thank you!

Hello, sorry, I have one more question. I was wondering if it's possible that you could release trained checkpoints, especially FiD models trained on the easy version of the dataset.

hi team! can you please provide us with the cleaned version of dataset. The data that you have provided seems very complex to understand. we could save lot of time in exploring and training.

The repository does not provide processed human_annotated train/test splits.
Since there is no fixed random seed in Process.ipynb, the generated human_annotated data will contain randomness.
Could the authors provide a deterministic , already processed human_annotation train/test split; or provide the seeds used in the experiments to generate datas , for a fair comparison in subsequent experiments
Thank you.