Issue with panx preprocessing about xtreme HOT 7 CLOSED

Thanks for the note, Stefan! We'll take a look at the best way to fix this. cc @JunjieHu

from xtreme.

Hi Stefan, I've made a new pull request to fix this issue. Please use the new utils_preprocess.py file to preprocess the data once @sebastianruder merges the PR.

from xtreme.

Hi Stefan, I've merged Junjie's PR. Can you please verify that the latest code fixes this issue? I'm closing this issue. Please verify and reopen if it still persists.

from xtreme.

@melvinjosej and @JunjieHu Thanks for the fix :)

I've just tested it, but the modifications in the download_data.sh (see 37abe88#diff-91e0bbb7c7da353deb7078bb3e3ed818) will lead to errors, because cding to the panx_dataset won't work - but the previous commit (see #15) was working :)

from xtreme.

@stefan-it Ahh, you're right. Does this PR bring it back to a working state?
#20

I'll merge it in once you confirm.

from xtreme.

PR is working :)

I can also confirm that there are no consect. empty lines in the datasets, I just re-ran the prediction for all languages 👍

from xtreme.

Great. I've merged my PR. Thanks again @stefan-it for verifying :)

from xtreme.