Test run for panx failing on subset of langs about xtreme HOT 3 OPEN

Hi @keratin7 , Thanks for posting the detail information here!

I notice that the error is due to empty examples in the data. You can see the output from your provided error message:

example.langs [] [] 0 ex_index 1 10634

I've updated the preprocessing script to remove some empty examples from the original file. You probably used an out-dated version of the code. I just check that the correct number of English examples should be 10004 rather than 10634 in your above log, and the correct number of German examples should be 10000.

Please use the current download script to process the panx data, and run the prediction by setting only "--do_prediction" in the script again.

from xtreme.

Thanks for getting back to me @JunjieHu!

I'm using the latest version of the code.
I couldn't find the AmazonPhotos.zip file on the shared cloud drive so I manually downloaded the en and de datasets and proprocessed them using this script before running the training and prediction.

Is the link provided in the download script as intended?

from xtreme.

Aah. I see that when the whole folder is downloaded, it saves as AmazonPhotos.zip, my bad. In any case, I downloaded individual en and de folders and preprocessed them using the same scripts. I still get the same error. Could you elaborate on what example.langs is supposed to have?

from xtreme.