jinensetpal / archimede Goto Github PK

Hey @jinensetpal ,

Raising an issue instead of sending an email since it is more practical for me to keep track of things inside Github. I was looking to your evaluation script and what can be probably done to make the evaluation of your approach more sound is the following:

• Shuffle the dataframe before taking the X and y; we may never know if entries are ordered for some reason, and this would greatly affect the performances of the model (probably not, but still worth doing as a good practice).

• I saw that you were loading some pickled test sets for evaluation: I guess those are a portion of the original training that you set apart for testing. Would you mind doing a 5-fold cross-validation instead? This would ensure that a) there's no data leakage between train and test and b) that current results aren't due to a "lucky split".

Best,

Gabriele