aks2203 / easy-to-hard-data Goto Github PK

Hi, I'm just starting to read your paper and review the code, and I'm wondering if the top
level README.md is a bit off.

eg here it says:
We sorted the chess puzzles provided by Lichess, and the first 600,000 easiest puzzles make up an easy training set. Testing can be done with any subset of puzzles with higher indices. The default test set uses indices 600,000 to 700,000.

but the chess test script uses batch size of 3000, thus it seems that the last sentence
should be "The default test set uses indices from 1.8M to 2.1M".
ref:
https://github.com/aks2203/easy-to-hard/blob/main/chess/launch/test_models.sh

I may of course misunderstand things, but I would expect that the data set, in isolation, is not
aware of the batch size.

thx,
Dave