confusion on epochs about gearnet HOT 3 OPEN

Hi, thanks for the question. In the original experiment, we train all models for 200 epochs. When I organize the code, I find training for 50 epochs can yield similar performance as training for 200 epochs. So it should be fine to train for just 50 epochs.

from gearnet.

Thank you for your reply! I have another question, when I try to reproduce the downstream tasks like EC, you mentioned in other issues that the batch size should be set to 8 using single GPU. But I found the finetuning process with low GPU-Util. If there is enough memory in GPU, can I enlarge the batch size without influencing the performance of Gearnet?
Thanks for your help!

from gearnet.

Actually, I think the downstream performance is very sensitive to batch size. For the current config, I find batch_size=8 is the best setting. Typically, if you want to change the batch_size, you also need to change the learning rate and number of epochs to get similar performance. You can try other batch_size to see whether similar performance can be achieved.

from gearnet.