Runing Time about dept HOT 2 CLOSED

Thank you for your help!

from dept.

Hi, thank you so much for your question.

The running time of each dataset/task depends on the size of the training data. For large datasets like MNLI, QQP, and QNLI, which have over 100,000 training examples, it often takes longer to train the model for optimal performance, potentially exceeding 10 hours on a single RTX 3090 GPU. This duration assumes no specific speed-enhancing training techniques are applied. Training times are shorter for datasets with fewer examples. In the few-shot learning setting, the training time is typically not very long, but we need to fine-tune the model on the source task before applying it to the target task, which introduces some additional training time.

In addition, we perform an exhaustive hyperparameter search for two learning rates, which introduces additional training costs. However, we will enjoy low memory and time costs during the inference.

from dept.