Efficiency question about the model about tbd-nets HOT 2 CLOSED

Exactly, since the modules aren't the same for each question (and even the length of each question varies) we have to construct a question-specific network and forward input one by one. Ultimately, this doesn't become too much of a problem.

I don't recall exactly how long things took, but it was in the neighbourhood of 45 minutes per epoch on CLEVR's 700,000 questions. A safe bet is somewhere between 30 minutes to an hour. The bulk of the work was on two different machines: a server with K80 GPUs and Xeon E5-2698 CPUs and a desktop with 1080TIs and i7-7700K CPU.

You can see the full training setup in our notebook. To summarize: batch size is 128 and we use 2 workers in the PyTorch DataLoader. All of our work was on a single GPU. If training time is a very big concern (you need it to take less than the roughly one day training took us) there are some inefficiencies that I'm sure could be improved upon.

Side note: I went ahead and edited your comment to fix the link.

from tbd-nets.

Great, thank you for the answer!

from tbd-nets.