A question in the "match" module about katago HOT 4 OPEN

What are you setting nnMaxBatchSize to?

from katago.

It is the default value 32.

Sorry, since I always use the default nnMaxBatchSize value in gtp_example.cfg, I neglected to adjust this value in match.

I noticed that the recommended nnMaxBatchSize value in GTP is (numSearchThreads / numNNServerThreadsPerModel), so I guess the recommended value is 288/3=96 in my settings?

from katago.

I tried to adjust nnMaxBatchSize from 32 to 96 with the first form, however it seems that it is still ~10% slower than the second form with nnMaxBatchSize 32.

from katago.

That's interesting!

• What happens if you use only 1x4090 instead of 3x4090?
• What happens if you run a different number of games in parallel than 288?
• Do you have any GPUs other than 4090 that you can test this on?
• What happens if you run only 1 game at a time, but that game has a large number of search threads?

It wouldn't be appropriate to make this a general recommendation, and definitely not appropriate to modify the code to do it automatically, if we can't understand how general this is and what causes it. I think we probably wouldn't modify the code to do it automatically regardless, because loading the model multiple times on the GPU takes extra memory, which might not be suitable for some users. But it would still be interesting to learn more about this.

from katago.