Recommended Alphazero training config parameters about open_spiel HOT 6 CLOSED

Thank you very much for the responses, i'll try these tips out and if i find anything interesting, will respond with my findings.

from open_spiel.

Hi,

Which AlphaZero: the python-only TF or C++ Libtorch based one?

@tewalds might have some insights for you. Timo, does this sound right?

from open_spiel.

I'm using the Libtorch based one because i thought the Tensorflow version is not in a usable state right now.

from open_spiel.

Thanks. The Python-only TF AZ should still work, it's just the C++ one that we never got to work externally.

In that case I think the best person to contact would be @christianjans, though I'm not sure if we ever tested this implementation thoroughly with GPUs. You can also try @mrdaliri who did his thesis using AZ on OpenSpiel Hex.

Unfortunately we don't have the time or resources to fully support a larger-scale AlphaZero so the ones in OpenSpiel are meant to be basic / correct example implementations. If you don't get it working, there are some larger-scale ones that can still be run on OpenSpiel games: see RLLib and muzero-general.

from open_spiel.

Quick heads-up, I contacted Tom Anthony who did his Ph.D. thesis in Hex and expect this to be faster. Possibly LibTorch is not properly setup to use cuda?

from open_spiel.

Hi, sorry for the late response on Libtorch AZ issues. I'm happy to hear your using it!

While working with Libtorch AZ in the past, I did try it with GPUs, but found that there wasn't a huge performance increase (at least with Clobber – the game I was playing).

I added the explicit_learning flag to alpha_zero_torch_example.cc which, when set to true, dedicates one GPU for NN weight updates, and the other GPUs for inference. I found that this sped up training, but I see that you already have this parameter set to true.

If you have 90 GB of memory, I wonder if an increased replay buffer size would be helpful in speeding up the training?

I also did a bit of a parameter writeup with the Python AZ which may be useful? I've attached it to this comment. I believe one of my findings there was that a high replay buffer size to replay buffer reuse ratio sped up the training.

The writeup was done in my early undergrad years so it definitely has its flaws haha. Let me know if you have any questions.

clobber_alphazero_writeup_2021.pdf

from open_spiel.