trojai / trojai_rl Goto Github PK

when I use this command as below to run wrapped_boxing.py, It always stopped at about 33% of 50000000 frames.
Is it normal?

$ nohup python -u wrapped_boxing.py --save_loc "../results/" --model_save_name BoxingFC512Model1ForIpynb.pt --num_clean_test 30 --num_triggered_test 30 --gpu > BoxingFC512Model1ForIpynb.log 2>&1 &

There are a few issues with running notebooks/boxing_example.ipynb

1. trojai_rl has an older gym version (0.15.3) requirement, but it appears one needs gym[accept-rom-license]==0.21.0 to get the Atari ROMs. Consequently, the numpy requirement also needs to be updated.
2. trojai_rl.datagen.envs.wrapped_boxing_public should be trojai_rl.datagen.envs.wrapped_boxing and trojai_rl.modelgen.architectures.atari_architectures_public is perhaps trojai_rl.modelgen.architectures.atari_architectures
3. Running train_boxing_model() produces the following error: TypeError: only integer scalar arrays can be converted to a scalar index
   The problem seems to be in torch_ac_optimizer.py. The code below produces a torch array when using argmax
   if argmax_action: action = torch.argmax(dist.probs) # get 'best' decision else: action = dist.sample()
   Using action = action.numpy()[0] before passing it to the step function appears to fix the issue, since the step function is expecting a scalar.

With these changes, we get qualitatively similar results on the new model trained by the notebook and the pretrained ones.