The generated memory.npy file is too large when training about dqn HOT 1 CLOSED

Hi,
Unfortunately, this is probably the case. Each element of the replay memory stores one transition, which consists of a current state, and a next state. Each state is 4 84x84 frames, for a total of 8 84x84 frames. When you have 500,000 of these transitions, and you try and store it to disk, it can take up quite a bit of space. My advice would be to eliminate the memory.npy saving, and just run the agent flat out.

Regarding the agent scoring zero, this is quite common. DQN performs quite poorly for a while. IF you look at slide 24 (http://joschu.net/docs/nuts-and-bolts.pdf), it notes that DQN is quite slow. On Breakout, I expect you'll see results after a few million frames (anywhere from 2 million to 5 million). Typically you have to leave it running overnight to see results. As such, a GPU is probably necessary.

Closing this issue, please reopen it if you have any more questions.

from dqn.