Comments (8)
Looks like it's currently not using GPU efficiently #21
from dqn-tensorflow.
I ran some experiments a while ago, and observed the same thing.
In my experiments (I did not using this repo), everything other than underlying NN library were same, and mini-batch was fed to GPU in the same manner (send mini-batch every time network is updated.) and yet Theano was faster than Tensorflow.
According to #2919 #3377, Tensorflow's seesion.run
method does things more than just feeding data to GPU. thus I guess that is adding overhead and making the training slower than Theano.
from dqn-tensorflow.
@mthrok The two issues are saying that feed_dict is slow (not session.run is slow). It's actually a good practice to avoid using feed_dict inside training loops to reduce overhead compared to other frameworks.
from dqn-tensorflow.
Has anyone solve the training slower problems? in my case, it almost 600hours training.
from dqn-tensorflow.
@ppwwyyxx So nice to see you here, Tensorpack author, why this repo's performance differ so much from your samples in tensorpack. I don't see major difference but this repo collect experince replay in the same thread with training. But does it matter? Or because you used ROM directly?
from dqn-tensorflow.
I don't know why. Maybe the use of feed_dict is the major reason.
Using thread improves speed but not significant in my case. Using rom should make no difference to speed.
from dqn-tensorflow.
@ppwwyyxx I failed to describe the issue clearly. The issue I run into this reop is the training best rewards is 30, no where to compare with the sample in your repo. I compared code but don't see major difference. I changed your sample by replacing ROM direcly with Gym, no code change with network or training, unfortunatly, the output from your code is as bad as this repo, the best reward simply 50 and doesn't make any progress any more after million steps.
So I guess the Gym envirement itself has bugs. But ROM is free of the issue, I though you were aware of this issue so you use ROM instead of Gym
from dqn-tensorflow.
It's not a bug. Gym environments (**-v0) is just a harder setting because it has more randomness. You can use other gym settings e.g. BreakoutDeterministic-v4 is closest to a naive atari wrapper.
However even with Breakout-v0, you should still be able to see better performance than 50.
from dqn-tensorflow.
Related Issues (20)
- Trouble loading checkpoints HOT 1
- How to run for more epochs HOT 1
- History is not updated with new game screen created after a terminal state is reached HOT 2
- CUDA_ERROR_OUT_OF_MEMOR HOT 1
- KeyError: '__flags' HOT 4
- unsupported operand type(s) for +: 'dict_values' and 'list HOT 3
- error memory HOT 1
- No updates to the learning rate
- clipped error function is not correct
- why the content is not match the readerme?
- How to run on Windows? HOT 2
- what's the difference between simple game and detail game
- about starting a new game and History HOT 1
- terminal in agent.py seem not handle properly HOT 1
- Any one who can share model details? HOT 3
- AssertionError: Cannot call env.step() before calling reset() HOT 1
- load failed HOT 3
- How to play the code?
- why the Breakout-v0's action_size is 4 ,but in the checkpoint is 6 HOT 1
- Ram consumption HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dqn-tensorflow.