Comments (2)
Hi @camigord,
Yes. So, this value is nothing special. Universe starter agent originally used tf.reduce_sum
instead of tf.reduce_mean
to compute the loss. The hyper-parameters in my A3C+ICM code were tuned to that setup. But it is a bad practice to sum across batch and channel dimensions. This is so because as one changes the environments, batch-size or network architecture, the other hyper-parameters will stop making sense. Hence, I switched out the tf.reduce_sum
with tf.reduce_mean
taking the constant factor out (e.g. 288, 20 etc.). This makes the code generalizable across different network architectures, input sizes and environments.
Moreover, to help the users understand the code better, I deliberately added this comment # TODO: historical accident ...
wherever the constants were factored out. Hope this answers your question.
from noreward-rl.
Alright, I have updated the comments in code to make them more informative. Hence, closing this issue now.
from noreward-rl.
Related Issues (20)
- Use this model with demo frames
- 安装报错
- a GAN idea HOT 1
- Feature normalization ?
- Question: I want to run a single worker HOT 2
- Tryinf to run simple atari pong HOT 1
- Convergence without LSTM
- Inconsistent actions between train and inference on Mario
- Questions about training
- Tried to run demo.py got syntax error HOT 3
- Generate maze code
- Could the algorithm be used on reinforced learning algorithms with experience reply?
- extrinsic and intrinsic combination HOT 1
- Failed to build doom-py HOT 2
- RuntimeError: go_vncdriver must be imported before tensorflow
- Tensor Board Version
- models.tar.gz cannot be downloaded HOT 3
- Update Code to Tensorflow 2.0
- What is the compatible python version?
- tensor flow not compatible and check sum issue in the requirement file
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from noreward-rl.