Git Product home page Git Product logo

a3c's Introduction

A3C

Deep reinforcement learning using an asynchronous advantage actor-critic (A3C) model written in TensorFlow.

This AI does not rely on hand-engineered rules or features. Instead, it masters the environment by looking at raw pixels and learning from experience, just as humans do.

Dependencies

  • NumPy
  • OpenAI Gym 0.10
  • Pillow
  • SciPy
  • TensorFlow 1.0

Learning Environment

Uses environments provided by OpenAI Gym.

Preprocessing

Each frame is transformed into a 47×47 grayscale image with 32-bit float values between 0 and 1. No image cropping is performed. Reward signals are restricted to -1, 0 and 1.

Network Architecture

The input layer consists of a 47×47 grayscale image.

Four convolutional layers follow, each with 32 filters of size 3×3 and stride 2 and each applying the rectifier nonlinearity.

A recurrent layer follows, consisting of 256 LSTM units.

Lastly, the network diverges into two output layers – one is a probability distribution over actions (represented as logits), the other is a single linear output representing the value function.

Acknowledgements

Implementation inspired by the OpenAI Universe reference agent.

Heavily influenced by DeepMind's seminal paper 'Asynchronous Methods for Deep Reinforcement Learning' (Mnih et al., 2016).

a3c's People

Contributors

andreimuntean avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

a3c's Issues

Score never improves

I've been training for something like 2.5 days now and have yet to improve beyond the following:

[2017-04-07 10:06:20,289] Finished episode. Total reward: -21. Length: 1245.

I'm using default hyperparameters, just using sh train.sh to launch training. Computer is an Intel Core i7-2600 CPU @ 3.40GHz. Eight threads run on my system -- do I need more than that to see improvement?

I did make a couple of changes, which I don't think should have affected the result but listing them here just in case:

  • Changed instances of python3 in train.sh to python (so that it would run with Python 2.7)
  • Added config.gpu_options.allow_growth = True after each invocation of tf.ConfigProto() (I am running on a GPU-enabled TF install and without the change, each thread would try to reserve almost all of the GPU memory, which would cause each thread after the second or third to fail with a CUDA out of memory error even though the code doesn't ever use the GPU.)

Any advice?

Thanks for the code what commands ..

WHat commands are required to run a small basic example. I have installed the pre-reqisites in an anaconda enviroment, but new to this process. So run ./train.sh

But how do I visualize what is happening, is it possible to perform simple Q learning for aari games on a CPU with this code?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.