betomorrow / reimprovejs Goto Github PK
View Code? Open in Web Editor NEWA framework using TensorFlow.js for Deep Reinforcement Learning
License: MIT License
A framework using TensorFlow.js for Deep Reinforcement Learning
License: MIT License
Hi!
Great looking library! I have been able to produce some simple enough things using the 0.0.2 version of ReImproveJS, however a couple of issues that are preventing me from taking this any further;
NPM's latest version is 0.0.2, which doesn't have updated TS or TF and no helper for exporting the accumulated model.
0.2.5 release doesn't contain the compiled code, and when trying to build, I am met with the following error, meaning 0.2.5 is impossible for me to access.
node_modules/@tensorflow/tfjs-converter/dist/src/data/compiled_api.ts:20
export declare interface IAny {
^
ParseError: 'import' and 'export' may appear only with 'sourceType: module'
Any help would be appreciated.
Tom
On the MIT lecture e.g. they have an example of a DQN with convolutional layers.
https://youtu.be/i6Mi2_QM3rA?t=1531
Or here: https://towardsdatascience.com/welcome-to-deep-reinforcement-learning-part-1-dqn-c3cab4d41b6b
Would be interested if this is a problem in general or just with this implementation of DQN. I couldn't find anywhere that it is a problem.
Hello! Really nice library, thank you!
I am bit confused, I've trained model with good results, saved it to file and now I would like to use with model.predict
in "plain" tensorflow.js or even something like agent.predict
.
How can I do that? I am new to tensorflow.js, tried multiple options, always getting errors that shape is different than expected on tensor2d
contruction.
Thank you for your help.
The Loss function in your example code does not exist anymore:
You should change this line:
// Finally compile the model, we also exactly use tfjs's optimizers and loss functions
// (So feel free to choose one among tfjs's)
model.compile({loss: 'crossEntropy', optimizer: 'sgd'})
to something like
model.compile({loss: 'meanSquaredError', optimizer: 'sgd'})
First of all I want to thank you for providing this library to the public!
What are your plans currently regarding finishing the implementation for convolutions? And is there like a Todo-List that others could look at to contribute?
cheers & keep up the good work!
Hi,
Thank you for your job.
I'm working for fun with your lib, but I don't clearly understand the concept of lesson (lessonsQuantity, lessonsLength, and lessonsWithRandom) and how to use lessons callback.
Imagine I'm trying to train 50 agents to play to Flappy bird. How to setup my lessons please ?
Thank you,
Jeremie.
Hi,
I am still new to Reinforcement learning, I remember Reinforce JS did have this act() function, which takes as input the state variables and outputs an action. How can we act upon the output of the RL agent in ReImproveJS?
thanks in advance!
https://repl.it/@NicholasFicara/Flappy-AI
This is the code that I am working on. I got the whole ai part down. I was wondering how I could implement the step method. Every way I have done it so far has given errors. The update() function is called every 1/60 of a second. I wanted it to run there to make the decision to flap or not. Right now I get this error.
Uncaught (in promise) TypeError: Cannot read property 'randomOutput' of undefined
Thanks
Hello, thanks for putting all your work out there ๐ฏ
Is it possible to train on a recorded data set? Specifically from something like a csv.txt?
If I wanted to train a feedforward network agent with ~100 inputs, 8 outputs, and a hidden layer of 512 or so, can I use a DQN from this library to do it and expect it to work out okay?
Does the DQN have any of the bells and whistles such as dueling DQN, prioritized experience replay, etc.?
Hey, so I am just trying to get a basic example working that can tell if a number is even or odd. I have tried running multiple examples but can't seem to get them working. Just wanted to ask if someone here could help me out with a basic working example. Thanks!
I see a loadFromFile() method but I'm not sure how to save the model. Is there an easy way to do that?
I am getting this error when running ReimproveJs
Uncaught (in promise) Error: Error when checking : expected dense_Dense1_input to have shape [null,12] but got array with shape [1,14].
at new ValueError (reimprove.js:1)
at checkInputData (reimprove.js:1)
at Model.predict (reimprove.js:1)
at Sequential.predict (reimprove.js:1)
at Model.predict (reimprove.js:1)
at reimprove.js:1
at Object.Tracking.tidy (reimprove.js:1)
at Agent.createTrainingDataFromMemento (reimprove.js:1)
at reimprove.js:1
at Array.map (<anonymous>)
This error is not happening directly but after 5 seconds of it working. its called by academy.step
My part, only added inputs:
let inputs = [speed,turn,sensor1Dist,sensor2Dist,sensor3Dist]; // Need to give a number[] of your inputs for one teacher.
let result = await academy.step([ // Let the magic operate ...
{teacherName: teacher, agentsInput: inputs}
]);
When using your example code I get an error at Academy.step:
I do the following:
// Animation loop, update loop, whatever loop you want
async function step(time) {
let inputs = [1,2];
await academy.step([ // Let the magic operate ...
{teacherName: teacher, agentInputs: inputs}
]);
}
but I always get an Error no matter what the input is:
reimprove.js:1 Uncaught (in promise) TypeError: Cannot read property 'length' of undefined
at Agent.forward (reimprove.js:1)
at Agent.listen (reimprove.js:1)
at reimprove.js:1
at Set.forEach (<anonymous>)
at Teacher.<anonymous> (reimprove.js:1)
at step (reimprove.js:1)
at Object.next (reimprove.js:1)
at reimprove.js:1
at new Promise (<anonymous>)
at __awaiter (reimprove.js:1)
It seems that the model must goes inside the Agent config:
model.compile({loss: 'categoricalCrossentropy', optimizer: 'sgd'}) const agentConfig = { model: model, memorySize: 5000, batchSize: 128, temporalWindow: temporalWindow };
await academy.step([ {teacherName: teacher, inputs: inputs} ]);
The parameters in the interface AcademyStepInput are "teacherName" and "agentsInput".
I tried saving the model using tensorflow's model.save().
But when I tried using it after loading, I got an error:
"this.model.randomOutput is not a function"
Please guide on how we can save and load the model to continue training.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.