shmuma / rl Goto Github PK
View Code? Open in Web Editor NEWRL experiments
License: GNU General Public License v3.0
RL experiments
License: GNU General Public License v3.0
Hello,
I'm doing my thesis on Rubik's cube solving with DL, mainly based on this paper.
Your article and code have been very valuable so far and definitely made my life easier. Though, just to make sure (and because this repo doesn't have a license listed), is it okay if I use your code and cite your article in my thesis?
Thanks!
-- Evelyn
Hi,
I try to do my own implementation as a personal learning (I use keras, not pytorch).
As I understood value should ideally represent a "distance" from the goal state, like:
In my case sometimes the value is increasing the further the state really is from the solved state:
The problem I think is that there will always be a state which is not in the training data, thus the above solution indeed has low squared error:
Take into account a chain in the training set (states that are neighbours and the "real, unknown" distance is increasing) with states S1, S2, S3...Sn (Here S1, ... Sn are in the training set, and the index measures the real distance, e.g. minimal rotations needed to solve the state).
V(S0) =0,V(S1)=1 |here the states fall apart| V(S2) = V(S3)-1=...=V(Sn)-1 = V(Sn+1)-1 is a good solution, since Sn+1 is not in the training set, thus V(Sn+1) could be anything (e.g. 10000).
Here V(S) is the value predicted by the Neural Net.
What am I missing? What makes the above solution invalid to converge to? It should be penalized somehow...
Thank you
In train.py, if config.weight_samples==True #in 74. row
Instead of 'simple' value_loss_t = value_loss_t.mean(), shouldn't we divide value_loss_t.sum() by the sum of weights (i.e. weighted average).
I think we don't want the loss function to be lower just because the weights in the batch was lower, i.e. sampling 'farther' states.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.