Comments (9)
Hi @voop, good catch, I'll get to it, thank you.
from pilco.
@kyr-pol I fixed the comment of @voop in branch rbf_controller
.
However, it seems that the refactoring I made in the RBF controller (the one to avoid rewriting gp0
) messed up with GPflow's optimization.
When trying to optimize the controller I get: No variables to optimize.
I think that whatever Param
is inside a GPR
model is not visible in the optimization of the policy. Maybe we could avoid having GPR
models inside RBF
but only use kernels?
from pilco.
Hm if that is the case with Param
s inside GPR
objects, yes it makes sense to just use the kernels. It is a bit weird and it seems worth looking into, but if we can get a quick fix we can look into it further later.
from pilco.
I think if ParamList from GPFlow is used instead of simple 'list' for storing models, it will be visible to the optimization.
However, changing it will cause other issues which I was not able to resolve...
from pilco.
Thanks @voop. I believe that we want to use ParamList
only for the RbfController
. This can be done by overriding the create_models
function of MGPR
.
I'll try to see if that works.
from pilco.
Okay, ParamList
made the parameters visible to the optimiser. I still had to create a fake GPR
class, where X and Y are Param
and not DataHolders
, as I think that GPflow
does not optimises over DataHolders
.
However, I am now getting NaNs on the cost of the policy. @kyr-pol can you take a look?
from pilco.
Yes I will get to it later tonight and see what I can do.
from pilco.
It seems that the NaNs were a numerical issue. By reducing the number of basis functions significantly (from 100 to 10, but 20 work too) the optimisation progresses.
Still, there are frequent failures, either in the Cholesky decompositions or the controller optimisation, which appear sometimes when using the LinearController as well. We should investigate these further.
from pilco.
Okay, that's good news. Let's close this and open a separate issue on numerical stability.
from pilco.
Related Issues (20)
- Computation of cross-covariance of state and action
- Question about MGPR.
- Error with cloudpickle
- Computation time for policy optimization HOT 3
- Reference for predicting with uncertain inputs with SMGPR HOT 1
- Gradient based policy optimisation. HOT 4
- SMGPR : the induced points are different for each model HOT 1
- calculate_factorizations question HOT 1
- Cost for trajectory following HOT 3
- Cholesky decomposition was not successful. The input might not be valid. HOT 2
- [BUG] mountain_car.py fails due to missing import
- What is the V for in the predict_given_factorizations HOT 1
- installation: issue with gast, tensorflow HOT 6
- How do you save your trained model? HOT 2
- Could you please share exact version of some dependency packages
- Performance issue in the definition of create_models, pilco/controllers.py(P1)
- AttributeError: 'Parameter' object has no attribute 'value'
- NotImplementedError: Cannot convert a symbolic (graph mode) `DeferredTensor` to a numpy array. HOT 2
- Is squash_sin() right? HOT 1
- Bugs in model update? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pilco.