cannontwo / research Goto Github PK

These experiments should evaluate the ability of the most recent version of PARL+Planning to learn to track trajectories under the following variations:

• Different trajectories
• Different numbers of references
• Different radii of references from trajectory
• Different sampled dimensions

As per my discussion with Zak today, a simpler PARL+Planning pipeline should be developed that interpolates a geometric lead path to generate a trajectory.

• Implement A* for top-level geometric search
• Implement cubic spline interpolation to get velocities and accelerations
• Implement trajectory following with PID or LQR
• Implement aggregate model which weights contribution by Voronoi overlap (via sampling)

This will be useful for making sure that learned controllers are actually stabilizing.

See the algorithm described in https://www.sciencedirect.com/science/article/pii/S1474667016363443. This is necessary in order to formulate an LP which can be solved for a PWA Lyapunov function.

Depends on #8

We can usefully refine polygons with regions that get mapped out of bounds without overly complicating the interior, which may make the LP feasible more quickly by reducing the number of polygons that we need to keep track of.

If the nominal model is lower-order than the true physical system, PARL+Planning should still be able to do something.

In the early stages of learning, the magnitude of learned dynamics can easily overpower the nominal model. I need to come up with a principled way to handle this. Possible solutions:

1. Make the assumption that there's a maximum single-timestep magnitude of deviation, and clip overlarge learned dynamics.
2. Reweight learned dynamics using some function of the number of datapoints, as a proxy for uncertainty.
3. Explicitly estimate uncertainty and use this to adjust learned dynamics (or prevent addition of uncertain dynamics)

This should be achievable using the existing Mesh class in cannon::graphics.

This paves the way for #5 (since the nominal model used for planning needs to have the same notion of continuity of time as the learned aggregate model).

I need to extract the logic that is specific to setting up a planning problem for OMPL from the PARL handling, and put both of these things into separate classes. This will help me to nail down the complicated OMPL stuff.

The unbounded Voronoi cells are not currently incorporated into the Voronoi polygon collection computed by parl_stability/voronoi.cpp, but it should be possible to include them by building Nef_polyedron_2 objects and then converted to polygons, as in parl_stability/transition_map.cpp.

This is necessary as otherwise the controlled system returned by Parl::get_controlled_system is not accurate, and so neither are the computed transition map or eventual Lyapunov computation.

See https://arxiv.org/pdf/1709.00376.pdf and https://murpheylab.github.io/pdfs/2019TROAbMu.pdf. This will be a slightly more challenging environment for PARL+Planning

Turns out I've been doing things unnecessarily inefficiently. See section 5.4 of https://link.springer.com/content/pdf/10.1007/BF00114723.pdf

Should be fairly simple; just use thread pool already implemented.

I need to design a sane way to encapsulate stopping/holding maneuvers, ICS, etc. that needs to surround replanning for PARL.

The planned and executed trajectories can be easily written to files, and then visualized using a sphere for each waypoint along the path (at simplest).

Parl currently contains a line_search method which can be extracted into an independent function or class in the cannon repo.

Rather than doing scatterplot-based rendering of found lyapunov functions, make another plotting script which renders the full polygon by color the vertices and using OpenGL interpolation. This will likely require a new interface to Plotter.

This can be done by implementing a function which takes a PWAFunc and a radius to cut it at, then returns a truncated PWAFunc.

At the very least, should visualize a heat map of number of datapoints across the aggregate model.

I've never rigorously tested PARL with and without the Adam optimizer. Given that PARL uses a line search now, it may be entirely unnecessary or inhibiting learning. The "without" version should simply directly apply line searched gradients.

As an extension to this issue, it might be interesting to try wolfe condition line search rather than the Armijo line search currently implemented.

This should aggregate the dynamics models learned by the path-following PARL agents, then be used for later planning.

This should use the polygon handling procedures in CGAL.

Need to verify that Lyapunov functions found by solving the relevant LP satisfy constraints. This can be done numerically and by plotting points on a fine grid over the estimated PI set.

Right now, PARL's performance is extremely constrained by LSTD matrix updates. If this matrix multiplication can be sped up, it would significantly affect PARL's speed. CUDA acceleration is likely the only way, as Eigen is already very optimized for CPU matrix multiplication.

Need to add computation of Omega_{ip} sets from https://ieeexplore.ieee.org/document/6426761 to parl_stability/transition_map.cpp.

This is specifically to be used with PARL.

This is a pre-requisite to modifying PARL to do tracking control. I want to verify that, with an appropriate reward function and a single reference point, PARL recovers LQR functionality.

This should compute a polygonal approximation to a positive invariant set for an input PWA function, as a relaxed notion of stability.

Since there is no Pickle for C++, need to implement an explicit HDF5 saving and loading procedure for PARL.

This wrapper should somehow store a planner to be used in the integrated algorithm. It is the object responsible for handling replanning, execution, etc.

The "ControlAffineKinematicCar" that I've implemented is really just halfway to a fully dynamic, second-order car model. I should just go all the way, and then use the links that Mark sent me to implement an LQR-based steering function to make planning feasible.

• https://github.com/KavrakiLab/ompl-private/blob/lqr-control/demos/Pendulum.cpp
• https://github.com/KavrakiLab/omplapp-private/blob/lqr-control/src/omplapp/apps/DynamicCarPlanning.cpp
• https://github.com/KavrakiLab/ompl-private/blob/lqr-control/src/ompl/control/src/LQRStatePropagator.cpp

I don't know if this is strictly necessary, but most presentations of Lyapunov function finding require it. Only the controlled system needs to have zero affine term, which means that the affine term in the controller can be set directly from the estimated dynamics during learning.

This will probably involve Lloyd's algorithm and some inspiration from the "bounded-error" approach to PWA system identification.

Use the four-region system from Example 3.2 of https://ieeexplore.ieee.org/document/1017553