krisbrud / gym-auv Goto Github PK

TODO:

• Remove dependency of global state as much as possible
• Decouple rendering from windowing code
• Get rid of unnecessary deps
• Pass the relevant state into the rendering code rather than giving the renderer access to the environment
• Make rendering configurable through normal config

Split run.py into different functions, files (and possibly classes?). See if there is anything useful for own thesis.

Format code with black formatter.

Since the sensor readings consists of 3 components (closeness, x-/y-velocity), it is probably a good idea to group the observations corresponding to the same input, i.e. going from a dimensionality [n_sensors * 3] to [n_sensors, 3].

• Check that it is possible to pass a dict etc to the policy, as the spatial structure of the LiDAR does not apply to the rest of the observation.

As users may want to use the environment without using stable_baselines and tensorflow, the project and it's dependencies should be pruned, such that testing may be done in smaller Docker containers and and gym_auv may be released as a PyPi package in the future.

Add periodic saving of model to disk using idiomatic RLlib features.

Profiler results show that the calls to intersection() in the simulate_sensor call account for a large amount of the runtime of the algorithm. Optimizing this may have a large positive impact on the simulation performance of the algorithm. Possible approaches may for instance be using an hierarchical checker, which filters out many of the instances that definitely are not collisions.

EivMeyer's original environment used his algorithm "Feasibility Pooling" as a preprocessing step in order to reduce the dimensionality of the observation space while still providing useful observations for the algorithm. As it is desirable to be able to do experiments with Convolutional Neural Networks for encoding this information, it should be refactored out and be an optional setting instead.

The 2D renderer assumes that the input is split into sectors, which isn't necessarily the case after #11. This should be fixed, as well as considering a refactor of the renderer.

Currently, the BaseEnvironment spawns a window by default to render the Environment, which is annoying and should not be the default behaviour.

Make some plots for qualitative evaluation of the agents, possibly refactoring existing code.

Feasibility pooling is now a part of the vessel code, making it cluttered. As it is stateless, it should rather be a wrapper, which makes the code less bug prone.