LearnToScore

This repo contains code to learn a utility function for Next-Best-View exploration with a depth camera.

The cpp folder contains a modified Octomap) implementation that allows to compute a viewpoint score when ground-truth voxels are known and also exposes functions to extract a 3D grid of the occupancy map on multiple scales relative to a defined camera pose.

A dataset of such 3D grids together with the ground-truth viewpoint score can be used to train a 3D ConvNet to regress a viewpoint utility score. The code to learn this 3D ConvNet is in python/reward_learning and written with TensorFlow. A script to record such datasets and evaluate the utility score regressor is also in this folder.

A useful OpenGL renderer written with Glumpy can be found in python/renderer.

The communication between the modified Octomap and the TensorFlow code is done with ROS. The images from the OpenGL renderer can be grabbed with ZeroMQ. Images can also be grabbed from Unreal Engine using a modified version of UnrealCV which can be found in the pybh submodule.

Task description and 3D grid extraction

Some results from real a scene