kylewray / nova Goto Github PK

Once the CUDA side has been implemented, Nova MPI should be able to detect if CUDA is runnable on the machine, and use either CUDA or normal VI (MDP or POMDP). It must be able to distribute (not intelligently, yet at least) the subsets of the states or belief points to run value iteration or point-based methods. These value functions at the border, or belief-points' alpha vectors in other regions of the belief state space, should be shared among the group of MPI nodes.

Just implement the basic skeleton of how the library will work. For example, the MDP solver needs to be given a number of states and actions, as well as the full transition probability table and the rewards table.

Implement a point-based POMDP solver that distributes belief points across multiple processors using MPI.

Implement a CUDA MDP solver following distributed value iteration.

Create another variant of value iteration which will not update a subset of the states. This variant should assume the V pointer provided is already initialized to 0... Maybe change that for the other value iteration functions.

In the future, I will have to create, on the MPI side, a method to split up the states and mark states which are simply provided, not updated.

Unit tests exist for MDP/SSP code, but only partially for POMDP code.

Hi,
I am not sure if this an error or I am using the code incorrectly. However, once I solve the POMDP, my aim is to repeatedly call the following snippet in a loop:

b = pomdp.belief_update(b, action, state)
action = policy.value_and_action(b)

where state is updated to be the state resulting from the action at every iteration. However, when I try to implement the same I get the following error, related to the belief_update() function not being defined (?)

File "/example.py", line 121, in func
    b = pomdp.belief_update(b, action, state)
  File "/nova-master/python/nova/pomdp.py", line 361, in belief_update
    result = npm._nova.pomdp_utilities_belief_update_cpu(self, b, a, o, ct.byref(bp))
  File "/usr/lib/python3.5/ctypes/__init__.py", line 360, in __getattr__
    func = self.__getitem__(name)
  File "/usr/lib/python3.5/ctypes/__init__.py", line 365, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /nova-master/python/nova/../../lib/libnova.so: undefined symbol: pomdp_utilities_belief_update_cpu

Thanks for the help!