Packaging all the featurizers within a single library about b-pro HOT 6 OPEN

@pierrelux I am planning to refactor the whole code during this summer. I'll let you know when I'm done with it. Thanks for the suggestion!

from b-pro.

Is there Python wrapper for this code you can share with me?

from b-pro.

I wrote a wrapper in the other branch. It is quite fast, there is just a small performance drop when compared to C++. However, I never managed to be able to obtain an equally fast implementation of Sarsa. I guess I was never able to use the representation sparseness properly in Python.

If you are interested, let me know. I can help you with the Python wrapper and maybe you are able to properly use sparsity in Python.

from b-pro.

@mcmachado Does the current sarsa implementation work? I can help implement sparsity properly in python if you need help.

from b-pro.

from b-pro.

When I wrote this issue more than two years ago, it was in the context of this wrapper that I had written: https://bitbucket.org/rllabmcgill/shallowpy/src/master/

It may still be useful for you. I was able to get Sarsa(0) to work by relying on the memory over-commitment feature in Linux which lets you "np.zero" huge arrays without actually using that memory until you write to it. It means that if you write your sarsa(0) with sparse updates through indexing methods only, it should run pretty fast. The problem is when you want to implement sarsa(lambda) with eligibility traces and need to decay your trace. You then end up writing a lot and the memory usage just explodes. That's when @mcmachado and I started discussing about porting their special "merging" logic from the C code into Python. It never quite happened, but it would be great if someone can do it.

from b-pro.