gvanderheide / discretemarkovchain Goto Github PK

It would be great to have this package available on conda-forge.
Something about creating a "feedstock" for it.
I will spend a 1/2 hour on this, then hope for better qualified help :)
https://conda-forge.github.io/#contribute

currently there is only support for tuples.

The following does not run well:

P = np.array([[0.5,0.5],[0.6,0.4]])
mc = markovChain(P)
mc.eigenMethod()

This gives the error:
ValueError: k must be less than ndim(A)-1, k=1

As temporary fix, the powerMethod is called when the the Markov chain consists of two states.

Hi, great work here! I'm working on a simple problem and would love your feedback. Essentially, I have these States or "buttons". I have let's say initial probabilities, of a user going from one button to another. After time t1, I want to know which button he "could" be on now(or predict). After every time 't' he presses a some button button, we update the transition matrix and predict the next button. Sounds simple I'm having a hard time incorporating this with the library.. Here's a screenshot for your reference:

I found this package super helpful for my research! There is an issue when the transition matrix is not sparse.

Issue
markovChain assumes that P is sparse. If P is not sparse (e.g. PageRank) using a sparse implementation leads to a memory explosion (and much slower code).

Solution

• allow the use to choose if P is sparse or dense
• write versions of the eigen-solvers that handle numpy arrays (should be mostly copy and paste)

mask is undefined in the README example, what is it?

Planned:

• Include other methods for determining steady state distributions, such as gauss-seidel and successive-over-relaxation. It is easy to implement unvectorized versions of these algorithms, but this will be rather slow in Python.

Fixed:

• Allow the user to specify its own sparse rate/probability matrix. This is useful and faster in case of band matrices. (Added in build 0.11)
• Save the initial matrix so it can easily be reused. Then different methods can be compared without generating the matrix again. (Added in build 0.11)

Possibile future additions:

• Generate the transition matrix using parellel processing. While I have some code that in principle can do this, it does not work on functions defined in a class and it can only be run after an if __name__ = "__main__" call.
• Introduce a new class for finite Markov chains and calculate important measures such as the expected time until absorption and the probability of being absorbed in a certain state.
• Include a class for average reward/cost Markov decision processes (a generalization of the Markov chains considered here).