Data driven Markov chain inference

[Konstantin Vasilev]

Hello, I'm really impressed by the Probability package and especially the Markov chains functionalities.

I went through the tutorials and examples, however, I am still looking for an example that infers states' transitions probabilities and dependencies on hidden states from real world big data.

From example, consider the following data structure:

• account_id
• datetime
• state
• hidden state

Given the above generic structure, how can I make an algo that follows the state transitions of each account and infers the set of possible transitions (e.g. it can infer that transition A->B is possible but if B->A is not found in the data, hence not considered possible), as well as their unconditional and conditional (on the hidden state) transition probabilities?

I would expect there to be such example already as many real world problems can be represented in the format above, hence the inference in such form will have very wide range of application. Moreover, it can easily scale with data  as learning about the transitions and their probabilities can be updated continuously.

If you're not aware of such examples, I would appreciate any general guidance and happy to share my progress afterwards.