Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Markov equilibrium
11 views
Skip to first unread message
RichD
Apr 22, 2013, 2:55:31 PM4/22/13
to
As I understand it, a Markov process is a probablisitic
sequence of state transiitons. It's completely specified
by the state transition matrix.
However, I sometimes see references to the
equlibrium condition, whether it's been reached,
etc. I don't get this, can anyone elaborate?
Thanks
--
Rich
sequence of state transiitons. It's completely specified
by the state transition matrix.
However, I sometimes see references to the
equlibrium condition, whether it's been reached,
etc. I don't get this, can anyone elaborate?
Thanks
--
Rich
Paul
Apr 22, 2013, 6:23:48 PM4/22/13
to
Some, but not all, Markov processes have a steady-state distribution (probability for being in each of the states). If the probability distribution for the current state equals the steady-state distribution, so will the distribution after the next transition (and any future transitions). In some cases you reach steady-state after a finite number of transitions. (Contrary to the current zombie craze, once you're dead, your probability of staying dead hovers around 1.0.) Sometimes you approach steady-state asymptotically. Sometimes there is no steady-state.
Paul
Paul
Rich Ulrich
Apr 22, 2013, 10:13:35 PM4/22/13
to
better -
You start with some frequencies existing at each state,
"starting state".
You apply the transition probabilities, and obtain new
frequencies. Do it again. Do it again. ...
Where the frequencies stabilize, if they do, is called an
equilibrium condition.
It is possible that the equilibrium will depend on the start.
--
Rich Ulrich
Ken Pledger
Apr 23, 2013, 6:21:27 PM4/23/13
to
In article
<13b0f307-dbde-4a3f...@mf10g2000pbb.googlegroups.com>,
RichD <r_dela...@yahoo.com> wrote:
> As I understand it, a Markov process is a probabilisitic
Example (using row vector notation). Transition matrix A =
(1/3 2/3)
(1/2 1/2).
Suppose the system has state vector (x y), i.e. there is
probability x that it is in the first state, and y for the second.
Then the next step of the process takes it to (x y)A. If you choose
some initial probabilities x and y, and iterate through quite a few
steps, you will find the state vector getting closer and closer to (3/7
4/7). If the systen were actually in that state, then the next step
would take it to (3/7 4/7)A which is again equal to (3/7 4/7).
(Work it out!)
(3/7 4/7) describes the equilibrium state, and this particular
process converges steadily towards that state. (Some others don't.)
You may be able to see that the equilibrium state vector is an
eigenvector of A corresponding to the eigenvalue 1.
HTH
Ken Pledger.
<13b0f307-dbde-4a3f...@mf10g2000pbb.googlegroups.com>,
RichD <r_dela...@yahoo.com> wrote:
> As I understand it, a Markov process is a probabilisitic
> sequence of state transiitons. It's completely specified
> by the state transition matrix.
>
> However, I sometimes see references to the
> equlibrium condition, whether it's been reached,
> etc. I don't get this, can anyone elaborate? ....
> by the state transition matrix.
>
> However, I sometimes see references to the
> equlibrium condition, whether it's been reached,
Example (using row vector notation). Transition matrix A =
(1/3 2/3)
(1/2 1/2).
Suppose the system has state vector (x y), i.e. there is
probability x that it is in the first state, and y for the second.
Then the next step of the process takes it to (x y)A. If you choose
some initial probabilities x and y, and iterate through quite a few
steps, you will find the state vector getting closer and closer to (3/7
4/7). If the systen were actually in that state, then the next step
would take it to (3/7 4/7)A which is again equal to (3/7 4/7).
(Work it out!)
(3/7 4/7) describes the equilibrium state, and this particular
process converges steadily towards that state. (Some others don't.)
You may be able to see that the equilibrium state vector is an
eigenvector of A corresponding to the eigenvalue 1.
HTH
Ken Pledger.
0 new messages