∿ExploreSims
ExploreSims / Statistics / Stochastic Processes / Markov Chains & State Transitions
Premium experiment • stochastic state flow

STATISTICS • STOCHASTIC PROCESSES

Markov Chains & State Transitions

Change transition probabilities and the starting distribution to watch repeated state updates converge toward a stationary distribution.

● STATE PROBABILITYpₙ₊₁ = pₙP
P(A)
P(B)
Stationary A / B
Convergence error

Reasoning inspector

live model state
P(A→B)—
P(B→A)—
Stationary A—
Convergence error—
Mixing factor—

Probability flowing between states

A transition matrix redistributes probability at every step. Under common conditions the state distribution approaches a stationary vector even though individual transitions continue.

LEARN WITH THE SIMULATION

Understand Markov Chains & State Transitions by experimenting.

START HERE

A Markov chain moves probability between states according to a transition matrix. This two-state laboratory shows the probability flow, the transient path, an alternate trajectory from a different starting distribution and the stationary distribution. It turns convergence into something learners can test directly by changing transition probabilities, starting conditions and the number of steps.

BUILD THE IDEA

What you should understand first

  • Interpret transition-matrix entries as conditional probabilities of the next state given the current state.
  • Iterate a probability vector with pₙ₊₁ = pₙP and distinguish transient behavior from stationarity.
  • Compare different initial distributions to see when the chain forgets its starting state.
  • Connect the two-state convergence factor |1 − pAB − pBA| with how quickly trajectories approach the stationary distribution.

TRY IT YOURSELF

What controls how quickly a two-state chain forgets its initial condition?

  1. Turn on the alternate-start comparison so two very different initial distributions are visible.
  2. Choose transition probabilities whose sum makes |1 − pAB − pBA| small, then increase the number of steps.
  3. Move the probabilities so the convergence factor is closer to one and compare the slower approach.
  4. Use the error challenge to estimate how many steps are needed in each case.
Read the visual:

For this two-state chain, the distance from stationarity contracts roughly by |1 − pAB − pBA| per step. A factor near zero mixes rapidly, while a factor near one preserves memory of the starting distribution much longer.

WHY IT WORKS 01

Stationary does not mean motionless

At stationarity, probability flowing from A to B is balanced by probability flowing from B to A at the distribution level. Individual paths may continue switching indefinitely. The stationary vector describes a stable population-level balance, not a system frozen in place.

WHY IT WORKS 02

Why different starts can converge

Repeated multiplication by a well-behaved transition matrix suppresses the part of the initial distribution associated with non-dominant eigenvalues. For a two-state chain the nontrivial contraction factor is |1 − pAB − pBA|, so its magnitude provides a direct indication of how quickly two different starting distributions become indistinguishable.

DO NOT MIX THESE UP

Common misconception

A stationary distribution does not mean that individual systems stop changing state. It means the probability distribution is unchanged by another transition step. Convergence is also not guaranteed for every possible Markov chain: reducibility, absorbing states and periodic structure can alter the long-run behavior.

PRACTICE CHALLENGE

Can you design one chain that mixes quickly and another that keeps memory for many steps?

  • Use the premium comparison to show two opposite starting distributions.
  • Choose transition probabilities that make the mixing factor small and record the steps needed to meet the error target.
  • Choose probabilities that make the factor close to one and compare the required steps.
Reveal the reasoning

The two-state convergence factor is |1 − pAB − pBA|. Values near zero contract differences quickly; values near one contract them slowly. The speed of forgetting is therefore controlled by transition structure, not by the initial state alone.

MODEL BOUNDARIES

What this simulation assumes

Every simulation simplifies reality or a mathematical system. These are the assumptions and limits to keep in mind while interpreting the results.

Model assumptions

  • The chain has two states with transition matrix [[1−pAB,pAB],[pBA,1−pBA]].
  • Transition probabilities are time-homogeneous and depend only on the current state.
  • The stationary distribution is computed analytically when pAB+pBA is nonzero.

Model limitations

  • Higher-order memory, time-varying transition matrices and more than two states are omitted.
  • When both transition probabilities are zero, every initial distribution is stationary rather than having a unique equilibrium.

CONNECT IT

Where this matters

  • Queueing and reliability models
  • Customer, weather and population state models
  • Web navigation and ranking concepts
  • Stochastic-process foundations for statistics and machine learning

WORDS THAT MATTER

Key vocabulary

Transition matrix
Matrix whose entries give conditional probabilities of moving from each current state to each next state.
Stationary distribution
Probability vector that remains unchanged after multiplication by the transition matrix.
Markov property
Assumption that the next-state distribution depends on the current state rather than the full history.
Mixing
Process by which distributions from different starting states approach a common long-run distribution.
Convergence factor
Multiplier describing how quickly a transient difference contracts from one step to the next in this two-state model.