Markov Matrices

Definition

Markov matrices (or stochastic matrices) describe transitions between states in a system. A matrix $M$ is Markov if all entries $m_{ij} \geq 0$ and each column sums to 1.

• How to read: “Matrix M is Markov if every entry m i j is nonnegative and each column sums to one.”
• Meaning: Each column is a probability distribution over next states—no negative probabilities, and all outcomes account for 100%.

Why It Matters

Markov matrices provide the steady-state solution for long-term population or system shifts; understanding how these matrices converge is the key to predicting where a system will eventually ‘land,’ regardless of its starting conditions.

Core Concepts

• Steady State ($x_\infty$): The state vector that remains unchanged under the transition: $M x_\infty = x_\infty$. This is an eigenvector with eigenvalue $\lambda = 1$.

  • How to read: “M times the steady state vector x infinity equals x infinity; with the eigenvalue lambda equal to one.”
  • Meaning: After many transitions, the system settles into a fixed probability distribution that the matrix leaves unchanged.

• Eigenvalues:

  • $\lambda = 1$ is always an eigenvalue.
    • How to read: “The eigenvalue lambda equals one is always an eigenvalue.”
    • Meaning: Conservation of total probability guarantees a steady-state direction.
  • All other eigenvalues satisfy $|\lambda_i| \leq 1$.
    • How to read: “The absolute value of each other eigenvalue is at most one.”
    • Meaning: No state can grow without bound under repeated transitions; the system cannot amplify probability.

• Convergence: For any starting vector $x_0$, the sequence $x_k = M^k x_0$ approaches the steady state (if $M$ is regular/positive).

  • How to read: “x k equals M to the power k times x zero.”
  • Meaning: Repeatedly applying the transition matrix drives any initial distribution $x_0$ toward the long-run steady state.

• Probability Vectors: The components of the state vector represent probabilities of being in each state, thus they sum to 1.

Connected Concepts

• Linear Transformation
• Equilibrium
• Eigenvalues and Eigenvectors