Condition Number

Definition

Condition number $c$ measures the sensitivity of a mathematical problem to changes or errors in the input data. For a matrix $A$ in the equation $Ax = b$, it is defined as: $c = \|A\| \|A^{-1}\|$

• How to read: “The condition number c equals the norm of A times the norm of A inverse.”
• Meaning: Measures sensitivity of $Ax = b$ to input errors—how much relative output error can be amplified from relative input error.

Why It Matters

It determines whether a numerical calculation is ‘trustworthy’ or if tiny rounding errors will explode into massive, incorrect answers.

Core Concepts

• Singular Values: In terms of SVD, $c = \sigma_{\max} / \sigma_{\min}$.
  • How to read: “The condition number c equals the ratio of sigma max to sigma min.”
  • Meaning: Ratio of largest to smallest stretching direction—large ratio means nearly singular (ill-conditioned).
• Accuracy Loss: If $c(A) = 10^k$, a small error in $b$ can be amplified by up to $10^k$ in the solution $x$. Roughly $k$ decimal digits of accuracy are lost.
• Well-conditioned vs. Ill-conditioned:
  • Well-conditioned: $c$ is small (near 1). Orthogonal matrices have $c = 1$.
  • Ill-conditioned: $c$ is large. A nearly singular matrix has an infinite condition number.
• Norm Dependency: The value depends on the choice of matrix norm ($\ell^1, \ell^2, \ell^\infty$).

Connected Concepts

• Singular Value Decomposition
• Matrix Algebra: Inverses
• Error Analysis