Pseudoinverse

Definition

Pseudoinverse (or Moore-Penrose Inverse) $A^+$ is a generalization of the matrix inverse for non-square or singular matrices. It provides the shortest least-squares solution to $Ax=b$.

Why It Matters

The pseudoinverse is the “failsafe” for linear algebra. In real-world data science, matrices are almost always noisy, singular, or non-square. Without $A^+$, we would be unable to find “best fit” solutions to complex systems in robotics, computer vision, or signal processing, leaving us paralyzed by mathematical “dead ends” where a standard inverse fails to exist.

Core Concepts

• Computation via SVD: If $A = U \Sigma V^T$, then $A^+ = V \Sigma^+ U^T$, where $\Sigma^+$ is the diagonal matrix of reciprocal singular values $1/\sigma_i$ (for $\sigma_i > 0$) and zero otherwise.
• How to read: “The A equals U Sigma V-transpose; the A-plus equals V Sigma-plus U-transpose.”
  • Meaning: SVD gives the pseudoinverse—invert nonzero singular values, zero out the rest.
• Shortest Solution: Among all vectors $x$ that minimize $\|Ax-b\|$, $x^+ = A^+b$ is the one with the smallest norm $\|x\|$.
• Projectors:
  • $A^+A$ projects onto the Row Space $C(A^T)$.
  • $AA^+$ projects onto the Column Space $C(A)$.
• Left/Right Inverses: If $A$ has full column rank, $A^+ = (A^T A)^{-1} A^T$ (left inverse). If $A$ has full row rank, $A^+ = A^T (A A^T)^{-1}$ (right inverse).
• How to read: “The A-plus equals the quantity A-transpose A inverse times A-transpose” or “A-transpose times the quantity A A-transpose inverse.”
  • Meaning / when to use: Tall skinny matrices use left inverse; short wide matrices use right inverse.

Connected Concepts

• Singular Value Decomposition
• Matrix Algebra: Inverses
• Linear Functions