Bayes' Theorem

Definition

Bayes’ Theorem is a mathematical formula for determining conditional probability. It describes the probability of an event based on prior knowledge of conditions that might be related to the event: $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$

• How to read: “The probability of A given B equals the probability of B given A times the probability of A, all over the probability of B.”
• Meaning: Update your belief in $A$ after observing evidence $B$ by scaling the prior $P(A)$ with how likely $B$ is when $A$ is true, then normalizing by total probability of $B$.

Why It Matters

It is the mathematical law of learning, providing the only rational way to update our beliefs when we see new evidence. In an uncertain world, Bayesian reasoning is the essential tool for maintaining an accurate view of reality.

Core Concepts

• Prior Probability $P(A)$: Initial degree of belief.
  • How to read: “The probability of A.”
  • Meaning: What you believed before seeing evidence $B$—the starting point.
• Likelihood $P(B|A)$: The probability of the evidence given the hypothesis.
  • How to read: “The probability of B given A.”
  • Meaning: How well hypothesis $A$ predicts or explains the evidence.
• Posterior Probability $P(A|B)$: The updated belief after observing new evidence.
  • How to read: “The probability of A given B.”
  • Meaning: Your revised belief after incorporating evidence $B$.
• Evidence $P(B)$: The total probability of the evidence.
  • How to read: “The probability of B.”
  • Meaning: Normalizing denominator—how likely the evidence is overall, across all hypotheses.

Connected Concepts

• Bayesian Updating
• Probability Theory
• Rationality
• Statistics