Law of Large Numbers

Definition

The Law of Large Numbers (LLN) is a theorem in probability and statistics that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed.

Why It Matters

Randomness averages out over time. The LLN provides the “statistical bedrock” for insurance, science, and gambling, ensuring that while individual events are unpredictable, the “long-run” average is as stable and reliable as a physical law.

Core Concepts

• Weak Law vs. Strong Law: The Weak Law (Khinchin’s Law) states that the sample average converges in probability towards the expected value. The Strong Law (Kolmogorov’s Law) states that the sample average converges almost surely to the expected value.

• Convergence: As $n \to \infty$, the sample mean $\bar{X}_n$ approaches the population mean $\mu$.

  • How to read: “As n approaches infinity, the sample mean X bar n approaches mu.”
  • Meaning / when to use: More trials make the sample average stabilize toward the true expected value—foundation of statistical reliability.

• Independence: The trials must be independent and identically distributed (i.i.d.) for the standard LLN to apply.

• Expected Value ($E[X]$): The theoretical long-run average value of a random variable.

  • How to read: “The expected value of X.”
  • Meaning: The weighted average outcome over infinitely many repetitions—the target the sample mean converges to.

Connected Concepts

• Central Limit Theorem
• Probability Theory
• Clustering Illusion
• Coincidence Mechanics