Definition
Normalization is the mathematical process of scaling or transforming a set of values or a function so that they conform to a standard, unified scale—most commonly scaling vectors to have a magnitude of 1, or adjusting a probability distribution so its total integral equals exactly 1.
How to read: The normalized vector v hat equals the vector v divided by the magnitude of v. Meaning / when to use: Used to convert any non-zero vector into a unit vector. It preserves the exact direction of the vector but forces its length to be 1, isolating the concept of “direction” from “magnitude.”
Why It Matters
Normalization allows for direct, apples-to-apples comparisons between wildly different datasets or systems. In machine learning, if you don’t normalize input data (e.g., comparing age in years to income in millions), the algorithm will be mathematically biased toward the larger numbers. In quantum mechanics and probability, if a function isn’t normalized to 1, it cannot represent physical reality, as the total probability of “something happening” must be exactly 100%.
Core Concepts
- Unit Vectors: In linear algebra, basis vectors are normalized so that moving “one step” along an axis is mathematically consistent.
- Probability Density: Scaling a function by a constant such that .
- Min-Max Scaling: In data science, transforming data features to fit within a specific range, usually .
- Z-Score Standardization: A specific form of normalization where data is centered around a mean of 0 with a standard deviation of 1, crucial for applying the Central Limit Theorem.