Definition
AI Hallucination is the phenomenon where a Large Language Model (LLM) generates output that is factually incorrect, nonsensical, or ungrounded in its training data, while presenting it with a high degree of confidence. It is a byproduct of the model’s objective to predict the “most probable next token” based on patterns rather than a reference to objective truth.
Why It Matters
It identifies the fundamental reliability gap in large language models, where plausible-sounding but false information is generated. Managing and mitigating these errors is the primary challenge for deploying AI in high-stakes fields like medicine, law, and engineering.
Core Concepts
- Next-Token Prediction: LLMs do not “reason” or “check facts”; they calculate probability distributions. If a factual gap exists in the training data, the model will “fill” it with the most statistically likely linguistic pattern.
- Confident Bullshit: A term from philosophy (Harry Frankfurt) applied to AI, where the system has no regard for the truth or falsity of its statements, only for the effectiveness of the communication.
- Probabilistic Drift: As a sequence grows longer, small errors in prediction can compound, leading the model farther away from logical or factual coherence.
- Training Data Bias: Hallucinations often mirror the myths, clichés, or common errors found in the massive, uncurated datasets used for training.
- Confabulation: The psychological term for creating false memories to fill gaps, often used as a more precise analogy for AI hallucinations than the visual “hallucination” of humans.