Definition
Generative Models are a class of artificial intelligence models that can generate new data samples that are similar to the training data. Unlike Discriminative Models, which learn the boundary between classes, generative models learn the underlying probability distribution of the data.
Why It Matters
Generative models represent the transition from AI as a ‘classifier’ to AI as a ‘creator’; by learning to produce new data rather than just labeling the old, these systems are fundamentally reshaping how we produce code, art, and scientific hypotheses.
Core Concepts
- Probabilistic Modeling: Estimating or to understand how the data was generated.
- How to read: “The probability of X, or the probability of X given Y.”
- Meaning: Generative models learn the joint or conditional probability distribution of data — how likely each sample (or sample given a label) is under the model.
- Large Language Models (LLMs): Generative models trained on text (e.g., GPT-4) that predict the next token in a sequence.
- Diffusion Models: Models used for image generation (e.g., Midjourney, Stable Diffusion) that learn to reverse a noise-adding process.
- Generative Adversarial Networks (GANs): A architecture where a “generator” and a “discriminator” compete, forcing the generator to create increasingly realistic data.