Definition
The Standard Model of AI defines intelligence as the ability of a machine to perform actions that can be expected to achieve its objectives. In this paradigm, objectives are provided as fixed mathematical entities (such as reward functions or goal states) which the machine then optimizes.
Why It Matters
The Standard Model of AI highlights the ‘competence risk’ of our current approach; it warns that building machines that are ‘too good’ at optimizing fixed, potentially misaligned objectives is the primary path to existential catastrophe.
Core Concepts
- Objective Optimization: The machine’s performance is measured by how well it satisfies the specific objective function it was given, rather than the true intent of the human designer.
- Fixed Goals: The assumption that the goal is correctly specified and should be pursued with maximum efficiency.
- Means-Ends Decoupling: The designer provides the “ends” (what to achieve), and the AI provides the “means” (how to achieve it).
- The Competence Problem: As AI becomes more competent, it becomes better at achieving the specified objective, which increases the danger if that objective is even slightly misaligned with human values.