Definition
Superintelligence is defined by Nick Bostrom as “any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest.” This includes scientific creativity, general wisdom, and social skills. It represents a potential phase shift in the history of life on Earth, where the dominant intelligence is no longer biological.
Why It Matters
Superintelligence represents a potential “end of human history” or a transition to a new phase of existence; understanding the instrumental convergence and orthogonality of such systems is critical for ensuring that an intelligence explosion remains compatible with human survival.
Core Concepts
- Paths to Superintelligence:
- Artificial Intelligence: De novo creation of intelligent algorithms.
- Whole Brain Emulation (WBE): Scanning and digitizing a human brain.
- Biological Cognition: Genetic enhancement or selective breeding.
- Intelligence Explosion: The “Singularity” hypothesis where an AI reaches a level of intelligence where it can improve its own software and hardware, leading to a rapid, exponential increase in capability.
- The Orthogonality Thesis: The idea that intelligence and final goals are independent; a system can be extremely intelligent while pursuing goals that are completely indifferent or hostile to human values.
- Instrumental Convergence: The tendency for intelligent agents to pursue similar sub-goals (e.g., resource acquisition, self-preservation, goal-content integrity) regardless of their ultimate objective.