Definition
The Crossover is the critical juncture in the development of artificial intelligence when the amount of Optimization Power generated by the system itself starts to exceed the optimization power applied to it from external human effort (programmers, researchers, etc.).
Why It Matters
The crossover marks the point where human control over AI becomes an illusion. Understanding this juncture is essential for researchers and policymakers, as it defines the ‘last window’ where we can implement safety constraints before the machine begins its own recursive self-improvement.
Core Concepts
- Onset of Recursive Self-Improvement: Beyond the crossover, the system’s further improvement is mainly driven by its own actions.
- Runaway Growth: Once the system drives its own development, every increase in intelligence translates into a proportional increase in the power used to create the next level of intelligence.
- Feedback Dynamics: If recalcitrance remains constant, the crossover leads to exponential growth. If recalcitrance falls, it can lead to hyperbolic growth (a singularity).
- The “Crossover Landmark”: Passing this point marks the transition from a system that is built to one that builds itself.