Definition
Coherent Extrapolated Volition (CEV) is a proposed objective function for a Friendly AI. Instead of following the literal commands of its creators, the AI would execute what humans would want if they knew more, thought faster, were more the people they thought they were, and had grown up further together.
Why It Matters
It provides a framework for aligning AI with human values even when we can’t define those values ourselves, preventing catastrophic ‘monkey’s paw’ outcomes.
Core Concepts
- Extrapolation Parameters:
- Knowing More: Access to all relevant non-moral facts.
- Thinking Faster: Subjective time acceleration to think things through more deeply.
- Being More Who We Wished We Were: Correcting for cognitive biases and “first-order” impulses that we disvalue.
- Grown Up Farther Together: Idealized social interaction to harmonize individual volitions.
- Initial Dynamic: CEV is a process that runs once and then replaces itself with whatever the result wishes (e.g., creating a world government or shutting itself down).
- Convergence vs. Divergence: The AI should only act on features of the extrapolation that are predictable with high confidence. If results diverge wildly, the system remains conservative.
- Coherence vs. Interference: The dynamic listens for widespread agreement (“yes”) but is especially sensitive to blocking votes (“no”).
- Rationales for CEV:
- Encapsulate Moral Growth: Avoids “value lock-in” of current human errors (e.g., slavery, cat-burning).
- Avoid Hijacking Destiny: Distributes the burden of determining the future from a small group of programmers to all of humanity.
- Conflict Mitigation: Different groups (e.g., Taliban vs. Swedish Humanist) can agree on CEV because they believe their side would prevail under idealized conditions.
- Preserve Autonomy: Humanity remains ultimately in charge of its own path, potentially choosing a “safety net” AI rather than a paternalistic one.