Mortdecai
d34f447e1f
Six Gemini agents ran autonomously through 35 research tasks covering falsifiability, retrocausality, consciousness, game theory, agricultural revolution, meaning crisis, AI cost curves, adoption S-curves, and more. 304KB of primary-source research with scholars, counterarguments, and data. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
63 lines
6.7 KiB
Markdown
63 lines
6.7 KiB
Markdown
# Task 13: Game Theory of Technology Races — Why No One Stops
|
|
|
|
## Executive Summary
|
|
The "Ratchet" effect (Paper 007) is fundamentally a game-theoretic outcome where individual rational actors are compelled to adopt and advance dangerous technologies due to competitive pressure. This research analyzes the structural reasons why collective warnings (the allegories) are consistently ignored. Key findings include:
|
|
* **The Multiplayer Prisoner's Dilemma:** In the race for AGI, the payoff for "winning" (trillions in value, strategic dominance) is so high that even a 10% risk of extinction is considered a "rational" bet by individual actors.
|
|
* **The Unilateralist's Curse:** With dozens of frontier labs and nations, the probability that *at least one* will ignore safety warnings and "pull the trigger" approaches 100%.
|
|
* **Coordination Models:** The **Montreal Protocol** (ozone) succeeded because it had clear economic alternatives; **Climate Change** and **AI** struggle because the "harmful" path is the primary driver of current economic growth.
|
|
|
|
## Key Scholars and Works
|
|
* **Nick Bostrom (*The Vulnerable World Hypothesis*):** Introduces the "Urn of Inventions" analogy—if we draw a "black ball" (a technology that makes destruction easy and cheap), civilization fails without total global control.
|
|
* **Stuart Russell (*Human Compatible*):** Describes the "Racing to the Precipice" dynamic where the economic value of AI ($10T+) makes it impossible for any single corporation to stop without being replaced.
|
|
* **Scott Alexander (*Meditations on Moloch*):** A foundational cultural text describing "Moloch" as the systemic force of competitive pressure that leads to suboptimal collective outcomes (arms races, clickbait, environmental collapse).
|
|
* **Thomas Schelling (*The Strategy of Conflict*):** Nobel-winning game theorist whose work on nuclear deterrence and "focal points" provides the framework for why international AI treaties are so difficult to verify.
|
|
|
|
## Supporting Evidence
|
|
|
|
### 1. The AI Arms Race as a Multiplayer Prisoner's Dilemma
|
|
* **The Setup:** Nations (US, China) and Corporations (OpenAI, Google, Meta) face a choice: Invest in Safety (S) or Invest in Capabilities (C).
|
|
* **The Payoff:** If all choose S, the world is safe but progress is slow. If one chooses C while others choose S, the "C-actor" gains a decisive strategic advantage (the "Singleton").
|
|
* **The Outcome:** To avoid being dominated by a "Singleton," everyone chooses C, leading to a race where safety is a secondary concern.
|
|
|
|
### 2. The Unilateralist's Curse
|
|
* **The Logic:** Even if 99 labs agree that a certain model is too dangerous to release, the "Curse" dictates that the 100th lab (perhaps less competent or more desperate) will release it anyway.
|
|
* **Open Source as Anti-Coordination:** The release of models like Llama 3 or Mistral makes coordination harder because the "code" is now outside the reach of centralized treaty-making.
|
|
|
|
### 3. The Collingridge Dilemma (The Timing Trap)
|
|
* **Information Horn:** When AI was in its infancy (1950-2010), we didn't know how to regulate it because we didn't know what it could do.
|
|
* **Power Horn:** Now that we know what it can do (2020-present), it is already becoming infrastructure (Microsoft 365, search, defense), making it too expensive/disruptive to stop.
|
|
|
|
## Counterarguments and Critiques
|
|
* **The "Ozone" Counter-Example:** Critics point to the **Montreal Protocol** as proof that humanity *can* coordinate. However, the search results show that the Montreal Protocol succeeded only because a profitable alternative (HCFCs) was already developed by the chemical giants (DuPont).
|
|
* **Soft Power Mediation:** The EU AI Act is an attempt to use "Regulatory Power" as a focal point for coordination, betting that the "Brussels Effect" will force global companies to adopt safer standards to access the European market.
|
|
* **Tribalism vs. Risk:** Some researchers (LeCun, Ng) argue that "Existential Risk" is a "tribal signal" used by incumbents to gatekeep the industry, suggesting the game is actually about **Regulatory Capture**, not survival.
|
|
|
|
## Historical Parallels and Case Studies
|
|
* **The Nuclear Arms Race:** The 1963 Limited Test Ban Treaty (LTBT) provides a model for "partial coordination." Nations agreed to stop atmospheric testing (visible harm) while continuing underground (invisible progress).
|
|
* **Antibiotic Resistance:** A "Tragedy of the Commons" case where individual doctors and patients take antibiotics for short-term health, leading to the collective long-term destruction of the drug's effectiveness.
|
|
* **The "Singleton" Scenario:** Historically, the "British Empire" or "Standard Oil" acted as singletons in their domains, but AI allows for a "singleton" that could potentially control the entire information layer of the species.
|
|
|
|
## Data Points
|
|
* **$600 Billion:** Estimated US AI capital expenditure in 2025-2026.
|
|
* **40-60%:** Global employment exposure to AI change (IMF).
|
|
* **98%:** Reduction in Ozone-depleting substances since 1990 (the success metric for Montreal).
|
|
* **Zero:** Measurable U.S. GDP growth attributed to AI in 2025 despite $700B investment (Goldman Sachs), illustrating the "Productivity J-Curve."
|
|
|
|
## Connections to the Series
|
|
* **Paper 007 (The Ratchet):** Game theory is the "engine" of the ratchet. It explains why, even when everyone sees the cliff, the structural incentives make it "rational" to keep accelerating.
|
|
* **Paper 006 (The Feedback Loop):** Recursive creation makes the arms race faster. If my AI can help me build a better AI, the "lead" I gain by defecting from a safety agreement becomes insurmountable in weeks, not years.
|
|
* **Paper 008 (Singularity as Compilation):** The race is not just for intelligence, but for the "Compilation Focal Point." The first entity to compile all human knowledge into a functional singleton wins the "Game of History."
|
|
|
|
## Rabbit Holes Worth Pursuing
|
|
* **The "Stag Hunt" vs. "Prisoner's Dilemma":** Which game more accurately describes AI alignment? (Stag Hunt allows for coordination if trust is high).
|
|
* **Byzantine Fault Tolerance in AI Governance:** Can we use blockchain-style consensus to verify model safety without seeing proprietary weights?
|
|
* **The "Oppenheimer Moment" for AI:** Why hasn't a major AI lab lead resigned in protest yet? What is the "resignation threshold" in game theory?
|
|
|
|
## Sources
|
|
* Bostrom, N. (2019). "The Vulnerable World Hypothesis." *Global Policy*.
|
|
* Russell, S. (2019). *Human Compatible: AI and the Problem of Control*.
|
|
* Alexander, S. (2014). "Meditations on Moloch." *Slate Star Codex*.
|
|
* Schelling, T. C. (1960). *The Strategy of Conflict*. Harvard University Press.
|
|
* IMF (2024). "Gen-AI: Artificial Intelligence and the Future of Work."
|
|
* Goldman Sachs Research (2025). "The AI Paradox: High Investment, Low GDP."
|