Moloch
In plain wordsA name, borrowed from an ancient god of sacrifice, for situations where competition forces everyone to act against their shared interests — as when AI companies feel pushed to race ahead despite the risks.
A personification of coordination failure: the dynamics by which competition, arms races and misaligned incentives push individuals, firms or nations into outcomes none of them want, because anyone who holds back alone is outcompeted. In AI discussion Moloch names the race among companies and countries to deploy ever more capable systems while cutting corners on safety, evaluation and social costs, and frames governance proposals — international agreements, compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI., shared safety commitments — as efforts to defeat Moloch by coordinating restraint. It is the rationalists'rationalist community, n.: A loose online community that prizes careful reasoning and spotting one's own biases; gathered around the forum LessWrong, it was one of the first groups to take the dangers of advanced AI seriously. vivid label for problems economists analyze as tragedies of the commons, prisoner's dilemmas and races to the bottom; 'multipolar trap', a phrase popularized in the same circles, names the same structure in plainer terms.
Nobody at either lab wants to skip the safety evals; that's Moloch, not malice.
Origin
Moloch is a Canaanite deity associated in the Hebrew Bible with child sacrifice; Allen Ginsberg's poem Howl (1956) made him a symbol of a devouring industrial system. Scott Alexander's essay 'Meditations on Moloch' (Slate Star Codex, July 2014) recast him as the god of runaway competition, and the usage spread through rationalist and AI-safety writing.
Usage note
Critics regard the mythic framing as fatalistic, since it can make contingent policy choices sound like laws of nature.
Related
- compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI.
- rationalist communityrationalist community, n.: A loose online community that prizes careful reasoning and spotting one's own biases; gathered around the forum LessWrong, it was one of the first groups to take the dangers of advanced AI seriously.
- AI pauseAI pause, n.: A proposed halt, temporary or indefinite, on building AI systems more powerful than today's, to give safety research and regulation time to catch up.
- existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown.
- TESCREALTESCREAL, abbr.: A critical label for a cluster of related belief systems, including transhumanism, effective altruism and longtermism, that its coiners say drive the push to build AGI and share roots with eugenics.
- Frontier AI Safety CommitmentsFrontier AI Safety Commitments, prop. n.: Promises made in 2024 by major AI companies to publish safety plans for their most advanced models, including the danger points at which they would stop if the risks could not be controlled.
Sources
- Alexander, S. (2014). Meditations on Moloch. Slate Star Codex, 30 July 2014.
Mentioned in
rationalist communityrationalist community, n.: A loose online community that prizes careful reasoning and spotting one's own biases; gathered around the forum LessWrong, it was one of the first groups to take the dangers of advanced AI seriously.