Toby Ord
In plain wordsAn Australian philosopher at Oxford who helped found effective altruism and estimated, in his book 'The Precipice', that unaligned AI is the largest threat to humanity's future this century.
Australian philosopher at the University of Oxford, a leading scholar of existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown. and one of the founders of effective altruismeffective altruism, n.: A movement that treats charity like a research problem, comparing causes by how much good each dollar or working hour achieves; many of its members came to rank risks from advanced AI as a top priority.. Giving What We Can, which he co-founded with William MacAskill in 2009, asks its members to promise a fixed share of their earnings to highly cost-effective charities, and in 2017 he and William MacAskill coined longtermismlongtermism, n.: The belief that we should care a great deal about how things go for humanity thousands or millions of years from now, since far more people may live then than have ever lived.. His book 'The Precipice: Existential Risk and the Future of Humanity' (2020) offered rough probabilities for the century ahead: about one in six for existential catastrophe overall, with unaligned AI the largest single source at about one in ten, ahead of engineered pandemics. A senior research fellow at Oxford's Future of Humanity InstituteFuture of Humanity Institute, prop. n.: Nick Bostrom's research center at Oxford University (2005–2024), where philosophers and scientists studied threats that could permanently derail humanity's future, advanced artificial intelligence among them. until its closure in 2024, he has advised the World Health Organization, the World Bank and the office of the UN Secretary-General, and has since written on how the scaling of AI computation, including test-time computetest-time compute, n.: The computing effort an AI model spends while answering a question, as opposed to while being trained. Letting a model think longer or try many answers can make it markedly smarter. and swarm scalingswarm scaling, n.: Getting better AI results by putting many AI agents on a task at once instead of letting one think longer — faster, but more expensive for each gain., bears on AI governance.
Related
- existential riskexistential risk, n.: A danger that could end humanity, or shut it out of a good future forever. Some experts count advanced AI among the largest such risks; others think this worry is overblown.
- longtermismlongtermism, n.: The belief that we should care a great deal about how things go for humanity thousands or millions of years from now, since far more people may live then than have ever lived.
- effective altruismeffective altruism, n.: A movement that treats charity like a research problem, comparing causes by how much good each dollar or working hour achieves; many of its members came to rank risks from advanced AI as a top priority.
- p(doom)p(doom), n.: Shorthand for someone's personal estimate of the chance that AI leads to catastrophe for humanity, such as extinction. People give it as a percentage, from nearly zero to nearly certain.
- global catastrophic riskglobal catastrophic risk, n.: A risk of disaster big enough to badly damage human civilization worldwide, such as killing a large share of humanity, even if humanity eventually recovers.
- Nick BostromNick Bostrom, prop. n.: A Swedish philosopher whose 2014 book 'Superintelligence' argued that AI smarter than humans could be catastrophic unless its goals are carefully controlled, and helped make AI risk a mainstream concern.
Sources
- Ord, T. (2020). The Precipice: Existential Risk and the Future of Humanity. Bloomsbury / Hachette.