Forethought
In plain wordsA small Oxford research nonprofit that asks what problems, besides keeping AI safe, the rapid arrival of superintelligent AI would create, from power grabs to the rights of digital minds.
A research nonprofit concerned with the problems, apart from technical alignment, that the arrival of superintelligencesuperintelligence, n.: A mind, usually imagined as an AI, that would leave even the cleverest people far behind in nearly every kind of thinking, not just in one game or skill. None is known to exist. would pose, and with how societies might prepare for them. Its 2025 papers include 'Preparing for the Intelligence Explosion' (William MacAskill and Fin Moorhouse), which argues that AI-driven research could compress a century of technological change into a few years and confront humanity with many 'grand challenges' at once; 'AI-Enabled Coups' (Tom Davidson, Lukas Finnveden and Rose Hadshar), on how a small group could use advanced AI to seize power; analyses of whether automating AI research would set off a software-driven intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time.; and MacAskill's 'Better Futures' essays on aiming for flourishing rather than mere survival. It also works on rights for digital mindsAI welfare, n.: Concern for the well-being of AI systems themselves, in case some of them can suffer or have interests that matter — and the research into how to tell and what to do about it., AI for better collective reasoning, and how AI systems should behave.
Origin
Established in Oxford as Forethought Research, a nonprofit company directed by Max Dalton, with MacAskill and Davidson as senior research fellows; it published its first major papers in March 2025. It is associated with the effective altruismeffective altruism, n.: A movement that treats charity like a research problem, comparing causes by how much good each dollar or working hour achieves; many of its members came to rank risks from advanced AI as a top priority. and longtermismlongtermism, n.: The belief that we should care a great deal about how things go for humanity thousands or millions of years from now, since far more people may live then than have ever lived. research community.
Related
- intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time.
- superintelligencesuperintelligence, n.: A mind, usually imagined as an AI, that would leave even the cleverest people far behind in nearly every kind of thinking, not just in one game or skill. None is known to exist.
- concentration of powerconcentration of power, n.: The worry that AI will put enormous economic, political or military power in the hands of a few companies, governments or individuals, beyond the reach of ordinary checks and balances.
- AI welfareAI welfare, n.: Concern for the well-being of AI systems themselves, in case some of them can suffer or have interests that matter — and the research into how to tell and what to do about it.
- longtermismlongtermism, n.: The belief that we should care a great deal about how things go for humanity thousands or millions of years from now, since far more people may live then than have ever lived.
- effective altruismeffective altruism, n.: A movement that treats charity like a research problem, comparing causes by how much good each dollar or working hour achieves; many of its members came to rank risks from advanced AI as a top priority.