IAEA for AI

A proposal for an intergovernmental organization that would oversee the most capable AI systems as the International Atomic Energy Agency (IAEA) oversees nuclear technology: setting safety standards, inspecting or auditing developers and data centers, verifying compliance with agreed limits such as compute thresholdscompute threshold, n.: A legal cutoff based on how much computation was used to train an AI model. Models trained with more than that amount face extra rules, on the theory that bigger training runs produce more capable models. or AI red linesAI red lines, n.: Limits on AI that would be forbidden worldwide no matter what — for example AI that copies itself without permission or controls nuclear weapons — which campaigners want governments to agree on., and perhaps controlling access to the most dangerous capabilities. Versions range from a scientific and standard-setting body to an agency with licensing and inspection powers backed by controls on chipscompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI.. Skeptics point to disanalogies: AI capability is diffuse and mostly privately developed, there is no equivalent of fissile material to count, and software is hard to verify.

Calls for an IAEA for AI usually founder on the question of what inspectors would actually count.