Paul Christiano

American researcher in AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this. whose early work made learning from human preferences practical. After a PhD in theoretical computer science at the University of California, Berkeley, he joined OpenAIOpenAI, prop. n.: The American company that makes ChatGPT and the GPT models. It began in 2015 as a nonprofit research lab and is now a for-profit company overseen by a nonprofit foundation., where he headed the team aligning its language models (2017–2021). He was lead author of 'Deep Reinforcement Learning from Human Preferences' (2017, with colleagues at OpenAI and DeepMind), which trained a reward modelreward model, n.: A separate AI model trained to score responses the way people would, so it can stand in for human judges while another model is trained on millions of examples. from people's comparisons of agent behavior and became the basis of reinforcement learning from human feedbackreinforcement learning from human feedback, n.: A way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well.. He also proposed iterated amplificationiterated amplification, n.: A proposed training scheme in which a person, helped by many copies of an AI, solves harder problems than either could alone; a new AI then learns to imitate that team, and the cycle repeats. and co-developed AI debateAI debate, n.: A proposed way to supervise very capable AI: two AI systems argue opposite answers to a question and a human judge decides who argued better, on the theory that lies are easier to expose than to defend. as approaches to scalable oversightscalable oversight, n.: The problem of how people can check and correct AI systems whose work is too difficult, fast or plentiful for humans to judge reliably alone — usually by enlisting AI to help with the checking.. In 2021 he founded the Alignment Research CenterAlignment Research Center, prop. n.: A small Berkeley nonprofit started by Paul Christiano that searches for mathematical ways to keep future AI systems, even ones far smarter than people, honest and under human control. (ARC), whose report that year framed the problem of eliciting latent knowledgeeliciting latent knowledge, n.: The unsolved problem of getting an AI to tell us what it actually knows — for instance, that a security camera has been tampered with — rather than what it predicts we would believe. and whose evaluations team, later spun out as METRMETR, prop. n.: A nonprofit research group that checks new AI systems for dangerous skills, best known for tracking how long the tasks are that AI agents can now finish on their own., pioneered dangerous-capability evaluationsdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself. of frontier models. In April 2024 he became head of AI safety at the US AI Safety InstituteAI Safety Institute, n.: A government agency that tests the most advanced AI models for dangerous abilities and studies how to make them safe. Britain and the United States set up the first ones in 2023., which was reorganized as the Center for AI Standards and InnovationCenter for AI Standards and Innovation, prop. n.: The US government's AI testing and standards center, part of the National Institute of Standards and Technology. It evaluates advanced AI models, American and foreign, for national-security risks and works with companies on voluntary testing. in 2025.