Skip to content
Last Dictionary
Plane
Governance, Policy & LawSafety & AlignmentSociety, Ethics & Economy
legal

human oversight

nounAlso: human control, human-on-the-loop, human-in-command

In plain wordsKeeping people able to watch, understand, question and, if necessary, stop or overrule an AI system, rather than letting it act unchecked.

  1. 1

    EU AI Act

    The duty to build high-risk AI systemshigh-risk AI system, n.: Under EU law, an AI system used where mistakes could seriously hurt people's safety or rights — in hiring, schooling, loans, policing, borders or products like medical devices — and which must therefore pass strict checks. so that people can effectively supervise them during operation, as a safeguard against harm to health, safety and fundamental rights, especially harm that the Act's other requirements do not remove. To a degree matching the risk, autonomy and setting, the provider must equip those supervisors to grasp what the system can and cannot do and read its outputs correctly, to watch for malfunctions and unexpected behavior, to resist over-trusting it (automation biasautomation bias, n.: The habit of trusting a computer's suggestion too much: following it even when it is wrong, or missing problems because the machine did not flag them.), and to set an output aside, overrule it or bring the system to a safe halt, for example with a stop button. Deployersprovider and deployer, phr.: The two main roles in EU AI law: the provider is whoever makes an AI system and puts it out under its own name; the deployer is whoever uses it for work or official purposes. must give the role to staff who are qualified, trained and empowered to perform it. For high-risk remote biometric identification, no action may follow a match until two such people have each checked it, with narrow exceptions.

    The hospital named two radiologists to exercise human oversight of the triage system.

  2. 2

    General

    In AI ethics and governance broadly, the principle that people remain able to monitor, intervene in and answer for automated decisions. It appears in the OECD AI PrinciplesOECD AI Principles, prop. n.: Guidelines for trustworthy AI that governments agreed through the OECD in 2019, the first such standard agreed between governments; many later laws and agreements borrowed from them. and the UNESCO Recommendation on the Ethics of AIUNESCO Recommendation on the Ethics of AI, prop. n.: A worldwide set of ethical principles for AI that all of UNESCO's member countries agreed in 2021, covering human rights, fairness, privacy, the environment, gender equality and education., and the EU's Ethics Guidelines for Trustworthy AIEthics Guidelines for Trustworthy AI, prop. n.: Guidelines written for the European Commission by an expert group in 2019, listing seven requirements AI should meet to be trustworthy; they paved the way for the EU AI Act. (2019) grade it in three degrees: human-in-the-loophuman-in-the-loop, adj.: Describes AI systems in which a person checks, corrects or approves the machine's work at key points, rather than letting it act entirely on its own., able to intervene in each decision; human-on-the-loop, monitoring operation with the power to step in; and human-in-command, deciding whether, when and how a system is used at all.

Origin

Human control of automation is an old theme in engineering, and 'meaningful human control' became a rallying phrase in debates on autonomous weapons from about 2013. 'Human agency and oversight' was the first of seven requirements in the EU High-Level Expert Group's Ethics Guidelines for Trustworthy AI (2019), and Article 14 of the AI Act (2024) turned it into law.

Usage note

Research on automation biasautomation bias, n.: The habit of trusting a computer's suggestion too much: following it even when it is wrong, or missing problems because the machine did not flag them. and overrelianceoverreliance, n.: Leaning on an AI system more than its reliability justifies: accepting its answers without checking, or letting it make decisions it often gets wrong. shows that nominal oversight can be hollow: a person who rubber-stamps outputs adds little protection, so commentators distinguish formal from effective oversight. Not the same as scalable oversightscalable oversight, n.: The problem of how people can check and correct AI systems whose work is too difficult, fast or plentiful for humans to judge reliably alone — usually by enlisting AI to help with the checking., an alignment research problem about supervising systems more capable than their overseers.

Related

See also
  • human-in-the-loophuman-in-the-loop, adj.: Describes AI systems in which a person checks, corrects or approves the machine's work at key points, rather than letting it act entirely on its own.
  • automation biasautomation bias, n.: The habit of trusting a computer's suggestion too much: following it even when it is wrong, or missing problems because the machine did not flag them.
  • high-risk AI systemhigh-risk AI system, n.: Under EU law, an AI system used where mistakes could seriously hurt people's safety or rights — in hiring, schooling, loans, policing, borders or products like medical devices — and which must therefore pass strict checks.
  • overrelianceoverreliance, n.: Leaning on an AI system more than its reliability justifies: accepting its answers without checking, or letting it make decisions it often gets wrong.
  • kill switchkill switch, n.: A way to quickly shut down an AI system, or halt its development, if it starts behaving dangerously — anything from an off button to a company's promise to stop.
  • provider and deployerprovider and deployer, phr.: The two main roles in EU AI law: the provider is whoever makes an AI system and puts it out under its own name; the deployer is whoever uses it for work or official purposes.
Contrast with
  • scalable oversightscalable oversight, n.: The problem of how people can check and correct AI systems whose work is too difficult, fast or plentiful for humans to judge reliably alone — usually by enlisting AI to help with the checking.

Defined in law or standards

EU AI Act (Reg. (EU) 2024/1689), Art. 14; deployer duty Art. 26(2)

Mentioned in

agentic AIagentic AI, n.: AI built to pursue goals on its own — planning, taking actions and adjusting along the way — instead of only responding to one prompt at a time. 'Agentic' describes how much of this independence a system has., EU AI ActEU AI Act, prop. n.: The European Union's law on artificial intelligence, the first broad AI law anywhere: it bans some uses, sets strict checks for risky ones and puts duties on the makers of the biggest AI models., high-risk AI systemhigh-risk AI system, n.: Under EU law, an AI system used where mistakes could seriously hurt people's safety or rights — in hiring, schooling, loans, policing, borders or products like medical devices — and which must therefore pass strict checks., levels of driving automationlevels of driving automation, n.: A six-step scale, from Level 0 to Level 5, describing how much of the driving a car's automation does and how much the human must still watch the road and be ready to take over., advanced driver-assistance systemadvanced driver-assistance system, n.: Car features that help the driver but do not replace them, such as automatic emergency braking, lane-keeping help, adaptive cruise control and blind-spot warnings., AI as normal technologyAI as normal technology, phr.: The view that AI, however important, will change society gradually, like electricity or the internet, through slow adoption by people and institutions, rather than as a sudden, uncontrollable superintelligence., algorithmic managementalgorithmic management, n.: When software, rather than a human manager, assigns work, sets pay, tracks performance and even disciplines or fires workers, as in ride-hailing, delivery and warehouse jobs., automation biasautomation bias, n.: The habit of trusting a computer's suggestion too much: following it even when it is wrong, or missing problems because the machine did not flag them., autonomyautonomy, n.: How much an AI system can act on its own without a person approving each step, from merely making suggestions to carrying out long tasks with no supervision at all., conformity assessmentconformity assessment, n.: The check, done by the maker or by an approved outside body, that a high-risk AI system meets the EU's legal requirements before it can be sold or used., Framework Convention on Artificial IntelligenceFramework Convention on Artificial Intelligence, prop. n.: The first binding international treaty on AI, drawn up by the Council of Europe, in which countries promise that AI used by governments and businesses will respect human rights, democracy and the rule of law., harmonised standardsharmonised standards, n.: Official European technical standards, written at the EU's request, that companies can follow to show their AI systems meet the law, instead of proving it from scratch., human-in-the-loophuman-in-the-loop, adj.: Describes AI systems in which a person checks, corrects or approves the machine's work at key points, rather than letting it act entirely on its own., levels of autonomylevels of autonomy, n.: Ladders that grade how independently an AI system acts — from a tool a person fully controls, through suggesting and asking for approval, up to acting on its own while people merely watch., overrelianceoverreliance, n.: Leaning on an AI system more than its reliability justifies: accepting its answers without checking, or letting it make decisions it often gets wrong., permission systempermission system, n.: The rules that decide what an AI agent may do on its own, what it must ask a person about first, and what it may never do — like a phone's settings for which apps can use the camera., provider and deployerprovider and deployer, phr.: The two main roles in EU AI law: the provider is whoever makes an AI system and puts it out under its own name; the deployer is whoever uses it for work or official purposes., remote biometric identificationremote biometric identification, n.: Using AI to recognize people at a distance — usually by matching faces in camera footage against a database — without their taking part. The EU strictly limits live police use of it in public places., responsibility gapresponsibility gap, n.: The problem that when an AI system that learns or acts on its own causes harm, it may be unclear whether anyone (its makers, users or operators) can fairly be held responsible., responsible AIresponsible AI, n.: An umbrella name for the practices organizations use to keep their AI systems fair, safe, private and accountable, from the first design meeting to everyday use., scalable oversightscalable oversight, n.: The problem of how people can check and correct AI systems whose work is too difficult, fast or plentiful for humans to judge reliably alone — usually by enlisting AI to help with the checking., trustworthy AItrustworthy AI, n.: AI that deserves people's trust because it works reliably, obeys the law, treats people fairly and can be checked, rather than AI that people merely happen to trust., UNESCO Recommendation on the Ethics of AIUNESCO Recommendation on the Ethics of AI, prop. n.: A worldwide set of ethical principles for AI that all of UNESCO's member countries agreed in 2021, covering human rights, fairness, privacy, the environment, gender equality and education., AI Basic ActAI Basic Act, prop. n.: South Korea's main AI law, in force since January 2026, which sets safety and transparency duties for AI used in sensitive areas and for AI-generated content.