trustworthy AI
In plain wordsAI that deserves people's trust because it works reliably, obeys the law, treats people fairly and can be checked, rather than AI that people merely happen to trust.
A policy and standards term for AI systems that warrant reliance because they meet a defined set of properties. The European Commission's High-Level Expert Group on AI (2019) framed trustworthy AI as lawful, ethical and robust, with seven requirements: human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental well-being; and accountability. The NIST AI Risk Management FrameworkNIST AI Risk Management Framework, prop. n.: A free, voluntary guide from the US government's standards agency that helps organizations spot, measure and reduce the risks of the AI systems they build or use. (2023) lists seven characteristics: valid and reliable; safe; secure and resilient; accountable and transparent; explainable and interpretable; privacy-enhanced; and fair with harmful bias managed. The concept frames the OECD AI PrinciplesOECD AI Principles, prop. n.: Guidelines for trustworthy AI that governments agreed through the OECD in 2019, the first such standard agreed between governments; many later laws and agreements borrowed from them. and much of the EU AI ActEU AI Act, prop. n.: The European Union's law on artificial intelligence, the first broad AI law anywhere: it bans some uses, sets strict checks for risky ones and puts duties on the makers of the biggest AI models..
The tender asks suppliers to show how their system meets each trustworthy-AI requirement.
Origin
Trustworthiness was an established goal in computer security and dependable computing (Microsoft's 'Trustworthy Computing' initiative dates from 2002). 'Trustworthy AI' became a policy term with the High-Level Expert Group's Ethics Guidelines for Trustworthy AI (April 2019), and the OECD AI Principles adopted the next month made it international.
Usage note
Philosophers stress the gap between being trusted and being trustworthy: persuasive design can raise trust without raising reliability, and automation biasautomation bias, n.: The habit of trusting a computer's suggestion too much: following it even when it is wrong, or missing problems because the machine did not flag them. is trust in excess of it. The aim is calibrated trust that tracks how reliable a system actually is.
Related
- responsible AIresponsible AI, n.: An umbrella name for the practices organizations use to keep their AI systems fair, safe, private and accountable, from the first design meeting to everyday use.
- AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect.
- robustnessrobustness, n.: An AI system's ability to keep working correctly and safely when conditions change — unfamiliar data, noisy inputs, or people deliberately trying to trick it.
- AI transparencyAI transparency, n.: Openness about AI: telling people when they are dealing with one, how a system was built and tested, and why it reached a particular decision.
- human oversighthuman oversight, n.: Keeping people able to watch, understand, question and, if necessary, stop or overrule an AI system, rather than letting it act unchecked.
- NIST AI Risk Management FrameworkNIST AI Risk Management Framework, prop. n.: A free, voluntary guide from the US government's standards agency that helps organizations spot, measure and reduce the risks of the AI systems they build or use.
- OECD AI PrinciplesOECD AI Principles, prop. n.: Guidelines for trustworthy AI that governments agreed through the OECD in 2019, the first such standard agreed between governments; many later laws and agreements borrowed from them.
- overrelianceoverreliance, n.: Leaning on an AI system more than its reliability justifies: accepting its answers without checking, or letting it make decisions it often gets wrong.
Sources
- High-Level Expert Group on Artificial Intelligence (2019). Ethics Guidelines for Trustworthy AI. European Commission.
- National Institute of Standards and Technology (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1.
Mentioned in
AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect., NIST AI Risk Management FrameworkNIST AI Risk Management Framework, prop. n.: A free, voluntary guide from the US government's standards agency that helps organizations spot, measure and reduce the risks of the AI systems they build or use., OECD AI PrinciplesOECD AI Principles, prop. n.: Guidelines for trustworthy AI that governments agreed through the OECD in 2019, the first such standard agreed between governments; many later laws and agreements borrowed from them., overrelianceoverreliance, n.: Leaning on an AI system more than its reliability justifies: accepting its answers without checking, or letting it make decisions it often gets wrong., responsible AIresponsible AI, n.: An umbrella name for the practices organizations use to keep their AI systems fair, safe, private and accountable, from the first design meeting to everyday use., algorithm aversionalgorithm aversion, n.: Losing faith in a computer's predictions after watching it slip up once or twice, and turning back to human judgment even though the machine remains the better forecaster., Ethics Guidelines for Trustworthy AIEthics Guidelines for Trustworthy AI, prop. n.: Guidelines written for the European Commission by an expert group in 2019, listing seven requirements AI should meet to be trustworthy; they paved the way for the EU AI Act., ISO/IEC 22989ISO/IEC 22989, prop. n.: An international standard that fixes agreed meanings for basic AI words — such as AI system, machine learning and trustworthiness — so that other standards, contracts and laws can use the same vocabulary.