machine ethics
In plain wordsThe effort to build AI systems that can tell right from wrong and act ethically themselves, rather than only being used ethically by people.
The field concerned with designing artificial agents whose own behavior is guided by ethical considerations, so-called artificial moral agents. Approaches range from top-down (encoding rules or theories, such as deontic logic, utilitarian calculation or Asimov-style lawsThree Laws of Robotics, n.: Three rules that science-fiction writer Isaac Asimov built into his fictional robots — do not harm people, obey orders, protect yourself — in stories that mostly show how such simple rules go wrong.) to bottom-up (learning moral judgments from examples or human feedback) and hybrids. James Moor distinguished ethical-impact agents, implicit ethical agents designed to avoid unethical outcomes, explicit ethical agents that reason about ethics, and full ethical agents. Large studies of human moral preferences, such as the Moral Machine experiment on self-driving-car dilemmas (about 40 million decisions), have informed the field. With language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., machine ethics overlaps with AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this. and value specification, as in constitutionsAI constitution, n.: A written set of principles that an AI company uses to shape an AI's values and behavior, both during training and as a public statement of what the AI should be like. and model specificationsmodel specification, n.: A public document in which an AI company spells out how its models are supposed to behave — what they should do, refuse and prioritize, and whom to obey when instructions conflict. that state the values a model should follow.
The machine ethics question is not whether the robot is used well, but whether it can recognize that an order is wrong.
Origin
The earliest known use is in Mitchell Waldrop's AI Magazine article 'A Question of Responsibility' (1987), which called for 'a theory and practice of machine ethics' inspired by Asimov's laws of robotics. It became the name of a research field with Michael Anderson, Susan Leigh Anderson and Chris Armen's 'Towards Machine Ethics' (2004) and the AAAI Fall Symposium on Machine Ethics (2005); Colin Allen, Wendell Wallach and Iva Smit's 'Why Machine Ethics?' (2006), James Moor's typology (2006) and Wallach and Allen's Moral Machines (2009) defined it.
Usage note
Distinct from AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect. in its usual sense, the ethics of how people build and use AI. Whether machines can be genuine moral agents, rather than systems that behave as if they were, is a question for philosophy of mind.
Related
- AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect.
- AI alignmentAI alignment, n.: Making an AI system genuinely try to do what the people it serves want — not a literal-minded, distorted or self-serving version of it. Also the research field working on this.
- value alignmentvalue alignment, n.: Building AI whose goals reflect human values — what people care about and consider right — rather than one that merely follows orders or chases a narrow target.
- AI constitutionAI constitution, n.: A written set of principles that an AI company uses to shape an AI's values and behavior, both during training and as a public statement of what the AI should be like.
- Three Laws of RoboticsThree Laws of Robotics, n.: Three rules that science-fiction writer Isaac Asimov built into his fictional robots — do not harm people, obey orders, protect yourself — in stories that mostly show how such simple rules go wrong.
- moral patienthoodmoral patienthood, n.: Being the kind of being whose interests matter morally for its own sake, so that it can be wronged. Babies and animals count, even though they are not responsible for what they do.
- responsibility gapresponsibility gap, n.: The problem that when an AI system that learns or acts on its own causes harm, it may be unclear whether anyone (its makers, users or operators) can fairly be held responsible.
- AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect.
Sources
- Moor, J. H. (2006). The Nature, Importance, and Difficulty of Machine Ethics. IEEE Intelligent Systems 21(4), 18–21.
- Wallach, W. & Allen, C. (2009). Moral Machines: Teaching Robots Right from Wrong. Oxford University Press.
- Awad, E. et al. (2018). The Moral Machine experiment. Nature 563, 59–64.
Mentioned in
AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect., responsibility gapresponsibility gap, n.: The problem that when an AI system that learns or acts on its own causes harm, it may be unclear whether anyone (its makers, users or operators) can fairly be held responsible., trolley problemtrolley problem, n.: A thought experiment asking whether you should divert a runaway tram so that it kills one person instead of five, often used to ask how self-driving cars should choose in an unavoidable crash.