AI Incident Database
In plain wordsA free public online collection of reports about real cases in which AI systems caused or nearly caused harm, kept so that builders and regulators can learn from past failures.
A public, crowdsourced archive of real-world cases in which deployed AI systems caused harm or came close to it; its founders took incident reporting in aviation and computer security as their model. Each entry groups one or more news or research reports describing the same event and indexes the systems, developers, deployers and harmed parties involved; editors apply taxonomies, such as one developed by the Center for Security and Emerging Technology, to classify harm type, severity and technology. The database is used to study patterns of failure, inform risk assessments and red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them., and support arguments for regulation. Coverage depends on what reaches the press, so it over-represents publicized consumer harms and under-represents failures inside organizations.
Before launch, the team searched the AI Incident Database for failures of similar hiring tools.
Origin
Launched in November 2020 under the Partnership on AIPartnership on AI, prop. n.: A forum, set up by big technology companies in 2016 and later joined by universities, charities and news organizations, that drafts voluntary guidelines for building and using AI responsibly., led by Sean McGregor, who described it in 'Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database' (2021). It is now run by the nonprofit Responsible AI Collaborative.
Related
- AI incidentAI incident, n.: A real-world case in which an AI system, through a flaw, a breakdown or the way someone used it, ends up hurting people, property or rights, as in an injury, a wrongful arrest or a privacy breach.
- Partnership on AIPartnership on AI, prop. n.: A forum, set up by big technology companies in 2016 and later joined by universities, charities and news organizations, that drafts voluntary guidelines for building and using AI responsibly.
- red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them.
- post-market monitoringpost-market monitoring, n.: The duty of an AI system's maker to keep watching how it performs after release, gathering evidence from real use so that problems are found and fixed.
- AI auditAI audit, n.: An independent check of an AI system, or of the company using it, against agreed standards — for example, testing whether a hiring tool treats women and men fairly.
Sources
- McGregor, S. (2021). Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. Proceedings of AAAI 2021 (IAAI).
Mentioned in
AI incidentAI incident, n.: A real-world case in which an AI system, through a flaw, a breakdown or the way someone used it, ends up hurting people, property or rights, as in an injury, a wrongful arrest or a privacy breach.