contextual integrity
In plain wordsA way of thinking about privacy: information is not simply public or private; what matters is whether it flows in ways that fit the expectations of the setting where it was shared.
A theory of privacyprivacy, n.: Your ability to control who knows what about you. AI raises new privacy questions because it learns from huge amounts of personal data and can infer things about people they never shared. holding that every flow of personal information is governed by norms specific to its context, and that privacy is violated when a flow breaches those norms — as when medical details told to a doctor reach an employer. A flow is described by five parameters: sender, recipient, subject, type of information, and transmission principle, such as confidentiality, consent or legal compulsion. The framework explains why aggregating or repurposing information that is individually public can still cause harm, which a simple public–private split cannot. In AI it has become a standard lens for judging whether language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. and agentsAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done. handle personal information appropriately — for instance, whether an assistant with access to a user's inbox reveals one person's secret to another — through benchmarks such as ConfAIde and PrivacyLens, and for designing agents that reason about which disclosures a context allows.
Under contextual integrity, the agent should not paste a colleague's medical-leave note into the team channel.
Origin
Introduced by the philosopher Helen Nissenbaum in 'Privacy as Contextual Integrity' (Washington Law Review, 2004) and developed in her book Privacy in Context (2010). Niloofar Mireshghallah and colleagues brought it into language-model evaluation with the ConfAIde benchmark (2023).
Related
- privacyprivacy, n.: Your ability to control who knows what about you. AI raises new privacy questions because it learns from huge amounts of personal data and can infer things about people they never shared.
- data protectiondata protection, n.: Laws and practices that control how organizations collect, use, store and share information about people, and that give people rights over it, such as seeing it or having it deleted.
- personally identifiable informationpersonally identifiable information, n.: Information that can identify a specific person, such as a name, address, phone number or ID number — or several facts that together point to one person.
- AI agentAI agent, n.: An AI system that does not just answer but acts: given a goal, it decides on steps, uses tools such as a browser or code editor, checks the results and keeps going until the job is done.
- surveillance capitalismsurveillance capitalism, n.: A business model in which companies collect huge amounts of data about people's behavior and use it to predict and influence what they will do, mainly to sell advertising.
Sources
- Nissenbaum, H. (2004). Privacy as Contextual Integrity. Washington Law Review 79(1).
- Mireshghallah, N. et al. (2024). Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory. ICLR 2024.
Mentioned in
privacyprivacy, n.: Your ability to control who knows what about you. AI raises new privacy questions because it learns from huge amounts of personal data and can infer things about people they never shared.