local differential privacy
In plain wordsA privacy method in which each person's device scrambles their data with random noise before sending it anywhere, so the company collecting it never sees anyone's true answer but can still estimate overall trends.
A variant of differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person. in which each individual randomizes their own data before it leaves their control, so that the guarantee holds even against the data collector. The classic mechanism is randomized response: answer a sensitive yes-or-no question truthfully with some probability and randomly otherwise; aggregated over many users, the true proportion can be estimated while any single answer is deniable. Because noise is added to every record rather than once to an aggregate, local privacy needs far more data for the same accuracy than the central model, where a trusted curator adds noise; intermediate designs such as the shuffle model and secure aggregationsecure aggregation, n.: A cryptographic trick that lets a server add up updates from many devices while learning only the total, never any single device's contribution. narrow the gap. It is used for telemetry such as browser settings and frequently typed words.
Local differential privacy let the keyboard team estimate popular new words without ever receiving what anyone typed.
Origin
Randomized response was proposed by Stanley Warner in 1965 for sensitive survey questions. The local model of differential privacy was formalized by Shiva Kasiviswanathan and colleagues in 'What Can We Learn Privately?' (FOCS 2008). Google's RAPPOR (Úlfar Erlingsson, Vasyl Pihur and Aleksandra Korolova, 2014) and Apple's 2016 deployment brought it to billions of devices.
Related
- differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person.
- privacy budgetprivacy budget, n.: The total amount of privacy risk a data holder agrees to spend; each analysis or release uses up part of it, and once it is gone, no more answers can be given safely.
- federated learningfederated learning, n.: Training one shared AI model across many phones, hospitals or companies without collecting their data in one place: each trains on its own data and sends back only model updates.
- secure aggregationsecure aggregation, n.: A cryptographic trick that lets a server add up updates from many devices while learning only the total, never any single device's contribution.
- privacy-preserving machine learningprivacy-preserving machine learning, n.: The family of techniques for training and using AI models while keeping the underlying personal or confidential data hidden — from adding careful noise to computing on encrypted data.
- differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person.
Sources
- Kasiviswanathan, S. P., Lee, H. K., Nissim, K., Raskhodnikova, S. & Smith, A. (2008). What Can We Learn Privately? FOCS 2008.
- Erlingsson, Ú., Pihur, V. & Korolova, A. (2014). RAPPOR: Randomized Aggregatable Privacy-Preserving Ordinal Response. ACM CCS 2014.
Mentioned in
differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person., federated learningfederated learning, n.: Training one shared AI model across many phones, hospitals or companies without collecting their data in one place: each trains on its own data and sends back only model updates., privacy budgetprivacy budget, n.: The total amount of privacy risk a data holder agrees to spend; each analysis or release uses up part of it, and once it is gone, no more answers can be given safely., secure aggregationsecure aggregation, n.: A cryptographic trick that lets a server add up updates from many devices while learning only the total, never any single device's contribution.