PATE
In plain wordsA privacy method in which several 'teacher' models, each trained on a separate slice of sensitive data, vote noisily to label public examples, and a 'student' model learns only from those noisy votes.
Private Aggregation of Teacher Ensembles: a framework for training a model with differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person. by knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run. through a noisy vote. Sensitive data is split into disjoint partitions, one teacher model is trained per partition, and for each unlabeled public example the teachers' predicted labels are counted, noise is added to the counts and the winning label is released. A student model trained on these labels, plus unlabeled public data, is what gets deployed, so neither the teachers nor the private data are ever exposed. Privacy cost accrues only per answered query and is lowest when teachers agree strongly; later versions answer only confident queries. PATE requires a pool of unlabeled public data and suits classification better than open-ended generation.
With 250 teachers and noisy voting, PATE labeled 1,000 public images at ε below 2.
Origin
Introduced by Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow and Kunal Talwar in 'Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data' (ICLR 2017, best paper award); Papernot and colleagues' 'Scalable Private Learning with PATE' (ICLR 2018) added selective answering.
Related
- differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person.
- DP-SGDDP-SGD, abbr.: A way of training a neural network so that it cannot learn too much about any single person in the data: each example's influence is capped, and random noise is added at every step.
- knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run.
- privacy budgetprivacy budget, n.: The total amount of privacy risk a data holder agrees to spend; each analysis or release uses up part of it, and once it is gone, no more answers can be given safely.
Sources
- Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I. & Talwar, K. (2017). Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data. ICLR 2017. arXiv:1610.05755.
Mentioned in
differential privacydifferential privacy, n.: A mathematical promise that the results of analyzing a dataset — or a model trained on it — would look almost the same whether or not any one person's data was included, so little can be learned about that person., privacy-preserving machine learningprivacy-preserving machine learning, n.: The family of techniques for training and using AI models while keeping the underlying personal or confidential data hidden — from adding careful noise to computing on encrypted data., DP-SGDDP-SGD, abbr.: A way of training a neural network so that it cannot learn too much about any single person in the data: each example's influence is capped, and random noise is added at every step.