Professional Documents
Culture Documents
Nips17 DPML Tutorial
Nips17 DPML Tutorial
Machine Learning
Theory, Algorithms, and Applications
Medical Records
Genetic Data
Search Logs
AOL Violates Privacy
AOL Violates Privacy
Netflix Violates Privacy [NS08]
Movies%
User%1%
User%2%
User%3%
Cancer Healthy
Correlations Correlations
Privacy
privacy barrier
Statistics
summary
statistic
Privacy-preserving synthetic
Data release
Private
data set sanitizer dataset
D ML model
Data Sanitizer Public
non-public
(sensitive) public
Property of Sanitizer
privacy barrier
summary
Statistics
statistic
Privacy-preserving synthetic
Private
data set sanitizer dataset
Data release
D ML model
Datanon-public
Sanitizer public
Public
Prior Knowledge:
A’s Genetic profile
A smokes
Case 1: Study
A has
cancer
Adversary
Cancer
[ Study violates A’s privacy ]
Prior Knowledge:
A’s Genetic profile
A smokes
Case 1: Study
A has
cancer
Adversary
Cancer
[ Study violates A’s privacy ]
Case 2: Study
Prior Knowledge:
A probably
A’s Genetic profile has cancer
A smokes
…through randomness
A( Data + )
Random have close
variables distributions
A(Data + )
How to ensure this?
A( Data + )
Random have close
variables distributions
A(Data + )
D D’
p[A(D) = t] p[A(D’) = t]
t
For all D, D’ that differ in one person’s value,
If A = ✏ -differentially private randomized algorithm, then:
p(A(D) = t)
sup log 0
✏
t p(A(D ) = t)
Differential Privacy [DMNS06]
D D’
p[A(D) = t] p[A(D’) = t]
t
For all D, D’ that differ in one person’s value,
If A = ✏ -differentially private randomized algorithm, then:
Max-divergence of p(A(D) = t)
sup log ✏
p(A(D)) and p(A(D’)) t
0
p(A(D ) = t)
Approx. Differential Privacy [DKM+06]
D D’
p[A(D) = t] p[A(D’) = t]
t
For all D, D’ that differ in one person’s value,
If A = (✏, ) -differentially private randomized algorithm, then:
h Pr(A(D) 2 S) i
max log ✏
S,Pr(A(D)2S)> Pr(A(D0 ) 2 S
Properties of
Differential Privacy
Property 1: Post-processing Invariance
privacy barrier
legitimate user 1 (✏1 , 1)
eriv ative
d
data
differentially non-private data derivative (✏1 ,
Private private post-
legitimate user 2 1)
data set algorithm processing
D data der
ivative
adversary (✏1 ,
(✏1 , 1) 1)
ATraining
1 (✏1 , ("
1) ,
2 2)
X X
Private ( ✏i , i)
data set A2 (✏2 , 2)
Cross-
validation
("3 , 3)
i i
D …
Testing ("4 , 4)
A
Problem:
Given function f, sensitive dataset D
Find a differentially private approximation to f(D)
Global Sensitivity of f:
The Global Sensitivity Method [DMNS06]
Global Sensitivity of f:
S(f) = | f(D) - f(D’)|
Domain(D)
D’
D
The Global Sensitivity Method [DMNS06]
Global Sensitivity of f:
S(f) = | f(D) - f(D’)|
dist(D, D’) = 1
Domain(D)
D’
D
The Global Sensitivity Method [DMNS06]
Global Sensitivity of f:
S(f) = max | f(D) - f(D’)|
dist(D, D’) = 1
Domain(D)
D’
D D’
D
Laplace Mechanism
Output where
✏-differentially
private
Laplace distribution:
Gaussian Mechanism
Output where
S(f )
Z⇠ N (0, 2 ln(1.25/ )) (✏, )-differentially
✏
private
Example 1: Mean
Problem:
Given function f(w, D), Sensitive Data D
Find differentially private approximation to
⇤
w = argmax f (w, D)
w
f(w, D) Pr(w)
w w
Example: Parameter Tuning
— Motivation
— What is differential privacy?
— Basic differential privacy algorithm design tools:
— The Global Sensitivity Method
— Laplace Mechanism
— Gaussian Mechanism
— Exponential Mechanism
Differential privacy in
estimation and prediction
Estimation and prediction problems
privacy barrier
f (w, ·) risk functional
Private (", )
data set
DP estimate of
D ŵ private estimator
argmin f (w, D)
sample size n w E[f (ŵ, z)] E[f (w⇤ , z)]
Xn
⇤ 1
w = argmin `(w, (xi , yi )) + R(w)
w n i=1
– – data set
D or D0 ?
– – D
w
0⇤
w
+
+ + –+
adversary
– + Private
– – data set
+ – – D 0
[CMS11, RBHT12]
Kernel learning: even worse
+
+ + –+
n
X
w(x) = ↵i k(x, xi ) – + +
i=1
– –
– –
[CMS11]
Privacy is compatible with learning
input
private
private sanitized
perturbation
database
database non-private
sanitized dataset non-non-
input
DPDP
optimization
database
optimization
preprocessing
ŵ
private
private
objective
perturbation algorithm
learning
D input
perturbation
(",(",)w
)
⇤
noise addition
input
perturbation
or
randomization
ŵoutput
input
privacy (", ) privacy
privacy
(", ) barrier barrier
barrier
Local Privacy
input
perturbation
input
perturbation
input
perturbation
sanitized non-
input database private
perturbation algorithm
input
perturbation
input
perturbation
privacy
(", ) barrier
private sanitized
database dataset non-
DP
private
preprocessing
learning
D (", )
ŵinput
privacy
barrier
private
database non-private
optimization
D w ⇤
noise addition
or
randomization
ŵoutput
(", ) privacy
barrier
non-private
preprocessing
private
database
DP
optimization
ŵobjective
D (", )
privacy
barrier
Objective Perturbation
Xn
1
J(w) = `(w, (xi , yi )) + R(w)
n i=1
ˆ
ŵpriv = argmin J(w)
w
⇤ >
ŵpriv = argmin J(w) +
w = argmin J(w) w b
w w
⇤
• Non-private optimization solves rJ(w) = 0 =) w
• Optimization is iterative
intermediate results leak information
Non-private SGD
Xn
1
J(w) = `(w, (xi , yi )) + R(w)
n i=1
e✏
g+
1 + e✏
1
ĝ =
B
g 1 + e✏
Both methods
2"✏
Differentially
private algorithm
Private
data set
subsample
D n
D data der
ivative
adversary (✏1 ,
(✏1 , 1) 1)
Training ("2 , 2)
Private
data set Cross-
validation
("3 , 3)
D
Testing ("4 , 4)
Composition property of
differential privacy
Basic composition: privacy loss is additive:
Training ("2 , 2)
Private
data set Cross-
validation
("3 , 3)
D
Testing ("4 , 4)
p(⌧2 |D)
1
2
|D)
p(⌧1p(⌧ 2 |D)
exp(" 2 )
p(⌧ |D)
p(⌧21|D 0)
0
exp("1 )
p(⌧1 |D )
p(⌧2 |D00 )
p(⌧1 |D )
⌧1 ⌧2
Privacy loss as a random
variable
Spectrum of (ε, δ)
guarantees means we can
trade off ε and δ when
analyzing a particular
mechanism.
• Composition again
(k 2i)", 1 (1 )k (1 i) gives a family of
(ε, δ) tradeoffs: can
quantify privacy loss
Pi 1 k by choosing any
`=0 ` e(k `)" e(k 2i+`)"
valid (ε, δ) pair.
i =
(1 + e" )k
[DRV10, KOV15]
Moments accountant
privacy barrier
Preprocessing ("1 , 1)
A
Training1 ("2 , 2)
Private
data set A 2 A (✏, )
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
Basic Idea: Directly calculate parameters (ε,δ)
from composing a sequence of mechanisms
More efficient than composition theorems
[ACG+16]
How to Compose Directly?
Given datasets D and D’ with one different record,
mechanism A, define privacy loss random variable as:
p(A(D) = t)
ZD,D0 = log 0
, w.p. p(A(D) = t)
p(A(D ) = t)
- Properties of ZD,D’
related to privacy loss of A
A
Training1 ("2 , 2)
Private
data set A 2 A (✏, )
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
Three Steps:
1. Calculate moment generating functions for A1, A2, ..
2. Compose
3. Calculate final privacy parameters
[ACG+16]
1. Stepwise Moments
privacy barrier
Preprocessing ("1 , 1)
A
Training1 ("2 , 2)
Private
data set A 2 A
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
Define: Stepwise Moment at time t of At at any s:
↵At (s) = sup log E[e sZD,D0
] (D and D’ differ
D,D 0 by one record)
[ACG+16]
2. Compose
privacy barrier
Preprocessing ("1 , 1)
A
Training1 ("2 , 2)
Private
data set A 2 A
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
A
Training1 ("2 , 2)
Private
data set A 2 A (✏, )
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
A
Training1 ("2 , 2)
Private
data set A 2 A (✏, )
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
A
Training1 ("2 , 2)
Private
data set A 2 A
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
A
Training1 ("2 , 2)
Private
data set A 2 A
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
A
Training1 ("2 , 2)
Private
data set A 2 A (✏, )
Cross-
validation
("3 , 3)
D …
Testing ("4 , 4)
[DRV10]
(better than linear)
epsilon
Moments
Accountant
#rounds of composition
[ACG+16]
How does it compare
on real data?
EM for MOG
with Gaussian
Mechanism
= 10 4
[PFCW17]
Summary
• Practical machine learning looks at the data many times.
posterior
original
L 1 , L2 , . . .
select random
lot new network
Private weights
data set
D
x L1 , x L2 , . . . compute
gradients
✓T
processed
posterior
moments clip and update
accountant add noise parameters
(", ) privacy
barrier
[ACG+16]
Overview of the algorithm
L 1 , L2 , . . .
select random
lot new network
Private weights
data set
D
x L1 , x L2 , . . . compute
gradients
✓T
moments clip and update
accountant add noise parameters
(", ) privacy
barrier
[ACG+16]
Effectiveness of DP deep learning
+ =
Prior ⇡(✓) Data X Posterior p(✓|X)
Advantage: General
Pitfalls:
— Intractable - only exact
distribution private
— Low statistical efficiency
even for large n
How to make posterior private?
Option 3: Approximate the OPS distribution via
Stochastic Gradient MCMC [WFS15]
−2
−2.5
Test−set log−likelihood
−3
−3.5
−4
Non−private HMM
−4.5 Non−private naive Bayes
Laplace mechanism HMM
OPS HMM (truncation multiplier = 100)
−5
−1 0 1
10 10 10
Epsilon (total)
Statistically efficient
Performance worse than non-private, better than OPS
Can do inference in relatively complex systems by building
up on this method — eg, time series clustering in HMMs
Differentially Private
Bayesian Inference
[JDH16, PFCW16]
Summary
• Deep learning
• Bayesian inference
Summary, conclusions,
and where to look next
Summary
https://simons.berkeley.edu/workshops/bigdata2013-4
http://www.nowpublishers.com/article/Details/TCS-042
Final Takeaways