Joint Distribution: Eral Rvs

Joint Distribution
• We may be interested in probability statements of sev-

eral RVs.
• Example: Two people A and B both flip coin twice.
X: number of heads obtained by A. Y : number of
heads obtained by B. Find P (X > Y ).
• Discrete case:
Joint probability mass function: p(x, y) = P (X =
x, Y = y).
– Two coins, one fair, the other two-headed. A ran-
domly chooses one and B takes the other.

1 A gets head 1 B gets head
X= Y =
0 A gets tail 0 B gets tail
Find P (X ≥ Y ).
1
• Marginal probability mass function of X can be ob-
tained from the joint probability mass function, p(x, y):
X
pX (x) = p(x, y) .
y:p(x,y)>0
Similarly:
X
pY (y) = p(x, y) .
x:p(x,y)>0
2
• Continuous case:
Joint probability density function f (x, y):
Z Z
P {(X, Y ) ∈ R} = f (x, y)dxdy
R
• Marginal pdf:
Z ∞
fX (x) = f (x, y)dy
−∞
Z ∞
fY (y) = f (x, y)dx
−∞
• Joint cumulative probability distribution function of
X and Y
F (a, b) = P {X ≤ a, Y ≤ b} − ∞ < a, b < ∞
• Marginal cdf:
FX (a) = F (a, ∞)
FY (b) = F (∞, b)
• Expectation E[g(X, Y )]:
P P
= R y R x g(x, y)p(x, y) in the discrete case
∞ ∞
= −∞ −∞ g(x, y)f (x, y)dxdy in the continuous case
3
• Based on joint distribution, we can derive
E[aX + bY ] = aE[X] + bE[Y ]
Extension:
E[a1X1 + a2X2 + · · · + anXn]
= a1E[X1] + a2E[X2] + · · · + anE[Xn]
• Example: E[X], X is binomial with n, p:

1 ith flip is head
Xi =
0 ith flip is tail
n
X n
X
X= Xi , E[X] = E[Xi] = np
i=1 i=1
• Assume there are n students in a class. What is the
expected number of months in which at least one stu-
dent was born. (Assume equal chance of being born
in any month).
Solution: Let X be the number of months some stu-
dents are born. Let Xi be the indicator RV for the
ith month
P12 in which some students are born. Then
X = i=1 Xi. Hence,
11 n
E(X) = 12E(X1) = 12P (X1 = 1) = 12 · [1 − ( ) ].
12
4
Independent Random Variables
• X and Y are independent if

P (X ≤ a, Y ≤ b) = P (X ≤ a)P (Y ≤ b)
• Equivalently: F (a, b) = FX (a)FY (b).

• Discrete: p(x, y) = pX (x)pY (y).
• Continuous: f (x, y) = fX (x)fY (y).
• Proposition 2.3: If X and Y are independent, then for
function h and g, E[g(X)h(Y )] = E[g(X)]E[h(Y )].
5
Covariance
• Definition: Covariance of X and Y

Cov(X, Y ) = E[(X − E(X))(Y − E(Y ))]
• Cov(X, X) = E[(X − E(X))2] = V ar(X).

• Cov(X, Y ) = E[XY ] − E[X]E[Y ].
• If X and Y are independent, Cov(X, Y ) = 0.
• Properties:
1. Cov(X, X) = V ar(X)
2. Cov(X, Y ) = Cov(Y, X)
3. Cov(cX, Y ) = cCov(X, Y )
4. Cov(X, Y + Z) = Cov(X, Y ) + Cov(X, Z)
6
Sum of Random Variables
• If Xi’s are independent, i = 1, 2, ..., n

n
X n
X
V ar( Xi ) = V ar(Xi)
i=1 i=1
n
X n
X
V ar( a i Xi ) = a2i V ar(Xi)
i=1 i=1
• Example: Variance of Binomial RV, sum of indepen-
dent Bernoulli RVs. V ar(X) = np(1 − p).
7
Moment Generating Functions
• Moment generating function of a RV X is φ(t)

φ(t) = E[etX ]
tx
P
x:p(x)>0 e p(x) X discrete
= R ∞ tx
−∞ e f (x)dx X continuous
• Moment of X: the nth moment of X is E[X n].

• E[X n] = φ(n)(t) | t = 0, where φ(n)(t) is the nth
order derivative.
• Example
1. Bernoulli with parameter p: φ(t) = pet + (1 − p),
for any t.
t
2. Poisson with parameter λ: φ(t) = eλ(e −1), for any
t.
• Property 1: Moment generation function of the sum
of independent RVs:
Xi, i = 1, ..., n are independent, Z = X1 + X2 + · · · +
Xn ,
n
Y
φZ (t) = φXi (t)
i=1
8
• Property 2: Moment generating function uniquely de-
termines the distribution.
• Example:
1. Sum of independent Binomial RVs
2. Sum of independent Poisson RVs
3. Joint distribution of the sample mean and sample
variance from a normal porpulation.
9
Important Inequalities
• Markov Inequality: If X is a RV that takes only non-

negative values, then for any a > 0
E[X]
P (X ≥ a) ≤ .
a
• Chebyshev’s Inequality: If X is a RV with mean µ
and variance σ 2, then for any value k > 0
σ2
P {|X − µ| ≥ k} ≤ 2 .
k
• Examples: obtaining bounds on probabilities.
10
Strong Law of Large Numbers
• Theorem 2.1 (Strong Law of Large Numbers): Let

X1, X2, ..., be a sequence of independent random
variables having a common distribution. Let E[Xi] =
µ. Then, with probability 1
X1 + X 2 + · · · + X n
→ µ as n → ∞
n
11
Central Limit Theorem
• Theorem 2.2 (Central Limit Theorem): Let X1, X2,

..., be a sequence of independent random variables
having a common distribution. Let E[Xi] = µ, V ar[Xi] =
σ 2. Then the distribution of
X1 + X2 + · · · + Xn − nµ
√
σ n
tends to the standard normal as n → ∞. That is
X1 + X2 + · · · + Xn − nµ
P{ √ ≤ z}
σZ n
z
1 2
→√ e−x /2dx = Φ(z)
2π −∞
• Example: estimate probability.
1. Let X be the number of times that a fair coin flipped
40 times lands heads. Find P (X = 20).
2. Suppose that orders at a restaurant are iid random
variables with mean µ = 8 dollars and standard
deviation σ = 2 dollars. Estimate the probability
that the first 100 customers spend a total of more
than $840. Estimate the probability that the first
100 customers spend a total of between $780 and
$820.
12

Joint Distribution: Eral Rvs

Uploaded by

Copyright:

Available Formats

Joint Distribution: Eral Rvs

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Joint Distribution: Eral Rvs

Uploaded by

Copyright:

Available Formats

Joint Distribution

• We may be interested in probability statements of sev-

• Example: E[X], X is binomial with n, p:

• X and Y are independent if

• Equivalently: F (a, b) = FX (a)FY (b).

• Definition: Covariance of X and Y

• Cov(X, X) = E[(X − E(X))2] = V ar(X).

• If Xi’s are independent, i = 1, 2, ..., n

• Moment generating function of a RV X is φ(t)

• Moment of X: the nth moment of X is E[X n].

• Markov Inequality: If X is a RV that takes only non-

• Theorem 2.1 (Strong Law of Large Numbers): Let

• Theorem 2.2 (Central Limit Theorem): Let X1, X2,

You might also like