Input Data Analysis (3)

Goodness-of-Fit Tests

1
 Input Data Analysis

• Activity I: Hypothesizing families of

distributions (“What does it look like?”)
• Activity II: Estimation of parameters
(“How is it represented?”)
• Activity III: Determining how
representative the fitted distributions are
(“How accurate is the representation?”)

2
 Goodness-of-Fit Test

• Used to test hypothesis that a given number

of data points (e.g., collected from the
system - field data) are independent samples
from a particular probability distribution
Observation: X1, X2, X3, …, Xn
H0: Xi’s are IID random variables with
distribution function F
e.g., uniform, normal, exponential, etc.

3
 Goodness-of-Fit Test
• Note:
– Failure to reject H0 should NOT be
interpreted as “accepting H0 as being true”
– Because these tests are not very powerful
for small to moderate sample size n (often
real field data are limited in size) - not
sensitive to subtle deviation between
sample data and the fitted distribution
– If n is very large, then these test will almost
always reject H0, since H0 is virtually never
exactly true
– But for practical reasons, “nearly” correct is
acceptable 4
 The Chi-Square Test
• Pearson (1900)
• First, divide the entire range of the fitted distribution
into k adjacent intervals with equal lengths except
the 1st and the last: [a0, a1), [a1, a2), [a2, a3), …, [ak-1,
ak]
a0 = -, ak = +  First: (-, a1), Last: [ak, +)
• Then tally
Nj = number of Xi’s in the jth interval [aj-1, aj)
for j = 1, 2, …, k (note: (j=1 to k) Nj = n)
• Compute the expected Proportion Pj of the Xi’s that
would fall in the jth interval
5
 The Chi-Square Test
• For continuous case
aj
pj   fˆ ( x)dx
a j 1

whrere f(x) is the density function of the fitted distribution

• For discrete data (our testing interest)

pj   pˆ ( xi )
a j 1  xi  a j

where pˆ is the mass function of the fitted distributi on

6
 The Chi-Square Test
• Test statistics K
2  
( N j  np j ) 2
j 1 np j

• Since npj is the expected number of n Xi’s

that would fall in the jth interval
– If H0 were true, we would expect 2 to be small
– Reject H0, if 2 is too large
• Testing for large n with an approximate 
– Reject H0 is: 2 > 2k-1,1- (L&K: Fig. 6.45)
where 2k-1,1- is the upper 1- critical point for a
Chi-Square distribution with k-1 d.f. 7
 The Chi-Square Test
• The Chi-Square test is only valid, i.e., of
level , asymptotically as n  
(L&K: Fig. 6.46)
• Also, need to estimate m parameters of
the fitted distribution (m  1)
• When maximum likelihood estimates are
used, if H0 is true, as n   the
distribution of 2 converges to a
distribution that lies between the
distribution functions of Chi-Square
distributions with k-1 and k-m-1 d.f.
9
 The Chi-Square Test
• Clear-Cut Cases
– If 2 > 2k-1,1-  reject H0
– If 2 < 2k-1,1-  do not reject H0
– What if 2k-m-1,1-  2  2k-1,1-
Recommend:
Reject H0 only if 2 > 2k-1,1-  conservative
i.e., the actual probability ’ of committing a type I
error (reject H0 when H0 is true) is at least as
small as the stated probability 
Result: loss of power (small probability of
rejecting a false H0) of the test
11
 The Chi-Square Test
• Clear-Cut Cases (Continue)
– Normally m2 and if k is fairly large, the
difference between 2k-m-1,1- and 2k-1,1- is very
small
• Difficulty:
– Choose of number of intervals
1. Equal length
2. Equal-probable (p1= p2 = … = pk)
- difficult in practice
• Less conservative: (maybe more practical)
– If 2 > 2k-m-1,1-  reject H0 12
 The Chi-Square Test
Ex: Time between Arrival (TBA). 60 data points
(all greater than 0) and Exp(20.1)
2
Cell Frequency Theoretical prop. ( N j  np j )
npj np j
- [0,10) 19 23.52 0.869
[10,20) 16 14.31 0.200
[20,30) 12 8.69 1.261
[30,40) 8 5.29 1.388
[40,50) + 5 3.21 1.236
 = 60 2 = 4.954
f(x) = (1/20.1) e-x/20.1 for x > 0
F(x) = 1 - e-x/20.1 [F(x) = (from 0 to x) f(x) dx]
df = 5 - 1 - 1 = 3  2 = 4.954 < 23,1-0.05 = 7.81 13
 The Chi-Square Test
• Ex: Inventory Model
– n = 156 observations on the (discrete)
number of items demanded in a week from
an inventory over a 3-year period. The
weekly demands are:
0 (59) 1 (26) 2 (24) 3 (18) 4 (12)
5 (5) 6 (4) 7 (3) 9 (3) 11 (2)
hj
0.4 0.346 Mass function
0.3
0.2
0.1
xj
0
0 1 2 3 4 5 6 7 8 9 10 11 14
 The Chi-Square Test
• Ex: Inventory Model (Continue)
– Summary Statistics
Min. = 0.000 Max. = 11.000 Mode = 0
Mean = 1.891 Median = 1.000 Variance = 5.285
Lexis ratio  = 2/ = 2.795
Skewness  = E[(X - )3]/(2)3/2 = 1.655
– Try to fit a geometric distribution geom(0.346)
• With roughly equal number of observations: 3 intervals
j Interval Nj npj ( N j  np j ) 2
np j
1 {0} 59 53.960 0.471
2 {1,2} 50 58.382 1.203
3 {3,4,…} 47 43.658 0.256
2 = 1.930
• Compare with 2 = 1.930 < 23-1, 0.90 = 4.605
H0 is not rejected at  = 0.10 level 15
 Kolmogorov-Smirnov (K-S) Test
• The Kolmogorov-Smirnov test (K–S test) is a form of minimum
distance estimation used as a non-parametric test of equality of
one-dimensional probability distributions used to compare a
sample with a reference probability distribution (one-sample K–S
test), or to compare two samples (two-sample K–S test).
• The Kolmogorov–Smirnov statistic quantifies a distance between
the empirical distribution of the sample and the cumulative
distribution function of the reference distribution, or between the
empirical distribution functions of two samples.
• The null distribution of this statistic is calculated under the null
hypothesis that the samples are drawn from the same distribution
(in the two-sample case) or that the sample is drawn from the
reference distribution (in the one-sample case).
• In each case, the distributions considered under the null
hypothesis are continuous distributions but are otherwise
unrestricted. 16
 Kolmogorov-Smirnov (K-S) Test
• Compare an empirical distribution function with
the distribution function of the hypothesized
distribution
• Advantages
– K-S tests do not require to group data in any way, so
no information is lost; this also eliminates the
troublesome problem of interval specification
– K-S tests are valid (exactly) for any sample size n (in
the all-parameters-known case), whereas Chi-Square
tests are valid only in an asymptotic sense
– K-S tests tend to be more powerful than Chi-Square
tests against many alternative distributions
17
 Kolmogorov-Smirnov (K-S) Test
• Disadvantages
– The range of applicability is more limited than
that for Chi-Square tests
– For discrete data, the required critical values
are not readily available and must be
computed using a complicated set of formulas
– The original form of the K-S test is valid only if
all the parameters of the hypothesized
distribution are known and the distribution is
continuous
• Allow for estimation of the parameters in
the cases of normal (lognormal),
exponential, and Weibull distributions

18
 Kolmogorov-Smirnov (K-S) Test
• An empirical distribution Fn(x) from data X1,…, Xn:

number of X i ' s  x
Fn ( x)   real numbers x
or n

where is the indicator function, equal to 1 if Xi ≤ x

and equal to 0 otherwise.
– Thus, Fn(x) is a step function
ˆ
– If ( x) is the fitted distribution function, a natural
F
assessment of goodness of fit is some measure between
ˆ
Fn(x) and F ( x) 19
Kolmogorov-Smirnov (K-S) Test

The K-S statistic Dn is the largest distance between the

empirical distribution function Fn(x) and the fitted
ˆ ( x)
F
distribution function for all values of x:

Dn  sup{ Fn ( x)  Fˆ ( x) }
x

20
 Kolmogorov-Smirnov (K-S) Test
• Dn can be calculated by:
 i ˆ  ˆ i 1
Dn  max{  F ( X ( i ) )}, Dn  max{F ( X ( i ) )  }
1 i  n n 1i  n n
and finally letting
Dn  max{Dn , Dn }
• Notes:
– Direct computation of Dn+ and Dn– requires sorting the data to
obtain X(i)’s
– For moderate values of n (up to several hundreds), sorting can
be done quickly by simple methods
– If n is large, sorting becomes expensive!
– A large value of Dn indicates a poor fit, so that it is to reject the
null hypothesis H0 if Dn exceeds some constant dn,1-, where  is
specified level of the test
21
 Kolmogorov-Smirnov (K-S) Test
• Case 1: If all parameters of F̂ are known
– None of the parameters is estimated in any
way from the data, the distribution of Dn does
not depend on F̂ , assuming that it is
continuous
– Instead of testing for Dn > dn,1-, we reject H0 if
0.11
( n  0.12  ) Dn  c1
n
where c1- are given in the all-parameters-known row
of Table 6.14
– This case is the original form of the K-S test
24
 Kolmogorov-Smirnov (K-S) Test
• Case 2: Suppose that the hypothesized distribution
is N(, 2) with both  and 2 unknown
– Estimate  and 2 by X(n) and S2(n), respectively
– Define the distribution function F̂ to be N(X(n), S2(n))
– Using this F̂ , Dn is computed in the same way
– We reject H0 if
0.85
( n  0.01  ) Dn  c'1
n
where c’1- are in the N(X(n), S2(n)) row of Table 6.14
– This case includes a K-S test for the lognormal distribution
if the Xi’s are the logarithms of the basic data points
26
 Kolmogorov-Smirnov (K-S) Test
• Case 3: Suppose the hypothesized distribution is
expo() with  unknown
  is estimated by its MLE X(n)
– Define F̂ to be the expo(X(n)) distribution function
– Using this F̂ , Dn is computed
– We reject H0 if
0.2 0.5
( Dn  )( n  0.26  ) Dn  c"1
n n
where c”1- are given in the Expo(X(n)) row of Table
6.14

27
 Kolmogorov-Smirnov (K-S) Test
• Case 4: Suppose the hypothesized distribution is
Weibull with both shape parameter  and scale
parameter  unknown
– Estimate parameters  and  by their respective MLEs
– F̂ is taken to be Weibull (MLEs of  and  )
– Dn is computed in the usual fashion
– We reject H0 if the adjusted K-S statistic n Dn is greater
than the modified critical value c*1- given in Table 6.15
– Note that critical values are available only for certain
sample sizes n, and that the critical values for n = 50
and  are, fortunately, very similar

28