Professional Documents
Culture Documents
Oracle® Crystal Ball: Reference and Examples Guide Release 11.1.2.4.850
Oracle® Crystal Ball: Reference and Examples Guide Release 11.1.2.4.850
Documentation Accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Documentation Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Chapter 1. Welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
About This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Technical Support and More . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
iii
Other Measures for a Data Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Skewness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Kurtosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Mean Standard Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Other Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Correlation Coefficient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Rank Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
Certainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Percentiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Simulation Sampling Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Monte Carlo Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Latin Hypercube Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Mean Confidence Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Standard Deviation Confidence Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Percentiles Confidence Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Random Number Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Goodness-of-Fit Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Chi-Square Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Kolmogorov-Smirnov and Anderson-Darling Statistics . . . . . . . . . . . . . . . . . . . . . . . 38
iv
Ppk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Cpm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Ppm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Z-LSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Z-USL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Zst . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Zst-total . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Zlt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Zlt-total . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
p(N/C)-below . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
p(N/C)-above . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
p(N/C)-total . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
PPM-below . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
PPM-above . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
PPM-total . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
LSL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
USL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Z-score Shift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
v
BetaPERT Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Binomial Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Discrete Uniform Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Exponential Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Gamma Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
Geometric Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Hypergeometric Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Logistic Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Lognormal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Maximum Extreme Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Minimum Extreme Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Negative Binomial Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Normal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Pareto Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Student’s t-Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Triangular Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
Uniform Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Weibull Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Yes-No Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Custom Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Additional Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Distribution Fitting Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Distribution Parameter Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Beta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
BetaPERT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Binomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Custom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Discrete Uniform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Exponential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Gamma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Geometric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Hypergeometric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Logistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Lognormal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Maximum Extreme Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Minimum Extreme Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Negative Binomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
vi
Pareto . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Poisson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Student’s t . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Triangular . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Uniform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Weibull . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Yes-No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
vii
F statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
t statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
p . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Historical Data Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Standard Deviation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Minimum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Maximum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Ljung-Box Statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Events Adjustment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Data Screening and Adjustment Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Outlier Detection Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Outlier and Missing Value Adjustment Methods . . . . . . . . . . . . . . . . . . . . . . . . 143
Techniques and Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Time-Series Prediction Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Standard Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Holdout Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Simple Lead Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
Weighted Lead Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
Classic Time-series Forecasting Method Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Classic Nonseasonal Forecasting Method Formulas . . . . . . . . . . . . . . . . . . . . . . 147
Classic Seasonal Forecasting Method Formulas . . . . . . . . . . . . . . . . . . . . . . . . . 149
Error Measure and Statistic Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Time-Series Forecast Error Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Prediction Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Autocorrelation Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
ARIMA Time-series Forecasting Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
ARIMA Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Estimation of ARIMA Model Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
ARIMA Constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Regression Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Calculating Standard Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Calculating Stepwise Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Regression Statistic Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
Statistics, Standard Regression with Constant . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Statistics, Standard Regression without Constant . . . . . . . . . . . . . . . . . . . . . . . 163
Partial F Statistic, Stepwise Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
viii
Chapter 7. OptQuest Examples and Reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
OptQuest Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Opening Example Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Recommended Run Preference Settings for Optimizations . . . . . . . . . . . . . . . . . . . 168
Product Mix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Product Mix Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Product Mix Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Product Mix OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Hotel Design and Pricing Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Hotel Design Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Hotel Design Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Hotel Design OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
Budget-constrained Project Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
Project Selection Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
Project Selection Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Project Selection OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Groundwater Cleanup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Groundwater Cleanup Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Groundwater Cleanup Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
Groundwater Cleanup OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Oil Field Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Oil Field Development Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
Oil Field Development Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Oil Field Development OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Portfolio Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Portfolio Revisited Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Portfolio Revisited Method 1: Efficient Frontier Optimization . . . . . . . . . . . . . . 189
Portfolio Revisited Method 2: Multiobjective Optimization . . . . . . . . . . . . . . . . 191
Method 3 — Arbitrage Pricing Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Tolerance Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Tolerance Analysis Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Tolerance Analysis Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
Tolerance Analysis OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
Inventory System Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Inventory System Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Inventory System Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
Inventory System OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Drill Bit Replacement Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
Drill Bit Replacement Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
ix
Drill Bit Replacement Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Drill Bit Replacement OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
Gasoline Supply Chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Gasoline Supply Chain Statement of Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Gasoline Supply Chain Spreadsheet Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Gasoline Supply Chain OptQuest Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
Optimization Tips and Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Model Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Optimization Models Without Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Optimization Models With Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Discrete, Continuous, or Mixed Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Linear or Nonlinear Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
Factors That Affect Optimization Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Simulation Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Number of Decision Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Base Case Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
Bounds and Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
Complexity of the Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
Simulation Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
Precision Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Sensitivity Analysis Using a Tornado Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
Maintaining Multiple Optimization Settings for a Model . . . . . . . . . . . . . . . . . . . . . 226
Other OptQuest Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
Automatic Resets of Optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
Constraint Formula Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
Minor Limit Violations With Continuous Forecasts . . . . . . . . . . . . . . . . . . . . . 227
Solutions Still Ranked Even With No Feasible Solution . . . . . . . . . . . . . . . . . . . 227
Referenced Assumption and Forecast Cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Decision Variables and Ranges With the Same Name . . . . . . . . . . . . . . . . . . . . 227
Linear Constraints Can Be Evaluated As Nonlinear . . . . . . . . . . . . . . . . . . . . . . 228
Evaluation Tolerances and Constraint Equality Statements . . . . . . . . . . . . . . . . 228
x
Probability Theory and Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
Random Variate Generation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
Specific Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Extreme Value Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Lognormal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Weibull Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Uncertainty Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
Spreadsheet Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Sequential Sampling with SIPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Time-series Forecasting References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
ARIMA Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
Regression Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
Optimization References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
OptQuest References and White Papers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Metaheuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Stochastic (Probabilistic) Optimization Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Multiobjective Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Optimization and Simulation in Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Financial Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Quality and Six Sigma Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Petrochemical Engineering Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Inventory System Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
xi
xii
Documentation Accessibility
For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at
http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.
13
14
Documentation Feedback
15
16
Welcome
1
In This Chapter
Introduction.................................................................................................17
About This Guide ...........................................................................................17
Technical Support and More ..............................................................................18
Introduction
Oracle Crystal Ball is a user-friendly, graphically oriented forecasting and risk analysis program
that takes the uncertainty out of decision-making.
Crystal Ball runs on several versions of Microsoft Windows and Microsoft Excel. For a complete
list of required hardware and software, see the current Oracle Crystal Ball Installation and
Licensing Guide.
17
Note: Because of round-off differences in various system configurations, you might obtain
calculated results that are slightly different from those in the examples.
18
Maximizing the Use of Crystal
2 Ball
In This Chapter
Introduction.................................................................................................19
Simulation Accuracy .......................................................................................19
Simulation Speed ..........................................................................................21
Sample Size ................................................................................................22
Correlated Assumptions ...................................................................................22
Crystal Ball and Multiple-processor Computers.........................................................23
Introduction
This chapter contains information that you can use to improve the overall performance of Crystal
Ball. These improvements occur in terms of the accuracy of the model or speed of the results.
Simulation Accuracy
The accuracy of the simulation is primarily governed by two factors:
l The number of trials, or length, of the simulation
l The sampling method
Generally speaking, the more trials you run in a simulation, the greater the accuracy of the
statistics and percentiles information. This greater accuracy comes at the expense of lengthier
simulation times and higher memory usages (see later sections on simulation speed and memory
usage). Also, for a given number of trials, the accuracy of the statistics and percentiles greatly
depends on the shape and nature of the forecast distribution.
If you are not sure how many trials to run for a specific level of accuracy, you can use the precision
control feature of to automatically determine the appropriate number of trials to run. For a
detailed picture of a simulation’s statistical accuracy, you can run the Bootstrap tool to generate
a forecast chart for each statistic or percentile of interest.
The sampling method is the other primary factor governing simulation accuracy. During a
simulation, Monte Carlo sampling generates natural, “what-if” type scenarios while Latin
Hypercube’s sampling is constrained, but more accurate.
19
Precision Control
The precision control feature sets how precise you want the forecast statistics to be. Crystal Ball
runs the simulation until the selected forecast statistics reach the required precision as
determined by calculating confidence intervals.
For more information about confidence intervals, see “Confidence Intervals” on page 35.
Generally speaking, as more trials are calculated, the confidence intervals narrow and statistics
become more precise. The precision control feature in uses this characteristic of confidence
intervals to determine when a specified precision of a statistic has been reached. It compares the
specified precision to the confidence interval. When the calculated confidence interval drops to
less than the specified precision, the simulation stops.
For each forecast, you can specify precision in either absolute terms in units of the forecast, or
in relative terms as percentages. These settings are made on the Precision tab of the expanded
Define Forecast dialog or the Forecast Preferences dialog. Each method, absolute or relative, has
its own benefits and drawbacks.
Specifying precision in absolute terms offers greater control of the simulation when the shape
and scale of the forecast distribution is roughly known. For example, for a Gross Profit forecast
(from the Vision Research model) that ranges from $25.5 to $64.0 million dollars, you can
require the precision of the mean to be within plus or minus $100,000 or some other convenient
measure of accuracy. However, with the same forecast range, an absolute accuracy of $1000 may
require an unreasonably large number of trials to reach. So, the drawback of using absolute
precision is that it may require experimentation to determine reasonable accuracy values.
Specifying precision in relative terms offers greater control of the simulation when the shape
and scale of the forecast distribution are largely unknown and you are interested in the accuracy
only as it relates to the overall distribution itself. In the previous Gross Profit example, you may
not know or care if the distribution ranges from $25,500 to $64,000 dollars or from $25.5 to
$64.0 million dollars. You may require only that the simulation's estimate of the mean fall within
plus or minus 5% of itself.
You may encounter the drawback of using relative precision when the forecast statistic is close
to zero. For example, suppose a forecast’s distribution straddles the break-even point of zero. A
relative precision of 5% of the mean, or roughly $0.5 million, results in a very small confidence
interval (relative to the full range width of $49.1 million) that may take an unexpectedly large
number of trials to satisfy.
Finally, combines the individual forecast precision options with the confidence level value found
in the Trials tab of the Run Preferences dialog to calculate confidence intervals. Generally, it is
a good idea to leave this value at 95% or 90% so that you can have a high degree of confidence
that the precision requirements have been met. However, if you have a large number of forecasts
defined with precision control set, you can adjust the confidence level up or down to globally
change the accuracy of all forecasts together.
Sampling Method
Selecting Monte Carlo or Latin Hypercube sampling affects how random numbers are generated
for individual assumptions.
20
In almost all cases, Latin Hypercube produces more accurate forecast statistics (especially the
mean) given the same number of trials as Monte Carlo, because it is a more consistent sampling
method. If you are primarily interested in the accuracy of the statistics, you should select Latin
Hypercube as the sampling method in the Sampling tab of the Run Preferences dialog.
If you are primarily interested in evaluating how the spreadsheet model behaves under various
what-if scenarios, you should select Monte Carlo as the sampling method. Monte Carlo produces
assumptions with the most randomness and simulates real-life situations the best.
Simulation Speed
Monte Carlo simulations can be very time-consuming. You can change a number of factors that
affect the speed of simulations. The factors are listed below in order of importance:
1. Change the Speed preferences.
In the Speed tab of the Run Preferences dialog are options that can substantially increase
the speed of the simulation.
You can, in increasing order of helpfulness:
l Update worksheets only every few seconds or so.
l Minimize Microsoft Excel workbooks during simulations.
l Minimize Microsoft Excel workbooks and suppress chart windows (the fastest
combination of settings).
2. Use the precision control feature.
The precision control feature can be used to either increase the precision of the simulations
or increase the speed of the simulations. If you set the confidence level to a high number,
the simulations will be more precise, but may run significantly longer. However, if you do
not need as precise a result, you can set the confidence level to a lower number and the
simulation speed will increase.
Using this feature to speed up the model will require you to experiment with different
confidence levels.
3. Reduce the size of the model by reducing the number of assumptions, forecasts, and
correlations.
Large models require more time per trial. For example, a model that takes 3 or 4 seconds
per recalculation cycle will take up to an hour to simulate 1,000 trials.
Greater numbers of assumptions and forecasts slow the simulation, especially if the
assumptions and forecasts are scattered across many spreadsheets in the model. Start by
examining the structure and nature of the model to locate possible efficiencies. You can also
use the sensitivity feature or the Tornado Chart tool to determine which assumptions
contribute the least amount of uncertainty to the most important forecasts. Freeze or
eliminate the least important assumptions from the simulation.
Correlated assumptions can also consume a significant amount of processing time; the time
grows geometrically as the number of correlated assumptions increases.
21
4. Reduce the use of other applications.
Quitting other applications and closing or minimizing windows can be helpful in reducing
overhead and increasing simulation speed.
5. Increase the system’s RAM.
The amount of RAM in the computer has a large effect on the speed of simulations. Modern
operating systems give applications such as spreadsheets the appearance of additional RAM
through the use of virtual memory.
Virtual memory enables you to run a greater number of applications than would otherwise
be possible, but slows down overall processing speed because the system is frequently
accessing the hard drive. If you hear the hard disk being used during a simulation, there may
not be enough RAM to hold all parts of the simulation. Buying more RAM or turning off
virtual memory (if possible) are solutions to this problem.
Sample Size
The sample size setting is located on the Sampling tab of the Run Preferences dialog. Sample
size, which is initially set to 500, affects Latin hypercube sampling. It divides each assumption’s
distribution into a number of intervals of equal probability. The sample size governs the number
of intervals for each distribution. Crystal BallCrystal Ball generates an assumption value for each
interval according to the interval’s probability distribution.
While any sample size greater than 100 should produce sufficiently acceptable results, you can
set this number higher to maximize accuracy. There is no absolute limit to sample size, although
samples greater than 100,000 work best with at least 1 GB RAM and may take a long time to run.
The increased accuracy resulting from the use of larger samples, however, requires additional
memory. If memory becomes an issue, reduce the sample size and consider adding more RAM.
Correlated Assumptions
There are practical limits to the size of correlation matrixes in a workbook. For matrixes built
with pairwise correlations (with or without coefficient cell references), the practical limit is about
1,000 correlations per matrix per worksheet (for example, a 50x50 fully specified matrix or a
matrix of 1,000 serially correlated assumptions). For linked matrixes, the practical limit is about
500 assumptions per matrix per worksheet (equivalent to approximately 125,000 pairwise
correlations).
To minimize the size of matrixes:
l Consider leaving assumptions with coefficients close to 0.0 as uncorrelated.
l Consider replacing assumptions with coefficients close to 1.0 or -1.0 with a formula in the
spreadsheet.
When the simulation starts, Crystal Ball checks each correlation matrix for consistency. (Usually,
the larger the correlation matrix, the greater your chance of inadvertently creating
inconsistencies.) If one or more matrixes are found to be inconsistent, Crystal Ball determines
22
whether adjustments to the correlation coefficients are possible. This process may take a long
time, depending on the number and size of the correlation matrixes.
If adjustments to the correlation coefficients are possible, the simulation stops with a dialog
prompting you to select a course of action. Choose from the following:
l Click OK to continue the simulation with adjusted coefficients.
l Click Edit Correlations to view or edit the adjusted correlations prior to running the
simulation. If you make any changes, Crystal Ball rechecks the matrix for consistency.
l To terminate the simulation, click Cancel.
For the first two options, the original correlation coefficients in the workbook are replaced
permanently with the adjusted ones. Note that pairwise correlations with coefficient cell
references are not adjusted.
After a simulation, you can use the Create Report feature to list original and adjusted correlation
coefficients for each assumption.
In general, users should see a linear increase in simulation speed with an increase in processor
speed. Using a computer with multiple processors and multiple threads can further enhance
performance as described in the following sections.
23
reduced, the time to run an entire simulation is also reduced. For two processors, the increase
in simulation speed can be roughly anywhere from 10% to 50%, depending on the model.
To use multithreading efficiently, you should be working with a spreadsheet model that:
l Is large (that is, it takes more than 0.5 sec for each recalculation).
l Can easily be divided into separate tasks by Microsoft Excel (for example, it may have
separate chains or groups of formulas that do not depend on each other).
l Is running at Normal Speed.
24
Statistical Definitions
3
In This Chapter
Introduction.................................................................................................25
Statistics ....................................................................................................25
Simulation Sampling Methods............................................................................34
Confidence Intervals .......................................................................................35
Random Number Generation .............................................................................37
Goodness-of-Fit Measures ................................................................................37
Introduction
This chapter provides formulas for the following types of statistics:
l “Measures of Central Tendency” on page 25
l “Measures of Variability” on page 27
l “Other Measures for a Data Set” on page 28
l “Other Statistics” on page 31
Statistics
This section discusses basic statistics used in Crystal Ball.
25
Mean
The mean of a set of values is found by adding the values and dividing their sum by the number
of values. The term “average” usually refers to the mean. For example, 5.2 is the mean, or average,
of 1, 3, 6, 7, and 9.
Formula:
Median
The median is the middle value in a set of sorted values. For example, 6 is the median of 1, 3, 6,
7, and 9 (recall that the mean is 5.2).
If an odd number of values exists, you find the median by ordering the values from smallest to
largest and then selecting the middle value.
If an even number of values exists, then the median is the mean of the two middle values.
Mode
The mode is the value that occurs most frequently in a set of values. The greatest degree of
clustering occurs at the mode.
The modal wage, for example, is the one received by the greatest number of workers. The modal
color for a new product is the one preferred by the greatest number of consumers.
In a perfectly symmetrical distribution, such as the normal distribution (the distribution on
the left, below), the mean, median, and mode converge at one point.
In an asymmetrical, or skewed, distribution, such as the lognormal distribution, the mean,
median, and mode tend to spread out, as shown in the second distribution (on the right) in the
following example (Figure 1).
26
Note: When running simulations, forecast data likely will be continuous and no value will occur
more than once. In such a case, sets the mode to ‘---’ in the Statistics view to indicate that
the mode is undefined.
Measures of Variability
The measures of variability for a data set are variance, standard deviation, and range (or range
width).
Variance
Variance is a measure of the dispersion, or spread, of a set of values about the mean. When values
are close to the mean, the variance is small. When values are widely scattered about the mean,
the variance is larger.
Formula:
For example, suppose your values are 1, 3, 6, 7, and 9. The mean is 5.2. The variance, denoted
by s2 , is calculated as follows:
Note: The calculation uses n - 1 instead of n to correct for the fact that the mean was calculated
from the data sample, thus removing one degree of freedom. This correction makes the
sample variances slightly larger than the variance of the entire population.
Standard Deviation
The standard deviation is the square root of the variance for a distribution. Like the variance, it
is a measure of dispersion about the mean and is useful for describing the “average” deviation.
See the description for the variance in the next section.
27
For example, you can calculate the standard deviation of the values 1, 3, 6, 7, and 9 by finding
the square root of the variance that is calculated in the variance example that follows.
Formula:
Coefficient of Variation
The coefficient of variation provides you with a measurement of how much your forecast values
vary relative to the mean value. Because this statistic is independent of the forecast units, you
can use it to compare the variability of two or more forecasts, even when the forecast scales differ.
For example, if you are comparing the forecast for a penny stock with the forecast for a stock on
the New York Stock Exchange, you would expect the average variation (standard deviation) of
the penny stock price to appear smaller than the variation of the NYSE stock. However, if you
compare the coefficient of variation statistic for the two forecasts, you will notice that the penny
stock shows significantly more variation on an absolute scale.
The coefficient of variation typically ranges from a value greater than 0 to 1. It might exceed 1
in a few cases in which the standard deviation of the forecast is unusually high. This statistic is
computed by dividing the standard deviation by the mean.
The coefficient of variation is calculated by dividing the standard deviation by the mean, as
follows:
coefficient of variation = s/x
To present this number as a percentage, multiply the result of the coefficient of variation
calculation by 100.
28
Skewness
A distribution of values (a frequency distribution) is said to be “skewed” if it is not symmetrical.
For example, suppose the curves in the example below represent the distribution of wages within
a large company (Figure 2).
Curve A illustrates positive skewness (skewed “to the right”), where most of the wages are near
the minimum rate, although some are much higher. Curve B illustrates negative skewness
(skewed “to the left”), where most of the wages are near the maximum, although some are much
lower.
If you describe the curves statistically, curve A is positively skewed and might have a skewness
coefficient of 0.5, and curve B is negatively skewed and might have a -0.5 skewness coefficient.
A skewness value greater than 1 or less than -1 indicates a highly skewed distribution. A value
between 0.5 and 1 or -0.5 and -1 is moderately skewed. A value between -0.5 and 0.5 indicates
that the distribution is fairly symmetrical.
Method:
Skewness is computed by finding the third moment about the mean and dividing by the cube
of the standard deviation.
Formula:
Kurtosis
Kurtosis refers to the peakedness of a distribution. For example, a distribution of values might
be perfectly symmetrical but look either very “peaked” or very “flat,” as illustrated below
(Figure 3).
29
Figure 3 Peaked and Flat Kurtosis
Suppose the curves in represent the distribution of wages in a large company. Curve A is fairly
peaked, because most of the employees receive about the same wage, with few receiving very
high or low wages. Curve B is flat-topped, indicating that the wages cover a wider spread.
Describing the curves statistically, curve A is fairly peaked, with a kurtosis of about 4. Curve B,
which is fairly flat, might have a kurtosis of 2.
A normal distribution usually is used as the standard of reference and has a kurtosis of 3.
Distributions with kurtosis values of less than 3 are described as platykurtic (meaning flat), and
distributions with kurtosis values of greater than 3 are leptokurtic (meaning peaked).
Method:
Kurtosis, or peakedness, is calculated by finding the fourth moment about the mean and dividing
by the quadruple of the standard deviation.
Formula:
Note: The mean standard error statistic provides information only on the accuracy of the mean
and can be used as a general guide to the accuracy of the simulation. The standard error
for other statistics, such as mode and median, probably will differ from the mean standard
error.
Formula:
30
where s = standard deviation and n = number of trials.
The error estimate might be inverted to show that the number of trials needed to yield a desired
error:
Other Statistics
These statistics describe relationships between data sets (correlation coefficient, rank
correlation) or other data measurements (certainty, percentile, confidence intervals).
Correlation Coefficient
Note: Crystal Ball uses rank correlation to determine the correlation coefficient of variables. For
more information on rank correlation, see “Rank Correlation” on page 32.
When the values of two variables depend upon one another in whole or in part, the variables
are considered correlated. For example, an “energy cost” variable likely will show a positive
correlation with an “inflation” variable. When the “inflation” variable is high, the “energy cost”
variable is also high; when the “inflation” variable is low, the “energy cost” variable is low.
In contrast, “product price” and “unit sale” variables might show a negative correlation. For
example, when prices are low, high sales are expected; when prices are high, low sales are
expected.
By correlating pairs of variables that have such a positive or negative relationship, you can
increase the accuracy of your simulation forecast results.
The correlation coefficient is a number that describes the relationship between two dependent
variables. Coefficient values range between -1 and 0 for a negative correlation and 0 and +1 for
a positive correlation. The closer the absolute value of the correlation coefficient is to either +1
or -1, the more strongly the variables are related.
When an increase in one variable is associated with an increase in another, the correlation is
called positive (or direct) and is indicated by a coefficient between 0 and 1. When an increase
in one variable is associated with a decrease in another variable, the correlation is called negative
(or inverse) and is indicated by a coefficient between 0 and -1. A value of 0 indicates that the
variables are unrelated to one another. The example below shows three correlation coefficients
(Figure 4).
31
Figure 4 Types of Correlation
For example, assume that total hotel food sales might be correlated with hotel room rates. Total
food sales likely will be higher, for example, at hotels with higher room rates. If food sales and
room rates correspond closely for various hotels, the correlation coefficient is close to 1.
However, the correlation might not be perfect (correlation coefficient is less than 1). Some people
might eat meals outside of the hotel, and others might skip some meals.
When you select a correlation coefficient to describe the relationship between two variables in
your simulation, you must consider how closely they are related. You should never need to use
an actual correlation coefficient of 1 or -1. Generally, you should represent these types of
relationships as formulas on your spreadsheet.
Formula:
Note: Crystal Ball uses rank correlation, also referred to as Spearman's rank correlation
coefficient, to correlate assumption values. This means that assumption values are
replaced by their rankings from lowest to highest value by the integers 1 to n, before
computing the correlation coefficient. This method allows distribution types to be ignored
when correlating assumptions.
Rank Correlation
A correlation coefficient measures the strength of the linear relationship between two variables.
However, if the two variables do not have the same probability distributions, they are not likely
related linearly. Under such circumstances, the correlation coefficient calculated on their raw
values has little meaning.
If you calculate the correlation coefficient using rank values instead of actual values, the
correlation coefficient is meaningful even for variables with different distributions.
You determine rank values by arranging the actual values in ascending order and replacing the
values with their rankings. For example, the lowest actual value will have a rank of 1; the next-
lowest actual value will have a rank of 2; and so on.
32
Crystal Ball uses rank correlation, also referred to as Spearman's rank correlation coefficient, to
correlate assumptions. The slight loss of information that occurs using rank correlation is offset
by two advantages:
l First, the correlated assumptions need not have similar distribution types. In effect, the
correlation function in is distribution-independent. The rank correlation method works even
when a distribution has been truncated at one or both ends of its range.
l Second, the values generated for each assumption are not changed; they are merely
rearranged to produce the desired correlation. In this way, the original distributions of the
assumptions are preserved.
Certainty
The forecast chart shows not only the range of results for each forecast, but also the probability,
or certainty, of achieving results within a range. Certainty is the percent chance that a forecast
value will fall within a specified range.
By default, the certainty range is from negative infinity to positive infinity. The certainty for this
range is always 100 percent. However, you might want to estimate the chance of a forecast result
falling in a specific range, say from zero to infinity (which you might want to calculate to ensure
that you make a profit).
For example, consider the forecast chart in Figure 5. If your objective is to make a minimum
return of $2,000,000, you might choose a range of $2,000,000 to +Infinity. In this case, the
certainty is almost 75 percent.
Percentiles
A percentile is a number on a scale of 0–100 that indicates the percent of a distribution that is
equal to or less than a value (by default). Standardized tests usually report results in percentiles.
If you are in the 95th percentile, then 95 percent of test takers had either the same score or a
lower score. This number does not mean that you answered 95 percent of the questions correctly.
33
You might have answered only 20 percent correctly, but your score was better than, or as good
as, 95 percent of the other test takers' scores.
Crystal Ball calculates percentiles of forecast values using an interpolation algorithm. This
algorithm is used for both discrete and continuous data, resulting in the possibility of having
real numbers as percentiles for even discrete data sets. If an exact forecast value corresponds to
a calculated percentile, accepts that as the percentile. Otherwise, proportionally interpolates
between the two nearest values to calculate the percentile.
Note: When calculating medians, does not use the proportional interpolation algorithm; it uses
the classical definition of median, described in “Median” on page 26.
Percentiles for a normal distribution look like the following figure (Figure 6).
34
Such random behavior is similar to how Monte Carlo simulation selects variable values at
random to simulate a model. When you roll a die, you know that a 1, 2, 3, 4, 5, or 6 will come
up, but you do not know which for any particular trial. It is the same with the variables that have
a known range of values and an uncertain value for any particular time or event (for example,
interest rates, staffing needs, stock prices, inventory, phone calls per minute).
Using Monte Carlo sampling to approximate the true shape of the distribution requires more
trials than Latin Hypercube.
Use Monte Carlo sampling to simulate “real world” what-if scenarios for your spreadsheet
model.
While a simulation runs, selects a random assumption value for each segment according to the
segment’s probability distribution. This collection of values forms the Latin Hypercube sample.
After has sampled each segment exactly once, the process repeats until the simulation stops.
The Sample Size option (displayed when you select Run Preferences, then Sample), controls the
number of segments in the sample.
Latin Hypercube sampling is generally more precise when calculating simulation statistics than
is conventional Monte Carlo sampling, because the entire range of the distribution is sampled
more evenly and consistently. Latin Hypercube sampling requires fewer trials to achieve the
same level of statistical accuracy as Monte Carlo sampling. The added expense of this method
is the extra memory required to track which segments have been sampled while the simulation
runs. (Compared to most simulation results, this extra overhead is minor.)
Use Latin Hypercube sampling when you are concerned primarily with the accuracy of the
simulation statistics.
Confidence Intervals
Because Monte Carlo simulation uses random sampling to estimate model results, statistics
computed on these results, such as mean, standard deviation and percentiles, always contain
some kind of error. A confidence interval (CI) is a bound calculated around a statistic that
attempts to measure this error with a given level of probability. For example, a 95 percent
confidence interval around the mean statistic is defined as a 95 percent chance that the mean
will be contained within the specified interval. Conversely, a 5 percent chance exists that the
35
mean will lie outside the interval. Shown graphically, a confidence interval around the mean
looks like Figure 8.
For most statistics, the confidence interval is symmetrical around the statistic, so that x = (CImax
- Mean) = (Mean - CImin). This accuracy lets you make statements of confidence such as “the
mean will lie within the estimated mean plus or minus x with 95 percent probability.”
Confidence intervals are important for determining the accuracy of statistics, hence, the accuracy
of the simulation. Generally speaking, as more trials are calculated, the confidence interval
narrows and the statistics become more accurate. The precision control feature of lets you stop
the simulation when the specified precision of the chosen statistics is reached. periodically checks
whether the confidence interval is less than the specified precision.
Notice that the Bootstrap tool in enables you to calculate the confidence intervals for any set of
statistics using empirically-based methods.
The following sections describe how calculates the confidence interval for each statistic.
where s is the standard deviation of the forecast, n is the number of trials, and z is the z value
based on the specified confidence level (to set the confidence level, from Run Preferences, select
Trials).
where s is the standard deviation of the forecast, k is the kurtosis, n is the number of trials, and
z is the z value based on the specified confidence level (from Run Preferences, select Trials).
36
Random Number Generation
Crystal Ball uses the random number generator described in the following iteration formula as
the basis for all nonuniform generators. For no starting seed value, takes the value of the number
of milliseconds elapsed since Windows started.
Method: Multiplicative Congruential Generator
This routine uses the iteration formula:
Comment:
The generator has a period of length of 231 - 2, or 2,147,483,646. This means that the cycle of
random numbers repeats after several billion trials. This formula is discussed in detail in the
Simulation Modeling & Analysis and Art of Computer Programming, Vol. II, references in
Appendix A, “Bibliography.”.
Goodness-of-Fit Measures
Subtopics
l Chi-Square Test
l Kolmogorov-Smirnov and Anderson-Darling Statistics
If you have historical data available, Crystal Ball’s distribution fitting feature can substantially
simplify the process of selecting a probability distribution when creating assumptions. Not only
is the process simplified, but the resulting distribution more accurately reflects the nature of the
data than if the shape and parameters of the distribution were estimated. Crystal Ball also
performs goodness-of-fit tests against forecast charts to identify which theoretical distribution
most closely matches the generated distribution.
Distribution fitting automatically matches historical or geenrated data against probability
distributions. A mathematical fit determines the set of parameters for each distribution that best
describe the characteristics of the data. Then, the closeness of each fit is judged using one of the
listed goodness-of-fit tests.
Chi-Square Test
The chi-square test can be thought of as a formal comparison of a histogram of the data with
the density or mass function of the fitted distribution. Essentially the range of the fitted
distribution is divided into adjacent intervals and the chi-square test is used to determine
whether there is a significant difference between the expected frequencies and the observed
frequencies in one or more intervals.
37
Kolmogorov-Smirnov and Anderson-Darling Statistics
The Kolmogorov-Smirnov and Anderson-Darling statistics are also known as Empirical
Distribution Function (EDF) statistics. Given a random sample, let X1 < X2 < ... < Xn be the
ordered statistics. The empirical cumulative distribution function (ECDF), Fn(x), is given by:
If F(x) is the cumulative distribution function of the fitted distribution, then any statistic
measuring the difference between Fn(x) and F(x) is called an EDF statistic. The Kolmogorov-
Smirnov statistic is a first -order measure of this difference; the Anderson-Darling statistic is a
quadratic form of the difference.
References:
D'agostino, R. B. and Stephens, M. A. Goodness-of-fit techniques. Marcel Dekker, Inc., NY.
1986.
Law, A. M. and Kelton, W. D. Simulation Modeling and Analysis, 3rd Ed. McGraw-Hill, NY.
2003.
38
Process Capability Tutorials and
4 Reference
In This Chapter
Process Capability Tutorials ...............................................................................39
Capability Metrics List .....................................................................................67
Process Capability Metrics Formulas.....................................................................69
“Tutorial 2 — Packaging Pump Design” on page 53, shows how to use Crystal Ball in design
quality programs such as Design for Six Sigma (DFSS).
Overall Approach
Suppose you are assigned the task of improving loan processing quality at a bank. You want to
minimize average processing time, keeping below the bank’s target. At the same time, you want
to evaluate and improve processing time consistency using the following techniques:
l Define — You plan to talk with stakeholders in various departments to learn how they are
currently working, how they view of the problem, and what outcome they hope to see.
39
l Measure — You will determine current processing times and the amount of variation within
them.
l Analyze — You plan to use Crystal Ball to help you discover process capability, or how
closely the processing times map to process requirements.
l Improve — You will use information gathered in the previous phases to design and
implement improvements in processing time and consistency.
l Control — You need to ensure permanent improvements by setting up a system of feedback
loops to continuously measure process results and change the output if measures exceed
desired control targets and control limits.
In the Define and Measure phases, you obtain and use information to build a model. Then, you
use Crystal Ball to simulate real-world variations that occur in processing loans. Finally, you can
use sensitivity analysis and further optimization to arrive at the best balance between time and
consistency.
The following sections give details of each phase:
l “Define Phase” on page 40
l “Measure Phase” on page 40
l “Analyze Phase” on page 41
l “Improve Phase” on page 41
l “Control Phase” on page 41
Define Phase
The bank’s loan processing procedures involve six steps. You interview staff involved with each
step and learn that marketing and management groups are hoping for completion of all six steps
within 96 working hours. Based on these interviews, you establish both the target and upper
specification limit (USL) at 96 hours. There is no lower specification limit (LSL), although
consistency (the sigma level) seems to be an issue. You decide to obtain more concrete data.
Measure Phase
You document time and time variation data for each processing step, as follows:
l Customer Inquiry involves the creation of an initial rate quote following the first customer
contact by phone, in-person visit, or the Internet. Historical data indicates this cycle time is
lognormally distributed with a mean of one hour and a standard deviation of 0.25 hours.
l Loan Application involves the completion of forms by customers, with or without staff
assistance. This step takes from one to five eight-hour days, but usually takes three days.
l Document Verification and processing occurs in the bank, where a loan specialist reviews
credit data, presents loan alternatives to customers, and independently verifies information.
This step usually takes between two and four eight-hour days. However, about two out of
ten times, the loan is suspended and takes between four and six days for additional reviews
and documentation.
40
l Underwriting, or loan approval, usually takes from one hour to 8 hours. All lengths of time
have the same likelihood of occurrence.
l Loan Closing includes the preparation of the final documentation, locking in an interest
rate, and arranging where to deposit funds. Historical data show that this step has a mean
of two days, but you do not yet know how the data are distributed.
l Loan Disbursement, moving the funds to the customer’s bank, usually takes 16 hours, with
some variation. Two-thirds of disbursement times fall within 12 and 20 hours.
After gathering this information, you decide to load it into Microsoft Excel and use Crystal Ball
to analyze it.
Analyze Phase
During the Analyze phase, you create a Microsoft Excel workbook that includes historical data
on loan closing times. Then, you can capture the time and variation information for each step
as a assumption. Finally, you can define overall processing time as a forecast. When the
simulation runs, you can view capability metrics to gain insight into the degree of process
variation, or inconsistency. You can also generate a sensitivity chart to learn more about the
main sources of inconsistency.
“Using the Loan Processing Model” on page 42, shows how to use a model in the Analyze and
Improve phases to accomplish the quality goals.
Improve Phase
You can experiment with the effects of changing the main sources of inconsistency. If you have
OptQuest, you can also define decision variables and optimize for the shortest processing time
and least time variation.
Control Phase
You set up a system of measuring cycle times for each step and analyzing them on a regular basis.
You run a simulation on real data once a month with a program of corrective actions to take
whenever results exceed desired limits by a specified amount.
41
Using the Loan Processing Model
Subtopics
l Starting Crystal Ball
l Activating the Process Capability Features
l Setting the Sampling Seed Value
l Opening the Example Model
l Reviewing the Parts of the Model
l Reviewing the Assumptions
l Examining the Custom Distribution
l Fitting a Distribution for Step 5
l Reviewing the Forecast
l Running Simulations
l Analyzing Simulations
l Investigating Improvement Possibilities
This section shows how to construct a model that effectively simulates each step of the loan
process with dual goals of minimizing processing time and maximizing time consistency. Then,
you run the model and analyze the output using Crystal Ball’s process capability features.
The sections listed above describe how to work with the loan processing model and analyze data
from it.
ä To confirm that the process capability features have been activated and activate them, if
necessary:
42
Setting the Sampling Seed Value
To more closely reproduce the results in this tutorial, you may want to set the sampling seed
value on the Sampling tab of the Run Preferences dialog.
43
l The assumption parameters beside each assumption show the distribution and the
parameters entered for each assumption. They are entered in each assumption as cell
references relative to cells in column C.
l The blue box at the bottom of the Simulated Cycle Time column is a forecast that represents
the total cycle time.
ä To explore this:
1 Select cell C18, the assumption for Step 3.
2 Select Define Assumption. , to display the Define Assumption dialog for that assumption.
Figure 10 shows the expanded Define Assumption dialog for Step 3, Document Verification.
The dialog has been expanded by clicking the More button, .
This dialog reflects the information from that 80% of the time, cycle time for this step ranges
from two to four days (16 to 32 hours) but 20% of the time, cycle time ranges from four to
six days (32 to 48 hours).
44
Figure 10 Custom Assumption for Step 3, Document Verification
2 Select Define Assumption, , to display the Distribution Gallery dialog for that cell.
3 Click the Fit button.
The Fit Distribution dialog opens.
4 Type the range =Data!B4:B103 or click the cell selector, , for the Range edit box. Then, click the
Data tab of the workbook and select cells B4 through B103.
When you accept the selection, the Fit Distribution dialog opens with the range entered.
5 Leave the other default settings as shown and click OK.
The Comparison Chart dialog opens (Figure 11). The Poisson distribution appears to be the
best fit.
45
Figure 11 Comparison Chart Dialog, Cell C22
Now, all assumptions for the Loan Process model are complete.
46
Reviewing the Forecast
The forecast in cell C26 represents total cycle time. To view its formula in the Microsoft Excel
formula bar, select cell C26. The formula is =SUM(C14:C24) — the sum of the assumptions for
Steps 1 through 6.
As established in the Define phase of this project, 96 hours is the loan processing target as
well as the USL.
2 Enter 96 in the Target edit box and click OK.
Running Simulations
At this point, the model’s assumptions and forecast are complete.
To run the simulation and generate a forecast chart, click the Start button .
Analyzing Simulations
When you run the simulation for this model, the Cycle Time forecast chart opens (Figure 14,
following). Because the forecast is defined with an upper specification limit and a target, the
chart is displayed in Split View with the forecast chart beside the process capability metrics. The
message with the metrics indicates that the distribution is non-normal (the normality test failed),
so capability metrics are calculated from the forecast values.
47
Figure 14 The Cycle Time Forecast with Process Capability Metrics
Notice that marker lines are displayed for the USL and target. Because the USL is defined, the
maximum certainty grabber is automatically set to that value. Because no LSL is defined, the
minimum certainty grabber is automatically set to –Infinity. The Certainty text box contains
73.23; about 73% of forecasted values fall below the USL.
Looking at the process capability metrics, the forecast distribution is not normal, so the Z scores
are not generally appropriate to use. The p(N/C)-above (defects above the USL) is about .27,
which confirms the certainty level shown in the forecast chart; more than one-quarter of the
forecasted values fall above the USL. The Cpk-upper and Cpk are about 0.25, where you had
hoped to see at least 1.00 (the 3-sigma level). The loan specialists you interviewed were correct
— cycle time variation is large.
You generate a sensitivity chart (Figure 15) to determine which of the loan processing steps has
the most influence on the Cycle Time forecast variance. To do this, select the forecast chart and
select Forecast, then Open Sensitivity Chart.
Figure 15 shows that the loan processing step with the greatest effect on cycle time variance is
Document Verification (Step 3). Obviously, this is an ideal target for improvement.
48
Figure 15 Sensitivity Chart for the Cycle Time Forecast
ä To do this:
49
Figure 16 Document Verification Assumption, Adjusted
This assumption was previously displayed in Figure 10. Because the probabilities in the table
are linked to workbook cells, when you adjusted the workbook, you also adjusted the
assumption probabilities.
5 Close the Define Assumption dialog and run the simulation.
6 Review the forecast chart in Figure 17.
50
Crystal Ball sets the upper certainty level at the USL/Target value (96). If the lower certainty
grabber is not located at –Infinity, click the lower grabber and drag it toward the Probability axis
as shown in Figure 17. Now, the certainty of cycle time falling at or below the USL is about 80%.
You decide to make a drastic change and edit the assumption parameters for Document
Verification as follows (change 32 to 24 in the first row; change 90% to 100% and 10% to 0%):
Now, there is 100% probability that all document verification times will fall between two and
three days.
Figure 18 shows that when you run with this new assumption definition, the certainty of falling
at or below the USL rises to about 94.5% and PPM-total drops to 53,309. Also, the distribution
is now normal; it was not normal for the last simulation. Z-USL and Zst-total are 1.61, the sigma
level for this simulation.
You review the sensitivity chart for this simulation (Figure 19) and find that the impact of Loan
Application is now much greater than Document Verification.
51
Figure 19 Sensitivity Chart after Adjusting Document Verification
You believe that Document Verification has been pared down about as much as it can be, so you
decide to experiment further with making the Loan Application cycle time shorter and more
consistent.
ä To do this:
1 Reset the simulation and change the Step 2 parameters to 8, 16, and 24.
2 Run the simulation.
Results are much better. Figure 20 on page 53 shows that the mean cycle time has dropped to
73 hours. The certainty of falling below the USL/Target value is 100% and the sigma level has
risen to 3.14. Cpk-upper and Cpk have risen over 1 which, given the shape and location of the
distribution, confirms that the sigma level is greater than 3.
52
Figure 20 Forecast Chart after Loan Application Adjustment
When you review the sensitivity chart (Figure 21), Loan Disbursement and Loan Closing are
now the main contributors to cycle time variation. You learn that these processes may be difficult
to improve further, so you decide to implement improvements to Document Verification and
Loan Application cycle times and then move into the Control phase, discussed in “Control
Phase” on page 41.
53
DFSS has five phases: Define, Measure, Analyze, Design, Validate (or Verify). The ultimate
mission of DFSS is to meet general Six Sigma goals of satisfying customer needs while minimizing
variation and to accomplish this as early in the development cycle as possible.
The following sections show how DFSS techniques can be used with Crystal Ball to simulate
various design options for a fluid pump in a food production setting:
l “Overall Approach” on page 54
l “Using the DFSS Liquid Pump model” on page 56
l “Using OptQuest to Optimize Quality and Cost” on page 63
Overall Approach
Suppose you are a pump manufacturer. You want to design a pump for a liquid packaging system
that draws processed fluids from a vat into jars at a consistent rate. You want to meet the specified
flow rate target and limits while maximizing performance and minimizing costs.
Using the phases of DFSS, you follow this overall design approach:
l Define — Obtain an appropriate flow rate target and limits from the customer and
recommend a type of pump based on customer considerations
l Measure — Consider the elements of the Known Flow Rate equation to determine the pump
design features that affect flow rate and obtain information about tolerances, costs, and so
on
l Analyze — Use Monte Carlo analysis and sensitivity analysis to study how different design
elements affect pump performance and affect production quality
l Design — Use optimization techniques with defined tradeoffs between flow rate
performance and total cost to determine the design parameters that best meet a defined
quality level while reducing cost
l Validate — Based on the optimized design parameters, create physical prototypes and test
to confirm that this design configuration yields the best-performing, most cost-effective
pump
In the Define and Measure phases, you obtain and use information to build a model. Then, you
use Crystal Ball to simulate real-world variations that occur in the actual manufacturing of the
product. Finally, if you have Crystal Ball Decision Optimizer, you can use sensitivity analysis
and further optimization to arrive at the best balance between quality and cost.
The following sections give details of each phase:
l “Define Phase” on page 55
l “Measure Phase” on page 55
l “Analyze Phase” on page 56
l “Design Phase” on page 56
l “Validate Phase” on page 56
54
The next main section, “Using the DFSS Liquid Pump model” on page 56, works through these
phases using a example model.
Define Phase
Marketing and production considerations determine the acceptable range of flow rates based
on how many units should be produced daily to meet market demand, what is useful production
time per day based on plant setup and productivity, and how many pumps are placed in each
plant.
The food processing company translates marketing requirements into these engineering
specifications and provides them to you:
l Lower specification limit (LSL) = 47.26 ml/sec
l Upper specification limit (USL) = 53.92 ml/sec
l Target = 50.595 ml/sec
Based on these requirements, you propose a displacement pump with a reciprocating piston.
Figure 22, following, shows a basic schematic of the system.
Measure Phase
You notice that the variables in the Known Flow Rate equation are piston radius, piston stroke
length, backflow rate, and motor speed. You observe that the in-house machine shop controls
the piston radius and stroke length while the backflow valve and motor are purchased
components.
The machine shop indicates it is capable of 3-sigma quality levels within a tolerance of ±1
millimeter (mm) for the piston radius and stroke length. The purchasing department asks you
to use an inexpensive backflow valve and motor.
55
Analyze Phase
During the Analyze phase, you create a Microsoft Excel workbook that captures the Known Flow
Rate equation, characteristics of the pump components used in the equation (piston radius,
piston stroke length, motor speed, and backflow rate), flow rate target and limits, machining
cost coefficients, plus nine motor and backflow valve cost options.
Then, you define assumption and forecast cells and simulate the design options.
Design Phase
If you have OptQuest, you can also define decision variables for:
l A range of possible piston radius measures and associated tolerances (related to standard
deviation through capability)
l A range of possible piston stroke lengths and associated tolerances,
l The different possible backflow valve and motor options
After this is done, you can set up quality requirements for flow rate and optimize for the least-
cost configuration that meets those quality requirements.
Validate Phase
This phase involves the physical creation of a prototype based on simulation forecasts followed
by physical testing to validate the design. Because so much preliminary work was done with
spreadsheet modeling instead of building additional prototypes, a final design can be developed
quickly and at minimal cost.
56
Starting Crystal Ball
If is not already started, start it as usual.
57
Reviewing the Parts of the Model
The liquid pump spreadsheet models a design effort with two basic goals: meeting the flow rate
target as closely and consistently as possible and minimizing cost. The following sections describe
six main parts of the model, indicated by numbers in Figure 23:
l “Known Flow Rate Formula” on page 58
l “Flow Rate Variable Assumptions” on page 58
l “Flow Rate and Cost Forecasts” on page 58
l “Flow Rate Target and Limits” on page 59
l “Cost Coefficients” on page 59
l “Motor and Backflow Valve Cost Options” on page 60
The Known Flow Rate formula specifies the relevant variables discussed earlier: piston radius,
piston stroke length, backflow rate, and motor speed.
The flow rate variable assumptions are assumptions that allow each of the flow rate variables,
listed under the Known Flow Rate formula (1), to be included in the Monte Carlo simulation.
Each assumption is defined as a normal distribution with a mean equal to the Nominal Initial
Value and a standard deviation equal to the StDev Initial Value. The Mean and Standard
Deviation parameters are not entered directly. Instead, they are entered as cell references to the
Initial Value cells.
Note: In this model, whenever you see the word “nominal”, it means approximate, designed,
or theoretical, as opposed to the actual measured value.
58
The Flow Rate Forecast is based on the Known Flow Rate formula (1).
ä To view it:
1 Select cell K23.
The formula in that cell is the Known Flow Rate formula (a function of the flow rate
variables) expressed in Microsoft Excel format.
2 Click the Define Forecast button.
3 Notice that the values in cells E24, E25, and E23 are displayed in the Define Forecast dialog as the
process capability lower specification limit (LSL), upper specification limit (USL), and target value,
respectively.
They are defined as cell references. To demonstrate this, click in one of the text boxes. The
related cell reference is displayed. (Note: If you do not see these text boxes in the Define
Forecast dialog, process capability features have not been activated in the Run Preferences
dialog.)
The Total Cost Forecast is the sum of the Component Cost Function column values in the
upper table. To view it:
4 Select cell K25.
Notice its formula is simply the sum of the component costs in cells J18 through J21. No
target or limits have been established for this forecast.
The flow rate target and upper and lower specification limits (4) are values defined by the food
processing plant in the Define phase of the DFSS process. The previous section shows how they
can be used to set the Target, LSL, and USL for the Flow Rate Forecast in cell K23.
Cost Coefficients
The cost coefficients table contains values used to calculate the component cost functions,
summed to yield the Total Cost Forecast (3).
ä To see how these are used to calculate the component cost functions:
1 Select cell J18.
This formula uses the Base Cost Coefficient of the piston, the initial value of the piston
radius, the Tolerance Cost Coefficient of the piston, and the initial value of the standard
deviation. The cost is directly proportional to the radius squared due to raw material usage
59
and inversely proportional to the standard deviation squared (a common method to relate
tolerance to cost).
2 Optional: View the contents of cells J19, J20, and J21.
These formulas are also similarly derived from cost coefficients or nominal costs relevant to
each particular flow rate variable.
The motor and backflow valve cost options are two sets of nine options each provided by the
motor and backflow valve vendors. Different performance specifications have different costs
associated with them.
Running Simulations
At this point you have created assumptions and forecasts.
To run the simulation and generate forecast charts, click the Start button, .
Analyzing Simulations
When you run the simulation for this model, two forecast charts are displayed.
The Total Cost Forecast (Figure 24, following) includes no assumption cells. It is displayed as a
single value instead of a range of predicted values. The forecasted total cost is $26.73 for the
pump (shown here with the Mean marker line displayed).
Because you entered a target and specification limits, the Flow Rate Forecast chart opens in Split
View with the forecast chart first and then the process capability metrics, shown in Figure 25.
60
Figure 25 The Flow Rate Forecast with Process Capability Metrics
Notice that marker lines are displayed for the LSL, Target, and USL values. (You may see different
marker lines; the Mean and Standard Deviation marker lines have been removed to simplify the
screenshot.) Because the LSL and USL are defined, the certainty grabbers are automatically set
to those values. According to the value in the Certainty text box for this simulation, 95.34% of
forecasted values fall between 47.2617 and 53.9283, the specification limits. The mean is about
one standard deviation lower than the target.
The distribution curve is normal. Looking at the process capability metrics, the Z-LSL shows
the LSL is 1.63 sigmas below the mean. You were hoping to see a value of at least 3. The Cp
capability index is lower than 1, so the short-term potential sigma level is less than 3.
You generate a sensitivity chart (Figure 26) to determine which of the flow rate variables has the
most influence on the Flow Rate Forecast.
61
Figure 15 shows that the flow rate variable with the greatest effect on the Flow Rate Forecast is
the piston radius. Reducing the standard deviation of the piston radius should affect the overall
flow rate variance and improve the process capability metrics. You wonder how reducing the
piston standard deviation by 50% would affect the flow rate forecast quality. You decide to adjust
the model and find out.
Adjusting Models
ä To see the effect of reducing the standard deviation of the piston radius:
1 Select cell H18.
2 Reduce the value by half, to 0.16667.
3 Enter this same value in the Upper Limit cell, I18.
Now, the table of flow rate variables is displayed as shown in Figure 27.
62
In Figure 28, quality appears much better than that shown in Figure 14. The mean is less than
1.5 standard deviations lower than the target, and the standard deviation is smaller. USL is almost
6 sigmas away from the mean (Z-USL = 5.68) and LSL is over 2 sigmas away (Z-LSL = 2.49).
The certainty of the forecasted flow rate falling within LSL and USL is now over 99%.
The Total Cost Forecast chart has also changed, as shown in Figure 29. The quality improvements
result in higher costs. Where the total cost per pump was $26.73, now it is $33.47, a 25% increase.
You and the team must determine whether to sacrifice quality to lower the cost or whether to
test modifications to the other three flow rate variable components as well.
63
Defining Decision Variables
To optimize these variables with OptQuest, a set of decision variables are defined in cells C20,
C21, E18, E19, H18, and H19 (Figure 30).
Note: If you changed the Piston Radius standard deviation Initial Value and Upper Limit
for the last part of the DFSS Pump tutorial, be sure to change them both back to their
original values as shown in Figure 30. Also, be sure that these values are included in
the decision variable for the Piston Radius standard deviation (Cell H18). Select it as
described in steps 1 and 2.
The upper and lower bound for piston machining variables are the same as those in the
Lower Limit and Upper Limit tables of Figure 30. For the motor speed and backflow valve
variables, the upper and lower bounds are the lowest and highest options (1 and 9,
respectively).
In choosing the piston machining values shown in Figure 30, you could identify raw stock
availability to define a range of piston radii and lengths for investigation. You could also identify
a feasible range of tolerances for machining the radius and length.
64
Running OptQuest
5 Click Next until you reach the Options panel (Figure 32, following).
65
Figure 32 OptQuest Options Panel
66
Option Optimized Value
Motor option 2
Long- Short-
Metric term term Description
Cp n y Short-term capability index indicating what quality level the forecast output is potentially capable
of producing. It is defined as the ratio of the specification width to the forecast width. If a Cp is
equal to or greater than 1, then a short-term 3-sigma quality level is possible.
Pp y n Long-term capability index indicating what quality level the forecast output is potentially capable
of producing. It is defined as the ratio of the specification width to the forecast width. If a Cp is
equal to or greater than 1, then a long-term 3-sigma quality level is possible.
Cpk-lower n y One-sided short-term capability index; for normally distributed forecasts, the ratio of the
difference between the forecast mean and lower specification limit over three times the forecast
short-term standard deviation; often used to calculate process capability indices with only a
lower specification limit.
Ppk-lower y n One-sided long-term capability index; for normally distributed forecasts, the ratio of the
difference between the forecast mean and lower specification limit over three times the forecast
long-term standard deviation; often used to calculate process capability indices with only a lower
specification limit.
67
Long- Short-
Metric term term Description
Cpk-upper n y One-sided short-term capability index; for normally distributed forecasts, the ratio of the
difference between the forecast mean and upper specification limit over three times the forecast
short-term standard deviation; often used to calculate process capability indices with only an
upper specification limit.
Ppk-upper y n One-sided long-term capability index; for normally distributed forecasts, the ratio of the
difference between the forecast mean and upper specification limit over three times the forecast
long-term standard deviation; often used to calculate process capability indices with only an
upper specification limit.
Cpk n y Short-term capability index (minimum of calculated Cpk-lower and Cpk-upper) that takes into
account the centering of the forecast with respect to the midpoint of the specified limits; a Cpk
equal to or greater than 1 indicates a quality level of 3 sigmas or better.
Ppk y n Long-term capability index (minimum of calculated Ppk-lower and Ppk-upper) that takes into
account the centering of the forecast with respect to the midpoint of the specified limits; a Ppk
equal to or greater than 1 indicates a quality level of 3 sigmas or better.
Cpm n y Short-term Taguchi capability index; similar to Cpk but considers a target value, which may not
necessarily be centered between the upper and lower specification limits.
Ppm y n Long-term Taguchi capability index; similar to Ppk but considers a target value, which may not
necessarily be centered between the upper and lower specification limits.
Z-LSL y y The number of standard deviations between the forecast mean and the lower specification
limit.
Z-USL y y The number of standard deviations between the forecast mean and the upper specification
limit.
Zst y n For short-term metrics when only one specification limit is defined, equal to Z-LSL if there is only
a lower specification limit or Z-USL if there is only an upper specification limit.
Zst-total n y For short-term metrics when both specification limits are defined, the number of standard
deviations between the short-term forecast mean and the lower boundary of combining all
defects onto the upper tail of the normal curve. Also equal to Zlt-total plus the Z-score shift value
if a long-term index is available.
Zlt n y For long-term metrics when only one specification limit is defined, equal to Z-LSL if there is only
a lower specification limit or Z-USL if there is only an upper specification limit.
Zlt-total y n For long-term metrics when both specification limits are defined, the number of standard
deviations between the long-term forecast mean and the lower boundary of combining all defects
onto the upper tail of the normal curve. Also equal to Zst-total minus the Z-score shift value if
a short-term index is available.
p(N/C)-total y y Probability of a defect outside the lower and upper specification limits; DPUTOTAL
PPM-below y y Defects below the lower specification limit, per million units
PPM-above y y Defects above the upper specification limit, per million units
68
Long- Short-
Metric term term Description
LSL y y Lower specification limit, the lowest acceptable value of a forecast involved in process capability,
or quality, analysis.
USL y y Upper specification limit, the highest acceptable value of a forecast involved in process capability
analysis.
Target y y The ideal target value of a forecast involved in process capability analysis.
Z-score shift y y An optional shift value to use when calculating long-term capability metrics. The default, set in
the Capability Options panel, is 1.5.
Note: Unless otherwise stated, formulas use the mean and standard deviation of the best- fitted
non-normal distribution of the data when normality checks fail for a dataset and capability
statistics are needed.
Cp
Short-term capability index indicating what quality level the forecast output potentially is
capable of producing. It is defined as the ratio of the specification width to the forecast width.
If a Cp is equal to or greater than 1, then a short-term 3-sigma quality level is possible.
Formula:
Pp
Long-term capability index indicating what quality level the forecast output is potentially capable
of producing. It is defined as the ratio of the specification width to the forecast width. If a Pp is
equal to or greater than 1, then a short-term 3-sigma quality level is possible.
Formula:
69
Cpk-lower
One-sided short-term capability index; for normally distributed forecasts, the ratio of the
difference between the forecast mean and lower specification limit over three times the forecast
short-term standard deviation; often used to calculate process capability indices with only a
lower specification limit.
Formula:
Ppk-lower
One-sided long-term capability index; for normally distributed forecasts, the ratio of the
difference between the forecast mean and lower specification limit over three times the forecast
long-term standard deviation; often used to calculate process capability indices with only a lower
specification limit.
Formula:
Cpk-upper
One-sided short-term capability index; for normally distributed forecasts, the ratio of the
difference between the forecast mean and upper specification limit over three times the forecast
short-term standard deviation; often used to calculate process capability indices with only an
upper specification limit.
Formula:
Ppk-upper
One-sided long-term capability index; for normally distributed forecasts, the ratio of the
difference between the forecast mean and upper specification limit over three times the forecast
long-term standard deviation; often used to calculate process capability indices with only an
upper specification limit.
Formula:
70
Cpk
Short-term capability index (minimum of calculated Cpk-lower and Cpk-upper) that takes into
account the centering of the forecast with respect to the midpoint of the specified limits; a Cpk
equal to or greater than 1 indicates a quality level of 3 sigmas or better.
Formula:
where:
Ppk
Long-term capability index (minimum of calculated Cpk-lower and Cpk-upper) that takes into
account the centering of the forecast with respect to the midpoint of the specified limits; a Ppk
equal to or greater than 1 indicates a quality level of 3 sigmas or better.
Formula:
where:
Cpm
Short-term Taguchi capability index; similar to Cpk but considers a target value, which may not
necessarily be centered between the upper and lower specification limits.
Formula:
71
Ppm
Long-term Taguchi capability index; similar to Ppk but considers a target value, which may not
necessarily be centered between the upper and lower specification limits.
Formula:
Z-LSL
The number of standard deviations between the forecast mean and the lower specification limit.
Formula:
Z-USL
The number of standard deviations between the forecast mean and the upper specification limit.
Formula:
Zst
For short-term data, ZST = ZTOTAL, expressed as Zst-total,
where
and
72
is the inverse normal cumulative distribution function, which assumes a right-sided tail.
In Microsoft Excel:
When displaying short-term metrics, ZST appears as Zst-total. This metric is equal to Z-LSL if
there is only a lower specification limit, or Z-USL if there is only an upper specification limit.
For long-term data, ZST = ZLT + ZScoreShift. When displaying long-term metrics, ZST appears
in the capability metrics table as Zst.
Note: Z scores typically are reported only for normal data. The maximum value for Z scores
calculated by from forecast data is 21.18.
Zst-total
For short-term metrics when both specification limits are defined, the number of standard
deviations between the short-term forecast mean and the lower boundary of combining all
defects onto the upper tail of the normal curve. Also equal to Zlt-total plus the Z-score shift
value if long-term metrics are calculated.
When short-term metrics are calculated, Zst-total is equivalent to ZST, described in the previous
section.
Zlt
For long-term data, ZLT = ZTOTAL, expressed as Zlt-total,
where
ZTOTAL =
and
is the inverse normal cumulative distribution function, which assumes a right-sided tail.
In Microsoft Excel:
When displaying long-term metrics, ZLT appears as Zlt-total. This metric is equal to Z-LSL if
there is only a lower specification limit or Z-USL if there is only an upper specification limit.
73
For short-term data, ZLT = ZST - ZScoreShift. When displaying short-term metrics, ZLT appears
in the capability metrics table as Zlt.
Note: Z scores typically are reported only for normal data. The maximum value for Z scores
calculated by from forecast data is 21.18.
Zlt-total
For long-term metrics when both specification limits are defined, the number of standard
deviations between the long-term forecast mean and the lower boundary of combining all defects
onto the upper tail of the normal curve. Also equal to Zst-total minus the Z-score shift value if
short-term metrics are calculated.
When long-term metrics are calculated, Zlt-total is equivalent to ZLT, described in the previous
section.
p(N/C)-below
Probability of a defect below the lower specification limit; DPUBELOW .
Formula:
where F is the area beneath the normal curve below the LSL, otherwise known as unity minus
the normal cumulative distribution function for the LSL (assumes a right-sided tail).
In Microsoft Excel:
Note: When calculated from the best-fitted non-normal distribution, p(N/C)-below = F(LSL),
where F is the cumulative distribution function for the best-fitted non-normal
distribution. When calculated from forecast values, p(N/C)-below is the percentile of the
values in the forecast that fall below the LSL.
p(N/C)-above
Probability of a defect above the upper specification limit; DPUABOVE.
Formula:
where F is the area beneath the normal curve above the USL, otherwise known as unity minus
the normal cumulative distribution function for the USL (assumes a right-sided tail).
74
In Microsoft Excel:
p(N/C)-total
Probability of a defect outside the lower and upper specification limits; DPUTOTAL.
Formula:
PPM-below
Defects below the lower specification limit, per million units.
Formula:
PPM-above
Defects above the upper specification limit, per million units.
Formula:
PPM-total
Defects outside both specification limits, per million units.
Formula:
LSL
Lower specification limit; the lowest acceptable value of a forecast involved in process capability,
or quality, analysis. User-defined by direct entry or reference when defining a forecast.
75
USL
Upper specification limit; the highest acceptable value of a forecast involved in process capability,
or quality, analysis. User-defined by direct entry or reference when defining a forecast.
Target
The ideal target value of a forecast involved in process capability analysis. User-defined by direct
entry or reference when defining a forecast.
Z-score Shift
An optional shift value to use when calculating long-term capability metrics. The default, set in
the Capability Options dialog box, is 1.5.
76
Probability Distribution
5 Examples and Reference
In This Chapter
Custom Distribution Examples ............................................................................77
Comparing Distribution Types.............................................................................91
Sequential Sampling with Custom Distributions ........................................................92
Formulas for Probability Distributions ....................................................................94
Distribution Fitting Methods............................................................................. 104
Distribution Parameter Defaults ........................................................................ 105
If none of the provided distributions fits the data, you can use the custom distribution to define
one. For example, a custom distribution can be especially helpful if different ranges of values
have specific probabilities. You can create a distribution of one shape for one range of values
and a different distribution for another range. You can describe a series of single values, discrete
ranges, or continuous ranges. This section uses real-world examples to describe the custom
distribution.
Since it is easier to understand how the custom distribution works with a hands-on example,
you may want to start and use it to follow the examples. To follow the custom distribution
examples, first create a new Microsoft Excel workbook then select cells as specified.
The listed sections show how to use the custom distribution. Also see Appendix A in the Oracle
Crystal Ball User's Guide.
77
2 Select Define, then Define Assumption.
The Distribution Gallery dialog opens.
3 Click the All category to select it.
4 Scroll to find the custom distribution, then click it.
5 Click OK.
Crystal Ball displays the Define Assumption dialog.
Using the custom distribution, a company can describe the probable retail cost of a new product.
The company decides the cost could be $5, $8, or $10. In this example, you will use the custom
distribution to describe a series of single values.
78
5 Click Enter.
Crystal Ball displays a relative probability of 1.00 for the value 10. A third single value bar
represents the value 10.
Figure 35 on page 79 shows the value bars for the values 5, 8, and 10, each with a relative
probability of 1.00.
Now, each value has a probability of 1. However, when you run the simulation, their total relative
probability becomes 1.00 and the probability of each value is reset to .3333.
ä To reset their probabilities before you run the simulation, follow these steps:
1 Click the bar with a value of 5.00.
Its value is displayed in the Value text box.
2 Type the probability as the formula =1/3 in the Probability text box and click Enter.
You could also enter a decimal — for example, 0.333333 — but the formula is more exact.
3 Follow steps 6 and 7 for the other two bars.
Crystal Ball rescales each value to a relative probability of 0.33 on the Relative Probability
scale along the first vertical axis (Figure 36 on page 80).
79
Figure 36 Single Values with Adjusted Probabilities
ä Before beginning example two, clear the values entered in example one as follows:
1 Right-click in the chart and select Clear Distribution from the menu.
2 Select Parameters, and then Continuous Ranges to enter value ranges.
3 Enter the first range of values:
l Type 5 in the Minimum text box.
l Type 15 in the Maximum text box.
l Type .75 in the Probability text box. This represents the total probability of all values
within the range.
4 Click Enter.
Crystal Ball displays a continuous value bar for the range 5.00 to 15.00, as in Figure 37 on
page 81, and returns the cursor to the Minimum text box. Notice that the height of the range
is 0.075. This represents the total probability divided by the width (number of units) in the
range, 10.
80
Figure 37 A Continuous Custom Distribution
ä You can change the probability and slope of a continuous range, as described in the following
steps:
1 Click anywhere on the value bar for the range 16 to 21.
The value bar changes to a lighter shade.
81
2 Select Parameters, and then Sloping Ranges.
Additional parameters are displayed in the Custom Distribution dialog (Figure 39 on page
82).
3 Set Height of Min. and Height of Max. equal to what currently is displayed in the chart, 0.05.
This can be an approximate value. Height of Min. is the height of the range Minimum and
Height of Max. is the height of the range Maximum.
4 Click Enter.
The range returns to its original color and its height is displayed unchanged.
5 Click in the range again to select it and set Height of Max. to 0.025. Then, click Enter.
The second side of the range drops to half the height of the first (Figure 40 on page 82).
The range is selected to show its parameters after the change.
6 Optional: Change the range from continuous to discrete values by adding a step value. Type .5 in the
Step text box and click Enter.
The sloped range is now discrete. Separate bars are displayed at the beginning and end of
the range and every half unit in between (16, 16.5, 17, 17.5 and so on until 21), as shown in
Figure 41 on page 83. If the discrete range represented money, it could only include whole
dollars and 50-cent increments.
You can enter any positive number in the Step text box. If you entered 1 in this example,
the steps would fall on consecutive integers, such as whole dollars. Leave the Step parameter
blank for continuous ranges.
82
Figure 41 A Sloped Discrete Range with Steps of .5
Although the bars have spaces between them, their heights and the width of the range they cover
are equal to the previous continuous sloped range and the total probability is the same.
While a second continuous range could have extended from 15 to 20, the second range in this
example starts at 16 rather than 15 to illustrate a discrete range because, unlike continuous
ranges, discrete ranges cannot touch other ranges.
With Crystal Ball, you can enter single values, discrete ranges, or continuous ranges individually.
You also can enter any combination of these three types in the same Custom Distribution dialog
as long as you follow these guidelines: ranges and single values cannot overlap one another;
however, the ending value of one continuous range can be the starting value of another
continuous range.
83
Figure 42 Four-column Custom Data Range
In this case, discrete ranges have the most parameters. So, you can create an assumption, select
Custom Distribution, and then select Parameters, and then Discrete Ranges before loading the
data.
If the data also included discrete sloping ranges, you could select Parameters, and then Sloping
Ranges before loading the data. The data table would then have five columns and could
accommodate all data types.
A column is displayed for each parameter in the current set (selected using the Parameters
menu). Parameters, then Discrete Ranges was selected before viewing the data table, so there
is a column in the data table for each discrete range parameter. Because the single value and
continuous ranges have subsets of the same group of parameters, their parameters will also
fit into the table.
2 Since the values are already on the worksheet, you can click Load Data to enter them into the Custom
Distribution dialog.
84
The Load Data dialog opens (Figure 44 on page 85).
The default settings are appropriate for most purposes, but the following other options are
available:
l When loading unlinked data, you can choose to replace the current distribution with
the new data or append new data to the existing distribution.
l If probabilities are entered cumulatively into the spreadsheet you are loading, select
Probabilities are cumulative. Then, determines the probabilities for each range by
subtracting the previous probability from the one entered for the current range. You
can select View, and then Cumulative Probability to display the data cumulatively in the
assumption chart.
3 Enter a location range for the data, in this case A2:D7. If the range has a name, you can enter the
name preceded by an = sign.
4 When all settings are correct, click OK.
Crystal Ball enters the values from the specified range into the custom distribution and plots the
specified ranges (Figure 45 on page 86).
85
Figure 45 Custom Data from Worksheet
Follow the rules in this section for loading data. If a data range has been named, you can enter
the name (preceded by the = sign) in the range text box of the Load Data dialog.
The listed sections show how to enter tables of data into custom distributions.
Unweighted Values
Single values are values that do not define a range. Each value stands alone. For a series of single
values with the same probabilities (unweighted values), use a one-column format or more than
five columns (Figure 46 on page 86). The values go in each cell and the relative probabilities
are all assumed to be 1.0. Select Parameters, and then Unweighted Values to enter these.
86
Figure 47 on page 87 shows the custom distribution created by loading the unweighted values
illustrated in Figure 46.
Weighted Values
For a series of single values all with different probabilities, use a two-column format. The first
column contains single values, the second column contains the probability of each value
(Figure 48 on page 87).
Note: Blank probabilities are interpreted as a relative probability of 1.0. Values with zero
probability should be explicitly entered with a probability of 0.0.
Figure 49 on page 88 shows the custom distribution created by loading the values illustrated
in Figure 48.
87
Figure 49 Weighted Values Loaded in a Custom Distribution
88
Figure 50 Mixed Ranges, Including Sloping Ranges
Figure 51 on page 89 shows the custom distribution created by loading the values illustrated
in Figure 50.
89
For example, row 20 inFigure 50 on page 89 shows a connected continuous sloping range. The
Minimum cell is blank because the Minimum value is equal to 7, the previous Maximum. The Height
of Min. is blank because it is equal to 6, the previous Height of Max.
Figure 53 on page 90 shows the custom distribution created by loading the data illustrated in
Figure 52.
90
l Blank probabilities are interpreted as a relative probability of 1.0. Values with zero
probability should be explicitly entered with a probability of 0.0.
l For continuous connected ranges, for either endpoint values or probabilities, if the starting
cell is blank, the previous end value is used as the start for this range.
l When you load a discrete value that exists in the table already, its probability is incremented
by 1. For continuous ranges, this is not allowed; an error message about overlapping ranges
opens.
Note: In versions of Crystal Ball earlier than 11.1.1.3.00, ranges or values with 0 probabilities
were removed. Sloping ranges with Height of Min. and Height of Max. equal to 0 were
also removed. The current rules are as follows:
In the simple view for weighted values, if users enter a probability of 0 for a value, the
users are prompted about deleting the value. For other types of values, 0 works as before
to delete the range or value.
In advanced (table) view for weighted values, if users enter a probability of 0 for a value,
it is ignored (or added, which is the same thing). For other types of values, 0 works as
before to delete the range or value.
If a custom distribution was created with unlinked data in a version of Crystal Ball earlier
than version 11.1.1.3.00, the frequency distribution does not change when the model is
loaded in Crystal Ball version 11.1.1.3.00 or later. However, if the input range was linked
for a custom distribution, the frequency distribution is updated according to current rules.
All new custom distributions, linked and unlinked, are evaluated according to current
rules.
91
The binomial and hypergeometric distributions are also closely related. As the number of trials
and the population size increase, the hypergeometric trials tend to become independent like the
binomial trials: the outcome of a single trial has a negligible effect on the probabilities of
successive observations. The differences between these two types of distributions become
important only when you are analyzing samples from relatively small populations. As with the
Poisson and binomial distributions, requires a similar amount of time to compute both the
binomial and hypergeometric distributions.
The yes-no distribution is simply the binomial distribution with Trials = 1.
The Weibull distribution is flexible. Actually, it consists of a family of distributions that can
assume the properties of several distributions. When the Weibull shape parameter is 1.0, the
Weibull distribution is identical to the exponential distribution. The Weibull location parameter
lets you set up an exponential distribution to start at a location other than 0.0. When the shape
parameter is less than 1.0, the Weibull distribution becomes a steeply declining curve. A
manufacturer may find this effect useful in describing part failures during a burn-in period.
When the shape parameter is equal to 2.0, a special form of the Weibull distribution, called the
Rayleigh distribution, results. A researcher may find the Rayleigh distribution useful for
analyzing noise problems in communication systems or for use in reliability studies. When the
shape parameter is set to 3.25, the Weibull distribution approximates the shape of the normal
distribution; however, for applications when the normal distribution is appropriate, us it instead
of the Weibull distribution.
The gamma distribution is also a flexible family of distributions. When the shape parameter is
1.0, the gamma distribution is identical to the exponential distribution. When the shape
parameter is an integer greater than one, a special form of the gamma distribution, called the
Erlang distribution, results. The Erlang distribution is especially useful in the areas of inventory
control and queueing theory, where events tend to follow Poisson processes. Finally, when the
shape parameter is an integer plus one half (e.g., 1.5, 2.5, etc.), the result is a chi-square
distribution, useful for modeling the effects between the observed and expected outcomes of a
random sampling.
When no other distribution seems to fit the historical data or accurately describes an uncertain
variable, you can use the custom distribution to simulate almost any distribution. The Load Data
button on the Custom Distribution dialog lets you read a series of data points or ranges from
value cells in the worksheet. If you like, you can use the mouse to individually alter the
probabilities and shapes of the data points and ranges so that they more accurately reflect the
uncertain variable.
92
structure between SIP variables without having to explicitly compute and define a matrix of
correlation coefficients.
SIPs can be represented by custom distributions in and can then be published and shared by
organizations using Crystal Ball’s Publish and Subscribe features in the Distribution Gallery.
The distribution is displayed in the Define Assumption dialog (Figure 54 on page 93).
You can select Edit, and then Add to Gallery to add the SIP to a custom library (Category) in the
Distribution Gallery. You can also use Publish and Subscribe on the Distribution Gallery
Categories menu to share the library of SIPs with others.
93
Running Simulations with SIPs
To run a simulation with one or more SIPs, define assumptions in the model as usual, pulling
in SIPs from a library (Category) in the Distribution Gallery as needed. When the simulation
runs, samples the SIP values sequentially instead of randomly. If the simulation has more trials
than the SIP has values, sampling wraps around to the first value of the distribution. If you are
using several SIPs with different numbers of values, a warning is displayed. You can either
continue, knowing that mismatched values will be sampled together after sampling wraps for
one SIP but not the other, or you can cancel the simulation and correct the problem. You can
also run fewer trials so that sampling doesn’t need to wrap for any SIP.
Correlations are not allowed for assumptions defined as SIPs, since the correlation structures
are implicitly defined for them. Warnings are displayed if correlations are defined for them.
Two Developer Kit calls, CB.DefineAssumND and CB.GetAssum, support the new parameter
for sequential sampling. The Developer Kit constant is called cbAsmIsSequential, a Boolean type.
Beta Distribution
Parameters: Minimum value (Min), Maximum value (Max), Alpha (α), Beta (β)
Formula:
for
where:
where:
94
Method 2: Rational Fraction Approximation method with a Newton Polish step
Comment: This method is used instead of Method 1 when Latin Hypercube sampling is in effect.
BetaPERT Distribution
Parameters: Minimum value (Min), Most likely value (Likely), Maximum value (Max)
Formula:
for
where:
Binomial Distribution
Parameters: Probability of success in each trial (p), Number of total trials (n)
Formula:
for i = 0,1,2,...n;
where:
95
Discrete Uniform Distribution
Parameters: Minimum value (Min), Maximum value (Max)
Formula:
Comment: This is the discrete equivalent of the uniform distribution, described in “ Uniform
Distribution” on page 103. In this distribution, a finite number of equally spaced values are
equally likely to be observed.
Exponential Distribution
Parameters: Success rate ( )
Formula:
Gamma Distribution
This distribution includes the Erlang and Chi-Square distributions as special cases.
Method 1:
96
When is greater than 1, Best’s rejection from a t density with 2 degrees of freedom.
Geometric Distribution
Parameters: Probability of success in each trial (p)
Formula:
for:
0 < p < 1 and i = 0, 1, 2,... ∞
where x = number of failures before the first success
Method: Inverse Transformation
Hypergeometric Distribution
Parameters:
Number of successful items in the population (Nx), sampled trials (n), population size (N)
Formula:
where:
for:
and N ≤ 1e5
and x = number of successful trials,
so Nx = number of successful items in the population.
Method: Direct Simulation
Comment: Computation time increases linearly with population size.
97
Logistic Distribution
Parameters:
Mean (μ), Scale (s)
Formula:
for:
where:
Lognormal Distribution
Parameters: Location (L), Mean ( ), Standard Deviation ( )
Mean =
Median =
Mode =
98
where ln = natural logarithm.
Formula:
for:
Arithmetic variance =
for:
99
where:
for:
where:
where:
100
Method: Direct Simulation through summation of Geometric variates
Comment: Computation time increases linearly with Shape.
Normal Distribution
This distribution is also known as the Gaussian distribution.
Parameters:
Mean ( ), Standard Deviation ( )
Formula:
for:
Pareto Distribution
Parameters:
101
Poisson Distribution
Parameters: Rate ( )
Formula:
for:
i = 0, 1, 2,... ∞ and 0 < λ <= 1e9
Method: Direct Simulation through Summation of Exponential Variates
Comment: Computation time increases linearly with Rate.
Student’s t-Distribution
Parameters: Midpoint (m), Scale (s), Degrees of Freedom (d)
Formula:
where:
and where:
and where:
= the gamma function
Triangular Distribution
Parameters: Minimum value (Min), Most likely value (Likeliest), Maximum value (Max)
Formula:
102
where:
Uniform Distribution
Parameters: Minimum value (Min), Maximum value (Max)
Formula:
Comment:
The generator has a period of length 231 - 2, or 2,147,483,646. This means that the cycle of random
numbers repeats after several billion trials. This formula is discussed in detail in the Simulation
Modeling & Analysis and Art of Computer Programming, Vol. II, references in Appendix A,
“Bibliography.”.
Weibull Distribution
A Weibull distribution with Shape = 2 is also known as the Rayleigh distribution.
Parameters:
where:
103
is the Gamma function.
Method: Inverse Transformation
Yes-No Distribution
This distribution is equivalent to the binomial distribution with Trials = 1. For details, see
“Binomial Distribution” on page 95.
Custom Distribution
Formula:
The formula consists of a lookup table of single data points, continuous ranges, and discrete
ranges. Each item in the table has a distinct probability relative to the other items. In addition,
ranges might be positively or negatively sloped, giving values on one side or the other a higher
probability of occurring.
Method: Sequential search of relative probabilities table.
Comments:
A Uniform variate is generated in the range (0, total relative probability). A sequential search of
the relative probabilities table is then performed. The Inverse Transformation method is used
whenever the uniform variate falls within a continuous or discrete range that is sloped in one
direction or the other.
Additional Comments
All of the nonuniform generators use the same uniform generator as the basis for their
algorithms.
The Inverse Transformation method is based on the property that the cumulative distribution
function for any probability distribution increases monotonically from zero to one. Thus, the
inverse of this function can be computed using a random uniform variate in the range (0, 1) as
input. The resulting values then have the desired distribution.
The Direct Simulation method actually performs a series of experiments on the selected
distribution. For example, if a binomial variate is being generated with Prob = .5 and Trials =
20, then 20 uniform variates in the range (0, 1) are generated and compared with Prob. The
number of uniform variates found to be less than Prob then becomes the value of the binomial
variate.
104
set. Sometimes, however, the MLEs do not exist for some distributions (for example, gamma,
beta). In these cases, Crystal Ball resorts to other natural parameter estimation techniques.
When the MLEs do exist, they exhibit desirable properties:
l They are minimum-variance estimators of the parameters.
l As the data set grows, the biases in the MLEs tend to zero.
For several of the distributions (for example, uniform, exponential), it is possible to remove the
biases after computing the MLEs to yield minimum-variance unbiased estimators (MVUEs) of
the distribution parameters. These MVUEs are the best possible estimators.
This section lists the initial values provides for the primary parameters in the Define Assumption
dialog.
If an alternate parameter set is selected as the default mode, the primary parameters are still
calculated as described below before conversion to the alternate parameters.
Note: Extreme values on the order of 1e±9 or ±1e16 may yield results somewhat different from
those listed here.
105
Beta
If the cell value is 0:
Minimum is -10.00
Maximum is 10.00
Alpha is 2
Beta is 3
Otherwise:
Minimum is cell value - (absolute cell value divided by 10)
Maximum is cell value + (absolute cell value divided by 10)
Alpha is 2
Beta is 3
For out-of-range values, such as ±1e300:
Minimum is 0
Maximum is 1
Alpha is 2
Beta is 3
BetaPERT
If the cell value is 0:
Likeliest is 0
Minimum is -10.00
Maximum is 10.00
Otherwise:
Likeliest is cell value
Minimum is cell value - (absolute cell value divided by 10)
Maximum is cell value + (absolute cell value divided by 10)
Binomial
If the cell value is between 0 and 1:
Probability is the cell value
Trials is 50
If the cell value is between 1 and 1e9 (the maximum number of binomial trials):
106
Probability (Prob) is 0.5
Trials is cell value
Otherwise:
Probability (Prob) is 0.5
Trials is 50
Custom
Initially empty.
Discrete Uniform
If the cell value is 0 or -1e9:
Minimum is 0
Maximum is 10
Otherwise:
Minimum is cell value - INT (absolute cell value divided by 10)
Maximum is cell value + INT (absolute cell value divided by 10)
Exponential
If the cell value is 0, rate is 1.0.
Otherwise, rate is 1 divided by the absolute cell value.
Gamma
If the cell value is 0:
Location is 0.00
Scale is 1.00
Shape is 2
Otherwise:
Location is cell value
Scale is absolute cell value divided by 10
Shape is 2
107
Geometric
If the cell value is greater than 0 and less than 1, probability is cell value.
Otherwise, probability is 0.2.
Hypergeometric
If the cell value is greater than 0 and less than 1:
Success is 100 times cell value
Trials is 50
Population size is 100
If the cell value is between 2 and the maximum number of Hypergeometric trials (1e5):
Success is cell value divided by 2 (rounded downward)
Trials is cell value divided by 2 (rounded downward)
Population size is cell value
Otherwise:
Success is 50
Trials is 50
Population size is 100
Logistic
If the cell value is 0:
Mean is 0
Scale is 1.0.
Otherwise:
Mean is cell value
Scale is absolute cell value divided by 10
Lognormal
If the cell value is greater than 0:
Mean is cell value
Standard deviation is absolute cell value divided by 10
Otherwise:
Mean is e
108
Standard deviation is 1.0
Negative Binomial
If the cell value is less than or equal to 0:
Probability is 0.2
Shape is 10
If the cell value is greater than 0 and less than 1:
Probability is cell value
Shape is 10
Otherwise, unless the cell value is greater than 100:
Probability is 0.2
Shape is cell value
If the cell value is greater than 100, the shape is 10.
Normal
If the cell value is 0:
109
Mean is 0
Standard deviation is 1.00
Otherwise, unless the cell value is more than 100:
Mean is cell value
Standard deviation is absolute cell value divided by 10.0
Pareto
If the cell value is between 1.0 and 1,000:
Location is cell value
Shape is 2
Otherwise:
Location is 1.00
Shape is 2
Poisson
If the cell value is less than or equal to 0, the rate is 10.00.
If the cell value is greater than 0 and less than or equal to the maximum rate (1e9), the rate is
the cell value.
Otherwise, the rate is 10.00
Student’s t
If the cell value is 0:
Midpoint is 0
Scale is 1.00
Degrees is 5
Otherwise:
Midpoint is cell value
Scale is absolute cell value divided by 10
Degrees is 5
Triangular
If the cell value is 0:
110
Likeliest is 0
Minimum is -10.00
Maximum is 10.00
Otherwise:
Likeliest is cell value
Minimum is cell value minus absolute cell value divided by 10
Maximum is cell value plus absolute cell value divided by 10
Uniform
If the cell value is 0:
Minimum is -10.00
Maximum is 10.00
Otherwise:
Minimum is cell value minus absolute cell value divided by 10.0
Maximum is cell value plus absolute cell value divided by 10.0
Weibull
If the cell value is 0:
Location is 0
Scale is 1.00
Shape is 2
Otherwise:
Location is cell value
Scale is absolute cell value divided by 10
Shape is 2
Yes-No
If the cell value is greater than 0 and less than 1, the probability of Yes(1) equals the cell value.
Otherwise, the probability of Yes(1) equals 0.5.
111
112
Predictor Examples and
6 Reference
In This Chapter
Predictor Examples ...................................................................................... 113
Important Predictor Concepts........................................................................... 124
Techniques and Formulas ............................................................................... 144
Predictor Examples
Subtopics
l About These Examples
l Inventory Control
l Company Finances
l Human Resources
113
l “Human Resources” on page 121
For more detailed product tutorials, see the Oracle Crystal Ball Predictor User's Guide.
Inventory Control
The initial reason that Monica needs to forecast is to maintain enough ingredients to keep up
with production. Monica's distributors give her discounts for buying in bulk. However, she must
balance this savings with maintaining product quality, which requires using the freshest
ingredients possible. Monica wants improved forecasting to help her place orders that give her
the best volume pricing while maintaining the quality of her products.
To follow along with this example, open Bakery.xlsx as described in “About These Examples”
on page 113.
The Sales Data worksheet (Figure 55) shows the daily sales data of each of these products from
the opening until the end of June 2015.
A summary of sales data for the three main products by week is displayed at the bottom of the
Operations worksheet (Figure 56 on page 115).
114
Figure 56 Weekly Sales Totals for Three Products
Summary tables at the top of the Operations worksheet enable Monica to summarize total sales
by price, units, and weight. She can also see details of main ingredients by product (Figure 57).
Monica wants to order monthly, one month in advance. The bakery has already received this
month’s delivery, which she placed last month. This month, she must place the order that will
be delivered at the end of this month for the next month, so she must forecast sales for the next
two months. Because she is in week 173 of her business, the forecast is for weeks 174 to 181.
115
ä To forecast the sales for weeks 174 to 181:
1 In the Bakery.xlsx workbook, click the Operations tab.
The Operations worksheet is displayed.
2 Select one cell—for example, C41—in the Historical Demand By Week table at the bottom of the
worksheet.
3 Start Predictor.
Predictor automatically selects all the data in that table.
4 Verify the following settings:
l The cell range $B$40:$E$213 is selected correctly on the Input Data panel, with headers,
dates, and data in columns settings also selected.
l The Data Attributes panel shows time periods are in weeks with Seasonality set to
AutoDetect.
l In the Methods panel, Multiple Linear Regression is cleared and all time-series methods
are selected
l Options settings are the defaults, RMSE and Standard forecasting.
5 Click Run.
6 In the Predictor Results window, set Periods to forecast to 8, and then click Paste.
7 In the Paste Forecasts to Spreadsheet dialog, use the following settings and click OK:
l Select At end of historical data to indicate where to paste results.
l Set Paste as to Random walk formulas.
l Select Include date series to list dates in the first column.
l Confirm that AutoFormat is selected.
The results paste to the end of the Historical Demand table as shown in Figure 58.
116
Figure 58 Forecasted Bakery Operations Results
The last four weeks of forecast values for each data series are automatically summed and placed
into the table at the top of the spreadsheet, in the Sales Forecast column (cells C9:C11). In this
table, the monthly sales forecast is converted to the number of items sold and then into the
weight of each product (Figure 59 on page 118).
117
Figure 59 Sales Forecast and Related Data for Last Four Weeks of Weekly Sales Table
The second table in Figure 59 takes the total weight of each product (in cells C15, C19, C23) and
calculates how much of each ingredient is required to produce that much product. The
ingredients for each are then summed in the third table (below the second table) into the total
amounts to order for the month (cells D31 to D34).
Based on the forecast, Monica should order:
l 12,080 pounds of flour
l 55 pounds of yeast
l 42 pounds of salt
l 125 pounds of tomatoes
Company Finances
Monica is always concerned about the bakery’s month-to-month cash flow (on a percent of sales
basis). Predictor can help her manage her inventory, and she can use it to predict her revenue
and understand her cash flow situation better. Understanding the bakery’s cash flow can, in turn,
help her better manage major capital expenditures.
Monica is considering two major capital expenditures: a flour silo and a delivery van. She wants
to start construction on the silo in July and purchase the delivery van in August. She needs to
118
forecast when the bakery can safely pay for these projects or whether the bakery must finance
them.
To follow along with this example, open Bakery.xlsx as described in “About These Examples”
on page 113. If Bakery.xlsx is already open from the previous example, select Reset to clear any
existing results.
Note: Any pasted data will remain until you clear it in Microsoft Excel.
The bakery cash flow information is laid out in the Cash Flow worksheet, shown in Figure 60.
This worksheet has a table at the bottom that summarizes the sales data for the bakery’s three
main products by month. You can forecast the next three months of revenue to decide when to
attempt the capital expenditures.
119
ä To forecast the next three months of revenue:
1 In the Bakery.xlsx workbook, click the Cash Flow tab.
The Cash Flow worksheet is displayed.
2 Select one cell—C36, for example—in the Historical Revenue By Month table at the bottom of the
worksheet.
3 Start Predictor.
Predictor automatically selects all the data in the Historical Revenue table.
4 Confirm the following settings:
l In the Input Data panel, the cell range $C$36:$AP$36 is selected correctly, with Data in
rows selected and no date or header settings selected.
l In Data Attributes, the first setting is Data is in periods with Seasonality set to
AutoDetect.
l In Methods, all time-series methods are selected and Multiple Linear Regression is cleared
(if available).
l Options settings are the defaults: RMSE and Standard forecasting.
5 Click Run.
6 In the Predictor Results window, enter 3 for Periods to forecast.
7 Click Paste, and then, in the Paste Forecasts to Spreadsheet dialog, confirm that results are set to
paste at the end of historical data as random walk formulas and click OK.
The results paste into the table at the bottom of the worksheet, cells AQ36 to AS36, and also
are displayed at the top of the worksheet (cells E4 to G4) as shown in Figure 61.
Tip: If ###... is displayed in place of the pasted data, make the columns wider.
120
Figure 61 Monthly Net Cash Flow Results
The revenue forecasts for the next three months are used to calculate the percentage expenses
in the second table.
The second table calculates the total expenses, and the third table calculates the necessary
expenditure for each extraordinary item. Below these tables is the cash flow summary for the
next three months, based on the forecasts. The net cash at the end of each month is what Monica
is looking for (row 27). Based on forecasted sales, the new net cash values are $20,922.05 for
July, $14,598.89 for August, and $39,634.45 for September. Based on the forecast, the bakery
should wait until September to buy the van.
Human Resources
Monica's Bakery is a labor-intensive operation that pays a competitive wage. However, to
maintain her target profitability, Monica must control labor costs. She knows many things are
done around the bakery that could be done by expensive machinery, such as kneading, mixing,
and forming. By accurately predicting her labor costs, she can decide when to invest in some of
this equipment to keep her total expenses within budget.
From her interest in economics, Monica knows that a few key macro-economic figures drive
labor costs, such as the Industrial Production Index, local CPI, and local unemployment. All of
these figures are available on the Internet on a monthly basis from the Bureau of Labor Statistics
and the Department of Commerce.
121
To follow along with this example, open Bakery.xlsx as described in “About These Examples”
on page 113. If Bakery.xlsx is already open from the previous example, select Reset to clear
existing results.
Monica has created her Labor Costs worksheet with an interactive table at the bottom that lists
the bakery's average hourly wage for each month and the monthly numbers for the three
economic indicators.
The average hourly wage depends on or is affected by the other three variables. Because of the
dependency, Monica decides to use regression instead of time-series forecasting. For regression,
the dependent variable is Monica’s Average Wage, and the other three are the independent
variables.
122
l In Methods, Non-seasonal Methods, ARIMA, and Multiple Linear Regression are selected
l Regression variables are defined as follows: Monica’s Average Wage is a dependent
variable, all the others are independent variables
l In Options, RMSE and Standard forecasting are selected
5 Click Run.
6 In the Predictor Results window, set Periods to forecast to 6 and click Paste.
7 In Paste as, set the controls to paste random walk formulas at the end of historical data with Include
date series and AutoFormat selected.
8 Click OK.
The results paste at the bottom of the table (cells B51 to F56) as shown in Figure 63.
Predictor first generates a regression equation to define the relationship between the
dependent and independent variables. Second, it uses the time-series forecasting methods
to forecast the independent variables individually. Third, Predictor uses those forecasted
values to calculate the dependent variable forecast values using the regression equation. See
Figure 64.
The forecast cells of the independent variables are simple value cells. The forecast cells of
the dependent variable are formula cells containing the regression equation and using the
forecast values from the independent variables.
The average wage in December is used to calculate the total increase in her payroll. Labor
costs actually decrease –2%. With these results, Monica decides that labor costs over the
next six months do not justify a major equipment capital purchase.
123
Figure 64 Predicted Labor Cost Increases
124
l Regression — Forecasts results using past relationships between a variable of interest and
several other variables that may influence it. This works best for situations in which you
need to identify the different effects of different variables. This category includes multiple
linear regression.
l Simulation — Randomly generates many scenarios for a model to forecast the possible
outcomes. This method works best where you may not have historical data, but you can
build the model of the situation to analyze its behavior.
l Qualitative — Uses subjective judgment and expert opinion to forecast results. These
methods work best for situations for which no historical data or models are available.
Predictor uses time-series and multiple linear regression for forecasting. uses simulation. Each
technique and method has advantages and disadvantages for particular types of data, so often
you may forecast the data using several methods and then select the method that yields the best
results.
The following section, “Classic Time-series Forecasting” on page 125, describes forecast
methods available in Predictor and their uses.
These sections offer information about other data and forecast methodologies:
l “Multiple Linear Regression” on page 136
l “Historical Data Statistics” on page 140
This section provides more information about classic time-series forecasting methods used in
Predictor. Information about ARIMA (also known as Box-Jenkins) time-series forecasting
methods are found in “ARIMA Time-series Forecasting Formulas” on page 157.
Time-series forecasting assumes that historical data is a combination of a pattern and some
random error. Its goal is to isolate the pattern from the error by understanding the pattern’s
level, trend, and seasonality. You can then measure the error using a statistical measurement to
describe both how well a pattern reproduces historical data and to estimate how accurately it
projects the data into the future. See “Time-series Forecasting Accuracy Measures” on page
132.
By default, Predictor tries all of the classic time-series methods in the Methods Gallery. It then
ranks them according to which method has the lowest error, depending on the error measure
selected in the Options pane. The method with the lowest error is the best method.
Two primary techniques of classic time-series forecasting are used in Predictor:
l “Classic Nonseasonal Forecasting Methods” on page 126 — Estimate a trend by removing
extreme data and reducing data randomness
125
l “Classic Seasonal Forecasting Methods” on page 129 — Combine forecasting data with an
adjustment for seasonal behavior
For information about regression forecasting methods, see “Multiple Linear Regression” on page
136. See for more information about the formulas Predictor uses for the nonseasonal and
seasonal forecasting methods described in the following sections.
Nonseasonal methods attempt to forecast by removing extreme changes in past data where
repeating cycles of data values are not present. The listed classic nonseasonal forecasting methods
are available.
For information about associated parameters, see “Classic Nonseasonal Forecasting Method
Parameters” on page 128.
Figure 65 Typical Single Moving Average Data, Fit, and Forecast Line
126
Double Moving Average (DMA) Nonseasonal Method
Applies the moving average technique twice, once to the original data and then to the resulting
single moving average data. This method then uses both sets of smoothed data to project forward.
Predictor can automatically calculate the optimal number of periods to average, or you can select
the number of periods to average.
This method is best for historical data with a trend but no seasonality. It results in a straight,
sloped-line forecast.
Figure 66 Typical Double Moving Average Data, Fit, and Forecast Line
Figure 67 Typical Single Exponential Smoothing Data, Fit, and Forecast Line
127
This method is best for data with a trend but no seasonality. It results in a straight, sloped-line
forecast.
Figure 68 Typical Double Exponential Smoothing Data, Fit, and Forecast Line
Figure 69 Typical Damped Trend Smoothing Data, Fit, and Forecast Line
128
Damped trend smoothing has three parameters: alpha, beta, and phi (all between 0 and 1, not
inclusive).
Figure 70 Typical Seasonal Additive Data, Fit, and Forecast Curve without Trend
129
forecast. The seasonal adjustment is multiplied by the forecasted level, producing the seasonal
multiplicative forecast.
This method is best for data without trend but with seasonality that increases or decreases over
time. It results in a curved forecast that reproduces the seasonal changes in the data.
Figure 71 Typical Seasonal Multiplicative Data, Fit, and Forecast Curve without Trend
130
Figure 73 Typical Holt-Winters’ Multiplicative Data, Fit, and Forecast Curve
Figure 74 Typical Damped Trend Additive Data, Fit, and Forecast Curve
131
Figure 75 Typical Damped Trend Multiplicative Data, Fit, and Forecast Curve
Each seasonal forecasting method uses some or all of these parameters, depending on the
forecasting method. For example, the seasonal additive forecasting method does not account
for trend, so it does not use the beta parameter. The damped trend methods use phi in addition
to the other three.
of the formulas refer to the actual points ( ) and the fitted points ( ). In the chart, the horizontal
axis illustrates the time periods (t) and the vertical axis illustrates the data point values.
132
Figure 76 Sample Data
Predictor measures the error using one of the methods described in the following sections:
l “RMSE” on page 133, below
l “MAD” on page 133
l “MAPE” on page 133
Another statistic, “Theil’s U” on page 134, is used as a relative accuracy measure. Also see
“Durbin-Watson” on page 134.
RMSE
RMSE (root mean squared error) is an absolute error measure that squares the deviations to
keep the positive and negative deviations from cancelling out one another. This measure also
tends to exaggerate large errors, which can help eliminate methods with large errors.
MAD
MAD (mean absolute deviation) is an absolute error measure that originally became very
popular (in the days before hand-held calculators) because it did not require the calculation of
squares or square roots. While it is still fairly reliable and widely used, it is most accurate for
normally distributed data.
MAPE
MAPE (mean absolute percentage error) is a relative error measure that uses absolute values.
MAPE has two advantages. First, the absolute values keep the positive and negative errors from
cancelling out each other. Second, because relative errors do not depend on the scale of the
dependent variable, this measure lets you compare forecast accuracy between differently scaled
time-series data.
133
Theil’s U
Theil’s U statistic is a relative accuracy measure that compares the forecasted results with the
results of forecasting with minimal historical data. It also squares the deviations to give more
weight to large errors and to exaggerate errors, which can help eliminate methods with large
errors (Table 2).
Table 2 Interpreting Theil’s U
where Yt is the actual value of a point for a given time period t, n is the number of data points,
and
Durbin-Watson
Detects autocorrelation at lag 1. This means that each time-series value influences the next value.
This is the most common type of autocorrelation. For the formula, see “Durbin-Watson
Statistic” on page 156.
This statistic can have any value between 0 and 4. Values indicate slow-moving, none, or fast-
moving autocorrelation (Table 3).
Table 3 Interpreting the Durbin-Watson Statistic
Less than 1 The errors are positively correlated. An increase in one period follows an increase in the previous period.
2 No autocorrelation.
More than 3 The errors are negatively correlated. An increase in one period follows a decrease in the previous period.
Avoid using independent variables that have errors with a strong positive or negative correlation,
because this can lead to an incorrect forecast for the dependent variable.
134
Time-series Forecasting Techniques
Subtopics
l Standard Forecasting
l Simple Lead Forecasting
l Weighted Lead Forecasting
l Holdout Forecasting
Classic time-series forecasting methods can use any of these techniques, but ARIMA methods
can use only Standard and Holdout forecasting.
Standard Forecasting
Standard forecasting optimizes the forecasting parameters to minimize the error measure
between the fit values and the historical data for the same period. For example, consider showing
historical data and calculated fit values for periods 1 through 7.
1 472 488
2 599 609
3 714 702
4 892 888
5 874 890
6 896 909
7 890 870
Predictor calculates the RMSE using the differences between the historical data and the fit data
from the same periods. For example:
(472-488)2 + (599-609)2 + (714-702)2 + (892-888)2 + ...
For standard forecasting, Predictor optimizes the forecasting parameters so that the RMSE
calculated in this way is minimized.
135
Simple Lead Forecasting
Simple lead forecasting optimizes the forecasting parameters to minimize the error measure
between the historical data and the fit values, offset by a specified number of periods (lead). Use
this forecasting technique when a forecast for some future time period has the greatest
importance, more so than the forecasts for the previous or later periods. For example, single
lead forecasting can be used if your company must order extremely expensive manufacturing
components two months in advance, making any forecast for two months out the most
important.
Holdout Forecasting
Holdout forecasting:
1. Removes the last few data points of the historical data.
2. Calculates the fit and forecast points using the remaining historical data.
3. Compares the error between the forecasted points and their corresponding, excluded,
historical data points.
4. Changes the parameters to minimize the error between the forecasted points and the
excluded points.
Predictor determines the optimal forecast parameters using only the non-holdout set of data.
Notice that if you have a small amount of data and want to use seasonal forecasting methods,
using the holdout technique may restrict you to nonseasonal methods.
For more information on the holdout technique and when to use it effectively, see the
Makridakis, Wheelwright, and Hyndman reference in the Appendix A, “Bibliography.”.
136
The goal of multiple linear regression is to find an equation that most closely matches the
historical data. “Multiple” indicates that you can use more than one independent variable to
define the dependent variable in the regression equation. “Linear” indicates that the regression
equation is a linear equation.
The linear equation describes how the independent variables (x1, x2, x3,...) combine to define
the single dependent variable (y). Multiple linear regression finds the coefficients for the
equation:
y = b0 + b1x1 + b2x2 + b3x3 + ... + e
where b1, b2, and b3, are the coefficients of the independent variables, b0 is the y-intercept
constant, and e is the error.
Equations with only one independent variable define a straight line. This uses a special case of
multiple linear regression called simple linear regression, with the equation:
y = b0 + b1x + e
where b0 is where the regression line crosses the graph's y axis, x is the independent variable, and
e is the error. When the regression equation has only two independent variables, it defines a
plane. When the regression equation has more than two independent variables, it defines a
hyperplane.
To find the coefficients of these equations, Predictor uses singular value decomposition. For
more information on this technique, see “Regression Methods” on page 159.
Regression Methods
Predictor uses one of three methods for calculating multiple linear regression:
l “Standard Regression” on page 138
137
l “Forward Stepwise Regression” on page 138
l “Iterative Stepwise Regression” on page 138
Standard Regression
Standard regression performs multiple linear regression, generating regression coefficients for
each independent variable you specify, no matter how significant.
138
2. Adds the independent variable with the most significant correlation (partial F statistic).
3. Checks the partial F statistic of the independent variables in the regression equation to see
if any became insignificant (have a probability below the minimum) with the addition of
the latest independent variable.
4. Removes the least significant of any insignificant independent variables one at a time.
5. Repeats step 3 until no insignificant variables remain in the regression equation.
6. Repeats steps 1 through 5 until one of the following occurs:
l The model runs out of independent variables.
l The regression reaches one of the stopping criteria (see the for information on how the
stopping criteria work).
l The same independent variable is added and then removed.
Regression Statistics
After Predictor finds the regression equation, it calculates several statistics to help you evaluate
the regression:
l “R2” on page 139
l “Adjusted R2” on page 139
l “Sum of Squared Errors (SSE)” on page 140
l “F statistic” on page 140
l “t statistic” on page 140
l “p” on page 140
R2
Coefficient of determination. This statistic indicates the percentage of the variability of the
dependent variable that the regression equation explains.
For example, an R2 of 0.36 indicates that the regression equation accounts for 36% of the
variability of the dependent variable.
Adjusted R2
Corrects R2 to account for the degrees of freedom in the data. In other words, the more data
points you have, the more universal the regression equation is. However, if you have only the
same number of data points as variables, the R2 may be deceptively high. This statistic corrects
for that.
139
For example, the R2 for one equation may be very high, indicating that the equation accounted
for almost all the error in the data. However, this value may be inflated if the number of data
points was insufficient to calculate a universal regression equation.
F statistic
Tests the significance of the regression equation as measured by R2. A significant value means
that the regression equation accounts for some of the variability of the dependent variable.
t statistic
Tests the significance of the relationship between the coefficients of the dependent variable and
an individual independent variable, in the presence of the other independent variables. A
significant value means that the independent variable contributes to the dependent variable.
p
Indicates the probability of the calculated F or t statistic being as large as it is (or larger) by
chance. A low p value is good and means that the F statistic is not coincidental and, therefore,
is significant. A significant F statistic means that the relationship between the dependent variable
and the combination of independent variables is significant.
Generally, p should be less than 0.05.
If an ARIMA model is displayed in the Method box, the transformation lambda and three
selection criteria values are also included in the Statistics group. For more information about
140
these values, see “ARIMA Time-series Forecasting Formulas” on page 157and the Appendix A,
“Bibliography.”.
Mean
The mean of a set of values is found by adding the values and dividing their sum by the number
of values. “Average” usually refers to the mean. For example, 5.2 is the mean or average of 1, 3,
6, 7, and 9.
Standard Deviation
The standard deviation is the square root of the variance for a distribution. Like the variance, it
is a measure of dispersion about the mean and is useful for describing the “average” deviation.
For example, you can calculate the standard deviation of the values 1, 3, 6, 7, and 9 by finding
the square root of the variance that is calculated in the variance example below.
The standard deviation, denoted as s, is calculated from the variance as follows:
where the variance is a measure of the dispersion, or spread, of a set of values about the mean.
When values are close to the mean, the variance is small. When values are widely scattered about
the mean, the variance is larger.
Minimum
The minimum is the smallest value in the data range.
Maximum
The maximum is the largest value in the data range.
Ljung-Box Statistic
Measures whether a set of autocorrelations is significantly different from a set of autocorrelations
that are all zero. See for the formula.
Events Adjustment
The events feature of Predictor enables users to define events, identifiable occurrences that have
affected historical data and could affect predicted data. Users can define events for historical and
predicted data.
When forecasting from historical data that includes a defined event, the effects of the event are
removed from the data and then the forecasting methods are fit to the adjusted data. If there are
repeating occurrences of events in the forecasted period, the effects of the events are applied to
the forecasts obtained.
141
Notice that the adjusted data does not appear in the charts. Predictor shows the original data
and the fits are adjusted visually to reflect the effects of the events. Any manual adjustments to
the forecasts are performed after the effects of the events are applied to them.
For more information about events, see the Oracle Crystal Ball Predictor User's Guide.
Historical data can have missing values and outliers, which are data points that differ significantly
from the rest of the data. Settings in the Data Attributes panel of the Predictor wizard enable
you to select several ways of handling missing values and identifying and adjusting outliers.
Because adjusted outliers are treated as missing values, both of these situations are discussed and
handled together.
In each case, the difference is calculated between historical data points and values calculated by
the various forecasting methods. These differences are called residuals. They can be positive or
negative depending on whether the historical value is greater than or less than the smoothed
value. Various statistics are then calculated on the residuals and these are used to identify and
screen outliers.
A certain number of values must exist before the data fit can begin. If outliers occur at the
beginning of the data, they are not detected.
Note: Time-series data is typically treated differently from other data because of its dynamic
nature, such as the pattern in the data. A time-series outlier need not be extreme with
respect to the total range of the data variation but it is extreme relative to the variation
locally.
142
This method can fail to detect outliers because the outliers increase the standard deviation. The
more extreme the outlier, the more the standard deviation is affected.
Missing values at the beginning of a data series are ignored. Missing values at the end of a data
series are allowed, but this condition is not ideal. The cubic spline interpolation method is
especially sensitive to data missing at the end of the series. If one or two values are missing, cubic
spline interpolation can be used. If multiple values are missing, neighbor interpolation provides
a better estimate.
Tip: An obvious outlier, such as a large data spike, should be replaced by a blank cell in the
original data set. Otherwise, neighbor interpolation is probably a better adjustment method,
especially if the specified neighbors do not include the spike. Because cubic spline
interpolation takes into account the whole data set, that adjustment method will be affected
by the outlier.
143
Cubic Spline Interpolation Method
Cubic spline interpolation is based on a drafting tool used to draw smooth curves through a
number of points. The spline tool consists of weights attached to a flat surface at the points to
be connected. A flexible strip is then bent across each of these weights, resulting in a smooth
curve.
The cubic spline interpolation is a piecewise continuous curve that passes through each value
in the data set. Each interval of the curve has a separate cubic polynomial, each with its own
coefficients. These are equivalent to the spline tool's weights. The cubic spline interpolation
method considers the entire data set when adjusting outliers and filling in missing values.
Note: To preserve the local nature of time series data, the number of neighbors on each side of
the missing value or values should be small, ideally n = 1 or n = 2.
Related terms:
144
l Time series—The original data, expressed as Yt
l Fit array—A retrofit of the time series, consisting of one-period-ahead forecasts performed
from the data of previous periods; expressed as Ft
l Residual array—A set of positive or negative residuals, expressed as rt, and defined as rt = Yt
– Ft
l RMSE—Root mean square error for forecasting, calculated as described in “RMSE” on page
153, where n is the number of periods for which a fit is available. RMSE depends on the
specific forecasting method and technique.
l Forecasts—Value projections calculated using the formula for the specific method; they are
1 to k periods ahead, where k is the number of forecasts required; also known as predictions.
l Standard error of forecasts—Used to calculate prediction intervals; see “Prediction
Intervals” on page 154
Standard Forecasting
In standard forecasting, if the method parameters are already provided by the user, the following
are calculated: RMSE and other error measures, forecasts, and standard error. If the parameters
are not provided by the user, then the parameters are optimized to minimize the error measure
between the fit values and the historical data for the same period.
Holdout Forecasting
In holdout forecasting:
l The last few data points are removed from the data series. The remaining historical data
series is called in-sample data, and the holdout data is called out-of-sample data. Suppose
p periods have been removed as holdout from a total of N periods.
l Parameters are optimized by minimizing the fit error measure for in-sample data. If method
parameters are provided by the user, those are used in the final forecasting.
l After the parameters are optimized, the forecasts for the holdout periods (p periods) are
calculated.
l The error statistics (RMSE, MAD, MAPE) are out-of-sample statistics, based on only the
numbers in the hold-out period. The RMSE for holdout forecasting is often called holdout
RMSE. The holdout error measures are the ones reported to the user and are used to sort
the forecasting methods.
l Other statistics such as Theil's U, Durbin-Watson, and Ljung-Box are in-sample statistics,
based on the non-holdout period.
l Final forecasting is performed on both the in-sample and out-of-sample periods (all N
periods) using the standard technique.
l The standard error for the forecasts is also calculated using all N periods.
To improve the optimized parameter values obtained for the method, holdout forecasting should
be used only when there are at least 100 data points for nonseasonal methods and 5 seasons for
145
seasonal methods. For best results, use no more than 5 percent of the data points as holdout, no
matter how large the number of total data points.
If the method parameters are already provided by the user, simple lead forecasting is performed
as described previously. If the parameters are not provided, then the parameters are optimized
to minimize the lead error measure (for example, lead RMSE). After the parameters are
optimized, the fit and the forecast are then calculated as for standard forecasting method.
146
l Method parameters are then obtained by minimizing the weighted lead RMSE.
l After the parameters are obtained, forecasts for future periods and the standard errors for
those forecasts are calculated as for standard forecasts.
If method parameters are provided by the user, weighted lead forecasting is performed as
described previously. If the parameters are not provided, they are optimized to minimize the
weighted lead error measure, such as weighted lead RMSE.
After parameters are optimized, the fit and the forecast are then calculated as for the standard
forecasting method. For a lead value = 1, weighted lead forecasting is the same as simple lead
forecasting and standard forecasting.
For ARIMA formulas, see “ARIMA Time-series Forecasting Formulas” on page 157.
This section contains formulas for the listed classic nonseasonal time-series forecasting methods.
(Fit)
(Forecast for period m) Ft+m = Ft
where the parameters are:
p—Order of moving average
147
Note: First fit is available from period (p + 1)
(Trend)
(Fit) Ft = Lt-1 + Tt-1
(Forecast for period m) Ft+m = Lt + m*Tt
Where the parameters are:
p—Order of moving average
Mt—First order moving average for period t
Mt'—Second order moving average for period t
148
Damped Trend Smoothing Formula
Equations for the nonseasonal damped trend smoothing method are obtained by introducing a
damp parameter (Φ) to the standard Holt formula for exponential smoothing of a linear trend,
or
Where is the level at time t, is the trend at time t, and is the m-period ahead
forecast at origin t. α and γ are the level and trend parameters. When Φ = 1, the method is
equivalent to the standard version of the Holt model with a linear trend.
This section contains formulas for the listed classic seasonal time-series forecasting methods.
Set Lt = P, St = Yt – P for t = 1 to s
Crystal Ball uses the following equations to calculate this method:
149
γ—Gamma
m—Number of periods ahead to forecast
s—Length of seasonality
Lt—Level of the series at time t
St—Seasonal component at time t
150
P=
Set: Lt = P, bt = 0, St = Yt – P, for t = 1 to s
For the remaining periods, use the following formulas:
Calculate: P =
Set: Lt = P, bt = 0, St = Yt/P, for t = 1 to s
For the remaining periods, use the following formulas:
151
α—Alpha
β—Beta
γ—Gamma
m—Number of periods ahead to forecast
s—Length of the seasonality
Lt—Level of the series at time t
bt—Trend of the series at time t
St—Seasonal component at time t
Where is the level at time t, is the trend at time t, is the seasonal component at time t,
and is the m period ahead forecast at origin t.α,γ, and δ are the level, trend, and
seasonal parameters. When Φ = 1, the method is the equivalent to the standard version of Holt-
Winters' seasonal additive model with a linear trend.
Where is the level at time t, is the trend at time t, is the seasonal component at time t,
and is the m period ahead forecast at origin t.α,γ, and δ are the level, trend, and
seasonal parameters. When Φ = 1, the method is the equivalent to the standard version of Holt-
Winters' seasonal multiplicative model with a linear trend.
152
Error Measure and Statistic Formulas
This section provides formulas for the following types of statistics used in Predictor:
l “Time-Series Forecast Error Measures” on page 153
l “Prediction Intervals” on page 154
l “Autocorrelation Statistics” on page 155
RMSE
Root mean squared error is an absolute error measure that squares the deviations to keep the
positive and negative deviations from canceling one another out. This measure also tends to
exaggerate large errors, which can help when comparing methods.
The formula for calculating RMSE:
where Yt is the actual value of a point for a given time period t, n is the total number of fitted
points, and
MAD
Mean absolute deviation is an error statistic that averages the distance between each pair of actual
and fitted data points.
The formula for calculating the MAD:
153
where Yt is the actual value of a point for a given time period t, n is the total number of fitted
points, and
MAPE
Mean absolute percentage error is a relative error measure that uses absolute values to keep the
positive and negative errors from canceling one another out and uses relative errors to enable
you to compare forecast accuracy between time-series models.
The formula for calculating the MAPE:
where Yt is the actual value of a point for a given time period t, n is the total number of fitted
points, and
Prediction Intervals
The prediction interval defines the range within which a predicted value has some probability
of occurring. Predictor uses an empirical method of calculating prediction intervals, using the
standard error of forecasts:
l For an m-period-ahead forecast, the error term rt(m) is defined as Yt – Ft(m), where Ft(m)
is the m-period-ahead fit for period t.
l The standard error of forecast for an m-period-ahead forecast is then expressed as
Assuming that forecast errors are normally distributed, the formula for predicting the future
value of
The empirical method is reasonably accurate when historical data amount is sufficiently large.
154
Autocorrelation Statistics
Measures of autocorrelation describe the relationship among values of the same data series at
different time periods.
The number of autocorrelations calculated is equal to the effective length of the time series
divided by 2, where the effective length of a time series is the number of data points in the series
without the pre-data gaps. The number of autocorrelations calculated ranges between a
minimum of 2 and a maximum of 400.
Autocorrelation formula:
Autocorrelation Probability
Autocorrelation probability is the probability of obtaining a certain autocorrelation for a
particular data series by chance alone, if the data were completely random. To calculate
autocorrelation probability:
l Calculate the standard error of autocorrelation:
where
SE(rk) = standard error of autocorrelation at lag k
ri = autocorrelation at lag i
k = the time lag
n = number of observations in the time series
Reference: Hanke et al. Business Forecasting. 7th ed. Prentice Hall, 2001. Chapter 3, pg 59–
60
l Calculate the t statistic:
155
l Calculate the p-value from the absolute t statistic; the probability is double the area of (1 –
CDF(t))
Durbin-Watson Statistic
The Durbin-Watson statistic calculates autocorrelation at lag 1.
The formula for calculating the Durbin-Watson statistic:
and the actual point (Yi) and n is the number of data points.
Ljung-Box Statistic
This statistic measures whether a set of autocorrelations is significantly different from a set of
autocorrelations that are all zero. The formula for calculating the Ljung-Box statistic:
where:
Q’ is the Ljung-Box statistic; the probability that the set of autocorrelations is the same as a set
of autocorrelations which are all 0.
n is the amount of data in the data sample.
h is the size of the set of autocorrelations used to calculate the statistic.
rk is the autocorrelation with a lag of k.
The size of the set of autocorrelations is equal to one-third the size of the data sample (or 100,
if the sample is greater than 300).
156
ARIMA Time-series Forecasting Formulas
Subtopics
l ARIMA Equations
l Estimation of ARIMA Model Coefficients
l ARIMA Constants
l Stationarity
This topic provides basic formulas for the ARIMA (autoregressive integrated moving average)
model implementation used in Predictor. For more information, see the references in the
ARIMA section of Appendix A, “Bibliography.”.
For classic time-series forecasting formulas, see “Classic Time-series Forecasting Method
Formulas” on page 147.
ARIMA Equations
l Equation for a p-th order autoregressive (AR) model — that is, AR(p) model:
Where {yt} is the data on which the ARMA model is to be applied. That means, the series is
already power-transformed and differenced, in that order. The parameters φ1, φ2 and so
on are AR coefficients.
l Equation for a q-th order moving average (MA) model — that is, MA(q) model:
Where {yt}, {φ}, and {θ} are as defined previously, and {Φ} and {Θ} are the seasonal
counterparts.
157
E. P., Jenkins, G. M., and Reinsel, G. C. Time Series Analysis: Forecasting and Control. 4th ed.
Hoboken, NJ: John Wiley & Sons. 2008.).
ARIMA Constants
The constant term in an ARIMA equation introduces deterministic trend into the model and
extends that trend indefinitely into the future. If the model includes a single difference (either
nonseasonal or seasonal) and the constant term is present, the trend is linear, where a twice-
differenced model has quadratic trend. The AutoSelect setting for ARIMA constants in the
Predictor ARIMA Options dialog leaves out the constant term in the model if it contains one or
more nonseasonal or seasonal differences.
When the constant term is included in the model, the value of the term is calculated by the
following equation:
Where φi are nonseasonal AR coefficients, Φi are seasonal AR coefficients, and μ is the mean
of the series.
Stationarity
ARIMA time-series forecasting assumes that the time series mean, variance, and autocorrelation
are stationary over time. This characteristic is called stationarity. If a time series statistic has
nonstationarity, it must be adjusted:
l Nonstationarity in the mean—In this case, the mean is not constant but drifts slowly. This
can be true for both seasonal and nonseasonal series and is removed by differencing the
series. The automatic ARIMA implementation of Predictor determines the amount of
nonseasonal differencing required to make a series stationary by using repeated KPSS
(Kwiatkowski-Phillips-Schmidt-Shin) tests with appropriate alpha values. For seasonal
series, repeated Canova-Hansen tests with appropriate alpha values are used.
l Nonstationarity in variance—In this case, the time series is heteroscedastic; the variance of
the data around the mean changes over time. This nonstationarity in variance is removed
by applying the Box-Cox transformation, a special type of power transformation:
, if lambda equals 0
Where the original series is {xt}, the transformed series is {zt}, and the power transformation
constant is lambda (λ).
Predictor determines a suitable value of lambda with an algorithm that uses the seasonality
information to divide the dataset into groups, and then tries to fina a lambda value that
renders the variance stationary across groups.
158
For users who want more control over the Box-Cox transformation, Predictor provides
commonly used power-transformation options, such as log transformation (lambda = 0)
or square-root transformation (lambda = 0.5), and even a custom transformation with a
user-selected lambda between –5 and +5 (inclusive). However, Predictor prevents the use
of custom lambda values that would result in transformed values being too large or too
small.
Regression Methods
Predictor supports two types of multiple linear regression, standard and stepwise (forward and
iterative). Some rules:
l Only standard forecasting is used for independent variables.
l Lags can be specified for each independent variable. They must be less than the effective
length of the series (not including pre-data gaps).
l The number of historical data points must be greater than or equal to the number of
independent variables, counting the included constant.
For details:
l “Calculating Standard Regression” on page 159
l “Calculating Stepwise Regression” on page 160
159
or surfaces. In these cases, the least squares technique returns no solution for the singular case
and extremely large parameters for the close-to-singular case.
Crystal Ball uses the matrix technique for singular value decomposition. Starting with:
y = bX
Following SVD, X can be rewritten:
X = [U][w][V]
where U, w, and V are the factor matrices. The matrix w, a square matrix of dimension (m+1)
by (m+1), is a diagonal matrix with the singular values (or eigenvalues). U and V are other factor
matrices..
The coefficients can then be calculated. For example, the b matrix is b = [V][w]-1[UT][y]
This can be written in matrix format as Y = bX + ∊, where Y and b are column vectors of
dimension n by 1 and X is a matrix of the dimension n by m, where n is the number of observations
and m is the number of independent variables. It is assumed that n > m.
Here, too, Predictor uses singular value decomposition (SVD) to determine b, the coefficients
of the regression equation. The only difference between this case and regression with a constant
is the dimension of the matrices.
For related regression statistics, see “Statistics, Standard Regression without Constant” on page
163.
160
l “Statistics, Standard Regression without Constant” on page 163
l “Partial F Statistic, Stepwise Regression” on page 165
The F statistic follows an F distribution with (m, n – m – 1) degrees of freedom. This information
is used to calculate the p-value of the F statistic.
161
Adjusted R2, Regression with Constant
You can calculate a regression equation by using the same number of data points as you have
equation coefficients. However, the regression equation will not be as universal as a regression
equation calculated using three times the number of data points as equation coefficients.
To correct the R2 for such situations, an adjusted R2 takes into account the degrees of freedom
of an equation. When you suspect that an R2 is higher than it should be, calculate the R2 and
adjusted R2. If the R2 and the adjusted R2 are close, then the R2 is probably accurate. If R2 is
much higher than the adjusted R2, you probably do not have enough data points to calculate
the regression accurately.
The formula for adjusted R2:
Adjusted R2 =
where n is the number of data points and m is the number of independent variables.
where n is the number of data points you have and m is the number of independent variables.
162
Coefficient
The coefficient of interest is expressed as bp, the pth component in the b vector.
where S is the standard error of estimate (SSE) and cpp is the diagonal element at (p,p) of the
matrix (XTX)-1.
t
If the F statistic in ANOVA and the corresponding p indicate a significant relationship between
the dependent and the independent variables as a whole, you then want to see the significance
of the relationship of the dependent variable to each independent variable. The t statistic tests
for the significance of the specified independent variable in the presence of the other independent
variables.
The formula for the t statistic:
where bp is the coefficient to check and se(bp) is the standard error of the coefficient.
p
The t statistic (“t” on page 163) follows a t distribution with (n – m – 1) degrees of freedom.
ANOVA, No Constant
ANOVA (analysis of variance) statistics for standard regression without a constant:
163
Table 6 ANOVA Statistics, Standard Regression without a Constant
The F statistic follows an F distribution with (m, n – m) degrees of freedom. This information
is used to calculate the p-value of the F statistic.
R2, No Constant
R2 is the coefficient of determination. This statistic represents the proportion of error for which
the regression accounts.
You can use many methods to calculate R2. Predictor uses the equation:
R2 can be extremely large in cases when the regression constant is omitted, even when the
correlation between Y and X is weak. Because it can be meaningless, many applications do not
mention this statistic. Predictor provides this statistic but it is not used for stepwise regression
when there is no regression constant.
where n is the number of data points and m is the number of independent variables.
Like R2 for regression without a constant, this is also a very large number without much meaning.
SSE, No Constant
SSE (standard error of measurement) is a measure of the amount the actual values differ from
the fitted values. The formula for SSE:
164
where n is the number of data points you have and m is the number of independent variables.
F, No Constant
The F statistic checks the significance of the relationship between the dependent variable and
the particular combination of independent variables in the regression equation. The F statistic
is based on the scale of the Y values, so analyze this statistic in combination with the p–value
(described in the next section). When comparing the F statistics for similar sets of data with the
same scale, the higher F statistic is better.
The formula for the F statistic is given in Table 5 on page 161.
PFt follows the F distribution with degrees of freedom equal to (1, Error DF at step t). Users
provide a maximum p-value, below which the variable is added to the regression.
For deletion of a variable, the partial F statistic for step t, (PFt):
PFt follows the F distribution with degrees of freedom equal to (1, Error DF at step t). Users
provide a maximum p-value, above which the variable is removed from the regression.
165
166
OptQuest Examples and
7 Reference
In This Chapter
OptQuest Examples ...................................................................................... 167
Optimization Tips and Notes ............................................................................ 216
OptQuest Examples
Subtopics
l Opening Example Models
l Recommended Run Preference Settings for Optimizations
l Product Mix
l Hotel Design and Pricing Problem
l Budget-constrained Project Selection
l Groundwater Cleanup
l Oil Field Development
l Portfolio Revisited
l Tolerance Analysis
l Inventory System Optimization
l Drill Bit Replacement Policy
l Gasoline Supply Chain
For those with Crystal Ball Decision Optimizer, this section presents a variety of examples using
OptQuest. These examples illustrate how to use spreadsheets to model optimization problems,
the key features of OptQuest, and the variety of applications for which you can use OptQuest.
To open example models, see “Opening Example Models” on page 168. Also see “Recommended
Run Preference Settings for Optimizations” on page 168.
Table 7, following, summarizes the examples in this chapter and the features illustrated.
Decision
Application Variables Type Constraints Requirements Illustrated Methods
167
Decision
Application Variables Type Constraints Requirements Illustrated Methods
Note: Most of the examples included here use one of the Advanced Options settings for
automatically stopping the optimization when either a solution confidence level or certain
number of non-improving solutions is reached. If you follow along with these examples,
your results should be similar but may not always be identical.
168
Tail-end percentiles and maximum and minimum range values generally require at least
2000 trials.
l Sampling tab — Sampling method set to Latin Hypercube. Latin Hypercube sampling
increases the quality of the solutions, especially the accuracy of the mean statistic.
l Sampling tab — Random Number Generation set to Use Same Sequence Of Random
Numbers with an Initial Seed Value of 999. The initial seed value determines the first number
in the sequence of random numbers generated for the assumption cells. This enables you
repeat simulations using the same set of random numbers to accurately compare the
simulation results. If you do not set an initial seed value, OptQuest will automatically pick
a random seed and use that starting seed for each simulation that is run.
Note: When a Crystal Ball forecast has extreme outliers, run the optimization with several
different seed values to test the solution’s stability.
After you define the assumptions, decision variables, and forecasts in Crystal Ball, you can begin
the optimization process in OptQuest.
Product Mix
Subtopics
l Product Mix Problem Statement
l Product Mix Spreadsheet Model
l Product Mix OptQuest Solution
The listed sections describe this problem and its OptQuest solution.
169
Limited amounts of ingredients are available for the next production cycle. Specifically, only
12,520 pounds of veal, 14,100 pounds of pork, 6,480 pounds of beef, and 10,800 pounds of casing
are available.
Complicating this situation is:
l The unit profits are only estimates because all customer contracts have not been finalized.
l The amount of casing used per unit may be more than anticipated because of production
losses due to tearing or partial rejections during inspection.
The problem is to determine how many pounds of each product to produce in order to maximize
the 5th percentile of operating profit without running out of meat ingredients or casing during
the manufacturing run, or violating any of the other limits.
170
Figure 78 Product Mix Problem Spreadsheet Model
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
171
Figure 79 shows the OptQuest solution. The optimal 5th percentile of operating profit is
$16,759.81, obtained by producing 1,039 pounds of bratwurst, 715 pounds of Italian sausage,
877 pounds of pepperoni, 1,377 pounds of Polish sausage, and 985 pounds of summer sausage.
172
Hotel Design and Pricing Problem
Subtopics
l Hotel Design Problem Statement
l Hotel Design Spreadsheet Model
l Hotel Design OptQuest Solution
A downtown hotel is considering a major remodeling effort and needs to determine the best
combination of rates and room sizes to maximize revenues.
Each market segment has its own price/demand elasticity. Estimates are:
Standard -3
Gold -1
Platinum -2
This means, for example, that a 1% decrease in the price of a standard room will increase the
number of rooms sold by 3%. Similarly, a 1% increase in the price will decrease the number of
rooms sold by 3%. For any proposed set of prices, the projected number of rooms of a given
type sold can be found using the formula:
Variable Description
E Elasticity
N New price
173
Variable Description
C Current price
The hotel owners want to keep the price of a standard room between $70 and $90, a gold room
between $90 and $110, and a platinum room between $120 and $149. All prices are in whole
dollar increments (discrete). Although the rooms may be renovated and reconfigured, there are
no plans to expand beyond the current 450-room capacity.
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
174
a requirement added in the Objectives panel. Specifically, the total room demand is
limited by a requirement using the forecast statistic Percentile (80), with an upper bound
of 450.
l This problem has three decision variables and no constraints.
2 On the Options panel, click Advanced Options and select Automatically stop after 500 non-improving
solutions.
3 Run the optimization.
The results are displayed in Figure 81. The mean of total revenue is $40,447.14 and room prices
are $108 for Gold, $120 for Platinum, and $81.00 for Standard.
The Crystal Ball simulation of this solution in verifies that the chance of demand exceeding
capacity is just slightly less than 20% (100% – 82.06%).
175
Figure 82 Hotel Pricing Solution (Percentiles View)
Using these figures, the finance manager has computed the expected return and the expected
profit for each project as shown in the following table.
Project Expected Revenue Success Rate Expected Return Initial Investment Expected Profit
176
Project Expected Revenue Success Rate Expected Return Initial Investment Expected Profit
Unfortunately, the available budget is only $2.0 million, and selecting all projects would require
a total initial investment of $2.8 million. Thus, the problem is to determine which projects to
select to maximize the total expected profit while staying within the budget limitation.
Complicating this decision is the fact that both the expected revenue and success rates are highly
uncertain.
177
Figure 83 Project Selection Problem Spreadsheet Model
The expected revenue and success rates are assumption cells in the Crystal Ball model. The
expected revenues have various distributions, while the success rates are modeled using a
binomial distribution with one trial. During the simulation, the outcomes in column D will be
either 0% or 100% (not successful or successful) with the probabilities initially specified. For
each simulated trial, the expected returns will either equal the expected revenue generated in
column C or zero. So, the expected profits can be positive or negative.
Although good solutions may be identified by inspection or by trial and error, basing a decision
on expected values can be dangerous because it doesn’t assess the risks. In reality, selecting R&D
projects is a one-time decision; each project will be either successful or not. If a project is not
successful, the company runs the risk of incurring the loss of the initial investment. Thus,
incorporating risk analysis within the context of the optimization is a very useful approach.
ä With Project Selection.xlsx open in Crystal Ball, start OptQuest from the Crystal Ball Run
menu. Then:
1 To obtain similar results, confirm that Run Preferences settings are as described in “Recommended Run
Preference Settings for Optimizations” on page 168.
2 Start the OptQuest wizard.
3 On the Objectives panel, set the objective to Maximize the Final Value of Total profit. Notice that there
are no requirements.
4 Click Next to step through the problem and notice the following:
l There are eight decision variables.
178
l One constraint represents the budget limitations. Notice the use of Microsoft Excel’s
SUMPRODUCT function in the constraint to create a linear combination of the decision
variables and investment amounts.
5 On the Options panel, click Advanced Options and select Automatically stop after 500 non-improving
solutions.
6 Run the optimization.
Figure 84 shows the results of an OptQuest optimization. The best solution selects projects 2, 4,
and 5.
Figure 85, the forecast chart for Total Profit, shows that the distribution of profits is highly
irregular, and depends on the joint success rate of the chosen projects. A risk of realizing a loss
exists. You may want to evaluate the risks associated with some of the other solutions identified
during the search.
179
Figure 85 Project Selection Solution Forecast Chart
Groundwater Cleanup
Subtopics
l Groundwater Cleanup Problem Statement
l Groundwater Cleanup Spreadsheet Model
l Groundwater Cleanup OptQuest Solution
180
l You have estimates of the contamination levels of the various chemicals. Each contaminant’s
concentration in the water is measured in micrograms per liter.
l The cancer potency factor (CPF) for each chemical is uncertain. The CPF is the magnitude
of the impact the chemical exhibits on humans; the higher the cancer potency factor, the
more harmful the chemical is.
l The population risk assessment must account for the variability of body weights and volume
of water consumed by the individuals in the community per day.
All these factors lead to the following equation for population risk:
This model shows the population risk (cell C25), which is the overall contamination risk to the
people in the community as a function of the factors shown in Table 11, following:
181
Table 11 Groundwater Cleanup Population Risk Factors
Volume Of Water Per Day C23 Interindividual variability of volume of water consumed each Normal, with lower bound of 0.
day.
Body Weight C22 Interindividual variability of body weights in the community. Normal, with lower bound of 0.
Remediation costs of the various cleanup methods (cells E8:E10) are a function of factors shown
in Table 12, following.
Fixed Costs C8:C10 Flat costs for each method to pay for initial setup. Triangular
Variable Costs D8:D10 Costs for each method based on how long the cleanup takes. Uniform
Efficiency D14 Percent of contaminants that the cleanup process removes. Each remediation None
method has a different cost for different efficiency levels.
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
182
4 On the Options panel, click Advanced Options and select Automatically stop after 500 non-improving
solutions.
5 Run the optimization.
The results are shown in Figure 87, following. The solution in minimizes costs at $10,924 while
keeping the risk level at 1.00E-04, rounded.
The distributions for the total remediation cost and the population risk are shown in and
Figure 89.
183
Figure 88 Groundwater Cleanup Total Remediation Cost Forecast Chart
This example concerns an oil company analysis of Net Present Value for a new asset.
184
actual data available, the discovery team wants to quantify and optimize the Net Present Value
(NPV) of this asset. You can simplify this analysis by representing the production profile by
three phases, shown in Table 13.
Phase Description
Build up The period when you drill wells to gain enough production to fill the facilities.
Plateau After reaching the wanted production rate (plateau), the period when you continue production at that rate as long as the reservoir
pressure is constant and until you produce a certain fraction of the reserves. In the early stages of development, you can only
estimate this fraction, and production greater than a certain rate influences plateau duration.
Decline The period when production rates, P, decline by the same proportion in each time step, leading to an exponential function:
P(t) = P(0) exp(-c*t)
where t is the time since the plateau phase ended and c is some constant.
With only estimates for the total Stock Tank Oil Initially In Place (STOIIP = reserve size) and
percent recovery amounts, the objective is to select a production rate, a facility size, and well
numbers to maximize some financial measure. In this example the measure used is the 10th
percentile (P90) of the NPV distribution. In other words the oil company wants to optimize an
NPV value which they are 90% confident of achieving or exceeding.
As described, the problem is neither trivial nor overly complex. A high plateau rate does not lose
any reserves, but it does increase costs with extra wells and larger facilities. However, facility
costs per unit decrease with a larger throughput, so selecting the largest allowed rate and selecting
a facility and number of wells to match may be appropriate.
185
Figure 90 Oil Field Development Problem Spreadsheet Model
Net present value (cell C30) of this oil field is based on:
l Total discounted reserves (cell C27)
l Oil margin (cell C13), which is equivalent to oil price minus operating costs
l Well costs (cell C28)
l Facilities cost (cell C29), which is determined for various production levels by a look-up
table
Facility capacity places a maximum limit on production rate, while the production rate of the
wells is defined as a normal distribution (cell C7).
The Production Profile table at the bottom of the model shows that the production phase
determines annual production rates. Cumulative oil production is calculated per year and is
then discounted at 10% (lognormal distribution in cell C10), resulting in a total discounted
reserves value. The model gives an oil (or profit) margin of $2.00 per barrel (bbl) and converts
total discounted reserves to present value dollars. Total well and facilities costs are then
subtracted for total project NPV.
186
Oil Field Development OptQuest Solution
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
ä Be sure Oil Field Development.xlsx is open in Crystal Ball Decision Optimizer. Then:
1 Start the OptQuest wizard.
As you click Next to step through the problem, notice:
l The objective is to maximize the 10th percentile (P90) of the NPV.
l There are no requirements.
l There are three decision variables: Wells to drill (cell C8), Facility size (cell C12), and
Plateau rate (cell C15).
l This problem has no constraints.
2 On the Options panel, click Advanced Options and select Automatically stop after 500 non-improving
solutions.
3 Run the optimization.
The results are shown in Figure 91. The 10th percentile of NPV is maximized at 199.93 with
a facility size of 100, a plateau rate of 14.5, and 13 wells to drill.
187
Figure 91 Oil Field Development Optimization Results
The Crystal Ball simulation of this solution in maximizes the 10th percentile (P90) of the NPV
with the same result shown in the OptQuest solution window..
188
Portfolio Revisited
Subtopics
l Portfolio Revisited Problem Statement
l Portfolio Revisited Method 1: Efficient Frontier Optimization
l Portfolio Revisited Method 2: Multiobjective Optimization
l Method 3 — Arbitrage Pricing Theory
This example concerns analysis of an investment portfolio with respect to risk as well as return.
The listed sections describe this problem and several ways to solve it using OptQuest.
When the investor maximized the portfolio return without regard to risk, OptQuest allocated
almost all the money to the investment with the highest return. This strategy did not result in a
portfolio that maintained risk at a manageable level. Only limiting the standard deviation of the
total expected return generated a more diversified portfolio. For more information about this
result, see the Oracle Crystal Ball Decision Optimizer OptQuest User's Guide.
OptQuest has a feature that creates an efficient frontier for you automatically. To use the Efficient
Frontier function in OptQuest, you need only define a requirement with a variable upper or
lower bound. OptQuest then calculates points within the variable requirement range.
The listed sections describe the model for this solution method and its OptQuest solution.
189
Efficient Frontier Spreadsheet Model
Open the Portfolio Revisited EF.xlsx workbook found in the Crystal Ball Examples folder. The
total expected return forecast, assumptions, and decision variables are the same as in the original
model, with the decision variables already defined.
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
The requirement has a variable upper bound for the standard deviation statistic (less than
or equal to $8,000).
190
The variable requirement bounds are $8,000 for the lower bound and $10,000 for the upper
bound in steps of $250.
3 Run the optimization for 2000 simulations (set in the Options panel of the OptQuest wizard). Run
Preferences settings are as described in “Recommended Run Preference Settings for Optimizations”
on page 168.
The results are shown in Figure 94. The mean of total expected return is maximized at $8,684
with fund allocations as follows: Aggressive Growth fund = $50,600; Growth and Income
fund = $32,400; Income fund = $17,000; and Money Market fund = $0.
When should you use the Efficient Frontier function? This method is useful when it is difficult
to determine reasonable lower or upper bounds for requirement statistics.
191
l Public health policies, requiring simultaneous minimization of risks to the population,
direct taxpayer costs, and indirect business regulation costs
l Electric power generation, requiring simultaneous optimization of operating costs,
reliability, and pollution control
192
Figure 95 Portfolio Revisited Spreadsheet Model
This new function (cell C22) contains the multiobjective relationship described by mean return
– (k * standard deviation) with the risk aversion constant (cell C19) broken out into a separate
cell. The mean return and standard deviation variables in this equation are automatically
extracted at the end of the simulation from the Total Expected Return forecast (cell C17). See
the Oracle Crystal Ball User's Guide for more information on the Auto Extract feature.
Note: Except where indicated, this example uses the recommended Crystal Ball run
preferences. See “Recommended Run Preference Settings for Optimizations” on page
168.
2. Start the OptQuest wizard.
As you click Next to step through the problem, notice:
l The objective refers to the new multiobjective function: Maximize the Final Value of Mean
minus stdev. The statistic to optimize is Final Value, to calculate only the statistical values
for the total expected return forecast at the end of the simulation. There are no
requirements.
l The decision variables and constraints are the same as previous Portfolio Allocation
examples.
193
3. On the Options panel, select Run for 2000 simulations, and then click Advanced Options and
select Automatically stop after 500 non-improving solutions.
4. Run the optimization for 2000 simulations.
The results are displayed in Figure 96. Mean minus std dev is maximized at $3,100.44 and
fund values are as follows:
l Aggressive Growth = $17,700
l Growth and Income = $7,300
l Income = $25,000
l Money Market = $50,000
After reviewing the results, close Portfolio Revisited.xlsx without saving it.
194
Method 3 — Arbitrage Pricing Theory
Subtopics
l Spreadsheet Model
l OptQuest Solution
A different approach to incorporating risk in a decision model is called Arbitrage Pricing Theory
(APT). APT does not ask whether portfolios are efficient. Instead, it assumes that a stock or
mutual fund's return is based partly on macroeconomic influences and partly on events unique
to the underlying company or assets (see Brealey and Myers, 1991, listed in the Appendix A,
“Bibliography.”). Further, this theory only considers macroeconomic influences, since
diversification, as in a portfolio, practically eliminates unique risk. Some macroeconomic
influences might include:
l The level of industrial activity
l The spread between short- and long-term interest rates
l The spread between low- and high-risk bond yields (see Chen et al. listed in the Appendix A,
“Bibliography.”)
A weighted sum of these influences determines the risk factor of an asset. APT provides estimates
of the risk factors for particular assets to these types of influences. Higher risk factors indicate
greater risk; lower factors indicate less risk. Assume that the risk factors per dollar allocated to
each asset are as shown in Table 15, “Sample Asset Risk Factors,” on page 195.
Using this method, the investor can specify a target level for the weighted (or aggregate) risk
factors, leading to a constraint that limits the overall risk. For example, suppose that the investor
can tolerate a weighted risk per dollar invested of at most 1.0. Anything above 1.0 is too risky
for the investor. Thus, the weighted risk for a $100,000 total investment must be at or below
100,000. If the investor distributed $100,000 equally among the four available assets, the return
would be:
0.03($25,000) + 0.05($25,000) + 0.07($25,000) + 0.11($25,000)= $7,000
And the total weighted risk would be:
-0.3($25,000) - 0.5(25,000) + 0.4(25,000) + 2.1(25,000) = $42,500
If this amount were greater than the limit of 100,000, this solution would not be feasible and
could not be chosen.
195
Spreadsheet Model
Open the Portfolio Revisited.xlsx worksheet found in the Crystal Ball Examples folder (Figure 95
on page 193). The total expected return forecast, assumptions, decision variables, and the
original constraint limiting the total investment to $100,000 are the same as in the original model.
The new item is a constraint limiting the total weighted risk (cell F13), calculated by:
OptQuest Solution
Note: This example uses run preferences recommended in “Recommended Run Preference
Settings for Optimizations” on page 168.
2 Start OptQuest.
3 When the Objectives panel opens, change the objective so it is the same as in Tutorial 2 in the Oracle
Decision Optimizer OptQuest User's Guide.
The objective should be set to Maximize the Mean of Total Expected Return.
There are no requirements. Click Next.
4 You do not need to change the decision variables. Click Next.
5 In the Constraints panel, add a new constraint that references cell F13, the total weighted risk. Note
that the formula in cell F13 uses the Excel SUMPRODUCT function to create a linear combination of the
risk factors and the decision variables. Limit this risk to $100,000 as shown inFigure 97 on page
196.
To do this:
l Click the blank row below the first constraint.
l Click Insert Reference.
l Point to F13 and click OK.
l Type =100000 after the cell reference.
196
6 To document the cell reference, select the second constraint and click Add Comment. In the Add
Comment dialog, type Cell F13 = the sum product of risk factors and decision
variables. When you click OK, the comment appears above the second constraint.
7 Click Next. In the Options panel, set the optimization to run for 1000 simulations.
8 Run the optimization.
The Crystal Ball simulation of this solution maximizes the mean of the total expected return at
$8,445 with the new constraint. Compare this to the original total expected return of $7,902
described in the Oracle Crystal Ball Decision Optimizer OptQuest User's Guide and originally
calculated using the different method of limiting risk with the standard deviation.
197
Tolerance Analysis
Subtopics
l Tolerance Analysis Problem Statement
l Tolerance Analysis Spreadsheet Model
l Tolerance Analysis OptQuest Solution
An engineer at an automobile design center needs to specify components for piston and cylinder
assemblies that work well together. To do this, he needs the dimensions of the components to
be within certain tolerance limits, while still selecting the most cost-efficient methods. This is
called an optimal stack tolerance analysis.
Note: This example involves concepts used only by Six Sigma and similar quality programs. If
you are not familiar with Crystal Ball’s process capability features, consider reviewing the
process capability appendix in the Oracle Crystal Ball User's Guide.
198
Tolerance Analysis Spreadsheet Model
Open the Tolerance Analysis.xlsx file (Figure 99).
The dimensions of the assemblies are a cumulation of their respective components’ statistical
dimensions. The difference in length between the cylinder assembly (cell C5) and the piston
assembly (cell C4) is the assembly gap (cell C6).
199
Component cost (cells F14:F18 and F23:F24) is a nonlinear function of quality specification.
The higher the specification, the higher the cost. Also notice that each component has a different
cost function associated with it.
In addition to the recommended options, before running OptQuest, in Crystal Ball select Run
Preferences and set:
l The maximum number of trials run to 2000
l The sampling method to Latin Hypercube
l The sample size to 500 for Latin Hypercube
On the Statistics tab of the Run Preferences dialog, select Calculate capability metrics.
Since the model is heavily dependent on the tails of the forecast distribution, these settings will
provide higher accuracy and will be adequate for this example. In actual practice, to gain better
accuracy, the engineer may want to run longer simulations of 5000 or 10,000 trials.
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
200
The cost and quality solution values are displayed in Figure 100. The assembly gap Zst-total is
3.93 with a total assembly cost of $60.00. Decision variable values range from 2.6 to 4.3 sigmas.
Figure 100 OptQuest Solution for Maximum Quality with a Cost Requirement
This example is adapted from James R. Evans and David L. Olson, Introduction to Simulation
and Risk Analysis. New York: Prentice-Hall, 1998.
201
l How much additional inventory to order or produce
l When to order or produce it
Although it is possible to consider these two decisions separately, they are so closely related that
a simultaneous solution is usually necessary. Typically, the objective is to minimize total
inventory costs. Total inventory costs typically include holding, ordering, shortage, and
purchasing costs.
In a continuous review system, managers continuously monitor the inventory position.
Whenever the inventory position falls at or below a level R, called the reorder point, the manager
orders Q units, called the order quantity. (Notice that the reorder decision is based on the
inventory position including orders and not the inventory level. If managers used the inventory
level, they would place orders continuously as the inventory level fell below R until they received
the order.) When you receive the order after the lead-time, the inventory level jumps from zero
to Q, and the cycle repeats.
In inventory systems, demand is usually uncertain, and the lead-time can also vary. To avoid
shortages, managers often maintain a safety stock. In such situations, it is not clear what order
quantities and reorder points will minimize expected total inventory cost. Simulation models
can address this question.
In this example, demand is uncertain and is Poisson distributed with a mean of 100 units per
week. Thus, the expected annual demand is 5,200 units.
Note: For large values of the rate parameter, , the Poisson distribution is approximately
normal. Thus, this assumption is tantamount to saying that the demand is normally
distributed with a mean of 100 and standard deviation of . The Poisson
distribution is discrete, thus eliminating the need to round off normally distributed
random variates
The traditional economic order quantity (EOQ) model suggests an order quantity:
For the EOQ policy, the reorder point should equal the lead-time demand; that is, place an order
when the inventory position falls to 200 units. If the lead-time demand is exactly 200 units, the
order will arrive when the inventory level reaches zero.
202
However, if demand fluctuates about a mean of 200 units, shortages will occur approximately
half the time. Because of the high shortage costs, the manager would use either a larger reorder
point, a larger order quantity, or both. In either case, the manager will carry more inventory on
average, which will result in a lower total shortage cost but a higher total holding cost. A higher
order quantity lets the manager order less frequently, thus incurring lower total ordering costs.
However, the appropriate choice is not clear. Simulation can test various reorder point/order
quantity policies.
203
Figure 101 Inventory System Problem Spreadsheet Model
In the actual simulation, the beginning inventory position and inventory level for each week
equals the ending levels for the previous week, except for the first week, which is specified in the
problem data. The demand is in column G as Crystal Ball assumptions.
Since all shortages are lost sales, the inventory level cannot be negative. Thus, the ending
inventory each week is:
Lost sales are computed by checking if demand exceeds available stock and computing the
difference.
The spreadsheet simulates 52 weeks, or one year of operation of the inventory system. Since the
objective is to minimize the mean total annual cost, cell P6 is defined as a forecast cell.
Column I determines whether the manager should place an order by checking if the beginning
inventory position minus the weekly demand is at or below the reorder point. The ending
inventory position is:
This formula may not be obvious. Clearly, if there are no lost sales, the ending inventory position
is simply the beginning position minus the demand plus any order that may have been placed.
If lost sales occur, computing the ending inventory position this way reduces it by the unfulfilled
demand, which is incorrect. Thus, you must add back the number of lost sales to account for
this.
204
In the ordering process, the manager places orders at the end of the week and receives orders at
the beginning of the week. Thus, in Figure 101, the order placed at the end of the first week with
a lead-time of 2 weeks will arrive at the beginning of the fourth week. Column L determines the
week an order is due to arrive, and a MATCH function is used in column E to identify whether
an order is scheduled to arrive.
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
Figure 102, following, shows optimization results. OptQuest identified the best solution as
having an order quantity of 330 and a reorder point of 320. The Performance Chart shows that
OptQuest quickly found a good solution value.
205
Figure 102 Inventory System Model Optimization Results
Because this optimization used step sizes of 5 and 10, you can fine-tune the solution by searching
more closely around the best solution using a smaller step size while also increasing the number
of trials per simulation for better precision. This is a good practice, since selecting too small a
step size initially consumes a lot of time or, if time is restricted, OptQuest may not find a good
solution. Thus, as the number of decision variables and range of search increases, use larger step
sizes and fewer trials initially. Later, refine the search around good candidates.
Figure 103 shows the results of an optimization with Order Quantity and Reorder Point bounded
to the range 300 to 360, with a step size of 1 and 5, respectively, and 1000 trials per simulation.
OptQuest identified the best solution as Order Quantity = 330 and Reorder Point = 320, the
same as the initial solution.
206
Figure 103 Inventory System—Second Optimization Results
Figure 104 shows the Crystal Ball forecast chart for the annual total costs for the second solution.
You can see that the distribution of total annual cost is highly concentrated around the mean,
but is also skewed far to the high-value end, indicating that very high values of cost are possible,
although not very likely. For such highly skewed distributions, run more trials than usual, since
statistics like the mean and tail-end percentiles can be susceptible to extreme outliers.
207
Drill Bit Replacement Policy
Subtopics
l Drill Bit Replacement Problem Statement
l Drill Bit Replacement Spreadsheet Model
l Drill Bit Replacement OptQuest Solution
This example was suggested from an example in Kenneth K. Humphreys, Jelen’s Cost and
Optimization Engineering. 3rd ed. New York: McGraw-Hill, 1991. 257-262.
For example, after 5 hours of consecutive use (starting with a new drill bit), the drill is able to
penetrate the terrain at a rate of:
Eventually, the bit must be replaced as the costs exceed the value of the well being drilled. The
problem is to determine the optimum replacement policy; that is, the drilling cycle, T hours,
between replacements.
T hours after replacing the bit, the total drilled depth in meters, M, is given by the integral of
Equation 4.2 from 0 to T, or:
208
l The drill bit replacement time, R, varies and is determined by a triangular distribution with
parameters 6.5, 7.5, and 9.
l The number of 10-hour days available per month, D, also varies due to the weather and the
number of days in a month, and is assumed to be triangular with parameters 24, 28, and 30.
With these assumptions, the profit/drilling cycle if the bit is replaced after T hours equals the
revenue obtained from drilling minus drilling expenses and replacement costs:
profit/drilling cycle = $60M - $425T - ($8,000 + $400R)
Assuming D ten-hour days per month, the average number of cycles per month is 10D/(T + R).
Therefore, the average profit per month is:
The objective is to find the value of T that maximizes the average profit per month.
Cell Assumption
C6 Replacement time, R.
One decision variable is defined in cell C12: the cycle time between replacements of the drill bit,
T.
209
Figure 105 Drill Bit Replacement Problem Spreadsheet Model
The model outputs are computed using the formulas developed in the previous section. The
drilling expenses in cell F7 include both the drilling costs and the replacement costs. The forecast
cell is F12, profit per month.
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
Figure 106, following, shows the OptQuest results. The best solution is to replace the drill bit
approximately every 19.9 hours.
210
Figure 106 Drill Bit Replacement Model Optimization Results
Figure 107, following, shows the OptQuest forecast chart and statistics for the simulation of this
solution. Since the profit per month has a relatively large standard deviation compared to the
mean (coefficient of variance=0.29), it is likely that the true profit/month is significantly higher
or lower than the mean objective value.
211
Gasoline Supply Chain
Subtopics
l Gasoline Supply Chain Statement of Problem
l Gasoline Supply Chain Spreadsheet Model
l Gasoline Supply Chain OptQuest Solution
This example shows how to determine the optimum amount of gasoline to transport between
different levels of a gasoline supply chain. The objective is to minimize the total cost, which
includes transportation costs and inventory holding costs at various points in the supply chain.
It is also important to minimize stockouts at various retail outlets. The complexity of the problem
arises from the fact that stochastic (variable) production exists at the refinery level and stochastic
demand exists at the retail outlet level.
The problem is to determine the amount of gasoline to transport between each level of the supply
chain to minimize the total operating cost, which is computed as the sum of transportation costs
and inventory holding costs. For business reasons, it is helpful to minimize stockouts at the ROs,
to a certain extent.
The following is a schematic diagram of the supply chain:
212
l The weekly supply of gasoline from the refinery (SP) follows a normal distribution with a
mean of 2000 gallons and standard deviation (s.d.) of 450 gallons.
l The weekly demands at ROs are distributed lognormally with means and standard deviations
of 400 gallons and 50 gallons, 500 gallons and 75 gallons, 650 gallons and 100 gallons
respectively at RO1, RO2, RO3.
l The inventory holding cost is a dollar for every five gallons.
l The transportation costs in dollars per gallon are as follows (notice that these costs include
transportation distances):
SP to SD1 = $15
SP to SD2 = $12.5
SD1 to RO1 = $6.5
SD1 to RO2 = $7.5
SD1 to RO3 = $9.0
SD2 to RO1 = $9.0
SD2to RO2 = $8.0
SD2 to RO3 = $7.0
l Existing inventories in gallons are:
Refinery: 200 gallons, SD1: 50 gallons, SD2: 100 gallons, RO1: 120 gallons, RO2: 180 gallons,
RO3: 80 gallons.
213
For this example, OptQuest can determine how much to supply at each of the SDs and ROs to
minimize the total expected cost while maintaining stockouts at ROs at an acceptable level.
Note: Except where indicated, this example uses the recommended Crystal Ball run preferences.
See “Recommended Run Preference Settings for Optimizations” on page 168.
214
Figure 109 Gasoline Supply Chain Constraints
Figure 110 shows that if the quantities of gasoline shown for each decision variable are
transported between the indicated destinations, the mean total cost will be $34,019.09 and the
95th percentile of the worst-case stockout will be -0.02, less than 0.
Figure 111, following, shows the Total Costs forecast chart and statistics for the simulation of
this solution. The Total Costs standard deviation ($94.01) is quite small relative to the mean
total cost ($34,019.09), suggesting that this cost forecast is an accurate representation of the true
weekly costs.
215
Figure 111 Total Costs Forecast Chart and Statistics
For those with Crystal Ball Decision Optimizer, this section describes the different factors that
affect how OptQuest searches for optimal solutions, including model types. Understanding how
these factors affect the optimization helps you control the speed and accuracy of the search.
This section also includes discussion of the Crystal Ball Tornado Chart tool and how you can
use it to analyze the sensitivity of the variables in your model and screen out minor decision
variables.
These tips and suggestions are followed by some notes to help you avoid unexpected results
when using OptQuest. They can also help you troubleshoot any difficulties that may occur.
216
Model Types
Subtopics
l Optimization Models Without Uncertainty
l Optimization Models With Uncertainty
l Discrete, Continuous, or Mixed Models
l Linear or Nonlinear Models
Selecting the right model for your scenario is essential for obtaining optimal results.
The solution to an optimization model provides a set of values for the decision variables that
optimizes (maximizes or minimizes) the associated objective. If the world were simple and the
future were predictable, all data in an optimization model would be constant, making the model
deterministic.
217
Figure 113 Schematic of an Optimization Model With Uncertainty
Stochastic models are much more difficult to optimize because they require simulation to
compute the objective. While OptQuest is designed to solve stochastic models using Crystal Ball,
it is also capable of solving deterministic models. shows that deterministic results are a single
value, while stochastic results are distributed over a curve.
218
l Mixed — Contain both discrete and continuous decision variables, or any of the other
decision variable types: binary, category, or custom.
For more information on discrete and continuous decision variables, see the Oracle Crystal Ball
User's Guide.
Figure 115 on page 219 shows that discrete variable distributions are a series of individual values
while continuous variable distributions are an infinite range of values without distinctive bounds
except the end points.
OptQuest can handle both linear and nonlinear objectives and constraints. For information on
defining linear or nonlinear constraints, see the Oracle Decision Optimizer OptQuest User's
Guide.
219
Factors That Affect Optimization Performance
Subtopics
l Simulation Accuracy
l Number of Decision Variables
l Base Case Values
l Bounds and Constraints
l Requirements
l Complexity of the Objective
l Simulation Speed
l Precision Control
There are many factors that influence the performance of OptQuest. For example, consider two
optimization methods, A and B, applied to an investment problem with the objective of
maximizing expected returns. When you evaluate the performance of each method, you must
look at which method:
l Finds an investment portfolio with a larger expected return
l Jumps to the range of high-quality solutions more quickly
Figure 117 on page 220 shows that although both methods find solutions with a similar expected
profit after 10 minutes of searching, method A jumps to the range of high-quality solutions faster
than B. For the criteria listed previously, method A performs better than method B.
While using OptQuest, you will obtain performance profiles similar to method A. OptQuest’s
search methodology (see the references in Appendix B) is very aggressive and attempts to find
high-quality solutions immediately, causing large improvements (with respect to the initial
solution) early in the search. This is critical when OptQuest can perform only a limited number
of simulations within the available time limit.
220
However, several factors affect OptQuest’s performance, and the importance of these factors
varies from one situation to another. The list at the beginning of this topic shows factors that
directly affect the search for an optimal solution, discussed in the following linked topics.
Simulation Accuracy
For sufficient accuracy, set the number of simulation trials to the minimum number necessary
to obtain a reliable estimate of the statistic being optimized. For example, you can reliably
estimate the mean with fewer trials than the standard deviation or a percentile.
General guidelines for determining the number of simulation trials necessary to obtain good
estimates are:
l 200 to 500 trials is usually sufficient for obtaining accurate estimates for the mean.
l At least 1000 trials are necessary for obtaining reasonable estimates for tail-end percentiles.
Empirical testing with the simulation model using the Crystal Ball Bootstrap tool (see the Oracle
Crystal Ball User's Guide) can help you find the appropriate number of trials for a given situation.
For very large numbers of decision variables, you may try running more simulations by lowering
the number of trials per simulation, at least initially. After you find an approximate solution,
you can rerun the optimization by using the approximate solution as suggested values, further
restricting the bounds on the decision variables, and increasing the number of trials to find more
accurate results.
221
l Requirements < 512
Requirements
While the search process benefits from the use of constraints and tight bounds, performance
generally suffers when you include requirements in the optimization model for two reasons:
l Requirements are very time-consuming to evaluate, since OptQuest must run an entire
simulation before determining whether the results are requirement-infeasible.
222
l To avoid running requirement-infeasible simulations, OptQuest must identify the
characteristics of solutions likely to be requirement-feasible. This makes the search more
complex and requires more time.
When you use requirements, you should increase the search time by at least 50% (based on the
time used for an equivalent problem without requirements).
OptQuest is designed to find global solutions for all types of objectives, especially complex
objectives like this one. However, for more complex objectives, you generally need to run more
simulations to find high-quality global solutions.
Simulation Speed
By increasing the speed of each simulation, you can increase the number of simulations that
OptQuest runs in a given time period. Some suggestions to increase speed are:
l Use precision control in Crystal Ball to stop simulations as soon as they reach a satisfactory
accuracy
l Reduce the size of your model
l Increase your system's RAM
l Reduce the number of assumptions and forecasts
l Quit other applications
The Oracle Crystal Ball User's Guide discusses these suggestions in more detail.
223
Precision Control
For some models, the accuracy of the statistics is highly dependent on the values of the decision
variables. In these cases, you can use Crystal Ball’s precision control feature to run a sufficient
number of trials for each simulation to achieve the necessary level of accuracy.
You can use Crystal Ball’s precision control feature for several purposes:
l When you are unsure of how to set the number of trials used for Crystal Ball simulations
l If you believe that the stability of the forecast statistics varies greatly depending on the
decision variable values
Precision control periodically calculates the accuracy of the forecast mean, standard deviation,
and any indicated percentile during the simulation. When the simulation reaches the target
accuracy, it stops, regardless of the number of trials already run.
This feature is especially useful for optimization models such as Portfolio Allocation, where the
forecast statistics are highly sensitive to the decision variables. When OptQuest selects
conservative investments, the variability of the expected return is low and the statistics are
relatively stable. When OptQuest selects aggressive investments, the variability is high and the
statistics are relatively less stable. Using precision control increases your forecast statistic
accuracy while avoiding running too many trials when a simulation reaches this accuracy quickly.
Notice that finding the appropriate precision control settings may require some trial and error.
It can be challenging to decide whether to use absolute or relative precision, what is the best
precision value in either case, and which statistics should receive precision control. For more
information on setting the precision control feature, see the Oracle Crystal Ball User's Guide.
ä To see the effects of using precision control with the Portfolio Allocation model:
1 In Oracle Crystal Ball Decision Optimizer, select Run, and then Run Preferences and change the
maximum number of trials from 1000 to 5000.
This maximum limit is always in effect, even when precision control is turned on. Therefore,
when using precision control, you must increase the maximum number of trials to let
precision control achieve the appropriate accuracy.
2 Turn on Precision Control.
a. Select cell C17.
b. Select Define, and then Define Forecast.
c. Click the More button in the Define Forecast dialog, then click the Precision tab.
d. Select Specify The Desired Precision For Forecast Statistics.
e. Select Mean.
f. Use an absolute precision of 1000 units.
3 Select Run, and then Run Preferences, , and set the following run preferences:
l Maximum number of trials to run set to 1000
l Sampling method set to Latin Hypercube
224
l Sample Size For Latin Hypercube set to 500
l Random Number Generation set to Use Same Sequence Of Random Numbers with an
Initial Seed Value of 999
4 Run another optimization.
Experiment with various other precision control settings to see the difference in the results.
Before running the Tornado Chart tool, run an initial optimization so that the base case values
of the decision variables are close to the optimal solution for your model. You can use the
225
Tornado Chart tool to measure the impact of your decision variables. For information, see the
Oracle Crystal Ball User's Guide.
The notes in this section can help you avoid problems while using OptQuest and can also assist
in troubleshooting any difficulties that may happen:
226
Array Formulas
Array formulas with brackets are supported by Microsoft Excel but are not allowed in OptQuest.
For example, suppose you enter a constraint as follows, referencing a named range:
MyRange > {0}
Date Formatting
It is possible to reference a decision variable cell formatted as a date (such as 2/19/1900) and
enter a constraint as follows in the OptQuest wizard's Constraints panel:
E2 > 2/19/1900
If you do this, OptQuest interprets it as 2 divided by 19 divided by 1900 and does not display an
error message.
This behavior is consistent with that of the Microsoft Excel formula bar. For best results, use the
Microsoft Excel DATE() function.
227
Linear Constraints Can Be Evaluated As Nonlinear
If you have a cell reference in an OptQuest constraint that is more than seven levels of formulas
removed from a decision variable, any constraint based on that cell will be evaluated as nonlinear
even though it may be linear.
228
Bibliography
A
In This Appendix
Crystal Ball Tools......................................................................................... 229
Monte Carlo Simulation ................................................................................. 230
Probability Theory and Statistics........................................................................ 231
Random Variate Generation Methods.................................................................. 231
Specific Distributions .................................................................................... 232
Uncertainty Analysis ..................................................................................... 232
Spreadsheet Design ..................................................................................... 233
Sequential Sampling with SIPs ......................................................................... 233
Time-series Forecasting References .................................................................... 233
Optimization References ................................................................................ 235
Bootstrap
Efron, Bradley, and Robert J. Tibshirani. “An Introduction to the Bootstrap,” Monographs on
Statistics and Applied Probability, vol. 57. New York: Chapman and Hall, 1993.
Mooney, C. Z., and R.D. Duval. “Bootstrapping: A Nonparametric Approach to Statistical
Inference,” Sage University Paper Series on Quantitative Applications in the Social Sciences,
series no. 07-095. Newbury Park, CA: Sage, 1993.
229
Morgan, M. Granger, and Max Henrion; with a chapter by Mitchell Small. Uncertainty: A Guide
to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. New York: Cambridge
University Press, 1990.
Two-Dimensional Simulation
Burmaster, David E., and Andrew M. Wilson. “An Introduction to Second-Order Random
Variables in Human Health Risk Assessments,” Human and Ecological Risk Assessment 2(4):
892-919 (1996).
Frey, H. Christopher, and David S. Rhodes. “Characterizing, Simulating, and Analyzing
Variability and Uncertainty: An Illustration of Methods Using an Air Toxics Emissions
Example,” Human and Ecological Risk Assessment 2(4): 762-797 (1996).
Hoffman, F.O., and J.S. Hammonds. “Propagation of Uncertainty in Risk Assessments: The
Need to Distinguish Between Uncertainty Due to Lack of Knowledge and Uncertainty Due to
Variability,” Risk Analysis 14(5): 707-712 (1994).
Rai, S. N., D. Drewski, and S. Bartlett. “A General Framework for the Analysis of Uncertainty
and Variability in Risk Assessment,” Human and Ecological Risk Assessment 2(4): 972-989
(1996).
230
Probability Theory and Statistics
Alder, Henry L., and Edward B. Roessler. Introduction to Probability and Statistics, 6th Ed. San
Francisco: W.H. Freeman & Co, 1977.
Biswas, Suddhendu. Topics in Statistical Methodology. New York: John Wiley & Sons, Inc., 1991.
Fraser, D.A.S. Probability and Statistics: Theory and Applications. North Scituate, MA: Duxbury
Press, 1976.
Johnson, Norman L., and Samuel Kotz. Distributions in Statistics: Continuous Multivariate
Distributions. New York: John Wiley & Sons, Inc., 1972.
Kapur, K.C., and L.R. Lamberson. Reliability in Engineering Design. New York: John Wiley &
Sons, 1977.
Larson, Harold J. Introduction to Probability Theory and Statistical Inference, 2nd Ed. New York:
John Wiley & Sons, 1974.
Logothetis, N., and V. Rothschild. Probability Distributions. New York: John Wiley & Sons, Inc.,
1986.
Spurr, William, and Charles P. Bonini. Statistical Analysis for Business Decisions. Homewood,
IL: Richard D. Irwin, Inc., 1973.
Wadsworth, George P., and Joseph G. Bryan. Introduction to Probability and Random
Variables. New York: McGraw-Hill Book Co., 1960.
Winkler, Robert L., and William L. Hays. Statistics: Probability, Inference, and Decision. New
York: Holt, Rinehart, and Winston, 1975.
231
Odeh, R.E., and J.O. Evans. “Percentage Points of the Normal Distribution,” Applied Statistics.
London: Royal Statistical Society, 1974.
Sedgewich, Robert. Algorithms. Reading, MA: Addison Wesley Publishing Co., 1983.
Specific Distributions
Subtopics
l Extreme Value Distribution
l Lognormal Distribution
l Weibull Distribution
Lognormal Distribution
Aitchison, J., and J.A. Brown. The Lognormal Distribution. New York: Cambridge University
Press, 1973.
Weibull Distribution
King, James R. Probability Charts for Decision Making, Rev. Ed. New York: Industrial Press, Inc.,
1981.
Henley, Ernest J., and Hiromitsu Kumamoto. Reliability Engineering and Risk Assessment.
Englewood Cliffs, NJ: Prentice Hall, Inc., 1981.
Uncertainty Analysis
Morgan, M. Granger, and Max Henrion; with a chapter by Mitchell Small. Uncertainty: A Guide
to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. New York: Cambridge
University Press, 1990.
Hoffman, F.O., and J.S. Hammonds. “Propagation of Uncertainty in Risk Assessments: The
Need to Distinguish Between Uncertainty Due to Lack of Knowledge and Uncertainty Due to
Variability,” Risk Analysis 14(5): 707-712 (1994).
International Atomic Energy Agency, “Evaluating the Reliability of Predictions Made Using
Environmental Transfer Models,” IAEA Safety Series No. 100:1-106. STI/PUB/835. Vienna,
Austria, 1989.
Kelton, W. David, and Averill M. Law. Simulation Modeling & Analysis, 3rd Ed. New York:
McGraw-Hill, Inc. 2000.
232
Spreadsheet Design
Powell, S.G., and K.R. Baker. The Art of Modeling with Spreadsheets: Management Science,
Spreadsheet Engineering, and Modeling Craft. Hoboken, NJ: John Wiley, 2003.
Ragsdale, C.T. Spreadsheet Modeling and Decision Analysis: A Practical Introduction to
Management Science. 5th Ed. Mason, OH: South-Western College Publishing, 2007.
Thommes, M.C. Proper Spreadsheet Design. Boston: Boyd and Fraser Publishing Co., 1992.
Forecasting
Bowerman, B.L., and R.T. O'Connell (contributor). Forecasting and Time Series: An Applied
Approach (The Duxbury Advanced Series in Statistics and Decision Sciences). Belmont, CA:
Duxbury Press, 1993.
DeLurgio, S.A., Sr. Forecasting Principles and Applications. Boston: Irwin/McGraw-Hill, 1998.
Hanke, J.E., and D.W. Wichern. Business Forecasting. 9th ed. Upper Saddle River:, NJ: Prentice
Hall, 2008.
Makridakis, S., S.C. Wheelwright, and R.J. Hyndman. Forecasting Methods and Applications. 3rd
ed. Hoboken, NJ: John Wiley & Sons, 1998.
Ragsdale, C. Spreadsheet Modeling and Decision Analysis: A Practical Introduction to Management
Science. 5th ed. Florence, KY: South-Western College Publishing, 2007.
233
Wei, W.W.S. Time Series Analysis: Univariate and Multivariate Methods. 2nd ed. New York:
Pearson, 2006.
ARIMA Forecasting
Box, G. E. P., Jenkins, G. M., and Reinsel, G. C. Time Series Analysis: Forecasting and Control.
4th ed. Hoboken, NJ: John Wiley & Sons. 2008.
Hamilton, J. D. Time Series Analysis. Princeton, NJ: Princeton University Press. 1st ed. 1994.
Hanke, J.E., and D.W. Wichern. Business Forecasting. 9th ed. Upper Saddle River, NJ: Prentice
Hall, 2008.
Liu, Lon-Mu. Time Series Analysis and Forecasting. 2nd ed. Villa Park, IL: Scientific Computing
Associates. 2006.
Makridakis, S., S.C. Wheelwright, and R.J. Hyndman. Forecasting Methods and Applications. 3rd
ed. Hoboken, NJ: John Wiley & Sons, 1998.
Wei, W. W. S. Time Series Analysis: Univariate and Multivariate methods. 2nd ed. New York:
Pearson, 2006.
Regression Analysis
Draper, N.R., and H. Smith. Applied Regression Analysis. 3rd ed. Hoboken, NJ: John Wiley &
Sons, 1998.
Golub, G.H., and C.F. Van Loan. Matrix Computations, 3rd ed. Baltimore: The Johns Hopkins
University Press, 1996.
Hanke, J.E., and D.W. Wichern. Business Forecasting. 9th ed. Upper Saddle River, NJ: Prentice
Hall, 2008.
Makridakis, S., S.C. Wheelwright, and R.J. Hyndman. Forecasting Methods and Applications. 3rd
ed. Hoboken, NJ: John Wiley & Sons, 1998.
Miller, A. Subset Selection in Regression. 2nd ed. Chapman & Hall/CRC, 2002.
234
Optimization References
Subtopics
l OptQuest References and White Papers
l Metaheuristics
l Stochastic (Probabilistic) Optimization Theory
l Multiobjective Optimization
l Optimization and Simulation in Practice
A variety of white papers concerning optimization are available on the Web site of OptTek
Systems, Inc., the company that developed the OptQuest calculation engine. For a list of papers
with abstracts, see http://www.opttek.com/News/WhitePapers.html
Metaheuristics
Glover, F., J.P. Kelly, and M. Laguna. “New Advances and Applications of Combining Simulation
and Optimization,” in Proceedings of the 1996 Winter Simulation Conference. Edited by J.M.
Charnes, D.J. Morrice, D.T. Brunner, and J.J. Swain, 1996: 144-152.
Glover, F., and M. Laguna. Tabu Search. Boston: Kluwer Academic Publishers, 1997.
Laguna, M. “Scatter Search,” in Handbook of Applied Optimization. P.M. Pardalos and M.G.C.
Resende (Eds.), Oxford Academic Press, 1999.
235
Multiobjective Optimization
Chankong, V., and Y.Y. Haimes. Multiobjective Decision Making: Theory and Methodology. New
York: North-Holland, 1983.
Hwang, C., and A. S. M. Masud. Multiple Objective Decision Making -Methods and
Applications. Berlin: Springer-Verlag, 1979.
Keeney, R., and Raiffa, H. Decisions with Multiple Objectives. New York: John Wiley, 1976.
Financial Applications
Brealey, R., and S. Myers. Principles of Corporate Finance. 4th ed. New York: McGraw-Hill, Inc.,
1991.
Chen, N., R. Roll, and S. Ross. “Economic Forces in the Stock Market.” Journal of Business, 59
(July 1986): 383-403.
Markowitz, H.M. Portfolio Selection. 2nd ed. Cambridge, MA: Blackwell Publishers Ltd., 1991.
236