Designation: D 5490 – 93 (Reapproved 2002)

Standard Guide for

Comparing Ground-Water Flow Model Simulations to Site-
Specific Information1
This standard is issued under the fixed designation D 5490; the number immediately following the designation indicates the year of
original adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. A
superscript epsilon (e) indicates an editorial change since the last revision or reapproval.

1. Scope priate safety and health practices and determine the applica-
1.1 This guide covers techniques that should be used to bility of regulatory limitations prior to use.
compare the results of ground-water flow model simulations to 1.9 This guide offers an organized collection of information
measured field data as a part of the process of calibrating a or a series of options and does not recommend a specific
ground-water model. This comparison produces quantitative course of action. This document cannot replace education or
and qualitative measures of the degree of correspondence experience and should be used in conjunction with professional
between the simulation and site-specific information related to judgment. Not all aspects of this guide may be applicable in all
the physical hydrogeologic system. circumstances. This ASTM standard is not intended to repre-
1.2 During the process of calibration of a ground-water flow sent or replace the standard of care by which the adequacy of
model, each simulation is compared to site-specific informa- a given professional service must be judged, nor should this
tion such as measured water levels or flow rates. The degree of document be applied without consideration of a project’s many
correspondence between the simulation and the physical hy- unique aspects. The word “Standard” in the title of this
drogeologic system can then be compared to that for previous document means only that the document has been approved
simulations to ascertain the success of previous calibration through the ASTM consensus process.
efforts and to identify potentially beneficial directions for 2. Referenced Documents
further calibration efforts.
1.3 By necessity, all knowledge of a site is derived from 2.1 ASTM Standards:
observations. This guide does not address the adequacy of any D 653 Terminology Relating to Soil, Rock, and Contained
set of observations for characterizing a site. Fluids2
1.4 This guide does not establish criteria for successful E 978 Practice for Evaluating Mathematical Models for the
calibration, nor does it describe techniques for establishing Environmental Fate of Chemicals3
such criteria, nor does it describe techniques for achieving 3. Terminology
successful calibration.
1.5 This guide is written for comparing the results of 3.1 Definitions:
numerical ground-water flow models with observed site- 3.1.1 application verification—using the set of parameter
specific information. However, these techniques could be values and boundary conditions from a calibrated model to
applied to other types of ground-water related models, such as approximate acceptably a second set of field data measured
analytical models, multiphase flow models, noncontinuum under similar hydrologic conditions.
(karst or fracture flow) models, or mass transport models. 3.1.1.1 Discussion—Application verification is to be distin-
1.6 This guide is one of a series of guides on ground-water guished from code verification which refers to software testing,
modeling codes (software) and their applications. Other stan- comparison with analytical solutions, and comparison with
dards have been prepared on environmental modeling, such as other similar codes to demonstrate that the code represents its
Practice E 978. mathematical foundation.
1.7 The values stated in SI units are to be regarded as the 3.1.2 calibration—the process of refining the model repre-
standard. sentation of the hydrogeologic framework, hydraulic proper-
1.8 This standard does not purport to address all of the ties, and boundary conditions to achieve a desired degree of
safety concerns, if any, associated with its use. It is the correspondence between the model simulations and observa-
responsibility of the user of this standard to establish appro- tions of the ground-water flow system.
3.1.3 censored data—knowledge that the value of a variable
in the physical hydrogeologic system is less than or greater
1
This guide is under the jurisdiction of ASTM Committee D18 on Soil and Rock
and is the direct responsibility of Subcommittee D18.21 on Ground Water and
2
Vadose Zone Investigations. Annual Book of ASTM Standards, Vol 04.08.
3
Current edition approved Nov. 15, 1993. Published January 1994. Annual Book of ASTM Standards, Vol 11.04.

Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.

1
 D 5490
than a certain value, without knowing the exact value. materials which are being modeled. Modeled aquifer hydro-
3.1.3.1 Discussion—For example, if a well is dry, then the logic properties should fall within realistic ranges for the
potentiometric head at that place and time must be less than the physical hydrogeologic system, as defined during conceptual
elevation of the screened interval of the well although its model development.
specific value is unknown.
3.1.4 conceptual model—an interpretation or working de- 5. Significance and Use
scription of the characteristics and dynamics of the physical 5.1 During the process of calibration of a ground-water flow
system. model, each simulation is compared to site-specific informa-
3.1.5 ground-water flow model—an application of a math- tion to ascertain the success of previous calibration efforts and
ematical model to represent a ground-water flow system. to identify potentially beneficial directions for further calibra-
3.1.6 hydrologic condition—a set of ground-water inflows tion efforts. Procedures described herein provide guidance for
or outflows, boundary conditions, and hydraulic properties that making comparisons between ground-water flow model simu-
cause potentiometric heads to adopt a distinct pattern. lations and measured field data.
3.1.7 residual—the difference between the computed and 5.2 This guide is not meant to be an inflexible description of
observed values of a variable at a specific time and location. techniques comparing simulations with measured data; other
3.1.8 simulation—in ground-water flow modeling, one techniques may be applied as appropriate and, after due
complete execution of a ground-water modeling computer consideration, some of the techniques herein may be omitted,
program, including input and output. altered, or enhanced.
3.1.8.1 Discussion—For the purposes of this guide, a simu-
lation refers to an individual modeling run. However, simula- 6. Quantitative Techniques
tion is sometimes also used broadly to refer to the process of 6.1 Quantitative techniques for comparing simulations to
modeling in general. site-specific information include calculating potentiometric
3.2 For definitions of other terms used in this guide, see head residuals, assessing correlation among head residuals, and
Terminology D 653. calculating flow residuals.
6.1.1 Potentiometric Head Residuals—Calculate the residu-
4. Summary of Guide als (differences) between the computed heads and the measured
4.1 Quantitative and qualitative comparisons are both es- heads:
sential. Both should be used to evaluate the degree of corre- r i 5 hi 2 H i (1)
spondence between a ground-water flow model simulation and
site-specific information. where:
4.2 Quantitative techniques for comparing a simulation with ri = the residual,
site-specific information include: Hi = the measured head at point i,
4.2.1 Calculation of residuals between simulated and mea- hi = the computed head at the approximate location where
sured potentiometric heads and calculation of statistics regard- Hi was measured.
ing the residuals. Censored data resulting from detection of dry If the residual is positive, then the computed head was too
or flowing observation wells, reflecting information that the high; if negative, the computed head was too low. Residuals
head is less than or greater than a certain value without cannot be calculated from censored data.
knowing the exact value, should also be used. NOTE 1—For drawdown models, residuals can be calculated from
4.2.2 Detection of correlations among residuals. Spatial and computed and measured drawdowns rather than heads.
temporal correlations among residuals should be investigated. NOTE 2—Comparisons should be made between point potentiometric
Correlations between residuals and potentiometric heads can heads rather than ground-water contours, because contours are the result
be detected using a scattergram. of interpretation of data points and are not considered basic data in and of
themselves.4 Instead, the ground-water contours are considered to reflect
4.2.3 Calculation of flow-related residuals. Model results features of the conceptual model of the site. The ground-water flow model
should be compared to flow data, such as water budgets, should be true to the essential features of the conceptual model and not to
surface water flow rates, flowing well discharges, vertical their representation.
gradients, and contaminant plume trajectories. NOTE 3—It is desirable to set up the model so that it calculates heads at
4.3 Qualitative considerations for comparing a simulation the times and locations where they were measured, but this is not always
with site-specific information include: possible or practical. In cases where the location of a monitoring well does
not correspond exactly to one of the nodes where heads are computed in
4.3.1 Comparison of general flow features. Simulations
the simulation, the residual may be adjusted (for example, computed heads
should reproduce qualitative features in the pattern of ground- may be interpolated, extrapolated, scaled, or otherwise transformed) for
water contours, including ground-water flow directions, use in calculating statistics. Adjustments may also be necessary when the
mounds or depressions (closed contours), or indications of times of measurements do not correspond exactly with the times when
surface water discharge or recharge (cusps in the contours). heads are calculated in transient simulations; when many observed heads
4.3.2 Assessment of the number of distinct hydrologic are clustered near a single node; where the hydraulic gradient changes
conditions to which the model has been successfully calibrated. significantly from node to node; or when observed head data is affected by
tidal fluctuations or proximity to a specified head boundary.
It is usually better to calibrate to multiple scenarios, if the
scenarios are truly distinct.
4.3.3 Assessment of the reasonableness or justifiability of 4
Cooley, R. L., and Naff, R. L., “Regression Modeling of Ground-Water Flow,”
the input aquifer hydrologic properties given the aquifer USGS Techniques of Water Resources Investigations, Book 3, Chapter B4, 1990.

2
 D 5490
6.1.2 Residual Statistics—Calculate the maximum and 1

5 6
n
2
minimum residuals, a residual mean, and a second-order (
i51
wi ~ r i 2 R ! 2
statistic, as described in the following sections. s 5 n (5)
6.1.2.1 Maximum and Minimum Residuals—The maximum ~n 2 1! ( wi
i51
residual is the residual that is closest to positive infinity. The
minimum residual is the residual closest to negative infinity. Of NOTE 6—Other norms of the residuals are less common but may be
two simulations, the one with the maximum and minimum revealing in certain cases.5,6 For example, the mean of the absolute values
of the residuals can give information similar to that of the standard
residuals closest to zero has a better degree of correspondence,
deviation of residuals.
with regard to this criterion. NOTE 7—In calculating the standard deviation of residuals, advanced
NOTE 4—When multiple hydrologic conditions are being modeled as statistical techniques incorporating information from censored data could
separate steady-state simulations, the maximum and minimum residual be used. However, the effort would usually not be justified because the
can be calculated for the residuals in each, or for all residuals in all standard deviation of residuals is only one of many indicators involved in
scenarios, as appropriate. This note also applies to the residual mean (see comparing a simulation with measured data, and such a refinement in one
6.1.2.2) and second-order statistics of the residuals (see 6.1.2.4). indicator is unlikely to alter the overall assessment of the degree of
correspondence.
6.1.2.2 Residual Mean—Calculate the residual mean as the
arithmetic mean of the residuals computed from a given 6.1.3 Correlation Among Residuals—Spatial or temporal
simulation: correlation among residuals can indicate systematic trends or
n
bias in the model. Correlations among residuals can be
(
i51
ri identified through listings, scattergrams, and spatial or tempo-
R 5 (2) ral plots. Of two simulations, the one with less correlation
n
among residuals has a better degree of correspondence, with
where: regard to this criterion.
R = the residual mean and 6.1.3.1 Listings—List residuals by well or piezometer, in-
n = the number of residuals. cluding the measured and computed values to detect spatial or
Of two simulations, the one with the residual mean closest to temporal trends. Figures X1.1 and X1.2 present example
zero has a better degree of correspondence, with regard to this listings of residuals.
criterion (assuming there is no correlation among residuals). 6.1.3.2 Scattergram—Use a scattergram of computed versus
6.1.2.3 If desired, the individual residuals can be weighted measured heads to detect trends in deviations. The scattergram
to account for differing degrees of confidence in the measured is produced with measured heads on the abscissa (horizontal
heads. In this case, the residual mean becomes the weighted axis) and computed heads on the ordinate (vertical axis). One
residual mean: point is plotted on this graph for each pair. If the points line up
n along a line with zero intercept and 45° angle, then there has
( wiri
i51
been a perfect match. Usually, there will be some scatter about
R5 n (3) this line, hence the name of the plot. A simulation with a small
n ( wi
i51
degree of scatter about this line has a better correspondence
with the physical hydrogeologic system than a simulation with
where wi is the weighting factor for the residual at point i. a large degree of scatter. In addition, plotted points in any area
The weighting factors can be based on the modeler’s judgment of the scattergram should not all be grouped above or below the
or statistical measures of the variability in the water level line. Figures X1.3 and X1.4 show sample scattergrams.
measurements. A higher weighting factor should be used for a 6.1.3.3 Spatial Correlation—Plot residuals in plan or sec-
measurement with a high degree of confidence than for one tion to identify spatial trends in residuals. In this plot, the
with a low degree of confidence. residuals, including their sign, are plotted on a site map or cross
NOTE 5—It is possible that large positive and negative residuals could section. If possible or appropriate, the residuals can also be
cancel, resulting in a small residual mean. For this reason, the residual contoured. Apparent trends or spatial correlations in the residu-
mean should never be considered alone, but rather always in conjunction als may indicate a need to refine aquifer parameters or
with the other quantitative and qualitative comparisons. boundary conditions, or even to reevaluate the conceptual
6.1.2.4 Second-Order Statistics—Second-order statistics model (for example, add spatial dimensions or physical pro-
give measures of the amount of spread of the residuals about cesses). For example, if all of the residuals in the vicinity of a
the residual mean. The most common second-order statistic is no-flow boundary are positive, then the recharge may need to
the standard deviation of residuals: be reduced or the hydraulic conductivity increased. Figure

H J
1 X1.5 presents an example of a contour plot of residuals in plan
n
2
( ~ri 2 R!2 view. Figure X1.6 presents an example of a plot of residuals in
i51 cross section.
s5 (4)
~n 2 1!
where s is the standard deviation of residuals. Smaller values 5
Ghassemi, F., Jakeman, A. J., and Thomas, G. A., “Ground-Water Modeling for
of the standard deviation indicate better degrees of correspon- Salinity Management: An Australian Case Study,” Ground Water, Vol 27, No. 3,
1989, pp. 384–392.
dence than larger values. 6
Konikow, L. F., Calibration of Ground-Water Models, Proceedings of the
6.1.2.5 If weighting is used, calculate the weighted standard Specialty Conference on Verification of Mathematical and Physical Models in
deviation: Hydraulic Engineering, ASCE, College Park, MD, Aug. 9–11, 1978, pp. 87–93.

3
 D 5490
6.1.3.4 Temporal Correlation—For transient simulations, also appear in contours of modeled heads.
plot residuals at a single point versus time to identify temporal 7.2 Hydrologic Conditions—Identify the different hydro-
trends. Temporal correlations in residuals can indicate the need logic conditions that are represented by the available data sets.
to refine input aquifer storage properties or initial conditions. Choose one data set from each hydrologic condition to use for
Figure X1.7 presents a typical plot of residuals versus time. calibration. Use the remaining sets for verification.
6.1.4 Flow-Related Residuals—Often, information relating 7.2.1 Uniqueness (Distinct Hydrologic Conditions)—The
to ground-water velocities is available for a site. Examples number of distinct hydrologic conditions that a given set of
include water budgets, surface water flow rates, flowing well input aquifer hydrologic properties is capable of representing is
discharges, vertical gradients, and contaminant plume trajec- an important qualitative measure of the performance of a
tories (ground-water flow paths). All such quantities are
model. It is usually better to calibrate to multiple conditions, if
dependent on the hydraulic gradient (the spatial derivative of
the conditions are truly distinct. Different hydrologic condi-
the potentiometric head). Therefore, they relate to the overall
tions include, but are not limited to, high and low recharge;
structure of the pattern of potentiometric heads and provide
conditions before and after pumping or installation of a cutoff
information not available from point head measurements. For
wall or cap; and high and low tides, flood stages for adjoining
each such datum available, calculate the residual between its
surface waters, or installation of drains. By matching different
computed and measured values. If possible and appropriate,
hydrologic conditions, the uniqueness problem is addressed,
calculate statistics on these residuals and assess their correla-
because one set of heads can be matched with the proper ratio
tions, in the manner described in 5.1 and 5.2 for potentiometric
of ground-water flow rates to hydraulic conductivities;
head residuals.
whereas, when the flow rates are changed, representing a
6.1.4.1 Water Budgets and Mass Balance—For elements of
different condition, the range of acceptable hydraulic conduc-
the water budget for a site which are calculated (as opposed to
tivities becomes much more limited.
specified in the model input) (for example, base flow to a
stream), compare the computed and the measured (or esti- 7.2.2 Verification (Similar Hydrologic Conditions)—When
mated) values. In addition, check the computed mass balance piezometric head data are available for two times of similar
for the simulation by comparing the sum of all inflows to the hydrologic conditions, only one of those conditions should be
sum of all outflows and changes in storage. Differences of included in the calibration data sets because they are not
more than a few percent in the mass balance indicate possible distinct. However, the other data set can be used for model
numerical problems and may invalidate simulation results. verification. In the verification process, the modeled piezomet-
6.1.4.2 Vertical Gradients—In some models, it may be ric heads representing the hydrologic condition in question are
more important to accurately represent the difference in heads compared, not to the calibration data set, but to the verification
above and below a confining layer, rather than to reproduce the data set. The resulting degree of correspondence can be taken
heads themselves. In such a case, it may be acceptable to as an indicator or heuristic measure of the ability of the model
tolerate a correlation between the head residuals above and to represent new hydrologic conditions within the range of
below the layer if the residual in the vertical gradient is those to which the model was calibrated.
minimized. NOTE 8—When only one data set is available, it is inadvisable to
6.1.4.3 Ground-Water Flow Paths—In some models, it may artificially split it into separate “calibration” and “verification” data sets.
be more important to reproduce the pattern of streamlines in It is usually more important to calibrate to piezometric head data spanning
the ground-water flow system rather than to reproduce the as much of the modeled domain as possible.
heads themselves (for example, when a flow model is to be NOTE 9—Some researchers maintain that the word “verification” im-
used for input of velocities into a contaminant transport plies a higher degree of confidence than is warranted.7 Used here, the
model). In this case, as with the case of vertical gradients in verification process only provides a method for estimating confidence
6.1.4.2 it may be acceptable to tolerate some correlation in intervals on model predictions.
head residuals if the ground-water velocity (magnitude and 7.3 Input Aquifer Hydraulic Properties—A good correspon-
direction) residuals are minimized. dence between a ground-water flow model simulation and
7. Qualitative Considerations site-specific information, in terms of quantitative measures,
may sometimes be achieved using unrealistic aquifer hydraulic
7.1 General Flow Features—One criterion for evaluating properties. This is one reason why emphasis is placed on the
the degree of correspondence between a ground-water flow ability to reproduce multiple distinct hydrologic stress sce-
model simulation and the physical hydrogeologic system is narios. Thus, a qualitative check on the degree of correspon-
whether or not essential qualitative features of the potentio-
dence between a simulation and the physical hydrogeologic
metric surface are reflected in the model. The overall pattern of
system should include an assessment of the likely ranges of
flow directions and temporal variations in the model should
hydraulic properties for the physical hydrogeologic system at
correspond with those at the site. For example:
the scale of the model or model cells and whether the
7.1.1 If there is a mound or depression in the potentiometric
properties used in the model lie within those ranges.
surface at the site, then the modeled contours should also
indicate a mound or depression in approximately the same
area.
7.1.2 If measured heads indicate or imply cusps in the 7
Konikow, L. F., and Bredehoeft, J. D., “Ground-Water Models Cannot Be
ground-water contours at a stream, then these features should Validated,” Adv. Wat. Res. Vol 15, 1992, pp. 75–83.

4
 D 5490
8. Report 9. Keywords
8.1 When a report for a ground-water flow model applica- 9.1 calibration; computer; ground water; modeling
tion is produced, it should include a description of the above
comparison tests which were performed, the rationale for
selecting or omitting comparison tests, and the results of those
comparison tests.

APPENDIX

(Nonmandatory Information)

X1. EXAMPLES

X1.1 Fig. X1.1 and Fig. X1.2 present sample listings of

residuals, as described in 6.1.3.1. These listings tabulate the
residuals for simulations of two hydrologic conditions with the
same model. Note that some of the wells do not have
measurements for both simulations. Simulated heads for these
wells are still reported as an aid to detecting temporal trends in
the heads for different aquifer stresses. Some censored water

FIG. X1.2 Example Listings of Residuals

level data were available for this site. For these data, the table
merely indicates whether or not the simulation is consistent
with the censored data.

X1.2 Fig. X1.3 and Fig. X1.4 show sample scattergrams, as

described in 6.1.3.2. The scattergram on Fig. X1.3 indicates a
FIG. X1.1 Example Listings of Residuals good match between modeled and measured potentiometric

5
 D 5490

FIG. X1.3 Sample Scattergram FIG. X1.4 Sample Scattergram

heads because there is little or no pattern between positive and

negative residuals and because the magnitude of the residuals X1.4 Fig. X1.7 shows a sample plot of measured and
is small compared to the total change in potentiometric head simulated potentiometric heads and their residuals for one well
across the site. The residuals shown on the scattergram on Fig. in a transient simulation, as described in 6.1.3.4. The upper
X1.4 have the same maximum, minimum, mean, and standard graph shows the measured potentiometric head at the well as
deviation as those shown on Fig. X1.3, but show a pattern of measured using a pressure transducer connected to a data
positiveresiduals upgradient and negative residuals downgra- logger. In addition, simulated potentiometric heads for the
dient. However, even though the statistical comparisons would same time period are also shown. The lower graph shows the
indicate a good degree of correspondence, this model may residuals. This example shows how residuals can appear
overestimate seepage velocities because the simulated hydrau- uncorrelated in a model that does not represent essential
lic gradient is higher than the measured hydraulic gradient. characteristics of the physical hydrogeologic system, in this
Therefore this model may need to be improved if the heads are case by not reproducing the correct number of maxima and
to be input into a mass transport model. minima.
X1.3 Fig. X1.5 and Fig. X1.6 show sample plots of
residuals in plan and cross-section, as described in 6.1.3.3. In
Fig. X1.5, there are sufficient data to contour the residuals. The
contours indicate potentially significant correlations between
residuals in the northwest and southwest corners of the model.
Along the river, the residuals appear to be uncorrelated. In Fig.
X1.6, residuals were not contoured due to their sparseness and
apparent lack of correlation.

6
 D 5490

FIG. X1.5 Sample Contours of Residuals Plan View

FIG. X1.6 Sample Plot of Residuals Section View

7
 D 5490

FIG. X1.7 Sample Temporal Residuals

ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.

This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.

This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or [email protected] (e-mail); or through the ASTM website
(www.astm.org).