Professional Documents
Culture Documents
Sensitivity Analyses: A Brief Tutorial With R Package Pse, Version 0.1.2
Sensitivity Analyses: A Brief Tutorial With R Package Pse, Version 0.1.2
This document presents a brief practical tutorial about the use of sensitivity
analyses tools in the study of ecological models. To read about the underlying
theory, please refer to our work in [1].
We presume for this tutorial that you are already familiar with the R programing
environment and with your chosen model. We will illustrate the techniques using a
simple model of population growth.
You should have installed R 1 along with an interface and text editor of your lik-
ing, the packages “sensitivity”, “Hmisc” and “ks”, and have downloaded the functions
available on our package “pse” available on
http://ecologia.ib.usp.br/let/doku.php?id=engl:tutorials:rcode
1 Input parameters
The first thing that must be done is to determine exactly what are the input pa-
rameters to your model. You should list which parameters should be investigated,
what are the probability distribution functions (PDFs) from which the parameter
values will be calculated, and what are the arguments to these PDFs.
In the examples, we will use a simple model of logistical population growth, in
which a population has an intrinsic rate of growth r, a carrying capacity of K and a
starting population of X0 . In each time step, the population may grow or diminish
after the following expression:
1
Parameter Distribution Arguments
r normal µ = 1.7, σ = 0.15
K normal µ = 40, σ = 1
X0 uniform min = 1, max = 50
We next translate this table to three R objects that will be used in the sensitivity
analyses, containing the names of the parameters, the distribution functions, and a
list containing the lists with all the parameters to the distribution functions:
2 Your model
The model that you wish to analyse must be formulated as an R function that
receives a data.frame, in which every column represent a different parameter, and
2
every line represents a different combination of values for those parameters. The
function must return an array with the same number of elements as there were lines
in the original data frame, and each entry in the array should correspond to the
result of running the model with the corresponding parameter combination. We
will cover the case in which a model outputs more than a single number in section
4.
If your model is already written in R, and accepts a single combination of values,
it is easy to write a “wrapper” using the function mapply to your model. In the
example below, the function oneRun receives three numbers, corresponding to r,
K and X0 , and returns a single value corresponding to the final population. The
function modelRun encapsulates this function, in a manner to receive a data.frame
containing all parameter combinations and returning the results in one array.
Make SURE that the order in which the parameters are defined above is the
same in which they are being passed to the function.
3
> library(pse)
> myLHS <- LHS(modelRun, factors, 200, q, q.arg, nboot=50)
The extra parameter nboot is used to bootstrap the correlation coefficients (see
below).
To access the values of the parameters used in the model, use the function
get.data(myLHS). To access the results, use get.results(myLHS).
With the object returned by the function LHS, we will exemplify in this section
four techniques that can be used: the uncertainty analysis using the ecdf, scatter-
plots of the correlation between each parameter and the result using the function
corPlot, partial rank correlation using the function plotprcc and agreement be-
tween different hypercube sizes with the function sbma.
4
3.1 ECDF
The ecdf, short for empirical cumulative distribution function, may be used to
illustrate the distribution of the model results, in our case the final population.
With this graph, we can see that the final population for our species is, with high
probability, between 35 and 45.
> plotecdf(myLHS)
1.0
0.8
Proportion <= x
0.6
0.4
0.2
0.0
32 34 36 38 40 42 44
LHS$res[, index.res]
n:200 m:0
5
3.2 Scatterplots
Here, we can see a scatterplot of the result as a function of each parameter. As all
the parameters are being changed for each run of the model, the scatterplots look
like they were randomly generated, even if the underlying model is deterministic.
This weird behaviour with larger values of r is an example of a bifurcation, and
will ultimately lead to chaotic solutions [3].
> corPlot(myLHS)
44
44
44
42
42
42
40
40
40
V1
V1
V1
38
38
38
36
36
36
34
34
34
32
32
32
r K X0
6
3.3 Partial correlation
The partial (rank) correlation coefficient (pcc or prcc) measures how strong are the
linear associations between the result and each input parameter, after removing the
linear effect of the other parameters.
The confidence intervals shown in this plot were generated by bootstraping.
> plotprcc(myLHS)
O1
1.0
0.5
0.0
0
−0.5
−1.0
r K X0
V1
r -1.521785833
K 0.982111569
X0 0.001878891
7
3.4 Agreement between runs
In order to decide whether our sample size was adequate or insufficient, we calculate
the Symmetric Blest Measure of Agreement (SBMA) between the PRCC coeffients
of two runs with different sample sizes.
[1] 0.375
8
4 Multiple response variables
In the previous section, we have examined a model that returned a single number,
namely, the final population. However, we might be interested in examining the
effects of the parameters in several distinct responses from the model. In our exam-
ple, we are interested in determining the effect of the parameters to the population
in each of the first 6 time steps. The theory and tools for this analysis remain
mostly the same. We will write our model to return an array now, as:
The hypercube is generated exactly in the same way. However, now we have
the option to give names (which will be used in the plots below) to each response
variable.
9
4.1 ECDF
The first plot we will produce will, again, be the ECDF. We may produce several
plots using the parameter “stack=FALSE”, or stack all the plots in the same graph,
using “stack=TRUE”:
2
0.8
Proportion <= x
0.6
4
0.4
0.2
6
0.0
3
1 5
10 20 30 40 50
dat
n:600 m:0
We may notice that the population values are spread over a wider range in the
first time steps, but converge to a narrow distribution on time 6.
10
4.2 Scatterplots
Next, we investigate the correlation plots for the variables with each input parame-
ter. To reduce the number of plots, we will present results just for the time steps 1,
3 and 6, using the “index.res” parameter, and supress the linear model from being
ploted with the parameter “add.lm”:
48
40
44
30
Time 1
Time 3
Time 6
20
40
10
36
1.4 1.6 1.8 2.0 1.4 1.6 1.8 2.0 1.4 1.6 1.8 2.0
r r r
20 25 30 35 40 45
48
40
44
30
Time 1
Time 3
Time 6
20
40
10
36
38 39 40 41 42 38 39 40 41 42 38 39 40 41 42
K K K
20 25 30 35 40 45
48
40
44
30
Time 1
Time 3
Time 6
20
40
10
36
0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50
X0 X0 X0
11
4.3 Partial correlation
The partial correlation plots also accept the “index.res” argument:
1.0
1.0
0.5
0.5
0.5
0.0
0.0
0.0
0
0
−0.5
−0.5
−0.5
−1.0
−1.0
−1.0
r K X0 r K X0 r K X0
12
4.4 Agreement between runs
We have seen how the function sbma measures the agreement between two runs
in the previous section. Now, we will use the function target.sbma to run several
hypercubes until a pair of runs provides us with an agreement equal to or better
than a limit specified in its first argument.
Then run the model using the data. To incorporate the results into the LHS
object, use the function tell:
Then you may proceed with the analyses using prcc, ecdf, etc.
13
References
[1] A. Chalom and P.I.K.L. Prado, Parameter space exploration of ecological models,
arXiv:1210.6278 [q-bio.QM], 2012.
14