Page 7 - Volume 4, Issue 1, January 2011

Uncertainty in pest risk analysis

5

On the contrary, computation is straightfor-

ward for models that are less complex and

less computationally intensive.

Output values can be presented in dif-

ferent ways. In general, it is not appropriate

to present all the computed model outputs

because the number of computed values is

generally very high (i.e. several thousands).

The recommended approach is to summa-

rize the output distributions by calculating

several key parameters such as mean, me-

dian, standard deviation, coefficient of vari-

ation, several extreme percentiles (1%, 5%,

10%, 90%, 95%, 99%). It is also useful to

show some graphical presentations of the

computed model outputs, like histograms

and cumulative probability distributions. All

these techniques have been applied in sev-

eral quantitative risk assessments (e.g. Koch

et al

., 2009; Peterson

et al

., 2009). When the

model includes several output variables, it is

useful to analyse the relationships between

these variables by drawing scatter plots or

by computing correlation coefficients.

3.4. Step (iv). Computation of sensitivity

indices

Sensitivity of model output to an un-

certain factor is commonly studied by us-

ing simple graphical presentation of model

outputs

versus

model inputs (e.g. Koch

et al

.,

2009; Giovanelli

et al

., 2010). This approach is

useful but not sufficient to assess and com-

pare the influence of the different input fac-

tors in a quantitative way. It is recommend-

ed to compute sensitivity indices for all the

uncertain factors in order to rank these fac-

tors according to their influence on the out-

puts.

A sensitivity index is a measure of the in-

fluence of an uncertain factor on a model

output variable. Factors whose values have

a strong effect on the model are character-

ized by high sensitivity indices. Non-influen-

tial factors are characterized by low sensi-

tivity indices. Sensitivity indices can thus be

used to rank uncertain factors and identify

those which should be measured or estimat-

ed more accurately.

A great diversity of sensitivity indices

has been proposed (e.g. Saltelli

et al

., 2000).

In local SA, sensitivity indices are based on

derivative calculation and correspond to the

slopes of the model output in the input fac-

tor space at a given set of values. In global

SA, sensitivity indices can be computed us-

ing a variety of techniques like ANOVA, cor-

relation between input factors and model

outputs, Fourier series, Monte Carlo simu-

lations, etc. (Saltelli

et al

., 2000). Sensitivity

indices can be computed using statistical

software (e.g. the package sensitivity of the

statistical software R

http://cran.r-project.

org/web/packages/sensitivity/index.html)

or more specialized software, such as Simlab

(http://simlab.jrc.ec.europa.eu

/), @Risk, or

Crystal ball. Examples of calculation of ANO-

VA-based sensitivity indices in quantitative

pest risk assessment can be found in EFSA

(2008b). Examples of correlation-based sen-

sitivity indices are provided in the case study

presented at the end of this paper.

3.5. Specific methods for analysing un-

certainty in model equations

Many models are now available for es-

timating risk of entry, establishment, and

spread. In some cases, it is difficult to choose

the most appropriate model for a given

question. For example, five different models

were used to map invasive species distribu-

tion by Roura-Pascual

et al

. (2009) and these

models led to different predictions generat-

ing uncertainty about the potential distribu-

tional area.

Two approaches have been proposed to

deal with this uncertainty, model compari-

son and model mixing. The latter approach

is also called consensual predictions or en-

semble forecasting. Model comparison aims

at assessing several candidate models in or-

der to select the model with the best predic-

tive performance. Several criteria have been

proposed to assess models for predicting in-

vasion (e.g. Smith

et al

., 1999; Townsend Pe-

terson

et al

., 2008) and the most popular

criterion is probably the area under the Re-

ceiver Operating Characteristic (ROC) curve,

which measures the ability of models to dis-

criminate presence and absence locations.