6、Conclusions and
discussions.
Three kinds of subsampling-ANOVA schemes (single-, multiple- and
full-subsampling) have been proposed and analyzed in this study. The
applicability of different subsampling ANOVA schemes are illustrated
through one simplified model and a rainfall-runoff conceptual model. To
evaluate the performance of different subsampling ANOVA schemes, the
traditional sobol’s method is also used as benchmark in the study. The
main purpose is to investigate the influence of different subsampling
ANOVA schemes on sensitivity analyses results. Based on the case
studies, some findings can be concluded:
1) The subsampling effectively diminishes the bias introduced by the
biased variance estimator. In the application of subsampling ANOVA
method, the parameter’s individual sensitivity is related to the
subsampling scheme. The subsampling process will reduce the subsampled
parameter’s individual sensitivity and increase the non-subsampled
parameter’s individual sensitivity. In other words, the difference of
sampling densities among parameters has great influence on
quantification of parametric sensitivities in hydrologic modeling.
2) For full–subsampling ANOVA method, the deviation decreased with the
parameters levels increased. The variation of the obtained parameters
sensitivities is small and the order of parameters influences (i.e.
sensitivity) would not change for three 3 or more parameter levels.
3) Compared with sobol’s method, the subsampling ANOVA methods can
significantly reduce the calculation requirements while achieve similar
calculation accuracy. Particularly, in order to get reliable parameter
sensitivity results, the full-subsampling scheme is
necessary,
and the 3 or more parameter levels are recommended.
In this study, the sobol’s method is considered as the benchmark to
evaluate the performance of the developed subsampling ANOVA approaches.
Even though the subsampling ANOVA approaches may not produce better
results than the sobol’s method, the proposed subsampling ANOVA
approaches, especially for the full-subsampling ANOVA method, have their
own essential strengths. Firstly, the sobol’s algorithm has high
computational cost and the number of model evaluations required for the
sobol’s indices to converge increases rapidly with the number of
parameters, making its efficiency questionable for complex hydrological
models (Herman et al., 2013,
Zhang et al., 2013,
Khorashadi Zadeh et al., 2017,
Shin et al., 2013). In comparison, the
subsampling ANOVA approaches can effectively reduce the computational
demands and generate reliable results (as shown in Table 1 and Table 3).
The
number of model evaluations is equal to the number of combinations with
all the parameter levels. However, as indicated in this paper, the
full-subsampling ANOVA approach can generate acceptable results with
three or four levels for each parameter. Thus, the computational cost
would be reduced greatly. Secondly, besides sensitivity analysis for
parameters with continuous values (Qi et
al., 2016c), the single-subsampling ANOVA algorithms has already been
applied to analyze the sensitivity of discrete or non-numeric elements
such as the statistical post processing scheme, precipitation products
and the hydrological model (Bosshard et
al., 2013, Qi et al., 2016a,
Qi et al., 2016b). Consequently, the
developed multiple-/full-subsampling ANOVA approaches can also handle
with sensitivity analysis for both numeric and non-numeric variables.
However, the sobol’s approach can only deal with numeric variables.
The approaches proposed in this study just serve as a first basis for
the application of subsampling ANOVA in hydrological model sensitivity
analysis under multiple uncertainties. The number of levels would
probably be higher to ensure robustness with a more complex model. The
subsampling ANOVA algorithms can not only reduce the computing cost
greatly, but also analyze the sensitivity of discrete or non-numeric
elements. Further research is encouraged to examine the applicability of
the subsampling ANOVA approaches in other non-numeric elements
sensitivity analysis.