Figure 4. Boxplots for
100 values of \(\text{SS}D_{\theta}\) for Case I, when designing three
sequential A-optimal experiment all at once using Bayes-Bayes, LO-LO,
Bayes-LO, and LO-Bayes approaches
Computation times for both Bayesian and LO approaches to MBDOE were
compared using MC simulations for the Bayes-Bayes and LO-Bayes
approaches. These two approaches use the same approach for parameter
estimation, but different methods for MBDOE, making it possible to
isolate the effects of Bayesian and LO approaches for MBDoE. Using a
core i5 laptop with 8 GB RAM, the average computations time for each
Bayes-Bayes run was 51.2 s, which is faster than 89.9 s on average for a
LO-Bayes run. Although this difference is relatively small for the
current case study, we anticipate that larger differences could occur
for larger models with more parameters and decision variables.
Case II: Results when moderately-informative initial guesses
are
available
Figure 5 compares boxplots for the 100 values of \(\text{SS}D_{\theta}\)for four approaches when designing three A-optimal experiments one at a
time using the prior parameter guesses in Case II. Similar patterns are
observed compared to Case I: as more experiments are designed and more
data become available, the mean and median values of boxplots for\(\text{SS}D_{\theta}\) becomes smaller. However, since the parameter
initial guesses are not as good as in Case I, the parameter estimates in
Case II are less accurate than in Case I. As in Case I, the Bayes-LO
approach provides the best parameter values on average. A key difference
between the results in Case I and Case II is that the LO approach tended
to estimate more parameters in Case II, due to higher initial parameter
uncertainties. Details concerning the frequency with which different
parameters were estimated are provided in the Supplementary Information.
Case III: Results when misinformed initial guesses are
used
Figure 6 shows boxplots for 100 values of \(\text{SS}D_{\theta}\)obtained for Case III, with the misinformed parameter initial guesses
described in the third row of Table 8. As expected, because parameter
initial guesses were worse than in Case I and II, the mean and median
for \(\text{SS}D_{\theta}\) are larger than Case I and II for all four
approaches. The Bayes-LO approach to designing experiments and
estimating parameters was the best approach, even though the modeler
believed he or she had better prior knowledge about the parameters than
was justifiable. These results suggest that the Bayes-LO approach is
somewhat robust to specification of misinformed prior information. Note
that the settings in Case II and Case III were also used to design three
experiments all-at-once instead of one at a time. As in Case I, the
Bayes-LO approach was the best and the parameter estimations resulting
from the three-at-once experiments were not as good as those obtained
using the one-at-a-time approach. Details are provided in the
Supplementary Information.