Introduction
Mathematical models are used in chemical and pharmaceutical industries
for analysis, design and control of chemical processes and for
maximizing product quality and profit.1,2 Especially
in pharmaceutical industries, models are important for Quality by Design
and development of continuous manufacturing processes, which are
becoming more widespread.3–5 Mathematical models for
pharmaceutical product development can be either empirical or
mechanistic.5–7 Although empirical models are
commonly used for pharmaceutical processes, they cannot reliably predict
the system behavior outside the range of operating conditions used for
model development.8 Therefore, fundamental models,
based on underlying chemistry and physics, are
preferred.9 These models usually contain unknown
parameters that require estimation using experimental
data.10 To obtain informative data, it is advantageous
to carefully plan the experiments aimed at parameter estimation using
design of experiment (DoE) techniques.11 As shown in
Table 1, optimal model-based design-of-experiments (MBDoE) techniques
select experiments to minimize uncertainties in parameters estimates or
model predictions.12–14 MBDoE techniques are
effective because they account for the structure of the model as well as
parameter and measurement uncertainties when selecting new run
conditions.13,15 Other benefits of MBDoE techniques,
compared to traditional factorial designs, are that they can be readily
used to design any number of experiments, e.g., one, three or seven
experiments, depending on available resources for
experimentation.15,16 MBDoE techniques have been
developed to satisfy a variety of objectives including minimizing total
variances of parameter estimates, minimizing the average variance of
model predictions, and designing experiments for model
discrimination.16
Table 1 shows several MBDoE objective functions that have been used for
development of chemical and pharmaceutical production
models.15–17 If modelers are interested in obtaining
accurate parameter estimates for their model, A-, D- or E- optimal
designs can be selected.15–17 Alternatively, G- and
V-optimal designs focus on obtaining accurate model predictions at
specified operating conditions of interest to the
modeler.18–21 All of these MBDoE techniques in Table
1 require computation of the inverse of the Fisher Information Matrix
(FIM ) when selecting experimental
settings.21–23 The FIM carries information
about how changes in parameter values can affect the model predictions
and is therefore crucial for both MBDoE calculations and parameter
inference.24 For nonlinear models, which are common in
chemical and pharmaceutical applications, computation of theFIM requires linearizing the model around some nominal
parameter values.17,25 If these nominal parameter
values are significantly different from the corresponding true values,
the selected MBDoE settings may lead to experimental data that are not
very informative.17,25 Sequential design approaches
are appealing because they enable updating of the parameter values, as
well as the experimental strategy, as more data become
available.26 Using sequential experimental designs,
valuable information from old experimental data can be used, which might
have been collected for other objectives than model
development.27,28
Computation of the objective functions for sequential MBDoE is
problematic if the FIM is noninvertible or ill-conditioned.
Typical causes are limited experimental data, strongly correlated
influences of different parameters, and parameters with little or no
influence on the model predictions.29 In chemical,
biochemical and pharmacological systems, models often contain a large
number of kinetic and transport parameters (e.g., 10-80 parameters)
which may result in noninvertible/ill-conditionedFIM s.30–34 To avoid this problem, several
approaches have been considered during sequential MBDoE calculations
including parameter subset selection,14,29,35pseudoinverse methods,21,36 Tikhonov
regularization,37–40 and Bayesian
approaches.13,41,42
The parameter-subset-selection approach uses a model-reduction
perspective.35,43,44 In one methodology, parameters
are ranked from most-estimable to least-estimable so that problematic
(low-ranked) parameters can be recognized and fixed at their nominal
values.35,45 In this way, experiments can be designed
using a well-conditioned reduced FIM that ignores problematic
parameters. Alternatively, pseudoinverse methods approximate the inverse
of the FIM (e.g., using the Moore-Penrose pseudoinverse) during
MBDoE calculations.21,36,46 In Tikhonov
regularization, a penalty is added to diagonal elements of theFIM to make it invertible.29,38–40 Bayesian
MBDoE using linear models results in Tikhonov penalties that account for
prior knowledge about parameters. However, for nonlinear models, the
situation can be considerably more complex, depending on how the
nonlinearity is treated.13,41,42 There is little
information in the literature regarding which approach is most
effective. In two previous articles, we considered pharmaceutical case
studies involving noninvertible FIM s. Two different approaches
were compared: i) a subset-selection-based approach that leaves out
problematic parameters (LO approach) and ii) a simpler approach that
uses a Moore-Penrose pseudoinverse in place of\(\mathbf{\text{FI}}\mathbf{M}^{\mathbf{-1}}\)(PI
approach).21,46 These case studies suggest that the LO
approach is often superior to the PI approach for designing both A- and
V-optimal experiments.21,46 A shortcoming of the LO
approach is that it can be complicated and computationally expensive due
to changes in the subset of parameters that is left out during MBDoE
calculations. This complication motivates us to find a more convenient
approach to deal with singular FIMs during MBDoE.
The focus of the current study is on a simplified Bayesian approach for
dealing with singular/ill-conditioned FIMs during MBDoE.
Bayesian approaches have been used in several past MBDoE studies for
chemical and biochemical systems.47–49 The main
benefit of the Bayesian MBDoE framework is that it accounts for prior
knowledge about plausible values of the model
parameters.13 However, many researchers raise concerns
about the use of Bayesian approaches in practical engineering
systems.13,50 Disadvantages of the Bayesian approach
include uncertainty about the reliability of assumptions made when
specifying prior information.49–51 Undesirable
computational complexity can also arise, depending on the assumptions
that are made. As a result, Bayesian MBDoE has not enjoyed widespread
applications in chemical process modeling.
Table 1. Optimality criteria for model-based design of
experiments 21,46