Background Information

Fisher Information Matrix for Nonlinear Models

Consider the nonlinear model:
\(\mathbf{Y=g}\left(\mathbf{d,\theta}\right)\mathbf{+\varepsilon}\)(1)
where \(\mathbf{Y\ \in\ }\mathbf{R}^{N}\) is a vector of stacked measured responses, g is the solution of equations that describe the system, \(\mathbf{d}\in\mathbf{R}^{r\times D}\) is a matrix of experimental settings (for \(r\) runs with \(D\) decision variables specified for each), \(\mathbf{\theta}\in\mathbf{R}^{p}\) is the vector of model parameters and\(\mathbf{\varepsilon}\mathbf{\in}\mathbf{R}^{N}\) is a vector of a measurement noise with diagonal covariance matrix\(\mathbf{\Sigma}_{\mathbf{y}}\in\mathbf{R}^{N\times N}\). For dynamic multi-response models with \(n\) sample times per run and \(v\)response variables, the total number of data values is \(N=nvr\). TheFIM is computed using a parametric sensitivity matrixS \({\in\mathbf{R}}^{N\times p}\) with elements:
\(S_{\text{ij}}=\left.\ \frac{\partial g\left(\mathbf{d},\mathbf{\theta}\right)}{\partial\theta_{j}}\right|_{{\hat{\theta}}_{k\neq j}}\)(2)
computed by linearizing the model around the best currently-available parameter values:53
The elements of \(\mathbf{S}\) should be scaled using parameter uncertainties \(s_{\theta_{j}}\) and measurement uncertainties\(s_{y_{i}}\) to reflect the modeler’s prior knowledge:54
\(Z_{\text{ij}}=S_{\text{ij}}\frac{s_{\theta_{j}}}{s_{y_{i}}}\) (3)
resulting in a scaled sensitivity matrix Z . The FIM is related to Z by:
\(\mathbf{FIM=}\mathbf{Z}^{T}\mathbf{Z}\) (4)
When performing sequential MBDoE calculations, Z contains two parts:21,46
\(\mathbf{Z=}\par \begin{bmatrix}\mathbf{Z}_{\mathbf{\text{old}}}\\ \mathbf{Z}_{\mathbf{\text{new}}}\\ \end{bmatrix}\) (5)
where \(\mathbf{Z}_{\mathbf{\text{old}}}\) corresponds to experimental settings and data from old experiments. The elements of\(\mathbf{Z}_{\mathbf{\text{old}}}\) are fixed during sequential MBDoE and elements of \(\mathbf{Z}_{\mathbf{\text{new}}}\) are determined by the optimizer. After each sequential design, elements of\(\mathbf{Z}_{\mathbf{\text{old}}}\) are updated based on the new parameter values and the number of rows in\(\mathbf{Z}_{\mathbf{\text{old}}}\) increases due to the recent experiments.

Parameter estimation with a noninvertible FIM

When estimating parameters, the FIM should be invertible, otherwise unique estimates for the parameters cannot be obtained.22,29 Several regularization approaches have been used to overcome this problem.38,39,55 One popular approach is to estimate a subset of the model parameters that are estimable, with the remaining parameters fixed at nominal values.23,45,56 Table 2 shows computational steps for a commonly used orthogonalization-based approach that ranks parameters from the most-estimable so problematic (unranked) parameters that lead to a noninvertible FIM can be determined.45,54 The ranking starts by computing the magnitude of each column of the scaled sensitivity matrix Z(Step 1). The parameter corresponding to the column with the highest magnitude is selected as the most-estimable parameter (Step 2). The columns of Z are then regressed onto columns of\(\mathbf{X}_{k}\), a matrix that contains columns from Z that correspond to the ranked parameters (Step 3). Residual matrix\(\mathbf{R}_{k}\) is then computed to remove correlation between columns for the unranked parameters and columns for the parameters that have already been ranked (Step 4). The next-most-estimable parameter is the one with the largest magnitude among columns of\(\mathbf{R}_{\mathbf{k}}\). In Step 5, the column corresponding to the next-most-estimable parameter is selected from the original\(\mathbf{Z}\) matrix and included in \(\mathbf{X}_{k}\), resulting in matrix \(\mathbf{X}_{k+1}\). Steps two to five are repeated to produce a ranked list with up to \(p\) parameters. The ranking stops when all of the parameters are ranked or at the iteration where\(\mathbf{X}_{k}^{T}\mathbf{X}_{k}\) (the reduced FIM ) becomes noninvertible. The remaining unranked parameters are categorized as problematic. They either have very little influence on the predicted responses or highly correlated effects with parameters on the ranked list.45,57 Using this orthogonalization-based ranking approach prior to parameter estimation helps to avoid numerical problems that would arise due to a noninvertible FIM .
Table 2. Orthogonalization algorithm 45,54