In keeping with GRADE guidance on grading the certainty of evidence in diagnostic test accuracy, we have considered the domains of imprecision and publication bias (19). There was considerable inconsistency in the reported sensitivity (ranging from 0.23 to 0.94) with minimal overlapping of the 95% CI (Figure 2). This did however improve when we considered only those studies looking at flow cytometric analysis of whole blood with a positive SI threshold of 2 (Figure 4). Specificity was found to be fairly consistent (ranging from 0.67 to 0.99). The specificity also demonstrated extensive overlapping of 95% CI, Figure 2, suggesting good consistency. Although there was variation in CI width for the reported sensitivity, the majority of studies (16 of 22, 73%) showed a 95% CI that was entirely above the sensitivity of 0.19 seen with sIgE, which is the relevant clinical comparison which we hope to improve upon with BAT. The 95% CI for specificity were much narrower than for sensitivity, demonstrating no need to lower the grading of the certainty of the evidence based on imprecision.
Publication bias was assessed for all 22 studies using a funnel plot (OLR Figure E1). The asymmetry suggests that there may be evidence of publication bias. However funnel plots may overestimate publication bias in meta-analyses of diagnostic test accuracy (23). Although one study showed BAT was more likely to be positive in those with a severe reaction (27), this work did not show any sensitivity-specificity relationship, and we have therefore not upgraded the certainty of evidence. Overall GRADE certainty of the evidence for sensitivity is “very low”, and for specificity is “low”, suggesting “the true effect might be markedly different from the estimated effect”.
Discussion
This work primarily highlights the significant variation in BAT sensitivity across all studies. Our primary finding from this work is that using flow cytometric analysis with an SI threshold of 2, BAT in penicillin allergy has an estimated summary point sensitivity of 51% (46% – 56%) and specificity of 89% (85% –93%). For comparison, sIgE, (another in vitro diagnostic recognized for use in penicillin allergy diagnosis), showed poorer sensitivity (19.3% (95%CI, 12 – 29)) but higher specificity (sIgE specificity of 97.4% (95% CI, 95.2%-98.6%)) than BAT (8).
Flow cytometric immediate analysis of whole blood was the most commonly described assay type (seventeen of twenty-two papers). All used CD63 as a marker of basophil activation. Only two studies looked at the use of CD203c as a marker of basophil activation, both suggested slightly improved performance over CD63 (25, 46). However only one of these defined the positive threshold used, and so comparative sub group analysis of could not be undertaken.
Subgroup analysis of the minimum number of identified basophils required for any single BAT test demonstrated very similar sensitivity and specificity, suggesting no statistically significant difference between the use of 500 or 1000 basophils. This is clinically pertinent as it suggested a smaller collection volume may suffice, thereby increasing the likelihood of collecting an usable sample from the patient.
One study looked at the use of a novel dendrimeric antigens (DeAns) as carrier molecules for benzylpenicilloyl and amoxicilloyl in dense and stable hapten-carrier conjugates (35). This did not provide any diagnostic benefit above the use of benzylpenicilloyl, amoxicilloyl or free penicillin in BAT in this small sample.
Two US studies were considered in the overall analysis and generation of an SROC curve (Figure 3). However, once we looked at studies using flow cytometric analysis of whole blood that also used the same positive threshold of an SI of 2 to generate out primary results of the summary point sensitivity and specificity (Figure 4), all twelve of these studies were in fact from European centres. Due to differences in prescribing practices, healthcare system structures and population genetics, these results may differ in different countries. Further work with greater geographic diversity would be valuable.