Recogniser evaluation and score cut-off
For each template we tested a range of score cut-offs, which is a
user-defined similarity threshold at which a template will return a
detection. This threshold alters the proportion of true positive and
false positive detections and is, therefore, an important part of
optimising call recognisers (Katz et al., 2016a). For each species, we
tested the recognisers at a low score cut-off of 3; thus, any call
instance that scored 3 or higher was returned as a detection by monitoR.
Optimal score cut-off for each template was determined by constructing
receiver operating characteristic (ROC) curves, a diagnostic tool that
optimises that trade-off between false positive and true positive rates
(Figure 2). We calculated true and false positive rates at score cut-off
increments of 0.2 and determined the optimum as the score cut-off where
true positives were greatest relative to false positives (i.e. the peaks
in Figure 2). We then retained these score cut-offs for the recogniser
evaluation.