Resolution:
## Figure 3.
Construction of confusion matrices for the semi-automated abstract screening strategy. The leftmost matrix represents citations that are labeled by the reviewer while
training the classification model. The middle matrix displays the predictions of the
trained model over the remaining unlabeled set of citations U. The rightmost matrix shows the corresponding crosstabulation at the end of "Level
1a" (see Figure 1). The quantities mentioned in this figure are used in the definition
of Yield and Burden, the chosen evaluation metrics (see Equations 1 and 2). Superscripts T and U refer to model training and applying the model to yet unlabeled citations, respectively.
tp^{[T|U]}: "true positives", tn^{[T|U]}: "true negatives", fp^{[T|U]}: "false positives", fn ^{[T|U]}: "false negatives". We assume that reviewers will never erroneously exclude a citation
that is eligible for systematic review, i.e. fn^{T }= 0.
Wallace |