Table 5

Inter-annotator agreement.

GO terms

Curator 1 +2

Curator 1+3

Curator 2+ 3

Average


Exact

47

35

35

39

Same Lineage

15

20

19

18

New Lineage

56

39

35

43

Correct

107

91

85

94

Incorrect

11

3

4

18

TOTAL

118

94

89

100

Precision

0.91

0.96

0.96

0.94

Recall

0.70

0.72

0.73

0.72

F-measure

0.79

0.82

0.83

0.82


Where precision is the fraction of manual GO term annotations that are correct (number of correct annotations / (number of correct annotations + number of incorrect annotations). Recall is defined as the fraction of correct GO term annotations that were successfully retrieved during manual annotation (number of correct annotations / number of correct annotations + (number of annotations from new lineage - number of incorrect annotations). New lineage annotations minus incorrect annotations represent total number of the GO terms that the curators should have correctly retrieved from the paper. F-measure = (balanced precision and recall) = 2 × P × R/(P+R).

Camon et al. BMC Bioinformatics 2005 6(Suppl 1):S17   doi:10.1186/1471-2105-6-S1-S17

Open Data