Table 3

Estimation of consistency in cases of initial agreement. We collected 200-200 randomly chosen examples from each type of corpus text to assess the level of consistency in cases when the two students provided identical annotation for the sentence (identical means here that all cues and scope boundaries were exactly the same) and they were compared to the annotation provided by the chief annotator. The agreement rates are given here.

NEGATION

keyword:

98.65%

left scope:

97.27%

right scope:

98.64%

full scope:

95.91%

SPECULATION

keyword:

99.63%

left scope:

99.25%

right scope:

99.63%

full scope:

98.88%

Vincze et al. BMC Bioinformatics 2008 9(Suppl 11):S9   doi:10.1186/1471-2105-9-S11-S9