Table 9

Overall performance evaluation.

Relation

Metric

notok (best system: non-syntactic)

+syndist (best system: syntactic)

baseline

IAA

CIAA


has_finding

P

63

74

65

R

82

77

76

F1

71

74

70

46

80


has_indication

P

44

43

0

R

47

37

0

F1

41

39

0

26

50


has_location

P

73

73

0

R

83

86

0

F1

76

78

0

55

80


has_target

P

59

71

0

R

68

66

0

F1

62

64

0

42

63


laterality_modifies

P

86

89

60

R

89

90

91

F1

85

89

72

73

94


negation_modifies

P

81

85

81

R

93

93

98

F1

85

88

88

66

93


sub_location_modifies

P

87

93

50

R

95

95

68

F1

90

94

58

49

96


Overall

P

64

71

36

R

76

74

48

F1

70

72

41

47

75


System best performance figures (from Tables 4 and 5), and comparison to baseline performance and to inter-annotator agreement scores.

Roberts et al. BMC Bioinformatics 2008 9(Suppl 11):S3   doi:10.1186/1471-2105-9-S11-S3

Open Data