Table 5

Results of meta-knowledge assignment and event extraction for EventMine-MK trained on the ST-MK corpus
Dimension Value Count R / P / F +GENIA (R/P/F) Majority (R/P/F)
KT Investigation 38 76.3 / 67.4 / 71.6 68.4 / 65.0 / 66.7 0.0 / 0.0 / 0.0
Analysis 132 70.5 / 69.9 / 70.2 72.7 / 73.3 / 73.0 0.0 / 0.0 / 0.0
Observation 412 80.6 / 77.4 / 79.0 78.6 / 75.2 / 76.9 100.0 / 40.5 / 57.7
Fact 59 33.9 / 76.9 / 47.1 40.7 / 70.6 / 51.6 0.0 / 0.0 / 0.0
Method 8 0.0 / 0.0 / 0.0 0.0 / 0.0 / 0.0 0.0 / 0.0 / 0.0
Other 368 77.4 / 73.8 / 75.6 76.6 / 74.0 / 75.3 0.0 / 0.0 / 0.0
Macro Average 1,017 56.5 / 60.9 / 57.3 56.2 / 59.7 / 57.3 16.7 / 6.8 / 9.6
Micro Average 1,017 74.6 / 74.6 / 74.6 73.9 / 73.9 / 73.9 40.5 / 40.5 / 40.5
CL L3 983 97.7 / 98.9 / 98.3 98.6 / 99.0 / 98.8 100.0 / 96.7 / 98.3
L2 30 73.3 / 48.9 / 58.7 76.7 / 62.2 / 68.7 0.0 / 0.0 / 0.0
L1 4 0.0 / 0.0 / 0.0 25.0 / 100.0 / 40.0 0.0 / 0.0 / 0.0
Macro Average 1,017 57.0 / 49.3 / 52.3 66.8 / 87.1 / 69.2 33.3 / 32.2 / 32.8
Micro Average 1,017 96.6 / 96.6 / 96.6 97.7 / 97.7 / 97.7 96.7 / 96.7 / 96.7
Polarity Positive 975 97.5 / 98.8 / 98.1 98.2 / 98.6 / 98.4 100.0 / 95.9 / 97.9
Negative 42 71.4 / 55.6 / 62.5 66.7 / 60.9 / 63.6 0.0 / 0.0 / 0.0
Macro Average 1,017 84.5 / 77.2 / 80.3 82.5 / 79.8 / 81.0 50.0 / 47.9 / 48.9
Micro Average 1,017 96.4 / 96.4 / 96.4 96.9 / 96.9 / 96.9 95.9 / 95.9 / 95.9
Manner High 56 78.6 / 63.8 / 70.4 76.8 / 65.2 / 70.5 0.0 / 0.0 / 0.0
Low 4 100.0 / 66.7 / 80.0 100.0 / 66.7 / 80.0 0.0 / 0.0 / 0.0
Neutral 957 97.2 / 98.7 / 97.9 97.4 / 98.6 / 98.0 100.0 / 94.1 / 97.0
Macro Average 1,017 91.9 / 76.4 / 82.8 91.4 / 76.8 / 82.8 33.3 / 31.4 / 32.3
Micro Average 1,017 96.2 / 96.2 / 96.2 96.3 / 96.3 / 96.3 94.1 / 94.1 / 94.1
Source Current 1,003 99.8 / 99.5 / 99.7 99.9 / 99.5 / 99.7 100.0 / 98.6 / 99.3
Other 14 64.3 / 81.8 / 72.0 64.3 / 90.0 / 75.0 0.0 / 0.0 / 0.0
Macro Average 1,017 82.1 / 90.7 / 85.9 82.1 / 94.8 / 87.4 50.0 / 49.3 / 49.7
Micro Average 1,017 99.3 / 99.3 / 99.3 99.4 / 99.4 / 99.4 98.6 / 98.6 / 98.6
Event (Task 1) 1,789 55.1 / 61.7 / 58.2
Event (Task 2) 1,795 54.1 / 60.8 / 57.2
Event (Task 3) 1,997 51.6 / 59.4 / 55.3

Meta-knowledge assignment performance is evaluated only on events correctly predicted by the system. Event extraction performance is evaluated using the ST evaluation tools. A majority class-based baseline is shown for reference. +GENIA shows the results obtained when the outputs of the GENIA-trained meta-knowledge assignment model are incorporated as additional features. (R)ecall/(P)recision/(F)score are reported.

Miwa et al.

Miwa et al. BMC Bioinformatics 2012 13:108   doi:10.1186/1471-2105-13-108

Open Data