Table 5 |
|||||
| Results of meta-knowledge assignment and event extraction for EventMine-MK trained on the ST-MK corpus | |||||
| Dimension | Value | Count | R / P / F | +GENIA (R/P/F) | Majority (R/P/F) |
| KT | Investigation | 38 | 76.3 / 67.4 / 71.6 | 68.4 / 65.0 / 66.7 | 0.0 / 0.0 / 0.0 |
| Analysis | 132 | 70.5 / 69.9 / 70.2 | 72.7 / 73.3 / 73.0 | 0.0 / 0.0 / 0.0 | |
| Observation | 412 | 80.6 / 77.4 / 79.0 | 78.6 / 75.2 / 76.9 | 100.0 / 40.5 / 57.7 | |
| Fact | 59 | 33.9 / 76.9 / 47.1 | 40.7 / 70.6 / 51.6 | 0.0 / 0.0 / 0.0 | |
| Method | 8 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | 0.0 / 0.0 / 0.0 | |
| Other | 368 | 77.4 / 73.8 / 75.6 | 76.6 / 74.0 / 75.3 | 0.0 / 0.0 / 0.0 | |
| Macro Average | 1,017 | 56.5 / 60.9 / 57.3 | 56.2 / 59.7 / 57.3 | 16.7 / 6.8 / 9.6 | |
| Micro Average | 1,017 | 74.6 / 74.6 / 74.6 | 73.9 / 73.9 / 73.9 | 40.5 / 40.5 / 40.5 | |
| CL | L3 | 983 | 97.7 / 98.9 / 98.3 | 98.6 / 99.0 / 98.8 | 100.0 / 96.7 / 98.3 |
| L2 | 30 | 73.3 / 48.9 / 58.7 | 76.7 / 62.2 / 68.7 | 0.0 / 0.0 / 0.0 | |
| L1 | 4 | 0.0 / 0.0 / 0.0 | 25.0 / 100.0 / 40.0 | 0.0 / 0.0 / 0.0 | |
| Macro Average | 1,017 | 57.0 / 49.3 / 52.3 | 66.8 / 87.1 / 69.2 | 33.3 / 32.2 / 32.8 | |
| Micro Average | 1,017 | 96.6 / 96.6 / 96.6 | 97.7 / 97.7 / 97.7 | 96.7 / 96.7 / 96.7 | |
| Polarity | Positive | 975 | 97.5 / 98.8 / 98.1 | 98.2 / 98.6 / 98.4 | 100.0 / 95.9 / 97.9 |
| Negative | 42 | 71.4 / 55.6 / 62.5 | 66.7 / 60.9 / 63.6 | 0.0 / 0.0 / 0.0 | |
| Macro Average | 1,017 | 84.5 / 77.2 / 80.3 | 82.5 / 79.8 / 81.0 | 50.0 / 47.9 / 48.9 | |
| Micro Average | 1,017 | 96.4 / 96.4 / 96.4 | 96.9 / 96.9 / 96.9 | 95.9 / 95.9 / 95.9 | |
| Manner | High | 56 | 78.6 / 63.8 / 70.4 | 76.8 / 65.2 / 70.5 | 0.0 / 0.0 / 0.0 |
| Low | 4 | 100.0 / 66.7 / 80.0 | 100.0 / 66.7 / 80.0 | 0.0 / 0.0 / 0.0 | |
| Neutral | 957 | 97.2 / 98.7 / 97.9 | 97.4 / 98.6 / 98.0 | 100.0 / 94.1 / 97.0 | |
| Macro Average | 1,017 | 91.9 / 76.4 / 82.8 | 91.4 / 76.8 / 82.8 | 33.3 / 31.4 / 32.3 | |
| Micro Average | 1,017 | 96.2 / 96.2 / 96.2 | 96.3 / 96.3 / 96.3 | 94.1 / 94.1 / 94.1 | |
| Source | Current | 1,003 | 99.8 / 99.5 / 99.7 | 99.9 / 99.5 / 99.7 | 100.0 / 98.6 / 99.3 |
| Other | 14 | 64.3 / 81.8 / 72.0 | 64.3 / 90.0 / 75.0 | 0.0 / 0.0 / 0.0 | |
| Macro Average | 1,017 | 82.1 / 90.7 / 85.9 | 82.1 / 94.8 / 87.4 | 50.0 / 49.3 / 49.7 | |
| Micro Average | 1,017 | 99.3 / 99.3 / 99.3 | 99.4 / 99.4 / 99.4 | 98.6 / 98.6 / 98.6 | |
| Event (Task 1) | 1,789 | 55.1 / 61.7 / 58.2 | |||
| Event (Task 2) | 1,795 | 54.1 / 60.8 / 57.2 | |||
| Event (Task 3) | 1,997 | 51.6 / 59.4 / 55.3 | |||
Meta-knowledge assignment performance is evaluated only on events correctly predicted by the system. Event extraction performance is evaluated using the ST evaluation tools. A majority class-based baseline is shown for reference. +GENIA shows the results obtained when the outputs of the GENIA-trained meta-knowledge assignment model are incorporated as additional features. (R)ecall/(P)recision/(F)score are reported.
Miwa et al. BMC Bioinformatics 2012 13:108 doi:10.1186/1471-2105-13-108
