Open Access Research article

Concept annotation in the CRAFT corpus

Michael Bada1*, Miriam Eckert2, Donald Evans1, Kristin Garcia1, Krista Shipley1, Dmitry Sitnikov3, William A Baumgartner1, K Bretonnel Cohen1, Karin Verspoor14, Judith A Blake3 and Lawrence E Hunter1

Author affiliations

1 Department of Pharmacology, University of Colorado Anschutz Medical Campus, Aurora, CO, USA

2 Department of Linguistics, University of Colorado Boulder, Boulder, CO, USA

3 Mouse Genome Informatics, The Jackson Laboratory, Bar Harbor, ME, USA

4 Victoria Research Lab, National ICT Australia, Melbourne, VIC, 3010, Australia

For all author emails, please log on.

Citation and License

BMC Bioinformatics 2012, 13:161  doi:10.1186/1471-2105-13-161

Published: 9 July 2012

Abstract

Background

Manually annotated corpora are critical for the training and evaluation of automated methods to identify concepts in biomedical text.

Results

This paper presents the concept annotations of the Colorado Richly Annotated Full-Text (CRAFT) Corpus, a collection of 97 full-length, open-access biomedical journal articles that have been annotated both semantically and syntactically to serve as a research resource for the biomedical natural-language-processing (NLP) community. CRAFT identifies all mentions of nearly all concepts from nine prominent biomedical ontologies and terminologies: the Cell Type Ontology, the Chemical Entities of Biological Interest ontology, the NCBI Taxonomy, the Protein Ontology, the Sequence Ontology, the entries of the Entrez Gene database, and the three subontologies of the Gene Ontology. The first public release includes the annotations for 67 of the 97 articles, reserving two sets of 15 articles for future text-mining competitions (after which these too will be released). Concept annotations were created based on a single set of guidelines, which has enabled us to achieve consistently high interannotator agreement.

Conclusions

As the initial 67-article release contains more than 560,000 tokens (and the full set more than 790,000 tokens), our corpus is among the largest gold-standard annotated biomedical corpora. Unlike most others, the journal articles that comprise the corpus are drawn from diverse biomedical disciplines and are marked up in their entirety. Additionally, with a concept-annotation count of nearly 100,000 in the 67-article subset (and more than 140,000 in the full collection), the scale of conceptual markup is also among the largest of comparable corpora. The concept annotations of the CRAFT Corpus have the potential to significantly advance biomedical text mining by providing a high-quality gold standard for NLP systems. The corpus, annotation guidelines, and other associated resources are freely available at http://bionlp-corpora.sourceforge.net/CRAFT/index.shtml webcite.