This article is part of the supplement: Proceedings of the 2009 AMIA Summit on Translational Bioinformatics
Comparison of concept recognizers for building the Open Biomedical Annotator
- Equal contributors
1 Centre for Biomedical Informatics, Stanford University, Stanford, CA 94305, USA
2 Department of Computer Science, Stanford University, Stanford, CA 94305, USA
BMC Bioinformatics 2009, 10(Suppl 9):S14 doi:10.1186/1471-2105-10-S9-S14Published: 17 September 2009
The National Center for Biomedical Ontology (NCBO) is developing a system for automated, ontology-based access to online biomedical resources. The system's indexing workflow processes the text metadata of diverse resources such as datasets from GEO and ArrayExpress to annotate and index them with concepts from appropriate ontologies. This indexing requires the use of a concept-recognition tool to identify ontology concepts in the resource's textual metadata. In this paper, we present a comparison of two concept recognizers – NLM's MetaMap and the University of Michigan's Mgrep. We utilize a number of data sources and dictionaries to evaluate the concept recognizers in terms of precision, recall, speed of execution, scalability and customizability. Our evaluations demonstrate that Mgrep has a clear edge over MetaMap for large-scale service oriented applications. Based on our analysis we also suggest areas of potential improvements for Mgrep. We have subsequently used Mgrep to build the Open Biomedical Annotator service. The Annotator service has access to a large dictionary of biomedical terms derived from the United Medical Language System (UMLS) and NCBO ontologies. The Annotator also leverages the hierarchical structure of the ontologies and their mappings to expand annotations. The Annotator service is available to the community as a REST Web service for creating ontology-based annotations of their data.