This article is part of the supplement: A critical assessment of text mining methods in molecular biology .

Report

Overview of BioCreAtIvE task 1B: normalized gene lists

Lynette Hirschman , Marc Colosimo , Alexander Morgan and Alexander Yeh

The MITRE Corporation, 202 Burlington Road, Bedford, MA 01730, USA

author email corresponding author email

BMC Bioinformatics 2005, 6(Suppl 1):S11doi:10.1186/1471-2105-6-S1-S11

Published:	24 May 2005

Abstract

Background

Our goal in BioCreAtIve has been to assess the state of the art in text mining, with emphasis on applications that reflect real biological applications, e.g., the curation process for model organism databases. This paper summarizes the BioCreAtIvE task 1B, the "Normalized Gene List" task, which was inspired by the gene list supplied for each curated paper in a model organism database. The task was to produce the correct list of unique gene identifiers for the genes and gene products mentioned in sets of abstracts from three model organisms (Yeast, Fly, and Mouse).

Results

Eight groups fielded systems for three data sets (Yeast, Fly, and Mouse). For Yeast, the top scoring system (out of 15) achieved 0.92 F-measure (harmonic mean of precision and recall); for Mouse and Fly, the task was more difficult, due to larger numbers of genes, more ambiguity in the gene naming conventions (particularly for Fly), and complex gene names (for Mouse). For Fly, the top F-measure was 0.82 out of 11 systems and for Mouse, it was 0.79 out of 16 systems.

Conclusion

This assessment demonstrates that multiple groups were able to perform a real biological task across a range of organisms. The performance was dependent on the organism, and specifically on the naming conventions associated with each organism. These results hold out promise that the technology can provide partial automation of the curation process in the near future.