The gene normalization task in BioCreative III
Authors:
- Zhiyong LuZLZhiyong Lu
- Hung-Yu KaoHKHung-Yu Kao
- Chih-Hsuan WeiCWChih-Hsuan Wei
- Minlie HuangMHMinlie Huang
- Jingchen LiuJLJingchen Liu
- Cheng-Ju KuoCKCheng-Ju Kuo
- Chun-Nan HsuCHChun-Nan Hsu
- Richard Tzong-Han TsaiRTRichard Tzong-Han Tsai
- Hong-Jie DaiHong-Jie Dai
- Naoaki OkazakiNONaoaki Okazaki
- Han-Cheol ChoHCHan-Cheol Cho
- Martin GernerMGMartin Gerner
Open Access
- 3 October 2011
- journal article
- research article
- Published by Springer Nature in BMC Bioinformatics
- Vol. 12 (S8) , S2
Abstract
We report the Gene Normalization (GN) challenge in BioCreative III where participating teams were asked to return a ranked list of identifiers of the genes detected in full-text articles. For training, 32 fully and 500 partially annotated articles were prepared. A total of 507 articles were selected as the test set. Due to the high annotation cost, it was not feasible to obtain gold-standard human annotations for all test articles. Instead, we developed an Expectation Maximization (EM) algorithm approach for choosing a small number of test articles for manual annotation that were most capable of differentiating team performance. Moreover, the same algorithm was subsequently used for inferring ground truth based solely on team submissions. We report team performance on both gold standard and inferred ground truth using a newly proposed metric called Threshold Average Precision (TAP-k).Keywords
This publication has 35 references indexed in Scilit:
- GeneTUKit: a software for document-level gene normalizationBioinformatics, 2011
- Threshold Average Precision (TAP-k): a measure of retrieval designed for bioinformaticsBioinformatics, 2010
- LINNAEUS: A species name identification system for biomedical literatureBMC Bioinformatics, 2010
- Disambiguating the species of biomedical named entities using natural language parsersBioinformatics, 2010
- iRefWeb: interactive analysis of consolidated protein interaction data and their supporting evidenceDatabase: The Journal of Biological Databases and Curation, 2010
- The WEKA data mining softwareACM SIGKDD Explorations Newsletter, 2009
- Integrating text mining into the MGI biocuration workflowDatabase: The Journal of Biological Databases and Curation, 2009
- A bioinformatics analysis of the cell line nomenclatureBioinformatics, 2008
- OntoGene in BioCreative IIGenome Biology, 2008
- Integrating high dimensional bi-directional parsing models for gene mention taggingBioinformatics, 2008