Skip to main content

Benchmarking Studies

Benchmarking

Guest Editors: Olga Vitek and Mark D. Robinson

Tools and methods are essential for all kinds of genomic and post-genomic studies. As increasing numbers of methods are published in certain fields, it can be difficult to keep track of best practices for their use. Large scale studies that benchmark these methods on a wide range of datasets can be extremely useful to the scientific community. Genome Biology has launched the special issue on Benchmarking Studies to provide a snapshot of methodological challenges of different methods and tools.


  1. Chloroplasts are intracellular organelles that enable plants to conduct photosynthesis. They arose through the symbiotic integration of a prokaryotic cell into an eukaryotic host cell and still contain their o...

    Authors: Jan A. Freudenthal, Simon Pfaff, Niklas Terhoeven, Arthur Korte, Markus J. Ankenbrand and Frank Förster
    Citation: Genome Biology 2020 21:254
    Content type: Research Published on:

  2. Positional weight matrix (PWM) is a de facto standard model to describe transcription factor (TF) DNA binding specificities. PWMs inferred from in vivo or in vitro data are stored in many databases and used in...

    Authors: Giovanna Ambrosini, Ilya Vorontsov, Dmitry Penzar, Romain Groux, Oriol Fornes, Daria D. Nikolaeva, Benoit Ballester, Jan Grau, Ivo Grosse, Vsevolod Makeev, Ivan Kulakovskiy and Philipp Bucher
    Citation: Genome Biology 2020 21:114
    Content type: Research Published on:

  3. Recent advancements in next-generation sequencing have rapidly improved our ability to study genomic material at an unprecedented scale. Despite substantial improvements in sequencing technologies, errors pres...

    Authors: Keith Mitchell, Jaqueline J. Brito, Igor Mandric, Qiaozhen Wu, Sergey Knyazev, Sei Chang, Lana S. Martin, Aaron Karlsberg, Ekaterina Gerasimov, Russell Littman, Brian L. Hill, Nicholas C. Wu, Harry Taegyun Yang, Kevin Hsieh, Linus Chen, Eli Littman…
    Citation: Genome Biology 2020 21:71
    Content type: Research Published on:

  4. Genome-wide pooled CRISPR-Cas-mediated knockout, activation, and repression screens are powerful tools for functional genomic investigations. Despite their increasing importance, there is currently little guid...

    Authors: Sunil Bodapati, Timothy P. Daley, Xueqiu Lin, James Zou and Lei S. Qi
    Citation: Genome Biology 2020 21:62
    Content type: Review Published on:

  5. The initiation and subsequent evolution of cancer are largely driven by a relatively small number of somatic mutations with critical functional impacts, so-called driver mutations. Identifying driver mutations...

    Authors: Hu Chen, Jun Li, Yumeng Wang, Patrick Kwok-Shing Ng, Yiu Huen Tsang, Kenna R. Shaw, Gordon B. Mills and Han Liang
    Citation: Genome Biology 2020 21:43
    Content type: Research Published on:

  6. Many functional analysis tools have been developed to extract functional and mechanistic insight from bulk transcriptome data. With the advent of single-cell RNA sequencing (scRNA-seq), it is in principle poss...

    Authors: Christian H. Holland, Jovan Tanevski, Javier Perales-Patón, Jan Gleixner, Manu P. Kumar, Elisabetta Mereu, Brian A. Joughin, Oliver Stegle, Douglas A. Lauffenburger, Holger Heyn, Bence Szalai and Julio Saez-Rodriguez
    Citation: Genome Biology 2020 21:36
    Content type: Research Published on:

  8. Large-scale single-cell transcriptomic datasets generated using different technologies contain batch-specific systematic variations that present a challenge to batch-effect removal and data integration. With c...

    Authors: Hoa Thi Nhu Tran, Kok Siong Ang, Marion Chevrier, Xiaomeng Zhang, Nicole Yee Shin Lee, Michelle Goh and Jinmiao Chen
    Citation: Genome Biology 2020 21:12
    Content type: Research Published on:

  9. With the expanding applications of mass cytometry in medical research, a wide variety of clustering methods, both semi-supervised and unsupervised, have been developed for data analysis. Selecting the optimal ...

    Authors: Xiao Liu, Weichen Song, Brandon Y. Wong, Ting Zhang, Shunying Yu, Guan Ning Lin and Xianting Ding
    Citation: Genome Biology 2019 20:297
    Content type: Research Published on:

  10. Sequencing technology and assembly algorithms have matured to the point that high-quality de novo assembly is possible for large, repetitive genomes. Current assemblies traverse transposable elements (TEs) and...

    Authors: Shujun Ou, Weija Su, Yi Liao, Kapeel Chougule, Jireh R. A. Agda, Adam J. Hellinga, Carlos Santiago Blanco Lugo, Tyler A. Elliott, Doreen Ware, Thomas Peterson, Ning Jiang, Candice N. Hirsch and Matthew B. Hufford
    Citation: Genome Biology 2019 20:275
    Content type: Research Published on:

    The Correspondence to this article has been published in Genome Biology 2024 25:4

  12. The Critical Assessment of Functional Annotation (CAFA) is an ongoing, global, community-driven effort to evaluate and improve the computational annotation of protein function.

    Authors: Naihui Zhou, Yuxiang Jiang, Timothy R. Bergquist, Alexandra J. Lee, Balint Z. Kacsoh, Alex W. Crocker, Kimberley A. Lewis, George Georghiou, Huy N. Nguyen, Md Nafiz Hamid, Larry Davis, Tunca Dogan, Volkan Atalay, Ahmet S. Rifaioglu, Alperen Dalkıran, Rengul Cetin Atalay…
    Citation: Genome Biology 2019 20:244
    Content type: Research Published on:

  13. Systematic interrogation of single-nucleotide variants (SNVs) is one of the most promising approaches to delineate the cellular heterogeneity and phylogenetic relationships at the single-cell level. While SNV ...

    Authors: Fenglin Liu, Yuanyuan Zhang, Lei Zhang, Ziyi Li, Qiao Fang, Ranran Gao and Zemin Zhang
    Citation: Genome Biology 2019 20:242
    Content type: Research Published on:

  14. Recent innovations in single-cell Assay for Transposase Accessible Chromatin using sequencing (scATAC-seq) enable profiling of the epigenetic landscape of thousands of individual cells. scATAC-seq data analysi...

    Authors: Huidong Chen, Caleb Lareau, Tommaso Andreani, Michael E. Vinyard, Sara P. Garcia, Kendell Clement, Miguel A. Andrade-Navarro, Jason D. Buenrostro and Luca Pinello
    Citation: Genome Biology 2019 20:241
    Content type: Research Published on:

  16. Elucidation of regulatory networks, including identification of regulatory mechanisms specific to a given biological context, is a key aim in systems biology. This has motivated the move from co-expression to ...

    Authors: Dharmesh D. Bhuva, Joseph Cursons, Gordon K. Smyth and Melissa J. Davis
    Citation: Genome Biology 2019 20:236
    Content type: Research Published on:

  17. A large number of analysis strategies are available for DNA methylation (DNAm) array and RNA-seq datasets, but it is unclear which strategies are best to use. We compare commonly used strategies and report how...

    Authors: Jeroen van Rooij, Pooja R. Mandaviya, Annique Claringbould, Janine F. Felix, Jenny van Dongen, Rick Jansen, Lude Franke, Peter A. C. ’t Hoen, Bas Heijmans and Joyce B. J. van Meurs
    Citation: Genome Biology 2019 20:235
    Content type: Research Published on:

  19. Human tissue is increasingly being whole genome sequenced as we transition into an era of genomic medicine. With this arises the potential to detect sequences originating from microorganisms, including pathoge...

    Authors: Abraham Gihawi, Ghanasyam Rallapalli, Rachel Hurst, Colin S. Cooper, Richard M. Leggett and Daniel S. Brewer
    Citation: Genome Biology 2019 20:208
    Content type: Research Published on:

  20. Accurate fusion transcript detection is essential for comprehensive characterization of cancer transcriptomes. Over the last decade, multiple bioinformatic tools have been developed to predict fusions from RNA...

    Authors: Brian J. Haas, Alexander Dobin, Bo Li, Nicolas Stransky, Nathalie Pochet and Aviv Regev
    Citation: Genome Biology 2019 20:213
    Content type: Research Published on:

  21. Single-cell RNA sequencing (scRNA-seq) is a powerful tool for studying complex biological systems, such as tumor heterogeneity and tissue microenvironments. However, the sources of technical and biological var...

    Authors: Ciara H. O’Flanagan, Kieran R. Campbell, Allen W. Zhang, Farhia Kabeer, Jamie L. P. Lim, Justina Biele, Peter Eirew, Daniel Lai, Andrew McPherson, Esther Kong, Cherie Bates, Kelly Borkowski, Matt Wiens, Brittany Hewitson, James Hopkins, Jenifer Pham…
    Citation: Genome Biology 2019 20:210
    Content type: Research Published on:

  23. Many high-throughput experiments compare two phenotypes such as disease vs. healthy, with the goal of understanding the underlying biological phenomena characterizing the given phenotype. Because of the import...

    Authors: Tuan-Minh Nguyen, Adib Shafi, Tin Nguyen and Sorin Draghici
    Citation: Genome Biology 2019 20:203
    Content type: Research Published on:

    The Correction to this article has been published in Genome Biology 2019 20:234

  24. A series of miRNA-disease association prediction methods have been proposed to prioritize potential disease-associated miRNAs. Independent benchmarking of these methods is warranted to assess their effectivene...

    Authors: Zhou Huang, Leibo Liu, Yuanxu Gao, Jiangcheng Shi, Qinghua Cui, Jianwei Li and Yuan Zhou
    Citation: Genome Biology 2019 20:202
    Content type: Research Published on:

  25. Challenges are achieving broad acceptance for addressing many biomedical questions and enabling tool assessment. But ensuring that the methods evaluated are reproducible and reusable is complicated by the dive...

    Authors: Kyle Ellrott, Alex Buchanan, Allison Creason, Michael Mason, Thomas Schaffter, Bruce Hoff, James Eddy, John M. Chilton, Thomas Yu, Joshua M. Stuart, Julio Saez-Rodriguez, Gustavo Stolovitzky, Paul C. Boutros and Justin Guinney
    Citation: Genome Biology 2019 20:195
    Content type: Open Letter Published on:

  26. Single-cell transcriptomics is rapidly advancing our understanding of the cellular composition of complex tissues and organisms. A major limitation in most analysis pipelines is the reliance on manual annotati...

    Authors: Tamim Abdelaal, Lieke Michielsen, Davy Cats, Dylan Hoogduin, Hailiang Mei, Marcel J. T. Reinders and Ahmed Mahfouz
    Citation: Genome Biology 2019 20:194
    Content type: Research Published on:

  27. The combination of experimental evolution with whole-genome resequencing of pooled individuals, also called evolve and resequence (E&R) is a powerful approach to study the selection processes and to infer the ...

    Authors: Christos Vlachos, Claire Burny, Marta Pelizzola, Rui Borges, Andreas Futschik, Robert Kofler and Christian Schlötterer
    Citation: Genome Biology 2019 20:169
    Content type: Research Published on:

  29. Alignment-free (AF) sequence comparison is attracting persistent interest driven by data-intensive applications. Hence, many AF procedures have been proposed in recent years, but a lack of a clearly defined be...

    Authors: Andrzej Zielezinski, Hani Z. Girgis, Guillaume Bernard, Chris-Andre Leimeister, Kujin Tang, Thomas Dencker, Anna Katharina Lau, Sophie Röhling, Jae Jin Choi, Michael S. Waterman, Matteo Comin, Sung-Hou Kim, Susana Vinga, Jonas S. Almeida, Cheong Xin Chan, Benjamin T. James…
    Citation: Genome Biology 2019 20:144
    Content type: Research Published on:

  31. In computational biology and other sciences, researchers are frequently faced with a choice between several computational methods for performing data analyses. Benchmarking studies aim to rigorously compare th...

    Authors: Lukas M. Weber, Wouter Saelens, Robrecht Cannoodt, Charlotte Soneson, Alexander Hapfelmeier, Paul P. Gardner, Anne-Laure Boulesteix, Yvan Saeys and Mark D. Robinson
    Citation: Genome Biology 2019 20:125
    Content type: Review Published on:

  32. In high-throughput studies, hundreds to millions of hypotheses are typically tested. Statistical methods that control the false discovery rate (FDR) have emerged as popular and powerful tools for error rate co...

    Authors: Keegan Korthauer, Patrick K. Kimes, Claire Duvallet, Alejandro Reyes, Ayshwarya Subramanian, Mingxiang Teng, Chinmay Shukla, Eric J. Alm and Stephanie C. Hicks
    Citation: Genome Biology 2019 20:118
    Content type: Research Published on:

  33. Structural variations (SVs) or copy number variations (CNVs) greatly impact the functions of the genes encoded in the genome and are responsible for diverse human diseases. Although a number of existing SV det...

    Authors: Shunichi Kosugi, Yukihide Momozawa, Xiaoxi Liu, Chikashi Terao, Michiaki Kubo and Yoichiro Kamatani
    Citation: Genome Biology 2019 20:117
    Content type: Research Published on: