Technological advances enabling massively parallel measurement of biological features — such as microarrays, high-throughput sequencing and mass spectrometry — have ushered in the omics era, now in its third decade. The resulting complex landscape of analytical methods has naturally fostered the growth of an omics benchmarking industry. Benchmarking refers to the process of objectively comparing and evaluating the performance of different computational or analytical techniques when processing and analysing large-scale biological data sets, such as transcriptomics, proteomics and metabolomics. With thousands of omics benchmarking studies published over the past 25 years, the field has matured to the point where the foundations of benchmarking have been established and well described. However, generating meaningful benchmarking data and properly evaluating performance in this complex domain remains challenging. In this Review, we highlight some common oversights and pitfalls in omics benchmarking. We also establish a methodology to bring the issues that can be addressed into focus and to be transparent about those that cannot: this takes the form of a spreadsheet template of guidelines for comprehensive reporting, intended to accompany publications. In addition, a survey of recent developments in benchmarking is provided as well as specific guidance for commonly encountered difficulties.

技术进步使得生物特征的大规模并行测量成为可能——例如微阵列、高通量测序和质谱——已经迎来了组学时代，现在已经进入了第三个十年。由此产生的复杂的分析方法自然促进了组学基准行业的发展。基准测试是指在处理和分析大规模生物数据集（例如转录组学、蛋白质组学和代谢组学）时客观比较和评估不同计算或分析技术性能的过程。过去 25 年里发表了数千项组学基准研究，该领域已经成熟到基准测试的基础已经建立并得到很好的描述。然而，生成有意义的基准测试数据并正确评估这个复杂领域的性能仍然具有挑战性。在这篇综述中，我们强调了组学基准测试中的一些常见疏忽和陷阱。我们还建立了一种方法，使可以解决的问题成为焦点，并对那些不能解决的问题保持透明：这采用综合报告指南电子表格模板的形式，旨在随出版物一起发布。此外，还提供了对基准测试最新发展的调查以及针对常见困难的具体指导。