22 papers · ranked by Valyu relevance
Lukas M. Weber, Wouter Saelens, Robrecht Cannoodt, Charlotte Soneson + 5 more
'Alexander Hapfelmeier' 'Paul P. Gardner' 'Anne-Laure Boulesteix' 'Yvan Saeys' 'Mark D. Robinson'] In computational biology and other sciences, researchers are frequently faced with a choice between several computational methods for performing data analyses. Benchmarking studies aim to rigorously compare the…
Jasper Albers, Jari Pronold, Anno Christopher Kurth, Stine Brekke Vennemo + 8 more
Modern computational neuroscience strives to develop complex network models to explain dynamics and function of brains in health and disease. This process goes hand in hand with advancements in the theory of neuronal networks and increasing availability of detailed anatomical data on brain connectivity. Large-scale…
Nils Japke, Christoph Witzko, Martin Grambow, David Bermbach
In this paper, we use a testbed microservice application which includes three performance issues to study the detection capabilities of both approaches. In extensive benchmarking experiments, we increase the severity of each performance issue stepwise, run both an application benchmark and the microbenchmark suite, and…
Martin Grambow, Christoph Laaber, Philipp Leitner, David Bermbach + 1 more
'Muhammad Aleem'] Performance problems in applications should ideally be detected as soon as they occur, i.e., directly when the causing code modification is added to the code repository. To this end, complex and cost-intensive application benchmarks or lightweight but less relevant microbenchmarks can be added to…
Martin Grambow, Д. А. Ковалев, Christoph Laaber, Philipp Leitner + 1 more
'David Bermbach'] Software performance changes are costly and often hard to detect pre-release. Similar to software testing frameworks, either application benchmarks or microbenchmarks can be integrated into quality assurance pipelines to detect performance changes before releasing a new application version.…
Xiaoqi Cabiria Liang, Nick Robertson, Marni Torkel, Sanghyun Kim + 3 more
The rapid growth of computational methods for the computational biology field highlights the critical role of benchmarking in guiding method selection. However, there is no standardised data structure that effectively links and stores datasets, performance metrics and available ground truth. Without such a unified and…
Salvador Capella-Gutierrez, Diana de la Iglesia, Juergen Haas, Analia Lourenco + 7 more
The dependence of life scientists on software has steadily grown in recent years. For many tasks, researchers have to decide which of the available bioinformatics software are more suitable for their specific needs. Additionally researchers should be able to objectively select the software that provides the highest…
Vandhana Krishnan, Sowmi Utiramerur, Zena Ng, Somalee Datta + 2 more
Benchmarking the performance of complex analytical pipelines is an essential part of developing Laboratory Developed Assays (LDT). Reference samples and benchmark calls published by Genome in a Bottle (GIAB) Consortium have enabled the evaluation of analytical methods. However, the performance of such methods is not…
Filippo Schiavio, Lubomír Bulej, Walter Binder
Developers often use microbenchmarks to choose the most performant implementation of a method or a class. On the Java Virtual Machine (JVM), this is commonly done using the Java Microbenchmark Harness (JMH) which addresses common pitfalls of measuring code performance on the JVM. However, even using JMH guidelines…
Trever Schirmer, Tobias Pfandzelter, David Bermbach
—Running microbenchmark suites often and early in the development process enables developers to identify performance issues in their application. Microbenchmark suites of complex applications can comprise hundreds of individual benchmarks and take multiple hours to evaluate meaningfully, making running those benchmarks…
Authors not listed
This paper presents an empirical comparison of process control algorithms, with particular emphasis on classical Proportional–Integral–Derivative (PID) control, Model Predictive Control (MPC), and neural network-based methods in the context of complex industrial plants. Since industrial sectors frequently demand…
Jianfeng Zhan
Currently, there is no consistent benchmarking across multi-disciplines. Even no previous work tries to relate different categories of benchmarks in multi-disciplines. This article investigates the origin and evolution of the benchmark term. Five categories of benchmarks are summarized, including measurement standards…
Vandhana Krishnan, Sowmithri Utiramerur, Zena Ng, Somalee Datta + 2 more
'Michael P. Snyder' 'Euan A. Ashley'] Background Benchmarking the performance of complex analytical pipelines is an essential part of developing Lab Developed Tests (LDT). Reference samples and benchmark calls published by Genome in a Bottle (GIAB) consortium have enabled the evaluation of analytical methods. The…
Neel Guha, Andy K. Zhang, Christine Tsang, Christopher D. Manning + 2 more
Despite substantial excitement around the use of AI in law, little information exists on the performance and associated risks of the domain’s widely marketed tools. Recent work, for instance, has demonstrated the significant potential for “hallucinations”-wherein models make up facts, law, and precedent-leading Chief…
Aleksandra Szmigiel, Ivan Gesteira Costa Filho, Ricardo Jose Gabrielli Barreto Campello
Clustering single-cell RNA-seq (scRNA-seq) data remains a major challenge due to high dimensionality and noise. Despite numerous bench-marking studies aiming to identify the best clustering methods, many suffer from methodological flaws that undermine their conclusions. A major challenge in benchmarking is selecting…
Authors not listed
Investigating the molecular structure of soil organic matter (SOM), along with its intramolecular interactions and interactions with other soil components and xenobiotics, is essential due to its ecological importance. However, the complexity and heterogeneity of SOM present significant challenges for systematic…
Peter Krusche, Len Trigg, Paul C. Boutros, Christopher E. Mason + 12 more
Assessing accuracy of NGS variant calling is immensely facilitated by a robust benchmarking strategy and tools to carry it out in a standard way. Benchmarking variant calls requires careful attention to definitions of performance metrics, sophisticated comparison approaches, and stratification by variant type and…
Authors not listed
Validating the performance of exchange-correlation functionals is vital to ensure the reliability of DFT calculations. Typically, these validations involve benchmarking datasets. Currently, such datasets are typically assembled in an unprincipled manner, suffering from uncontrolled chemical bias, and limiting the…
Authors not listed
Accurate benchmarks are key to assessing the accuracy and robustness of computational methods, yet most available benchmark sets focus on equilibrium geometries, limiting their utility for applications involving non-equilibrium structures such as ab initio molecular dynamics and automated reaction-path exploration. To…
Kobi Felton, Jan Rittig, Alexei Lapkin
In the fine chemicals industry, reaction screening and optimisation are essential to development of new products. However, this screening can be extremely time and labor intensive, especially when intuition is used. Machine learning offers a solution through iterative suggestions of new experiments based on past…
Zheng Li, Liam O’Brien, Maria Kihl
Software engineering considers performance evaluation to be one of the key portions of software quality assurance. Unfortunately, there seems to be a lack of standard methodologies for performance evaluation even in the scope of experimental computer science. Inspired by the concept of "instantiation" in…
Dimitar Georgiev, Simon Vilms Pedersen, Ruoxiao Xie, Álvaro Fernández-Galiana + 2 more
Raman spectroscopy is a non-destructive and label-free chemical analysis technique, which plays a key role in the analysis and discovery cycle of various branches of science. Nonetheless, progress in Raman spectroscopic analysis is still impeded by the lack of software, methodological and data standardisation, and the…