23 papers · ranked by Valyu relevance
Serghei Mangul, Lana S. Martin, Brian L. Hill, Angela Ka-Mei Lam + 4 more
'Margaret G. Distler' 'Alex Zelikovsky' 'Eleazar Eskin' 'Jonathan Flint'] Computational omics methods packaged as software have become essential to modern biological research. The increasing dependence of scientists on these powerful software tools creates a need for systematic assessment of these methods, known as…
Théo Matricon, Mathieu Acher, Helge Spieker, Arnaud Gotlieb
Benchmarking is a common practice in software engineering to assess the qualities and performance of software variants, coming from multiple competing systems or from configurations of the same system. Benchmarks are used notably to compare and understand variant performance, fine-tune software, detect regressions, or…
Salvador Capella-Gutierrez, Diana de la Iglesia, Juergen Haas, Analia Lourenco + 7 more
The dependence of life scientists on software has steadily grown in recent years. For many tasks, researchers have to decide which of the available bioinformatics software are more suitable for their specific needs. Additionally researchers should be able to objectively select the software that provides the highest…
Japke, Nils, Koch, Sebastian + 4 more
—Performance regressions in large-scale software systems can lead to substantial resource inefficiencies, making their early detection critical. Frequent benchmarking is essential for identifying these regressions and maintaining service-level agreements (SLAs). Performance benchmarks, however, are resourceintensive…
Martin Grambow, Christoph Laaber, Philipp Leitner, David Bermbach + 1 more
'Muhammad Aleem'] Performance problems in applications should ideally be detected as soon as they occur, i.e., directly when the causing code modification is added to the code repository. To this end, complex and cost-intensive application benchmarks or lightweight but less relevant microbenchmarks can be added to…
Eva Martín del Pico, Josep Lluis Gelpi, Salvador Capella-Gutiérrez
Software plays a crucial and growing role in research. Unfortunately, the computational component in Life Sciences research is challenging to reproduce and verify most of the time. It could be undocumented, opaque, may even contain unknown errors that affect the outcome, or be directly unavailable, and impossible to…
Seth Russell, Tellen D. Bennett, Debashis Ghosh, Sebastian Ventura
Today’s computational researchers are expected to be highly proficient in using software to solve a wide range of problems ranging from processing large datasets to developing personalized treatment strategies from a growing range of options. Researchers are well versed in their own field, but may lack formal training…
Nils Japke, Christoph Witzko, Martin Grambow, David Bermbach
In this paper, we use a testbed microservice application which includes three performance issues to study the detection capabilities of both approaches. In extensive benchmarking experiments, we increase the severity of each performance issue stepwise, run both an application benchmark and the microbenchmark suite, and…
Izaskun Mallona, Charlotte Soneson, Ben Carrillo, Almut Lütge + 5 more
Benchmarking, which involves collecting reference datasets and demonstrating method performance, is a requirement for the development of new computational tools, but also becomes a domain of its own to achieve neutral comparisons of methods. Although a lot has been written about how to design and conduct benchmark…
Martin Grambow, Д. А. Ковалев, Christoph Laaber, Philipp Leitner + 1 more
'David Bermbach'] Software performance changes are costly and often hard to detect pre-release. Similar to software testing frameworks, either application benchmarks or microbenchmarks can be integrated into quality assurance pipelines to detect performance changes before releasing a new application version.…
Anthony Sonrel, Almut Luetge, Charlotte Soneson, Izaskun Mallona + 16 more
Computational methods represent the lifeblood of modern molecular biology. Benchmarking is important for all methods, but with a focus here on computational methods, benchmarking is critical to dissect important steps of analysis pipelines, formally assess performance across common situations as well as edge cases, and…
Robbe Devreese, Caroline Jachmann, Bart Van Puyvelde, Holda A. Anagho-Mattanovich + 46 more
Mass spectrometry (MS)-based proteomics is a well-established strategy for analyzing complex biological mixtures. Many MS instruments and data acquisition strategies are available, and the data they acquire differ substantially, thus requiring tailored analysis algorithms. Hence, many dedicated bioinformatics workflows…
Trever Schirmer, Tobias Pfandzelter, David Bermbach
—Running microbenchmark suites often and early in the development process enables developers to identify performance issues in their application. Microbenchmark suites of complex applications can comprise hundreds of individual benchmarks and take multiple hours to evaluate meaningfully, making running those benchmarks…
Jasper Albers, Jari Pronold, Anno Christopher Kurth, Stine Brekke Vennemo + 8 more
Modern computational neuroscience strives to develop complex network models to explain dynamics and function of brains in health and disease. This process goes hand in hand with advancements in the theory of neuronal networks and increasing availability of detailed anatomical data on brain connectivity. Large-scale…
Michael P. Lynch, Laurent Gatto, Aedin C. Culhane
Reproducibility, reusability, and portability were identified as key requirements for analysis pipelines in genomics. A previous study identified that less than 4% of notebooks used for biomedical research were fully reproducible. Reusable and portable software are required to ensure compatibility with federated and…
Aymeric Blot, Justyna Petke
Performance is a key quality of modern software. Although recent years have seen a spike in research on automated improvement of software's execution time, energy, memory consumption, etc., there is a noticeable lack of standard benchmarks for such work. It is also unclear how such benchmarks are representative of…
Authors not listed
The analysis of molecular dynamics (MD) simulations is a critical but fragmented process, often requiring researchers to chain together multiple software tools and write bespoke scripts for routine structural and dynamic analyses. This workflow complexity creates a significant barrier to efficiency, standardization…
Authors not listed
Accurate benchmarks are key to assessing the accuracy and robustness of computational methods, yet most available benchmark sets focus on equilibrium geometries, limiting their utility for applications involving non-equilibrium structures such as ab initio molecular dynamics and automated reaction-path exploration. To…
Samuel Genheden, Esben Bjerrum
We introduce a framework for benchmarking multi-step retrosynthesis methods, i.e. route predictions, called PaRoutes. The framework consists of two sets of 10,000 synthetic routes extracted from the patent literature, a list of stock compounds, and a curated set of reactions on which one-step retrosynthesis models can…
Authors not listed
Alchemical Free Energy methods are gaining traction in computer-aided drug discovery. An expanding array of methodologies is available for the setup, execution and analysis of relative binding free energy (RBFE) calculations. However, the sharing of algorithms and protocols developed by different organisations is often…
Authors not listed
Investigating the molecular structure of soil organic matter (SOM), along with its intramolecular interactions and interactions with other soil components and xenobiotics, is essential due to its ecological importance. However, the complexity and heterogeneity of SOM present significant challenges for systematic…
Charly Empereur-mot, Luca Pesce, Davide Bochicchio, Claudio Perego + 1 more
We present Swarm-CG, a versatile software for the automatic parametrization of bonded parameters incoarse-grained (CG) models. By coupling state-of-the-art metaheuristics to Boltzmann inversion, Swarm-CG performs accurate parametrization of bonded terms in CG models composed of up to 200 pseudoatomswithin 4h-24h on…
Charly Empereur-mot, Luca Pesce, Davide Bochicchio, Claudio Perego + 1 more
We present Swarm-CG, a versatile software for the automatic parametrization of bonded parameters in coarse-grained (CG) models. By coupling state-of-the-art metaheuristics to Boltzmann inversion, Swarm-CG performs accurate parametrization of bonded terms in CG models composed of up to 200 pseudoatoms within 4h-24h on…