27 papers · ranked by Valyu relevance
Sheldon Smith, Ethan Robinson, Timmy Frederiksen, T S Stevens + 3 more
'Tomáš Černý' 'Miroslav Bureš' 'Davide Taibi'] Abstract—Testing microservice systems involves a large amount of planning and problem-solving. The difficulty of testing microservice systems increases as the size and structure of such systems become more complex. To help the microservice community and simplify…
Japke, Nils, Koch, Sebastian + 4 more
—Performance regressions in large-scale software systems can lead to substantial resource inefficiencies, making their early detection critical. Frequent benchmarking is essential for identifying these regressions and maintaining service-level agreements (SLAs). Performance benchmarks, however, are resourceintensive…
Anasua Sarkar, Yang Yang, Mauno Vihinen
Development of new computational methods and testing their performance has to be done on experimental data. Only in comparison to existing knowledge can method performance be assessed. For that purpose, benchmark datasets with known and verified outcome are needed. High-quality benchmark datasets are valuable and may…
Nils Japke, Christoph Witzko, Martin Grambow, David Bermbach
In this paper, we use a testbed microservice application which includes three performance issues to study the detection capabilities of both approaches. In extensive benchmarking experiments, we increase the severity of each performance issue stepwise, run both an application benchmark and the microbenchmark suite, and…
Wilhelm Hasselbring
In empirical software engineering, benchmarks can be used for comparing different methods, techniques and tools. However, the recent ACM SIGSOFT Empirical Standards for Software Engineering Research do not include an explicit checklist for benchmarking. In this paper, we discuss benchmarks for software performance and…
Huang Huang, Gertrude-Emilia Costin
The concept of benchmark materials in the personal care and cosmetic industry is often based on the positioning of prototypes relative to the leading brand within a given product category. The main points of comparison are the market ranking, performance regarding consumers’ perception, aesthetics, cost, complexity of…
Martin Grambow, Christoph Laaber, Philipp Leitner, David Bermbach + 1 more
'Muhammad Aleem'] Performance problems in applications should ideally be detected as soon as they occur, i.e., directly when the causing code modification is added to the code repository. To this end, complex and cost-intensive application benchmarks or lightweight but less relevant microbenchmarks can be added to…
Salvador Capella-Gutierrez, Diana de la Iglesia, Juergen Haas, Analia Lourenco + 7 more
The dependence of life scientists on software has steadily grown in recent years. For many tasks, researchers have to decide which of the available bioinformatics software are more suitable for their specific needs. Additionally researchers should be able to objectively select the software that provides the highest…
Haidong Yi, Alec Plotkin, Natalie Stanley
Modern single-cell data analysis relies on statistical testing (e.g. differential expression testing) to identify genes or proteins that are up-or down-regulated in relation to cell-types or clinical outcomes. However, existing algorithms for such statistical testing are often limited by technical noise and cellular…
Izaskun Mallona, Almut Luetge, Ben Carrillo, Daniel Incicau + 4 more
bioinformatics Authors: ['Izaskun Mallona' 'Almut Luetge' 'Ben Carrillo' 'Daniel Incicau' 'Reto Gerber' 'Anthony Sonrel' 'Charlotte Soneson' 'Mark D. Robinson'] We describe an alpha version of a new benchmarking system, Omnibenchmark, to facilitate benchmark formalization and execution in solo and community efforts.…
Marc Pagès-Gallego, Jeroen de Ridder
Nanopore basecalling is a difficult task which requires the use of complex algorithms and neural network models to achieve competitive accuracies. These algorithms, called basecallers, are developed continuously both by ONT and the scientific community in an effort to improve basecalling accuracies. With the rapidly…
Xavier Devroey, Alessio Gambi, Juan Pablo Galeotti, René Just + 3 more
'Fitsum Meshesha Kifetew' 'Annibale Panichella' 'Sebastiano Panichella'] Researchers and practitioners have designed and implemented various automated test case generators to support effective software testing. Such generators exist for various languages (e.g., Java, C#, or Python) and various platforms (e.g., desktop…
Rebecca F. Alford, Jeffrey J. Gray
Energy functions are fundamental to biomolecular modeling. Their success depends on robust physical formalisms, efficient optimization, and high-resolution data for training and validation. Over the past 20 years, progress in each area has advanced soluble protein energy functions. Yet, energy functions for membrane…
Rebecca Jane Edwards, Peter Yeates, Janet Lefroy, Robert McKinley
Aligning examiners judgements to a shared standard is desirable within Objective Structured Clinical Exams (OSCEs) because it accords with OSCEs’ epistemic assumptions and purpose. Video-based benchmarking (VBB) involves examiners scoring station specific videos and comparing their judgements against an agreed score.…
Authors not listed
The construction of large benchmark sets has accelerated advancement of quantum chemistry methods, especially in density functional theory and lower-cost methods. However, these large benchmark sets can be unsuitable for cutting-edge method development, because research codes developed for fundamentally new approaches…
Sher Afgun Khan, Muhammad Abdul Qadir, Muhammad Azeem Abbas, Muhammad Tanvir Afzal + 1 more
'Muhammad Tanvir Afzal' 'Ozgu Can'] OWL2 semantics are becoming increasingly popular for the real domain applications like Gene engineering and health MIS. The present work identifies the research gap that negligible attention has been paid to the performance evaluation of Knowledge Base Systems (KBS) using OWL2…
Suzanne Ackloo, Rima Al-awar, Rommie E. Amaro, Cheryl H. Arrowsmith + 35 more
Computational approaches in drug discovery and development hold great promise, with artificial intelligence methods undergoing widespread contemporary use, but the experimental validation of these new approaches is frequently inadequate. We are initiating Critical Assessment of Computational Hit-finding Experiments…
Alberto Florez Prada, Darren J. Hart
Validating bioinformatics pipelines and benchmarking sequence processing algorithms requires reliable test datasets. Existing read simulation tools rely on reference genomes and empirical error profiles, lacking fine-grained control over specific targeted DNA constructs and controlled error injection.…
Andrea Bombarda, Angelo Gargantini
Combinatorial testing is a widely adopted technique for efficiently detecting faults in software. The quality of combinatorial test generators plays a crucial role in achieving effective test coverage. Evaluating combinatorial test generators remains a challenging task that requires diverse and representative…
Alex Lepauvre, Rony Hirschhorn, Katarina Bendtz, Liad Mudrik + 1 more
The replication crisis in experimental psychology and neuroscience has received much attention recently. This has led to wide acceptance of measures to improve scientific practices, such as preregistration and registered reports. Less effort has been devoted to performing and reporting the results of systematic tests…
Piotr Plecinski, Nataliia Bokla, Tamara Klymkovych, Mykhailo Melnyk + 2 more
'Wojciech Zabierowski' 'Diogo Gomes'] Reading and analyzing data from sensors are crucial in many areas of life. IoT concepts and related issues are becoming more and more popular, but before we can process data and draw conclusions, we need to think about how to design an application. The most popular solutions today…
Authors not listed
The incredible advances in deep learning architectures have not only led to successful protein structure prediction methods, but also to many interesting new methods related to protein-ligand docking and co-folding. The most recent biomolecular foundation model, Boltz-2, even claimed to approach the performance of…
Authors not listed
Proteochemometric models (PCM) are used in computational drug discovery to leverage both protein and ligand representations for bioactivity prediction. While machine learning (ML) and deep learning (DL) have come to dominate PCMs, often serving as scoring functions, rigorous evaluation standards have not always been…
Hagen Eike Keßel, Stefan Masjosthusmann, Kristina Bartmann, Jonathan Blum + 8 more
In the field of hazard assessment, Benchmark concentrations (BMC) and their associated uncertainty are of particular interest for regulatory decision making. The BMC estimation consists of various statistical decisions to be made, which depend largely on factors such as experimental design and assay endpoint features.…
Ian Knight, Khanh Tang, Olivier Mailhot, John Irwin
Molecular docking is a widely used technique for leveraging protein structure in ligand discovery, but as a method, it remains difficult to utilize due to limitations that have not been adequately addressed. Despite some progress towards automation, docking still requires expert guidance, hindering its adoption by a…
Michael D. Hogan, Lisa J. Carnahan, Robert J. Carpenter, David W. Flater + 9 more
Our high technology society continues to rely more and more upon sophisticated measurements, technical standards, and associated testing activities. This was true for the industrial society of the 20th century and remains true for the information society of the 21st century. Over the last half of the 20th century…
Fergus Boyles, Charlotte M Deane, Garrett Morris
Machine learning scoring functions for protein-ligand binding affinity have been found to consistently outperform classical scoring functions when trained and tested on crystal structures of bound protein-ligand complexes. However, it is less clear how these methods perform when applied to docked poses of complexes. We…