24 papers · ranked by Valyu relevance
Sebastian Raschka
The correct use of model evaluation, model selection, and algorithm selection techniques is vital in academic machine learning research as well as in many industrial settings. This article reviews different techniques that can be used for each of these three subtasks and discusses the main advantages and disadvantages…
Gary S Collins, Paula Dhiman, Jie Ma, Michael M Schlussel + 9 more
Evaluating the performance of a clinical prediction model is crucial to establish its predictive accuracy in the populations and settings intended for use. In this article, the first in a three part series, Collins and colleagues describe the importance of a meaningful evaluation using internal, internal-external, and…
Pankaj Chejara, Luis P. Prieto, Adolfo Ruiz-Calleja, María Jesús Rodríguez-Triana + 3 more
'María Jesús Rodríguez-Triana' 'Shashi Kant Shankar' 'Reet Kasepalu' 'Andreas Savakis'] Multimodal Learning Analytics (MMLA) researchers are progressively employing machine learning (ML) techniques to develop predictive models to improve learning and teaching practices. These predictive models are often evaluated for…
Ben Hutchinson, Negar Rostamzadeh, Christina M. Greer, Katherine Heller + 1 more
'Katherine Heller' 'Vinodkumar Prabhakaran'] Forming a reliable judgement of a machine learning (ML) model's appropriateness for an application ecosystem is critical for its responsible use, and requires considering a broad range of factors including harms, benefits, and responsibilities. In practice, however…
Maulik K. Nariya, Caitlin E. Mills, Peter K. Sorger, Artem Sokolov
The true accuracy of a machine learning model is a population-level statistic that cannot be observed directly. In practice, predictor performance is estimated against one or more test datasets, and the accuracy of this estimate strongly depends on how well the test sets represent all possible unseen datasets. Here we…
Pola Schwöbel, Luca Franceschi, Muhammad Bilal Zafar, Keerthan Vasist + 6 more
'Keerthan Vasist' 'A. Malhotra' 'Tomer Shenhar' 'Pinal Tailor' 'Pınar Yilmaz' 'Michael J. Diamond' 'Michele Donini'] fmeval is an open source library to evaluate large language models (LLMs) in a range of tasks. It helps practitioners evaluate their model for task performance and along multiple responsible AI…
Thomas Wöhling, Alvaro Oliver Crespo Delgadillo, Moritz Kraft, Anneli Guthke
'Anneli Guthke'] Title: Abstract Groundwater level observations are used as decision variables for aquifer management, often in conjunction with models to provide predictions for operational forecasting. In this study, we compare different model classes for this task: a spatially explicit 3D groundwater flow model…
Jennifer A. Rohrs, Sahak Z. Makaryan, Stacey D. Finley
Systems biology combines computational modeling with quantitative experimental measurements to study complex biological processes. Here, we outline an approach for parameterizing and validating a systems biology model to yield predictive tool that can generate testable hypotheses and expand biological understanding.
Jacek A Kopec, Philippe Finès, Douglas G Manuel, David L Buckeridge + 9 more
Background Computer simulation models are used increasingly to support public health research and policy, but questions about their quality persist. The purpose of this article is to review the principles and methods for validation of population-based disease simulation models. Methods We developed a comprehensive…
Keith D. Harris, Guy Hadari, Gili Greenbaum
Modelling the dynamics of biological processes is ubiquitous across the ecological and evolutionary disciplines. However, the increasing complexity of these models poses a challenge to the dissemination of model-derived results. Often only a small subset of model results are made available to the scientific community…
Birger Johansson, Trond A. Tjøstheim, Christian Balkenius
System-level brain modeling is a powerful method for building computational models of the brain and allows biologically motivated models to produce measurable behavior that can be tested against empirical data. System-level brain models occupy an intermediate position between detailed neuronal circuit models and…
Feng Feng, Zhenru Chen, Jianyuan Ni, Yuanxun Zhang + 3 more
Drinking water is essential to public health and socioeconomic growth. Therefore, assessing and ensuring drinking water supply is a critical task in modern society. Conventional approaches to analyzing and controlling drinking water quality are labor-intensive and costly with a low throughput. Machine learning (ML) is…
Kate E. Dray, Joseph J. Muldoon, Niall M. Mangan, Neda Bagheri + 1 more
Mathematical modeling is invaluable for advancing understanding and design of synthetic biological systems. However, the model development process is complicated and often unintuitive, requiring iteration on various computational tasks and comparisons with experimental data. Ad hoc model development can pose a barrier…
Florian E. Dorner, Vivian Y. Nastl, Moritz Hardt
twice the data Authors: ['Florian E. Dorner' 'Vivian Y. Nastl' 'Moritz Hardt'] High quality annotations are increasingly a bottleneck in the explosively growing machine learning ecosystem. Scalable evaluation methods that avoid costly annotation have therefore become an important research ambition. Many hope to use…
Authors not listed
Computational blind challenges offer critical, unbiased assessment opportunities to assess and accelerate scientific progress, as demonstrated by a breadth of breakthroughs over the last decade. We report the outcomes and key insights from an open science community blind challenge focused on computational methods in…
Authors not listed
Background: Janus Kinase 2 (JAK2) is a key kinase in cellular signal transduction. Its abnormal activation is closely related to various myeloproliferative neoplasms and inflammatory diseases. Developing selective JAK2 inhibitors is an important direction in drug discovery. Accurate prediction of compound inhibitory…
Daniel Wallach, Taru Palosuo, Peter Thorburn, Zvi Hochman + 48 more
Predicting wheat phenology is important for cultivar selection, for effective crop management and provides a baseline for evaluating the effects of global change. Evaluating how well crop phenology can be predicted is therefore of major interest. Twenty-eight wheat modeling groups participated in this evaluation. Model…
Mithun Chakraborty, Sanmay Das, Allen Lavoie
We present a simple theoretical framework, and corresponding practical procedures, for comparing probabilistic models on real data in a traditional machine learning setting. This framework is based on the theory of proper scoring rules, but requires only basic algebra and probability theory to understand and verify.…
Utku Boran Torun, Veli Karakaya, Ali Babar, Eray Tüzün
Large Language Models (LLMs) are increasingly embedded in software engineering (SE) tools, powering applications such as code generation, automated code review, and bug triage. As these LLM-based AI for Software Engineering (AI4SE) systems transition from experimental prototypes to widely deployed tools, the question…
Justas Birgiolas, Vergil Haynes, Padraig Gleeson, Richard C. Gerkin + 2 more
As researchers develop computational models of neural systems with increasing sophistication and scale, it is often the case that fully de novo model development is impractical and inefficient. Thus arises a critical need to quickly find, evaluate, re-use, and build upon models and model components developed by other…
Daniel R Balcarcel, Sanjiv D Mehta, Celeste G Dixon, Charlotte Z Woods-Hill + 3 more
Prognostic models developed for use in the intensive care unit (ICU) can inform treatment decisions and improve patient care. However, despite extensive research, few models have contributed to improved patient-centred outcomes. A major limitation is that the influence of treatment interventions on patient outcomes…
Fuzhan Rahmanian, Robert M. Lee, Dominik Linzner, Kathrin Michel + 4 more
Predicting and monitoring battery life early and across chemistries is a significant challenge due to the plethora of degradation paths, form factors, and electrochemical testing protocols. Existing models typically translate poorly across different electrode, electrolyte, and additive materials, mostly require a fixed…
Authors not listed
Ensuring the trustworthiness of machine learning (ML) models in high-stake applications is crucial. One such application is predicting anti-cancer drug sensitivity, where ML models are built with the final goal of integrating them into treatment recommendation systems for personalized medicine. Here, we propose a…
Authors not listed
Machine learning holds significant promise for accelerating biomarker discovery in clinical proteomics, yet its real-world impact remains limited by widespread methodological pitfalls and unrealistic expectations. In this perspective, we critically examine the integration of machine learning into clinical proteomics…