27 papers · ranked by Valyu relevance
John D. Jakeman, Lorena A. Barba, Joaquim R. R. A. Martins, Thomas O'Leary-Roseberry
"Thomas O'Leary-Roseberry"] Scientific machine learning (SciML) models are transforming many scientific disciplines. However, the development of good modeling practices to increase the trustworthiness of SciML has lagged behind its application, limiting its potential impact. The goal of this paper is to start a…
Joost Mertens, Joachim Denil
One of the challenges in twinned systems is ensuring the digital twin remains a valid representation of the system it twins. Depending on the type of twinning occurring, it is either trivial, such as in dashboarding/visualizations that mirror the system with real-time data, or challenging, in case the digital twin is a…
José Camacho
The validation of a data-driven model is the process of assessing the model's ability to generalize to new, unseen data in the population of interest. This paper proposes a set of general rules for model validation. These rules are designed to help practitioners create reliable validation plans and report their results…
Orestis Efthimiou, Michael Seo, Konstantina Chalkou, Thomas Debray + 2 more
'Matthias Egger' 'Georgia Salanti'] Predicting future outcomes of patients is essential to clinical practice, with many prediction models published each year. Empirical evidence suggests that published studies often have severe methodological limitations, which undermine their usefulness. This article presents a…
Agus Sudjianto, Aijun Zhang
This paper presents a comprehensive overview of model validation practices and advancement in the banking industry based on the experience of managing Model Risk Management (MRM) since the inception of regulatory guidance SR11-7/OCC11- 12 over a decade ago. Model validation in banking is a crucial process designed to…
Najma Taimoor, Semeen Rehman
—This paper presents a verification-based methodology to validate the model of personalised health conditions. The model identifies the values that may result in unsafe, unreachable, in-exhaustive, and overlapping states threaten otherwise threaten the patient's life by producing false alarms by accepting suspicious…
Mahmood Alaghmandan, Olga Streltchenko
In light of recent allegations of misconduct concerning highly regarded academic research projects in behavioural science (see [6] and [10]) we review model risk management practices commonly applied in financial institutions. We maintain these are transferable to academic research with clear benefits for the quality…
Péter Király, Ramóna Kiss, Dániel Kovács, Amine Ballaj + 1 more
We investigated the relevance of the validation principles on the Quantitative Structure Activity Relationship models issued by Organization for Economic and Co-operation and Development. We checked the goodness-of-fit, robustness and predictivity categories in linear and nonlinear models using benchmark datasets. Most…
M. A. E. Binuya, E. G. Engelhardt, W. Schats, M. K. Schmidt + 1 more
'E. W. Steyerberg'] Background Clinical prediction models are often not evaluated properly in specific settings or updated, for instance, with information from new markers. These key steps are needed such that models are fit for purpose and remain relevant in the long-term. We aimed to present an overview of…
David E Carlson, Ricardo Chavarriaga, Yiling Liu, Fabien Lotte + 1 more
'Bao-Liang Lu'] Title: Abstract Objective. Machine learning’s (MLs) ability to capture intricate patterns makes it vital in neural engineering research. With its increasing use, ensuring the validity and reproducibility of ML methods is critical. Unfortunately, this has not always been the case in practice, as there…
Evgueni Jacob, Angélique Perrillat-Mercerot, Jean-Louis Palgen, Adèle L’Hostis + 5 more
Over the past several decades, metrics have been defined to assess the quality of various types of models and to compare their performance depending on their capacity to explain the variance found in real-life data. However, available validation methods are mostly designed for statistical regressions rather than for…
Matthew Francis Dixon
Agentic artificial intelligence systems introduce a new class of model risk. Unlike traditional predictive models, autonomous agents continuously acquire information, form beliefs regarding latent states of the environment, generate forecasts, select actions, and adapt their behavior over time. Existing validation…
Fulya Akpinar Singh, Nasrin Afzal, Shepard J. Smithline, Craig J. Thalhauser
'Craig J. Thalhauser'] Validation of a quantitative model is a critical step in establishing confidence in the model’s suitability for whatever analysis it was designed. While processes for validation are well-established in the statistical sciences, the field of quantitative systems pharmacology (QSP) has taken a more…
Jonathan Karr, Rahuman S. Malik-Sheriff, James Osborne, Gilberto Gonzalez-Parra + 14 more
'Gilberto Gonzalez-Parra' 'Eric Forgoston' 'Ruth Bowness' 'Yaling Liu' 'Robin Thompson' 'Winston Garira' 'Jacob Barhak' 'John Rice' 'Marcella Torres' 'Hana M. Dobrovolny' 'Tingting Tang' 'William Waites' 'James A. Glazier' 'James R. Faeder' 'Alexander Kulesza'] During the COVID-19 pandemic, mathematical modeling of…
Giuseppe Gallitto, Robert Englert, Balint Kincses, Raviteja Kotikalapudi + 4 more
Multivariate predictive models play a crucial role in enhancing our understanding of complex biological systems and in developing innovative, replicable tools for translational medical research. However, the complexity of machine learning methods and extensive data pre-processing and feature engineering pipelines can…
Maurice HT Ling
Modeling and simulation are recognized as important aspects of the scientific method for more than 70 years but its adoption in biology has been slow. Debates on its representativeness, usefulness, and whether the effort spent on such endeavors is worthwhile, exist to this day. Here, I argue that most of learning is…
Keith D. Harris, Guy Hadari, Gili Greenbaum
Modeling the dynamics of biological processes is ubiquitous across the ecological and evolutionary disciplines. However, the increasing complexity of these models poses a significant challenge to the dissemination of model-derived results. With the existing requirements of scientific publishing, most often only a small…
Udit Surya Saha, Michele Vendruscolo, Anne E. Carpenter, Shantanu Singh + 2 more
Recent advances in machine learning methods for materials science have significantly enhanced accurate predictions of the properties of novel materials. Here, we explore whether these advances can be adapted to drug discovery by addressing the problem of prospective validation - the assessment of the performance of a…
Nicolas Lartillot
There is still no consensus as to how to select models in Bayesian phylogenetics, and more generally in applied Bayesian statistics. Bayes factors are often presented as the method of choice, yet other approaches have been proposed, such as cross-validation or information criteria. Each of these paradigms raises…
Matthew Witman, Peter Schindler
Machine learning (ML) models in the materials sciences that are validated by overly simplistic cross-validation (CV) protocols can yield biased performance estimates for downstream modeling or materials screening tasks. This can be particularly counterproductive for applications where the time and cost of failed…
Xuan-Truc Dinh Tran, Tieu-Long Phan, Van-Thinh To, Ngoc-Vi Nguyen Tran + 4 more
3D pharmacophore models describe the ligand’s chemical interactions in their bioactive conformation. They offer a simple but sophisticated approach to decipher the chemically encoded ligand information, making them a valuable tool in Drug Design. Our research summarized the key studies for applying 3D pharmacophore…
Esther Heid, Charles J. McGill, Florence H. Vermeire, William H. Green
Characterizing uncertainty in machine learning models has recently gained interest in the context of machine learning reliability, robustness, safety, and active learning. Here, we separate the total uncertainty into contributions from noise in the data (aleatoric) and shortcomings of the model (epistemic), further…
Maria H. Rasmussen, Chenru Duan, Heather J. Kulik, Jan Halborg Jensen
With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet…
Chen Jie, Huang Min, Chen Bin, Sun Ziwen
Evaluating the effectiveness of education management requires the integration of multi-source data and information. Based on data modeling technology, combined with data enhancement and transfer learning methods, this paper analyzes the differences in the allocation of education management resources in six universities…
Chi Zhang, Dmytro Antypov, Matthew J Rosseinsky, Matthew Stephen Dyer
Machine learning has found wide application in the materials field, particularly in discovering structure-property relationships. However, its potential in predicting synthetic accessibility of materials remains relatively unexplored due to the lack of negative data. In this study, we employ several one-class…
Authors not listed
Ensuring the trustworthiness of machine learning (ML) models in high-stake applications is crucial. One such application is predicting anti-cancer drug sensitivity, where ML models are built with the final goal of integrating them into treatment recommendation systems for personalized medicine. Here, we propose a…
Authors not listed
Accurate determination of the metabolic fate of xenobiotics is essential for ensuring their safety and efficacy. While in vivo and in vitro methods remain the gold standard for assessing metabolic properties, they are both costly and time-consuming. In silico metabolism prediction models offer complementary solutions…