25 papers · ranked by Valyu relevance
Jacek A Kopec, Philippe Finès, Douglas G Manuel, David L Buckeridge + 9 more
Background Computer simulation models are used increasingly to support public health research and policy, but questions about their quality persist. The purpose of this article is to review the principles and methods for validation of population-based disease simulation models. Methods We developed a comprehensive…
Lourdes Cucurull‐Sanchez, Michael J. Chappell, Vijayalakshmi Chelliah, S. Y. Amy Cheung + 9 more
'S. Y. Amy Cheung' 'Gianne Derks' 'Mark Penney' 'Alex Phipps' 'Rahuman S. Malik‐Sheriff' 'Jon Timmis' 'Marcus J. Tindall' 'Piet H. van der Graaf' 'Paolo Vicini' 'James W. T. Yates'] The lack of standardization in the way that quantitative and systems pharmacology (QSP) models are developed, tested, and documented…
Marco Viceconti, Francesco Pappalardo, Blanca Rodriguez, Marc Horner + 2 more
'Jeff Bischoff' 'Flora Musuamba Tshinanu'] Title: Highlights 1. • Regulators now consider also evidences produced in silico. 2. • We need accepted methods to evaluate the credibility of models. 3. • In this paper we describe the use of the ASME V&V-40 technical standard. 4. • We also discuss its application to various…
John D. Jakeman, Lorena A. Barba, Joaquim R. R. A. Martins, Thomas O'Leary-Roseberry
"Thomas O'Leary-Roseberry"] Scientific machine learning (SciML) models are transforming many scientific disciplines. However, the development of good modeling practices to increase the trustworthiness of SciML has lagged behind its application, limiting its potential impact. The goal of this paper is to start a…
José Camacho
The validation of a data-driven model is the process of assessing the model's ability to generalize to new, unseen data in the population of interest. This paper proposes a set of general rules for model validation. These rules are designed to help practitioners create reliable validation plans and report their results…
Jennifer A. Rohrs, Sahak Z. Makaryan, Stacey D. Finley
Systems biology combines computational modeling with quantitative experimental measurements to study complex biological processes. Here, we outline an approach for parameterizing and validating a systems biology model to yield predictive tool that can generate testable hypotheses and expand biological understanding.
Kevin Vanslette, Tony Tohme, Kamal Youcef‐Toumi
We construct and propose the "Bayesian Validation Metric" (BVM) as a general model validation and testing tool. We find the BVM to be capable of representing all of the standard validation metrics (square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical…
You Ling, Sankaran Mahadevan
This paper develops new insights into quantitative methods for the validation of computational model prediction. Four types of methods are investigated, namely classical and Bayesian hypothesis testing, a reliability-based method, and an area metric-based method. Traditional Bayesian hypothesis testing is extended…
Mahmood Alaghmandan, Olga Streltchenko
In light of recent allegations of misconduct concerning highly regarded academic research projects in behavioural science (see [6] and [10]) we review model risk management practices commonly applied in financial institutions. We maintain these are transferable to academic research with clear benefits for the quality…
David E Carlson, Ricardo Chavarriaga, Yiling Liu, Fabien Lotte + 1 more
'Bao-Liang Lu'] Title: Abstract Objective. Machine learning’s (MLs) ability to capture intricate patterns makes it vital in neural engineering research. With its increasing use, ensuring the validity and reproducibility of ML methods is critical. Unfortunately, this has not always been the case in practice, as there…
Evgueni Jacob, Angélique Perrillat-Mercerot, Jean-Louis Palgen, Adèle L’Hostis + 5 more
Over the past several decades, metrics have been defined to assess the quality of various types of models and to compare their performance depending on their capacity to explain the variance found in real-life data. However, available validation methods are mostly designed for statistical regressions rather than for…
Marco Viceconti, Miguel A. Juárez, Cristina Curreli, Marzio Pennisi + 2 more
'Giulia Russo' 'Francesco Pappalardo'] Abstract— Different research communities have developed various approaches to assess the credibility of predictive models. Each approach usually works well for a specific type of model, and under some epistemic conditions that are normally satisfied within that specific research…
Todd Oliver, Gabriel Terejanu, Christopher S. Simmons, Robert Moser
The ultimate purpose of most computational models is to make predictions, commonly in support of some decision-making process (e.g., for design or operation of some system). The quantities that need to be predicted (the quantities of interest or QoIs) are generally not experimentally observable before the prediction…
Fulya Akpinar Singh, Nasrin Afzal, Shepard J. Smithline, Craig J. Thalhauser
'Craig J. Thalhauser'] Validation of a quantitative model is a critical step in establishing confidence in the model’s suitability for whatever analysis it was designed. While processes for validation are well-established in the statistical sciences, the field of quantitative systems pharmacology (QSP) has taken a more…
Richard C. Gerkin, Justas Birgiolas, Russell J. Jarvis, Cyrus Omar + 1 more
Validating a quantitative scientific model requires comparing its predictions against many experimental observations, ideally from many labs, using transparent, robust, statistical comparisons. Unfortunately, in rapidly-growing fields like neuroscience, this is becoming increasingly untenable, even for the most…
Giuseppe Gallitto, Robert Englert, Balint Kincses, Raviteja Kotikalapudi + 4 more
Multivariate predictive models play a crucial role in enhancing our understanding of complex biological systems and in developing innovative, replicable tools for translational medical research. However, the complexity of machine learning methods and extensive data pre-processing and feature engineering pipelines can…
Udit Surya Saha, Michele Vendruscolo, Anne E. Carpenter, Shantanu Singh + 2 more
Recent advances in machine learning methods for materials science have significantly enhanced accurate predictions of the properties of novel materials. Here, we explore whether these advances can be adapted to drug discovery by addressing the problem of prospective validation - the assessment of the performance of a…
Matthew Witman, Peter Schindler
Machine learning (ML) models in the materials sciences that are validated by overly simplistic cross-validation (CV) protocols can yield biased performance estimates for downstream modeling or materials screening tasks. This can be particularly counterproductive for applications where the time and cost of failed…
Xuan-Truc Dinh Tran, Tieu-Long Phan, Van-Thinh To, Ngoc-Vi Nguyen Tran + 4 more
3D pharmacophore models describe the ligand’s chemical interactions in their bioactive conformation. They offer a simple but sophisticated approach to decipher the chemically encoded ligand information, making them a valuable tool in Drug Design. Our research summarized the key studies for applying 3D pharmacophore…
Esther Heid, Charles J. McGill, Florence H. Vermeire, William H. Green
Characterizing uncertainty in machine learning models has recently gained interest in the context of machine learning reliability, robustness, safety, and active learning. Here, we separate the total uncertainty into contributions from noise in the data (aleatoric) and shortcomings of the model (epistemic), further…
Liwei Cao, Danilo Russo, Vassilios S. Vassiliadis, Alexei Lapkin
A mixed-integer nonlinear programming (MINLP) formulation for symbolic regression was proposed to identify physical models from noisy experimental data. The formulation was tested using numerical models and was found to be more efficient than the previous literature example with respect to the number of predictor…
Maria H. Rasmussen, Chenru Duan, Heather J. Kulik, Jan Halborg Jensen
With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet…
Chi Zhang, Dmytro Antypov, Matthew J Rosseinsky, Matthew Stephen Dyer
Machine learning has found wide application in the materials field, particularly in discovering structure-property relationships. However, its potential in predicting synthetic accessibility of materials remains relatively unexplored due to the lack of negative data. In this study, we employ several one-class…
Authors not listed
Ensuring the trustworthiness of machine learning (ML) models in high-stake applications is crucial. One such application is predicting anti-cancer drug sensitivity, where ML models are built with the final goal of integrating them into treatment recommendation systems for personalized medicine. Here, we propose a…
Authors not listed
Accurate determination of the metabolic fate of xenobiotics is essential for ensuring their safety and efficacy. While in vivo and in vitro methods remain the gold standard for assessing metabolic properties, they are both costly and time-consuming. In silico metabolism prediction models offer complementary solutions…