24 papers · ranked by Valyu relevance
Han Bao, Clayton Scott, Masashi Sugiyama
This document is a corrigendum to Bao et al. (2020), which used the wrong definition of calibration from Steinwart (2007). This corrigendum uses the correct definition. Fortunately, all of the main results from the published version still hold, with occasional minor changes. This corrigendum also removes an erroneous…
Ruihao Li, Chunlian Fu, Wei Yi, Xiaodong Yi
The low-cost Inertial Measurement Unit (IMU) can provide orientation information and is widely used in our daily life. However, IMUs with bad calibration will provide inaccurate angular velocity and lead to rapid drift of integral orientation in a short time. In this paper, we present the Calib-Net which can achieve…
Lars van der Laan, Ahmed Alaa
Conformal Prediction Authors: ['Lars van der Laan' 'Ahmed Alaa'] Ensuring model calibration is critical for reliable predictions, yet popular distribution-free methods, such as histogram binning and isotonic regression, provide only asymptotic guarantees. We introduce a unified framework for Venn and Venn-Abers…
Bernardo Ávila Pires, Csaba Szepesvári
In this paper we refine the process of computing calibration functions for a number of multiclass classification surrogate losses. Calibration functions are a powerful tool for easily converting bounds for the surrogate risk (which can be computed through well-known methods) into bounds for the true risk, the…
Ben Dai
in Classification Authors: ['Ben Dai'] Empirical risk minimization (ERM) with a computationally feasible surrogate loss is a widely accepted approach for classification. Notably, the convexity and calibration (CC) properties of a loss function ensure consistency of ERM in maximizing accuracy, thereby offering a wide…
Zhiguo Tan, Xing Zhang, Shuhua Teng, Ling Wang + 2 more
'Jesús Morales'] Extrinsic parameter calibration is the foundation and prerequisite for LiDAR and camera data fusion of the autonomous system. This technology is widely used in fields such as autonomous driving, mobile robots, intelligent surveillance, and visual measurement. The learning-based method is one of the…
Tatiana Krikella, Joel A. Dubin
Precision medicine is accelerating rapidly in the field of health research. This includes fitting predictive models for individual patients based on patient similarity in an attempt to improve model performance. We propose an algorithm which fits a personalized predictive model (PPM) using an optimal size of a similar…
Tomasz Kuśmierczyk, Joseph Sakaya, Arto Klami
Bayesian decision theory outlines a rigorous framework for making optimal decisions based on maximizing expected utility over a model posterior. However, practitioners often do not have access to the full posterior and resort to approximate inference strategies. In such cases, taking the eventual decision-making task…
Dan Levi, Liran Gispan, Niv Giladi, Ethan Fetaya + 1 more
Predicting not only the target but also an accurate measure of uncertainty is important for many machine learning applications, and in particular, safety-critical ones. In this work, we study the calibration of uncertainty prediction for regression tasks which often arise in real-world systems. We show that the…
Xiaoqian Jiang, Aditya Menon, Shuang Wang, Jihoon Kim + 2 more
'Lucila Ohno-Machado' 'Shree Ram Singh'] Historically, probabilistic models for decision support have focused on discrimination, e.g., minimizing the ranking error of predicted outcomes. Unfortunately, these models ignore another important aspect, calibration, which indicates the magnitude of correctness of model…
Hansol X. Ryu, Manoj Srinivasan
Studying how humans perceive patterns in visually presented data is useful for understanding data-based decision-making and potentially understanding visually mediated sensorimotor control. We conducted experiments to examine how human subjects perform the simplest machine learning or statistical estimation tasks…
Ke Wang, Qingwen Xue, Jian John Lu, Ediriweera Desapriya + 1 more
'Kazuko Okamura'] Identifying high-risk drivers before an accident happens is necessary for traffic accident control and prevention. Due to the class-imbalance nature of driving data, high-risk samples as the minority class are usually ill-treated by standard classification algorithms. Instead of applying preset…
Telmo M. Silva Filho, Hao Song, Miquel Perelló-Nieto, Raúl Santos‐Rodríguez + 2 more
'Raúl Santos‐Rodríguez' 'Meelis Kull' 'Peter Flach'] This paper provides both an introduction to and a detailed overview of the principles and practice of classifier calibration. A well-calibrated classifier correctly quantifies the level of uncertainty or confidence associated with its instance-wise predictions. This…
Maria H. Rasmussen, Chenru Duan, Heather J. Kulik, Jan Halborg Jensen
With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet…
Husam Abdulnabi, J. Timothy Westwood
A quantitative measurement can have variation, referred to here as measurement variation, which is a probability distribution. Machine Learning models typically produce a prediction corresponding to the mode of the measurement variation. The Deviation Error is a novel metric, described here, to assess predictions that…
Xudong Lv, Shuo Wang, Dong Ye, Aboelmagd Noureldin
As an essential procedure of data fusion, LiDAR-camera calibration is critical for autonomous vehicles and robot navigation. Most calibration methods require laborious manual work, complicated environmental settings, and specific calibration targets. The targetless methods are based on some complex optimization…
Laura Marie Helleckes, Michael Osthege, Wolfgang Wiechert, Eric von Lieres + 1 more
High-throughput experimentation has revolutionized data-driven experimental sciences and opened the door to the application of machine learning techniques. Nevertheless, the quality of any data analysis strongly depends on the quality of the data and specifically the degree to which random effects in the experimental…
Authors not listed
Physics-based coarse-grained (CG) models are widely used in (bio)molecular simulations, yet their parameterization remains challenging and labor-intensive. In this work, we demonstrate how recently developed gradient-based optimization methods can substantially accelerate the refinement of CG force field (FF)…
Gaspard Dussert, Simon Chamaillé-Jammes, Stéphane Dray, Vincent Miele
In this paper, we investigate whether deep learning models for species classification in camera trap images are well calibrated, i.e. whether predicted confidence scores can be reliably interpreted as probabilities that the predictions are true. Additionally, as camera traps are often configured to take multiple photos…
Authors not listed
Machine learning (ML) models are increasingly used in quantum chemistry, but their reliability hinges on uncertainty quantification (UQ). In this study, we compare two prominent UQ paradigms—Deep Evidential Regression (DER) and Deep Ensembles—on the QM9 and WS22 datasets, with a specific emphasis on the role of post…
Chonghuan Zhang, Qianyue Zhang, Alexei Lapkin
Biochemical transformations may allow significant improvements in synthetic efficiency of complex functional molecules through reduction in the number of synthetic steps or avoidance of harsh conditions and/or toxic solvents/reactants. Yet, there is a limited access to biochemical reaction data, which reduces the…
María Peña Fernández, Lara Lloret Iglesias, Jesús Marco de Lucas
One of the most compelling ideas for bridging neuroscience and artificial neural networks is the establishment of a framework based on three main components: network architecture, optimization mechanism, and loss (or objective) function to be minimized. While the first two components have been extensively explored, the…
Esther Heid, Charles J. McGill, Florence H. Vermeire, William H. Green
Characterizing uncertainty in machine learning models has recently gained interest in the context of machine learning reliability, robustness, safety, and active learning. Here, we separate the total uncertainty into contributions from noise in the data (aleatoric) and shortcomings of the model (epistemic), further…
Shomik Verma, Miguel Rivera, David O. Scanlon, Aron Walsh
Understanding the excited state properties of molecules provides insights into how they interact with light. These interactions can be exploited to design compounds for photochemical applications, including enhanced spectral conversion of light to increase the efficiency of photovoltaic cells. While chemical discovery…