Search · four archives
Search · four archives
21 papers · ranked by Valyu relevance
Sheng Liu, Jonathan Niles‐Weed, Narges Razavian, Carlos Fernandez‐Granda
'Carlos Fernandez‐Granda'] We propose a novel framework to perform classification via deep learning in the presence of noisy annotations. When trained on noisy labels, deep neural networks have been observed to first fit the training data with clean labels during an "early learning" phase, before eventually memorizing…
Aditya Golatkar, Alessandro Achille, Stefano Soatto
Regularization is typically understood as improving generalization by altering the landscape of local extrema to which the model eventually converges. Deep neural networks (DNNs), however, challenge this view: We show that removing regularization after an initial transient period has little effect on generalization…
Alessio Galatolo, Alfred Nilsson, Roderick Karlemstrand, Yineng Wang
The memorization problem is well-known in the field of computer vision. Liu et al. propose a technique called Early-Learning Regularization, which improves accuracy on the CIFAR datasets when label noise is present. This project replicates their experiments and investigates the performance on a real-world dataset with…
Madhu S. Advani, Andrew M. Saxe, Haim Sompolinsky
We perform an analysis of the average generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant “high-dimensional” regime where the number of free parameters in the network is on the order of or even larger than the number of examples in the dataset. Using random…
Christopher W. Bartlett, Jamie Bossenbroek, Yukie Ueyama, Patricia McCallinhart + 5 more
Early stopping is an extremely common tool to minimize overfitting, which would otherwise be a cause of poor generalization of the model to novel data. However, early stopping is a heuristic that, while effective, primarily relies on ad hoc parameters and metrics. Optimizing when to stop remains a challenge. In this…
Ruoqi Shen, Liyao Gao, Yi-An Ma
Early stopping is a simple and widely used method to prevent over-training neural networks. We develop theoretical results to reveal the relationship between the optimal early stopping time and model dimension as well as sample size of the dataset for certain linear models. Our results demonstrate two very different…
Abdulaziz Albahr, Marwan Albahar, Mohammed Thanoon, Muhammad Binsawad
'Muhammad Binsawad'] Heart diseases are characterized as heterogeneous diseases comprising multiple subtypes. Early diagnosis and prognosis of heart disease are essential to facilitate the clinical management of patients. In this research, a new computational model for predicting early heart disease is proposed. The…
Xinghua Liu, Ming Cao
—The paper proposes a novel regularization procedure for machine learning. The proposed high-order regularization (HR) provides new insight into regularization, which is widely used to train a neural network that can be utilized to approximate the action-value function in general reinforcement learning problems. The…
Maryam Dialameh, Ali Hamzeh, Hossein Rahmani
—Regularization plays a vital role in the context of deep learning by preventing deep neural networks from the danger of overfitting. This paper proposes a novel deep learning regularization method named as DL-Reg, which carefully reduces the nonlinearity of deep networks to a certain extent by explicitly enforcing the…
Nassim Sohaee
In Machine Learning, prediction quality is usually measured using different techniques and evaluation methods. In the regression models, the goal is to minimize the distance between the actual and predicted value. This error evaluation technique lacks a detailed evaluation of the type of errors that occur on specific…
Romik Ghosh, Dana Mastrovito, Stefan Mihalas
The human brain readily learns tasks in sequence without forgetting previous ones. Artificial neural networks (ANNs), on the other hand, need to be modified to achieve similar performance. While effective, many algorithms that accomplish this are based on weight importance methods that do not correspond to biological…
Authors not listed
Inverse molecular design aims to generate novel chemical structures that satisfy multiple property constraints, yet reinforcement-learning (RL) fine-tuning can be sensitive to how objectives are converted into a scalar reward. Here, we systematically analyze how scalarization choices and stabilization mechanisms shape…
Zhe Li, Josue Ortega Caro, Evgenia Rusak, Wieland Brendel + 5 more
Machine learning models have difficulty generalizing to data outside of the distribution they were trained on. In particular, vision models are usually vulnerable to adversarial attacks or common corruptions, to which the human visual system is robust. Recent studies have found that regularizing machine learning models…
Marijn van Vliet, Riitta Salmelin
Linear machine learning models “learn” a data transformation by being exposed to examples of input with the desired output, forming the basis for a variety of powerful techniques for analyzing neuroimaging data. However, their ability to learn the desired transformation is limited by the quality and size of the example…
Jeonghwan Cheon, Se-Bum Paik
Human infants begin life with limited visual capacities, such as low acuity and poor color sensitivity, due to gradual sensory maturation. In contrast, machine learning models are trained on high-fidelity inputs from the outset, often leading to shortcut learning and overfitting to spurious correlations. Here, we show…
Guanghui Zhang, Xinran Wang, Steven J. Luck
Regularization has been extensively used in multivariate pattern classification (MVPA; decoding) of EEG data to mitigate the risk of overfitting. N-fold cross-validation is also used to mitigate this risk, and it is often combined with averaging across trials to improve the signal-to-noise ratio. However, the impact of…
Omisa Jinsi, Margaret M. Henderson, Michael J. Tarr, Sathishkumar V. E.
'Sathishkumar V. E.'] Humans are born with very low contrast sensitivity, meaning that inputs to the infant visual system are both blurry and low contrast. Is this solely a byproduct of maturational processes or is there a functional advantage for beginning life with poor visual acuity? We addressed the impact of poor…
Sanjar Adilov
Machine learning models for molecular-property prediction typically work with molecular representations in the form of fingerprints, descriptors, or graphs. In case of fingerprints and descriptors, molecular representations usually comprise thousands of features, which causes the curse of dimensionality for many…
Authors not listed
Electrochemical impedance spectroscopy (EIS) coupled with distribution of relaxation times (DRT) analysis is a robust framework for characterizing electrochemical systems. However, DRT deconvolution is often plagued by spurious peaks, hindering accurate process identification and quantitative parameter estimation. To…
Authors not listed
Machine learning models are increasingly applied to heterogeneous materials datasets spanning different synthesis routes, measurement protocols, and structural classes. Although multi-task and representation-learning approaches are commonly used to improve predictive performance, the latent representations learned by…
Authors not listed
Machine olfaction—the artificial replication of the sense of smell—faces significant challenges due to the absence of large, standardized training datasets. Unlike vision, language, and audio models, which benefit from extensive corpora such as ImageNet, GLUE, and AudioSet, olfaction lacks scaled equivalents and…