Search · four archives

Results for early-learning regularization

or ask it as a questionAsk the stacks →

ArchivesarXiv PubMed bioRxiv chemRxiv

WhenAny time Past month Past year Past 5 years

OrderMost relevant Most cited Newest

15 papers · ranked by Valyu relevance

01
PPubMed24 Aug 2020
High-dimensional dynamics of generalization error in neural networks
Madhu S. Advani, Andrew M. Saxe, Haim Sompolinsky
We perform an analysis of the average generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant “high-dimensional” regime where the number of free parameters in the network is on the order of or even larger than the number of examples in the dataset. Using random…
82relevance
02
PPubMed12 Jan 2023
Invasive or More Direct Measurements Can Provide an Objective Early-Stopping Ceiling for Training Deep Neural Networks on Non-invasive or Less-Direct Biomedical Data
Christopher W. Bartlett, Jamie Bossenbroek, Yukie Ueyama, Patricia McCallinhart + 5 more
Early stopping is an extremely common tool to minimize overfitting, which would otherwise be a cause of poor generalization of the model to novel data. However, early stopping is a heuristic that, while effective, primarily relies on ad hoc parameters and metrics. Optimizing when to stop remains a challenge. In this…
73relevance
03
PPubMed30 Mar 2024
Prediction Consistency Regularization for Learning with Noise Labels Based on Contrastive Clustering
Xinkai Sun, Sanguo Zhang, Shuangge Ma, Sotiris Kotsiantis
In the classification task, label noise has a significant impact on models’ performance, primarily manifested in the disruption of prediction consistency, thereby reducing the classification accuracy. This work introduces a novel prediction consistency regularization that mitigates the impact of label noise on neural…
72relevance
04
PPubMed11 Nov 2021
Computational Learning Model for Prediction of Heart Disease Using Machine Learning Based on a New Regularizer
Abdulaziz Albahr, Marwan Albahar, Mohammed Thanoon, Muhammad Binsawad
'Muhammad Binsawad'] Heart diseases are characterized as heterogeneous diseases comprising multiple subtypes. Early diagnosis and prognosis of heart disease are essential to facilitate the clinical management of patients. In this research, a new computational model for predicting early heart disease is proposed. The…
70relevance
05
PPubMed1 Jun 2021
BEAN: Interpretable and Efficient Learning With Biologically-Enhanced Artificial Neuronal Assembly Regularization
Yuyang Gao, Giorgio A. Ascoli, Liang Zhao
In an attempt to test the influence of BEAN regularization on the generalizability of DNNs in the scenarios where the training samples are extremely limited, we conducted a few-shot learning from scratch task, i.e., without the help of any additional side tasks and pre-trained models (Kimura et al., ). Notice that in…
67relevance
06
PPubMed11 Mar 2020
Variational Networks: An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration
Alexander Effland, Erich Kobler, Karl Kunisch, Thomas Pock
We investigate a well-known phenomenon of variational approaches in image processing, where typically the best image quality is achieved when the gradient flow process is stopped before converging to a stationary point. This paradox originates from a tradeoff between optimization and modeling errors of the underlying…
65relevance
07
PPubMed28 Jan 2023
Error and optimism bias regularization
Nassim Sohaee
In Machine Learning, prediction quality is usually measured using different techniques and evaluation methods. In the regression models, the goal is to minimize the distance between the actual and predicted value. This error evaluation technique lacks a detailed evaluation of the type of errors that occur on specific…
65relevance
08
PPubMed18 May 2021
Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks
Abdulkadir Canatar, Blake Bordelon, Cengiz Pehlevan
A theoretical understanding of generalization remains an open problem for many machine learning models, including deep networks where overparameterization leads to better performance, contradicting the conventional wisdom from classical statistics. Here, we investigate generalization error for kernel regression, which…
63relevance
09
PPubMed4 Jan 2022
Adversarially Robust Learning <i> via </i> Entropic Regularization Jagatap et al. Adversarial Training With Entropic Regularization
Gauri Jagatap, Ameya Joshi, Animesh Basak Chowdhury, Siddharth Garg + 1 more
'Chinmay Hegde'] In this paper we propose a new family of algorithms, ATENT, for training adversarially robust deep neural networks. We formulate a new loss function that is equipped with an additional entropic regularization. Our loss function considers the contribution of adversarial samples that are drawn from a…
63relevance
10
PPubMed25 May 2026
Proximal regularization of deep residual neural networks applied to high-dimensional genomic data
Yuhua Fan, Ilkka Launonen, Mikko J Sillanpää, Patrik Waldmann
High-dimensional genomic datasets contain complex patterns shaped by substantial biological noise, which pose major challenges for predictive modeling in genetics and breeding. Residual neural networks (ResNets) provide a powerful framework for capturing nonlinear genomic effects, but often overfit in settings where…
63relevance
11
PPubMed6 Jan 2023
Early experience with low-pass filtered images facilitates visual category learning in a neural network model Training with blurred images improves basic-level category learning
Omisa Jinsi, Margaret M. Henderson, Michael J. Tarr, Sathishkumar V. E.
'Sathishkumar V. E.'] Humans are born with very low contrast sensitivity, meaning that inputs to the infant visual system are both blurry and low contrast. Is this solely a byproduct of maturational processes or is there a functional advantage for beginning life with poor visual acuity? We addressed the impact of poor…
62relevance
12
PPubMed5 Aug 2013
Iterative Reweighted Noninteger Norm Regularizing SVM for Gene Expression Data Classification
Jianwei Liu, Shuang Cheng Li, Xionglin Luo
Support vector machine is an effective classification and regression method that uses machine learning theory to maximize the predictive accuracy while avoiding overfitting of data. L2 regularization has been commonly used. If the training dataset contains many noise variables, L1 regularization SVM will provide a…
62relevance
13
PPubMed3 Oct 2019
Big in Japan: Regularizing Networks for Solving Inverse Problems
Johannes Schwab, Stephan Antholzer, Markus Haltmeier
Deep learning and (deep) neural networks are emerging tools to address inverse problems and image reconstruction tasks. Despite outstanding performance, the mathematical analysis for solving inverse problems by neural networks is mostly missing. In this paper, we introduce and rigorously analyze families of deep…
60relevance
14
PPubMed26 Sep 2014
Hessian-Regularized Co-Training for Social Activity Recognition Hessian-Regularized Co-Training
Weifeng Liu, Yang Li, Xu Lin, Dacheng Tao + 2 more
'Kewei Chen'] Co-training is a major multi-view learning paradigm that alternately trains two classifiers on two distinct views and maximizes the mutual agreement on the two-view unlabeled data. Traditional co-training algorithms usually train a learner on each view separately and then force the learners to be…
59relevance
15
PPubMed31 Jul 2020
Statistical Learning Signals for Complex Visual Images in Macaque Early Visual Cortex
Victor Vergnieux, Rufin Vogels
Animals of several species, including primates, learn the statistical regularities of their environment. In particular, they learn the temporal regularities that occur in streams of visual images. Previous human neuroimaging studies reported discrepant effects of such statistical learning, ranging from stronger…
58relevance

Results for early-learning regularization

High-dimensional dynamics of generalization error in neural networks

Invasive or More Direct Measurements Can Provide an Objective Early-Stopping Ceiling for Training Deep Neural Networks on Non-invasive or Less-Direct Biomedical Data

Prediction Consistency Regularization for Learning with Noise Labels Based on Contrastive Clustering

Computational Learning Model for Prediction of Heart Disease Using Machine Learning Based on a New Regularizer

BEAN: Interpretable and Efficient Learning With Biologically-Enhanced Artificial Neuronal Assembly Regularization

Variational Networks: An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration

Error and optimism bias regularization

Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks

Adversarially Robust Learning <i> via </i> Entropic Regularization Jagatap et al. Adversarial Training With Entropic Regularization

Proximal regularization of deep residual neural networks applied to high-dimensional genomic data

Early experience with low-pass filtered images facilitates visual category learning in a neural network model Training with blurred images improves basic-level category learning

Iterative Reweighted Noninteger Norm Regularizing SVM for Gene Expression Data Classification

Big in Japan: Regularizing Networks for Solving Inverse Problems

Hessian-Regularized Co-Training for Social Activity Recognition Hessian-Regularized Co-Training

Statistical Learning Signals for Complex Visual Images in Macaque Early Visual Cortex

High-dimensional dynamics of generalization error in neural networks

Invasive or More Direct Measurements Can Provide an Objective Early-Stopping Ceiling for Training Deep Neural Networks on Non-invasive or Less-Direct Biomedical Data

Prediction Consistency Regularization for Learning with Noise Labels Based on Contrastive Clustering

Computational Learning Model for Prediction of Heart Disease Using Machine Learning Based on a New Regularizer

BEAN: Interpretable and Efficient Learning With Biologically-Enhanced Artificial Neuronal Assembly Regularization

Variational Networks: An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration

Error and optimism bias regularization

Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks

Adversarially Robust Learning <i> via </i> Entropic Regularization Jagatap et al. Adversarial Training With Entropic Regularization

Proximal regularization of deep residual neural networks applied to high-dimensional genomic data

Early experience with low-pass filtered images facilitates visual category learning in a neural network model Training with blurred images improves basic-level category learning

Iterative Reweighted Noninteger Norm Regularizing SVM for Gene Expression Data Classification

Big in Japan: Regularizing Networks for Solving Inverse Problems

Hessian-Regularized Co-Training for Social Activity Recognition Hessian-Regularized Co-Training

Statistical Learning Signals for Complex Visual Images in Macaque Early Visual Cortex