Search · four archives
Search · four archives
14 papers · ranked by Valyu relevance
Amal Rannen Triki, Maxim Berman, Matthew B. Blaschko
Deep neural networks (DNNs) have become increasingly important due to their excellent empirical performance on a wide range of problems. However, regularization is generally achieved by indirect means, largely due to the complex set of functions defined by a network and the difficulty in measuring function complexity.…
Yuichi Yoshida, Takeru Miyato
We investigate the generalizability of deep learning based on the sensitivity to input perturbation. We hypothesize that the high sensitivity to the perturbation of data degrades the performance on it. To reduce the sensitivity to perturbation, we propose a simple and effective regularization method, referred to as…
Guillaume Lecué, Shahar Mendelson
Our approach gives a common framework that may be used in the analysis of learning problems and regularization problems alike. In particular, it sheds some light on the role various notions of sparsity have in regularization and on their connection with the size of subdifferentials of Ψ in a neighbourhood of the true…
Qianru Liu, Rui Wang, Yuesheng Xu, Mingsong Yan
We consider a regularization problem whose objective function consists of a convex fidelity term and a regularization term determined by the `1 norm composed with a linear transform. Empirical results show that the regularization with the `1 norm can promote sparsity of a regularized solution. It is the goal of this…
Meena Jagadeesan, Ilya Razenshteyn, Suriya Gunasekar
We provide a function space characterization of the inductive bias resulting from minimizing the `2 norm of the weights in multi-channel convolutional neural networks with linear activations and empirically test our resulting hypothesis on ReLU networks trained using gradient descent. We define an induced regularizer…
R. Flamary, N. Jrad, R. Phlypo, M. Congedo + 1 more
This work investigates the use of mixed-norm regularization for sensor selection in event-related potential (ERP) based brain-computer interfaces (BCI). The classification problem is cast as a discriminative optimization framework where sensor selection is induced through the use of mixed-norms. This framework is…
Nurdan Ayse Saran, Fatih Nar, Charles Elkan
This study presents a novel numerical approach that improves the training efficiency of binary logistic regression, a popular statistical model in the machine learning community. Our method achieves training times an order of magnitude faster than traditional logistic regression by employing a novel Soft-Plus…
Michaël Unser
The minimization of a data-fidelity term and an additive regularization functional gives rise to a powerful framework for supervised learning. In this paper, we present a unifying regularization functional that depends on an operator L and on a generic Radon-domain norm. We establish the existence of a minimizer and…
Rixon Crane, Fred Roosta
We consider regularization of non-convex optimization problems involving a non-linear leastsquares objective. By adding an auxiliary set of variables, we introduce a novel regularization framework whose corresponding objective function is not only provably invex, but it also satisfies the highly desirable…
Johannes Schwab, Stephan Antholzer, Markus Haltmeier
Deep learning and (deep) neural networks are emerging tools to address inverse problems and image reconstruction tasks. Despite outstanding performance, the mathematical analysis for solving inverse problems by neural networks is mostly missing. In this paper, we introduce and rigorously analyze families of deep…
Loïc Shi-Garrier, Nidhal Carla Bouaynaya, Daniel Delahaye, Boris Ryabko
'Boris Ryabko'] Despite their remarkable performance, deep learning models still lack robustness guarantees, particularly in the presence of adversarial examples. This significant vulnerability raises concerns about their trustworthiness and hinders their deployment in critical domains that require certified levels of…
Mehrsa Pourya, Sebastian Neumayer, Michael Unser
We propose a regularization scheme for image reconstruction that leverages the power of deep learning while hinging on classic sparsity-promoting models. Many deep-learning-based models are hard to interpret and cumbersome to analyze theoretically. In contrast, our scheme is interpretable because it corresponds to the…
Yuhua Fan, Ilkka Launonen, Mikko J Sillanpää, Patrik Waldmann
High-dimensional genomic datasets contain complex patterns shaped by substantial biological noise, which pose major challenges for predictive modeling in genetics and breeding. Residual neural networks (ResNets) provide a powerful framework for capturing nonlinear genomic effects, but often overfit in settings where…
Leon Bungert, Martin Burger, Yury Korolev, Carola-Bibiane Schönlieb
We study variational regularisation methods for inverse problems with imperfect forward operators whose errors can be modelled by order intervals in a partial order of a Banach lattice. We carry out analysis with respect to existence and convex duality for general data fidelity terms and regularisation functionals.…