19 papers · ranked by Valyu relevance
Nicholas G. Polson, James G. Scott, Brandon T. Willard
In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful in statistics and machine learning for obtaining optimization solutions for composite functions. Our approach exploits closedform solutions of proximal operators and envelope representations based on the Moreau…
Dmitriy Drusvyatskiy
In this short survey, I revisit the role of the proximal point method in large scale optimization. I focus on three recent examples: a proximally guided subgradient method for weakly convex stochastic approximation, the prox-linear algorithm for minimizing compositions of convex functions and smooth maps, and Catalyst…
Kevin L. Keys, Hua Zhou, Kenneth Lange
Proximal distance algorithms combine the classical penalty method of constrained minimization with distance majorization. If f(x) is the loss function, and C is the constraint set in a constrained minimization problem, then the proximal distance principle mandates minimizing the penalized loss…
Giovanni Chierchia, Nelly Pustelnik, Jean‐Christophe Pesquet, Béatrice Pesquet‐Popescu
'Béatrice Pesquet‐Popescu'] We propose a proximal approach to deal with a class of convex variational problems involving nonlinear constraints. A large family of constraints, proven to be effective in the solution of inverse problems, can be expressed as the lower level set of a sum of convex functions evaluated over…
Stanley Osher, Howard Heaton, Samy Wu Fung
Title: Significance Many objective functions do not admit explicit formulas for their proximal operators. Moreover, these operators often cannot be estimated using exact gradients (e.g., when objectives are accessible via an oracle). In this work, we give a formula for accurately approximating proximal operators using…
Filip Nikolovski, Irena Stojkovska, Katerina Saneva, Zoran Hadži-Velkov
'Zoran Hadži-Velkov'] Regularization is a widely recognized technique in mathematical optimization. It can be used to smooth out objective functions, refine the feasible solution set, or prevent overfitting in machine learning models. Due to its simplicity and robustness, the gradient descent (GD) method is one of the…
Alfonso Landeros, Oscar Hernan Madrid Padilla, Hua Zhou, Kenneth Lange
'Kenneth Lange'] The current paper studies the problem of minimizing a loss f(x) subject to constraints of the form Dx ∈ S, where S is a closed set, convex or not, and D is a matrix that fuses parameters. Fusion constraints can capture smoothness, sparsity, or more general constraint patterns. To tackle this generic…
Ziqiang Shi
In this work, we generalized and unified recent two completely different works of Jascha [9] and Lee [2] respectively into one by proposing the proximal stochastic Newton-type gradient (PROXTONE) method for optimizing the sums of two convex functions: one is the average of a huge number of smooth convex functions, and…
Kenneth Lange, Kevin L. Keys
The MM principle is a device for creating optimization algorithms satisfying the ascent or descent property. The current survey emphasizes the role of the MM principle in nonlinear programming. For smooth functions, one can construct an adaptive interior point method based on scaled Bregman barriers. This algorithm…
Pascal Fernsel, Fabiana Zama, Elena Loli Piccolomini
Classical approaches in cluster analysis are typically based on a feature space analysis. However, many applications lead to datasets with additional spatial information and a ground truth with spatially coherent classes, which will not necessarily be reconstructed well by standard clustering methods. Motivated by…
Stéphane Chrétien, Christophe Guyeux, Bastien Conesa, Régis Delage-Mouroux + 3 more
'Régis Delage-Mouroux' 'Michèle Jouvenot' 'Philippe Huetz' 'Françoise Descôtes'] Background Non-Negative Matrix factorization has become an essential tool for feature extraction in a wide spectrum of applications. In the present work, our objective is to extend the applicability of the method to the case of missing…
Hani N. Alsafadi, John Stegmayr, Victoria Ptasinski, Iran Silva + 3 more
The respiratory epithelium consists of multiple, functionally distinct cell-types and is maintained by regionally-specific progenitor populations which repair the epithelium following injury. Several in vitro methods exist for studying lung epithelial repair using primary murine lung epithelial cells, but isolation…
Arni Sturluson, Ali Raza, Grant D. McConachie, Daniel Siderius + 2 more
Nanoporous materials (NPMs) selectively adsorb and concentrate gases into their pores, and thus could be used to store, capture, and sense many different gases. Modularly synthesized classes of NPMs, such as covalent organic frameworks (COFs), offer a large number of candidate structures for each adsorption task. A…
Sungho Shin, Ophelia Venturelli, Victor M. Zavala
We present a nonlinear programming (NLP) framework for the scalable solution of parameter estimation problems that arise in dynamic modeling of biological systems. Such problems are computationally challenging because they often involve highly nonlinear and stif differential equations as well as many experimental data…
Kazunori D Yamada
In the deep learning era, a gradient descent method is the most common method to optimize parameters of neural networks. Among various mathematical optimization methods, a gradient descent method is the most naive method. Although controlling a learning rate of the method is necessary for quick convergence, the…
Ray Zirui Zhang, Christopher E. Miles, Xiaohui Xie, John S. Lowengrub
We propose a new neural network based method for solving inverse problems for partial differential equations (PDEs) by formulating the PDE inverse problem as a bilevel optimization problem. At the upper level, we minimize the data loss with respect to the PDE parameters. At the lower level, we train a neural network to…
Stephan Grein, David R. Penas, Daniel Weindl, Polina Lakrisenko + 2 more
Dynamic models are central to the computational life sciences but typically contain unknown parameters that must be inferred from experimental data. High-throughput measurements have made this task increasingly challenging, yielding high-dimensional search spaces and non-convex objectives with many local optima. This…
Hua-Dong Xiong, Li Ji-An, Marcelo G. Mattar, Robert C. Wilson
Cognitive modeling provides a formal method to articulate and test hypotheses about cognitive processes. However, accurately and reliably estimating model parameters remains challenging due to common issues in behavioral science, such as limited data, measurement noise, experimental constraints, and model complexity.…
Authors not listed
Solving optimization problems, especially for nonlinear and constrained systems, is a challenge. Decades of specialized algorithms have been developed for general and special cases of root finding, minimization (including constraints), for parameter estimation, and mapping connected spaces. These approaches typically…