14 papers · ranked by Valyu relevance
Nir Shlezinger, Santiago Segarra, Yi Zhang, Dvir Avrahami + 3 more
Optimization methods play a central role in signal processing, serving as the mathematical foundation for inference, estimation, and control. While classical iterative optimization algorithms provide interpretability and theoretical guarantees, they often rely on surrogate objectives, require careful hyperparameter…
Demelas, Francesca, Roux, Joseph Le + 8 more
This paper presents Bundle Network, a learning-based algorithm inspired by the Bundle Method for convex non-smooth minimization problems. Unlike classical approaches that rely on heuristic tuning of a regularization parameter, our method automatically learns to adjust it from data. Furthermore, we replace the iterative…
Yao Shu, Jiongfeng Fang, Ying Tiffany He, F. Richard Yu
Parallelized Iterations Authors: ['Yao Shu' 'Jiongfeng Fang' 'Ying Tiffany He' 'F. Richard Yu'] First-order optimization (FOO) algorithms are pivotal in numerous computational domains such as machine learning and signal denoising. However, their application to complex tasks like neural network training often entails…
Daouda Sow, Kaiyi Ji, Ziwei Guan, Yingbin Liang
Bilevel optimization has found extensive applications in modern machine learning problems such as hyperparameter optimization, neural architecture search, meta-learning, etc. While bilevel problems with a unique inner minimal point (e.g., where the inner function is strongly convex) are well understood, such a problem…
Benjamin Grimmer, Alex L. Wang
This paper considers nonsmooth convex optimization with either a subgradient or proximal operator oracle. In both settings, we identify algorithms that achieve the recently introduced game-theoretic optimality notion for algorithms known as subgame perfection. Subgame perfect algorithms meet a more stringent…
Adrien Taylor, Yoel Drori
We present an optimal gradient method for smooth strongly convex optimization. The method is optimal in the sense that its worst-case bound on the distance to an optimal point exactly matches the lower bound on the oracle complexity for the class of problems, meaning that no black-box first-order method can have a…
Ruichen Jiang, Ali Kavis, Qiujiang Jin, Sujay Sanghavi + 1 more
Optimization Authors: ['Ruichen Jiang' 'Ali Kavis' 'Qiujiang Jin' 'Sujay Sanghavi' 'Aryan Mokhtari'] We propose adaptive, line search-free second-order methods with optimal rate of convergence for solving convex-concave min-max problems. By means of an adaptive step size, our algorithms feature a simple update rule…
Jason Rader, Terry Lyons, Patrick Kidger
We introduce Optimistix: a nonlinear optimisation library built in JAX and Equinox. Optimistix introduces a novel, modular approach for its minimisers and least-squares solvers. This modularity relies on new practical abstractions for optimisation which we call search and descent, and which generalise classical notions…
Nikita Belokonev, Artem Melnikov, Maninadh Podapaka, Karan Pinto + 2 more
'Markus Pflitsch' 'Michael Perelshtein'] Chemical component design is a computationally challenging procedure that often entails iterative numerical modeling and authentic experimental testing. We demonstrate a novel optimization method, Tensor train Optimization (TetraOpt), for the shape optimization of components…
Yifan Yang, Hao Ban, Minhui Huang, Shiqian Ma + 1 more
Analysis Authors: ['Yifan Yang' 'Hao Ban' 'Minhui Huang' 'Shiqian Ma' 'Kaiyi Ji'] Bilevel optimization has recently attracted considerable attention due to its abundant applications in machine learning problems. However, existing methods rely on prior knowledge of problem parameters to determine stepsizes, resulting in…
Y. Pan
In this project, we reviewed a paper that deals graph-structured convex optimization (GSCO) problem with the approximate Frank-Wolfe (FW) algorithm. We analyzed and re-implemented the original algorithm and introduced some extensions based on that. Then we conducted experiments to compare the results and concluded that…
Serena Landers, Sahil Pontula, Shiekh Zia Uddin, Sachin Vaidya + 2 more
We introduce the CLUSTER algorithm (\textbf{c}oordinate-\textbf{l}evel \textbf{u}pdate \textbf{s}trategy for \textbf{t}rust-region step \textbf{e}valuation \textbf{r}efinement) for local derivative-free optimization problems where there is a cost to changing each parameter (or clusters of parameters). For example, this…
Hugo Silva, Martha White
Network? Authors: ['Hugo Silva' 'Martha White'] Oftentimes, machine learning applications using neural networks involve solving discrete optimization problems, such as in pruning, parameter-isolation-based continual learning and training of binary networks. Still, these discrete problems are combinatorial in nature and…
Saeed Asadi, Sonia Gharibzadeh, Shiva Zangeneh, Masoud Reihanifar + 2 more
Multidimensional Surface 3D Visualizations and Initial Point Sensitivity Authors: ['Saeed Asadi' 'Sonia Gharibzadeh' 'Shiva Zangeneh' 'Masoud Reihanifar' 'Mehrzad Rahimi' 'Lazim Abdullah'] This study examines several renowned gradient-based optimization techniques and focuses on their computational efficiency and…