Search · four archives
Search · four archives
17 papers · ranked by Valyu relevance
Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros + 5 more
'Nadathur Satish' 'Narayanan Sundaram' 'Mostofa Patwary' 'Prabhat' 'Ryan P. Adams'] Bayesian optimization is an effective methodology for the global optimization of functions with expensive evaluations. It relies on querying a distribution over functions defined by a relatively cheap surrogate model. An accurate model…
Stefan Falkner, Aaron Klein, Frank Hutter
Modern deep learning methods are very sensitive to many hyperparameters, and, due to the long training times of state-of-the-art models, vanilla Bayesian hyperparameter optimization is typically computationally infeasible. On the other hand, bandit-based configuration evaluation approaches based on random search lack…
Shashank Shekhar, Adesh Bansode, Asif Salim
—Most of the machine learning models have associated hyper-parameters along with their parameters. While the algorithm gives the solution for parameters, its utility for model performance is highly dependent on the choice of hyperparameters. For a robust performance of a model, it is necessary to find out the right…
Daniel T. Chang
Deep learning models are full of hyperparameters, which are set manually before the learning process can start. To find the best configuration for these hyperparameters in such a high dimensional space, with time-consuming and expensive model training / validation, is not a trivial challenge. Bayesian optimization is a…
Julien-Charles Lévesque, Christian Gagné, Robert Sabourin
In this paper, we bridge the gap between hyperparameter optimization and ensemble learning by performing Bayesian optimization of an ensemble with regards to its hyperparameters. Our method consists in building a fixed-size ensemble, optimizing the configuration of one classifier of the ensemble at each iteration of…
Kevin Swersky, Jasper Snoek, Ryan P. Adams
In machine learning, the term "training" is used to describe the procedure of fitting a model to data. In many popular models, this fitting procedure is framed as an optimization problem, in which a loss is minimized as a function of the parameters. In all but the simplest machine learning models, this minimization…
Rubén Martínez-Cantín
BayesOpt is a library with state-of-the-art Bayesian optimization methods to solve nonlinear optimization, stochastic bandits or sequential experimental design problems. Bayesian optimization is sample efficient by building a posterior distribution to capture the evidence and prior knowledge for the target function.…
Sigrid Passano Hellan, Christopher G. Lucas, Nigel Goddard
Transfer learning for Bayesian optimisation has generally assumed a strong similarity between optimisation tasks, with at least a subset having similar optimal inputs. This assumption can reduce computational costs, but it is violated in a wide range of optimisation problems where transfer learning may nonetheless be…
Tinu Theckel Joy, Santu Rana, Sunil Gupta, Svetha Venkatesh
In this paper, we develop a Bayesian optimization based hyperparameter tuning framework inspired by statistical learning theory for classifiers. We utilize two key facts from PAC learning theory; the generalization bound will be higher for a small subset of data compared to the whole, and the highest accuracy for a…
Abhilash Nandy, Chandan Kumar‐Sinha, Deepak Mewada, Soumya Sharma
In this report we survey Bayesian Optimization methods focussed on Multi-Armed Bandit Problem. We take the help of the paper "Portfolio Allocation for Bayesian Optimization" [1]. We report a small literature survey on the acquisition functions and the types of portfolio strategies used in the papers [1] [2]. We also…
Wenyi Wang, William J. Welch
We propose an algorithm for a family of optimization problems where the objective can be decomposed as a sum of functions with monotonicity properties. The motivating problem is optimization of hyperparameters of machine learning algorithms, where we argue that the objective, validation error, can be decomposed as…
Dipti Jasrasaria, Edward O. Pyzer‐Knapp
—Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration…
Antoine Blanchard, Themistoklis P. Sapsis
In Bayesian optimization, accounting for the importance of the output relative to the input is a crucial yet challenging exercise, as it can considerably improve the final result but often involves inaccurate and cumbersome entropy estimations. We approach the problem from the perspective of importance-sampling theory…
Oskar Gustafsson, Mattias Villani, Pär Stockhammar
Bayesian models often involve a small set of hyperparameters determined by maximizing the marginal likelihood. Bayesian optimization is a popular iterative method where a Gaussian process posterior of the underlying function is sequentially updated by new function evaluations. An acquisition strategy uses this…
Ran Rubin
Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a…
Doniyor Ulmasov, Caroline Baroukh, Benoît Chachuat, Marc Peter Deisenroth + 1 more
'Marc Peter Deisenroth' 'Ruth Misener'] Bayesian Optimization (BO) is a data-efficient method for global black-box optimization of an expensive-to-evaluate fitness function. BO typically assumes that computation cost of BO is cheap, but experiments are time consuming or costly. In practice, this allows us to optimize…
Anil Ramachandran, Sunil Gupta, Santu Rana, Cheng Li + 1 more
'Svetha Venkatesh'] Bayesian optimisation is a well-known sample-efficient method for the optimisation of expensive black-box functions. However when dealing with big search spaces the algorithm goes through several low function value regions before reaching the optimum of the function. Since the function evaluations…