Search · four archives
Search · four archives
15 papers · ranked by Valyu relevance
Shashank Shekhar, Adesh Bansode, Asif Salim
—Most of the machine learning models have associated hyper-parameters along with their parameters. While the algorithm gives the solution for parameters, its utility for model performance is highly dependent on the choice of hyperparameters. For a robust performance of a model, it is necessary to find out the right…
Lei Cui, Yangguang Li, Xin Lu, Dong An + 1 more
Bayesian Optimization (BO) is a common solution to search optimal hyperparameters based on sample observations of a machine learning model. Existing BO algorithms could converge slowly even collapse when the potential observation noise misdirects the optimization. In this paper, we propose a novel BO algorithm called…
Park, Joon-Hyun, Cheon, Mujin + 4 more
The performance of Bayesian optimization (BO), a highly sample-efficient method for expensive black-box problems, is critically governed by the selection of its hyperparameters, including the kernel and acquisition functions. This presents a significant practical challenge: an inappropriate combination of these can…
Carl Hvarfner, Erik Hellsten, Frank Hutter, Luigi Nardi
Gaussian processes are the model of choice in Bayesian optimization and active learning. Yet, they are highly dependent on cleverly chosen hyperparameters to reach their full potential, and little effort is devoted to finding good hyperparameters in the literature. We demonstrate the impact of selecting good…
Sigrid Passano Hellan, Christopher G. Lucas, Nigel Goddard
Transfer learning for Bayesian optimisation has generally assumed a strong similarity between optimisation tasks, with at least a subset having similar optimal inputs. This assumption can reduce computational costs, but it is violated in a wide range of optimisation problems where transfer learning may nonetheless be…
Jiazhao Zhang, Ying Hung, Chung-Ching Lin, Zicheng Liu
Optimization Authors: ['Jiazhao Zhang' 'Ying Hung' 'Chung-Ching Lin' 'Zicheng Liu'] Choosing appropriate hyperparameters plays a crucial role in the success of neural networks as hyperparameters directly control the behavior and performance of the training algorithms. To obtain efficient tuning, Bayesian optimization…
Xilu Wang, Yaochu Jin, Sebastian Schmitt, Markus Olhofer
Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the development of new Bayesian optimization algorithms and their applications. Hence, this paper attempts to provide a comprehensive and updated…
Huayuan Sun
—The choices of hyperparameters have critical effects on the performance of machine learning models. In this paper, we present a general framework that is able to construct an adaptive optimizer, which automatically adjust the appropriate algorithm and parameters in the process of optimization. Examining the method of…
Eduardo C. Garrido‐Merchán
Several scenarios require the optimization of non-convex black-box functions, that are noisy expensive to evaluate functions with unknown analytical expression, whose gradients are hence not accessible. For example, the hyper-parameter tuning problem of machine learning models. Bayesian optimization is a class of…
Ankur Sinha, Satender Gunwal, Shivam Kumar
—Hyperparameter optimization in machine learning is often achieved using naive techniques, such as random and grid search, that only lead to an approximate set of hyperparameters. Although techniques such as Bayesian optimization perform an intelligent search, it does not guarantee an optimal solution. A major drawback…
Wenhu Li, Niki van Stein, Thomas Bäck, Elena Raponi
Bayesian optimization (BO) is a powerful class of algorithms for optimizing expensive black-box functions, but designing effective BO algorithms remains a manual, expertise-driven task. Recent advancements in Large Language Models (LLMs) have opened new avenues for automating scientific discovery, including the…
Jonathan Gornet, Yiannis Kantaros, Bruno Sinopoli
—We introduce Hyperparameter Controller (Hyper-Controller), a computationally efficient algorithm for hyperparameter optimization during training of reinforcement learning neural networks. HyperController optimizes hyperparameters quickly while also maintaining improvement of the reinforcement learning neural network…
Farhad Mirkarimi
Many real-world tasks require optimizing expensive black-box functions accessible only through noisy evaluations, a setting commonly addressed with Bayesian optimization (BO). While Bayesian neural networks (BNNs) have recently emerged as scalable alternatives to Gaussian Processes (GPs), traditional BNN-BO frameworks…
André L. Marchildon, David W. Zingg
The probabilistic surrogates used by Bayesian optimizers make them popular methods when function evaluations are noisy or expensive to evaluate. While Bayesian optimizers are traditionally used for global optimization, their benefits are also valuable for local optimization. In this paper, a framework for…
Mike Diessner, Kevin J. Wilson, Richard D. Whalley
expensive experiments and simulations with changing environmental conditions Authors: ['Mike Diessner' 'Kevin J. Wilson' 'Richard D. Whalley'] Experiments in engineering are typically conducted in controlled environments where parameters can be set to any desired value. This assumes that the same applies in a…