Search · four archives
Search · four archives
7 papers · ranked by Valyu relevance
Răzvan Andonie, Adrian-Catalin Florea
Nearly all model algorithms used in machine learning use two dierent sets of parameters: the training parameters and the meta-parameters (hyperparameters). While the training parameters are learned during the training phase, the values of the hyperparameters have to be specified before learning starts. For a given…
Adrian-Catalin Florea, Răzvan Andonie
We introduce an improved version of Random Search (RS), used here for hyperparameter optimization of machine learning algorithms. Unlike the standard RS, which generates for each trial new values for all hyperparameters, we generate new values for each hyperparameter with a probability of change. The intuition behind…
Shashank Shekhar, Adesh Bansode, Asif Salim
—Most of the machine learning models have associated hyper-parameters along with their parameters. While the algorithm gives the solution for parameters, its utility for model performance is highly dependent on the choice of hyperparameters. For a robust performance of a model, it is necessary to find out the right…
Patrick Koch, Oleg Golovidov, Steven D. Gardner, Brett Wujek + 2 more
'Joshua Griffin' 'Yan Xu'] Machine learning applications often require hyperparameter tuning. The hyperparameters usually drive both the efficiency of the model training process and the resulting model quality. For hyperparameter tuning, machine learning algorithms are complex black-boxes. This creates a class of…
Mathias Lechner, Ramin Hasani, Philipp Neubauer, Sophie Neubauer + 1 more
'Daniela Rus'] Hyperparameter tuning is a fundamental aspect of machine learning research. Setting up the infrastructure for systematic optimization of hyperparameters can take a significant amount of time. Here, we present PyHopper, a black-box optimization platform designed to streamline the hyperparameter tuning…
Gonzalo I. Diaz, Achille Fokoue, Giacomo Nannicini, Horst Samulowitz
A major challenge in designing neural network (NN) systems is to determine the best structure and parameters for the network given the data for the machine learning problem at hand. Examples of parameters are the number of layers and nodes, the learning rates, and the dropout rates. Typically, these parameters are…
Takayuki Okuno, Akiko Takeda, Akihiro Kawana
We propose a bilevel optimization strategy for selecting the best hyperparameter value for the nonsmooth ℓp regularizer with 0 < p ≤ 1. The concerned bilevel optimization problem has a nonsmooth, possibly nonconvex, ℓp-regularized problem as the lower-level problem. Despite the recent popularity of nonconvex ℓp…