17 papers · ranked by Valyu relevance
Andreas Mayr, Harald Binder, Olaf Gefeller, Matthias Schmid
Background: The concept of boosting emerged from the field of machine learning. The basic idea is to boost the accuracy of a weak classifying tool by combining various instances into a more accurate prediction. This general concept was later adapted to the field of statistical modelling. Nowadays, boosting algorithms…
Zhiyuan He, Danchen Lin, Thomas Lau, Mike Wu
Proposed by Freund and Schapire (1997), boosting is a general issue of constructing an extremely accurate prediction with numerous roughly accurate predictions. Addressed by Friedman (2001, 2002) and Natekin and Knoll (2013), the Gradient Boosting Machines (GBM) seeks to build predictive models through back-fittings…
Hristos Tyralis, Georgia Papacharalampous
Machine learning algorithms have been extensively exploited in energy research, due to their flexibility, automation and ability to handle big data. Among the most prominent machine learning algorithms are the boosting ones, which are known to be "garnering wisdom from a council of fools", thereby transforming weak…
Robert E. Schapire
Boosting is a general method of generating many simple classification rules and combining them into a single, highly accurate rule. This paper reviews the AdaBoost boosting algorithm and some of its underlying theory, and then looks at some of the challenges of applying AdaBoost to bidding in complicated auctions and…
Gérard Biau, Benoı̂t Cadre, Laurent Rouvìère
Gradient tree boosting is a prediction algorithm that sequentially produces a model in the form of linear combinations of decision trees, by solving an infinite-dimensional optimization problem. We combine gradient boosting and Nesterov's accelerated descent to design a new algorithm, which we call AGB (for Accelerated…
Fabio Sigrist
We introduce a novel boosting algorithm called 'KTBoost' which combines kernel boosting and tree boosting. In each boosting iteration, the algorithm adds either a regression tree or reproducing kernel Hilbert space (RKHS) regression function to the ensemble of base learners. Intuitively, the idea is that discontinuous…
Nan Wang
The fields of machine learning and mathematical optimization increasingly intertwined. The special topic on supervised learning and convex optimization examines this interplay. The training part of most supervised learning algorithms can usually be reduced to an optimization problem that minimizes a loss between model…
Guo Guang-xu, Songcan Chen
— Schapire's margin theory provides a theoretical explanation to the success of boosting-type methods and manifests that a good margin distribution (MD) of training samples is essential for generalization. However the statement that a MD is good is vague, consequently, many recently developed algorithms try to generate…
Aleksandr Y. Aravkin, Giulio Bottegal, Gianluigi Pillonetto
Boosting combines weak (biased) learners to obtain effective learning algorithms for classification and prediction. In this paper, we show a connection between boosting and kernelbased methods, highlighting both theoretical and practical applications. In the context of `2 boosting, we start with a weak linear learner…
Mohammad Taha Toghani, Genevera I. Allen
—Boosting methods are among the best generalpurpose and off-the-shelf machine learning approaches, gaining widespread popularity. In this paper, we seek to develop a boosting method that yields comparable accuracy to popular AdaBoost and gradient boosting methods, yet is faster computationally and whose solution is…
Kaidong Wang, Yao Wang, Qian Zhao, Deyu Meng + 1 more
—It is known that Boosting can be interpreted as a gradient descent technique to minimize an underlying loss function. Specifically, the underlying loss being minimized by the traditional AdaBoost is the exponential loss, which is proved to be very sensitive to random noise/outliers. Therefore, several Boosting…
Kyongche Kang, Jack Michalak
Machine Learning focuses on the construction and study of systems that can learn from data. This is connected with the classification problem, which usually is what Machine Learning algorithms are designed to solve. When a machine learning method is used by people with no special expertise in machine learning, it is…
Robert M. Freund, Paul Grigas, Rahul Mazumder
In this paper we analyze boosting algorithms [15, 21, 24] in linear regression from a new perspective: that of modern first-order methods in convex optimization. We show that classic boosting algorithms in linear regression, namely the incremental forward stagewise algorithm (FSε) and least squares boosting…
Michał Sołtys, Szymon Jaroszewicz
Uplift modeling is an area of machine learning which aims at predicting the causal effect of some action on a given individual. The action may be a medical procedure, marketing campaign, or any other circumstance controlled by the experimenter. Building an uplift model requires two training sets: the treatment group…
Richard Nock, Yishay Mansour
Boosting is a highly successful ML-born optimization setting in which one is required to computationally efficiently learn arbitrarily good models based on the access to a weak learner oracle, providing classifiers performing at least slightly differently from random guessing. A key difference with gradient-based…
Fei Wang, Zhongheng Li, Fang He, Rong Wang + 2 more
'Feiping Nie'] Abstract—The AdaBoost algorithm has the superiority of resisting overfitting. Understanding the mysteries of this phenomena is a very fascinating fundamental theoretical problem. Many studies are devoted to explaining it from statistical view and margin theory. In this paper, we illustrate it from…
Abraham J. Wyner, Matthew Olson, Justin Bleich, David Mease
There is a large literature explaining why AdaBoost is a successful classifier. The literature on AdaBoost focuses on classifier margins and boosting's interpretation as the optimization of an exponential likelihood function. These existing explanations, however, have been pointed out to be incomplete. A random forest…