Online Agnostic Boosting via Regret Minimization
Nataly Brukhim,Xinyi Chen,Elad Hazan,Shay Moran
Boosting is a widely used machine learning approach based on the idea of aggregating weak learningrules. While in statistical learning numerous boosting methods exist both in the realizable and agnosticsettings, in online learning they exist only in the realizable case. In this work we provide the first agnosticonline boosting algorithm; that is, given a weak learner with only marginally-better-than-trivial regretguarantees, our algorithm boosts it to a strong learner with sublinear regret.Our algorithm is based on an abstract (and simple) reduction to online convex optimization, whichefficiently converts an arbitrary online convex optimizer to an online booster. Moreover, this reductionextends to the statistical as well as the online realizable settings, thus unifying the 4 cases of statistical/online and agnostic/realizable boosting.
