PAC-Bayes Analysis Beyond the Usual Bounds
Omar Rivasplata,Ilja Kuzborskij,Csaba Szepesvari,John Shawe-Taylor
Specifically, we present a basic PAC-Bayes inequality for stochastic kernels, from which one may derive extensions of various known PAC-Bayes bounds as well as novel bounds. We clarify the role of the requirements of fixed u2018data-freeu2019 priors, bounded losses, and i.i.d. data. We highlight that those requirements were used to upper-bound an exponential moment term, while the basic PAC-Bayes theorem remains valid without those restrictions. We present three bounds that illustrate the use of data-dependent priors, including one for the unbounded square loss.
