Bound on Target Domain Error
Tags: #machine learning #transfer learningEquation
$$\epsilon_{T}(h) \le \hat{\epsilon}_{S}(h) + \sqrt{\frac{4}{m}(d \log \frac{2em}{d} + \log \frac{4}{\delta })} + d_{\mathcal{H}}(\tilde{\mathcal{D}}_{S}, \tilde{\mathcal{D}}_{T}) + \lambda \\ \lambda = \lambda_{S} + \lambda_{T}$$Latex Code
\epsilon_{T}(h) \le \hat{\epsilon}_{S}(h) + \sqrt{\frac{4}{m}(d \log \frac{2em}{d} + \log \frac{4}{\delta })} + d_{\mathcal{H}}(\tilde{\mathcal{D}}_{S}, \tilde{\mathcal{D}}_{T}) + \lambda \\
\lambda = \lambda_{S} + \lambda_{T}
Have Fun
Let's Vote for the Most Difficult Equation!
Introduction
Equation
Latex Code
\epsilon_{T}(h) \le \hat{\epsilon}_{S}(h) + \sqrt{\frac{4}{m}(d \log \frac{2em}{d} + \log \frac{4}{\delta })} + d_{\mathcal{H}}(\tilde{\mathcal{D}}_{S}, \tilde{\mathcal{D}}_{T}) + \lambda \\
\lambda = \lambda_{S} + \lambda_{T}
Explanation
I will explain this equation in more details. Domain adaptation literatures prove that the test error on target domain \epsilon_{T}(h) is bounded by three terms: 1. the empirical estimate of training errors on the source domain \hat{\epsilon}_{S}(h); 2. the distance divergence between source domain and target domain d(Ds, Dt), 3. Fixed term of VC-Dimension(d), sample size of source domain m, e as the natural logarithm. \lambda denotes a fixed term as the sum of \lambda_{S} and \lambda_{T}, which represent the errors of models training on Ds and Dt respectively. From the above analysis, we can see that if data source Ds and Dt are similar(the divergence between source and target domain distribution Ds and Dt is small), the error on target domain will also be bounded, that's how models trained on source domain will perform better on similar distributed target domains. You can check more detailed information in this NIPS 2006 paper by Shai Ben-David for more details.
Reply