Inspired by co-training, many multi-view semi-supervised kernel methods are based on the following idea: find a function in each of multiple Reproducing Kernel Hilbert Spaces (RKHSs) such that (a) the chosen functions make similar predictions on unlabeled examples, and (b) the combined prediction given by the sum, or average, of the chosen functions performs well on labeled examples. In this paper, we construct a single, new RKHS with a data-dependent ``co-regularization'' norm that reduces these approaches to standard supervised learning. The reproducing kernel function of this RKHS can be explicitly derived and plugged into any kernel method, greatly extending the theoretical and algorithmic scope of co-regularization. In particular, in conjunction with well-known results, we substantially simplify, and also improve, the derivation of Rademacher complexity bounds for co-regularization given in earlier work. We propose a co-regularization based algorithmic alternative to Manifold Regularization~\cite{BelNiySin06} which leads to major empirical improvements on semi-supervised tasks.
Discussion