In this paper we propose to boost products of simple base learners. Similarly to trees, we call the base learner as a subroutine but in an iterative rather than recursive fashion. The main advantage of the proposed method is its simplicity and computational efficiency. Experimental results on benchmark datasets indicate that boosting products of decision stumps is one of the best generic multi-class classification algorithms: it clearly outperforms boosted trees, and on the MNIST dataset it achieves the second best result among no-domain-knowledge algorithms after deep belief nets. As a second contribution, we propose an improved base learner for nominal features and show that boosting the product of two of these new subset indicator base learners solves the maximum margin matrix factorization problem, used to formalize the collaborative filtering task. On a small benchmark dataset, we obtain experimental results comparable to the semi-definite-programming-based solution at a much lower computational cost.
Discussion