ECAI 2004 Conference Paper

[full paper]

A generalized quadratic loss exploiting target information for Support Vector Machines

Filippo Portera, Alessandro Sperduti

The standard SVM formulation for binary classification is based on the Hinge loss function, where errors are considered to be independent. Due to this, local information in the feature space which can be useful to improve the prediction model is disregarded. In this paper we address this problem by defining a generalized quadratic loss where the co-occurrence of errors is weighted according to a kernel similarity measure in the feature space. In particular the proposed approach weights pairs of errors according to the distribution of the related patterns in the feature space. The generalized quadratic loss includes also target information in order to penalize errors on pairs of patterns that are similar and of the same class. We show that the resulting dual problem can be expressed as a hard margin SVM in a different feature space when the co-occurrence error matrix is invertible. The existence of the inverse can be assured by generating the co-occurrence error matrix via an exponential kernel. We compare our approach against a standard SVM using the Hinge loss on some binary classification tasks. Experimental results obtained for different instances of the co-occurrence error matrix on these problems, show an improvement in the performance.

Keywords: SVM, Classification, Machine Learning, Loss function

Citation: Filippo Portera, Alessandro Sperduti: A generalized quadratic loss exploiting target information for Support Vector Machines. In R.López de Mántaras and L.Saitta (eds.): ECAI2004, Proceedings of the 16th European Conference on Artificial Intelligence, IOS Press, Amsterdam, 2004, pp.628-632.