ECAI 2004 Conference Paper

[full paper]

Unsupervised Instance-Based Learning Techniques of Feature Weighting do not perform so badly!

Héctor Núñez, Miquel Sànchez-Marrè

Major hypothesis that will be proved in this paper is that unsupervised learning techniques of feature weighting are not significant worse than supervised methods as is commonly supposed in the machine learning community. This paper tests the power of unsupervised feature weighting techniques for predictive tasks within several domains. The paper analyses several unsupervised and supervised feature weighting techniques, and proposes new unsupervised feature weighting techniques. Two unsupervised entropy-based weighting algorithms are proposed and tested against all other techniques. The techniques are evaluated in terms of predictive accuracy on unseen cases, measured by a ten-fold cross-validation process. Of course, the label class is not taken into account to find out the weights within the unsupervised methods, but it is for the predictive accuracy computation. The testing has been done using thirty-four data sets from the UCI Machine Learning Database Repository and other sources. Unsupervised weighting methods assign weights to attributes without any knowledge about class labels, so this task is considerably more difficult. It has commonly been assumed that unsupervised methods would have a worse performance than supervised ones, as they do not use any domain knowledge to bias the process. To really confirm or not this hypothesis, a performance analysis among several supervised and unsupervised methods have been done. Major result of the study is that unsupervised methods really are not so bad! Moreover, one of the unsupervised learning methods has shown a very promising behaviour when faced against domains with many irrelevant features, reaching similar performance as some of the best supervised methods.

Keywords: Feature Weighting, Unsupervised Instance-Based Learning, Machine Learning

Citation: Héctor Núñez, Miquel Sànchez-Marrè: Unsupervised Instance-Based Learning Techniques of Feature Weighting do not perform so badly!. In R.López de Mántaras and L.Saitta (eds.): ECAI2004, Proceedings of the 16th European Conference on Artificial Intelligence, IOS Press, Amsterdam, 2004, pp.102-106.