ECAI 2004 Conference Paper

[full paper]

Optimal Brain Surgeon Variants For Optimization

Mohammed Attik, Laurent Bougrain, Frédéric Alexandre

The determination of the optimal architecture of a multilayer perceptron (MLP) to solve a specific problem is a difficult task. Several approaches based on saliency analysis have been developed in this field. In this paper we prove that the OBS does not gives an optimal architecture then we propose to apply the OBS method several times to achieve this goal. We also present the advantage of hybrid methods for optimization. A new hybrid method is proposed which use the Flexible Optimal Brain Surgeon (F-OBS) specialized in variable selection method. A comparison of our approaches to standard techniques for architecture optimization designed for MLP is presented. Simulation results obtained on the Monk's problem illustrate the specificities of each method described in this paper.

Keywords: Neural Networks, Pruning, Weight Saliency, Optimization

Citation: Mohammed Attik, Laurent Bougrain, Frédéric Alexandre: Optimal Brain Surgeon Variants For Optimization. In R.López de Mántaras and L.Saitta (eds.): ECAI2004, Proceedings of the 16th European Conference on Artificial Intelligence, IOS Press, Amsterdam, 2004, pp.957-958.