Effect of normalization on feature extraction and classification in Deep Belief Networks

Use a good set of features able to code images is of major importance for good classification scores. Most of the methods used the last few years are based on empirically determined features such that GiST, SURF or SIFT detectors. An alternative way is to try to compute an alphabet of features from which the initial set of images is statistically likely to have been generated. This recent approach based on Restricted Boltzmann Machines has been popularized by Hinton [?] who proposed an efficient algorithm for computing the underlying generative model. One of the major interest of the approach is that it is grounded on statistical theory of image reconstruction and that the RBM layers can be stacked so that the initial features can be non-linearly combined. These deep belief networks (DBNs) is reminiscent of the way the ventral pathway of primate cortex code images and scenes. The code obtained at the output of the network can be used for classification. In particular it has been used for semantic place recognition (SPR) in robotics [?], a problem in which a robot has to find its present location on the basis of the visual aspect of the scene. We have previously shown [?] that these DBNs provide state of the art results. For this purpose, the initial data were normalized as usual [] using a whitening procedure. However, when considering brain models, this whitening procedure is non-realistic since it is difficult to account for global computations in the brain. This is one of the reason why, in the present work, we replaced this global normalization procedure by a local one. Unexpectedly, this procedure gave better SPR classification results. In this paper …