The paper "Single Layer Networks in Unsupervised Feature Learning" by Coates, Lee, Ng is one of the most interesting on this years NIPS in my opinion. [edit] It's now available online! ( pdf ) [/edit] It follows a very simple idea: Compare "shallow" unsupervised feature extraction on image data using classification. Datasets that are used are NORB and CIFAR 10, two of the most used datasets in the deep learning community. Filters are learned on image patches and then features are computed in a very simple pyramid over the image. These are then classified using a linear SVM. The approaches that are compared are: K-Means soft K-Means Sparse Autoencoder Sparse RBM Here soft K-Means is an ad-hoc method that the authors thought up as being a natural extension of K-Means. It is a local coding based on the k nearest neighbors of a point. A cross validation was performed to find the best patchsize, number of features and distance between sample points for f...