Peekaboo

Since I felt that our CUDA library, CUV, was a little lifeless without some examples on how it can be used, I started to work on a K-means implementation using it. The idea is to have something like from cuv_python import kmeans [clusters, assignements]=kmeans(data) run k-means on the GPU. There are some papers on kmeans on the GPU out there but I thought I see how far a little coding will get me. As an (maybe a little atypical) example, I choose MNIST as a dataset. My experiments were with k=10 or k=20 but this should not be a restriction on the implementation. First, I tried to do it without adding anything to CUV, just using simple matrix operations that were already there. The result is something like this: clusters=mnist[:,rand_indices] mnist_dev=cp.push(mnist) # copy('F') is necessary so we can slice later on clusters_dev=cp.push(clusters.copy("F")) norms = cp.dev_matrix_cmf(mnist_dev.w, 1) cp.reduce_to_row(norms.vec,mnist_dev,cp.reduce_functor.ADD_S...

The paper "Single Layer Networks in Unsupervised Feature Learning" by Coates, Lee, Ng, that I talked about in this post , is now available online ( pdf )! Thanks to Olivier Grisel from metasploit for pointing that out.

The paper "Single Layer Networks in Unsupervised Feature Learning" by Coates, Lee, Ng is one of the most interesting on this years NIPS in my opinion. [edit] It's now available online! ( pdf ) [/edit] It follows a very simple idea: Compare "shallow" unsupervised feature extraction on image data using classification. Datasets that are used are NORB and CIFAR 10, two of the most used datasets in the deep learning community. Filters are learned on image patches and then features are computed in a very simple pyramid over the image. These are then classified using a linear SVM. The approaches that are compared are: K-Means soft K-Means Sparse Autoencoder Sparse RBM Here soft K-Means is an ad-hoc method that the authors thought up as being a natural extension of K-Means. It is a local coding based on the k nearest neighbors of a point. A cross validation was performed to find the best patchsize, number of features and distance between sample points for f...