Peekaboo

Since I felt that our CUDA library, CUV, was a little lifeless without some examples on how it can be used, I started to work on a K-means implementation using it. The idea is to have something like from cuv_python import kmeans [clusters, assignements]=kmeans(data) run k-means on the GPU. There are some papers on kmeans on the GPU out there but I thought I see how far a little coding will get me. As an (maybe a little atypical) example, I choose MNIST as a dataset. My experiments were with k=10 or k=20 but this should not be a restriction on the implementation. First, I tried to do it without adding anything to CUV, just using simple matrix operations that were already there. The result is something like this: clusters=mnist[:,rand_indices] mnist_dev=cp.push(mnist) # copy('F') is necessary so we can slice later on clusters_dev=cp.push(clusters.copy("F")) norms = cp.dev_matrix_cmf(mnist_dev.w, 1) cp.reduce_to_row(norms.vec,mnist_dev,cp.reduce_functor.ADD_S...

This one is from the live of a C++ programmer. As you might remember, I am working on the CUV Library for CUDA programming in Python. The library uses quite a lot of templates and meta-programming. Since we want to have all functionality in Python, we need to instantiate all the templates. After some refactoring, I got the helpful error message _cuv_python.so: undefined symbol: _ZN3cuv6detail20apply_binary_functorINS_6vectorIfNS_16dev_memory_spaceEjEES4_NS2_IhS3_jEEhEEvRT_RKT0_RKT1_RKNS_13BinaryFunctorERKiRKT2_SL_ Ah, right. That one. Well, often it is possible to guess which function this is about. But since the instantiations are not so straight-forward, I had no idea which function that was. So I wanted to look that up somehow. I tried to google how to find the corresponding function, but without much success. Probably I was missing the right keywords. After some fiddling around, I finally got it: Load your program/library in gdb. You can get a list of all symbols in t...

As promised, my group recently published our Restricted Boltzmann Machine implementation . It is based upon the CUV Library that is being developed here. The idea is to combine the ease of programming of Python with the computing power of the GPU. We used this implementation for several papers and it grew a lot over time. Here is a list of most of the features: Restricted Boltzmann Machine Training With n-step Contrastive Divergence With persistent Contrastive Divergence Weight decay, momentum, batch-learning Binary or gaussian visible nodes Restricted Boltzmann Machine Evaluation Sampling from the model Visualizing Filters Annealed Importance Sampling for approximating the partition function Calculating the partition function exactly Visualization and saving of hidden representations Stacking RBMs to Deep Belief Networks Sampling from DBNs Deep Boltzmann Machine Training With n-step Contrastive Divergence With persistent Contrastive Divergence Deep Boltzmann Ma...

My colleague Hannes uploaded a simple Multi Layer Perceptron as a demo for our CUV library. It is written entirely in Python and classifies the MNIST dataset of handwritten digits. He also wrote a blog post explaining the design and use in detail. The code is very easy to understand and to expand. But it is very fast - of course, using CUDA - and can serve as the basis for many experiments. There are also convolution routines in the library and its even possible to extend the network to a convolutional neural network entirely in Python and on the GPU.

The CUDA library from my working group was updated again today. Features of the library include matrix and vector operations on CPU and GPU using NVidia Cuda. Much work went into a clean design and ease of use. Beside the C++ interface there are also Python wrappers for very easy coding using the GPU. We mainly use this library for neural networks and restricted Boltzmann machines but it is quite generic and everybody who uses dense or diagonal matrix operations can benefit from it. New features include convenience functions in Python and Image Pyramids on the GPU. There are also some minor fixes. If everything goes as planned there will be another mayor update quite soon.