Peekaboo

This is my favourite part of every converence and workshop: quotes and fun facts :) There are some things researchers will never write in a paper but that they really like to tell you. Also many professors actually have a pretty good humor (or at least one that is as nerdy as mine). Please not that even though I put the quotes into quotation marks, they might not be completely accurate. Most lecturers can talk pretty fast and I am usually taking notes in the old-school paper way... Ponce: About learned dictionaries and filters: "Dictionary elements don't have semantic meaning. People like to look at them, I don't know why." About denoising using structured sparsity: "We don't know anything about image processing. The finish guys are way better. But the sparse model still works better." Lambert: About 1-vs-all training for multi class classification: "Everyone is using that. But no one knows why it works." Francis Bach (?): Abo...

There was an interesting talk by Jitendra Malik about "Rich Representations for Learning Visual Recognition" and thereafter a panel discussion with Jitendra Malik, Yann LeCun, Geoff Hinton, Tomaso Poggio, Kai Yu, Yoshua Bengio and Andrew Ng. Many "deep" topics were touched but there is one or two ideas that I found the most noteworthy. The first is the idea by Malik to do "hyper supervision". This is his idea of doing the exact opposite than weak supervision: The training examples are labeled very precisely and with lots of extra information. This makes it possible to find more interesting intermediate representations. It also gives the learning algorithm more to work on. In his introduction he said: "Learning object recognition from bounding boxes is like learning language from a list of sentences." If I understand his ideas correctly, he thinks that is its necessary to have additional clues - like 3D information, tracking and time consist...

Here are some of the quotes of this conference that were more on the lighter side and that made me at least chuckle: Josh Tenenbaum about 1970s linguists: "They didn't have any computers these days. At least none that we would now recognize as computer." Geoff Hinton on Josh Tenenbaums talk about "How to grow a mind": "You made the right generalization of Deep Learning from one example." (Hinting at Josh Tenenbaums interest in one-shot-learning) Some more of Geoff Hinton: On optimizing a non-convex likelihood function: "In good neural network fashion we add noise and momentum and hope for the best." On the same topic: "Now we allow the propabilities to add up to 4. I accidentally forgot to normalize them and they were 2. When I normalized them to one, the result got worse. So I went in the other direction." David W. Hogg in the Sam Roweis symposium: "In astronomy, we work at the photon level. We don't have big b...