Machine Learning Toolkits

Wow. So much to read today.
While following link upon link, I found so many great toolkits that I think it is worth listing them here.
One of the greatest sources was the GNU/Linux AI & Alife HOWTO.

[edit] It's been a while since I wrote this blog post but still many people seem to find it, so here a quick update. After looking into many libraries, I started using scikit-learn and then using it exclusively. Now I am a regular contributor. It is a fast growing project with great documentation resources, many algorithms and it is just so easy to use. Also, working with Python and the Python crowd is fun. I heartly recommend it. [/edit]

Here goes:
  • Vowpal Wabbit - project on very fast online gradient descent by Yahoo research (C++)
  • VFML (Very Fast Machine Learning) - library for very fast decision trees and Bayes networks (C++)
  • Stochastic Gradient Descent - library for SVMs with stochastic gradient descent (C++)
  • Maximum Entropy Modeling Toolkit for Python and C++ - the name says it all
  • Elefant - toolkit that includes kernel methods, optimization strategies and belief propagation. It has a gui
  • Milk - toolkit for python that includes SVMs, decision trees, kNN, PCA, Kmeans, NMF and feature selection
  • Peach - pure Python library that includes neural networks, fuzzy logic, genetic algorithms and swarm intelligence
  • Pebl - python library and command line application for learning the structure of a Bayesian network
  • Machine Learning: An Algorithmic Perspective - Actually a book. But with MANY MANY MANY examples online. All in Python. MOST AWESOME! - I just ordered the book
  • dbacl - a digramic Bayesian classifier - a collection of command line tools for Bayesian classification particularly for spam filtering
  • Shark - Modular library including neural networks, kernel methods, discrete and continuous optimization, fuzzy logic and control and mixtures density models (C++)
  • PyMVPA - python module including more classifiers, regression and feature selection methods than can be listed here. Do a cross-validated classifier sweep and parameter search in < 10 lines of python.
  • Monte - gradient based learning in Python - Python module that contains neural networks, Kmeans, logistic regression with a focus on parametric models
  • scikit-learn - python module with good API. Includes SVMs, generalized linear models, gaussian mixture models, mean-shift, feature selection and ranking and data management and many more.
  • mlpy - Python module that includes Wavelet transforms, Kernel methods, FDA, PDA, LASSO, LARS, feature selection and ranking and data management. Very clean interface.
  • Modular toolkit for Data Processing - Python toolkit for data processing. In my opinion the API needs a little getting used to. Includes PCA, Kmeans, RMBs, FastICA, Neural Gas, SVms, Perceptrons and many more.
  • Orange - Data mining through visual programming or Python. Large toolbox that includes great visualization features, classifiers, data management, regression and clustering. Definitely worth trying.
  • Weka - A classic tool for all data mining. Contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. Can be used via interface, scripting or java.

Comments

  1. Yaroslav BulatovNovember 8, 2010 at 8:39 PM

    There's also RapidMiner, PyBrain, Apache Mahout, LibLinear, and that's just from the first couple of pages of http://www.delicious.com/tag/machinelearning

    ReplyDelete
  2. Leo BakkerJanuary 26, 2014 at 3:38 PM

    Thanks for this overview

    ReplyDelete
  3. AnonymousMay 14, 2014 at 3:08 PM

    Thanks for your note about scikit-learn.
    I'd like to test this library for a bayesian network work but I have some difficulties to create a simple bayesian network. Do you know where I could find an example of a simple implementation using scikit-learn?

    ReplyDelete
  4. Andreas MuellerMay 14, 2014 at 7:13 PM

    This comment has been removed by the author.

    ReplyDelete
  5. UnknownApril 4, 2015 at 12:07 PM

    http://scikit-learn.org/stable/modules/naive_bayes.html
    @Stig

    ReplyDelete

Post a Comment

Popular posts from this blog

Machine Learning Cheat Sheet (for scikit-learn)

Image
As you hopefully have heard, we at scikit-learn are doing a user survey (which is still open by the way). One of the requests there was to provide some sort of flow chart on how to do machine learning. As this is clearly impossible, I went to work straight away. This is the result: [edit2] clarification: With ensemble classifiers and ensemble regressors I mean random forests , extremely randomized trees, gradient boosted trees , and the soon-to-be-come weight boosted trees (adaboost). [/edit2] Needless to say, this sheet is completely authoritative.
Read more

A Wordcloud in Python

Image
Last week I was at Pycon DE , the German Python conference. After hacking on scikit-learn a lot last week, I decided to to something different on my way back, that I had planned for quite a while: doing a wordl -like word cloud . I know, word clouds are a bit out of style but I kind of like them any way. My motivation to think about word clouds was that I thought these could be combined with topic-models to give somewhat more interesting visualizations. So I looked around to find a nice open-source implementation of word-clouds ... only to find none. (This has been a while, maybe it has changed since). While I was bored in the train last week, I came up with this code . A little today-themed taste:
Read more

MNIST for ever....

Image
[update] This post is a bit old, but many people still seem interested. So just a short update: Nowadays I would use Python and scikit-learn to do this. Here is an example of how to do cross-validation for SVMs in scikit-learn. Scikit-learn even downloads MNIST for you. [/update] MNIST is, for better or worse, one of the standard benchmarks for machine learning and is also widely used in then neural networks community as a toy vision problem. Just for the unlikely case that anyone is not familiar with it: It is a dataset of handwritten digits, 0-9, in black on white background. It looks something like this: There are 60000 training and 10000 test images, each 28x28 gray scale. There are roughly the same number of examples of each category in the test and training datasets. I used it in some papers myself even though there are some reasons why it is a little weird. Some not-so-obvious (or maybe they are) facts are: - The images actually contain a 20x20 patch of digi
Read more

Python things you never need: Empty lambda functions

Ever wanted to create an empty Python function in-line? No? Me neither. But my coworker does... We found the answer: <pre> x = lambda : None </pre> If you ever needed this, please tell me ;) [Edit] Apparently some people do need it :) Btw, most people seem to try Most people try lambda: pass first. But you have to remember, the thing at the right side of the colon is the return value of the function, not the body! So an expression is needed, not a statement. [\Edit]
Read more