mloss.org new software

MLPACK 0.1

Thu, 20 Nov 2008 01:48:19 -0000

MLPACK is the first comprehensive scalable machine learning library. Developed by the Fundamental Algorithmic and Statistical Tools laboratory (FASTlab), MLPACK and its core functions library FASTlib are the much needed filling of an existing void. Previously, researchers had to either (a) settle for poorly-scaling collections of methods implemented for academic purposes, (b) hunt down the often difficult to find and difficult to apply yet fast code writen by algorithms' developers, or (c) reimplement solutions to their specific analysis problems from scratch. With MLPACK, we offer a fourth option, in which researchers may find all the methods they need designed favoring both speed and usability.

MLPACK currently includes a wide range of the following efficient algorithms:

$k$-nearest neighbor classifier.
FastICA.
Hidden Markov Models.
Information Maximization algorithm for ICA.
Kalman filter.
Kernel density estimation algorithm using series expansion.
Mixture of Gaussians using maximum likelihood and L2 error.
Naive Bayes classifier.
Nelder-Mead/Quasi-Newton optimizer.
Series expansion library for Gaussian kernel in $O(p^D)$ and $O(D^p)$ expansions.
Support Vector Machine classifier and regression.
Sequential Minimal Optimization algorithm for SVM.

r-cran-earth 2.1-2

Stephen Milborrow,Trevor Hastie, Rob Tibshirani — Tue, 18 Nov 2008 10:07:00 -0000

Multivariate Adaptive Regression Spline Models: Build regression models using the techniques in Friedman's papers "Fast MARS" and "Multivariate Adaptive Regression Splines". (The term "MARS" is copyrighted and thus not used in the name of the package.)

Adaptive Resonance Theory for Unsupervised Learning 1.0

Tomas Hudik, Jan Zizka — Thu, 13 Nov 2008 16:40:28 -0000

This software package includes the ART algorithms for unsupervised learning only. It is a family of four programs based on different ART algorithms (ART 1, ART 2A, ART 2A-C and ART Distance). All of them are clustering algorithms and they are command-line programs. They are able to process only numerical continuous values (with the exception of ART 1) and cannot handle missing values. ART algorithms do not need to know a number of how many clusters should be created in advance.

DeltaLDA 0.1

David Andrzejewski — Wed, 12 Nov 2008 21:53:00 -0000

This software implements the DeltaLDA model, which is a modification of the Latent Dirichlet Allocation (LDA) model. DeltaLDA can use multiple topic mixing weight priors to jointly model multiple corpora with a shared set of topics. The inference method is Collapsed Gibbs sampling. The program can also be used to do "standard" LDA as a special case, and is implemented as a Python C extension module.

MLPY Machine Learning Py 1.2.7

Tue, 11 Nov 2008 12:13:33 -0000

We introduce mlpy, a high-performance Python package for predictive modeling. It makes extensive use of NumPy to provide fast N-dimensional array manipulation and easy integration of C code. Mlpy provides high level procedures that support, with few lines of code, the design of rich Data Analysis Protocols (DAPs) for predictive classification and feature selection. Methods are available for feature weighting and ranking, data resampling, error evaluation and experiment landscaping. The package includes tools to measure stability in sets of ranked feature lists, of special interest in bioinformatics for functional genomics, for which large scale experiments with up to 10^6 classifiers have been run on Linux clusters and on the Grid.

The modular structure of mlpy allows easily adding new algorithms to each of the 7 categories in which the package is organized. They are:

Classification. For each algorithm, distinct methods are deployed for the training and the testing phases (whenever possible, real valued prediction can be obtained). The implemented algorithms are in the families of SVMs-Support Vector Machines (four kernels available), DA-Discriminant Analysis (Fisher, Penalized and Spectral Regression) and Nearest Neighbours.

Feature weighting. A total of nine methods is made available to obtain weights from models such as SVMs or DAs; classifier-independent methods for weighting features are also implemented, including I-RELIEF and Discrete Wavelet Transform.

Feature ranking. Two main schemas are used for selecting and ranking purposes, belonging either to the Recursive Feature Elimination or the Recursive Forward Selection family (for a total of six variants).

Resampling methods. The classification and feature ranking operations can be organized within a sampling procedure such as Textbook/Monte-Carlo cross validation (stratification over labels is available), leave-one-out or user-defined train/test split schema.

Metric functions. Performance assessment can be evaluated by a set of different measures, including Error, Accuracy, Matthews Correlation Coefficient, Area Under the ROC Curve. Variability can assessed by Standard Deviation or Bootstrap Confidence Intervals.

Feature list analysis. The ordered lists from the feature ranking experiments can be analyzed in terms of stability (Canberra indicator, extraction/position indicator) and an optimal list can be retrieved (Borda count).

Landscaping tools. A system of executable scripts to be used off-the-shelf to tabulate performance (e.g. Error, MCC and stability measures) on a grid of different experimental conditions by a basic DAP implementation (resampling by k-fold or Monte Carlo CV, training, feature ranking, test).

mlpy is a project developed by the MPBA research unit at FBK, the Bruno Kessler Foundation in Trento, Italy (http://mpba.fbk.eu).

dlib 17.12

Davis King — Tue, 11 Nov 2008 03:02:31 -0000

A C++ toolkit containing machine learning algorithms and tools that facilitate creating complex software in C++ to solve real world problems.

The library provides efficient implementations of the following algorithms:

support vector machines for classification
relevance vector machines for regression and classification
reduced set approximation of SV decision surfaces
online kernel RLS regression
online kernelized centroid estimation/one class classifier
kernel k-means clustering
radial basis function networks
kernelized recursive feature ranking
Bayesian network inference using junction trees or MCMC

The library also comes with extensive documentation and example programs that walk the user through the use of these machine learning techniques.

RL Glue and Codecs -- Glue 3.0 RC2 and Codecs R338

Brian Tanner, Adam White, Richard S. Sutton — Tue, 11 Nov 2008 01:20:00 -0000

RL-Glue allows agents, environments, and experiments written in Java, C/C++, Matlab, Python, and Lisp to inter operate, accelerating research by promoting software re-use in the community.

Note: This is a release candidate, not a final release. We are actively soliciting feedback from the community about any problems with the software or documentation that can be improved before a final release in Nov/Dec 2008.

Update posted: Oct 11/2008, Big change for main RL-Glue and C-Codec to use const-pointers instead of structs by value in parameters and return types. Breaks backward compatibility. See the tech manual in docs.

Update posted: Oct 8/2008, fixed memory leak in RL-Glue, fixed skeleton experiment build on Linux, updated some Cygwin compatibility stuff.

Overview

Inspired by related psychological theory, in computer science, reinforcement learning is a sub-area of machine learning concerned with how an agent ought to take actions in an environment so as to maximize some notion of long-term reward. Reinforcement learning algorithms attempt to find a policy that maps states of the world to the actions the agent ought to take in those states... -- Wikipedia Reinforcement Learning Article

RL-Glue is a set of common guidelines for the reinforcement learning community to follow to allow us to share and compare agents and environments with greater ease. The software implementation of RL-Glue is the reusable glue interface to connect the basic parts of a learning experiment.

RL-Glue supports interaction between agents, environments, and experiment programs in two different modes. In direct-compile mode, all three modules are written in C/C++ and compiled together into a single executable program.

In the more flexible socket mode: agents, environments, and experiments use inter-process communication through sockets, either locally on one computer or over the network or Internet. In socket mode, agents, environments, and experiments written in a variety of languages can interact with each other transparently. The language-specific software that allows creations from a particular language to connect to RL-Glue is called a codec. We currently have codecs for:

C/C++
Java
Python
Matlab
Lisp

Members of the reinforcement learning community are welcome to write their own language-or-project specific codecs to use with RL-Glue. The Lisp codec is an example of a user-contributed codec. There are currently codecs in development to connect projects as diverse as: a real-time strategy game, an atari emulator, and a robot to RL-Glue.

The RL-Glue software project, combined with the RL-Glue codecs are a powerful tool that allows members of the reinforcement learning community to re-use each others agents, environments, and experiment programs to help quicken the pace of research. Before RL-Glue most researches implemented their own experiment protocol, making collaboration difficult.

RL-Glue has been the base for the last few reinforcement learning competitions, and that trend will continue with the 2009 Reinforcement Learning Competition.

What's new in RL-Glue 3.0

- A new homepage: http://glue.rl-community.org/

Revamped build system (autotools) for maximum cross-platform install compatibility (Linux, Unix, MacOS, Cygwin) -Installing has never been simpler: >$ ./configure >$ make >$ sudo make install
RL-Glue now installs to /usr/local
- Headers and libs in standard search paths
  - Compiling agents/environments/experiments has never been easier: >$ gcc MyAgent.c -lrlagent -o myAgent.exe
Codecs for C/C++, Java, Python, MATLAB AND LISP <--- MATLAB AND LISP!
charArray (String) observation and action types!
Documentation
- General RL-Glue Overview
- RL-Glue Technical Manual
- A manual for each codec!

History of RL-Glue

We can trace RL-Glue back as far as 1996 through a project by Rich Sutton and Juan Carlos Santamaria called RL-Interface. Since then, the project has gone through several designs and languages. Over time the objectives of the project became more ambitious - it grew from being a convenient calling convention within a single language to a complete protocol allowing all sorts of various languages to communicate with each other.

Model Monitor 1.0

Troy Raeder, Nitesh V. Chawla — Fri, 07 Nov 2008 18:07:20 -0000

Model Monitor is a Java toolkit for the systematic evaluation of classifiers under changes in distribution. It provides methods for detecting distribution shifts in data, comparing the performance of multiple classifiers under shifts in distribution, and evaluating the robustness of individual classifiers to distribution change. As such, it allows users to determine the best model (or models) for their data under a number of potential scenarios. Additionally, Model Monitor is fully integrated with the WEKA machine learning environment, so that a variety of commodity classifiers can be used if desired.

Some of the techniques implemented in the software come from our papers:

David A. Cieslak and Nitesh V. Chawla "Detecting Fracture Points in Classifier Performance", 7th IEEE Conference on Data Mining, pp. 123-132, 2007.

David A. Cieslak and Nitesh V. Chawla "A Framework for Monitoring Classifiers' Performance: When and Why Failure Occurs?", Knowledge and Information Systems 2008.

Interested parties can find them on our website:

http://www.nd.edu/~dial

r-cran-mboost 1.0-4

Torsten Hothorn, Peter Buhlmann, Thomas Kneib, Matthias Schmid, Benjamin Hofner — Fri, 07 Nov 2008 00:00:00 -0000

Model-Based Boosting: Functional gradient descent algorithms (boosting) for optimizing general loss functions utilizing componentwise least squares, either of parametric linear form or smoothing splines, or regression trees as base learners for fitting generalized linear, additive and interaction models to potentially high-dimensional data.

WEKA 3.5.8

Waikato Machine Learning Group — Thu, 06 Nov 2008 23:01:06 -0000

The Weka workbench contains a collection of visualization tools and algorithms for data analysis and predictive modelling, together with graphical user interfaces for easy access to this functionality. The main strengths of Weka are that it is freely available under the GNU General Public License, very portable because it is fully implemented in the Java programming language and thus runs on almost any computing platform, contains a comprehensive collection of data preprocessing and modeling techniques, and is easy to use by a novice due to the graphical user interfaces it contains.

Weka supports several standard data mining tasks, more specifically, data preprocessing, clustering, classification, regression, visualization, and feature selection. All of Weka's techniques are predicated on the assumption that the data is available as a single flat file or relation, where each data point is described by a fixed number of attributes (normally, numeric or nominal attributes, but some other attribute types are also supported). Weka provides access to SQL databases using Java Database Connectivity and can process the result returned by a database query. It is not capable of multi-relational data mining, but there is separate software for converting a collection of linked database tables into a single table that is suitable for processing using Weka. Another important area that is currently not covered by the algorithms included in the Weka distribution is sequence modeling.

Weka's main user interface is the Explorer, but essentially the same functionality can be accessed through the component-based Knowledge Flow interface and from the command line. There is also the Experimenter, which allows the systematic comparison of the predictive performance of Weka's machine learning algorithms on a collection of datasets.

The Explorer interface has several panels that give access to the main components of the workbench. The Preprocess panel has facilities for importing data from a database, a CSV file, etc., and for preprocessing this data using a so-called filtering algorithm. These filters can be used to transform the data (e.g., turning numeric attributes into discrete ones) and make it possible to delete instances and attributes according to specific criteria. The Classify panel enables the user to apply classification and regression algorithms (indiscriminately called classifiers in Weka) to the resulting dataset, to estimate the accuracy of the resulting predictive model, and to visualize erroneous predictions, ROC curves, etc., or the model itself (if the model is amenable to visualization like, e.g., a decision tree). Weka contains many of the latest sophisticated methods, such as support vector machines, gaussian processes, random forests, but also classic methods like C4.5, ANNs, bagging, boosting, etc. The Associate panel provides access to association rule learners that attempt to identify all important interrelationships between attributes in the data. The Cluster panel gives access to the clustering techniques in Weka, e.g., the simple k-means algorithm. There is also an implementation of the expectation maximization algorithm for learning a mixture of normal distributions. The next panel, Select attributes provides algorithms for identifying the most predictive attributes in a dataset. The last panel, Visualize, shows a scatter plot matrix, where individual scatter plots can be selected and enlarged, and analyzed further using various selection operators.

What's new since version 3.5.6?

New classifiers: Bayesian logistic regression, discriminitive multinomial naive Bayes for text classification, functional, decision table-naive Bayes hybrid, averaged one dimensional estimators with subsumption resolution, J48 with grafting and classification via clustering meta classifier
Subset by expression, reservoir sampling, RELAGGS and random subset filters.
Latent semantic analysis attribute transformer; cost sensitive attribute selection; linear forward selection and subset size forward selection search methods.
Improved output in Logistic, NaiveBayes, EM and SimpleKMeans.
Plugin support for the KnowledgeFlow.
Ability to execute knowledge flows outside of the GUI.
Output predictions for a run of cross-validation and percentage split on the command line.
Instance weights can now be specified in a standard ARFF file.
GUI for Bayes net classifiers
Manhattan and Chebyshev distance functions

[1] Ian H. Witten; Eibe Frank (2005). Data Mining: Practical machine learning tools and techniques, 2nd Edition. Morgan Kaufmann, San Francisco.

[2] G. Holmes; A. Donkin and I.H. Witten (1994). Weka: A machine learning workbench. Proc Second Australia and New Zealand Conference on Intelligent Information Systems, Brisbane, Australia.

[3] S.R. Garner; S.J. Cunningham, G. Holmes, C.G. Nevill-Manning, and I.H. Witten (1995). Applying a machine learning workbench: Experience with agricultural databases. Proc Machine Learning in Practice Workshop, Machine Learning Conference, Tahoe City, CA, USA 14-21.

[4] P. Reutemann; B. Pfahringer and E. Frank (2004). Proper: A Toolbox for Learning from Relational Data with Propositional and Multi-Instance Learners. 17th Australian Joint Conference on Artificial Intelligence (AI2004). Springer-Verlag.

[5] Ian H. Witten; Eibe Frank, Len Trigg, Mark Hall, Geoffrey Holmes, and Sally Jo Cunningham (1999). Weka: Practical Machine Learning Tools and Techniques with Java Implementations. Proceedings of the ICONIP/ANZIIS/ANNES'99 Workshop on Emerging Knowledge Engineering and Connectionist-Based Information Systems 192-196.

[6] Gregory Piatetsky-Shapiro (2005-06-28). KDnuggets news on SIGKDD Service Award 2005.