pegasos is a pure-python package for fitting SVM and logistic models using the Primal Estimated sub-GrAdient SOlver. This implementation is based on the google tool sofia-ml. The package has an sklearn-like interface so can easily be used with existing sklearn functionality. At each training step, the pegasos solver randomly samples a batch from the training data. The runtime of the training algorithm scales linearly with the regularization parameter lambda and the number of training steps; as such the model is well suited to large datasets. For details on the training algorithm see:

http://eprints.pascal-network.org/archive/00004062/01/ShalevSiSr07.pdf

• sparse or dense matrix support
• binary classification (multiclass via sklearn.multiclass)
• balanced class weightings via training loops
• probabilistic predictions for logistic model
• model serialisation via cPickle

See example.py for how to use the library.

There are benchmarks against sklearn's SGDClassifier in the benchmarks folder.

samples   pegasos  liblinear  libsvm
------------------------------------
10^4      4.08     0.55       10.42
10^5      4.09     17.35      2638.62
10^6      4.63     230.71     *
10^7      6.87     3318.32    *

* libsvm times are missing because the models converge sometime around the heat-death of the universe

The near-constant training time of pegasos is due to the constant number of training steps. For larger datasets the number of iterations should be increased. A grid-search on the lambda regularization parameter may also be benifical. The accuracy of the classifiers is generally ordered as libsvm > liblinear > pegasos but the differences are only 0.5-1%

Note that training time will increase by a constant amount for sparse matrices

Requirements are:

• scikit-learn >= 0.13.1
• numpy >= 1.7.1
• scipy >= 0.10.1

and nose for tests:

python setup.py nosetests

• more tests
• training batches (with online learning)