This is an implementation of a training algorithm for a linear classifier.

The training algorithm is based on the Gauss-Seidel method for solving systems of linear equations. Some adjustments had to be made to apply the method to a linear classifier:

• w*x+b > 0 inequality for positive samples is converted to w*x+b = 1
• w*x+b < 0 inequality for negative samples is converted to w*x+b = -1
• If the A matrix has zeros on the main diagonal, some weights won't be updated for that iteration
• The training data is tiled to form problems with a square A matrix

It is possible for the problem to converge even in linearly inseparable problems, in which case we hope to minimize the empirical risk according to the hinge-loss function.

• numpy
• python 3.6

• jupyter (to try the notebook)
• matplotlib (to plot the resulting hyperplane)

• Wikipedia on Gauss-Seidel method
• Wikipedia on Perceptrons
• Wikipedia on Gauss-Seidel SOR