Implemented a multi-class SVM that supports multi-core parallel computing
In this project, we have developed a multi-class Support Vector Machine (SVM) capable of harnessing multicore parallel processing. Applied to the task of handwritten digit classification, this classifier not only demonstrates remarkable training and prediction velocity but also achieves superior accuracy. The multi-class SVM employs a One-vs-One (OvO) decomposition strategy, and facilitates the use of kernel tricks. In one of the assessments, with a dataset characterized by 256 features, 10 categories, 8,566 training instances, and 2,432 test instances, the classifier, when employing the Gaussian kernel, has been observed to secure an accuracy exceeding 90% on the test set. Furthermore, the adoption of parallel computing significantly elevates performance, offering a speed enhancement of over 100% compared to single-threaded computations.
Additional Information:
- Implemented an interface for selecting the type of kernel function.
- The demo uses the Iris flower dataset and can be executed directly with the command
python nlsvm_ovo_poly_kernel_multi_process.py
. zip.train
andzip.test
are the training samples and test samples of the handwritten digit dataset, respectively.- The quadratic programming solver used is
osqp
- The testing environment is
python 3.9.7
, and it requires a minimum ofpython 3.9
- The open-source license used is
MIT License
example:
if __name__ == '__main__':
from datetime import datetime
import os
from sklearn.model_selection import train_test_split
import pandas as pd
start_time = datetime.now()
# Change directory to the script file location
os.chdir(os.path.dirname(__file__))
# Load training and testing data
data_train = pd.read_csv(
"zip.train", delimiter=" ", header=None).to_numpy()
data_test = pd.read_csv("zip.test", delimiter=" ", header=None).to_numpy()
X_train = data_train[:, 1:].astype(float)
y_train = data_train[:, 0].astype(int).astype(str)
X_test = data_test[:, 1:].astype(float)
y_test = data_test[:, 0].astype(int).astype(str)
print("Training on the handwritten digits dataset...")
model = nlsvm(X_train, y_train, C=10.0, degree=6)
print("Testing the classifier...")
y_predict = predict(model, X_test)
print(f"Test set accuracy: {accuracy(y_predict, y_test) * 100:.2f}%\n")
end_time = datetime.now()
print("Total time taken: ", (end_time - start_time).seconds, "seconds\n")
Training on the handwritten digits dataset...
Categories: ['0' '1' '2' '3' '4' '5' '6' '7' '8' '9']
A total of 45 classifiers are required for this task...
Starting processes...
Classifier training completed: 3/45 ['0', '3']
Classifier training completed: 4/45 ['0', '4']
Classifier training completed: 5/45 ['0', '5']
Classifier training completed: 2/45 ['0', '2']
Classifier training completed: 8/45 ['0', '8']
......
Classifier training completed: 42/45 ['6', '9']
Classifier training completed: 43/45 ['7', '8']
Classifier training completed: 44/45 ['7', '9']
Classifier training completed: 45/45 ['8', '9']
Training complete!
Number of training samples: 7291; Dimensions: 256; Number of classes: 10
Testing the classifier...
Number of test samples: 2432
Test set accuracy: 94.82%
Total time taken: 352 seconds