huawei-noah / streamdm-cpp Goto Github PK

stream Machine Learning in C++

Home Page: http://huawei-noah.github.io/streamDM-Cpp/

C++ 99.44% Makefile 0.56%

streamdm-cpp's Introduction

streamDM-C++: C++ Stream Data Mining

streamDM in C++ implements extremely fast streaming decision trees in C++ for big data streams. It is a project developed at Huawei Noah's Ark Lab. streamDM in C++ is licensed under Apache Software License v2.0.

The main advantages of streamDM in C++ over other C/C++ data stream libraries are the following:

Faster than VFML in C and MOA in Java.
Evaluation and learners are separated, not linked together.
It contains several methods for learning numeric attributes.
It is easy to extend and add new methods.
The adaptive decision tree is more accurate and does not need an expert user to choose optimal parameters to use.
It contains powerful ensemble methods.
It is much faster and uses less memory.

Getting Started

First download and build streamDM in C++:

git clone https://github.com/huawei-noah/streamDM-Cpp.git
cd streamDM-Cpp
make

Download a dataset:

wget "http://downloads.sourceforge.net/project/moa-datastream/Datasets/Classification/covtypeNorm.arff.zip"
unzip covtypeNorm.arff.zip

Evaluate the dataset:

./streamdm-cpp "EvaluatePrequential -l (HoeffdingTree -l NBAdaptive) -r ArffReader -ds covtypeNorm.arff -e (BasicClassificationEvaluator -f 100000)"

Methods

streamDM in C++ executes tasks. Tasks can be evaluation tasks as "EvaluatePrequential" or "EvaluateHoldOut" and the parameters needed are a learner, a stream reader, and an evaluator.

The methods currently implemented are: Naive Bayes, Logistic Regression, Perceptron, Majority Class, Hoeffding Tree, Hoeffding Adaptive Tree, and Bagging.

The stream readers currently implemented support Arff, C45, and LibSVM formats.

streamdm-cpp's People

Stargazers

Watchers

streamdm-cpp's Issues

How to run this project

Hi
I need steps to run this project
which IDE to run on it

can't build for lack of file

code/src/core/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/core/../Common.h:43:0,
from code/src/core/InstanceInformation.cpp:20:
code/src/core/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/core/../Common.h:43:0,
from code/src/core/SparseInstance.cpp:26:
code/src/core/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/utils/Log.cpp:18:0:
code/src/utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/utils/../Common.h:43:0,
from code/src/utils/CommandLineParser.cpp:20:
code/src/utils/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/utils/../Common.h:43:0,
from code/src/utils/Configurable.cpp:19:
code/src/utils/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/utils/LearnerModel.cpp:19:0:
code/src/utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/streams/../Common.h:43:0,
from code/src/streams/ArffReader.h:21,
from code/src/streams/ArffReader.cpp:18:
code/src/streams/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/streams/../Common.h:43:0,
from code/src/streams/C45Reader.h:21,
from code/src/streams/C45Reader.cpp:18:
code/src/streams/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/streams/../Common.h:43:0,
from code/src/streams/CSVReader.cpp:27:
code/src/streams/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/streams/../Common.h:43:0,
from code/src/streams/LibSVMReader.h:30,
from code/src/streams/LibSVMReader.cpp:25:
code/src/streams/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/evaluation/../Common.h:43:0,
from code/src/evaluation/BasicClassificationEvaluator.cpp:19:
code/src/evaluation/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/evaluation/../Common.h:43:0,
from code/src/evaluation/NullEvaluator.cpp:19:
code/src/evaluation/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/../Common.h:43:0,
from code/src/learners/Learner.cpp:19:
code/src/learners/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Functions/../../../Common.h:43:0,
from code/src/learners/Classifiers/Functions/Logisticregression.cpp:19:
code/src/learners/Classifiers/Functions/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Functions/../../../Common.h:43:0,
from code/src/learners/Classifiers/Functions/Majorityclass.cpp:2:
code/src/learners/Classifiers/Functions/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Functions/../../../Common.h:43:0,
from code/src/learners/Classifiers/Functions/Perceptron.cpp:19:
code/src/learners/Classifiers/Functions/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Bayes/../../../Common.h:43:0,
from code/src/learners/Classifiers/Bayes/Naivebayes.cpp:20:
code/src/learners/Classifiers/Bayes/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Bayes/observer/../../../../Common.h:43:0,
from code/src/learners/Classifiers/Bayes/observer/NumericAttributeClassObserver.h:22,
from code/src/learners/Classifiers/Bayes/SimpleNaiveBayesStatistics.cpp:19:
code/src/learners/Classifiers/Bayes/observer/../../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Meta/../../../Common.h:43:0,
from code/src/learners/Classifiers/Meta/Bagging.cpp:20:
code/src/learners/Classifiers/Meta/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Bayes/observer/../../../../Common.h:43:0,
from code/src/learners/Classifiers/Bayes/observer/NominalAttributeClassObserver.h:22,
from code/src/learners/Classifiers/Bayes/observer/NominalAttributeClassObserver.cpp:18:
code/src/learners/Classifiers/Bayes/observer/../../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Bayes/observer/../../../../Common.h:43:0,
from code/src/learners/Classifiers/Bayes/observer/NumericAttributeClassObserver.h:22,
from code/src/learners/Classifiers/Bayes/observer/NumericAttributeClassObserver.cpp:18:
code/src/learners/Classifiers/Bayes/observer/../../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Trees/../../../Common.h:43:0,
from code/src/learners/Classifiers/Trees/ADWIN.h:23,
from code/src/learners/Classifiers/Trees/ADWIN.cpp:18:
code/src/learners/Classifiers/Trees/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Trees/../../../Common.h:43:0,
from code/src/learners/Classifiers/Trees/InstanceConditionalTest.h:22,
from code/src/learners/Classifiers/Trees/AttributeSplitSuggestion.h:22,
from code/src/learners/Classifiers/Trees/AttributeSplitSuggestion.cpp:18:
code/src/learners/Classifiers/Trees/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Trees/../../../Common.h:43:0,
from code/src/learners/Classifiers/Trees/HTNode.h:24,
from code/src/learners/Classifiers/Trees/HATNode.h:25,
from code/src/learners/Classifiers/Trees/HATNode.cpp:18:
code/src/learners/Classifiers/Trees/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Trees/../../../Common.h:43:0,
from code/src/learners/Classifiers/Trees/HoeffdingTree.h:24,
from code/src/learners/Classifiers/Trees/HoeffdingAdaptiveTree.h:23,
from code/src/learners/Classifiers/Trees/HoeffdingAdaptiveTree.cpp:19:
code/src/learners/Classifiers/Trees/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Trees/../../../Common.h:43:0,
from code/src/learners/Classifiers/Trees/HoeffdingTree.h:24,
from code/src/learners/Classifiers/Trees/HoeffdingTree.cpp:18:
code/src/learners/Classifiers/Trees/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Trees/../../../Common.h:43:0,
from code/src/learners/Classifiers/Trees/InstanceConditionalTest.h:22,
from code/src/learners/Classifiers/Trees/AttributeSplitSuggestion.h:22,
from code/src/learners/Classifiers/Trees/HTAttributeClassObserver.h:23,
from code/src/learners/Classifiers/Trees/HTAttributeClassObserver.cpp:18:
code/src/learners/Classifiers/Trees/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Trees/../../../Common.h:43:0,
from code/src/learners/Classifiers/Trees/HTNode.h:24,
from code/src/learners/Classifiers/Trees/HTNode.cpp:18:
code/src/learners/Classifiers/Trees/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/learners/Classifiers/Trees/../../../Common.h:43:0,
from code/src/learners/Classifiers/Trees/InstanceConditionalTest.h:22,
from code/src/learners/Classifiers/Trees/InstanceConditionalTest.cpp:18:
code/src/learners/Classifiers/Trees/../../../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/tasks/../Common.h:43:0,
from code/src/tasks/EvaluatePrequential.h:21,
from code/src/tasks/EvaluatePrequential.cpp:18:
code/src/tasks/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/tasks/../Common.h:43:0,
from code/src/tasks/EvaluateHoldOut.h:21,
from code/src/tasks/EvaluateHoldOut.cpp:19:
code/src/tasks/../utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
In file included from code/src/Common.h:43:0,
from code/src/main.cpp:18:
code/src/utils/Log.h:26:31: fatal error: log4cpp/Category.hh: No such file or directory
#include <log4cpp/Category.hh>
^
compilation terminated.
makefile:80: recipe for target 'all' failed
make: *** [all] Error 1

excuse for interrupt you, can you give me some useful advice to build streamDM-cpp?

can not support libsvm format data train/predict

When compare the streamDM-Cpp with other online algorithm, in our internal dataset, we found this library indeed can not support libsvm format data input. With further debugging, SpareInstance Class hasn't implemented completely.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.