Git Product home page Git Product logo

xgboost's Introduction

xgboost: eXtreme Gradient Boosting

An optimized general purpose gradient boosting library. The library is parallelized, and also provides an optimized distributed version. It implements machine learning algorithm under gradient boosting framework, including generalized linear model and gradient boosted regression tree.

Contributors: https://github.com/tqchen/xgboost/graphs/contributors

Turorial and Documentation: https://github.com/tqchen/xgboost/wiki

Questions and Issues: https://github.com/tqchen/xgboost/issues

Examples Code: Learning to use xgboost by examples

Distributed Version: Distributed XGBoost

Notes on the Code: Code Guide

Learning about the model: Introduction to Boosted Trees

  • This slide is made by Tianqi Chen to introduce gradient boosting in a statistical view.
  • It present boosted tree learning as formal functional space optimization of defined objective.
  • The model presented is used by xgboost for boosted trees

What's New

Features

  • Sparse feature format:
    • Sparse feature format allows easy handling of missing values, and improve computation efficiency.
  • Push the limit on single machine:
    • Efficient implementation that optimizes memory and computation.
  • Speed: XGBoost is very fast
    • IN demo/higgs/speedtest.py, kaggle higgs data it is faster(on our machine 20 times faster using 4 threads) than sklearn.ensemble.GradientBoostingClassifier
  • Layout of gradient boosting algorithm to support user defined objective
  • Distributed and portable
    • The distributed version of xgboost is highly portable and can be used in different platforms
    • It inheritates all the optimizations made in single machine mode, maximumly utilize the resources using both multi-threading and distributed computing.

Build

  • Run bash build.sh (you can also type make)
  • If you have C++11 compiler, it is recommended to type make cxx11=1
    • C++11 is not used by default
  • If your compiler does not come with OpenMP support, it will fire an warning telling you that the code will compile into single thread mode, and you will get single thread xgboost
  • You may get a error: -lgomp is not found
    • You can type make no_omp=1, this will get you single thread xgboost
    • Alternatively, you can upgrade your compiler to compile multi-thread version
  • Windows(VS 2010): see windows folder
    • In principle, you put all the cpp files in the Makefile to the project, and build
  • OS X:
    • For users who want OpenMP support using Homebrew, run brew update (ensures that you install gcc-4.9 or above) and brew install gcc. Once it is installed, edit Makefile by replacing:
    export CC  = gcc
    export CXX = g++
    
    with
    export CC  = gcc-4.9
    export CXX = g++-4.9
    
    Then run bash build.sh normally.

Version

  • This version xgboost-0.3, the code has been refactored from 0.2x to be cleaner and more flexibility
  • This version of xgboost is not compatible with 0.2x, due to huge amount of changes in code structure
    • This means the model and buffer file of previous version can not be loaded in xgboost-3.0
  • For legacy 0.2x code, refer to Here
  • Change log in CHANGES.md

XGBoost in Graphlab Create

xgboost's People

Contributors

antinucleon avatar cblsjtu avatar ericchendm avatar giuliohome avatar hetong007 avatar kalenhaha avatar mhue avatar pommedeterresautee avatar smly avatar tcfuji avatar tqchen avatar travisbrady avatar yepyao avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.