Git Product home page Git Product logo

ftrl_proximal_lr's Introduction

AsyncFTRL

A multithreaded implementation of FTRL-Proximal algorithm for Logistic Regression. This library is parallelized by an asynchronous procedure, followed the work of Downpour SGD by Jeffrey Dean etc.

Related Work:

Features

  • LibSVM file format
  • Multithreaded accelerated

Get Started

  • Single thread mode: ./ftrl_train -f input_file -m model_output [-t test_file]
  • Multithread mode: ./ftrl_train -f input_file -m model_output [-t test_file] --thread 0

Play with Async FTRL

Most of the time async ftrl works pretty well and you don't need to touch async ftrl related parameters. But if dosen't work, you may try the following:

  • sync-step: number of push/fetch steps to sync up with global model, default is 3. you may try 2/1 if default param fails.
  • warmstarting: train a single model using a small fraction of the data before async ftrl start.
    • --burn-in fraction : set fraction of data used to train a single model before async ftrl start.

ftrl_proximal_lr's People

Contributors

bingzhengw avatar bingzhengwei avatar isnowfy avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

splade poseidon1214 huangpingchun nkhuyu lizhangzhan ty01csbaidu albert1988 isnowfy zhyhooo pandasasa gongzishuye alexchao2012 duthchao scofield0li baodier cnevd gaoliqiang bayronp zhxwmessi leaveslr wangjunbo enyun alexqdh lcytzk liaoyubinai zeonyu birdgun simplejian casywang erikluo xietingcandice lukebelieves ibean fullofinspiration alvis-huang defaultrobot jizhihui gaosimin10 rivenzoo wwens zhouyonglong iwii0425 alienfeel liuhao10119 leavingseason zhanglistar wandouqiang sandy4321 tomzhang zhaoyulai colinbebrave rfgo boxianlai moxuetianya jerrycatleung sayplsay nanma zxhymjcx finesure2017 dukeyuan yueyedeai skymsg ldh127 youdaodc

ftrl_proximal_lr's Issues

Why validation-loss is -nan?

The instance is like this

-1 8111:2.9488658905029297 11507:1.0 13082:1.0 13178:1.0 16133:1.0 22000:1.0 22888:1.0 28221:1.0 36004:1.0 36542:1.0 37596:1.0 38549:1.0 48206:1.0 48786:1.0 56133:1.0 56694:1.0 58444:
1.0 59045:1.0 80219:1.0 83143:1.0 83336:1.0 85436:1.0 90750:2.9488658905029297 109126:1.0 109126:1.0 110542:1.0 111070:1.0 113837:1.0 116355:1.0 119025:1.0 124098:1.0 128976:1.0 129492:1.0 1299
54:1.0 134077:1.0 137757:1.0 138089:1.0 139260:1.0 140558:1.0 142435:1.0 144453:1.0 146199:2.9488658905029297 148378:1.0 148690:1.0 151094:1.0 162239:1.0 163439:1.0 167017:1.0 170853:1.0 171369
:1.0 171885:1.0 172801:1.0 176027:1.0 177852:2.9488658905029297 180997:1.0 181929:1.0 181946:1.0 183749:1.0 186802:1.0 190239:1.0 190705:1.0 197947:1.0 203165:1.0 206100:1.0 210524:1.0 214260:1
.0 221477:1.0 233497:1.0 234544:1.0 236809:1.0 237575:1.0 251918:1.0 262375:1.0 263600:1.0 267415:1.0 268116:1.0 275559:1.0 275898:1.0 276946:1.0 277961:1.0 281753:1.0 283133:1.0 285053:1.0 286
215:1.0 291402:1.0 522376:1.0 2209491:1.0 2340160:1.0
.

And I use this command ./ftrl_proximal_lr/ftrl_train -f mergedTraining.txt -m trainModel2 -t mergedValidation.txt, but the validatio-loss is always -nan.

Could you please figure out the mistake I made?

Also, when I add --double, segmentation fault occurred.

I find that the function GetWeight returns a -nan weight and the idx was 2340160 which exceeds the max idx (i.e. 2209010) in the trainModel2.

Many thanks.

start from error

when set parameter: start-from
then the parameters like alpha lose its function,
and was set to be -0.21

Why train-loss is inf?

The train set is like this

0 1:520418 2:2610 4:1 5:6 7:1910 8:1 9:78834 10:114126 11:0.69076284107 12:9984 13:10364 14:0.963334619838 15:298 16:114 17:0.00378009488292 18:0.00144607656595 19:850 20:850 21:0.0482352941176
0 1:519803 2:2610 4:1 5:6 7:1488 8:1 9:77867 10:112870 11:0.689882165323 12:7646 13:7898 14:0.968093188149 15:207 16:85 17:0.00265837903091 18:0.00109160491607 19:850 20:850 21:0.0482352941176
0 1:520437 2:2610 4:1 5:6 7:970 8:1 9:74048 10:107716 11:0.687437335215 12:11436 13:11702 14:0.977268842933 15:202 16:89 17:0.00272796024201 18:0.00120192307692 19:850 20:850 21:0.0482352941176
1 1:538828 2:2610 3:1 5:1 7:505 8:1 9:77028 10:112655 11:0.68375127602 12:11726 13:11964 14:0.98010698763 15:352 16:105 17:0.00456976683803 18:0.00136314067612 19:850 20:850 21:0.0482352941176
0 1:538747 2:2610 3:1 5:1 7:559 8:1 9:15905 10:22648 11:0.702269516072 12:2912 13:2952 14:0.986449864499 15:38 16:15 17:0.00238918579063 18:0.000943099654197 19:850 20:850 21:0.0482352941176
0 1:536412 2:2610 3:1 5:1 7:1216 8:1 9:72868 10:105857 11:0.688362602379 12:8242 13:8384 14:0.983062977099 15:117 16:61 17:0.00160564308064 18:0.000837130153154 19:850 20:850 21:0.0482352941176
0 1:560173 2:2610 4:1 5:4 7:17 9:4514 10:7180 11:0.628690807799 12:374 13:379 14:0.986807387863 15:69 16:11 17:0.0152857775809 18:0.0024368630926 19:1502 20:1502 21:0.00266311584554
0 1:436555 2:2610 3:1 5:1 7:2150 8:1 9:180779 10:263298 11:0.686594657005 12:1727 13:1813 14:0.952564809708 15:373 16:96 17:0.00206329274971 18:0.000531035131293 19:88 20:88 21:0.0113636363636
......

And I use this command ./ftrl_train -f ../../gen_feature/ftrl_train.txt -m model --alpha 0.01 --beta 0.1 --l1 0.1 --l2 0.2 --thread 5, but the train-loss is always inf.

Could you please figure out the mistake I made?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.