Git Product home page Git Product logo

fastparallel's Introduction

fastparallel ci

Zero-overhead parallel function call for node.js. Also supports each and map!

Benchmark for doing 3 calls setImmediate 1 million times:

benchSetImmediate*1000000: 1378.514ms
benchAsyncParallel*1000000: 1740.304ms
benchAsyncEach*1000000: 1566.517ms
benchAsyncMap*1000000: 1687.518ms
benchNeoParallel*1000000: 1388.223ms
benchNeoEach*1000000: 1473.006ms
benchNeoMap*1000000: 1402.986ms
benchInsyncParallel*1000000: 1957.863ms
benchInsyncEach*1000000: 1383.822ms
benchInsyncMap*1000000: 1822.954ms
benchItemsParallel*1000000: 1690.118ms
benchParallelize*1000000: 1570.064ms
benchFastParallel*1000000: 1536.692ms
benchFastParallelNoResults*1000000: 1363.145ms
benchFastParallelEachResults*1000000: 1508.134ms
benchFastParallelEach*1000000: 1325.314ms

Obtained on node 12.18.2, on a dedicated server.

If you need zero-overhead series function call, check out fastseries. If you need a fast work queue check out fastq. If you need to run fast waterfall calls, use fastfall.

js-standard-style

The major difference between version 1.x.x and 2.x.x is the order of results, this is now ready to replace async in every case.

Example for parallel call

var parallel = require('fastparallel')({
  // this is a function that will be called
  // when a parallel completes
  released: completed,

  // if you want the results, then here you are
  results: true
})

parallel(
  {}, // what will be this in the functions
  [something, something, something], // functions to call
  42, // the first argument of the functions
  done // the function to be called when the parallel ends
)

function something (arg, cb) {
  setImmediate(cb, null, 'myresult')
}

function done (err, results) {
  console.log('parallel completed, results:', results)
}

function completed () {
  console.log('parallel completed!')
}

Example for each and map calls

var parallel = require('fastparallel')({
  // this is a function that will be called
  // when a parallel completes
  released: completed,

  // if you want the results, then here you are
  // passing false disables map
  results: true
})

parallel(
  {}, // what will be this in the functions
  something, // functions to call
  [1, 2, 3], // the first argument of the functions
  done // the function to be called when the parallel ends
)

function something (arg, cb) {
  setImmediate(cb, null, 'myresult')
}

function done (err, results) {
  console.log('parallel completed, results:', results)
}

function completed () {
  console.log('parallel completed!')
}

Caveats

The done function will be called only once, even if more than one error happen.

This library works by caching the latest used function, so that running a new parallel does not cause any memory allocations.

Why it is so fast?

  1. This library is caching functions a lot.

  2. V8 optimizations: thanks to caching, the functions can be optimized by V8 (if they are optimizable, and I took great care of making them so).

  3. Don't use arrays if you just need a queue. A linked list implemented via processes is much faster if you don't need to access elements in between.

  4. Accept passing a this for the functions. Thanks to this hack, you can extract your functions, and place them in a outer level where they are not created at every execution.

License

ISC

fastparallel's People

Contributors

mcollina avatar gnought avatar geek avatar dependabot[bot] avatar alessioalex avatar robertslando avatar mcdonnelldean avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.