Git Product home page Git Product logo

async-await-parallel's Introduction

async-await-parallel travis npm

This module is a simple utility for limiting the concurrency of awaiting async arrays in ES7. It replaces Promise.all when using async await much like async.mapLimit is commonly used in place of the analogous async.map when using callback-style async functions.

Installation

npm install async-await-parallel

This module uses async and await and therefore requires Node >= 7.

Background

Normally, when you have an array of async operations that you want to await on, you would use Promise.all.

await Promise.all([
  async () => { ... },
  async () => { ... },
  async () => { ... },
  async () => { ... },
  async () => { ... },
])

Unfortunately, there's nothing built into ES7's implementation of async await that allows you to limit the concurrency of how many async handlers are running at once.

This is problematic in many common scenarios such as performing operations on each file in a directory or downloading a batch of URLs without opening too many sockets or clogging IO bandwidth.

Usage

async-await-parallel allows you to set a maximum concurrency for an array of async results you want to await on.

const parallel = require('async-await-parallel')

await parallel([
  async () => { ... },
  async () => { ... },
  async () => { ... },
  async () => { ... },
  async () => { ... },
], 2)

In this example, a max concurrency of 2 is set, so no more than 2 of the async functions may execute concurrently. Async functions will be executed in order once previous ones resolve.

API

/**
 * Invokes an array of async functions in parallel with a limit to the maximum
 * number of concurrent invocations. Async functions are executed in-order and
 * the results are mapped to the return array.
 *
 * Acts as a replacement for `await Promise.all([ ... ])` by limiting the max
 * concurrency of the array of function invocations.
 *
 * If any single task fails (eg, returns a rejected Promise), the pool will drain
 * any remaining active tasks and reject the resulting Promsie.
 *
 * @param {Array<async Function(Void) => Any>} thunks
 * @param {Number?} concurrency Max concurrency (defaults to 5)
 *
 * @return {Promise<Array<Any>>}
 */
async function parallel (thunks, concurrency = 5)

Inspiration

  • async.mapLimit equivalent functionality for callbacks
  • co-parallel equivalent functionality for co generators
  • async-parallel this library is heavily inspired by this TypeScript library for async / await by Dave Templin (simplified, converted from TS to ES, and added tests)

License

MIT. Copyright (c) 2017 Vidy.

async-await-parallel's People

Contributors

transitive-bullshit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

async-await-parallel's Issues

thunk.call is not a function

When running the following code with node 8.2.1:
const promiseArray = awsObjects.map(async function(obj) {
if (await shouldUploadToADL(obj)) {
await downloadFileFromS3(obj);
await uploadFileToADL(obj.Key)
}
});

await parallel(promiseArray, 5);

and i'm getting the following error:
thunk.call is not a function.

Nevermind , I found the issue!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.