streamline.js is a language tool to simplify asynchronous Javascript programming.

Instead of writing hairy code like:

function archiveOrders(date, cb) {
  db.connect(function(err, conn) {
    if (err) return cb(err);
    conn.query("select * from orders where date < ?", [date], function(err, orders) {
      if (err) return cb(err);
      helper.each(orders, function(order, next) {
        conn.execute("insert into archivedOrders ...", [order.id, ...], function(err) {
          if (err) return cb(err);
          conn.execute("delete from orders where id=?", [order.id], function(err) {
            if (err) return cb(err);
            next();
          });
        });
      }, function() {
        console.log("orders have been archived");
        cb();
      });
    });
  });
}

you write:

function archiveOrders(date, _) {
  var conn = db.connect(_);
  conn.query("select * from orders where date < ?", [date], _).forEach_(_, function(_, order) {
    conn.execute("insert into archivedOrders ...", [order.id, ...], _);
    conn.execute("delete from orders where id=?", [order.id], _);
  });
  console.log("orders have been archived");
}

and streamline transforms the code and takes care of the callbacks!

No control flow APIs to learn! You just have to follow a simple rule:

Replace all callbacks by an underscore and write your code as if all functions were synchronous.

Streamline is not limited to a subset of Javascript. You can use all the features of Javascript in your asynchronous code: conditionals, loops, try/catch/finally blocks, anonymous functions, chaining, this, etc.

Streamline also provides futures, and asynchronous variants of the EcmaScript 5 array functions (forEach, map, etc.).

npm install streamline -g

http://coolwanglu.github.io/vim.js/web/vim.html (emscripten + streamline.js + @coolwanglu's magic touch).

$ cat > hello._js
console.log('hello ...');
setTimeout(_, 1000);
console.log('... world');
^D
$ _node hello

$ cat > hello._coffee
console.log 'hello ...'
setTimeout _, 1000
console.log '... world'
^D
$ _coffee hello

$ cat > hello._js
#!/usr/bin/env _node
console.log('hello ...');
setTimeout(_, 1000);
console.log('... world');
^D
$ chmod +x hello._js
$ ./hello._js

$ cat > hello._coffee
#!/usr/bin/env _coffee
console.log 'hello ...'
setTimeout _, 1000
console.log '... world'
^D
$ chmod +x hello._coffee
$ ./hello._coffee

$ _node -c .

The second one is to create your own loader with the register API. See the loader example for details.

Compiling will give you the fastest startup time because node will directly load the compiled *.js files but the register API has a cache option which comes close and the loader saves you a compilation pass.

• The first one is to compile the source with _node --standalone -c. The compiler generates vanilla Javascript code that you can load with <script> directives in an HTML page. See the eval unit test for an example.
• You can also transform the code in the browser with the transform API. All the necessary JS code is available as a single lib/transform-all.js file. See the streamlineMe example.
• A third option is to use the streamline-require infrastructure. This is a very efficient browser-side implementation of require that lets you load streamlined modules as well as vanilla Javascript modules in the browser.

Streamline gives you the choice between generating regular callback-based asynchronous code, generating code that takes advantage of the fibers library, or generating code for JavaScript generators.

The callback option produces code that does not have any special runtime dependencies.

The fibers option produces simpler code but requires that you install the fibers library (easy: npm install fibers). This option gives superior development experience: line numbers and comments are preserved in the transformed code; you can step with the debugger through asynchronous calls without having to go through complex callbacks, etc.

The fibers option can be activated by passing the --fibers option to the _node command or by setting the fibers option when registering streamline (see the streamline.register(options) function.

The generators option produces code for harmony generators. It uses the galaxy module as runtime. It requires node.js version >= 0.11.4 or an experimental browser (latest Chrome Canary). This options produces code which is similar to what you get with the fibers option, just a bit heavier because of the yield keywords.

The generators option can be activated by passing the --generators option to the _node command or by setting the ganerators option when registering streamline. If you run it with a loader you have to pass the --harmony option to node.

There are also fast variants of the fibers and generators options. See below.

function lineCount(path, _) {
  return fs.readFile(path, "utf8", _).split('\n').length;
}

lineCount("README.md", function(err, result) {
  if (err) return console.error("ERROR: " + err.message);
  console.log("README has " + result + " lines.");
});

function myStreamlineFunction(p1, p2, _) {
  var result = functionReturningAPromise(p1, p2).then(_, _);
  // do something with result
}

function callingStreamlineAsPromise(p1, p2) {
  var p = myStreamlineFunction(p1, p2);
  p.then(function(result) {
    // do something with result
  }, function(err) {
    // handle error
  });
}

function countLines(path, _) {
  return fs.readFile(path, "utf8", _).split('\n').length;
}

function compareLineCounts(path1, path2, _) {
  // parallelize the two countLines operations
  var n1 = countLines(path1, !_);
  var n2 = countLines(path2, !_);
  // get the results and diff them
  return n1(_) - n2(_);
}

See the futures wiki page for details.

The flows module contains utilities to deal with futures. For example flows.collect to wait on an array of futures and flows.funnel to limit the number of concurrent operations.

function dirLines(dir, _) {
  return fs.readdir(dir, _).map_(_, function(_, file) {
    return fs.readFile(dir + '/' + file, 'utf8', _).split('\n').length;
  });
}

function dirLines(dir, _) {
  // process 8 files in parallel
  return fs.readdir(dir, _).map_(_, 8, function(_, file) {
    return fs.readFile(dir + '/' + file, 'utf8', _).split('\n').length;
  });
}

See the documentation of the builtins module for details.

Note: you must install the companion galaxy-stack package to get async stack traces in generators mode.

Exception handling also works with futures and promises. If a future throws an exception before you try to read its result, the exception is memorized by the future and you get it at the point where your try to read the future's result. For example:

try {
  var n1 = countLines(badPath, !_);
  var n2 = countLines(goodPath, !_);
  setTimeout(_, 1000); // n1 fails, exception is memorized
  return n1(_) - n2(_); // exception is thrown by n1(_) expression.
} catch (ex) {
  console.error(ex.stack); // exception caught here
}

request(options, function(err, response, body) {
  // ...
});

var results = request(options, [_]);
// will be better with destructuring assignment.
var response = results[0];
var body = results[1];

var response = request(options, _);

You can even use language features which are not available in JavaScript. For example you can specify a default callback (see #218 for full details):

fn = (p1, p2, _ = (e) -> throw e if e) ->
  # do something

For details see the fast mode wiki page

Streamline also provides stream wrappers that simplify stream programming. These wrappers used to be included in the streamline npm package but they have now been moved to a separate ez-streams package.

You can seamlessly debug streamline code thanks to JavaScript source maps. See this video for a quick demo.

To activate this feature, pass the --source-map options to _node or _coffee, or set the sourcemap option if you register via a loader.

Streamline code can be instrumented to produce flame graphs.

Two cool things about these flame graphs:

• The pyramids correspond to stacks of asynchronous calls, not to raw V8 stacks.
• The tool generates two graphs: a pure CPU graph and a graph that combines CPU and I/O delays.

See streamline-flamegraph for details.

The tutorial shows streamline.js in action on a simple search aggregator application.

The diskUsage examples show an asynchronous directory traversal that computes disk usage.

You can see how streamline transforms the code by playing with the online demo.

Read the FAQ.

If you don't find your answer in the FAQ, post to the mailing list, or file an issue in GitHub's issue tracking.

• ez-streams: easy streams, with array-like API (filter, map, reduce, foreach, some, every), transforms (json, csv and xml streaming parsers and formatters), parallelization, buffering, etc.

The following packages contain API wrappers for streamline.js:

• streamline-fs: wrapper for node's fs module. It fixes the fs.exists call and it wraps the entire API for streamline's fast mode.
• streamline-streams: historical streaming module for streamline. It implements the low level bits of the ez-steams module (see above). If you want a rich streaming APIs you should use ez-streams instead.
• streamline-mongodb: wrappers for mongodb's native node.js driver. You only need this wrapper if you use the fast mode.

There are also some helper packages for express:

• express-streamline
• streamline-express

The following packages use streamline.js:

• streamline-require: a light and efficient require infrastructure for modules in the browser.
• streamline-pdfkit: a fork of pdfkit in which all the sync calls have been eliminated.
• streamline-zip: a fork of node-native-zip with async deflate.

The tutorial and FAQ are must-reads for starters.

The API is documented here.

For support and discussion, please join the streamline.js mailing list.

See the AUTHORS file.

Special thanks to Marcel Laverdet who contributed the fibers implementation and to Geoffry Song who contributed sourcemap support.

Streamline.js is licensed under the MIT license.