Git Product home page Git Product logo

cadence's Introduction

Writing software in C, shell, Node.js and Go. Managing infrastructure with Kubernetes, ELK, Kafka, Consul/Vault, Terraform and Pulumi. Formerly Wink, currently available for contract.

Overview

My most popular project is Timezone. A IANA DB timzone library for Node.js or the browser that does timezone aware date math and localized time formatting.

Currently working on documentation for these featured projects. They are the most interesting projects in this account.

The projects in this account come together to create four primary projects. Memento a pure-JavaScript relational database, Compassion a Paxos-based atomic log, Packet a binary packet parser and serializer generator and Prolific a logging system for Node.js applications that won't lose messages.

As of October 2021 my primary focus is on documentation of all this work. Please be patient as I work though the README.mds.

Database

Strata gets a lot of attention because it is a pure-JavaScript b-tree for Node.js. A proper file-backed b-tree that can write to either a directory tree or a write-ahead log. The write-ahead log is itself useful and you can find it in WriteAhead.

Memento is a practical application of Strata. It is a pure-JavaScript noSQL relational database for Node.js. It is ACID. It is atomic with all or nothing transactions, isolated in that changes made during a transaction are only visible to that transaction, and durable in that there are no parietal writes and committed writes will survive a system crash. It is based a multi-version concurrency control model and I've a collection of libraries to build MVCC databases off of Strata.

IndexedDB is a pure-JavaScript implementation of IndexedDB for Node.js. It is build on top of Memento and is primarily a proof of concept and is used to leverage the Web Platform Test suite for IndexedDB to get a through testing of Memento.

Consensus

Compassion is an atomic log based on a Paxos. People are drawn to the Paxos implementation via NPM but repo it is the core Paxos algorithm without networking components. If you want to use that Paxos implementation you should instead use Compassion. Compassion provides the service discovery and network communication. It implements an async/await interface you can use to build applications. You can use Conference to implement a map/reduce, which ends up being a useful concept in consensus applications, allowing you to take actions based on whether or not all participants have been notified. Currently sketching out Addendum as a proof-of-concept for Compassion that implements the etcd v2 API in Node.js using Paxos instead of Raft.

Packet Parsing

Packet is a binary parser generator for Node.js that generates pure-JavaScript whole or incremental parsers and serializers from a syntax-bashed JavaScript definition language. These generated parsers and serializers ought to be as performant as any parser or serializer you would write by hand.

Logging

Prolific is more than yet another logging library. It is both performant due to asynchronous message processing during normal operation, and durable due to synchronous message processing upon uncaught exception. Unlike other logging libraries Prolific will not lose your parting stack trace. The final messages will be captured and redirected to the same logging stream as your runtime messages. This is kind of a big deal.

Structured Concurrency

Destructible is a structured concurrency library I created before I learned about structured concurrency. It has a different interface than the Python libraries like Trio with it's nurseries, but the underlying concepts are the same, launching multiple concurrent paths of execution and having them rendezvous in a single promise upon completion. Destructible manages multiple async/await call stacks and when an exception occurs in any one of them it will gather all exceptions from an async/await call combine them into an elaborate report of all stack traces through that single promise.

Turnstile performs parallel concurrency with an explicit work queue that can be monitored rather than the implicit work queue of spawning an arbitrary number of async function calls. Fracture is a more elaborate parallel concurrency library with the ability to enqueue work in the work queue from the work queue in a way that will avoid deadlock.

Avenue is a module that implements what are essentially Go channels, channels that can be processed synchronously or asynchronously by zero, one or more consumers (i.e. multiplexed.) The consumption decisions are deferred so that the producer does not have to concern itself with the concurrency considerations of the consumer.

cadence's People

Contributors

flatheadmill avatar jwerle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

cadence's Issues

Are named functions re-entrant.

I say yes.

list.forEach(function (file) { 
  file = path.resolve(directory, file);
  async(function () { 

    fs.stat(file, async());

  }, function (stat) { 

    if (stat.isDirectory()) { 
      tree.push(file);
      async(directory)(null);
    } else { 
      fs.readFile(file, 'utf8', async());
    } 

  }, function (body) { 

    if (/proof/.test(body)) count++;

  });
});

Add named functions to context, pass by parameter.

Instead of assigning a function to a variable in scope, the function is added by its name to the context. Added benefit of not adding the function to the global scope. Need to test this on Internet Explorer.

Return a function.

Return a function that is either run with or without a callback. Without a callback, on exception we throw. With a callback, on exception we call the callback. The callback can accept a return value.

Allow early exit.

Allow early exit by calling cadence directory with an Error as the only argument, or null as the first argument.

Catch all exception handling.

Instead of requiring an exception, we can decide that the default it to throw.

In fact, why not return a function, then if you call that function without arguments, it specifies it's own exception handler?

Gather parallel executions.

When someone creates multiple callbacks with the cadence function, gather the results into array results for the context.

Loops.

Loops parallel and serial:

cadence(function (cadence) {

  records = [];

  cadence(function () {
    new Array(length - offset).forEach(function (x, i) {
      cadence(function () {
        cursor.get(i, callback("got"));
      }, function (got) {
        records[i] = got;
      });
    });
  });

  cadence(function () { 
    var i = offset;
    cadence(new Array(10), function each (item, index) {
      cursor.get(i, callback());
    }, function (got) {
      records[i] = got;
    }, function () {
      cadence(loop);
    });
  });

}, callback);

Support splats.

Sorting out how to invoke parameters. Previously thought that anything after cadence would be variable arguments, but that is going to break Proof, because users of Proof won't know that async is special. The will list async followed by ok, equal and ok and equal are treated as parameters.

My new thought is that we stick with leading arguments always. We do not treat cadence as special. Then we create a special syntax for splats. A function parameters named $splat will mean the end of the argument list, and absorb any extra arguments. The $ indicates that it will not be stored in the cadence context. A parameter named $splat$1 means that all arguments will be absorbed by the splat except for one last one which will be assigned to the next variable.

This way, if someone say ok, equal, async in proof, ok and equal will not be assigned, because there are no values passed into the function. Calling from the outside, you might always want to lead with splat, or have it there, to protect your cadence, but then, cadence will always be cadence, so we might know that cadence means end of arguments. In any case, within the cadence, the cadence context is in the control of the cadence author, so they simply have to avoid collisions.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.