Git Product home page Git Product logo

luster's Introduction

Luster NPM version Build status

Dependency status Development Dependency status

Core features

  • No worker code modification is necessary.
  • Provides common solution for master process.
  • Maintains specified quantity of running workers.
  • Runs groups of workers on the different ports for 3rd party load balancing (nginx or haproxy, for example).
  • Allows configuration via JSON, JS or anything that can be required out of the box.
  • Zero downtime successive workers' restart.
  • Simple and flexible API for building extensions and development of custom master-workers solutions.

Node.js versions support

In [email protected] we dropped support for node<4. If you desperately need to make it run on older node versions, use [email protected].

Quick start

Install luster module and save it as runtime dependency:

$ npm install --save luster

Write minimal required configuration file for luster:

$ echo '{ "app" : "worker.js" }' > ./luster.conf.json

Run the cluster:

$ ./node_modules/.bin/luster

Read configuration manual to know more about luster features.

Configuration

How luster tries to resolve a path to configuration file

Following example written in plain JavaScript, not JSON, so you can name it luster.conf.js to launch luster without options, or pass the configuration file path as the first argument to the script:

$ ./node_modules/.bin/luster ./configs/my_luster_configuration.js

Internally, luster tries to call the require() in the following way:

require(path.resolve(process.cwd(), process.argv[2] || './luster.conf'));

Annotated example of configuration

module.exports = {
    // required, absolute or relative path to configuration file
    // of worker source file
    app : "./worker.js",

    // workers number
    // number of cpu threads is used by default
    workers : 4,

    // options to control workers startup and shutdown processes
    control : {
        // time to wait for 'online' event from worker
        // after spawning it (in milliseconds)
        forkTimeout : 3000,

        // time to wait for 'exit' event from worker
        // after disconnecting it (in milliseconds)
        stopTimeout : 10000,

        // if worker dies in `exitThreshold` time (in milliseconds) after start,
        // then its' `sequentialDeaths` counter will be increased
        exitThreshold : 5000,

        // max allowed value of `sequentialDeaths` counter
        // for each worker; on exceeding this limit worker will
        // be marked as `dead` and no more automatic restarts will follow.
        allowedSequentialDeaths : 10,

        // if falsy, worker is considered ready after 'online' event
        // it happens between forking worker and executing it
        // if truly, worker is considered ready
        // when you call require('luster').ready inside of it
        // notice that it's only affect startup/restart logic
        // worker will start handling requests right after you call 'listen' inside of it
        triggerReadyStateManually : false
    },

    // use "server" group if you want to use web workers
    server : {
        // initial port for the workers;
        // can be tcp port number or path to the unix socket;
        // if you use unix sockets with groups of the workers,
        // then path must contain '*' char, which will be replaced
        // with group number
        //
        // worker can get port number to listen from the environment variable
        // `port`, for example:
        // > server.listen(process.env.port)
        port : 8080,

        // number of workers' groups; each group will
        // have its own port number (port + group number)
        groups : 2
    },

    // extensions to load
    // each key in the "extensions" hash is a npm module name
    extensions : {
        // luster-log-file extension example
        "luster-log-file" : {
            stdout : "/var/log/luster/app.stdout.log",
            stderr : "/var/log/luster/app.stderr.log"
        },

        // luster-guard extension example
        "luster-guard" : {
            include: [ '**/*.js' ],
            exclude: [ '**/node_modules/**' ]
        }
    },

    // if extensions' modules can't be resolved as related to
    // luster module or worker path, then absolute path
    // to the directory, which contains extensions modules
    // must be declared here:
    extensionsPath : "/usr/local/luster-extensions",

    // max time to wait for extensions initialization
    extensionsLoadTimeout : 10000,

    // if your app or used extensions extensively use luster
    // internal events then you can tweak internal event emitters
    // listeners number limit using following option.
    // default value is `100`, option must be a number else EventEmitter
    // throws an error on configuration.
    maxEventListeners : 100
};

Extensions

Extensions development

Extensions is a simple Node.js module, which must export object with configure function, which will be called during master and worker configuration.

Synchronous extension initialization:

module.exports = {
    configure : function(config, clusterProcess) {
        // has `get` method:
        // var someProp = config.get('some.property.path', defaultValue);
        this.config = config;

        if (clusterProcess.isMaster) {
            this.initializeOnMaster(clusterProcess);
        } else {
            this.initializeOnWorker(clusterProcess);
        }
    }
}

Asynchronous extension initalization:

module.exports = {
    initializeOnMaster : function(master, done) {
        // emulate async operation
        setTimeout(function() {
            // do something
            done();
        }, 500);
    },

    initializeOnWorker : function(worker, done) {
        // emulate async operation
        setTimeout(function() {
            // do something
            done();
        }, 300);
    },

    configure : function(config, clusterProcess, done) {
        // has `get` method:
        // var someProp = config.get('some.property.path', defaultValue);
        this.config = config;

        if (clusterProcess.isMaster) {
            this.initializeOnMaster(clusterProcess, done);
        } else {
            this.initializeOnWorker(clusterProcess, done);
        }
    }
}

To enable asynchronous initalization of an extension, configure function must be declared with 3 or more arguments, where 3-rd argument is callback, which must be called by extensions when initialization has been finished. Callback accepts one optional argument: an error, if initalization failed.

Debuggability

If you are somehow lost in how master-worker interaction works, feel free to use NODE_DEBUG=luster:eex when launching your app. For example, you can check it within luster examples folder:

cd examples/custom_master_and_ipc/
NODE_DEBUG=luster:eex npm run start

You will see the sequence of events both on master and workers, along with underlying IPC messages.

luster's People

Contributors

an9eldust avatar corpix avatar golyshevd avatar h4 avatar isqua avatar mutantcornholio avatar omgtehlion avatar slnpacifist avatar wayx avatar

Watchers

 avatar  avatar

Forkers

mutantcornholio

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.