Mesh your Seneca.js microservices together - no more service discovery!
seneca-mesh
This plugin allows you to wire up Seneca microservices using automatic
meshing. It uses the SWIM gossip algorithm for automatic service
discovery within the microservice network.
To join the network, all a service has to do is contact one other
service already in the network. The network then shares information
about which services respond to which patterns. There is no need to
configure the location of individual services anywhere.
Many thanks to Rui Hu for the excellent
swim module that makes this
work.
If you're using this module, and need help, you can:
If you are new to Seneca in general, please take a look at
senecajs.org. We have everything from tutorials to sample apps to
help get you up and running quickly.
Seneca compatibility
Supports Seneca versions 3.x and above.
Install
To install, use npm
npm install seneca-balance-client
npm install seneca-mesh
The seneca-mesh plugin depends on the seneca-balance-client plugin.
And in your code:
require('seneca')()
.use('mesh', { ... options ... })
Using Windows? seneca-mesh uses some native modules, so make sure to
configure
msbuild.
Quick Example
Create a microservice. The service translates color names into
hex values.
// color-service.js
var Seneca = require('seneca')
Seneca()
// Uncomment to get detailed logs
// .test('print')
// provide an action for the format:hex pattern
.add('format:hex', function (msg, reply) {
// red is the only color supported!
var color = 'red' === msg.color ? '#FF0000' : '#FFFFFF'
reply({color: color})
})
// load the mesh plugin
.use('mesh', {
// this is a base node
isbase: true,
// this service will respond to the format:hex pattern
pin: 'format:hex'
})
Run the service (and leave it running) using:
Create a client for the service. This client joins the mesh network,
performs an action, and then leaves.
// color-client.js
var Seneca = require('seneca')
Seneca({log: 'test'})
// load the mesh plugin
.use('mesh')
// send a message out into the network
// the network will know where to send format:hex messages
.act({format: 'hex', color: 'red'}, function (err, out) {
// prints #FF0000
console.log(out.color)
})
Run the client in a separate terminal using:
The client finds the service using the mesh network. In this simple
case, the color-service
is configured as a base node, which means
that it listens on a pre-defined local UDP port. The client checks for
base nodes on this port.
Notice that the client did not need any infomation about the service
location.
To join a network, you do need to know where the base nodes are. Once
you've joined, you don't even need the bases anymore, as the network keeps
you informed of new services.
To find base nodes, seneca-mesh provides support for discovery via
configuration, multicast, service registries, and custom
approaches. Base nodes are not used for service discovery. They
serve only as a convenient means for new nodes to join the network.
In the above example, UDP multicast was used by default. In production
you'll need to choose a discovery mechanism suitable for your network.
The examples folder contains code for this
example, and other scenarios demonstrating more complex network
configurations:
- local-dev-mesh: local development, including a web service API.
- multicast-discovery: multicast allows base nodes to discover each other - zero configuration!
- consul-discovery: base node discovery using a service registry, when multicast is not available.
As a counterpoint to mesh-based configuration, the
local-dev example is a reminder of the
burden of traditional service location.
Development monitor
You can monitor the status of your local development network using the
monitor
option:
// monitor.js
Seneca({tag: 'rgb', log: 'silent'})
.use('mesh', {
monitor: true
})
This prints a table of known services to the terminal. Use keys
Ctrl-C
to quit, and p
to prune failed services. In the case of the
multicast-discovery example, the
monitor will output something like:
Deployment
Seneca-mesh has been tested under the following deployment configurations:
- Single development machine using localhost (loopback network interface)
- Multiple machines using VirtualBox (enable host network)
- Docker containers using host networking (--net="host")
- Docker swarm using an overlay network (not multicast not supported here by Docker)
- Amazon Web Services on multiple instances (multicast not supported by Amazon)
See the test and test/docker folders for example code.
See also the [Full system](#Full systems) examples for deployment configurations.
Multicast service discovery is the most desirable from an ease of
deployment perspective, as you don't have to do anything - base nodes
discover each other, and services discover base nodes. Unfortunately
multicast networking is often not supported by the underlying network.
As best-practice deployment model, consider running a least one base
node per machine. This provides considerable redundancy for services
joining the network.
Base discovery
Once a service has joined the SWIM network, it will find all the other
services. SWIM solves that problem for you, which is why it is so
awesome.
But you stil have to join the network initially. You can do so by
pointing a new service at any other service, and it will "just
work". However in practice it is useful to have the concept of a base
node that provides bootstrapping functionality as a its primary
purpose. The problem then reduces to finding base nodes.
Note: not all base nodes need to alive - you can provide a list of
base nodes containing nodes that are down. SWIM will continue anyway
so long as at least one node is up.
Seneca-mesh provides the following strategies:
-
defined: the base nodes are pre-defined and provided to the
service via configuration or environment variables. This is no
worse than having other kinds of well-known services in your
system, such as databases. By following a consistent approach you
can provide a list of nodes dynamically - e.g. using the AWS CLI
to list all instances in your VPC (aws ec2 describe-instances
).
-
custom: you can provide a custom function that returns a list of
bases, resolved by your own custom approach.
-
registry: load the list of bases from a key-value registry such
as Consul. This strategy leverages the
seneca-registry
set of plugins, so you can use not only consul, but also etcd,
ZooKeeper, and so on.
-
multicast: base nodes broadcast their existence via IP
multicast. New services briefly listen to the broadcast to get the
list of base nodes, and then drop out. This keeps broadcast
traffic to a minimum. Note: you need to get the broadcast address
right for your network - time to run ifconfig -a
!
-
guess: If a base node is running locally, then the service can
find it by searching at the default location: UDP 127.0.0.1:39999.
If you've specified a different IP for the service to bind to,
then that IP will also be checked. This is the usual mode for
local development.
The strategies are executed in the order listed above. By default,
seneca-mesh only moves onto the next strategy if the current one
failed to produce any bases (this is configurable).
Message flows
Each service speficies the messages patterns that it cares about using
the pin setting. As a convenience, you can use pin at the top
level of the options, however the more complete form is an array of
patterns specifications listed in the listen option.
Thus
seneca.use('mesh', {
pin: 'foo:bar'
})
is equivalent to:
seneca.use('mesh', {
listen: [
{pin: 'foo:bar'}
]
})
Each entry in the listen array specifies the listening models for a
given pattern. In particular, you can specify that the listening model:
- consume: assume the message is from a work queue; consume the message, and generate a reply. This is the default.
- observe: assume the message is published to multiple services; do not generate a reply
As an example, consider a microservice that generates HTML
content. The get:content
message expects a reply containing the HTML
content, and is intended for just one instance of the service, to
avoid redundant work. The clear:cache
message is published to all
instances of the service to indicate that underlying data for the HTML
content has changed, and the content must be regenerated for the next
get:content
message. Define the mesh patterns as follows:
seneca.use('mesh', {
listen: [
{pin: 'get:content'}, // model:consume; the default
{pin: 'clear:cache', model:'observe'}
]
})
Seneca-mesh uses the
HTTP transport by default. To
use other transports, you can add additional options to each entry of the listen
array. These options are passed to the transport system as if you have
called seneca.listen
directly:
seneca.use('redis-transport')
seneca.use('mesh', {
listen: [
{pin: 'get:content'}, // model:consume; the default
{pin: 'clear:cache', model:'observe', type:'redis'}
]
})
Message Patterns
role:mesh,get:members
You can send this message to any node, and the response will be a list
of all known patterns in the network.
Here's a useful little service that lets you submit messages to the network via a REPL:
require('seneca')({
tag: 'repl',
log: { level: 'none' }
})
.use('mesh')
.repl({
port: 10001,
alias: {
m: 'role:mesh,get:members'
}
})
And on the command line:
The alias m
can be used as a shortcut.
Options
The seneca-mesh plugin accepts the following set of options. Specify these when loading the plugin:
require('seneca')
.use('mesh', {
// options go here
})
The options are:
-
isbase: Make this node a base node. Default: false.
-
bases: An array of pre-defined base nodes. Specify strings in
the format: 'IP:PORT'. Default: [].
-
pin: the action pattern that this service will respond to. Default: null
-
listen: an array of action patterns that this service will respond to. Default: null
-
stop: base node discovery stops as soon as a discovery
strategies provides a list of suggested nodes. Default: true
-
discover: define the base node discovery options:
-
defined: use defined base nodes, specified via the bases
option.
- active: activate this discovery strategy. Default: true
-
custom: provide a function with signature function (seneca, options, bases, next)
that returns an array of base nodes. See
unit test single-custom
for
an example.
-
registry: use the role:registry
patterns to load the list of
base nodes. Set to false to disable. Default is a set of
sub-options - see code for details.
- active: activate this discovery strategy. Default: true
-
multicast: base nodes broadcast their existence via IP
multicast. New services briefly listen to the broadcast to get the
list of base nodes, and then drop out. This keeps broadcast
traffic to a minimum. Note: you need to get the broadcast address
right for your network - time to run ifconfig -a
!
-
guess: Guess the location of a base by assuming it is on the
same host. Default: true.
- active: activate this discovery strategy. Default: true
Full systems
You can review the source code of these example projects to see seneca-mesh in action:
Test
To run tests, use npm:
Contributing
The Seneca.js org encourages open and safe participation.
If you feel you can help in any way, be it with documentation, examples,
extra testing, or new features please get in touch.
License
Copyright (c) 2015-2016, Richard Rodger and other contributors.
Licensed under MIT.