xogroup / bunnybus Goto Github PK
View Code? Open in Web Editor NEWEnterprise bus facade for creating a data transit system.
Enterprise bus facade for creating a data transit system.
We would like to be able to control BunnyBus
like a remote for turn on and off subscriptions. There are times when it is useful to be able to shut down processing of a queue even when there are messages in the queue so that debugging can occur. On the same note, turning the process back on when necessary.
Create some documentation around examples. Should add examples into a new file named Example.md
and linked from README.md
When booting up new BunnyBus
fresh, when publish()
is called in parallel vigorously, connection handling does not mitigate parallel instantiation which is a race condition.
Expected only a single connection to start up
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
{ Error: read ECONNRESET
at exports._errnoException (util.js:1022:11)
at TCP.onread (net.js:569:26) code: 'ECONNRESET', errno: 'ECONNRESET', syscall: 'read' }
CLI Tool to help with the following
pgql 'select * from table' | bunnybus -c config.json publish --options.routeKey 'someKey'
bunnybus -c config.json subscribe -q 'queue1' --confirmation ack | tee history.log | bunnybus -c config.json publish --options.routeKey 'someKey'
bunnybus -c config.json subscribe -q 'queue1' --confirmation ack | peco > selected.log
Goal is to fork this off into a bunnybus-cli
project.
When the configuration does not contain value to instantiate a good connection, the runtime hangs.
it.only('should not freak out', (done) => {
// initialize with bad configs to trigger the catch
let bunnyBus = new BunnyBus({
server: 'fake'
});
return bunnyBus.publish({event: 'thing.event_happened', content: false})
.catch(() => {
// re-up with default configs
bunnyBus = new BunnyBus();
return bunnyBus.publish({event: 'thing.event_happened', content: true})
.then(() => {
expect(true).to.be.true();
done();
})
.catch(done);
})
.catch(done);
})
retryCount
of a message is equal maxRetryCount
, bunnybus will auto reject the message and put it into an error queue;headers.RetryCount
will be equal to maxRetryCount
https://github.com/xogroup/bunnybus/blob/master/lib/index.js#L644retryCount
will affects message processing when consume the error queue.retryCount
when reject a message;Just rephrasing from fail to set
to something along the lines of fail to override the EventEmitter
to clarify that default logging behavior will not be broken on failed contract validation.
Remove npm-shrinkwrap.json
.
Move commands from Make to npm scripts.
Any other unneeded artifacts.
Prepare for Travis CI / Circle CI build system.
I'm trying to publish a message to a RabbitMQ delayed message exchange (type 'x-delayed-message', as defined by the RabbitMQ delayed message plug-in), using BunnyBus. When I declare this exchange as 'globalExchange', I run into an error, because BunnyBus 'assumes' that this exchange is of type 'topic'. This is hardcoded in the _publish function.
It would be nice to make this 'globalExchangeType' configurable, or perhaps make the _publish function agnostic of the exchange type for existing exchanges.
I got the following error message:
Operation failed: ExchangeDeclare; 406 (PRECONDITION-FAILED) with message "PRECONDITION_FAILED - inequivalent arg 'type' for exchange 'Bot' in vhost '/': received 'topic' but current is ''x-delayed-message''"
On calling reject
, bunnybus actually calls an ack on the amqplib driver - https://github.com/xogroup/bunnybus/blob/master/lib/index.js#L654
This is bad both because it's inefficient and because it means deadlettering policies won't be correctly applied (and rabbit won't display rejects/deadletters correctly in stats).
Therefore we should instead create the error queue on first connection (like we do with regular queue) and correctly swap over to rejects.
The current connection object does not track amqp
close events. This is a hole in error handling and needs to be mitigated.
Add an option to new BunnyBus()
to allow the subscribe()
method to ignore header checks when pumping messages off of the queue.
Create a BunnyBus
identifier for messages originating from publish()
, requeue()
, send()
and reject
. This will enable the APIs for subscribe()
and get()
to be able to identify if messages should be processed as part of the BunnyBus
ecosystem. Any messages without such identifier should be automatically ack()
ed off or reject()
ed to the error queue with some sort of header value signifying the reason.
The motivation behind this is to allow for multitenancy of a single Rabbit cluster between two different driver set. In the even vhost
configuration was not designed correctly, messages may be sent and received between incompatible coupling.
In addition, this identifier should perhaps also contain a optional semver version identifier. There might be two possible take on this where we use the semver of the BunnyBus
version based off of what exist in package.json
or we also allow the end client to supply one through publish([options])
or send([options])
Test should provide proof against:
sinon
spy on bunnyBus.publish
like so: sinon.spy(bunnyBus, 'publish');
bunnyBus.publish
and chain a Promise (the default native Promise or any particular library) onto it instead of a callback.console.log(bunnyBus.publish.callCount)
after the call and see that it is 2 instead of the expected 1. If you look at the args
passed to the two spied calls, they are identical. The event is actually only published once but the spy leads you to believe that it was published twice.When you spy on bunnyBus.publish
, call bunnyBus.publish
once and return a Promise, then the spy should only see one call of bunnyBus.publish
.
When you spy on bunnyBus.publish
, call bunnyBus.publish
once and return a Promise, then the spy tracks two identical calls of bunnyBus.publish
.
The problem seems to be happening here in the code. publish
is called once with the original invocation, and then it is called again if no callback was provided. This results in the sinon
spy accurately registering two calls when the rabbit event was only published once. If you are spying on bunnyBus.publish
in your tests and expecting each call to represent an event that was published then this behavior is misleading.
Rename all private methods specified in the documentation to start with _
. So it is clear to end users the separation of public and private methods through naming conventions.
Add logging calls where apprioriate.
With the introduction of Node8 as LTS in October. It is time to shore up testing around await
/async
testing.
We want to make sure async
/await
calls have similar expected behavior as the raw Promise invoked ones.
All bunnybus class methods in the public API should support ES6 native promises when no callback is provided.
The current integration tests written with the callback contract should be copied into a second file integration-promise.js
and re-written to utilize the promise API.
Currently it seems I can't add custom options when publishing/sending a message
One of the use case I'm thinking is expiration
Is that intended or a missing feature ?
This is an issue tracker to help as a high level checklist for items needing to be accomplished.
rabbitmq
imagecallback
Promise
TODO
in code base and fix.Promise
returns object when callback
is not providedrabbit-killswitch
support
EventEmitter
interface on to BunnyBus
log.info
, log.error
, log.fatal
, log.warn
, log.debug
Right now trying to subscribe to an _error
queue with bunnybus binds new events (causing regular events to flow directly into the _error queue), and creates _error_error
queues which is undesirable behavior.
It would be nice to be able to disable this behavior with subscribe, or have the 'get' method work like subscribe.
This should also be captured in documentation.
return bunnyBus.subscribe('hapi-reviews-reminder', {
'reviews.webhook': function(message, ack, reject, requeue) {
return myOtherHandler(message, ack, reject, requeue);
}
});
In this case myOtherHandler
will lock up and not ack(), reject(), requeue() or throw any errors.
Either the docs should specify I need to rebind my scope, or bunnybus shouldn't depend on an explicit scope in message handlers.
Node 14 and 16 to be supported engines
Only Node 10 and 12 are supported and trying to install bunnybus on projects using newer versions of Node yields warnings.
I took note of issue 84 and the relevant pull request.
I'm trying to send a message directly to the queue and make it persistent. See the following code fragment:
const sendOptions = { persistent: true };
this.bunnyBus.send(message, queueName, sendOptions)
I expected the created message to be persistent.
The message was created, but not persistent according to RabbitMQ. However, the following fragment of code did work as expected:
const publishOptions = { persistent: true };
this.bunnyBus.publish(message, publishOptions)
I've also tried to use deliveryMode: 2; but unfortunately this had the same result. Both messages (the sent and published ones) targeted the same, durable queue.
Create a new method that takes in a JS Object that describes the rabbit topology and creates it.
Add documentation around error types within API.md
.
The subscribe()
interface does not properly identify the routeKey
when only the message within the payload.content
contains it.
send()
with a payload like { event : 'a', prop1 : 'value1' }
.subscribe()
with a handler listening to a
.The handler listening to a
should get the message routed to it.
BunnyBus
swallows the message because the event
value was not identified.
We want to allow for any type of A+ Promise based framework to be monkey patched into to our runtime. Most important, we want the ability to use bluebird
. We also don't want to manage the this dependency internally and allow BunnyBus to be Promise framework agnostic while leverage the default ES6 Promise as a safe default.
Rename the following header properties and fix the test to adhere as well.
createAt
-> createdAt
errorAt
-> erroredAt
Rearranging the hierarchy of the documentation for readability.
Add headers to separate BunnyBus getters/setters from methods.
The existing handlers contract consist of (message, ack, reject, requeue) => {}
. The message
is the deserialized data from payload.content
. But there maybe times when it is desirable to also ascertain the other meta information associated to the payload such has transactionId
and retryCount
.
I would like to allow for this feature when subscribe(queue, [options])
is called with an options.subscribeMeta
set to true
. When the condition is met, then the handlers
contract can change to (message, meta, ack, reject, requeue) => {}
.
meta
will be a complex object where the scope meta.headers
will contain all of the information contained within payload.properties.headers
.
Make a version of the load test for the new Promise paths.
Create equal coverage for both node callback and Promise code paths.
For the subscribe()
contract, the handlers currently contain a hash pair for routeKey
/event
assigned with a handler of signature (message, ack, reject, requeue) => {}
. The routeKey
is used to bind queue to the exchange for message routing. This routeKey
can accept wild cards of the form *
and #
. When we create a handler and assign it a full wild card or partially wild card value, the binding takes place on RabbitMQ, but BunnyBus doesn't respect it correctly.
Example,
routeKey
of value *
is used. A messages comes through and lands in the queue with a concrete routeKey
value of systemA.message-created
. This message will not be routed to the handler because the value does not match what was registered. We need to put in place a better matching system than just checking hasProperty()
.
This is the offending LOC.
Calling bunnyBus.subscriptions.clearAll();
should close all connections. Currently the only way to stop the dangling connection is to call the internal methods. This dangling connection will cause test runners to hang and other weird/undesirable behavior.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.