mattpker / node-is-master Goto Github PK

There's a unique index on the 'startDate' column, however, nothing enforces the uniqueness. It appears that an expectation that new Date() will always produce unique result. In our case, this is certainly not the case.

Our logs are filling up with about 15 million entries which look like this:

{ [MongoError: insertDocument :: caused by :: 11000 E11000 duplicate key error index: foo-production.nodes.$startDate_1 dup key: { : new Date(1438769951417) }]

(This is across several clusters in several databases, so the indexes and dates involved vary some).

I think there may be an infinite loop bug here:

https://github.com/mattpker/node-is-master/blob/master/is-master.js#L97

There is an attempt to detect the duplicate case and try a new startDate, but it appears to me that the lines below should change the startDate and updatedDate on this.worker, not this.

It appears to be this bug is causing an infinite loop in our system, since the function recurses and tries the same wrong value over and over, logging each time....

I'm looking at a patch in our fork right now.

Hi @mattpker ,

Thought you should include somewhere on your readme, some information to explain that mongodb's document expiry only runs every minute or so. As a result, if the master dies, it can take a minute or so for a new master to be promoted. Even longer if based on the default expiry/timeout settings (which I have PR'd to improve hopefully)

I understand this is just a caveat of the approach being used - but it is really important to communicate these so people know before spending too much time investigating for themselves.

Thanks for writing this cool module 👍

As a default value, the collection name of 'nodes' may conflict with an existing collection due to generic name. Also, the name "node" immediately evokes node.js for a Node developer, which is not related to what the collection contains.

By passing an additional option to Mongoose, we can avoid having the name converted to 'is-masters':

http://stackoverflow.com/questions/22391706/is-there-a-way-to-prevent-mongodb-adding-plural-form-to-collection-names

This method seems like it would work well with Redis as well. Any thoughts on supporting that?

after about 20 minutes or so of the app running ( cluster of 4 via pm2 ), i am getting this error:

{ [MongoError: E11000 duplicate key error index: my_db.forks.$startDate_1 dup key: { : null }]
name: 'MongoError',
message: 'E11000 duplicate key error index: my_db.forks.$startDate_1 dup key: { : null }',
driver: true,
index: 0,
code: 11000,
errmsg: 'E11000 duplicate key error index: my_db.forks.$startDate_1 dup key: { : null }' }

running pm2 v0.15.10 on node v5.0.0 on a linux ec2 box

any ideas?