oortcloud / node-ddp-client Goto Github PK

So with the switch from ws to faye-websockets the use_ssl_strict option no longer works. Lack of this is also probably the underlying cause for #20.

faye-websockets uses the underlying node TLS module for SSL but doesn't pass along the rejectUnauthorized property. See https://github.com/faye/faye-websocket-node/blob/master/lib/faye/websocket/client.js#L22-L34.

The problem with the ws module is it requires compiling native C++ code. Doing so is a bit problematic on Windows.

There's a few different potential ways of fixing this:

• Remove use_ssl_strict and document it as such. Add a workaround note, which is pass a in a trusted certificate in ca property.
• Issue/Pull request with faye-websockets to pass along the rejectUnauthorized property.
• Issue/Pull request with ws to include precompiled binaries for the native extension, and switch back. This would be the most work but highest performance.

When a sockjs request fails, the clients tries to recover.
When a websocket request fails while connecting, the client does not reconnect.

For example, if the hostname cannot be resolved the sockjs connection will try to reconnect while the websocket connection won't.

Hi, I'm trying to use DDP to communicate with Meteor using method calls. One of my methods returns a Meteor ObjectID (EJSON type oid). Everything works okay if I turn EJSON support off, except that dates and oids are not sent/recieved in the correct formats. Turning EJSON support on fixes the date type, but introduces a crash:

/Users/vsi/jobWorker/node_modules/ddp/node_modules/meteor-ejson/ejson.js:215
      return converter(obj.$value);
             ^
TypeError: undefined is not a function
  at Object.builtinConverters.fromJSONValue (/Users/vsi/jobWorker/node_modules/ddp/node_modules/meteor-ejson/ejson.js:215:14)
  at fromJSONValueHelper (/Users/vsi/jobWorker/node_modules/ddp/node_modules/meteor-ejson/ejson.js:313:28)
  at /Users/vsi/jobWorker/node_modules/ddp/node_modules/meteor-ejson/ejson.js:286:21
  at Function._.each._.forEach (/Users/vsi/jobWorker/node_modules/ddp/node_modules/meteor-ejson/node_modules/underscore/underscore.js:86:24)
  at EJSON._adjustTypesFromJSONValue (/Users/vsi/jobWorker/node_modules/ddp/node_modules/meteor-ejson/ejson.js:284:5)
  at Object.EJSON.fromJSONValue (/Users/vsi/jobWorker/node_modules/ddp/node_modules/meteor-ejson/ejson.js:325:5)
  at Object.EJSON.parse (/Users/vsi/jobWorker/node_modules/ddp/node_modules/meteor-ejson/ejson.js:337:16)
  at [object Object].DDPClient._message (/Users/vsi/jobWorker/node_modules/ddp/lib/ddp-client.js:117:18)
  at [object Object].<anonymous> (/Users/vsi/jobWorker/node_modules/ddp/lib

I've figured out that this is because the EJSON package doesn't have a built-in custom type handler for oid. I have found the Meteor code implementing this (https://github.com/meteor/meteor/blob/devel/packages/minimongo/objectid.js) but don't see any way to get ahold of the EJSON object used by the DDP package to register this code using EJSON.addType() Is there some way to do this that I am missing, or do I need to change the DDP package to accomplish this?

I have a Raspberry Pi running a Node application. I am sharing internet with the Pi through a connected ethernet cord. The node application is to connect with a Meteor application running on my laptop. I have the DDP connection working from the Raspberry Pi to my meteor application hosted on a server, but I can't connect to my local version.

I've tried my computer's IP address and 'localhost', but both result in an error. The same error occurs when no Meteor app is running. Here is a sample error when trying localhost:

Network error: ws://localhost:3000/websocket: connect ECONNREFUSED

Does anyone have any ideas of what I could try? Developing with a deployed application is obviously less than optimal and I think I was able to get this working in September just by specifying localhost?

Ideally, the node-ddp-client would be as simple as possible, and not try to keep track of the ddp collections. Keeping track of collections could be another library that encapsulates the raw ddp client. In particular, for our use case we need to keep track of some collections in a more structured way, and some subscriptions don't correspond to a collection at all. Thus the collections in ddp-client are just wasted memory for me.

What do you guys think of a PR that gives an option to disable the collections in ddp-client?

Future versions of DDP will be able to tell a client using this version that it is out of date. How does this happen? Via an error??

We should give the developer some way to know that this has happened so they can alert the user that they need to upgrade or some such (Personally I am thinking of meteorite + atmosphere DDP version transitions).

Hi,

I'd be interesting to is it possible to support sock transport rather than directly using websockets.
In some of our apps, we don't use websockets and it's hard to connect to it. We are using CloudFlare and no WS transport is allowed.

If I send a PR with proper SockJS support, any change it will get accepted?

Shouldn't it modify collections before notifying listeners that changes have occurred?

Hey,
if the connection fails while calling a method, the method will never be resolved even if I've enabled autoReconnect. For example:

async function test() {

    const a = new DDPClient();
    await a.connect();

   // We shut down the DDP server to simulate a failing connection
    setTimeout(async () => {
        const testy = await a.call("categories.get");
        console.log(testy);
    },5000)
}

The promise/callback will never be resolved when we've enabled useSockJs:true.

I'm getting an error with express:

http.js:673
    throw new Error('Can\'t set headers after they are sent.');
          ^
Error: Can't set headers after they are sent.
    at ServerResponse.OutgoingMessage.setHeader (http.js:673:11)
    at ServerResponse.res.setHeader (/root/test_client/node_modules/express/node_modules/connect/lib/patch.js:59:22)
    at ServerResponse.res.set.res.header (/root/test_client/node_modules/express/lib/response.js:518:10)
    at ServerResponse.res.send (/root/test_client/node_modules/express/lib/response.js:124:10)
    at exports.ddptest (/root/test_client/routes/test.js:1059:25)
    at DDPClient._message (/root/test_client/routes/test.js:116:7)
    at WebSocket.DDPClient._prepareHandlers (/root/test_client/routes/test.js:78:10)
    at WebSocket.EventEmitter.emit (events.js:99:17)
    at Receiver.self._receiver.ontext (/root/test_client/node_modules/ws/lib/WebSocket.js:544:10)
    at Receiver.opcodes.1.finish (/root/test_client/node_modules/ws/lib/Receiver.js:397:14)

Previously I used the ddp client but had to use a custom iteration when pre1 was released. The error is usually caused by some kind of inifinite loop

Using a custom express route:

var client = new DDPClient({
  host: "localhost",
  port: 3000
});

client.connect(function () {
  client.call("ddp_test", ["value"], function (err, result) {
    console.log(result);
    client.close();
  })
});

I'm trying to run a headless Meteor process, so without the web interface. To this end I removed the meteor-platform package and manually added the underscore, random, ejson, check, mongo, ddp and meteor packages. However, this doesn't work with node-ddp-client as the connected callback is never called.

Any idea why the ddp package by itself doesn't suffice for node-ddp-client to connect?

I have a feeling this might be a stupid question, but I figure it's better to ask than make a mess of things!

Because I have a DDP stream running from my Node app to a Meteor server somewhere in the cloud, is it possible to then send command back over that same link? I'm trying to achieve the ability to send configuration, etc to the Node app without the user needing to punch a hole in their firewall, etc. etc.

Alternatively I was looking into STUN/TURN/ICE but means I need to do a whole bunch of research into how to deploy such things.

Hello.
I found an interesting problem. It only happened when I debugged my code.
If for some reason I lose the connection of DDP(not closed or error) if I call .subscribe or .call the callback is never called.
Debugging I lost the connection because the socket can't resolve ping-pong process and it set its readyState to 2
I debugged really deep and I found this code

// .../node_modules/ddp/lib/ddp-client.js
DDPClient.prototype.subscribe = function(name, params, callback) {
  ...
  self._callbacks[id] = callback;
  ...
  self._send({
    msg    : "sub",
    ...
  });
};
DDPClient.prototype._send = function(data) {
  this.socket.send(
    EJSON.stringify(data)
  );
};

// .../node_modules/ddp/node_modules/faye-websocket/lib/faye/websocket/api.js
  send: function(data) {
    if (this.readyState > API.OPEN) return false;
    if (!(data instanceof Buffer)) data = String(data);
    return this._driver.messages.write(data);
  },

This condition if (this.readyState > API.OPEN) is true and the socket never sends the message but the DDP added the callback but it never has an answer message for this callback and it will been waiting forever for this callback.

Without debugging this error never happens, but I think is really dangerous that the DDP don't manage this problem, it can happen and the DDP will wait forever.

Anybody had this problem?

Can you please add a license, so that we know what the terms of distribution are?

I'm opening this here as I'm about to switch Meteorite / Atmosphere to port 80 / no SSL.

Probably not really ideal but there are too many people with problems right now.

It'd be good figure out what the true issue is. I guess it's a problem with either ws or node itself.

When I have a free moment I'll come back and make a reproducible test case.

Hi,
This is a great package, and I have used it effectively with react native and iOS. However, now that react-native has support for the Websocket module, can this work for Android as well? And if so, it would be great to see some examples of it. With the following code, I get errors: `let ddpclient = new DDPClient({
url: 'ws://localhost:3000/websocket'
});

console.log('DDP', ddpclient);
ddpclient.connect((error, wasReconnect) => {
  if (error) {
    console.log('DDP connection error!');
    return;
  }
  if (wasReconnect) {
    console.log('Reestablishment of a connection');
  }
  console.log('Connected!');
});
ddpclient.subscribe('messages', [], ()=> {
  console.log('messages complete:');
  console.log(ddpclient.collections.messages);
});

ddpclient.on('message', (msg) => {
  console.log('ddp message: ' + msg);
})`

I am getting Error: process.nextTick is not a function
when I use observer, please help me

In JS function.call expects parameters to be passed in directly instead of an array of parameters...

Also, Meteor.call accepts direct parameters while Meteor.apply accepts an array

The proposal:

Change from:
DDPClient.prototype.call and DDPClient.prototype.callWithRandomSeed
to:
DDPClient.prototype.send and DDPClient.prototype.sendWithRandomSeed

I have the following code on a vanilla node client:

var DDPClient = require("ddp");

var ddpclient = new DDPClient({
  // All properties optional, defaults shown 
  host: "localhost",
  port: 3000,
  ssl: false,
  autoReconnect: true,
  autoReconnectTimer: 500,
  maintainCollections: true,
  ddpVersion: '1',
  useSockJs: true
});

ddpclient.connect(function(error, wasReconnect) {
  if (error) {
    console.log('DDP connection error!');
    return;
  }

  if (wasReconnect) {
    console.log('Reestablishment of a connection.');
  }

  console.log('Connected!');

  ddpclient.subscribe(
    'test', [],
    function() {
      console.log('Subscribed to `test`');

      var observer = ddpclient.observe("test");
      observer.added = function(id, doc) {
        console.log("[ADDED] to", observer.name, ":", id, 'doc:', doc);
      };
      observer.changed = function(id, oldFields, clearedFields, newFields) {
        console.log("[CHANGED] in", observer.name,":", id);
        console.log("[CHANGED] old field values:", oldFields);
        console.log("[CHANGED] cleared fields:", clearedFields);
        console.log("[CHANGED] new fields:", newFields);
      };
    }
  );
});

And I have a meteor server with this publication:

Meteor.publish('test', function() {
  this.added('test', 'testId', { num: Math.random() });
  this.ready();
});

When I start up the node client, this is my console output:

Connected!
Subscribed to `test`

However, once I start triggering a bunch of changed items, they do start coming in:

Meteor.publish('test', function() {
  const id = 'test';

  this.added('test', id, { num: Math.random() });

  let handle = Meteor.setInterval(() => {
    this.changed('test', id, { num: Math.random() });
  }, 1000);

  this.ready();

  this.onStop(() => {
    Meteor.clearInterval(handle);
  });
});

Console output:

Connected!
Subscribed to `test`
[CHANGED] in test : test
[CHANGED] old field values: { num: 0.945352595532313 }
[CHANGED] cleared fields: []
[CHANGED] new fields: { num: 0.13286670367233455 }
[CHANGED] in test : test
[CHANGED] old field values: { num: 0.13286670367233455 }
[CHANGED] cleared fields: []
[CHANGED] new fields: { num: 0.5646781949326396 }
...
...

Currently there is a function or public field to know if the connection is connected/closed.

It is possible add one

Hi,
I am trying to connect to my meteor application hosted on galaxy with a node server that is hosted on digital ocean Unbuntu 14.04. When I run my node server locally, it connects to the meteor server fine and runs the job. However, when i deploy the node server to DO and run it, standalone using node or with pm2, the connection times out. Is there any reason it would work fine on my local machine but not once it is deployed to DO?
Thanks!

meteor/meteor#4191 suggests that the easiest way to create a DDP client (collections and all) is to create a meteor app with the standard packages stripped out and just adding ddp-client, writing whatever the client is doing in the server/ code. Then, bundling this would create a DDP client application that could be run independently with Node.

Is this an accurate description, and if so, does that make this package obsolete?

Given a collection, suppose that there exist some finite set of subscriptions with different parameters over that collection.

Now, if I want to remove a particular subscription, how can that be accomplished using this package's syntax?

Is there any kind of stop/remove subscription method?

This is more a point for discussion than a bug. Currently, node-ddp-client uses the ws library (used by socket.io). I'd like to switch that out for faye-websocket, which is what sockJs uses (which of course is what meteor uses). My primary reason for that is that ws has C code and requires a gyp-build. Gyp builds add a form of platform dependency, and don't always go smoothy in npm install, especially on windows. I hear that the 'cool kids' also like faye more, but I haven't listened enough to those arguments to have a real opinion.

Is this something you guys would be interested in? I'm happy to make the PR. It's important to us because our madeye client-side node app needs to install as smoothly and as cross-platform-ly as possible, and each gyp-build is an obstacle to that.

I'm using this DDP client to run Cucumber tests. In between my tests I close the connection and reopen it, in between which ddp.collections is not cleaned. This results in the same documents getting pushed into it multiple times.

I currently solve this by doing:

ddp.close();
ddp.collections = {};

The DDP client depends on https://github.com/sixolet/ejson and https://github.com/phillyqueso/node-srp, but neither of them are currently being maintained. Both the Meteor EJSON and SRP packages have been updated since the npm packages were published.

I talked to Slava on IRC and it looks like they might maintain these, but not before a new hire comes on board.

@tmeasday could you set up ddp-ejson and ddp-srp repositories in oortcloud, and I'll maintain them in the meantime?

Hello everyone, a question.
What are the reason for having a reconnected in DDP client?
Currently my app after open and close many parallel DDP client to the same DDP server it reconnects, why is it loosing the connection?
My DDP server doesn't crashes or show any error.
I am thinking about the overload with one DDP server, because each DDP client is subscribing and calling a method (depends the data).

There are some limit about having opened DDP client

I want to switch EJSON support to on by default in the 0.4.0 release. @jagill has a few PRs awaiting merge, one of which is fairly extensive.

For some uses of the DDP client this will be a breaking change, however, my reasoning is the current default settings parse documents in collections differently than Meteor does, and the DDP client ought to be brought in line on this.

Does anyone have strong feelings on this change?

I pushed a new branch that adds EJSON support. I included the EJSON smart package files in the ddp client as a temporary measure, until there's an EJSON npm package. I spoke with sixolet on IRC and it's on her list of things to do.

Feel free to check it out. It could use a bit more testing, but generally speaking it works.

Everything connects fine after viewing #79 until what seems like a Connection Idle Disconnect.

Currently using it in a node project hosted on DO in connection with a meteor app hosted on Galaxy but it seems like if the connection is idle and there are no processes happening, it will disconnect.

We have reconnect measures in place but it continues to disconnect anyway. Are there more ways to keep a sustained connection?

I thought I could use this in a browser environment, but Webpack complains that it can't find the net and tls modules, which seems to mean this package only runs on the server-side where those Node.js can be imported.

Is there a way to use in the browser?

I tried "XXX.meteor.com" with port 80. It doesn't work.
Any solution to this?

I'm connecting a node app to a meteor app that uses email/password authentication.

The ddp call to login works and a token is returned. Calls to authenticated methods are failing and I think its due my code not using the token correctly. Does the token need to be passed to .call() when making an authenticated call, if so how. If not, does the ddp lib manage the token itself (saving it on a per-connection basis and sending it along with every .call() )

The answer to this would make a good addition to the README. Thanks

I'm trying to understand why the method added removed and changed never trigger on my system. I create a little server in meteor that publish a collection like this

Meteor.publish('coll',function(){
  return Coll.find();
});

and in my nodejs software I write this code:

  var ddp = new DDPClient({
    host: this.config.server.host,
    port: this.config.server.port,
    autoReconnect: true,
    autoReconnectTimer: 500,
    maintainCollections: true,
    useSockJs: true
  });

ddp.connect(function(err, wrec) {
  DDPLogin.loginWithEmail(ddp, config.server.user, config.server.passwd, function (err, userInfo) {
        ...
       ddp.subscribe('mp3s');
       var obs = ddp.observe('coll');
       obs.added = function(id) {
           console.log('evvai');
           entry = ddp.collections.coll.find(id);
          console.log(entry);
       };
  });
  ....
});

I have a local meteor instance running with the ROOT_URL set to http://localhost:3000/admin/. I can't seem to figure out the correct settings the DDP client should be set for this to work. Everything works when I remove the ROOT_URL environment variable.

var ddpClient = new DDP({
  host: 'localhost',
  port: 3000,
  //url: 'wss://localhost:3000/admin/websocket',
  //useSockJs: true,
  //ddpVersion: '1'
});

ddpClient.connect(function(error, wasReconnect) {
  if (error) {
    console.log('DDP connection error!');
    return;
  }

  console.log('connected!');
})

The commented out properties are different things I've tried without any success. What am I missing here? Note that I will be using the cluster package in the future so SockJS is required.

Hi there
I've handled socket-error and socket-close events with your sample code but when I manually disconnect the client (like removing the ethernet cable) those handlers never goes in action.
If I shutted down the server, I see socket-close is works well.
Does it flashes any simple solution on your mind?
Any help will be appreciated :)
Thanks :)

ddpclient.on('socket-close', function(code, message) {
  console.log("Close: %s %s", code, message);
});

ddpclient.on('socket-error', function(error) {
  console.log("Error: %j", error);
});

Hi there,
I'm running Apache with SSL on my production server.
So I'm forwarding meteor calls with ProxyPass to inside Meteor app.

When I run the app on local development site, it runs perfectly but when I push the app to production site, it does not.

It's response is:

Error during WebSocket handshake: Unexpected response code: 500

Does it flashed anything for you?
I'm looking further but any help will be appreciated.
Thanks

node-ddp-client uses an old version of the request module that has a security vulnerability. See https://nodesecurity.io/advisories/130.

i am using ddp with react project and it showing me
Error: process.binding is not supported
▶ 19 stack frames were collapsed.
./src/Api/Message.js
src/Api/Message.js:3

1 | import React from 'react'
2 |

3 | var DDPClient = require("ddp");
4 |
5 | const SERVER_URL = 'wss:///be77b069.ngrok.io/websocket'
6 | export function Message(text)

Would be neat to bump this. 0.9.4 doesn't support SNI (which is needed for Galaxy).

0.11.0 seems to work fine.

I have noticed a problem in my code or with the node-ddp-client package. When i have have a node-ddp-client connected to the server, and the server for whatever reason goes down, the cpu usages and the memory of the node script goes up.

Is there anything i could do to change this behavior?

Note: This behavior occurs also when the only thing the script does is to make a connection to the server. and no other code is running

Greetings,

Etienne

I was wondering why in the example.js line 50 the result is undefined?

 setTimeout(function () {
/*
 * Call a Meteor Method
 */
ddpclient.call(
  "deletePosts",             // name of Meteor Method being called
  ["foo", "bar"],            // parameters to send to Meteor Method
  function (err, result) {   // callback which returns the method call results
    console.log("called function, result: " + result);
  },
  function () {              // callback which fires when server has finished
    console.log("updated");  // sending any updated documents as a result of
    console.log(ddpclient.collections.posts);  // calling this method
  }
);

}, 3000);

We are probably mature enough to deserve one of these :)

Hello,

When I try:

var ddpclient = new DDPClient({ host: urls.build.hostname, port: urls.build.port, autoReconnect: true });

 // When we are connected / reconnected then run the handed function
ddpclient.connect(function(error) {
   console.log('RE-/CONNECTED');
 });

It works only during the first connection. No more, when reconnecting. (autoReconnect is set to true).

I opened up an issue here: phillyqueso/node-srp#1 on the SRP code that this module uses.

Trying to login with a bad username throws an error, instead of passing one to the login callback. Can this behavior be changed? Seems a bit messy.

Thanks!
Patrick

I have a node server which makes ddp client connections to push data from a Drupal installation to a Meteor app. Been working great until some fellow devs installed newer version of the ddp client.

Node server using ddp-client: http://cgit.drupalcode.org/spectacle/tree/modules/spectacle_core/ddp.js

When I install [email protected], the default dependencies are (these do not work for me):
[email protected]
└── [email protected]

I am not able to get the ddp server above to work unless I use the dependency versions below:
[email protected]
└── [email protected]

The package.json file we are having to use to make things work for now: http://cgit.drupalcode.org/spectacle/tree/modules/spectacle_core/package.json

Any thoughts on what is going on or how to debug further. Glad to help. :) Thanks!

I just noticed this while reading the 0.8.2 release notes, but am not sure if this is going to affect how this package works. Any thoughts on how this will affect node-ddp before I upgrade to 0.8.2?

In some situation after running for a long time (and I suppose, due to network problems), it is possible that calling a method of DDPClient results in an Error('not opened') being thrown by WebSocket.

This is what the exception looks like:

/usr/local/lib/node_modules/pi-steroid/node_modules/ddp/node_modules/ws/lib/WebSocket.js:175
    else throw new Error('not opened');
               ^
Error: not opened
    at WebSocket.send (/usr/local/lib/node_modules/pi-steroid/node_modules/ddp/node_modules/ws/lib/WebSocket.js:175:16)
    at DDPClient._send (/usr/local/lib/node_modules/pi-steroid/node_modules/ddp/lib/ddp-client.js:77:15)
    at DDPClient.call (/usr/local/lib/node_modules/pi-steroid/node_modules/ddp/lib/ddp-client.js:222:8)

This should be properly catched by DDPClient and transformed into a socket-error event (and a reconnection if automatic reconnection is enabled).