Git Product home page Git Product logo

seneca-mongo-store's Introduction

Seneca

A Seneca.js data storage plugin

seneca-mongo-store

npm version Build Coveralls Dependency Status Maintainability DeepScan grade Gitter chat

js-standard-style

This module is a plugin for Seneca.js. It provides a storage engine that uses MongoDb to persist data.

If you're using this module, and need help, you can:

If you are new to Seneca in general, please take a look at senecajs.org. We have everything from tutorials to sample apps to help get you up and running quickly.

Seneca compatibility

Supports Seneca versions 3.x

Supported functionality

All Seneca data store supported functionality is implemented in seneca-store-test as a test suite. The tests represent the store functionality specifications.

Install

To install, simply use npm. Remember you will need to install Seneca.js separately.

npm install seneca
npm install seneca-mongo-store

Quick Example

var seneca = require('seneca')()
seneca
  .use("entity")
  .use('mongo-store', {
    uri: 'mongodb://120.0.0.1:27017/dbname'
  })

seneca.ready(function () {
  var apple = seneca.make$('fruit')
  apple.name  = 'Pink Lady'
  apple.price = 0.99
  apple.save$(function (err,apple) {
    console.log( "apple.id = "+apple.id  )
  })
})

Connection Options

You can connect to MongoDB in a few different ways:

// URI pattern which gets passed directly to the native MongoDB .connect() method
seneca.use('mongo-store', {
  uri: 'mongodb://120.0.0.1:27017/dbname',
  options: {}
})

// Key based connection gets transformed into a mongodb:// URI
seneca.use('mongo-store', {
  name: 'dbname',
  host: '127.0.0.1',
  port: 27017,
  options: {}
})

The options also gets passed into the MongoDB .connect() method. Refer to the Connection Settings documentation for a list of those options.

Usage

You don't use this module directly. It provides an underlying data storage engine for the Seneca entity API:

var entity = seneca.make$('typename')
entity.someproperty = "something"
entity.anotherproperty = 100

entity.save$(function (err, entity) { ... })
entity.load$({id: ...}, function (err, entity) { ... })
entity.list$({property: ...}, function (err, entity) { ... })
entity.remove$({id: ...}, function (err, entity) { ... })

Query Support

The standard Seneca query format is supported:

  • .list$({f1:v1, f2:v2, ...}) implies pseudo-query f1==v1 AND f2==v2, ....

  • .list$({f1:v1, ..., sort$:{field1:1}}) means sort by f1, ascending.

  • .list$({f1:v1, ..., sort$:{field1:-1}}) means sort by f1, descending.

  • .list$({f1:v1, ..., limit$:10}) means only return 10 results.

  • .list$({f1:v1, ..., skip$:5}) means skip the first 5.

  • .list$({f1:v1, ..., fields$:['fd1','f2']}) means only return the listed fields.

Note: you can use sort$, limit$, skip$ and fields$ together.

  • .list$({f1:v1, ..., sort$:{field1:-1}, limit$:10}) means sort by f1, descending and only return 10 results.

Native Driver

As with all seneca stores, you can access the native driver, in this case, the node-mongodb-native collection object using entity.native$(function (err,collection) {...}). Below we have included a demonstration on how to write a SQL query using Mongo aggregate in Seneca:

SELECT cust_id, count(*) FROM orders GROUP BY cust_id HAVING count(*) > 1
var aggregateQuery = [
  {
    $group: { _id: "$cust_id", count: { $sum: 1 } }
  },
  {
    $match: { count: { $gt: 1 } }
  }
];

orders_ent.native$(function (err, db) {
	var collection = db.collection('orders');
	collection.aggregate(aggregateQuery, function (err, list) {
		if (err) return done(err);
		console.log("Found records:", list);
	});
});

You can also use: entity.list$({f1:v1,...}, {native$:[{-mongo-query-}, {-mongo-options-}]}) which allows you to specify a native mongo query per node-mongodb-native

Contributing

The Senecajs org encourages open participation. If you feel you can help in any way, be it with documentation, examples, extra testing, or new features please get in touch.

To run tests with Docker

Build the Mongo Docker image:

npm run build

Start the Mongo container:

npm run start

Stop the Mongo container:

npm run stop

While the container is running you can run the tests into another terminal:

npm run test

Testing for Mac users

Before the tests can be run you must run docker-machine env default and copy the docker host address (example: '192.168.99.100'). This address must be inserted into the test/mongo.test.js file as the value for the host variable (uri). The tests can now be run.

License

Copyright (c) 2012 - 2021, Richard Rodger and other contributors. Licensed under MIT.

seneca-mongo-store's People

Contributors

adrieankhisbe avatar bamse16 avatar blainsmith avatar code-jace avatar colmharte avatar eoinsha avatar floridemai avatar georgigriffiths avatar gkubisa avatar lilsweetcaligula avatar marian2js avatar maximshe avatar maxymshg avatar mazzy89 avatar mcdonnelldean avatar mihaidma avatar mirceaalexandru avatar nherment avatar rjrodger avatar zamiel49 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

seneca-mongo-store's Issues

Latest version is not on npm

I see that the mongodb dependency was updated a few months ago to ~2.1.2 but that does not reflect in the package from npm.

Please bump up the version and push to npm.

Update seneca-test-store to 1.x.x

Currently the seneca-test-store dependency is 0.3.0 (this is the store interface version it implements)
Needs to be updated to 1.0.0 or later, these test versions include a lot more tests.

using seneca mongodb to update multiple documents

Is there any way to update multiple documents from senecajs mongodb module.

    entity.save$(function (err, entity) { ... })
    entity.load$({id: ...}, function (err, entity) { ... })
    entity.list$({property: ...}, function (err, entity) { ... })
    entity.remove$({id: ...}, function (err, entity) { ... })

Mongo Indexes

Hey, I was curious what the best practice would be to setup indexes on my mongo DB database while using this plugin with my Seneca services. For example setting an index for an email address field for a user document. Each group (roles) of related services are in a separate project so I can deploy to different instances. Thanks!

seneca .load not matching _id field from local database

Hi,

Seneca not matching _id field from local database which i recently migrated from live database. Though same query working on live database.

userEntity.loadPromise({ '_id': new ObjectID(msg.userId) })

Only difference is local mongodb version is 3.2.8 and compose(live) one is 2.6.9.
Query from robomongo tool on local database working fine.do we missing something in this?
Thanks in advance.

Error when running Seneca app on Docker container

I have node.js + seneca app running on docker container which need to access to MongDb server running on the Host machine. I used --net=host when started docker container, but Seneca service is failing to run due to connecting timeout issue. When the app is running locally on host machine everything works fine.
Any advice?

changed connection array and doesn't kept backward compatibility

Merge pool request 9465822
just stopped our connection and throwing errors directly.

Connection Syntax Changes such as this should be added only after BOLD notifications and on Readme also.
Also plugin should give proper errors to show that Connection Syntaxes changed.

we just spend a lot of time to dig on this. hope this should not happen to others.

Move to senecajs

@rjrodger could you please move this project to senecajs?
I could be assigned as temporary maintainer to update it.

ObjectId on sub-objects

Hey everyone!

Is it possible to set an ObjectId to a sub-object besides using the native$ method? If I try to do that and use the save$ method, seneca throws a giant error on my face.

quick example error

seneca.use('mongo-store', {
  uri: 'mongodb://120.0.0.1:27017/dbname'
})
.use("basic")
.use("entity",{mem_store: false})

seneca.ready(function () {
  var apple = seneca.make$('fruit')
  apple.name  = 'Pink Lady'
  apple.price = 0.99
  apple.save$(function (err,apple) {
    console.log( "apple.id = "+apple.id  )
  })
})

seneca mongo store quick sample wont work

I'm using the quick example but it didn't work.
My current setup:

node v8.17
seneca 3.22
seneca-basic 0.5
seneca-entity 2.5
seneca-mongo-store 1.5

File: node index.js

const seneca = require('seneca')();
seneca.use('entity');
seneca.use('mongo-store', {
  uri: 'mongodb://localhost:27017/seneca_testdb'
});

seneca.ready(() => {
  var apple = seneca.make$('fruit')
  apple.name = 'Pink Lady'
  apple.price = 0.99
  apple.save$((err,apple) => {
    console.log( "apple.id = "+apple.id  )
  });
});

But when I run this, I got this error:

{
  "kind": "fatal",
  "level": 600,
  "plugin": "seneca",
  "tag": "3.22.0",
  "id": "q1cgduxozeft/1597741532370/15838/3.22.0/-",
  "code": "ready_failed",
  "notice": "seneca: Ready function failed: instance.private$.get_instance is not a function",
  "err": {
    "eraro": true,
    "orig": {},
    "code": "ready_failed",
    "seneca": true,
    "package": "seneca",
    "msg": "seneca: Ready function failed: instance.private$.get_instance is not a function",
    "details": {
      "message": "instance.private$.get_instance is not a function",
      "errmsg": "instance.private$.get_instance is not a function",
      "errline": "at Entity.log (/home/admin/sample_seneca/node_modules/seneca/lib/logging.js:266:36)",
      "orig$": {},
      "message$": "instance.private$.get_instance is not a function"
    },
    "callpoint": "at Entity.log (/home/admin/sample_seneca/node_modules/seneca/lib/logging.js:266:36)",
    "fatal$": true
  },
  "isot": "2020-08-18T09:05:32.867Z",
  "when": 1597741532867,
  "level_name": "fatal",
  "seneca_id": "q1cgduxozeft/1597741532370/15838/3.22.0/-"
}

Any idea on how to resolve this issue? Thanks

How do I update / modify a document in mongo using seneca-mongo-store?

Hi,

I have tried inserting and deleting a document using seneca-mongo-store and successfully able to insert and delete data in mongo.. How do I update / modify a document from mongo?

const seneca = require('seneca')();
seneca.use('entity');
seneca.use('mongo-store', {
name: 'test',
host: 'localhost',
port: 27017
});

var Device = null;

module.exports =
{
connect : function(done) {
seneca.use('mongo-store', {
name: 'test',
host: 'localhost',
port: 27017
});
console.log('connected');
//if(err) { console.error(err); data({result : 'failed'}) }
seneca.ready(done);
},

addDevice : function(data,done){
    console.log('In addDevice function')
    var newDevice = seneca.make$('sdeviceabccd')
    newDevice.data = data
    console.log("data entered..")
    newDevice.save$(function (err,newDevice) {
        console.log("newDevice:" +newDevice.id)
        if(err) { console.log("newDevice") } 
        done({result : 'success'});
    });
},  

 //Deleting the entry from DB
deleteDevice : function(data,done){
    console.log(data)
    console.log('printed datad..')
    var newData = seneca.make$('sdeviceabccd')
    newData.remove$({'id': '57b2fce913aaa81998573fe5'}, function (err, newData) { 
        console.log('deleted data'); 
        done({result : 'success'}); 
    });
},      

ListDevice : function(done) { 
    console.log('listed..');
    var sounDa = seneca.make$('sdeviceabccd');
    sounDa.list$(function (err, sounDa) {   
    console.log('data listed..' +sounDa)
    if(err) { 
        console.log("listed dev") 
    } 
    done({result : 'success'}); 
})

},
}
can anyone tell me how do I update using the same? I dont find option for update$ in github.

And this is the main file which I am executing:

device = require('./souhom');

device.connect(function(done){
    console.log('connected');
    data = 'devicedata'

device.addDevice(data,function(retval){
    console.log(retval.result); 

device.deleteDevice({ "_id" : "57b2fd09b95a6719acf5f38d"},function(ret) {
    console.log(ret);
    console.log("-----");
device.ListDevice(function(ret) {
    console.log(ret.result);

});
});
});
});

Quick example timeout

I just started up a mongodb and ran the quick example code with node.
The DB is running, accessible and empty.

What am I doing wrong?

Seneca Fatal Error
==================

Message: seneca: Action name:mongo-store,plugin:define,role:seneca,seq:2,tag:undefined failed: [TIMEOUT].

Code: act_execute

Details: { message: '[TIMEOUT]',
  pattern: 'name:mongo-store,plugin:define,role:seneca,seq:2,tag:undefined',
  fn: [Function: plugin_definition],
  cb: [Function: noop],
  instance: 'Seneca/9xs0slqk29ei/1475137197818/14391/3.2.1/-',
  'orig$': [Error: [TIMEOUT]],
  'message$': '[TIMEOUT]',
  plugin: {} }

Stack: 
    at Object.act_tm [as ontm] (/media/data/PROJECTS/Unity/gameServer/simple_server/game-server/node_modules/seneca/seneca.js:922:48)
    at timeout_check [as _repeat] (/media/data/PROJECTS/Unity/gameServer/simple_server/game-server/node_modules/seneca/node_modules/gate-executor/gate-executor.js:213:16)
    at wrapper [as _onTimeout] (timers.js:275:11)
    at Timer.listOnTimeout (timers.js:92:15)

Instance: Seneca/9xs0slqk29ei/1475137197818/14391/3.2.1/-
  ALL ERRORS FATAL: action called with argument fatal$:true (probably a plugin init error, or using a plugin seneca instance, see senecajs.org/fatal.html)
    at Seneca.act_done (/media/data/PROJECTS/Unity/gameServer/simple_server/game-server/node_modules/seneca/seneca.js:1054:29)

When: 2016-09-29T08:20:20.483Z

Log: {kind:null,plugin:seneca,tag:3.2.1,id:9xs0slqk29ei/1475137197818/14391/3.2.1/-,code:act_execute,notice:seneca: 

[...]

Timeout Error: How to remove using ID?

Here is the code

seneca.add({role: "coupons-store", cmd: "delete"}, function(args, respond){
    var coupons = seneca.make$("coupons");
    coupons.remove$({id: args.id}, function(err, entity){
        if(err) return respond(error);

        respond(null, {value: true});   
    });
});

Seems like I am passing the ID wrong way. It gives timeout error. But the document gets deleted from the coupons collection.

Please help me to fix this.

Message: seneca: Action cmd:list,role:entity failed: not authorized for query on collection

Hi,
we are facing this issue with compose.io mongodb 2.6.9 database.
The connection intermittently failing for different query each time, as we have multiple queries within one call. Look like bug is related with below links.
https://jira.mongodb.org/browse/NODE-719
https://jira.mongodb.org/browse/NODE-722

Adding error stack below.

Seneca Fatal Error
==================

Message: seneca: Action cmd:list,role:entity failed: not authorized for query on app<id>.<collection_name>.

Code: act_execute

Details: { 
  gate: false,
  ungate: true,
  desc: undefined,
  cb: [Function],
  fn: [Function],
  time: { start: 1468403687939, end: 1468403688527 },
  'orig$': 
   { [MongoError: not authorized for query on app<id>.<collection_name>
     name: 'MongoError',
     message: 'not authorized for query on app<id>.<collection_name>',
     '$err': 'not authorized for query on app<id>.<collection_name>',
     code: 13 },
  'message$': 'not authorized for query on app<id>.<collection_name>',
  message: 'not authorized for query on app<id>.<collection_name>',
  pattern: 'cmd:list,role:entity',
  instance: 'Seneca/2.1.0/6615/-',
  plugin: {} }

Please can you help us into this. how we can resolve the things, as we are using cloud database.

Listing entity with a query containing: $or mongo operator is no more possible

Since the introduction of automatic conversion from array of values to a "$in" keyword... it is no more possible to perform query containing "$or" keyword.

As mongo is expecting an array value after such property, it is now failing as the real query sent converts the array to object with a "$in" property!

It misses the following test at the end of fixquery method:

        //delete q.id
      } else {
        for (var qp in q) {
          if ('id' !== qp && !qp.match(/\$$/)) {

// FIX START HERE
            // Preserving original mongo operators (like $or, $and, ...) which are expecting an array ....
            if (qp.startsWith('$')) {
              qq[qp] = q[qp]
            } else if (Array.isArray(q[qp])) {
// FIX STOP HERE
            
              qq[qp] = { $in: q[qp] }
            } else {
              qq[qp] = q[qp]
            }
          }
        }
      }
    }

seneca list$ query performance

db.getCollection('result').find({ Id: 'k3h5kj34h53k4j54667gfh' }).explain("executionStats")

If we run above query in Robomongo it is taking 0.13-0.16 sec

if we run this query with seneca list$ it is taking 1260 ms = 1.2 - 1.6 sec

we have already added indexes on appropriate collections.

Is there anything we are missing here?

using $in on _id field does not work

If I run the following query I get an empty list:

var query = {id: {$in: ["52416d63d39323866a000002", "52416d63d39323866a000003"]}}
entity.list$(query, function(err, result){
});

This is because the ids in the list are never translated into ObjectID. I'd expect this to work since there's no other way to run this query that is intuitive.

upsert tests one level deeper than other tests

  Sorting
    Load
      ✔ 70) should support ascending order (2 ms)
      ✔ 71) should support descending order (1 ms)
    List
      ✔ 72) should support ascending order (2 ms)
      ✔ 73) should support descending order (2 ms)
    Remove
      ✔ 74) should support ascending order (2 ms)
      ✔ 75) should support descending order (3 ms)
  upsert tests
    Upserts
      matches on 1 upsert$ field
        ✔ 76) updates the entity (31 ms)

Crash on using seneca-mongo-store with seneca 0.5.3+

The project runs fine with seneca 0.5.2, but it crashes with the following error message with seneca 0.5.3/0.5.4. Log from seneca 0.5.3:

/Users/marius/Documents/Repositories/massive-tyrion/node_modules/seneca-mongo-store/node_modules/mongodb/lib/mongodb/connection/server.js:529
        throw err;
              ^
TypeError: Object.keys called on non-object
    at Function.keys (native)
    at EventEmitter.defaultresponder
(/Users/marius/Documents/Repositories/massive-tyrion/node_modules/seneca/lib/seneca.js:723:11)
    at EventEmitter.<anonymous>
(/Users/marius/Documents/Repositories/massive-tyrion/node_modules/seneca/lib/seneca.js:692:19)
    at /Users/marius/Documents/Repositories/massive-tyrion/node_modules/seneca/lib/seneca.js:1038:16
    at store_unit
(/Users/marius/Documents/Repositories/massive-tyrion/lib/incident.js:246:20)
    at EventEmitter.<anonymous>
(/Users/marius/Documents/Repositories/massive-tyrion/lib/incident.js:288:26)
    at /Users/marius/Documents/Repositories/massive-tyrion/node_modules/seneca/lib/seneca.js:1038:16
    at /Users/marius/Documents/Repositories/massive-tyrion/node_modules/seneca-mongo-store/lib/mongo-store.js:316:15
    at /Users/marius/Documents/Repositories/massive-tyrion/node_modules/seneca-mongo-store/node_modules/mongodb/lib/mongodb/collection.js:1010:5
    at Cursor.nextObject
(/Users/marius/Documents/Repositories/massive-tyrion/node_modules/seneca-mongo-store/node_modules/mongodb/lib/mongodb/cursor.js:653:5)

Not able to run basic example

I was exploring seneca-mongo-store with the example provided and I realize that its storing data into memory. How do I force it to write into mongo.

Code I am using:

var seneca = require('seneca')()
seneca.use('entity')

seneca.use('mongo-store', {
  uri: 'mongodb://127.0.0.1:27017/testdb'
})

seneca.ready(function () {
  var dev = seneca.make$('device')
  dev.id  = 2
  dev.name = 'pii'
  dev.save$(function (err,dev) {
    console.log( "dev.id = "+dev.id  )
  })
})

Also I tried to add {default_plugins: {'mem-store':false}} but it generates error.

2016-05-06T09:44:17.451Z iet7mymndo4p/1462527856926/8385/- WARN act     -               -       -       seneca: No matching action pattern found for { cmd: 'save',  role: 'entity',  ent: $-/-/device;id=2;{name:pii},  name: 'device',  base: undefined,  zone: undefined }, and no default result provided (using a default$ property).  act_not_found   {args:{ cmd: 'save',  role: 'entity',  ent: $-/-/device;id=2;{name:pii},  name: 'device',  base: undefined,  zo       Error: seneca: No matching action pattern found for { cmd: 'save',  role: 'entity',  ent: $-/-/device;id=2;{name:pii},  name: 'device',  base: undefined,  zone: undefined }, and no default result provided (using a default$ property).
    at Object.errormaker [as error] (/home/cts328367/webservice/node_modules/seneca/node_modules/eraro/eraro.js:94:15)
    at Object.execute_action [as fn] (/home/cts328367/webservice/node_modules/seneca/seneca.js:1088:25)
    at Immediate._onImmediate (/home/cts328367/webservice/node_modules/seneca/node_modules/gate-executor/gate-executor.js:136:14)
    at tryOnImmediate (timers.js:543:15)
    at processImmediate [as _immediateCallback] (timers.js:523:5)

Inserting duplicates with unique index results in timeout rather than error

Assuming that:

db.collection('test').ensureIndex({ foo: 1 }, { unique: true });

Using this task:

Seneca.add('act:test', (args, done) => {
  const model = this.make$('test');
  model.foo = 'bar';
  model.save$(done);
}

Doing this:

Seneca.add('act:test', err => {
  Seneca.add('act:test', err => {
    expect(err).to.exist;
  });
});

The first call will succeed, but the second call will log a mongodb error to the console but never call the callback. This results in a task timeout rather than an error being returned to the caller.

non-deterministic test failure

Failed tests:

  1. shared tests Upserts bombarding the store with near-parallel upserts has no race condition - creates a single new entity:
  seneca: Action cmd:save,role:entity failed: E11000 duplicate key error collection: senecatest.users index: email_1 dup key: { : "[email protected]" }.

MongoError: E11000 duplicate key error collection: senecatest.users index: email_1 dup key: { : "[email protected]" }
at MessageStream.messageHandler (/Users/richard/Projects/seneca/mongo-store/node_modules/mongodb/lib/cmap/connection.js:263:20)
at MessageStream.emit (events.js:315:20)
at processIncomingData (/Users/richard/Projects/seneca/mongo-store/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
at MessageStream._write (/Users/richard/Projects/seneca/mongo-store/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
at writeOrBuffer (internal/streams/writable.js:358:12)
at MessageStream.Writable.write (internal/streams/writable.js:303:10)
at Socket.ondata (internal/streams/readable.js:719:22)
at Socket.emit (events.js:315:20)
at addChunk (internal/streams/readable.js:309:12)
at readableAddChunk (internal/streams/readable.js:284:9)
at Socket.Readable.push (internal/streams/readable.js:223:10)
at TCP.onStreamRead (internal/stream_base_commons.js:188:23)

Either this is a real issue to be solved, or the test is validating a feature that is not supported

Can't run minimal Seneca app with mongo-store plugin

I've just tried to run code below, but got an error: "Database name must be a string". Error occurs for both forms of plugin init: by uri & key-based approach. Without mongo-store plugin all works well. Can you please help with this problem?

var seneca = require('seneca')()

seneca.use('mongo-store', {
  uri: 'mongodb://120.0.0.1:27017/dbname'
})

seneca.ready(function () {})

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.