loopbackio / loopback-datasource-juggler Goto Github PK

Feature request after asking on SO here : http://stackoverflow.com/questions/21484963/can-i-use-bitwise-operators-to-request-a-loopback-model

Summary :

Kind of filter that can be used with Loopback :

// Just an example of syntax, it does not do bitwise filter
Inventory.find({where: {status: {gt: 4}});

With a direct connection to MongoDB, we can do :

// Select items with 3rd bit on in `status` field
db.inventory.find({status: {$mod: [4, 0]}});

I suppose most database engines support this operation.

Request : add this possibility into loopback filters

In case you observe the Validations/configure function, you'd notice that there are undefined values being inserted into _validations (first member, attr name):

https://github.com/strongloop/loopback-datasource-juggler/blob/master/lib/validations.js#L607

Apparently, args is not preprocessed correctly, causing faulty/duplicate entries to be created.

This possibly caused by LDL required: true (vs. presence validation), here:

https://github.com/strongloop/loopback-datasource-juggler/blob/master/lib/model-builder.js#L357

Contrary to create(), upsert() does not call any before/after hooks at all.

In addition, it appears to be impossible for any of the beforeHooks to cancel the pending action (save for example). next(new Error('Halted')) apparently doesn't work.

Should we expose the following functions of relations.js in API docs?

• Relation.relationNameFor()
• scopeMethods.add()
• scopeMethods.remove()
  They are in API docs now, but I'm not sure if they are intended for external use.

Shouldn't this be the case? Perhaps this is due to how removeById/deleteById/destroyById are being used? Because of this, it's easy to miss the right remote hook method invocations too.

As a workaround, I'm using Model.on('deleted', ...) which seems to be triggered either way.

Currently trying to query by ID

Activity.findOne({where: {id: $stateParams.id}},
function(activtiy) {
console.log(activtiy)
$scope.currentEvent = activtiy;

});

I get an incorrect object. It seems to always give me the first one if the collection. Tried the same with the find() and it returns all of the objects in the collection.

Here is a simple example app.

var lb = require('loopback');
var user = lb.Model.extend('user');
var account = lb.Model.extend('account');
var memory = lb.memory();

user.attachTo(memory);
account.attachTo(memory);

user.belongsTo(account);

account.create(function(err, a) {
  try {
    a.user.create({
      name: 'joe'
    }, function(err, u) {
      if(err) return console.log('err:', err);
      console.log('done.', u);
    });
  } catch(e) {
    console.log('caught', e);
  }
});

user.create(function(err, u) {
  try {
    u.account.create({
      balance: 0
    }, function(err, account) {
      if(err) return console.log('err:', err);
      console.log('done.', account);
    });
  } catch(e) {
    console.log('caught', e);
  }
});

Here is the output:

caught [TypeError: Cannot call method 'create' of undefined]
caught [TypeError: Object function (refresh, p) {
        if (arguments.length === 1) {
            p = refresh;
            refresh = false;
        } else if (arguments.length > 2) {
            throw new Error('Method can\'t be called with more than two arguments');
        }
        var self = this;
        var cachedValue;
        if (!refresh && this.__cachedRelations && (typeof this.__cachedRelations[methodName] !== 'undefined')) {
            cachedValue = this.__cachedRelations[methodName];
        }
        if (p instanceof ModelBaseClass) { // acts as setter
            this[fk] = p[idName];
            this.__cachedRelations[methodName] = p;
        } else if (typeof p === 'function') { // acts as async getter
            if (typeof cachedValue === 'undefined') {
                this.__finders__[methodName].apply(self, [this[fk], function(err, inst) {
                    if (!err) {
                        self.__cachedRelations[methodName] = inst;
                    }
                    p(err, inst);
                }]);
                return this[fk];
            } else {
                p(null, cachedValue);
                return cachedValue;
            }
        } else if (typeof p === 'undefined') { // acts as sync getter
            return this[fk];
        } else { // setter
            this[fk] = p;
            delete this.__cachedRelations[methodName];
        }
    } has no method 'create']

following hasMany relation defined in model

var RouteDefinition = {
    name: {
        type: String
    },
    company: {
        type: String,
        lowercase: true,
        trim: true,
        required: true,
        validate: /\S+/
    },
    deliveryDate: Date,
    etr: {
        distance: Number,
        time: Number,
        stops: Number
    },
    deliveryInProgress: {
        type: Number
    },
    note: String,
    state: {
        type: String,
        enum: states,
        default: 'Initial'
    },
    deviceTokens: [String],
    stats: {
        routeStart: Date,
        routeEnd: Date,
        lastModified: {
            type: Date,
            default: Date.now
        },
        lastModifiedBy: {
            type: String
        },
        createdAt: {
            type: Date,
            default: Date.now
        }
    }
};

var options = {
    'relations': {
        'deliveries': {
            'model': 'delivery',
            'type': 'hasMany',
            'foreignKey': 'routeId'
        }
    }
};
var Route = Model.extend('deliveryRoute', RouteDefinition, options);

is not recognized.

So following call
http://localhost:3000/api/deliveryRoutes?filter[include][deliveries]
results in following error:

{
  "error": {
    "name": "Error",
    "status": 500,
    "message": "Relation \"deliveries\" is not defined for deliveryRoute model",
    "stack": "Error: Relation \"deliveries\" is not defined for deliveryRoute model\n    at /private/var/www/mapilary-api-strongloop/node_modules/loopback-datasource-juggler/lib/include.js:95:12\n    at Function.Inclusion.include (/private/var/www/mapilary-api-strongloop/node_modules/loopback-datasource-juggler/lib/include.js:53:7)\n    at Function.f (/private/var/www/mapilary-api-strongloop/node_modules/loopback-datasource-juggler/lib/jutil.js:100:15)\n    at Function.f [as include] (/private/var/www/mapilary-api-strongloop/node_modules/loopback-datasource-juggler/lib/jutil.js:100:15)\n    at /private/var/www/mapilary-api-strongloop/node_modules/loopback-connector-mongodb/lib/mongodb.js:444:33\n    at /private/var/www/mapilary-api-strongloop/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:162:16\n    at commandHandler (/private/var/www/mapilary-api-strongloop/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:706:16)\n    at /private/var/www/mapilary-api-strongloop/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/db.js:1806:9\n    at Server.Base._callHandler (/private/var/www/mapilary-api-strongloop/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/base.js:442:41)\n    at /private/var/www/mapilary-api-strongloop/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/server.js:485:18"
  }
}

The only workaround is to defined relation in app.js:

Route.hasMany('deliveries', {model: Delivery, foreignKey: 'routeId'});

Sample project:
https://github.com/mauron85/mapilary-api-strongloop

Testing environment:
slc v2.2.3 (node v0.10.26)

├─┬ [email protected]
├─┬ [email protected]
├─┬ [email protected]
├── [email protected]
├─┬ [email protected]
├─┬ [email protected]
│ ├── [email protected]
└─┬ [email protected]

I am trying to set the default of a hasMany relationship to an empty array. However, based on the docs there doesn't seem to be a way to do so.

Also I tried to make a datemade property and default it to "new Date()" but instead when making it, it will save the string rather than making a new date. When I save the json file as new Date() without the qoutes it gives an error

Consider the use-case described in #85: a product hasAndBelongsToMany categories.

One can get all products of a category CAT-ID without category data by calling

GET /api/categories/CAT-ID/products

To keep the API consistent with what is available for "product belongsTo category" relationship, LoopBack should support also a request similar to this one:

GET /api/products?filter[where][categoryId]=CAT-ID

The where expression should probably allow multiple ids, something along the lines "where categoryIds include {id-list}".

In relations.js at line 134:

  if (params.through) {
    var fk2 = i8n.camelize(anotherClass.modelName + '_id', true);

should be:

  if (params.through) {
    var fk2 = this.dataSource.idName(anotherClass.modelName) ||
      i8n.camelize(anotherClass.modelName + '_id', true);

It would be even better the following definition:

"relations": {
      "images": {
        "type": "hasMany",
        "model": "image",
        "foreignKey": "maltKey", // malt_image.malt_key => malt.malt_key
        "through": "malt-image"
        "throughField": "imageKey", // malt_image.image_key => image.image_key
      }
    }

However, that does not solve the issue, later in scope.js at line 100:

        Object.keys(targetClass._scopeMeta).forEach(function (name) {

_scopeMeta is undefined. If I skip that, then in scope.js at line 74:

          return targetClass.find(params, function (err, data) {

targetClass is the through model (malt-image) instead of the target model (image). It is more interesting that find() is undefined. I abandoned at this point.

var Space = app.loopback.getModel('Space');
var User = app.loopback.getModel('User');
var Account = app.loopback.getModel('Account');

Space.hasAndBelongsToMany('users', { through: Account });
Space.hasMany('accounts');

User.hasAndBelongsToMany(Space, { as: 'spaces', through: Account });
User.hasMany('accounts');

Example: /api/spaces/client-a/users?filter[where][username]=toby // result: []
Example: /api/spaces/client-a/users?filter[order]=username // result: unordered array

Regarding sorting, since models are fetched individually (byId vs. find), ordering is not applied either. This might very well be a limitation of how juggler handles relations at the moment (very flexible regarding mixed datasources, but not optimal performance wise).

I think the best approach would be to gather the related ids from the through-model first, then issue a second query on the actual target model, scoped to the previously retrieved ids - this is not only more efficient, but should enable the filtering capabilities as expected.

Consider the following test in test/relations.test.js, describe("hasAndBelongsToMany"):

    it('should destroy all related instances', function(done) {
      Article.findOne(function (e, article) {
        article.tags.destroyAll(function(err) {
          if (err) return done(err);

          article.tags(true, function(err, list) {
            if (err) return done(err);
            list.should.have.length(0);
          });
        });
      });
    });

// result
// AssertionError: expected [ null ] to have a length of 0 but got 1

/to @raymondfeng

Add support for multiple id columns to CRUD operations.

Let's say an Order belongsTo a User. Now we want to get an Order with the user information loaded, we can do:

/api/orders/?filter[include]=user

It returns:

{
   orderId: ...,
   userId: ...,
   __cachedRelations: {
    user: {...}
  }
}

Ideally, it should return

{
   orderId: ...,
   userId: ...,
   user: {...}
}

malformed REST filter can cause crash in multiple filters.

loopbackio/loopback-connector-mysql#34
@raymondfeng confirms this is an issue in the memory connector

loopbackio/loopback-connector-mysql#33

When calling rest endpoint /api/model/?filter[where][and][]=string_here which is an invalid filter causes an uncaught error.

/loopback-connector-mysql/lib/mysql.js:407
Object.keys(conds).forEach(function (key) {
^
TypeError: Object.keys called on non-object

Unless I'm missing something, this currently doesn't work as expected:

Model.validate('name', function(err) {
  if (this.name === 'foo') err();
}, { message: 'invalid name', code: 'nofoo' });

Expected error reporting:

"codes": {
  "name": [
    "nofoo"
  ]
}

When possible, please promote master branch to production to publish JSDoc changes.

The skipValidation checks return immediately, without handing a callback function (which is the case with async validations, like validatesUniquenessOf). The code exists early, and program flow is halted.

Hello,

The hasAndBelongsToMany does not seem to work with the following models definition:

{
  "email": {
    "options": {
      "base": "Email"
    },
    "dataSource": "mail",
    "public": false
  },
  "user": {
    "options": {
      "base": "User",
      "scopes": {
        "admins": {
          "where": {
            "admin": true
          }
        },
        "partners": {
          "where": {
            "partner": true
          }
        },
        "advertisers": {
          "where": {
            "advertiser": true
          }
        },
        "publishers": {
          "where": {
            "publisher": true
          }
        }
      },
      "relations": {
        "accessTokens": {
          "model": "accessToken",
          "type": "hasMany",
          "foreignKey": "userId"
        },
        "channels": {
          "model": "channel",
          "type": "hasMany",
          "foreignKey": "userId"
        },
        "templates": {
          "model": "template",
          "type": "hasMany",
          "foreignKey": "userId"
        }
      }
    },
    "properties": {
      "id": {
        "type": "string"
      },
      "admin": {
        "type": "boolean"
      },
      "partner": {
        "type": "boolean"
      },
      "advertiser": {
        "type": "boolean"
      },
      "publisher": {
        "type": "boolean"
      }
    },
    "dataSource": "mongodb",
    "public": true
  },
  "accessToken": {
    "options": {
      "base": "AccessToken"
    },
    "dataSource": "mongodb",
    "public": true
  },
  "application": {
    "options": {
      "base": "Application"
    },
    "properties": {
      "id": {
        "type": "string"
      }
    },
    "dataSource": "mongodb",
    "public": true
  },
  "acl": {
    "options": {
      "base": "ACL"
    },
    "properties": {
      "id": {
        "type": "string"
      }
    },
    "dataSource": "mongodb",
    "public": true
  },
  "roleMapping": {
    "options": {
      "base": "RoleMapping"
    },
    "properties": {
      "id": {
        "type": "string"
      }
    },
    "dataSource": "mongodb",
    "public": false
  },
  "role": {
    "options": {
      "base": "Role",
      "relations": {
        "principals": {
          "type": "hasMany",
          "model": "roleMapping",
          "foreignKey": "roleId"
        }
      }
    },
    "properties": {
      "id": {
        "type": "string"
      }
    },
    "dataSource": "mongodb",
    "public": true
  },
  "scope": {
    "options": {
      "base": "Scope"
    },
    "properties": {
      "id": {
        "type": "string"
      }
    },
    "dataSource": "mongodb",
    "public": true
  },
  "push": {
    "options": {
      "base": "Push",
      "plural": "push"
    },
    "properties": {
      "id": {
        "type": "string"
      }
    },
    "dataSource": "push"
  },
  "installation": {
    "options": {
      "base": "Installation"
    },
    "dataSource": "mongodb",
    "public": true
  },
  "notification": {
    "options": {
      "base": "Notification"
    },
    "properties": {
      "id": {
        "type": "string"
      }
    },
    "dataSource": "mongodb",
    "public": true
  },
  "channel": {
    "options": {
      "relations": {
        "categories": {
          "model": "category",
          "type": "hasAndBelongsToMany",
          "foreignKey":"categoryId"
        },
        "ads": {
          "model": "ad",
          "type": "hasMany",
          "foreignKey": "channelId"
        },
        "pricing": {
          "model": "pricing",
          "type": "belongsTo",
          "foreignKey": "pricingId"
        }
      }
    },
    "properties": {
      "id": {
        "type": "string"
      },
      "userId": {
        "type": "string"
      },
      "pricingId": {
        "type": "string"
      },
      "type": {
        "type": "string"
      },
      "keywords": {
        "type": "array"
      },
      "meta": {
        "type": "string"
      },
      "created": {
        "type": "date"
      },
      "modified": {
        "type": "date"
      }
    },
    "public": true,
    "dataSource": "mongodb",
    "plural": "channels"
  },
  "category": {
    "options": {
      "relations": {
        "channels": {
          "model": "channel",
          "type": "hasAndBelongsToMany",
          "foreignKey": "categoryId"
        },
        "templates": {
          "model": "template",
          "type": "hasAndBelongsToMany",
          "foreignKey":"templateId"
        }
      }

    },
    "properties": {
      "id": {
        "type": "string"
      },
      "name": {
        "type": "string"
      },
      "created": {
        "type": "date"
      },
      "modified": {
        "type": "date"
      }
    },
    "public": true,
    "dataSource": "mongodb",
    "plural": "categories"
  },
  "ad": {
    "options": {
      "relations": {
        "channel": {
          "model": "channel",
          "type": "belongsTo",
          "foreignKey": "channelId"
        }
      }
    },
    "properties": {
      "id": {
        "type": "string"
      },
      "channelId": {
        "type": "string"
      },
      "alt": {
        "type": "string"
      },
      "url": {
        "type": "string"
      },
      "width": {
        "type": "number"
      },
      "height": {
        "type": "number"
      },
      "views": {
        "type": "number"
      },
      "ctr": {
        "type": "number"
      },
      "created": {
        "type": "date"
      },
      "modified": {
        "type": "date"
      }
    },
    "public": true,
    "dataSource": "mongodb",
    "plural": "ads"
  },
  "template": {
    "options": {
      "relations": {
        "channel": {
          "model": "channel",
          "type": "belongsTo",
          "foreignKey": "channelId"
        },
        "categories": {
          "model": "category",
          "type": "hasAndBelongsToMany",
          "foreignKey":"categoryId"
        }
      }
    },
    "properties": {
      "id": {
        "type": "string"
      },
      "userId": {
        "type": "string"
      },
      "map": {
        "type": "object"
      },
      "channelId": {
        "type": "string"
      },
      "html": {
        "type": "string"
      },
      "keywords": {
        "type": "array"
      },
      "created": {
        "type": "date"
      },
      "modified": {
        "type": "date"
      }
    },
    "public": true,
    "dataSource": "mongodb",
    "plural": "templates"
  },
  "pricing": {
    "options": {
      "relations": {
        "type": {
          "model": "pricingType",
          "type": "belongsTo",
          "foreignKey": "pricingTypeId"
        },
        "channels": {
          "model": "channel",
          "type": "hasMany",
          "foreignKey": "pricingId"
        }
      }
    },
    "properties": {
      "id": {
        "type": "string"
      },
      "pricingTypeId": {
        "type": "string"
      },
      "bid": {
        "type": "number"
      },
      "created": {
        "type": "date"
      },
      "modified": {
        "type": "date"
      }
    },
    "public": true,
    "dataSource": "mongodb",
    "plural": "pricings"
  },
  "pricingType": {
    "options": {
      "relations": {
        "pricings": {
          "model": "pricing",
          "type": "hasMany",
          "foreignKey": "pricingTypeId"
        }
      }
    },
    "properties": {
      "id": {
        "type": "string"
      },
      "name": {
        "type": "string"
      },
      "created": {
        "type": "date"
      },
      "modified": {
        "type": "date"
      }
    },
    "public": true,
    "dataSource": "mongodb",
    "plural": "pricingTypes"
  }
}

Any idea where the issue might be?

When i post the {"categoryId":"535babcb7aa13b693e9a3f1e"} to templates/535bac1c7aa13b693e9a3f20/categories, it does not seem to work, because then, when i make a GET request to templates/535bac1c7aa13b693e9a3f20/categories, all i get is an empty array, and there is not much else to help me out as far as documentation goes.
The content of the templatecategory collection is:

{ "templateId" : "535ba93d53ad7def3d45eae2", "categoryId" : ObjectId("535ba98a53ad7def3d45eae3"), "_id" : ObjectId("535ba98a53ad7def3d45eae4") }
{ "templateId" : "535ba93d53ad7def3d45eae2", "categoryId" : ObjectId("535baa2053ad7def3d45eae5"), "_id" : ObjectId("535baa2053ad7def3d45eae6") }
{ "templateId" : "535bac1c7aa13b693e9a3f20", "categoryId" : ObjectId("535bac507aa13b693e9a3f21"), "_id" : ObjectId("535bac507aa13b693e9a3f22") }
{ "templateId" : "535bac1c7aa13b693e9a3f20", "categoryId" : ObjectId("535bac6c7aa13b693e9a3f23"), "_id" : ObjectId("535bac6c7aa13b693e9a3f24") }
{ "templateId" : ObjectId("535bad327aa13b693e9a3f25"), "categoryId" : "535babcb7aa13b693e9a3f1e", "_id" : ObjectId("535bad327aa13b693e9a3f26") }
{ "templateId" : ObjectId("535bad667aa13b693e9a3f27"), "categoryId" : "535babcb7aa13b693e9a3f1e", "_id" : ObjectId("535bad667aa13b693e9a3f28") }

I am using connector-loopback-mongodb for the data, and it doesn't seem to work

Cheers,

Alex.

Background

Remoting supports instance methods. Its very cumbersome and error prone. I'd like to remove the functionality. We only really depend on it for relations if at all.

A Cleaner Relation API

Setting aside relationship definition for now. I think we can offer a better base api for relations. Currently we have:

var myModel = new MyModel(...);
myModel.relatedModelName.create(...);

We can keep this API, but rebuild it on top of something like this:

MyModel.createRelated(idOrFindOneFilter, 'relatedModelName', data);

Then myModel.relatedModelName.create and others would just delegate to MyModel.createRelated(...).

We would implement these in dao.js which would add support for remoting these.

While the _ indicates this to be a private method, it would be beneficial for application-level code to be able to re-use this implementation. Please consider making this public, or refactor this into a seperate method that can be used to normalize a (generic) object's properties.

http://cassandra.apache.org/

There are a few open source drivers in Node:

• https://github.com/racker/node-cassandra-client
• https://github.com/jorgebay/node-cassandra-cql
• https://github.com/simplereach/helenus

I have two Models defined; Location and Address, defined as follows:

"address": {
    "properties": {
        "address1": {
            "type": "string",
            "required": true
        }
    },
    "public": true,
    "dataSource": "db",
    "plural": "addresses"
},  
"location": {
    "properties": {
        "title": {
            "type": "string"
        },
        "address":{
            "type": "address",
            "required": true
        }
    },
    "public": true,
    "dataSource": "db",
    "plural": "locations"
}

When I Create an object based on the address Model via the API Explorer, the required constraint on address1 behaves as expected and I must supply a value to create the object.

When I Create an object based on the location Model via the API Explorer, the required constraint on the address works as expected, but the required constraint on the address1 of the address Model does not. I am able to create an address without an address1, which is not what I would expect.

I am using a memory datasource, node v0.10.24 and loopback v1.8.6.

The same issue is noted for JugglingDB here: 1602/jugglingdb#391

Example: We assert uniqueness of username in a user model. If two users are created with the same username very close together, then both will succeed, since the uniqueness check is a round trip (really just counting existing instances with the same username) and both will pass uniqueness before either creation is made.

findById seems to work in 'hasMany' relations like this(https://github.com/strongloop/loopback-datasource-juggler/blob/master/examples/relations.js#L54), but in this case(https://github.com/strongloop/loopback-datasource-juggler/blob/master/examples/relations.js#L117), if I use

assembly.parts.findById(....)

It returns the part regardless the part has any connection with the assembly, just like the following works:

Part.findById(...)

BTW, is there any other way to check out whether a model exists in another model's relation?

LoopBack models should be able to attach to datasources backed by messaging connectors to:

1. Send messages to a queue
2. Receive messages from a queue
3. Listen on messages from a queue
4. Publish to a topic
5. Subscribe to a topic.

Consider the following setup: "Product belongsTo Category".

For a product with id 1, one can get the category it belongs to via

Product.findById(1, function(err, prod) {
  prod.category(function(err, cat) {
    // see "cat"
  });
});

The method is not exposed via the REST API, although it should be.

GET /products/1/category
404 Not Found

The primary reason is that the relation function is not enumerable (code).

Note that changing 'enumerable' to true is not enough, as the method is invoked with a wrong "this" argument when called via remoting.

@raymondfeng I am wondering how did you test the pull request #86 which was supposed to fix the issue?

Hey, so a really nice feature to steal from ActiveRecord would be dependent deletions. For example, when a user with the User.hasMany(Posts, { dependent: true }) relation is deleted, their posts should also be deleted or you'd have orphaned data.

We have all the info we need to do cascading deletes. I'm thinking of implementing like this:

Book.belongsTo(Library); // should enable cascading delete

Still a thought in progress. Feel free to ignore for now. If anyone is aware of a good example for configuring / enabling cascading deletes please post a link.

I ran across this issue when dealing with related documents with a Mongo DB backend. I have a models.json file that looks contains something similar to:

{
  "service": {
    "properties": {
      "slug": {
        "type": "String"
      },
      "title": {
        "type": "String"
      },
      "link": {
        "type": "String"
      },
      "subtitle": {
        "type": "String"
      },
      "intro": {
        "type": "String"
      },
      "content": {
        "type": "String"
      },
      "icon": {
        "type": "Object"
      },
      "image": {
        "type": "Object"
      }
    },
    "public": true,
    "dataSource": "db"
  },
  "serviceCategory": {
    "options": {
      "relations":{
        "services": {
          "model": "service",
          "type": "hasMany"
        }
      }
    },
    "properties": {
      "title": {
        "type": "String"
      },
      "tagline": {
        "type": "String"
      },
      "subtitle": {
        "type": "String"
      },
      "intro": {
        "type": "String"
      },
      "content": {
        "type": "String"
      },
      "image": {
        "type": "object"
      }
    },
    "public": true,
    "dataSource": "db",
    "plural": "serviceCategories"
  }
}

When retrieving services with a request similar to /api/serviceCategories/<id>/services I get back objects that have a serviceCategoryId field with the string equivalent of a Mongo ID. If a change is made to one of those retrieved documents and it is updated via a PUT request, what was an ObjectId is replaced with a string and Mongo loses the relationship. (i.e. the document is no longer retrievable via the original request.)

In a conversation over IRC rfeng suggested that creating a custom ObjectId type for Mongo might be a solution to this issue and that I should file a bug here for direction on how to go about that. Thinking about the issue, I'm wondering if this should be handled when the initial relationship is defined, that appears to be where the foreign key field is implicitly created. In either event, an ObjectId type is likely a requirement.

Feature request

Current Model.upsert() creates or update a single model instance in underlying storage (as expected)

There is currently no API facility to update multiple model instances, which can be pretty handy for example to update status fields on a large amount of lines, or such purpose.

New Model feature proposed by raymond here :

Model.update(query, data, cb)

query param would have the same definition as in find-like methods

data param would contain model field in the same format accepted by updateAttributes

Currently this works, and includes the files as an embedded array in the results:

/api/products?filter[include]=files while:

/api/products/1?filter[include]=files does not work. No relationships are included.

You'd expect this to just work, albeit skipping the 'filter' and just specify 'include'.

Perhaps an 'after remote hook' can take care of this?

https://groups.google.com/forum/#!topic/loopbackjs/PWB4naSgYZg

I have relations setup with a model which includes a property similar to this:

versions: [
  {
    id:         String
  }
]

If the versions property is actually an object, rather than an array (due to an error in saving), loopback crashes when it attempts to resolve relations involving this model.

This is the error I get:

/Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/connection/base.js:245
        throw message;
              ^
Error: Items must be an array: [object Object]
  at new List (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/list.js:25:11)
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/model.js:178:39
  at Array.forEach (native)
  at Function.ModelClass.forEachProperty (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/model-builder.js:271:51)
  at ModelConstructor.ModelBaseClass._initProperties (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/model.js:164:8)
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/dao.js:557:13
  at Array.forEach (native)
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/dao.js:554:12
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/include.js:59:11
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/node_modules/async/lib/async.js:113:21
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/node_modules/async/lib/async.js:24:16
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/include.js:124:9
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/node_modules/async/lib/async.js:111:13
  at Array.forEach (native)
  at _each (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/node_modules/async/lib/async.js:32:24)
  at Object.async.each (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/node_modules/async/lib/async.js:110:9)
  at processIncludeItem (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/include.js:106:11)
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/include.js:57:5
  at /Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/node_modules/async/lib/async.js:111:13
  at Array.forEach (native)
  at _each (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/node_modules/async/lib/async.js:32:24)
  at Object.async.each (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/node_modules/async/lib/async.js:110:9)
  at Function.Inclusion.include (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/include.js:56:9)
  at Function.f (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/jutil.js:100:15)
  at Function.f [as include] (/Users/zackbloom/s/API/node_modules/loopback-datasource-juggler/lib/jutil.js:100:15)
  at /Users/zackbloom/s/API/node_modules/loopback-connector-mongodb/lib/mongodb.js:519:33
  at /Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/cursor.js:163:16
  at commandHandler (/Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/cursor.js:709:16)
  at /Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/db.js:1846:9
  at Server.Base._callHandler (/Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/connection/base.js:445:41)
  at /Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/connection/server.js:468:18
  at [object Object].MongoReply.parseBody (/Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
  at [object Object].<anonymous> (/Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/connection/server.js:426:20)
  at [object Object].EventEmitter.emit (events.js:95:17)
  at [object Object].<anonymous> (/Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:201:13)
  at [object Object].EventEmitter.emit (events.js:98:17)
  at Socket.<anonymous> (/Users/zackbloom/s/API/node_modules/mongodb/lib/mongodb/connection/connection.js:339:18)
  at Socket.EventEmitter.emit (events.js:95:17)
  at Socket.<anonymous> (_stream_readable.js:746:14)
  at Socket.EventEmitter.emit (events.js:92:17)
  at emitReadable_ (_stream_readable.js:408:10)
  at emitReadable (_stream_readable.js:404:5)
  at readableAddChunk (_stream_readable.js:165:9)
  at Socket.Readable.push (_stream_readable.js:127:10)
  at TCP.onread (net.js:528:21)

The error itself isn't terrible, but I'd certainly rather it be returned through the HTTP request, rather than crashing the entire app.

I've tried wrapping parts of that stack in a try block, but it looks like the mongodb connector does some fancy magic with nextTick to bypass the actual stack.

I use this example to show the situation

/**
 * Module Dependencies
 */
var loopback = require('loopback'),
    app = require('../app'),
    ds = app.datasources.db;

/**
 * Cancha config
 */
var config = {
    "properties": {
        "age": {
            type: "number",
            required: true
        }
    },
    "options": {
        schema: true
    }
}

/**
 * Cancha Model
 */
var Cancha = ds.createModel('cancha', config.properties, config.options);
app.model(Cancha); //Expose it to REST API

module.exports = Cancha;

If you POST this

{
    "age": []
}

the Array is casted to 0 ( good) and this is the response

{
  "age": 0,
  "id": 2
}

The problem arises when we POST a object

{
    "age": {}
}

Then, the age, which shoud not be null or empty ( we are requiring a value with required: true), is null

{
  "age": null,
  "id": 3
}

So, we are bypassing the validation

An app crashes when a 'list' (array) property doesn't receive the correct input:

"tags": {
  "type": ["string"]
}

REST Request (pseudo code): POST { "tags": "one" } (should have been array not string)

Error: could not create List from JSON string: at new List (<...>/node_modules/loopback-datasource-juggler/lib/list.js:16:13)

This should be handled gracefully by any means, not bail out completely! (perhaps even cast a scalar to an single-item array). A more extensible approach would be custom formatters/converters/coercing - see #128

Example: GET http://localhost:4444/api/entries/count?where=1

/loopback-datasource-juggler/lib/connectors/memory.js:339
  var keys = Object.keys(where);
                    ^
TypeError: Object.keys called on non-object

Where does the actual implementation of property formatters (trim, uppercase, ...) come from? Is it provided by a connector (or a database engine itself)? I could not find any references in-code, and it'd be interesting to be able to provide your own formatters (like urlify/slugify for example).

Consider the use-case described in #85: a product hasAndBelongsToMany categories.

At the moment, it's possible to retrieve a category by id with all its products via findOne:

GET /api/categories/findOne?filter[where][id]=CAT-ID&filter[include]=products

We should add support for include to findById too, so that the request can be simplified to e.g.

GET /api/categories/CAT-ID?include=products

The 'DataSource.prototype.discoverSchemas' only discovers relations of type 'belongsTo'. I can't find an way to determine relations of all types in Package?

https://github.com/strongloop/loopback-datasource-juggler/blob/master/lib/datasource.js#L992

See: https://github.com/strongloop/loopback-datasource-juggler/blob/master/lib/connectors/memory.js#L379

To make the comparison follow MongoDB for example, the value should intersect at least once (by casting any value to an array).

Something like (needs a more fitting and well-tested solution, without dependencies):

if (example.inq) {
  if (!value) return false;
  var sample = [].concat(value || []);
  return _.intersect(example.inq, sample).length > 0;
}

When we have a schema like this

{
    "location": {
        type:"GeoPoint",
        required: true
    }
}

and we post a value for location which is not a GeoPoint representation of lat and lgn, like this

{
    "age": "this is not a lat and lng"
}

Instead of returning a "user friendly" like other validations, something like this

{
  "error": {
    "name": "ValidationError",
    "status": 422,
    "message": "The Model instance is not valid. See `details` property of the error object for more info.",
    "statusCode": 422,
    "details": {
      "context": "cancha",
      "codes": {
        "age": [
          "presence"
        ]
      },
      "messages": {
        "age": [
          "can't be blank"
        ]
      }
    },
    "stack": "ValidationError: The Model instance is not valid... "
  }
}

Instead it throws an AssertionError killing the Node.js process

AssertionError: lat must be a number when creating a GeoPoint
    at new GeoPoint (/home/foo/bar/changa/node_modules/loopback-datasource-juggler/lib/geo.js:134:3)

This force us to do the validation manually, thing I don't know how to do. I tried with this without success

try{
    Cancha.create({
        "location": "wrong geopoint"
    }, function(err, cancha){
        if(err)
            return console.log(err);
        return console.log(cancha.isValid());
    })
}catch(ev){
    console.log(ev);
}

Consider the following three examples of models.json passed to loopback().boot():

// 1. works
AccessToken: {
  options: {
    base: 'AccessToken'
  },
  dataSource: 'db'
},
User: {
  options: {
    base: 'User',
    relations: {
      accessTokens: {
        model: 'AccessToken',
        type: 'hasMany',
        foreignKey: 'userId'
      }
    }
  },
  dataSource: 'db'
},

// 2. does not work, User.prototype.accesTokens is undefined when called from User.login
User: {
  options: {
    base: 'User',
    relations: {
      accessTokens: {
        model: 'AccessToken',
        type: 'hasMany',
        foreignKey: 'userId'
      }
    }
  },
  dataSource: 'db'
},
AccessToken: {
  options: {
    base: 'AccessToken'
  },
  dataSource: 'db'
},

// 3. lowercase name of `accessToken`, it works
User: {
  options: {
    base: 'User',
    relations: {
      accessTokens: {
        model: 'accessToken',
        type: 'hasMany',
        foreignKey: 'userId'
      }
    }
  },
  dataSource: 'db'
},
accessToken: {
  options: {
    base: 'AccessToken'
  },
  dataSource: 'db'
},

I am not sure what is the exact source of the problem, but this does not feel right. It may be a problem of app.boot, in which case juggler should provide an API that app.boot can use to prevent it.

/to @raymondfeng

This would be an interesting feature to support. It means that the model class/type is stored alongside the key.

Somewhat related: given that model inheritance is possible, it might make sense to at least support one level of this hierarchy for fetching:

Animal > Mammal
Animal > Reptile

Mammal.find() would find only Mammals, whereas:
Animal.find() would find both Mammals and Reptiles.

See: https://github.com/briankircho/mongoose-schema-extend for an example of this.

Essentially, being able to control the default association scopes ('where') should allow this, and other functionality, like setting the default sort order for these relations.

Following this discussion
https://groups.google.com/forum/#!topic/loopbackjs/2UhjYRnR5kU

I'd suggest to modify BaseSQL.find method, as following

/**
 * Find a model instance by id
 * @param {String} model The model name
 * @param {*} id The id value
 * @param {Function} callback The callback function
 */
BaseSQL.prototype.find = function find(model, id, callback) {
  var props = this._models[model].properties;
  var idQuery = (id === null || id === undefined)
    ? this.idColumnEscaped(model) + ' IS NULL'
    : this.idColumnEscaped(model) + ' = ' + this.toDatabase(props[this.idColumn(model)], id);
  var sql = 'SELECT * FROM ' +
    this.tableEscaped(model) + ' WHERE ' + idQuery + ' LIMIT 1';

  this.query(sql, function (err, data) {
    if (data && data.length === 1) {
      data[0].id = id;
    } else {
      data = [null];
    }
    callback(err, this.fromDatabase(model, data[0]));
  }.bind(this));
};

and I think we need the same for BaseSQL.exists, BaseSQL.save and BaseSQL.delete because all of them use "id" as integer. :(

As discussed in https://groups.google.com/forum/#!topic/loopbackjs/z7bW9UFJzCU

I recently noticed some issues or sub-optimal behaviour with loopback models storage in the database :

• How index are created (for exampe: mongodb foreign key are not indexed, at least for many-to-many)
• How interface tables items are dealt with, for example deleting links when a parent object is deleted (cascade)

Example : when i remove items in table Product and/or Category, via loopback API, items in product and category collection are deleted, but not productcategory collection

In a RDBMS, I suspect the best practice is to define FK and cascade behaviour in the DB engine, but in noSQL like mongodb, should not this be taken care of by Loopback ?

This is probably related to the fix for the ObjectId issue. Loopback is using a Array like object called List to wrap Array passed over the api. Internally, that List instances look contain an items array that stores that actual content. When an updateAttributes is used to make changes to a model with an Array attribute, it is written to the database as a List rather than a raw array and causes subsequent reads to fail, crashing node.

/Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/connection/base.js:242
        throw message;
              ^
TypeError: Object #<Object> has no method 'forEach'
    at Array.List (/Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-datasource-juggler/lib/list.js:47:8)
    at /Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-datasource-juggler/lib/model.js:174:39
    at Array.forEach (native)
    at Function.ModelClass.forEachProperty (/Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-datasource-juggler/lib/model-builder.js:274:51)
    at ModelConstructor.ModelBaseClass._initProperties (/Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-datasource-juggler/lib/model.js:160:8)
    at Function.<anonymous> (/Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-datasource-juggler/lib/dao.js:323:11)
    at /Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-connector-mongodb/lib/mongodb.js:249:17
    at /Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/collection/query.js:147:5
    at Cursor.nextObject (/Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:733:5)
    at commandHandler (/Users/csalch/code/bestfit/surdell/express-rest/node_modules/loopback-connector-mongodb/node_modules/mongodb/lib/mongodb/cursor.js:713:14)

I'm not sure what a good fix for this would be as it will likely break the ObjectID stuff again.

One would expect these to be implemented when creating a custom Scope. Also:

DataAccessObject.scope should accept scope methods, as defineScope allows this.