Git Product home page Git Product logo

linvodb3's Introduction

LinvoDB

LinvoDB is a Node.js/NW.js/Electron persistent DB with MongoDB / Mongoose-like features and interface.

Features:

  • MongoDB-like query language
  • Persistence built on LevelUP - you can pick back-end
  • NW.js/Electron friendly - JS-only backend is level-js or Medea
  • Performant - steady performance unaffected by DB size - queries are always indexed
  • Auto-indexing
  • Live queries - make the query, get constantly up-to-date results
  • Schemas - built-in schema support
  • Efficient Map / Reduce / Limit

Coming soon:

  • Streaming cursors
  • Distributed dataset

Relationship to NeDB

LinvoDB is based on NeDB, the most significant core change is that it uses LevelUP as a back-end, meaning it doesn't have to keep the whole dataset in memory. LinvoDB also can do a query entirely by indexes, meaning it doesn't have to scan the full database on a query.

In general:

  • LinvoDB is better for large datasets (many objects, or large objects) because it doesn't keep the whole DB in memory and doesn't need to always scan it
  • LinvoDB does the entire query through the indexes, NeDB scans the DB
  • Both LinvoDB and NeDB play well with NW.js (node-webkit). LinvoDB can be initialized with the JS-only level-js back-end.
  • NeDB is ultra-fast because the DB is in memory, LinvoDB's performance is comparible to MongoDB. LinvoDB is faster for large datasets.
  • LinvoDB has live queries, map/reduce and schema support.
  • Both LinvoDB and NeDB are unsuitable for huge datasets (big data)
  • Combining NeDB's in-memory data and LinvoDB's full-indexed queries would yield even better performance. If you want to sacrifice memory for query performance, you can use LinvoDB with a backend that works like that or with LevelDB + increased LRU cache

Install, Initialize, pick backend

Install:

npm install linvodb3 level-js # For NW.js, using level-js
npm install linvodb3 leveldown # For pure node.js, using LevelDB

Initialize:

var LinvoDB = require("linvodb3");

// The following two lines are very important
// Initialize the default store to level-js - which is a JS-only store which will work without recompiling in NW.js / Electron
LinvoDB.defaults.store = { db: require("level-js") }; // Comment out to use LevelDB instead of level-js
// Set dbPath - this should be done explicitly and will be the dir where each model's store is saved
LinvoDB.dbPath = process.cwd(); 

var Doc = new LinvoDB("doc", { /* schema, can be empty */ })

Initialization, detailed:

var LinvoDB = require("linvodb3");
var modelName = "doc";
var schema = { }; // Non-strict always, can be left empty
var options = { };
// options.filename = "./test.db"; // Path to database - not necessary 
// options.store = { db: require("level-js") }; // Options passed to LevelUP constructor 
var Doc = new LinvoDB(modelName, schema, options); // New model; Doc is the constructor

LinvoDB.dbPath // default path where data files are stored for each model
LinvoDB.defaults // default options for every model

Insert / Save

The native types are String, Number, Boolean, Date and null. You can also use arrays and subdocuments (objects). If a field is undefined, it will not be saved.

If the document does not contain an _id field, one will be automatically generated (a 16-characters alphanumerical string). The _id of a document, once set, cannot be modified.

// Construct a single document and then save it
var doc = new Doc({ a: 5, now: new Date(), test: "this is a string" });
doc.b = 13; // you can modify the doc 
doc.save(function(err) { 
	// Document is saved
	console.log(doc._id);
});

// Insert document(s)
// you can use the .insert method to insert one or more documents
Doc.insert({ a: 3 }, function (err, newDoc) {
	console.log(newDoc._id);
});
Doc.insert([{ a: 3 }, { a: 42 }], function (err, newDocs) {
	// Two documents were inserted in the database
	// newDocs is an array with these documents, augmented with their _id

	// If there's an unique constraint on 'a', this will fail, and no changes will be made to the DB
	// err is a 'uniqueViolated' error
});

// Save document(s)
// save is like an insert, except it allows saving existing document too
Doc.save([ doc, { a: 55, test: ".save is handy" } ], function(err, docs) { 
	// docs[0] is doc
	// docs[1] is newly-inserted document with a=55 and has an assigned _id

	// Doing that with .insert would throw an uniqueViolated error for _id on doc, because it assumes all documents are new
});

Querying

Use find to look for multiple documents matching you query, or findOne to look for one specific document. You can select documents based on field equality or use comparison operators ($lt, $lte, $gt, $gte, $in, $nin, $ne, $regex, $exists). You can also use logical operators $or, $and and $not. See below for the syntax.

var Planet = new LinvoDB("planet", { /* schema, can be empty */ })

// Let's say our datastore contains the following collection
Planet.save([ 
	{ _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false, satellites: ['Phobos', 'Deimos'] },
	{ _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true, humans: { genders: 2, eyes: true } },
	{ _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false },
	{ _id: 'id4', planet: 'Omicron Persei 8', system: 'futurama', inhabited: true, humans: { genders: 7 } },
	{ _id: 'id5', completeData: { planets: [ { name: 'Earth', number: 3 }, { name: 'Mars', number: 2 }, { name: 'Pluton', number: 9 } ] } }
], function() { 

// Finding all planets in the solar system
Planet.find({ system: 'solar' }, function (err, docs) {
  // docs is an array containing documents Mars, Earth, Jupiter
  // If no document is found, docs is equal to []
});

// Finding all inhabited planets in the solar system
Planet.find({ system: 'solar', inhabited: true }, function (err, docs) {
  // docs is an array containing document Earth only
});

// Use the dot-notation to match fields in subdocuments
Planet.find({ "humans.genders": 2 }, function (err, docs) {
  // docs contains Earth
});

// Use the dot-notation to navigate arrays of subdocuments
Planet.find({ "completeData.planets.name": "Mars" }, function (err, docs) {
  // docs contains document 5
});

Planet.find({ "completeData.planets.0.name": "Earth" }, function (err, docs) {
  // docs contains document 5
  // If we had tested against "Mars" docs would be empty because we are matching against a specific array element
});

// You can also deep-compare objects. Don't confuse this with dot-notation!
Planet.find({ humans: { genders: 2 } }, function (err, docs) {
  // docs is empty, because { genders: 2 } is not equal to { genders: 2, eyes: true }
});

// Find all documents in the collection
Planet.find({}, function (err, docs) {
});

// The same rules apply when you want to only find one document
Planet.findOne({ _id: 'id1' }, function (err, doc) {
  // doc is the document Mars
  // If no document is found, doc is null
});


}); // end of .save()

Operators ($lt, $lte, $gt, $gte, $in, $nin, $ne, $exists, $regex)

The syntax is { field: { $op: value } } where $op is any comparison operator:

  • $lt, $lte: less than, less than or equal
  • $gt, $gte: greater than, greater than or equal
  • $in: member of. value must be an array of values
  • $ne, $nin: not equal, not a member of
  • $exists: checks whether the document posses the property field. value should be true or false
  • $regex: checks whether a string is matched by the regular expression. Contrary to MongoDB, the use of $options with $regex is not supported, because it doesn't give you more power than regex flags. Basic queries are more readable so only use the $regex operator when you need to use another operator with it (see example below)
// $lt, $lte, $gt and $gte work on numbers and strings
Planet.find({ "humans.genders": { $gt: 5 } }, function (err, docs) {
  // docs contains Omicron Persei 8, whose humans have more than 5 genders (7).
});

// When used with strings, lexicographical order is used
Planet.find({ planet: { $gt: 'Mercury' }}, function (err, docs) {
  // docs contains Omicron Persei 8
})

// Using $in. $nin is used in the same way
Planet.find({ planet: { $in: ['Earth', 'Jupiter'] }}, function (err, docs) {
  // docs contains Earth and Jupiter
});

// Using $exists
Planet.find({ satellites: { $exists: true } }, function (err, docs) {
  // docs contains only Mars
});

// Using $regex with another operator
Planet.find({ planet: { $regex: /ar/, $nin: ['Jupiter', 'Earth'] } }, function (err, docs) {
  // docs only contains Mars because Earth was excluded from the match by $nin
});

Array fields

When a field in a document is an array the query is treated as a query on every element and there is a match if at least one element matches.

// If a document's field is an array, matching it means matching any element of the array
Planet.find({ satellites: 'Phobos' }, function (err, docs) {
  // docs contains Mars. Result would have been the same if query had been { satellites: 'Deimos' }
});

// This also works for queries that use comparison operators
Planet.find({ satellites: { $lt: 'Amos' } }, function (err, docs) {
  // docs is empty since Phobos and Deimos are after Amos in lexicographical order
});

// This also works with the $in and $nin operator
Planet.find({ satellites: { $in: ['Moon', 'Deimos'] } }, function (err, docs) {
  // docs contains Mars (the Earth document is not complete!)
});

Logical operators $or, $and, $not

You can combine queries using logical operators:

  • For $or and $and, the syntax is { $op: [query1, query2, ...] }.
  • For $not, the syntax is { $not: query }
Planet.find({ $or: [{ planet: 'Earth' }, { planet: 'Mars' }] }, function (err, docs) {
  // docs contains Earth and Mars
});

Planet.find({ $not: { planet: 'Earth' } }, function (err, docs) {
  // docs contains Mars, Jupiter, Omicron Persei 8
});

// You can mix normal queries, comparison queries and logical operators
Planet.find({ $or: [{ planet: 'Earth' }, { planet: 'Mars' }], inhabited: true }, function (err, docs) {
  // docs contains Earth
});

Sorting and paginating

If you don't specify a callback to find, findOne or count, a Cursor object is returned. You can modify the cursor with sort, skip and limit and then execute it with exec(callback).

var Planet = new LinvoDB("planet", { /* schema, can be empty */ })

var doc1,doc2,doc3,doc4;

Planet.save([
	doc1 = { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false, satellites: ['Phobos', 'Deimos'] },
	doc2 = { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true, humans: { genders: 2, eyes: true } },
	doc3 = { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false },
	doc4 = { _id: 'id4', planet: 'Omicron Persei 8', system: 'futurama', inhabited: true, humans: { genders: 7 } }
], function() { 

// No query used means all results are returned (before the Cursor modifiers)
Planet.find({}).sort({ planet: 1 }).skip(1).limit(2).exec(function (err, docs) {
  // docs is [doc3, doc1]
});

// You can sort in reverse order like this
Planet.find({ system: 'solar' }).sort({ planet: -1 }).exec(function (err, docs) {
  // docs is [doc1, doc3, doc2]
});

// You can sort on one field, then another, and so on like this:
Planet.find({}).sort({ firstField: 1, secondField: -1 }) ...   // You understand how this works!

}); // end of .save

Counting documents

You can use count to count documents. It has the same syntax as find. For example:

// Count all planets in the solar system
Planet.count({ system: 'solar' }, function (err, count) {
  // count equals to 3
});

// Count all documents via cursor
Planet.find({}).count(function (err, count) {
  // count equals to 4
});

Map / Reduce / Filter / Aggregate

Besides the standard pagination and sorting Cursor methods, we have the filter, map and reduce modifiers. Before seeing the examples, you should know that you can combine any of these modifiers in any order/way and all will be executed. For example, you can run a regular query with .find and then run a reduce on it. No matter how you combine those modifiers, the order of execution is: query, filter, sort, limit/skip, map, reduce, aggregate.

The basic syntax is:

Cursor.map(function(val){ return val })

Cursor.reduce(function reducer(a,b), initial);

Cursor.filter(function(val) { return true /* or false*/ }); // truthy / falsy values accepted

Cursor.aggregate(function(res) { /* do something to the result of the query right before serving */ return res })

// Let's assume this dataset
var Planet = new LinvoDB("planet", { /* schema, can be empty */ })
Planet.save([
	doc1 = { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false, satellites: ['Phobos', 'Deimos'] },
	doc2 = { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true, humans: { genders: 2, eyes: true } },
	doc3 = { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false },
	doc4 = { _id: 'id4', planet: 'Omicron Persei 8', system: 'futurama', inhabited: true, humans: { genders: 7 } }
], function() { 

// Map/reduce capabilities
Planet.find({ system: 'solar' }).sort({ planet: 1 })
.map(function(x) { return x.planet })
.reduce(function(a, b) { return a+", "+b }, "")
.exec(function(err, res) { 
	// res is "Earth, Jupiter, Mars"
});

// The largest number of human genders
Planet.find({ "humans.genders": { $exists: true } })
.map(function(x) { return x.humans.genders })
.reduce(function(a,b) { return Math.max(a,b) }, 0)
.exec(function(err,res) { 
	// res is 7
});

// Combine map and filter only
// As you can see, you can use map if you want to project only a part of the data
Planet.find({})
.filter(function(x){ return x.planet.length > 5 })
.map(function(x){ return { planet: x.planet } })
.exec(function(err,res) { 
	// res is [{ planet: 'Jupiter' }, { planet: 'Omicron Persei 8' }]
});

// Use aggregate to emulate count
Planet.find({}).aggregate(function(res){ return res.length }).exec(function(err,res) { 
	// res is 4
});

// Combine all the methods, because we can
Planet.find({ system: "solar" }) // we have Mars, Earth, Jupiter remaining
.sort({ inhabited: 1 })
.limit(2) // Earth falls out, we have Mars, Jupiter
.filter(function(x){ return x.planet.length > 5 }) // only Jupiter remains
.map(function(x) { return x.planet })
.reduce(function(a,b) { return a+" "+b }, "planets are:")
.aggregate(function(res) { return res+", those are uninhabited and in the solar system, with a long name" })
.exec(function(err,res){ console.log(res) }); // "planets are: Jupiter, those are uninhabited and in the solar system, with a long name"

}); // end .save()

Live Queries

Once you have a Cursor object, returned by calling find without a callback, you can turn it into a live query, meaning the .res property will always be up-to-date results from the query. Of course, all modifiers, such as limit, skip, sort, map, reduce, filter and aggregate will still apply.

An event will be emitted when the result is updated - liveQueryUpdate on the model itself.

Seriously consider if live queries can be utilized in your application - if you need particular results continuously, using live queries is extremely efficient, since you don't have to re-read the database but results are kept up-to-date as you update the documents.

// Let's assume this dataset
var Planet = new LinvoDB("planet", { /* schema, can be empty */ })
Planet.save([
	{ _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false, satellites: ['Phobos', 'Deimos'] },
	{ _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true, humans: { genders: 2, eyes: true } },
	{ _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false },
	{ _id: 'id4', planet: 'Omicron Persei 8', system: 'futurama', inhabited: true, humans: { genders: 7 } }
], function(err, docs) { 

var live = Planet.find({ system: "solar" }).sort({ inhabited: -1 }).limit(2).live(); // don't forget you can still use map, reduce, filter, aggregate

Planet.on("liveQueryUpdate", function() { 
	// we'll log this twice, once on the initial result and then again once we update document
	// we won't get this emitted if we modify the dataset with documents that do not fit the query ({ system: solar })
	console.log(live.res);
});

setTimeout(function() {
	docs[1].inhabited = false; // Earth catastrophe 
	docs[1].save(); // Save Earth
}, 666);

}); // end .save()

Angular Disclaimer

If you plan to use Live Queries with AngularJS and update scope on the liveQueryUpdated event please be careful. First, I recommend using $digest when possible instead of $apply (dirty-check only the current scope). Second, I recommend debouncing the event before running the $scope.$apply() event to avoid $apply being called many times because of heavy DB use at a moment.

Updating

Re-saving a document

doc.save() - you can use save on a document instance to re-save it, therefore updating it.

// Let's use the same example collection as in the "finding document" part
// { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false }
// { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true }
// { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false }
// { _id: 'id4', planet: 'Omicron Persia 8', system: 'futurama', inhabited: true }

Planet.findOne({ planet: 'Earth' }, function(err, doc) {
	doc.inhabited = false;
	doc.save(function(err) { /* we have updated the Earth doc */ }); 
});

Atomic updating

Doc.update(query, update, options, callback) will update all documents matching query according to the update rules:

  • query is the same kind of finding query you use with find and findOne
  • update specifies how the documents should be modified. It is either a new document or a set of modifiers (you cannot use both together, it doesn't make sense!)
    • A new document will replace the matched docs
    • The modifiers create the fields they need to modify if they don't exist, and you can apply them to subdocs. Available field modifiers are $set to change a field's value, $unset to delete a field and $inc to increment a field's value. To work on arrays, you have $push, $pop, $addToSet, $pull, and the special $each. See examples below for the syntax.
  • options is an object with two possible parameters
    • multi (defaults to false) which allows the modification of several documents if set to true
    • upsert (defaults to false) if you want to insert a new document corresponding to the update rules if your query doesn't match anything. If your update is a simple object with no modifiers, it is the inserted document. In the other case, the query is stripped from all operator recursively, and the update is applied to it.
  • callback (optional) signature: err, numReplaced, newDoc
    • numReplaced is the number of documents replaced
    • newDoc is the created document if the upsert mode was chosen and a document was inserted

Note: you can't change a document's _id.

// Let's use the same example collection as in the "finding document" part
// { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false }
// { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true }
// { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false }
// { _id: 'id4', planet: 'Omicron Persia 8', system: 'futurama', inhabited: true }

// Replace a document by another
Planet.update({ planet: 'Jupiter' }, { planet: 'Pluton'}, {}, function (err, numReplaced) {
  // numReplaced = 1
  // The doc #3 has been replaced by { _id: 'id3', planet: 'Pluton' }
  // Note that the _id is kept unchanged, and the document has been replaced
  // (the 'system' and inhabited fields are not here anymore)
});

// Set an existing field's value
Planet.update({ system: 'solar' }, { $set: { system: 'solar system' } }, { multi: true }, function (err, numReplaced) {
  // numReplaced = 3
  // Field 'system' on Mars, Earth, Jupiter now has value 'solar system'
});

// Setting the value of a non-existing field in a subdocument by using the dot-notation
Planet.update({ planet: 'Mars' }, { $set: { "data.satellites": 2, "data.red": true } }, {}, function () {
  // Mars document now is { _id: 'id1', system: 'solar', inhabited: false
  //                      , data: { satellites: 2, red: true }
  //                      }
  // Not that to set fields in subdocuments, you HAVE to use dot-notation
  // Using object-notation will just replace the top-level field
  Planet.update({ planet: 'Mars' }, { $set: { data: { satellites: 3 } } }, {}, function () {
    // Mars document now is { _id: 'id1', system: 'solar', inhabited: false
    //                      , data: { satellites: 3 }
    //                      }
    // You lost the "data.red" field which is probably not the intended behavior
  });
});

// Deleting a field
Planet.update({ planet: 'Mars' }, { $unset: { planet: true } }, {}, function () {
  // Now the document for Mars doesn't contain the planet field
  // You can unset nested fields with the dot notation of course
});

// Upserting a document
Planet.update({ planet: 'Pluton' }, { planet: 'Pluton', inhabited: false }, { upsert: true }, function (err, numReplaced, upsert) {
  // numReplaced = 1, upsert = { _id: 'id5', planet: 'Pluton', inhabited: false }
  // A new document { _id: 'id5', planet: 'Pluton', inhabited: false } has been added to the collection
});

// If you upsert with a modifier, the upserted doc is the query modified by the modifier
// This is simpler than it sounds :)
Planet.update({ planet: 'Pluton' }, { $inc: { distance: 38 } }, { upsert: true }, function () {
  // A new document { _id: 'id5', planet: 'Pluton', distance: 38 } has been added to the collection  
});

// If we insert a new document { _id: 'id6', fruits: ['apple', 'orange', 'pear'] } in the collection,
// let's see how we can modify the array field atomically

// $push inserts new elements at the end of the array
Planet.update({ _id: 'id6' }, { $push: { fruits: 'banana' } }, {}, function () {
  // Now the fruits array is ['apple', 'orange', 'pear', 'banana']
});

// $pop removes an element from the end (if used with 1) or the front (if used with -1) of the array
Planet.update({ _id: 'id6' }, { $pop: { fruits: 1 } }, {}, function () {
  // Now the fruits array is ['apple', 'orange']
  // With { $pop: { fruits: -1 } }, it would have been ['orange', 'pear']
});

// $addToSet adds an element to an array only if it isn't already in it
// Equality is deep-checked (i.e. $addToSet will not insert an object in an array already containing the same object)
// Note that it doesn't check whether the array contained duplicates before or not
Planet.update({ _id: 'id6' }, { $addToSet: { fruits: 'apple' } }, {}, function () {
  // The fruits array didn't change
  // If we had used a fruit not in the array, e.g. 'banana', it would have been added to the array
});

// $pull removes all values matching a value or even any query from the array
Planet.update({ _id: 'id6' }, { $pull: { fruits: 'apple' } }, {}, function () {
  // Now the fruits array is ['orange', 'pear']
});
Planet.update({ _id: 'id6' }, { $pull: { fruits: $in: ['apple', 'pear'] } }, {}, function () {
  // Now the fruits array is ['orange']
});



// $each can be used to $push or $addToSet multiple values at once
// This example works the same way with $addToSet
Planet.update({ _id: 'id6' }, { $push: { fruits: {$each: ['banana', 'orange'] } } }, {}, function () {
  // Now the fruits array is ['apple', 'orange', 'pear', 'banana', 'orange']
});

Removing

Removing a document instance

// if you have the document instance at hand, you can just
Doc.findOne({ planet: 'Mars' }, function(err, doc) {
	doc.remove(function() {
		// done
	});	
});

Removing from the collection

Doc.remove(query, options, callback) will remove all documents matching query according to options

  • query is the same as the ones used for finding and updating
  • options only one option for now: multi which allows the removal of multiple documents if set to true. Default is false
  • callback is optional, signature: err, numRemoved
// Let's use the same example collection as in the "finding document" part
// { _id: 'id1', planet: 'Mars', system: 'solar', inhabited: false }
// { _id: 'id2', planet: 'Earth', system: 'solar', inhabited: true }
// { _id: 'id3', planet: 'Jupiter', system: 'solar', inhabited: false }
// { _id: 'id4', planet: 'Omicron Persia 8', system: 'futurama', inhabited: true }

// Remove one document from the collection
// options set to {} since the default for multi is false
Planet.remove({ _id: 'id2' }, {}, function (err, numRemoved) {
  // numRemoved = 1
});

// Remove multiple documents
Planet.remove({ system: 'solar' }, { multi: true }, function (err, numRemoved) {
  // numRemoved = 3, all planets from the solar system were removed
});

Events

// Hook-like
Doc.on('save', function(doc) { }) // Will be called before saving a document - no matter if using save, insert or update methods. You can modify the document in this event, it's essentially a hook
Doc.on('insert', function(doc) { }) // Will be called before saving a new document - again, no matter if using save/insert/update methods. You can modify the document in this event
Doc.on('remove', function(doc) { }) // Before removing a document; called with the document about to be removed

Doc.on('construct', function(doc) { }) // When a document is constructed

// After operation is complete
Doc.on('inserted', function(docs) { }) // Called after inserting new documents is complete; docs is an array of documents
Doc.on('updated', function(docs) { }) // Called after updating documents is complete; docs is an array of documents
Doc.on('removed', function(ids) { }) // Called after removing documents is complete; ids is an array of ids

Schemas

You can define a schema for a model, allowing you to enforce certain properties to types (String, Number, Date), set defaults and also define properties with getter/setter. Since schema support is implemented deep in LinvoDB, you can query on fields which are getter/setter-based and rely that types/defaults are always going to be enforced.

NOTE: when constructing a model with a schema, please specify options object after the schema, otherwise schema will be treated as options: new LinvoDB(name, schema, options)

Schemas are defined as an object of specs for each property. The spec can have properties:

  • type - the type to be enforced, can be String, Number, Date along with "string", "number", "date" alternative syntax. Can also be a RegExp instance in case you want to validate against that expression.
  • default - the default value; must comply to the type obviously
  • enumerable - whether this property will be enumerable
  • get - getter, cannot be used with type/default
  • set - setter, cannot be used with type/default
  • index, sparse, unique - booleans, whether to create an index and it's options

If type is all you need, you can shorthand the property to the type only, e.g. { name: String }. You can also define a property as an "array of" by setting it to [spec], for example [String] for an array of strings. Nested objects are supported.

var Person = new LinvoDB("person", { 
	name: { type: String, default: "nameless" }, // default value
	age: Number, // shorthand to { type: ... }
	created: Date, 
	address: { // nested object
		line1: String,
		line2: String
	},
	department: { type: String, index: true }, // you can use the schema spec to define indexes
	favNumbers: [Number], // array of
	firstName: { get: function() { return this.name.split(" ")[0] } }
}, { });

var p = new Person();
// p is { name: 'nameless', age: 0, created: /* date when created */, address: { line1: "", line2: "" }, favNumbers: [] }

p.name = 23;
// p.name becomes "23"

p.created = "10/23/2004"; 
// p is 23 October 2004, date object

p.favNumbers.push(22);
p.favNumbers.push("42"); // favNumbers will be [22, 42] ; the string will be cast to a number
p.favNumbers.push("forty five"); // nothing happens, can't cast
// p.favNumbers is [22, 42]

p.name = "John Smith"; 
// p.firstName is "John"

p.save(function() { 
	// Person is saved
	// You can even query on virtual properties

	Person.find({ firstName: "John" }, function(err, res) { /* res will be [p] */ });
});

Model - static & instance methods

// var doc = new Doc(); // create a new instance
// Or get it from query results

doc.remove(function(err) { /* removes the document*/ })
doc.save(function(err) { /* saves the document*/ })
doc.copy(); // returns a copy of the document

You can define additional functions for both the model and the document instances.

Planet.static("findAllSolar", function(cb) { return Planet.find({ system: 'solar' }).exec(cb) });
Planet.findAllSolar(function(err,res) {  /* res is all planets in the solar system */  });

Planet.method("findSameSystem", function(cb) { return Planet.find({ system: this.system }).exec(cb) });
Planet.findOne({ planet: 'Earth' }, function(err, doc) {
	doc.findSameSystem(function(err,res) { /* res is all planets in the solar system */ })
});

Indexing

Indexing in LinvoDB is automatic, although you can turn that off ({autoindex: false} in model options, not recommended). Defining indexes, in case you need you enforce a unique constraint, happens with Doc.ensureIndex({ fieldName: "name", unique: true }).

The full syntax is Doc.ensureIndex(options, cb), where callback is optional and get passed an error if any (usually a unique constraint that was violated). ensureIndex can be called when you want, even after some data was inserted, though it's best to call it at application startup. The options are:

  • fieldName (required): name of the field to index. Use the dot notation to index a field in a nested document.
  • unique (optional, defaults to false): enforce field uniqueness. Note that a unique index will raise an error if you try to index two documents for which the field is not defined.
  • sparse (optional, defaults to false): don't index documents for which the field is not defined. Use this option along with "unique" if you want to accept multiple documents for which it is not defined.

You can remove a previously created index with Doc.removeIndex(fieldName, cb).

NOTE compound indexes are currently not supported.

Promises with Bluebird

Even though LinvoDB does not support Promises out-of-the-box, it can easily be made promise-friendly using Bluebird's promisification feature:

var LinvoDB = require("linvodb3");
var Promise = require("bluebird");

var Planet = new LinvoDB('planet', {});

Promise.promisifyAll(Planet.find().__proto__);
// As of this line, LinvoDB APIs now have promise-returning methods with *Async suffix.
// All the callback-based APIs are still there and will work as before.

Planet.find({ system: 'solar' }).limit(10).execAsync().then(function(docs) {
	// use docs somehow
}).catch(function(err) {
	// handle errors
});

// or, if you use ES7 async / await:

try {
	var docs = await Planet.find({ system: 'solar' }).limit(10).execAsync();
	// use docs somehow
} catch (err) {
	// handle errors
}

Utilization

Stremio - LinvoDB was created specifically because NeDB started to behave suboptimally with >300 movie/series metadata objects, which were pretty large. Reduced memory usage from ~500MB to ~100MB. Live queries, schemas and map/reduce helped create a much cleaner codebase.

If you wish to add something here, contact me at [email protected]

License

See License

Help

Pull requests are always welcome. :)

linvodb3's People

Contributors

antarasi avatar apsaarin avatar b1rdex avatar bengl avatar c10h22 avatar danielventurini avatar depeele avatar dependabot[bot] avatar firefly2442 avatar ivshti avatar jamessharp avatar jasonrhodes avatar jonhester avatar kenspirit avatar lfthomaz avatar louischatriot avatar mitsos1os avatar nicoder avatar orlandodemauro avatar paulbjensen avatar pleochism avatar sbruchmann avatar szwacz avatar timshadel avatar vladikoff avatar wolfwings avatar yurysolovyov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

linvodb3's Issues

Why cursor is not an EventEmitter?

I think it may be useful, especially with live queries. For now, update of one live query invokes handlers of the same collection for other live queries. It's not good, because any handler can emit a re-render of different parts of the screen or make some hard computations.

P.S. In my project, I've extended Cursor specially for live queries, added some useful features (like optional binding for angular scope for cleanup and digest) and made it EventEmitter for emitting result event. It works great.

Regex Type not working

Hello,

I have schema with two regex type fields, but I cannot get them to work as expected.
None of them initializes to default and I am not able to save/update them either.
Here is my schema and some sample code:

'use strict';

var LinvoDB = require('linvodb3')
LinvoDB.dbPath = '/valid/path/here'

var schema = {
    name: { type: String, index: true, unique: true },
    text: { type: String, default: '' },
    exec: { type: String, default: '' },
    icon: { type: String, default: '' },
    data: { type: String, default: '_no_data_' },
    type: { type: String, default: '_system_' },
    regex0: { type: RegExp, default: new RegExp() },
    regex1: { type: RegExp, default: new RegExp() }
}

var AppSchema = new LinvoDB('apps', schema, {})

var data = [new AppSchema({
    name: 'Google Search',
    text: 'Search whatever on the net',
    exec: 'netSearch',
    icon: 'google.png',
    type: '_system_',
}), new AppSchema({
    name: 'Github',
    text: 'Search whatever on Github',
    exec: 'githubSearch',
    data: 'https://www.github.com/search?q=',
    icon: 'github.png',
    type: '_web_app_',
    regex0: new RegExp('^github (.*)', 'i')
})]


AppSchema.save( data, ( err ) => {
    console.log( err ) // logs null
})

AppSchema.findOne({ exec: 'netSearch' }, ( err, doc ) => {
    console.log(doc.data) // logs '_no_data_'
    console.log(doc.regex0) // logs undefined
})

AppSchema.findOne({ exec: 'githubSearch' }, ( err, doc ) => {
    console.log(doc.regex0) // logs undefined
    console.log(doc.regex1) // logs undefined
    doc.regex1 = new RegExp('^git (.*)', 'i');
    doc.save(( err, newDoc ) => {
        console.log(newDoc.regex1) // logs undefined
    })
})

doc.save did not populate _id

hi,

im using:
linvodb3 v "3.7.3"

for some reason, doc.save did not saved to ._id (after save event)

var doc = new Doc({});
doc['name'] = 'test';
doc.save(function(err) { 
        // Document is saved
        console.log('id: ', doc._id); // return id: undefined
      });

please advice

thanks

Default store is undefined

Hi, i trying to use linvoDB with Electron, but when add this line

LinvoDB.defaults.store = { db: require("medeadown") };

I got output: TypeError: LinvoDB.defaults.store is not a function

In the code LinvoDB.defaults only has autoLoad and autoIndex attributes.

Odd Capitalization Issue

This may be my noobness, but I'm having an odd issue with capitalization. I've created a database where I'm storing the path for each file. That path is like this:

I:\path\to\file\image.png

but when it's getting stored, the drive is lowercase:

i:\path\to\file\image.png

I'm not exactly sure why this is happening and it doesn't really bother me, except that it's not consistent when I'm doing queries. So when I'm iterating over my files to make sure no new ones have been added, it's not finding them cause the drive letters don't match. I know I can just make the first letter lowercase, but not sure what kind of effect that'll have on Unix platforms so I'm just wondering if there's a way to fix this in general?

Thanks for any assistance.

events.js:163 Uncaught Error: Uncaught, unspecified "error" event. (true)

my code:

var LinvoDB = nodeRequire("linvodb3");
var modelName = "doc";
var schema = { }; // Non-strict always, can be left empty
var options = { };

LinvoDB.dbPath = "./db/";

var pickerDB = new LinvoDB('picker', schema, options);

this code can work:

pickerDB.insert({'s':'c'}, function (err, newDoc) {
    dlog('插入成功');
    console.log(newDoc._id);
});

but this code doesn't work:

`

pickerDB.find({  }).sort({ }).exec(function (err, docs) {
        console.log(docs);
        console.log(err);
    });

`

and show error:

image

It seems a problem in leveldown。
but if the leveldown is not properly installed, why can i insert data success .

Store HTML in LimvoDB

I'm looking for a DB storage for an app built on NW.js or Electron, and LinvoDB seems to be suitable.

Long story short: I want to make something like Sublime Text (I don't know exactly how it works, so I will make a guess)
· Creating a new TAB, all content inside is store in localStorage/IndexedDB/...
· When the file needs to be saved, it stores the content in localStorage and then create the file.
· If the file already exists, the localStorage content overwrites it.

You may have question why I'm not using files directly, and it's because the first request: We don't know if the user will save the file, but we don't want to lose all the content if the app is closed.

Searching the web, I find that is bad practice to pass HTML content (is for WYSIWYG editor) through JSON, but I can't think of other solution.. I'd have to use encodeURI and decode when retrieve the data to the #editor or the file saved.

So..
How would you do a «Notepad - Sublime Text» editor using PounchDB, or NeDB, or LimvoDB using Electron/NW.js engine?

I'm using:
· Electron
· Angular
· I don't know yet what DB should I choose.

Thank you!

Failing to call via remote/Electron

Calling this from renderer code:

var db = window.remote.require('linvodb3');
var medea = window.remote.require("medeadown");
var path = window.remote.require('path');
var User = new db('users', {}, { filename: path.join(__dirname, "data") + 'users.db', store: { db: medea }});

Error thrown:

Uncaught Error: db.open is not a function
TypeError: db.open is not a function
at LevelUP.open (/Volumes/Seagate/development/techops-clock/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:116:6)
at new LevelUP (/Volumes/Seagate/development/techops-clock/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:86:8)
at LevelUP (/Volumes/Seagate/development/techops-clock/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:46:12)
at Function.Model.initStore (/Volumes/Seagate/development/techops-clock/node_modules/linvodb3/lib/model.js:79:100)
at new Model (/Volumes/Seagate/development/techops-clock/node_modules/linvodb3/lib/model.js:52:35)
at EventEmitter. (/Volumes/Seagate/development/techops-clock/node_modules/electron-prebuilt/dist/Electron.app/Contents/Resources/atom.asar/browser/lib/rpc-server.js:181:13)
at emitThree (events.js:97:13)
at EventEmitter.emit (events.js:175:7)
at EventEmitter. (/Volumes/Seagate/development/techops-clock/node_modules/electron-prebuilt/dist/Electron.app/Contents/Resources/atom.asar/browser/api/lib/web-contents.js:64:23)
at emitTwo (events.js:87:13)

Have deleted data appeared after the restart the computer

I have to add data and delete data,And the operation was successful.But when I restart the computer after, the first retrieval of these data have been deleted.The second retrieval, has been deleted data is not appear again.It can happen to every time to restart the computer.

Tests failing - rollback updates

It must be either because of a dependency that is updated or due to the commits int he last month.

  3) Database Update If a multi update fails on one document, previous updates should be rolled back:
     Error: timeout of 10000ms exceeded
      at null.<anonymous> (/Users/ivogeorgiev/linvodb3/node_modules/mocha/lib/runnable.js:158:14)
      at Timer.listOnTimeout (timers.js:110:15)

  4) Database Update If an index constraint is violated by an update, all changes should be rolled back:
     Error: timeout of 10000ms exceeded
      at null.<anonymous> (/Users/ivogeorgiev/linvodb3/node_modules/mocha/lib/runnable.js:158:14)
      at Timer.listOnTimeout (timers.js:110:15)

  5) Database Update Updates are atomic:
     AssertionError: expected null to equal undefined


  6) Database Using indexes Updating indexes upon document update Updating docs still works as before with indexing:
     AssertionError: expected null to equal undefined

deleted document seems to randomly returned

Hi,
sorry to disturb again :-D

ive a scripts that clean up all data,
and query for existing data and check to makesure data returned 0.
and reinsert data from a csv file.
when i do a regex search, it seems to randomly return data together with deleted data.
it happen 50% of the time. im using this with nodewebkit,
and it seems to happen when i refreshed the nodewebkit browser with the exact same query.

var LinvoDB = require("linvodb3");
var modelName = "doc";
var schema = { }; // Non-strict always, can be left empty
var options = { filename: 'data/toc.db', store: {db: require('medeadown')} };
var Doc = new LinvoDB(modelName, schema, options); // New model; Doc is the constructor

Doc.remove({}, {multi: true}, function (err, rows) {
  if(err) {
    throw err;
  } else {
    console.log('removed: '+rows +' rows.');
    check_exists_docs();
  }
});

function check_exists_docs() {
  Doc.find({}, function(err, res) {
    console.log('should be empty rows:', res.length);
    res.length.should.eql(0);
    do_import(); //does the insertion
  });
}

when i deleted the data directory and reinsert,
its working fine.
i suspect the deleted docs seems to be returned.

the more time i import, the more rows it return.

protobuf support

Enough said, support protobufs as document format. In that case we have to have strict schemas.

Capped collections

LinvoDB needs to support capped collections, like MongoDB

With the events system, this shouldn't be too hard to implement as an add-on module.
The logic will be:

  1. when new documents are inserted (listen to the inserted event
  2. lookup an index we'll be capping on directly (.indexes[idxName]. ...) - e.g. an index by mtime or ctime
  3. take the documents of this index which are not in the first N (cap size) and remove them with Doc.remove()

I can't get nw.js to load the code example database

I get the error below every time I open my app in nw.js, I am using the code right from example page. Am I missing something or is this a issue with nw.js? Thank you!

Uncaught node.js Error

OpenError: doc.db: No such file or directory
at eval (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.XLDg8g/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:118:34)
at eval (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.XLDg8g/node_modules/medeadown/medeadown.js:28:14)
at eval (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.XLDg8g/node_modules/medeadown/node_modules/leveldown-open/leveldown-open.js:15:18)
at Object.cb as oncomplete

This is the code:

var LinvoDB = require("linvodb3");
LinvoDB.defaults.store = { db: require("medeadown") }; // Comment out to use LevelDB instead of Medea

var Doc = new LinvoDB("doc", { /* schema, can be empty */ })

// Construct a single document and then save it

var doc = new Doc({ a: 5, now: new Date(), test: "this is a string" });
doc.b = 13; // you can modify the doc
doc.save(function(err) {
// Document is saved
console.log(doc._id);
});

// Insert document(s)
// you can use the .insert method to insert one or more documents
Doc.insert({ a: 3 }, function (err, newDoc) {
console.log(newDoc._id);
});
Doc.insert([{ a: 3 }, { a: 42 }], function (err, newDocs) {
// Two documents were inserted in the database
// newDocs is an array with these documents, augmented with their _id

// If there's an unique constraint on 'a', this will fail, and no changes will be made to the DB
// err is a 'uniqueViolated' error

});

// Save document(s)
// save is like an insert, except it allows saving existing document too
Doc.save([ doc, { a: 55, test: ".save is handy" } ], function(err, docs) {
// docs[0] is doc
// docs[1] is newly-inserted document with a=55 and has an assigned _id

// Doing that with .insert would throw an uniqueViolated error for _id on doc, because it assumes all documents are new

});

How to stop an event listener?

I'm listening to some data using

MyModel.on('inserted', function(docs){
    console.log(`inserted docs`, docs);
});

How do I stop listening to this static handler?

Repopulating data after inserts

On README.md, it says

Doc.insert([{ a: 3 }, { a: 42 }], function (err, newDocs) {
    // Two documents were inserted in the database
    // newDocs is an array with these documents, augmented with their _id

    // If there's an unique constraint on 'a', this will fail, and no changes will be made to the DB
    // err is a 'uniqueViolated' error

    console.log(newDocs);

    // Sample result: 
    // Document {a: 3, _id: "QT4Lcq1EWSFU8Ug2"}    
});

But newDocs doesn't seem to return an array of documents, it only returns the first inserted element.

At line 282: https://github.com/Ivshti/linvodb3/blob/master/lib/model.js#L282

PS:
This will return an array of inserted documents.

Doc.on('inserted', function (docs) {

});

[Feature Request] Importing and Exporting

I don't seem to see this feature in the API docs. (Please correct me if I am wrong. :D)

I am constantly working with some data (in csv format) with around 20k~100k rows per file and I would like to insert them into persistent data source using linvodb3 in my Electron App via drag and drop.
I was wondering if there is such an API to do importing and exporting?

It would be great if we can also output it back to things like tsv, csv, xml, etc.

Or should I stick with batch insertion instead? (At least for importing)

Preloading db

Is it possible to preload db before quering? Like netdb.loadDatabase?

Should Model.reload() be called when initializing an application with an existing database?

I'm using linvodb3 with medea. If I run this script:

MyModel.insert({ _id: "foo" }, function(err, doc) {
    console.log(err, doc);
});
MyModel.insert({ _id: "foo" }, function(err, doc) {
    console.log(err, doc);
});

The first call succeeds and inserts the document. The second call generates an error, as expected. But then if I re-run the script, after the database already exists on disk, the exact same sequence occurs.

When running the script a second time, I'd expect the first call to fail, since a document with _id: "foo" had been inserted on the previous run. But that doesn't happen.

If I call MyModel.reload(function() { }) at the beginning of this test script, then the first insert does fail, as expected.

The reload() method isn't mentioned anywhere in the documentation. I only happened to see it being used in db.test.js.

Is the app expected to call reload() every time it's launched? I would've expected the index to be saved to disk and automatically reloaded when the model is created.

What can stop end user to directly edit database files?

For example there is some application that uses linvodb, what stops end user to edit files directly with text editor? or is there some way to eliminate this case? Generally im talking about nwjs application, app will be shipped with /db folder where all database files will be. Any ideas how this can be stopped?

error in readme.md

In example code readme.md:

Doc.insert([{ a: 3 }, { a: 42 }], function (err, newDocs) {
    // Two documents were inserted in the database
    // newDocs is an array with these documents, augmented with their _id
    // my additions
    console.log(typeof newDocs) // object
    console.log(newDocs); //Document { a: 3, _id: 'kzWdZ3K0O4KSDV3z' }
});

In Insert callback function 2nd parametr isn't Array. It's 1st insered Object.

upd: dublicate issue #20

Creating Schemas with nw.js causes errors.

This is the error I get when I open:
LevelUPError: Could not locate LevelDOWN, try npm install level down

So I install Leveldown and this is the error I get:
LevelUPError: Installed version of LevelDOWN (1.3.0) does not match required version (~0.10.0)

----------- ERROR without LevelDOWN installed

LevelUPError: Could not locate LevelDOWN, try npm install leveldown
at getLevelDOWN (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.MDZ5rk/node_modules/linvodb3/node_modules/levelup/lib/util.js:55:11)
at LevelUP.open (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.MDZ5rk/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:113:37)
at new LevelUP (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.MDZ5rk/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:86:8)
at LevelUP (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.MDZ5rk/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:46:12)
at Function.Model.initStore (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.MDZ5rk/node_modules/linvodb3/lib/model.js:79:100)
at new Model (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.MDZ5rk/node_modules/linvodb3/lib/model.js:52:35)
at eval (file:///var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.MDZ5rk/main.js:16:14)

----------- ERROR with LevelDOWN installed

LevelUPError: Installed version of LevelDOWN (1.3.0) does not match required version (~0.10.0)
at getLevelDOWN (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.G62FcM/node_modules/linvodb3/node_modules/levelup/lib/util.js:59:11)
at LevelUP.open (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.G62FcM/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:113:37)
at new LevelUP (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.G62FcM/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:86:8)
at LevelUP (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.G62FcM/node_modules/linvodb3/node_modules/levelup/lib/levelup.js:46:12)
at Function.Model.initStore (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.G62FcM/node_modules/linvodb3/lib/model.js:79:100)
at new Model (/private/var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.G62FcM/node_modules/linvodb3/lib/model.js:52:35)
at eval (file:///var/folders/ff/n5r157j92tdc6322388y_2th0000gn/T/.org.chromium.Chromium.G62FcM/main.js:16:14)

----------- CODE I'M RUNNIG

var os = require('os');
var LinvoDB = require("linvodb3");
var modelName = "persons";
var schema = { }; // Non-strict always, can be left empty

options.filename = "/temp"; // Path to database - not necessary
options.store = { db: require("medeadown") }; // Options passed to LevelUP constructor

var Person = new LinvoDB("person", {
name: { type: String, default: "nameless" }, // default value
age: Number, // shorthand to { type: ... }
created: Date,
address: { // nested object
line1: String,
line2: String
}
}, { });

koa

Can it will be support koa like mongoose!

Multiple Stores

Is it possible to have multiple data stores in Linvodb like NeDB? How would you go about selecting the different data stores?

I guess you could just use different file names for different stores or is there a better option?

When I run the update operation, the error occurred: Cannot redefine property

This my schema

var User = new LinvoDB('user',{
id:{type:Number,index:true,unique:true},
user:{
    id:{type:Number,index:true,unique:true},
    username:{type:String,index:true,unique: true},
    userLabelName:String,
    portraitUrl:String,
    email:String,
    phoneNumber:String,
    birthday:String,
    firstName:String
}
},{});

module.exports = User;

when I execute below code:

UserDoc.update({id:userprofile.id},userprofile,{upsert:true},function(){});

Uncaught Exception:
TypeError: Cannot redefine property: email
at Function.defineProperty (native)
at E:\electron\app\app\node_modules\linvodb3\lib\schemas.js:70:10
at forOwn (E:\electron\app\app\node_modules\linvodb3\node_modules\lodash\dist\lodash.js:2106:15)
at Function.forEach (E:\electron\app\app\node_modules\linvodb3\node_modules\lodash\dist\lodash.js:3303:9)
at construct (E:\electron\app\app\node_modules\linvodb3\lib\schemas.js:58:4)
at E:\electron\app\app\node_modules\linvodb3\lib\schemas.js:68:20
at forOwn (E:\electron\app\app\node_modules\linvodb3\node_modules\lodash\dist\lodash.js:2106:15)
at Function.forEach (E:\electron\app\app\node_modules\linvodb3\node_modules\lodash\dist\lodash.js:3303:9)
at Object.construct (E:\electron\app\app\node_modules\linvodb3\lib\schemas.js:58:4)
at Document (E:\electron\app\app\node_modules\linvodb3\lib\document.js:799:11)

EMFILE: too many open files

I am getting this error on mac osx 10.9.5 when trying to open my database.

Uncaught OpenError: EMFILE: too many open files, open '/Users/bjorn/Library/Application Support/xxxxxx/data/sales_documents.db/38.medea.hint'

I have been trying to increase the max open files with ulimit -n 2048 with no success.

Im not sure if this is caused by the LinvoDB module or medeadown.

Database files not deleting?

Hello! I have a small issue with file deletions in my application. Basically, it seems that the database only unlinks(?) the entries from the database but the files remain in the computer. Sometimes deleted entries seem to be reindexed randomly on refresh or startup.

Opening db

let LinvoDB = require("linvodb3");
LinvoDB.defaults.store = { db: require("medeadown") };
LinvoDB.dbPath = process.cwd();
let Doc = new LinvoDB("doc", { text: String, time: Date});
let doc = new Doc();

This is the command I use to delete everything:

    Doc.remove({ }, { multi: true }, function (err, numRemoved) {
      console.log("Removed: " + numRemoved);
    });

This is how I delete an entry:

 Doc.remove({ _id: id }, {}, function (err, numRemoved) {
      doc.save(function(err) {
        if (err) {
          console.error("Delete error:" + err);
        }
      });
      console.log("This is the item's ID that will be deleted: " + id);
    });

This is how I add an entry

Doc.update({ _id: id }, { $set: { text: text } }, function (err, numReplaced) {
      doc.save(function(err) { /* saving the document */ });
});
  };

This is how I populate on startup

 Doc.find({}).sort({time: 1})
  .filter(e => (e.text !== undefined))
  .map(res => {populateTable(res.text, moment.unix(res.time).format($("#time").val()), res._id);})
  .exec(function(err, res) {
  });

Sorry if I pasted too much stuff in here. If it's not an actual issue with Linvodb I'd greatly appreciate some support! It's the biggest bug I currently have in my app and can't seem to get the hang of it...

Thank you!

why no one want a promise version

good job dude. and it is really very fast, more than mongodb.
and I think most people don't like callback, so why we don't have a promise support?

electron v1.2.3 node.js v6.10.0 can't save doc

The problem is:

With electron v0.37.8 and node.js v5.7, linvodb3 work normal, but when I updated electron 1.2.3 (with node v6.10), linvodb3 can not save data, and no error was throwed.

Close method

Hi

Is there a close method to clean up on shutdown?

Thanks

Adhering to Unique parameter

I'm seeming to have trouble with the unique option. Neither of the below options work when inserting duplicate docs (as in the docs get inserted).

var Movies = new LinvoDB("movies", {
  title: String,
  id: {
    type: String,
    index: true,
    unique: true
  },
}, {});
var Movies = new LinvoDB("movies", {
  title: String,
  id: String
}, {});

Movies.ensureIndex({ fieldName: "id", unique: true })

Is it normal that Doc.find({}) is that slow ?

Hello !

I recently switched from NeDB to LinvoDB3 for Museeks, but I have some trouble with database initial load speed.

I know why NeDB is ultra-fast, and I know Linvo is not meant to be as fast as NeDB, but I got these results (3500 documents):

Track.find({}; (err, res) => {});

Type Size Load speed
Library with covers ≈ 1GB ≈ 20'000ms
Library without covers 2.7MB 2461.152ms

It seems like

  • the size of a document influences load speed a bit
  • number of documents influences load speed a lot

I'm using level-js

Is this the expected behavior/speed ? or have I done something wrong ?
Would levelDB be faster ?

DB cannot be accessed by multiple processes

When I try to connect to the DB with the 2nd app, I get an error:

OpenError: IO error: lock /home/ryan/node/maxscrafts/app/linvodb3/Users/LOCK: Resource temporarily unavailable
at .../node_modules/linvodb3/node_modules/levelup/lib/levelup.js:119:34
at .../node_modules/leveldown/node_modules/abstract-leveldown/abstract-leveldown.js:39:16

code:
new LinvoDB(schema.name, schema.schema, {
filename: path.join(root_dir, "app/linvodb3", schema.name)
}),

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.