A LevelUP plugin to merge multiple ranges into a single sorted stream using a subkey for ordering. Useful when doing range queries against multiple key-based secondary indexes. Supports standard LevelUP stream options and naïve pagination via a skip
option.
npm install level-merged-stream
Given a LevelUP instance db
:
var mergedStream = require('level-merged-stream');
db = mergedStream(db);
As per db.createReadStream()
but with the following additional options:
-
'ranges'
(array): Thestart
/end
ranges as you'd provide tocreateReadStream
. Each range is streamed and merged according to the value privided bysubkey
. Each range must be ordered bysubkey
, i.e. any range prefix must be constant for the entire range for overall ordering to be consistent. -
'subkey'
(function, default: identity): Selects the subkey from the key that is used for ordering the results. -
'comparator'
(function, default: primitive comparison): A subkey comparator function if primitive comparison is insufficient, .e.g. for Buffers or custom keys. -
'dedupe'
(boolean/function, default:false
): Iftrue
, when two or more subkeys are considered equal, only the first result will be streamed. The key and/or value returned could come from any underlying range stream. If a function, it should returntrue
if two keys passed are considered equal for deduplication purposes. -
'skip'
(number, default:0
): The number of results to skip in the merged stream. Useful for pagination when also usinglimit
.
As per db.createKeyStream()
this streams only the keys.
As per db.createValueStream()
this streams only the values.
examples/readme.js
:
var path = require('path');
var level = require('level');
var mergedStream = require('level-merged-stream');
var location = path.join(__dirname, '/.db');
var db = level(location);
db = mergedStream(db);
db.batch()
.put('a1', '1')
.put('b2', '2')
.put('c3', '3')
.put('d4', '4')
.put('a5', '5')
.put('b6', '6')
.put('c7', '7')
.put('d8', '8')
.write(function () {
db.mergedReadStream({
// Only stream the 'a's and 'c's
ranges: [
{ start: 'a', end: 'b' },
{ start: 'c', end: 'd' }
],
// Ignore the first character for sorting
subkey: function (key) {
return key.slice(1);
},
skip: 1,
limit: 2
})
.on('data', console.log)
.on('end', function () {
db.close(function () {
level.destroy(location);
});
});
});
Output:
{ key: 'c3', value: '3' }
{ key: 'a5', value: '5' }
MIT