iterator should be a stream about node-leveldb HOT 11 OPEN

in the meantime i am using this workaround

function getLast(cb) {
  db.iterator(function(err, iterator) {
    if (err) return cb(err)
    iterator.last(function(err) {
      if (err) return cb(err)
      iterator.current(function(err, key, val) {
        cb(err, key)
      })
    })
  })
}

then i can use the returned value as the end parameter in forRange

from node-leveldb.

I have further improved upon the above design here: max-mapper/plumbdb@0cf5212

essentially I am using the seek, current and next functions on Iterator wrapped in a readable Stream

from node-leveldb.

I am not opposed to this but do not have much time right now. If anyone wants to take a stab at it that would great.

from node-leveldb.

@maxogden Does the patch supplied resolve your issue? It seems like a different approach to a common problem.

from node-leveldb.

the above patch makes this library more usable but this issue should stay open as a stream would be a more ideal API

from node-leveldb.

So you mean write wrapper that iterates on your behalf and emits data events (ie a stream?). Iterating over key-values in leveldb is FAST. Imagine iterating over every key in a very large database, without back pressure I bet something would break. Run out of ram because there's too many events in the queue? Or perhaps the overhead from crossing the V8->C++ boundary would just grind your app to a screeching halt.

I can understand the desire for a stream interface, it sounds like it make more sense as helper module for people who understand that you shouldn't iterate over a large set with it. This module should match the leveldb C++ interface as much as possible IMHO.

from node-leveldb.

streams support backpressure. I get that it should match the c++ interface but it's a node binding and streams are part of node core

from node-leveldb.

@maxogden, I like your suggestion of exposing a stream. I may take a stab at this. Stay tuned.

from node-leveldb.

👍

from node-leveldb.

Fwiw I've been happy with the stream implementation in rvagg/levelup

Sent from my iPhone

On Mar 7, 2013, at 12:18 PM, Michael Phan-Ba [email protected] wrote:

—
Reply to this email directly or view it on GitHub.

from node-leveldb.

Wow, rvagg/levelup looks like it's really matured, and has a nice little ecosystem. Last I looked, there wasn't an "obvious" winner among the leveldb drivers. @maxogden, in your opinion, is there any reason to resurrect this node-leveldb project, or have you been satisfied with levelup? No need to reinvent the wheel.

from node-leveldb.