Handle memory and storage limitations about fitbit-asap HOT 3 CLOSED

Since we are very often reading the queue, we might as well have one global queue variable to reduce memory usage.
At this moment, if there is a large queue, we might be needlessly hitting memory limits before the garbage collector has a chance to clear our scoped queue variables.

Or we might just null the scoped queue variables immediately after use.

from fitbit-asap.

The current version of the code base resolves the memory issue by storing only the message IDs in memory—the messages themselves are stored as individual files on disk.

As far as running out of disk space, this library is already self-limiting via the timeout option. For applications where very high disk usage is likely, I think it makes more sense for this to be managed at the application level.

I'm closing this issue for now.

from fitbit-asap.

Hi @daveyjones, I'm using your library and I'm having similar concerns. I suppose the "timeout" option is not enough to protect against hitting the memory /storage bound. In current source code, this option works to reduce memory overhead when process() is called, but what if the PeerSockect connection is off for a rather long time? Then we would've kept enqueuing (writing to disk) non-stop till we hit the memory bound, which might trigger an exception and potentially crash the app(I'm not sure how Fitbit handles this)? As a developer, I don't want to risk crashing the app or losing newer messages instead of older ones, and nor do I have a callback for calling asap.send() to gain some control over this issue.

Maybe we should call process() more often? e.g. In function enqueue(), instead of calling process() only when queue is previously empty, we call it every time.

from fitbit-asap.