pressly / chainstore Goto Github PK

Let's say I have 3 stores: Memstore, BoltDB (async) and s3 (async).

given chain.Put(ctx, key, data)

• Memstore succeeds
• BoltDB fails
• s3 succeeds

What should we do? Should we automatically re-try storing the value into the BoltDB again? What if the DB is down for a bit? Should we have a retry queue per Store?

Or since we have LRU, should we have something like PUT down ON DELETE functionality on chain?

The repo has a .travis.yml file but it seems like it's not working. Maybe we can create a main test using boltdb and mockstore, just like the example, except without s3 and update this travis file to run it.

batched writes are way faster for some data stores, ie. for boltdb: https://gist.github.com/benbjohnson/59b57e3772bfb7a65fbf

Should be able to support both.. current interface is: New(capacity int64, *anotherStore chainstore.Store) .. maybe we should make it New("500MB", store) or New("500 items", store) .. using a simple string?

or multiple .New() methods would do it as well.. .NewValueCapacity() and default .New() is to manage capacity by number of keys

I think PR #22 broke the build and now chainstore can't be installed.

go get -u github.com/pressly/chainstore
# github.com/pressly/chainstore
../../pressly/chainstore/timeout.go:25:7: undefined: cancel
../../pressly/chainstore/timeout.go:26:8: undefined: cancel
../../pressly/chainstore/timeout.go:31:7: undefined: cancel
../../pressly/chainstore/timeout.go:32:8: undefined: cancel
../../pressly/chainstore/timeout.go:37:7: undefined: cancel
../../pressly/chainstore/timeout.go:38:8: undefined: cancel

I really like this Idea from the TODO section, and think it's pretty important.

So I wanted to get the ball rolling and open up discussion as to how it might be implemented.

The current boltdb library was archived but https://github.com/coreos/bbolt is being maintained and is considered as a drop in replacement.
Can you replace to bbolt?

see: http://dave.cheney.net/2014/10/17/functional-options-for-friendly-apis

Each chainstore action should be based on a context.Context request context. The benefit here is a request chain can be cancelled with a timeout or cancel function. I've found in imgry that this can be useful in case some requests take to long to complete and instead the goroutine never finishes.

It will make for a much more robust store.

See pressly/imgry#23

All of the stores clients should use ctxhttp or something.