KV metadata in API about python-etcd3 HOT 7 OPEN

I have a big experience of working with all projects mentioned above and I think approaches used by these projects are fine (they are solving it's purpose). The only difference with gRPC is that they had implement response object on their own (except python-consul, they simply used dict for that :)) due to protocol nature (kazoo is using binary protocol on very low level and python-etcd and python-consul are working with JSON).

Our situation is better (I think), because grpcio-tools already did a lot of work for us and classes are available for usage. Some of them could be used as it is, others make sense to inherit (and extend) and only in a few cases we need to implement something new.

from python-etcd3.

My current thought is to borrow from kazoo's api design, but make the stat optional. Something like:

>>> etcd.get("/key")
b"some-value"
>>> value, stat = etcd.get("/key", stat=True)

This keeps the basic api simple to use, but allows for access to the extra metadata if it's required. Having a function that changes its return type based of parameters is a little ugly, though.

from python-etcd3.

Having a function that changes its return type based of parameters is a little ugly, though.

Yes, it is very ugly...
And don't forget that together with key metadata we also need cluster-wide metadata (especially ResponseHeader.revision).
If the key hasn't been updated very long time watch on mod_revision will fail because history was already compacted, but if you use ResponseHeader.revision for that it will work fine.
In this sense python-etcd approach looks more sane, but the difference is - we don't have a tree structure anymore, so we will not have a root element and children, but only children (list).
And for the sanity it should remain list even for a single result. The only difference must be - if the single key was requested and it is not there - raise an exception.

from python-etcd3.

Why not simply always return the metadata? Users who don't want it can always indicate that they don't care by saying data, _ = etcd.get('/key').

Alternatively, have two separate functions: get and get_with_metadata. If a user decides they now need the metadata, they can simply change their function call to get it. Both options avoid the need to have return types that change based on context.

from python-etcd3.

@Lukasa you make a good point, it's not much extra effort to just throw away the extra data.

from python-etcd3.

@kragniz, I've noticed that you started implementing it, but you forgot one important thing:

together with key metadata we also need cluster-wide metadata (especially
ResponseHeader.revision). If the key hasn't been updated very long time, watch
on mod_revision will fail because history was already compacted. But, if you use
ResponseHeader.revision for that it will work fine.

I have experience of working with all three libraries mentioned above in the Patroni project. In order to implement everything reliable I had to override some of the methods and classes of original projects, therefore I am really familiar not only with their API but with internals as well.

from python-etcd3.

@CyberDem0n hang on, it's not finished yet!

from python-etcd3.