Enahce Cluster metadata algorithm about scalecube-cluster HOT 5 CLOSED

Here the guess was almost correct, with small correction. All gossips spread very fast, all gossips about update come to the remote node sooner that the node 2 completes the first call to fetch metadata.

from scalecube-cluster.

The attempt to avoid the extra calls that is based on incarnationsis as follows: inside metadata we store it's version(incarnation). Any time we receive gossib that notifies about member update - we compare membershipEvent's incarnation to the incarnation for which we have stored metadata.

Make call to fetch metadata if only incarnation in membershipEvent is higher that the one in metadata.

from scalecube-cluster.

D 0008:59,103 i.s.c.m.MetadataStoreImpl Updated metadata: {key=0, incarnation=1} for local member ([email protected]:55103) [sc-cluster-55103-1] D 0008:59,104 i.s.c.m.MetadataStoreImpl Updated metadata: {key=1, incarnation=2} for local member ([email protected]:55103) [sc-cluster-55103-1] D 0008:59,104 i.s.c.m.MetadataStoreImpl Updated metadata: {key=2, incarnation=3} for local member ([email protected]:55103) [sc-cluster-55103-1] D 0008:59,105 i.s.c.m.MetadataStoreImpl Updated metadata: {key=3, incarnation=4} for local member ([email protected]:55103) [sc-cluster-55103-1] D 0008:59,105 i.s.c.m.MetadataStoreImpl Updated metadata: {key=4, incarnation=5} for local member ([email protected]:55103) [sc-cluster-55103-1]

At remote member:
I 0008:59,162 i.s.c.m.MembershipProtocolImpl Member updated: [email protected]:55103(1). Currently stored metadata for it: 0 [sc-cluster-55104-2] D 0008:59,163 i.s.c.m.MetadataStoreImpl Getting metadata for member [email protected]:55103 [sc-cluster-55104-2] I 0008:59,163 i.s.c.m.MembershipProtocolImpl Member updated: [email protected]:55103(2). Currently stored metadata for it: 0 [sc-cluster-55104-2] D 0008:59,164 i.s.c.m.MetadataStoreImpl Getting metadata for member [email protected]:55103 [sc-cluster-55104-2] I 0008:59,164 i.s.c.m.MembershipProtocolImpl Member updated: [email protected]:55103(3). Currently stored metadata for it: 0 [sc-cluster-55104-2] D 0008:59,165 i.s.c.m.MetadataStoreImpl Getting metadata for member [email protected]:55103 [sc-cluster-55104-2]

Here we see that all calls to fetch metadata were made at the moment we had the very old version of metadata (incarnation 0). That's according to the algorithm. But doesn't fix the issue.

from scalecube-cluster.

What might be done to avoid extra calls but maybe doesn't worth it:

Don't notify the cluster about too requent metadata updates. Instead - collect frequent updates in batches (based on some timeframe) and create just one incarnation for the batch of updates.
This will reduce not only number of RPC-s from all the members (but local) in cluster to fetch metadata (one per property update), but also number of member update events.

from scalecube-cluster.

Agreed to close. Thanks for such deep investigation @snripa .

from scalecube-cluster.