Comments (13)
Gossip protocol is definitely the right way to go. As per our discussion, a new node would be provided IP address of an existing node, so it could jump start the protocol.
Few questions/requirements:
- What happens when the entire cluster is brought up together, and we don't have any existing IP address? I suppose rolling start would be required in such a scenario?
- The implementation of Gossip should allow us to pass meta information, like runtime and load statistics about the server, so we can make better judgements about shard reassignment.
- Instead of taking only 1 IP address, take a list so multiple servers could be checked.
We'll also have to determine what our communication protocol would be. I looked at grpc.io, but it's focused around protocol buffers. We're using Flatbuffers, which aren't supported. So, I see no big benefit in going that direction. We should stick to the standard net/rpc package provided by Go. Byte array based request and response should be sufficient, because Flatbuffers.
from dgraph.
Btw, server failure detection isn't really required from Gossip protocol. MultiRaft is going to tackle that part.
from dgraph.
What happens when the entire cluster is brought up together, and we don't have any existing IP address? I suppose rolling start would be required in such a scenario?
No. Each new node will always connect to at least an existing node. And after connection is established, the topology will be shared periodically. So, let's say node A
is the first in the cluster and nodes B
and C
connect simultaneously. A
will tell B
and C
there are no more nodes connected, but after a configurable amount of time, A
will tell B
that C
exists and C
that B
exists. B
and C
will most probably connect to each other and share topologies themselves. If Z
has connected to B
, B
will infect everyone it's connected to, and so on.
The implementation of Gossip should allow us to pass meta information, like runtime and load statistics about the server, so we can make better judgements about shard reassignment.
As I said in the channel, this is definitely one advantage of going with serf
. It provides a cluster messaging facility where you can do this kind of stuff. It also includes network tomography, so one node could eventually know the nearest-K nodes.
We should stick to the standard net/rpc package provided by Go. Byte array based request and response should be sufficient, because Flatbuffers.
I'd say we should stick with whatever allows for any payload format. Narrowing it to net/rcp
may be too much. Also, By default [gRPC] uses protocol buffers, Googleโs mature open source mechanism for serializing structured data (although it can be used with other data formats such as JSON)
.
Btw, server failure detection isn't really required from Gossip protocol. MultiRaft is going to tackle that part.
Node failure is important in terms of cluster membership management. We're not talking about sharding/replication and its consensus here.
from dgraph.
No. Each new node will always connect to at least an existing node.
I think you misunderstood my question. My question is specifically regarding the case when you bring up the cluster the first time, so in your example, bringing up A, B, and C together. In that case, there's no previously existing node; so does that mean, we have to do rolling start?
Regarding grpc, as I mentioned above, it uses protocol buffers, but we use Flatbuffers. I don't want to mix up the two, unless absolutely needed. Which is why I propose using net/rpc
, instead.
Node failure is important in terms of cluster membership management.
I think there's some sort of overlap here between Raft and Gossip, that we should be careful of, design wise. Probably more of a note for me at this point, than for you.
from dgraph.
bringing up A, B, and C together. In that case, there's no previously existing node; so does that mean, we have to do rolling start?
Here's what I said:
So, let's say node A is the first in the cluster and nodes B and C connect simultaneously.
So A is the existing node, while only B and C connect together.
Regarding grpc, as I mentioned above, it uses protocol buffers, but we use Flatbuffers. I don't want to mix up the two, unless absolutely needed. Which is why I propose using net/rpc, instead.
I don't support going with gRPC as well. Neither choosing net/rpc
. Tbh, I want to adopt an existing solution and not implement this from scratch.
from dgraph.
What new technologies would existing solution bring in? How hard would it be to write our own solution with net/rpc
and flatbuffers
?
from dgraph.
What new technologies would existing solution bring in?
Not sure I follow the question, but basically something like serf
would allow for:
- Gossip discovery
- Cluster membership management
- Cluster messaging
- Network tomography
How hard would it be to write our own solution with net/rpc and flatbuffers
The discovery process should be pretty doable. The membership management and messaging would be tricky. The network tomography should be hard.
from dgraph.
I don't support going with gRPC as well. Neither choosing net/rpc.
What I mean is, if you don't use either of these for communication, what do you use? That's what I mean when I ask about the new technologies that your solution would bring in. I'm not asking about the functionality the library would bring in -- that's clear a separate discussion.
from dgraph.
As I said above
I'd say we should stick with whatever allows for any payload format. Narrowing it to net/rcp may be too much.
I just meant that I don't feel like narrowing the messaging transport in a decision right now. Deciding to go with net/rpc
may be enough for anyone not picking it just because they we're thinking about something else.
serf
leverages on memberlist
, which uses pure TCP and UDP for messaging exchange.
from dgraph.
memberlist
seems closest to the minimum we need to implement Gossip. But, it lacks the ability to send various stats along with the Gossip pings. To send stats / custom messages, we'd have to call SendTo
, on top of Gossip periodically; which will increase the amount of communication required unnecessarily.
I think it seems like writing our own bare bones Gossip impl would give us the right customizability, without any of the unneeded features at this point at the extra cost they might come in.
from dgraph.
Why not Raft Protocol ? https://github.com/hashicorp/raft
Like Influxdb does.
from dgraph.
Raft is only meant for consensus, not discovery or cluster management.
However, serf and memberlist use that Raft implementation.
On Jan 19, 2016 20:13, "Joao Martins" [email protected] wrote:
Why not Raft Protocol ? https://github.com/hashicorp/raft
Like Influxdb does.โ
Reply to this email directly or view it on GitHub
#12 (comment).
from dgraph.
We're using RAFT only. When a new node wants to join, it can talk to any member of the cluster. That member can convey the master for the entire cluster. The new node then talks to the master, and joins the cluster. This is sufficient for now.
from dgraph.
Related Issues (20)
- [FEATURE]: @custom directive to make batch requests to services that accept them
- [QUESTION]: <Snapshot Isolation Testing> HOT 20
- [QUESTION]: IDE for DQL HOT 2
- [QUESTION]: data writing is slow HOT 2
- [BUG]: Online Restore graphql API has no interface to get status
- [ENHANCEMENT]: Simplify Administrative Operations with REST API
- [ENHANCEMENT]: Support case-insensitive sorting of strings HOT 1
- [QUESTION]: full text search - Polish lang support HOT 1
- [ENHANCEMENT]: Sorting using facets
- [BUG]: crash gracefully when dgraph binary version is incompatible with data on disk HOT 3
- [How to use regexp in @filter]: <regexp use> HOT 3
- dgraph performance
- Query profiler: Add an ability to get performance statistics of queries
- [BUG]: Dgraph.Allow-Origin CORS setting does not work as docs claim
- [QUESTION]: Can i write some extension for db
- Can you recommend several graphic libraries that display graph data on the web or Flutter side?
- [BUG]: When build oss degraph alpha exit with error : flags: acl / encryption is an enterprise-only feature
- Cancel schema and improve it to automatically generate schema, modify schema, and update schema based on data without human intervention, and ensure compatibility with historical data. HOT 1
- [Documentation]: <Running dgraph tests on Mac OS> HOT 1
- /run.sh: line 16: dgraph-ratel: command not found HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dgraph.