Comments (5)
Here is my initial design sketch.
We define MultiRaftAdapter so requests to each Raft group are directed properly. For the receiver side this looks ok to me but question is on the sender side.
The text in red is a example of request to send. Here the problem is RaftCore knows it's group id. Ideally, each RaftCore can be ignorant of the group id: the knowledge about the group id is managed outside e.g. in the MultiRaftAdapter.
from lolraft.
One another possible motivation is that this extension will benefit for performance of a normal application (not only sharded like kvs). The application may have two consensus in the cluster in which the two are independent. For example, one log for distributed sequencer and one for application log; they are logically independent. If we have multi-raft distinguished by group_id, we have use gid=1 for the former and gid=2 for the latter.
Now I am willing to do this. But it will be a big change that will take a long long time.
from lolraft.
I would love to use this feature fwiw
from lolraft.
With k8s we can deploy many Raft nodes in separate namespace. (Well, container technology is revolution in distributed computing) This means we don't need to distinguish nodes by their ports any more when they are located in the same physical server.
However, as they are independent Raft nodes they have own Tonic server (and Tokio runtime). Apparently, this consumes memory footprint for each instance. But more importantly, there will be created a hundreds of thread pools and OS scheduler needs to take care of thousands of threads, the overhead will not be negligible in my experience. In conclusion we can't put many (100~) nodes in the same server and the number of shards is restricted therefore.
With this feature in-library Multi-Raft, only one Raft node is put per server and shards coexist in the same Raft node where they are distinguished by 64bits IDs. As a result, we can make 10000~ shards to boost the performance.
from lolraft.
This is a very simple example: There are 3 nodes and 3 shards each 2 replicas.
Imagine we split the key space into more shards (like 10000) each maintaining 2 replicas as well.
Other than the resource problem (memory, too many pooled threads), this incurs massive number of heartbeat. The number of heartbeat per interval is proportional to the number of Raft node so in this case the order of 10k. If the heartbeat interval is 100ms, there will be 100 heartbeats in 1 ms, 1 heartbeat in 10us which involves TCP send/recv and protobuf encode/decode. I think this will be totally useless.
My idea is separate the heartbeat layer from Raft node level to phy. node level.
from lolraft.
Related Issues (20)
- Multi-raft
- Add versioning to membership
- Rework: RaftCore is too monolithic HOT 1
- Support duplicated requests HOT 3
- Load balancer and stable connection HOT 2
- Avoid duplicated response HOT 2
- Multi-Raft: config dissemination HOT 2
- The project name has too low googlability.
- Use cargo-chef in testapp Docker
- Run test in GHA with docker compose
- Bump http-serde to 2.0
- Release v0.10 HOT 1
- Implement reflection HOT 1
- Write doc
- Multiplexed heartbeat HOT 3
- Extend testapp for testing multi-raft
- CI: Cache test artifact
- multi_raft_test with L50 fails
- test: Initialize lanes in parallel
- Include lane_id in the log
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lolraft.