nervosnetwork / overlord Goto Github PK

View Code? Open in Web Editor NEW

78.0 13.0 23.0 660 KB

Overlord consensus protocol.

Home Page: https://nervosnetwork.github.io/overlord/

License: MIT License

Rust 99.23% Shell 0.21% Makefile 0.56%

consensus bft rust library cargo muta

overlord's Introduction

Overlord

Overlord consensus protocol.

Introduction

Overlord is a new consensus protocol that decouple the consensus process from the execution process.

Detailed introduction: 中文|English

Usage

From cargo

[dependencies]
overlord = "0.4"

Overlord takes turns to become the leader by default. If you want to choose a leader randomly, add the random_leader feature to the dependency as below.

[dependencies]
overlord = { version = "0.4", features = ["random_leader"] }

Example

We simulated a salon scene to show an example of using overlord.

A distributed system for reaching a consensus on the content of a speech is realized by simulating the dialogue between speakers through the communication between threads.

Run the example by cargo run --example salon, and the system will output the agreed speech content in turn. Click here to see the detail.

It will check whether different speakers agree on the content of the speech.

Projects using Overlord

Muta, a high-performance blockchain framework.
Axon, a layer2 of CKB that is compatible with Ethereum.

overlord's People

Contributors

Stargazers

Watchers

overlord's Issues

TODO List

feat: Wal mod
feat: integration tests
refactor: only next leader will wait for block interval delay
refactor: do commit when receiving a precommitQC that hash is not null
refactor: structured log output

Speakers in salon tried talking to ghost and panicked

The salon example seems broken after the 0.2.0 release. It panics at unwrapping None in transmit_to_relayer to get the sender. I add some logs:

    async fn transmit_to_relayer(
        &self,
        _ctx: Context,
        name: Bytes,
        words: OverlordMsg<Speech>,
    ) -> Result<(), Box<dyn Error + Send>> {
        match self.talk_to.get(&name) {
            Some(to) => to.send(words).unwrap(),
            None => println!("get name {:?} failed, words are: `{:?}`\n", name, words),
        }
        Ok(())
    }

     Running `target\debug\examples\salon.exe`
get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xcf\xdeP\x80\x1c\xe4D\x9f!\x85", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"\xcf\xdeP\x80\x1c\xe4D\x9f!\x85" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xbd\x88\xa6M\x89UN\xf3t/", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"\xbd\x88\xa6M\x89UN\xf3t/" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\x1e\x1b\x97\x07\x1cd2?rP", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"\x1e\x1b\x97\x07\x1cd2?rP" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"f2s\x1d]@\x93\xd1f\x12", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"f2s\x1d]@\x93\xd1f\x12" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xcb\xcdh7JhP\xa0>h", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"\xcb\xcdh7JhP\xa0>h" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xda2\xa1\xa9\xeb\xd3\xd8\x0e\xd23", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"\xda2\xa1\xa9\xeb\xd3\xd8\x0e\xd23" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"N\x8d[\x89q<}ju\xf6", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"N\x8d[\x89q<}ju\xf6" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"#P\x8dk<OZ\x19\x91\xbf", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"#P\x8dk<OZ\x19\x91\xbf" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"h\x88\xbf7\xf82Re\xbe6", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"h\x88\xbf7\xf82Re\xbe6" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xee\xaf\xce\xa5\xb6\xf4\x93\xad\xb4\x1e", vote: Vote { height: 1000, round: 0, vote_type: Prevote, block_hash: b"" }, voter: b"\xee\xaf\xce\xa5\xb6\xf4\x93\xad\xb4\x1e" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xda2\xa1\xa9\xeb\xd3\xd8\x0e\xd23", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"\xda2\xa1\xa9\xeb\xd3\xd8\x0e\xd23" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xbd\x88\xa6M\x89UN\xf3t/", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"\xbd\x88\xa6M\x89UN\xf3t/" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xee\xaf\xce\xa5\xb6\xf4\x93\xad\xb4\x1e", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"\xee\xaf\xce\xa5\xb6\xf4\x93\xad\xb4\x1e" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\x1e\x1b\x97\x07\x1cd2?rP", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"\x1e\x1b\x97\x07\x1cd2?rP" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"N\x8d[\x89q<}ju\xf6", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"N\x8d[\x89q<}ju\xf6" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xcf\xdeP\x80\x1c\xe4D\x9f!\x85", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"\xcf\xdeP\x80\x1c\xe4D\x9f!\x85" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"\xcb\xcdh7JhP\xa0>h", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"\xcb\xcdh7JhP\xa0>h" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"#P\x8dk<OZ\x19\x91\xbf", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"#P\x8dk<OZ\x19\x91\xbf" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"h\x88\xbf7\xf82Re\xbe6", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"h\x88\xbf7\xf82Re\xbe6" })`

get name b"" failed, words are: `SignedVote(SignedVote { signature: b"f2s\x1d]@\x93\xd1f\x12", vote: Vote { height: 1000, round: 0, vote_type: Precommit, block_hash: b"" }, voter: b"f2s\x1d]@\x93\xd1f\x12" })`

In height: 1000, commit with : "d3da64281b8f570be410"
In height: 1001, commit with : "8d9f345ddb0516e515f2"
In height: 1002, commit with : "4f97550acff268401068"
In height: 1003, commit with : "9226f8ce75acb2b7acbe"
In height: 1004, commit with : "e89ea2e506228d76dab4"
In height: 1005, commit with : "1a9045433ae50fb8df22"
In height: 1006, commit with : "980183753e04d11fc05c"
In height: 1007, commit with : "fb4ee4cd8cf5ff7cb089"
In height: 1008, commit with : "42e82237cbc2e533f0fc"
In height: 1009, commit with : "76b1eb7f27b4e300fa6f"
In height: 1010, commit with : "cdb02ac2fd3d33e50181"
In height: 1011, commit with : "b21bb448ee232c8cd6af"
In height: 1012, commit with : "704bf7cbf08a87b72e17"
In height: 1013, commit with : "0c66c4c8c834a25a585d"

It looks like speakers are sending message to a nameless peer.

Since this example code doesn't change much, maybe it's a problem with the overlord?

block interval as shockwave

There are four nodes run in one server, and set block interval as 3s.

When we check the real block interval:

10   "timestamp": "2022-06-16 20:02:30 +00:00",
11  "timestamp": "2022-06-16 20:02:33 +00:00",
12  "timestamp": "2022-06-16 20:02:33 +00:00", 
13  "timestamp": "2022-06-16 20:02:36 +00:00",
14  "timestamp": "2022-06-16 20:02:37 +00:00",

It looks like 3 0 3 1 as shock wave.

For overlord, all node will record consensus cost time.
In next round, leader will sleep a while (block interval - cost) to keep block interval and followers run into consensus workflow directly.

For example real consensus cost is x:
height 1 all nodes run into consensus workflow directly, and all nodes record cost x
height 2 leader will sleep (3-x) and x for real consensus. leader record cost x, and followers record cost 3(when leader sleep, followers waiting for proposal, so it's also cost for followers).
height 3 leader(as follower in previous height) will sleep 0 and x for real consensus.
height 4 repeat height 2

So that's why real block interval looks like shock wave.

As simple as possible:

height  sleep  real-cost
   1       0         x
   2      3-x       x
   3       0         x

As compare, another plan is all node record consensus cost and sleep a while (block interval - cost) in the end of consensus workflow.
As simple as possible:

height  real-cost    sleep
   1         x         3-x
   2         x         3-x
   3         x         3-x

We will found that: sleep time at height 3 should be 3-x.
And it means if the cost of followers in height 2 didn't include the time leader sleep, everything will be OK.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.