This is a simple distributed in-memory key-value storage which has an HTTP interface and uses hashicorp/raft internally.
I wanted to try to build something with Raft, so I decided to create yet another key-value storage. It will run elections to choose a new leader automatically if the current leader fails. It's very simple in-memory key-value storage which is not meant to be a reliable KV DB and not for production usage of course :)
Useful links:
- hashicorp/raft implementation
- goraft/raft (unmaintained)
- thesecretlivesofdata: raft explanation
- raft.github.io
This project has configured docker containers ready to use. Start:
make runIt will start three docker containers (3 nodes) and you will see something like that
...
node_2_1 | [GIN-debug] Listening and serving HTTP on :8080
node_1_1 | [raft][WARN] raft: AppendEntries to {Voter 10.1.0.102:4002 10.1.0.102:4002} rejected, sending older logs (next: 1)
node_2_1 | [raft][WARN] raft: Failed to get previous log: 4 log not found (last: 0)
node_1_1 | [raft][INFO] raft: pipelining replication to peer {Voter 10.1.0.102:4002 10.1.0.102:4002}
node_3_1 | [raft][DEBUG] raft-net: 10.1.0.103:4003 accepted connection from: 10.1.0.101:49686
node_2_1 | [raft][DEBUG] raft-net: 10.1.0.102:4002 accepted connection from: 10.1.0.101:39628
node_1_1 | [DEBUG] state=Leader leader=10.1.0.101:4001
node_3_1 | [DEBUG] state=Follower leader=10.1.0.101:4001
node_2_1 | [DEBUG] state=Follower leader=10.1.0.101:4001
node_1_1 | [DEBUG] state=Leader leader=10.1.0.101:4001
...
Now you can make requests to set and get keys:
########### Get value
~/ > curl 'http://127.0.0.1:4001/keys/some-key/'
{"value":""} # empty, we don't have anything yet
########### Set value
~/ > curl 'http://127.0.0.1:4001/keys/some-key/' -H 'Content-Type: application/json' -d '{"value": "some-value"}'
{"value":"some-value"} # saved
########### Get value again
~/ > curl 'http://127.0.0.1:4001/keys/some-key/'
{"value":"some-value"} # hooray!Try to stop the leader node, see how they elect a new leader and have fun! :)
API is very simple:
POST /keys/<key>/
Headers:
Content-Type: application/jspn
Request:
{
"value": "some-value"
}
---------------------------------
GET /keys/<key>/
Response:
{
"value": "some-value"
}
docker-compose.yml file contains prepared cluster with three nodes. Basically, they are copies of the image from Dockerfile.
They will be exposed on ports 4001, 4002, 4003.
# we have a full cluster with three nodes in it: leader and two followers
node_1_1 | [DEBUG] state=Leader leader=10.1.0.100:4000
node_2_1 | [DEBUG] state=Follower leader=10.1.0.100:4000
node_3_1 | [DEBUG] state=Follower leader=10.1.0.100:4000
# let's stop the leader
> docker stop raftexample_node_1_1
raftexample_node_1_1 exited with code 137
# and the election process begins
node_3_1 | [raft][INFO] raft: Node at 10.1.0.103:4000 [Candidate] entering Candidate state in term 17
node_3_1 | [raft][INFO] raft: Election won. Tally: 2
node_3_1 | [raft][INFO] raft: Node at 10.1.0.103:4000 [Leader] entering Leader state
node_2_1 | [DEBUG] state=Follower leader=10.1.0.103:4000
node_3_1 | [DEBUG] state=Leader leader=10.1.0.103:4000
...
# now we have a new leader and one follower. Let's start old leader
> docker start raftexample_node_1_1
node_1_1 | [raft][DEBUG] raft-net: 10.1.0.100:4000 accepted connection from: 10.1.0.103:52494
node_1_1 | [DEBUG] state=Follower leader=10.1.0.103:4000
node_2_1 | [DEBUG] state=Follower leader=10.1.0.103:4000
node_3_1 | [DEBUG] state=Leader leader=10.1.0.103:4000
# old leader started and it has follower role now- More tests
- Ability to POST key to any node in the cluster, it will forward request to the leader automatically
- Fuzzing Raft for Fun
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent