guille / raftcore Goto Github PK

Hi,

First, thanks a lot for this work.
I am playing with RaftCore and IAPIRaftConnector.

My goal is to host the cluster (+ 1 node) and the API in a ConsoleApp in .NET 6 using OWIN.

Perhaps I am missing something but I can't find the requestvote & appendentries API endpoints.

APIRaftConnector.cs

private async Task<string> SendRequestVote(int term, uint candidateId, int lastLogIndex, int lastLogTerm) {
{
  ...
  var response = await client.PostAsync(baseURL + "requestvote", content); // <=== requestvote
  ...
}

private async Task<string> SendAppendEntries(int term, uint leaderId, int prevLogIndex, int prevLogTerm, List<LogEntry> entries, int leaderCommit)
{
  ...
  var response = await client.PostAsync(baseURL + "appendentries", content); // <=== appendentries
  ...
}

NodesController.cs

[HttpGet("[controller]")]
public IEnumerable<IEnumerable<string>> GetNodesInfo() {
  ...
}

// GET /nodes/log/:elems
[HttpGet("[controller]/log/{elems}")]
public JsonResult GetLogs(int elems) {
  ...
}

// GET /nodes/sm
[HttpGet("[controller]/sm")]
public IEnumerable<string> GetStateMachines() {
  ...
}

// PATCH /nodes
[HttpPatch("[controller]/{id}")]
public string SwitchState(int id) {
  ...
}

// POST /nodes/requests
[HttpPost("[controller]/requests")]
public string MakeRequest() {
  ...
}

Where is my mistake?
Thanks.

After a long time of execution, a replicated Raft node can get massive. The algorithm designers developed a protocol to compact the logs to reduce the space necessary to store and replicate the log data structure.

You can learn more about this feature reading the Raft paper §7

• Test the class APIRaftConnector located inside RaftCore/Connections/Implementations/. Make sure it works properly.

• Make the endpoints configurable.

Don't know if you are still working on this, but I looked at your implementation and found some issues - just a FYI.

A lot of the operations in your implementation are not thread safe.

An example of this is, your heartbeats interval can exceed the time it takes to complete a heartbeat, as a result you can have two "AppendEntries" running concurrently, this causes issues particularly when both threads reach

toApply.ForEach(x => StateMachine.Apply(x.Command));

applying your logs twice.

You should have dedicated threads that loop this check and log application.

One of the most interesting parts about the Raft algorithm is the ability to swap between cluster configurations and sizes. The algorithm offers a protocol to do so safely without losing any data or any of the guarantees it provides. The functionality would allow the administrators of the clusters to replace, add or remove machines from a cluster without having to restart it.

You can learn more about this feature reading the Raft paper §6

The product developed ships with two basic state machine implementations and two basic
connectors.

More state machines could be added to cover more complicated use cases (for example, a state machine that executes entries on a PostgreSQL database).

The same could be done for connectors, supporting configurable APIs instead of a pre-fixed one, or
supporting SOAP services, for example.

Hello,

could you provide me a sample for MyIPConnector please.

Regards

Although the designed implementation can withstand network connectivity issues, it does not offer full support for more severe circumstances, such as application crashes or power outages.

The main solution of this problem is considered in the Raft algorithm paper, and consists of persisting some of the node’s state. This would require adding a protocol to both serialize part of the node’s state and reload this configuration after node crashes. In the developed implementation, all state is volatile.

in RequestVote method,should we compare with term > CurrentTerm and vote true?

if(term>CurrentTerm)
 voteGranted = true;