Git Product home page Git Product logo

camomile's Introduction

Camomile

Camomile is an implementation of the Kademlia protocol done during the course Mobile and distributed computing systems (D7024E) at Luleå University of Technology.

A short project report with system architecture and a design specification can be found here: doc/d7024e_project_kademlia_report.pdf

Project status

Build status Test coverage Camomile docs
Build Status Code Coverage Camomile Documentation

Build Camomile

Download and install Protoc into /usr/local:

curl -sfL -o /tmp/protoc.zip https://github.com/protocolbuffers/protobuf/releases/download/v3.9.1/protoc-3.9.1-linux-x86_64.zip
sudo unzip /tmp/protoc.zip -d /usr/local/  # Dangerous!!!

Get dependencies, generate and build the binaries:

go get -u github.com/golang/protobuf/protoc-gen-go
go generate ./...
go build ./cmd/...

Run as cluster

Build the Docker container:

docker build . -t dhtnode:latest

Start the cluster script:

./bin/run-cluster.sh <num> # Change <num> to the number of nodes to run.

Done!

REST API

Reference

Method Path Form Fields Header Code Description
GET /{key} N/A Origin: {id} 200 OK Retrieves a value by its hash key.
POST / value={value} Location: /{key} 202 Accepted Saves a value in the DHT network.
DELETE /{key} N/A N/A 204 No Content Orders the DHT network to forget a value.

Examples

Save value

ξ curl -iF 'value=ABC, du är mina tankar' 127.0.0.1:8080/
HTTP/1.1 202 Accepted
Location: /bde0e9f6e9d3fabd5bf6849e179f0aee485630f6d5c1c4398517cc1543fb9386
Date: Mon, 07 Oct 2019 13:42:02 GMT
Content-Length: 23
Content-Type: text/plain; charset=utf-8

ABC, du är mina tankar

Retrieve value

ξ curl -i 127.0.0.1:8080/bde0e9f6e9d3fabd5bf6849e179f0aee485630f6d5c1c4398517cc1543fb9386
HTTP/1.1 200 OK
Origin: 3a6b713115697a45658aac4ac5eb1714e6f985cb1826d2b5cc53562e2d490157
Date: Mon, 07 Oct 2019 13:42:44 GMT
Content-Length: 23
Content-Type: text/plain; charset=utf-8

ABC, du är mina tankar

Forget value

ξ curl -iX DELETE 127.0.0.1:8080/bde0e9f6e9d3fabd5bf6849e179f0aee485630f6d5c1c4398517cc1543fb9386
HTTP/1.1 204 No Content
Date: Mon, 07 Oct 2019 13:44:49 GMT
Content-Length: 0

FAQ

Some nodes logs sendto: invalid argument when running the cluster script.

Too many nodes running at the same host may result in a "Neighbour Table Overflow". To fix this, increase the kernels internal ARP cache size.

These settings should be enough for a cluster of 50 nodes:

sysctl -w net.ipv4.neigh.default.gc_interval=3600
sysctl -w net.ipv4.neigh.default.gc_stale_time=3600
sysctl -w net.ipv4.neigh.default.gc_thresh1=1024
sysctl -w net.ipv4.neigh.default.gc_thresh2=4096
sysctl -w net.ipv4.neigh.default.gc_thresh3=8192

camomile's People

Contributors

edvinnn avatar hjortsberg avatar wstrm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

camomile's Issues

Bug: (r *Table).Add doesn't ping on full bucket

2019-10-07-170943_964x388_scrot

I'm not 100% sure how this one would be solved.

We'd get a dependency cycle if the routing table calls any function in the DHT. So this logic would probably be added to the DHT and not (r *Table).Add.

One way to do it:

  1. DHT gets a message from node A.
  2. DHT checks if the bucket is full, with a new route.Table method, i.e.
    (r *Table).Full(id node.ID) bool.
  3. DHT gets a...
    a. true: Go to 4.
    b. false: Add new contact A, and go to 6.
  4. DHT retrieves the contact at the head of the bucket (least recently seen, B), using a new method (r *Table).Head(id node.ID) route.Contact
  5. DHT pings the least recently seen node, B, and if it gets a...
    a. Response: Ignore new contact A.
    b. No response: Add new contact A (evicting node B).
  6. Done.

Implement protobuf

Implement these messages:

message Message { MessageID : Bytes }
message Ping { Message* }
message Pong { Message* }
message Store { Message* Value* }
message FindNode { Message* NodeID : 160 bit }
message NodeInfo { Message* IP : bytes Port : uInt NodeID : bytes }
message FindValue { Message* Key : bytes }
message Value { Message* Key : bytes Value : UTF-8 }

Add documentation draft

The required content of the lab report is described in the Workflow section in the lab assignment text. For example:

  • draft architecture description
  • initial backlog, prioritized with objective to first reach a minimal working solution
  • solution outlines for your prioritized items on the backlog and objectives in the lab assignment

The report shall be uploaded at midnight 2 days before your review slot (so reviewers will have one day to check it). Book your time slot with a supervisor for review. The link for booking a time slot is here:

Construct initial CLI

Example of format usage: ./dhtctl -address <ipv6ip> -put "UTF8 string"

-address: string
    - put: string (value) ret:string (key)
    - get: string (key)    ret:string (value)
    - exit: bool

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.