clockworksoul / smudge Goto Github PK

Currently, this project supports three means of configuring its behavior, none of which are as fleshed out as they could/should be:

1. Using the Golang API proper
2. Environment variables
3. The CLI (which ostensibly exists as a proof-of-concept for the API)

I've been throwing around the idea developing a distinct Smudge server that can provide -- perhaps by running sidecar next to a larger service -- all Smudge capabilities, without the service needing to implement the Go API. The server could then communicate with the Smudge sidecar via some language-agnostic means

This leads me to a couple of questions:

First, is this worth even doing? Is there any use for this?

Second, I don't think it makes sense to support both CLI and environment variables for specifying variables, so I'm leaning towards dropping environment variable support entirely. Does anybody have any opinions about this?

Third, I think this would be best served by creating a separate server project that uses the API. Does anybody have any thoughts about this?

Thanks for your thoughts!

Piece of code at the end of the README creates a listener at port 9999. My question is, where is this code located? On all member nodes? Or a separate server?

Thanks

It is possible that calling AddBroadcastListener from concurrent threads will cause it to lose the registration.
Reason:
https://github.com/clockworksoul/smudge/blob/master/events.go#L36 is not protected.

I'm getting the following error if I try announcing presence on an IPv6 network:

2018-04-23T15:11:21Z |INFO| Announcing presence on [ff02::1]:9998
2018-04-23T15:11:21Z |ERRO| dial udp [ff02::1]:9998: connect: invalid argument

The error occurs here.

I need to validate this issue as it may be caused by my IPv6 stack... I decided to create this issue for reference. It can be assigned to me...

Both got the same descriptions which are OnChange().
Instead, the first one should be description of OnBroadcast().

It is listed in known issues, but is it planned?

In https://github.com/clockworksoul/smudge/blob/master/broadcast.go#L281 - you take a lock to see if the msg is already there, release the lock, and if it isn't (which is likely), take the same lock again.
Since the check you do outside the lock is very cheap, it is better to hold on to the lock and do the work in one shot.

Hi again,

I was building my project with flag -race as someone suggested me.
Suddenly, I found that this warning occurred: https://pastebin.com/Nz7idxhZ

I barely use smudge, not even broadcasting, which you can see from my code: https://github.com/fe1t/blockchain_programming_in_golang/tree/fix_race

Most of the code are in server.go:ConfigServer func(), some other is only in cli.go for customizing logger.

I'm not sure if this will cause the problem.

If you could help me check this, I'll be appreciate and thankful.

This project looks similar to serf and memberlist from hashicorp, yours and serf are based on swim.

In the docker section of the readme there are two images referenced:

Testing the Docker image
You can test Smudge locally using the Docker image. First create a network to use for your Smudge nodes and then add some nodes to the network.

For IPv4 you can use the following commands:

docker network create smudge
docker run -i -t --network smudge --rm clockworksoul/smudge:latest /smudge
# you can add nodes with the following command
docker run -i -t --network smudge --rm clockworksould/smudge:latest /smudge -node 172.20.0.2
To try out Smudge with IPv6 you can use the following commands:

docker network create --ipv6 --subnet fd02:6b8:b010:9020:1::/80 smudge6
docker run -i -t --network smudge6 --rm clockworksoul/smudge:latest /smudge
# you can add nodes with the following command
docker run -i -t --network smudge6 --rm clockworksould/smudge:latest /smudge -node [fd02:6b8:b010:9020:1::2]:9999
Building the binary with the Go compiler

Thanks in advance.

I normally connect to university network and get public IP.

When I'm using Smudge for transmitting data (1000-1200 bytes/data), sometimes broadcast messages are lost or very slow.

So I'm here for an advice. I need Smudge to run and broadcast messages only in my localhost.

Here are some of my code.

func ConfigServer() error {
	port, err := strconv.Atoi(NODE_ID)
	if err != nil {
		return err
	}

	// Set configuration options
	smudge.SetListenIP(net.ParseIP(baseAddress))
	smudge.SetListenPort(port)
	smudge.SetHeartbeatMillis(500)
	smudge.SetMaxBroadcastBytes(2000)
	smudge.SetLogThreshold(smudge.LogOff)
	smudge.SetMulticastEnabled(false)
	smudge.SetClusterName("KU-Coin")

	smudge.AddStatusListener(MyStatusListener{})
	smudge.AddBroadcastListener(MyBroadcastListener{})

	// Add a new remote node. Currently, to join an existing cluster you must
	// add at least one of its healthy member nodes.

	if nodeAddress != knownNodes[0] {
		node, err := smudge.CreateNodeByAddress(knownNodes[0])
		if err == nil {
			smudge.AddNode(node)
		} else {
			return err
		}
	}

	// start the server
	go func() {
		smudge.Begin()
	}()

	// Handle SIGINT and SIGTERM.
	// quit := make(chan os.Signal, 2)
	// signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
	// <-quit
	return nil
}

I've set baseAddress to "127.0.0.1" but still got no luck.

Thanks in advance

The SWIM document (section 4.2, page 6) defines the messages as:

• { Suspect M_j: M_i suspects M_j }
• { Alive M_j: M_l knows M_j is alive }
• { Confirm M_j: M_h declares M_j as faulty }

You are only passing the info of M_j (let's call it the target node) but you are not passing M_i/M_l/M_h (let's call it the source node). It may not seem to be needed but the source node is quite important. For example, when non-trustworthy nodes can be connected to the network. A mechanism for avoiding node impersonation would also be needed for this purpouse. Other examples of uses for the source address and port come to mind like prioritizing sending him an alive message so that the timeout doesn't proc, etc.

I love this implementation and am currently in the process of writing C bindings for it. One question I have is this note in the documentation:

No WAN support: only local-network, private IPs are supported.

Is this still true, despite there existing the ability to enter an IP (which can very well be public?) If so, what's restricting this program from supporting a WAN?

Request for IPv6 support:

Some considerations:

• IPv6 does not modify UDP except for the checksum. In both cases, the UDP checksum adds the bytes of the source and target addresses, which in IPv6 are 4 times longer, but I'm quite sure that this field is calculated by the underlying UDP library, so this should not have an impact in Smudge.
• Transmitting IPv6 addresses inside the messages, however, does have an impact. There are two approaches to this. Use a flag that allows to distinguish IP versions and then use 4 or 16 bytes as necesary, or always use 16 bytes and embed the IPv4 in an IPv6 format (127.0.0.1 -> 7F.00.00.01 -> 0000:0000:0000:0000:0000:FFFF:7F00:0001 -> ::FFFF:7F00:1) what basically means using the last 4 bytes to store the IPv4 address and set all the bits of the 2 previous bytes. Golang's net.IP type will understand this addresses as IPv4.
• IPv6 increases the minimum message size that every implementation needs to subbort to 1280 bytes - 8 bytes for the UDP header - 40 bytes for the IPv6 header leaves 1232 bytes for the IPv6 extension headers plus the application payload. In IPv4 the maximum IP header is of 60 bytes and the UDP header 8 extra ones what makes the 576 minimum accepted package turn into 508 for the application.

The broadcasting feature is nice and all, but people have struggled with making it work in ways that it wasn't designed for, in large part because the way it works isn't obvious.

Some documentation on this would be 👍.

The code here: https://github.com/clockworksoul/smudge/blob/master/broadcast.go#L98
will read from the indexCounter without any thread safety. If two threads attempt to send a message at the same time, they'll both get the same index number, and one of those messages will be lost

It would be nice if programs that integrate Smudge could bring their own logger.

I'm willing to create a PR for this. Hopefully somewhere in the coming weeks.