Git Product home page Git Product logo

go-sockaddr's Introduction

go-sockaddr

sockaddr Library

Socket address convenience functions for Go. go-sockaddr is a convenience library that makes doing the right thing with IP addresses easy. go-sockaddr is loosely modeled after the UNIX sockaddr_t and creates a union of the family of sockaddr_t types (see below for an ascii diagram). Library documentation is available at https://godoc.org/github.com/hashicorp/go-sockaddr. The primary intent of the library was to make it possible to define heuristics for selecting the correct IP addresses when a configuration is evaluated at runtime. See the docs, template package, tests, and CLI utility for details and hints as to how to use this library.

For example, with this library it is possible to find an IP address that:

Or any combination or variation therein.

There are also a few simple helper functions such as GetPublicIP and GetPrivateIP which both return strings and select the first public or private IP address on the default interface, respectively. Similarly, there is also a helper function called GetInterfaceIP which returns the first usable IP address on the named interface.

sockaddr CLI

Given the possible complexity of the sockaddr library, there is a CLI utility that accompanies the library, also called sockaddr. The sockaddr utility exposes nearly all of the functionality of the library and can be used either as an administrative tool or testing tool. To install the sockaddr, run:

$ go get -u github.com/hashicorp/go-sockaddr/cmd/sockaddr

If you're familiar with UNIX's sockaddr struct's, the following diagram mapping the C sockaddr (top) to go-sockaddr structs (bottom) and interfaces will be helpful:

+-------------------------------------------------------+
|                                                       |
|                        sockaddr                       |
|                        SockAddr                       |
|                                                       |
| +--------------+ +----------------------------------+ |
| | sockaddr_un  | |                                  | |
| | SockAddrUnix | |           sockaddr_in{,6}        | |
| +--------------+ |                IPAddr            | |
|                  |                                  | |
|                  | +-------------+ +--------------+ | |
|                  | | sockaddr_in | | sockaddr_in6 | | |
|                  | |   IPv4Addr  | |   IPv6Addr   | | |
|                  | +-------------+ +--------------+ | |
|                  |                                  | |
|                  +----------------------------------+ |
|                                                       |
+-------------------------------------------------------+

Inspiration and Design

There were many subtle inspirations that led to this design, but the most direct inspiration for the filtering syntax was OpenBSD's pf.conf(5) firewall syntax that lets you select the first IP address on a given named interface. The original problem stemmed from:

  • needing to create immutable images using Packer that ran the Consul process (Consul can only use one IP address at a time);
  • images that may or may not have multiple interfaces or IP addresses at runtime; and
  • we didn't want to rely on configuration management to render out the correct IP address if the VM image was being used in an auto-scaling group.

Instead we needed some way to codify a heuristic that would correctly select the right IP address but the input parameters were not known when the image was created.

go-sockaddr's People

Contributors

amfern avatar angrycub avatar dadgar avatar dominik-lekse avatar jefferai avatar jen20 avatar magiconair avatar mattbostock avatar rboyer avatar schmichael avatar sean- avatar slackpad avatar swayne275 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-sockaddr's Issues

Possible IP comparison issue due to multiple possible representations

The code at https://github.com/hashicorp/go-sockaddr/blob/master/ipaddrs.go#L47 makes the assumption that IPv4 addresses are always stored in a 4 byte long byte slice and similarly a IPv6 address is stored as a 16 byte slice.

This assumption however does not hold, as the net.IP's state (https://golang.org/pkg/net/#IP):

Note that in this documentation, referring to an IP address as an IPv4 address or an IPv6 address is a semantic property of the address, not just the length of the byte slice: a 16-byte slice can still be an IPv4 address.

Due to this byte slice based comparisons of IP addresses may not be always as meaningful as the user might expect, and the same IPv4 with different internal representation my cause comparison mismatches, resulting in unforseen anomalies.

Note, this issue report was not initiated by my use of this package and noticing a bug, rather by creating an extension proposal for go vet and that signalling the possible misuse in this library.

Cheers :)

Unable to Install

Hi,

Got this when trying to install:

# github.com/posener/complete/cmd/install
go/src/github.com/posener/complete/cmd/install/install.go:102: undefined: os.Executable
# github.com/hashicorp/go-sockaddr
go/src/github.com/hashicorp/go-sockaddr/ifaddrs.go:140: undefined: strings.Compare
go/src/github.com/hashicorp/go-sockaddr/ifaddrs.go:181: undefined: strings.Compare
go/src/github.com/hashicorp/go-sockaddr/ifaddrs.go:728: signRE.Copy undefined (type *regexp.Regexp has no field or method Copy)
go/src/github.com/hashicorp/go-sockaddr/ifaddrs.go:1201: whitespaceRE.Copy undefined (type *regexp.Regexp has no field or method Copy)
go/src/github.com/hashicorp/go-sockaddr/ifaddrs.go:1244: whitespaceRE.Copy undefined (type *regexp.Regexp has no field or method Copy)
go/src/github.com/hashicorp/go-sockaddr/ifaddrs.go:1264: ifNameRE.Copy undefined (type *regexp.Regexp has no field or method Copy)
go/src/github.com/hashicorp/go-sockaddr/ifaddrs.go:1265: ipAddrRE.Copy undefined (type *regexp.Regexp has no field or method Copy)
go/src/github.com/hashicorp/go-sockaddr/route_info_solaris.go:12: routeInfo redeclared in this block
	previous declaration at go/src/github.com/hashicorp/go-sockaddr/route_info_linux.go:8
go/src/github.com/hashicorp/go-sockaddr/route_info_solaris.go:18: NewRouteInfo redeclared in this block
	previous declaration at go/src/github.com/hashicorp/go-sockaddr/route_info_linux.go:14
go/src/github.com/hashicorp/go-sockaddr/route_info_solaris.go:26: routeInfo.GetDefaultInterfaceName redeclared in this block
	previous declaration at go/src/github.com/hashicorp/go-sockaddr/route_info_linux.go:29
go/src/github.com/hashicorp/go-sockaddr/ifaddrs.go:1265: too many errors

"dummy" network adapters do not work with `GetInterfaceIP`

A picture console log is worth 1000 words:

TL;DR: interfaces of type "dummy" (like a loopback, but usually not given an address in the 127.x space) are not supported(?) by the GetInterfaceIP template function

  1. Grab the binary to play around:
root@ip-172-25-26-131:~/go/bin# go get -u github.com/hashicorp/go-sockaddr/cmd/sockaddr
  1. List all the network interfaces on the host:
root@ip-172-25-26-131:~/go/bin# ip -o link show | awk -F': ' '{print $2}'
lo
ens5
dummy0
docker0
  1. demonstrate the tool works:
root@ip-172-25-26-131:~/go/bin# ./sockaddr eval -d 'GetInterfaceIP "ens5"'
[0] in: "{{GetInterfaceIP \"ens5\"}}"
[0] out: "172.25.26.131"

root@ip-172-25-26-131:~/go/bin# ifconfig ens5
ens5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 172.25.26.131  netmask 255.255.240.0  broadcast 172.25.31.255
        inet6 fe80::e9:d4ff:fe2c:7eb4  prefixlen 64  scopeid 0x20<link>
        inet6 2600:1f1c:61e:1701:d208:ba80:cf7c:9e9e  prefixlen 128  scopeid 0x0<global>
        ether 02:e9:d4:2c:7e:b4  txqueuelen 1000  (Ethernet)
        RX packets 159045  bytes 89658689 (89.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 83925  bytes 13061624 (13.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  1. test the tool with the intended interface:
root@ip-172-25-26-131:~/go/bin# ./sockaddr eval -d 'GetInterfaceIP "dummy0"'
[0] in: "{{GetInterfaceIP \"dummy0\"}}"
[0] out: ""

a Nil result was not expected.

  1. Show the expected result:
root@ip-172-25-26-131:~/go/bin# ifconfig dummy0
dummy0: flags=195<UP,BROADCAST,RUNNING,NOARP>  mtu 1500
        inet 169.254.1.1  netmask 255.255.255.255  broadcast 0.0.0.0
        inet6 fe80::1cc3:f6ff:fe14:c441  prefixlen 64  scopeid 0x20<link>
        ether 1e:c3:f6:14:c4:41  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 11  bytes 770 (770.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
  1. show how the dummy interface came into existence:
root@ip-172-25-26-131:~/go/bin# cat /etc/systemd/network/dummy0.netdev
# Creates a "dummy" network interface
# we'll configure this interface with a link-local address
# See: https://www.freedesktop.org/software/systemd/man/systemd.netdev.html
##

[NetDev]
Name=dummy0
Kind=dummy

Additional info that may be helpful:

root@ip-172-25-26-131:~/go/bin# lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 18.04.2 LTS
Release:	18.04
Codename:	bionic

For context, this issue was discovered while troubleshooting this specific configuration of consul:
hashicorp/consul#5371

The `sockaddr.Include` and `sockaddr.Exclude` do not have "private" selector.

Hi @hairyhenderson,

I've noticed that the documentation mentions that the sockaddr.Exclude and sockaddr.Include support a "private" selector, but it does not seem to be the case, for instance:

$ gomplate -i '{{ sockaddr.GetAllInterfaces | sockaddr.Include "private" "" }}'
Error: template: <arg>:1:39: executing "<arg>" at <sockaddr.Include>: error calling Include: invalid include selector "private"

Which is correct as per the following:

go-sockaddr/ifaddrs.go

Lines 963 to 994 in 6d291a9

// IncludeIfs returns an IfAddrs based on the passed in selector.
func IncludeIfs(selectorName, selectorParam string, inputIfAddrs IfAddrs) (IfAddrs, error) {
var includedIfs IfAddrs
var err error
switch strings.ToLower(selectorName) {
case "address":
includedIfs, _, err = IfByAddress(selectorParam, inputIfAddrs)
case "flag", "flags":
includedIfs, _, err = IfByFlag(selectorParam, inputIfAddrs)
case "name":
includedIfs, _, err = IfByName(selectorParam, inputIfAddrs)
case "network":
includedIfs, _, err = IfByNetwork(selectorParam, inputIfAddrs)
case "port":
includedIfs, _, err = IfByPort(selectorParam, inputIfAddrs)
case "rfc", "rfcs":
includedIfs, _, err = IfByRFCs(selectorParam, inputIfAddrs)
case "size":
includedIfs, _, err = IfByMaskSize(selectorParam, inputIfAddrs)
case "type":
includedIfs, _, err = IfByType(selectorParam, inputIfAddrs)
default:
return IfAddrs{}, fmt.Errorf("invalid include selector %q", selectorName)
}
if err != nil {
return IfAddrs{}, err
}
return includedIfs, nil
}

To achieve this (somewhat), one would have to use the rfc selector with an appropriate numeric RFC code. Alternatively, we could send a Pull Request against go-sockaddr to add this include/exclude filter.

Would you like to send a Pull Request to update documentation?

Bind consul to correct private IP when GetPrivateInterfaces returns more than one.

Hey,

I don't know GO.
I'm running consul on DigitalOcean and I need to bind it to a private address.
My issue is:

DigitalOcean has two private IP's because of it's Floating IP functionality, so all Droplets come with two private IP's.

[root@xxx-consul-server-0 ~]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    0      0        0 eth0
10.19.0.0       0.0.0.0         255.255.0.0     U     0      0        0 eth0
10.135.0.0      0.0.0.0         255.255.0.0     U     0      0        0 eth1
link-local      0.0.0.0         255.255.0.0     U     1002   0        0 eth0
link-local      0.0.0.0         255.255.0.0     U     1003   0        0 eth1
xxxxxxxxx     0.0.0.0         255.255.240.0   U     0      0        0 eth0

I was trying to run this with a command I checked in consul's page:

'GetPrivateInterfaces | include "network" "10.0.0.0/8" | attr "address"'

But it returns the "Floating IP" private IP instead of the Droplet's private IP.

So, I understood that with this command I was returning both IP addresses, filtering the cidr block and fetching one of the IP's. But the one it was being returned is the wrong one.

I got to this dumb solution:

'GetPrivateInterfaces | include "network" "10.0.0.0/8" | sort "-name" | attr "address"'

Where I just invert the sort by name and it returns me the one I want.

I'm wondering how should this be correctly done.

Thank you all.

Route detection fails on CoreOS due to /sbin/ip not being present

Followup from hashicorp/nomad#2696
Code tries to execute /sbin/ip route, which fails on CoreOS - they ship iproute2 in /usr/bin/ip

Related line from Nomad server log:
Jun 07 19:32:01 ip-10-249-200-172 nomad[22507]: ==> Failed to parse HTTP advertise address: Unable to parse default advertise address: Unable to parse address template "{{ GetPrivateIP }}": unable to execute sockaddr input "{{ GetPrivateIP }}": template: sockaddr.Parse:1:3: executing "sockaddr.Parse" at <GetPrivateIP>: error calling GetPrivateIP: fork/exec /sbin/ip: no such file or directory

GetDefaultInterfaces can't support equivalent routing

I have a machine with default equivalent routing,than GetDefaultInterfaces cant't work。
example:

ip route

default
nexthop via 223.110.247.129 dev bond0 weight 1
nexthop via 223.110.247.129 dev bond1 weight 1

Please, use fixtures instead of live commands for tests

Hi,

In the course of upgrading the version of Consul in Debian, I have to package this library. While doing so, I get all kinds of test failures that seem to stem from the fact that my network configuration does not match many assumptions in the tests. This makes the tests very fragile, and almost useless for the general case. It would be a lot more robust if instead of calling /sbin/ip you save the output of the command and use that in mocks.

Thanks.

Why arent receiving all text message and calls to my device.

Listen, This is becoming extremely depressing. I live in Atlanta, GA. I'm not interested in any clouds, cloud flare, or was. I'm very disappointed that my text aren't reaching my phone. Have you guys blocked my server and ports. I've never seen IP address this long. Secondly, why does my device show ad chrome on maps. If I can confirm AWS or cloud flare is blocking my network, I promise there will be ligations. This is awful

Create a function to collect public interfaces from nodes that aren't aware of it

There are a few techniques that could be used:

3rd party DNS

dig +short myip.opendns.com @resolver1.opendns.com
## or ##
dig TXT +short o-o.myaddr.l.google.com @ns1.google.com

Instance Metadata

AWS

curl http://checkip.amazonaws.com

Digital Ocean

curl http://169.254.169.254/metadata/v1/interfaces/public/0/ipv4/address

Azure

curl -H Metadata:true "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/publicIpAddress?api-version=2017-08-01&format=text"

The DNS based scheme has the most general utility and could solve for all cases, but does require a third party (so it could vanish or change). The provider metadata route is the most encapsulated, but requires an implementation per provider (assuming that they provide instance metadata... a fairly safe assumption, but an assumption nonetheless)

Create a separate Go module for cmd/sockaddr

go-sockaddr is used in HashiCorp core modules such as go-secure-stdlib/parseutil.
go-sockaddr Go module includes cmd/sockaddr which depends on modules used only for this CLI tool. Those features bring their own dependencies which are not needed by the core go-sockaddr functionality.

I propose to drop cmd/sockaddr/vendor and to create a separate Go module for cmd/sockaddr:

cd cmd/sockaddr
go mod init github.com/hashicorp/go-sockaddr/cmd/sockaddr
rm -Rf vendor
go get github.com/hashicorp/[email protected]
go get github.com/mitchellh/[email protected]
go get github.com/mitchellh/[email protected]
go get github.com/ryanuber/[email protected]+incompatible
go get github.com/hashicorp/[email protected]
git add go.mod go.sum
cd ..
go mod tidy
git add go.mod go.sum
git commit -m "Create Go module for cmd/sockaddr"
git tag v1.0.3
git push --tags
cd cmd/sockaddr
go get github.com/hashicorp/[email protected]
go mod tidy
git add go.mod go.sum
git commit
git tag cmd/go-sockaddr/v1.0.3
git push --tags

Code review pre-integration w/Consul

Going to use this as a checklist for review since it's hard to review a repo in-place.

  • sockaddr.go: Remove commented-out Listen* interface methods in SockAddr
  • sockaddr.go: Comment for NewSockAddr() about order is wrong, you check for UNIX last
  • sockaddr.go: In NewSockAddr() final error should probably format with a %q
  • sockaddr.go: Delete commented-out NewPort()
  • sockaddr.go: In the To* functions, why can't you do a normal type switch - why query the type enum and then try to cast?
  • sockaddr_test.go: Do you have tests elsewhere for IPv6 and UNIX types (add TODOs for now)
  • sockaddrs.go: In Less() there's an unreachable panic() that's confusing; should delete it
  • sockaddrs.go: in AscType() you can get rid of the panic with a final else
  • ifaddrs.go: in GetIfSockAddrs() it should bail out with an error if NewIPAddr() fails, not just continue
  • ifaddrs.go: we don't need the panic() in GetDefaultInterface() since there's already a well-defined error return if that happens
  • ifaddrs.go: IfByNameExclude(), IfByNameInclude(), IfByTypeExclude(), and IfByTypeInclude() don't need the error wrap - it'll just repeat the same message
  • ifaddrs_test.go: needs better coverage of the different filter functions (add TODOs for now)
  • ipv4addr.go: delete commented-out code in Cmp()
  • ipv4addr.go: i'm having trouble following the logic in CmpRFC()
  • ipv6addr.go: there's a stale comment from the IPv4 version of the NewIPv6Addr() function

Fail test for go1.19

After switching to go1.19 I got fail test

=== RUN   TestSockAddr_IPAddrs_IPAddrsByNetworkSize/0
    ipaddrs_test.go:340: [2] Sort equality failed: expected 128.95.120.2:8600, received 128.95.120.2
    ipaddrs_test.go:340: [3] Sort equality failed: expected 128.95.120.2, received 128.95.120.2:8600

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.