Git Product home page Git Product logo

gobetween's Introduction

Stand With Ukraine

gobetween

Tag Build Status Go Report Card Docs Docker Snap Status Telegram License

gobetween - modern & minimalistic load balancer and reverse-proxy for the ☁️ Cloud era.

Current status: Maintenance mode, accepting PRs. Currently in use in several highly loaded production environments.

Features

  • Fast L4 Load Balancing

  • Clear & Flexible Configuration with TOML or JSON

    • File - read configuration from the file
    • URL - query URL by HTTP and get configuration from the response body
    • Consul - query Consul key-value storage API for configuration
  • Management REST API

    • System Information - general server info
    • Configuration - dump current config
    • Servers - list, create & delete
    • Stats & Metrics - for servers and backends including rx/tx, status, active connections & etc.
  • Discovery

    • Static - hardcode backends list in the config file
    • Docker - query backends from Docker / Swarm API filtered by label
    • Exec - execute an arbitrary program and get backends from its stdout
    • JSON - query arbitrary http url and pick backends from response json (of any structure)
    • Plaintext - query arbitrary http and parse backends from response text with customized regexp
    • SRV - query DNS server and get backends from SRV records
    • Consul - query Consul Services API for backends
    • LXD - query backends from LXD
  • Healthchecks

    • Ping - simple TCP ping healthcheck
    • Exec - execute arbitrary program passing host & port as options, and read healthcheck status from the stdout
    • Probe - send specific bytes to backend (udp, tcp or tls) and expect a correct answer (bytes or regexp)
  • Balancing Strategies (with SNI support)

    • Weight - select backend from pool based relative weights of backends
    • Roundrobin - simple elect backend from pool in circular order
    • Iphash - route client to the same backend based on client ip hash
    • Iphash1 - same as iphash but backend removal consistent (clients remain connecting to the same backend, even if some other backends down)
    • Leastconn - select backend with least active connections
    • Leastbandwidth - backends with least bandwidth
  • Integrates seamlessly with Docker and with any custom system (thanks to Exec discovery and healthchecks)

  • Single binary distribution

Architecture

gobetween

Usage

Hacking

Debug and Test

Run several web servers for tests in different terminals:

  • $ python -m SimpleHTTPServer 8000
  • $ python -m SimpleHTTPServer 8001

Instead of Python's internal HTTP module, you can also use a single binary (Go based) webserver like: https://github.com/udhos/gowebhello

gowebhello has support for SSL sertificates as well (HTTPS mode), in case you want to do quick demos of the TLS+SNI capabilities of gobetween.

Put localhost:8000 and localhost:8001 to static_list of static discovery in config file, then try it:

  • $ gobetween -c gobetween.toml

  • $ curl http://localhost:3000

Enable profiler and debug issues you encounter

[profiler]
enabled = true     # false | true
bind    = ":6060"  # "host:port"

Performance

It's Fast! See Performance Testing

The Name

It's a play on words: gobetween ("go between").

Also, it's written in Go, and it's a proxy so it's something that stays between 2 parties 😄

License

MIT. See LICENSE file for more details.

Authors & Maintainers

All Contributors

Community

  • Join gobetween Telegram group here.

Logo

Logo by Max Demchenko

gobetween's People

Contributors

ctrlok avatar eric-lindau avatar erincandescent avatar falcosuessgott avatar illarion avatar jtopjian avatar kikom avatar knicknic avatar kron4eg avatar nickdoikov avatar pitan avatar shantanugadgil avatar smerschjohann avatar techiscool avatar tpo avatar williamscs avatar yousong avatar yyyar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gobetween's Issues

Expose servers stats

server:

  • active connections
  • total connections
  • traffic rx/tx (total)
  • current traffic rx/tx (gb,mb,kb,b,/s)
  • backends
    • status : live | down
    • inbound traffic (total/avg(1 sec))
    • outbound traffic (total/avg(1 sec))
    • active connections
    • total connections

Add TLS client auth support to Docker discovery

Due to security reasons add optional ability to use TLS and client authentication while querying Docker API, providing additional parameters in docker discovery config.

Proposed format:

[servers.<name>.discovery]
# ...
kind = 'docker'
# ...
docker_tls_enabled = false # false by default for backward compatibility

# required if docker_tls_enabled = true
docker_tls_cert_path = '/path/to/cert.pem'
docker_tls_key_path = '/path/to/key.pem'
docker_tls_cacert_path = '/path/to/cacert.pem'

More info: https://docs.docker.com/engine/security/https/

Mesos/Marathon Discovery

For future releases we can add mesos/marathon support via their rest api.

https://mesosphere.github.io/marathon/docs/generated/api.html#

api enteryponint - ip/hostname:port
api service enterypoint - /v2/apps/[app_id]/tasks

Response:

Body

Type: application/json

Example:

{
  "tasks": [
    {
      "appId": "/minecraft/survival-world",
      "host": "srv7.hw.ca1.mesosphere.com",
      "id": "minecraft_survival-world.564bd685-4c30-11e5-98c1-be5b2935a987",
      "ports": [
        31756
      ],
      "slaveId": null,
      "stagedAt": "2015-08-26T20:23:39.463Z",
      "startedAt": "2015-08-26T20:23:44.678Z",
      "version": "2015-04-17T04:00:14.171Z"
    }
  ]
}

Type: text/plain

minecraft_survival-world    10013   srv7.hw.ca1.mesosphere.com:31756

Web admin panel

Basic functionality for gobetween frontend :

1. password auth
2. servers list
3. Stats sheet/graph  for each server 
3.1 for server : 
     incoming connections quantity,
     active connections
     traffic (rx/tx) 
3.2 for each backend node:
      check status (ok,failed)
      connection quantity    
      active connections
       traffic (rx/tx)
       weight
       priority
4. gobetween configuration constructor
     basic functionality
     add/delete servers
     add/delete backend nodes for static list
     change discovery type for each server/edit backend variables
     change check type /check variables 

list is unfinished....

multiplie same name server creation cause of api and stat freeze

steps to reproduce
1)start gobetween without any servers configured
2) add server via api
3) add same server second dime
4) possible need to continue step 3)
5) check ip:apiport/dump and ip:apiport/servers//stats
some of them or both links stops to respond

Container IP's for Docker discovery

As user i would like to have an option to return private container IP and service Port pairs during discovery for cases where Docker Networking is used and load balancer can reach services directly (without port mapping).

Add TLS support to REST API

#  [api.tls]                        # (optional) Enable HTTPS
#  cert_path = "/path/to/cert.pem"  # Path to certificate
#  key_path = "/path/to/key.pem"    # Path to key

SRV Discovery fails when number of service DNS records is >= 6

When amount of SRV records in service is 6 or more discovery fails with error:

2016-07-19 18:21:39 [INFO ] (srvFetch): Fetching consul.service.consul:8600 nginx.service.consul.
2016-07-19 18:21:39 [ERROR] (discovery): srv error dns: buffer size too small retrying in 2s
2016-07-19 18:21:39 [INFO ] (discovery): Applying failpolicy keeplast

After service has been scaled down, problem went away

after some time gobetween stops receive connections

netstat -anp |grep 9200 |wc -l

178

{
    "active_connections": 6,
    "rx_total": 1562227305,
    "tx_total": 1724183714,
    "rx_second": 21308,
    "tx_second": 55111,
    "backends": [
        {
            "host": "10.0.0.53",
            "port": "9200",
            "priority": 1,
            "weight": 1,
            "stats": {
                "live": true,
                "total_connections": 9692,
                "active_connections": 1,
                "rx": 234993851,
                "tx": 133001288,
                "rx_second": 187,
                "tx_second": 318
            }
        },
        {
            "host": "10.0.0.54",
            "port": "9200",
            "priority": 1,
            "weight": 1,
            "stats": {
                "live": true,
                "total_connections": 9839,
                "active_connections": 1,
                "rx": 216484944,
                "tx": 131607080,
                "rx_second": 0,
                "tx_second": 491
            }
        },
        {
            "host": "10.0.0.55",
            "port": "9200",
            "priority": 1,
            "weight": 1,
            "stats": {
                "live": true,
                "total_connections": 9629,
                "active_connections": 1,
                "rx": 166526541,
                "tx": 2506771,
                "rx_second": 39,
                "tx_second": 32
            }
        },
        {
            "host": "10.0.0.51",
            "port": "9200",
            "priority": 1,
            "weight": 1,
            "stats": {
                "live": true,
                "total_connections": 9849,
                "active_connections": 2,
                "rx": 776211040,
                "tx": 1453758841,
                "rx_second": 34081,
                "tx_second": 80542
            }
        },
        {
            "host": "10.0.0.52",
            "port": "9200",
            "priority": 1,
            "weight": 1,
            "stats": {
                "live": true,
                "total_connections": 9508,
                "active_connections": 0,
                "rx": 168079725,
                "tx": 3470986,
                "rx_second": 39,
                "tx_second": 32
            }
        }
    ]
}
# dumped on 2016-07-13 04:55:44.270808288 +0000 UTC

[logging]
  level = "info"
  output = "/var/log/gobetween.log"

[api]
  enabled = true
  bind = "10.0.0.8:8888"

[defaults]
  max_connections = 0
  client_idle_timeout = "0"
  backend_idle_timeout = "0"
  backend_connection_timeout = "0"

[servers]
  [servers.elasticsearch]
    max_connections = 10000
    client_idle_timeout = "1m"
    backend_idle_timeout = "1m"
    backend_connection_timeout = "3m"
    bind = "10.0.0.8:9200"
    protocol = "tcp"
    balance = "weight"
    [servers.elasticsearch.discovery]
      kind = "static"
      failpolicy = "keeplast"
      interval = "0"
      timeout = "0"
      static_list = ["10.0.0.51:9200", "10.0.0.52:9200", "10.0.0.53:9200", "10.0.0.54:9200", "10.0.0.55:9200"]
    [servers.elasticsearch.healthcheck]
      kind = "ping"
      interval = "10s"
      passes = 3
      fails = 3
      timeout = "1s"

last logs

2016-07-12 22:00:16 [INFO ] (proxy): read tcp 10.0.0.8:38724->10.0.0.53:9200: use of closed network connection
2016-07-12 22:00:18 [INFO ] (proxy): read tcp 10.0.0.8:38281->10.0.0.52:9200: use of closed network connection
2016-07-12 22:00:55 [INFO ] (proxy): read tcp 10.0.0.8:57795->10.0.0.55:9200: i/o timeout
2016-07-12 22:00:55 [INFO ] (proxy): read tcp 10.0.0.8:9200->10.0.0.60:45876: use of closed network connection
2016-07-12 22:01:13 [INFO ] (proxy): read tcp 10.0.0.8:59927->10.0.0.53:9200: i/o timeout
2016-07-12 22:01:13 [INFO ] (proxy): read tcp 10.0.0.8:9200->10.0.0.60:45347: use of closed network connection
2016-07-12 22:01:14 [INFO ] (proxy): read tcp 10.0.0.8:35520->10.0.0.54:9200: i/o timeout
2016-07-12 22:01:14 [INFO ] (proxy): read tcp 10.0.0.8:9200->10.0.0.60:45348: use of closed network connection
2016-07-12 22:01:18 [INFO ] (proxy): read tcp 10.0.0.8:9200->10.0.0.60:45344: i/o timeout
2016-07-12 22:01:19 [INFO ] (proxy): read tcp 10.0.0.8:9200->10.0.0.60:45345: i/o timeout

Docker single host respond with wildcard ip 0.0.0.0:port

Instead of swarm that respond with :

ec9ec5753ee2        test1/company:test   "/bin/bash -c 'start   46 hours ago        Up 46 hours         5.5.5.5:39599->9999/tcp, 5.5.5.5:50300->9000/udp,  5.5.5.5:39600->9000/tcp         n1.node.test.com/12121212

docker node

ec9ec5753ee2        test1/company:test   "/bin/bash -c 'start   46 hours ago        Up 46 hours         0.0.0.0:39599->9999/tcp, 0.0.0.0:50300->9000/udp, 0.0.0.0:39600->9000/tcp         n1.node.test.com/12121212

So, by default in case of single remote docker host discovery we can`t determine docker containers IP.
It can be fixed in Linux by running :

docker run -d -p=192.168.123.138:80:80 tutum/hello-world

but under windows we have :

PS C:\Docker> docker -H tcp://127.0.0.1:2375 run -l proxied=true -d -p 10.0.0.4:8083:80 helloworld
edd0a1ea04851d5d0c817efb2fd3ea0ee6e3f31f89b3fb53a220ab07feead00b
C:\Windows\system32\docker.exe: Error response from daemon: failed to create endpoint hungry_ptolemy on network nat: Windows does not support host IP addresses in NAT settings.

 PS C:\Docker> docker -H tcp://127.0.0.1:2375 run -l proxied=true -d --ip 10.0.0.4 -p 8083:80 helloworld
f4069038a829d617718bc80e4b33fb3a10468fc155b1742dba5319653703650b
C:\Windows\system32\docker.exe: Error response from daemon: User specified IP address is supported on user defined networks only.

it needs to eliminate this case for windows compatibility and allow to use linux docker also with wildcard binding

for this we need check does discovery return 0.0.0.0:port and change port with variable :
docker_binded_host=IP/hostname

Disable/enable backend via api

need to add such functionality via api without deleting/recreating server:

  1. disable backend server ip:port (continuously). This should disable all traffic to this node and exclude it from active backend pool (same as for faulty healthcheck)
  2. Enable backend server ip:port (continuously). This should enable(rebalance according to a policy) traffic to this node and exclude it from active backend pool (same as for healthy healthcheck)
  3. if this server disappear from backend pool - delete it`s "disable" flag.

Add go get into Makefile

Hi ALL

For build gobeetwen at the clear system add this lines into Makefile

go get github.com/BurntSushi/toml
go get github.com/Sirupsen/logrus
go get github.com/elgs/gojq
go get github.com/fsouza/go-dockerclient
go get github.com/gin-gonic/gin
go get github.com/hashicorp/consul/api
go get github.com/miekg/dns
go get github.com/spf13/cobra

Error: dial tcp :0: getsockopt: connection refused

Hello,
Im trying gobetween on my dev machine - Im using docker-machine & boot2docker (1.12.3)

This is my simple docker compose:

version: '2'

networks:
  backend:

services:
  app:
    image: php:7.0-alpine
    expose:
      - 8080
    volumes:
      - .:/data
    command: php -S 0.0.0.0:8080 /data/index.php
    networks:
      - backend
    labels:
      - "scale.app=true"

  gobetween:
    image: yyyar/gobetween
    depends_on:
      - app
    ports:
      - "80:80"
    volumes:
      - "./gobetween/conf:/etc/gobetween/conf/:rw"
      - "/var/run/docker.sock:/var/run/docker.sock"
    networks:
      - backend

Gobetween conf looks like this:

[logging]
level = "debug"

[servers.app]
bind = "0.0.0.0:80"
protocol = "tcp"
balance = "roundrobin"

  [servers.app.discovery]
    interval = "10s"
    timeout = "2s"
    kind = "docker"
    docker_endpoint = "unix://var/run/docker.sock"  # Docker / Swarm API
    docker_container_label = "scale.app=true"  # label to filter containers
    docker_container_private_port = 8080   # gobetween will take public container port for this private port
    docker_container_host_env_var = "HOSTNAME"

Gobetween is able to discover new backends, when i try to scale up the app, but proxying is not working - see log output:

app_1        | PHP 7.0.13 Development Server started at Fri Nov 25 20:08:30 2016
gobetween_1  | 2016-11-25 20:08:31 [INFO ] (manager): Initializing...
gobetween_1  | 2016-11-25 20:08:31 [INFO ] (server): Creating 'app': 0.0.0.0:80 roundrobin docker none
gobetween_1  | 2016-11-25 20:08:31 [INFO ] (scheduler): Starting scheduler
gobetween_1  | 2016-11-25 20:08:31 [INFO ] (manager): Initialized
gobetween_1  | 2016-11-25 20:08:31 [INFO ] (api): API disabled
gobetween_1  | 2016-11-25 20:08:31 [INFO ] (dockerFetch): Fetching unix://var/run/docker.sock scale.app=true 8080
gobetween_1  | 2016-11-25 20:08:40 [DEBUG] (server.handle): Accepted 10.211.55.2:52649 -> [::]:80
gobetween_1  | 2016-11-25 20:08:40 [DEBUG] (server.handle): Accepted 10.211.55.2:52648 -> [::]:80
gobetween_1  | 2016-11-25 20:08:40 [ERROR] (server.handle): dial tcp :0: getsockopt: connection refused
gobetween_1  | 2016-11-25 20:08:40 [ERROR] (server.handle): dial tcp :0: getsockopt: connection refused

Am I missing something?

[feature request] gobetween register itself in Consul

Hi,
Would it be possible to add a feature where gobetween can register itself in the Consul it would
use for service discovery.
This would helpful when the IP of the Load Balancer may change across reboots.

As of now I am making do with a hackish workaround ...
A consul agent on the load balancer machine. and a cron job which get the IP of eth0 and recreates
a service definition in '/etc/consul/'.

Regards,
Shantanu

Windows exec discovery failed

exec discovery script :

    @ECHO OFF
    echo webseerver1.loc:80 weight=1
    echo(
    echo webserver2.loc:80 weight=1

discovery config section :

    [servers.sample.discovery]  
    kind = "exec"
    exec_command = ['D:\scripts\exec_discovery.bat'] 
    exec_command = ['node', 'D:\scripts\test.js'] 

Gobetween can discovery only last server from discovery script because of a newline character difference between windows and linux.

Need to make discovery regexp for exec discovery configurable.

Basic REST API functionality

Add build-in REST API server functionality

  • list / get / create / remove servers
  • save current config / restore config from config path selected during startup

mutiplie errors during gobetween functioning

2016-07-12 13:28:05 DEBUG: Accepted 10.0.0.50:57617 -> 10.0.0.8:9200
2016-07-12 13:28:05 DEBUG: Begin 10.0.0.50:57617 -> 10.0.0.8:9200 -> 10.0.0.55:9200
2016-07-12 13:28:05 INFO : read tcp 10.0.0.8:49356->10.0.0.55:9200: use of closed network connection
2016-07-12 13:28:05 DEBUG: End 10.0.0.50:57617 -> 10.0.0.8:9200 -> 10.0.0.55:9200
2016-07-12 13:28:05 DEBUG: Accepted 10.0.0.50:57618 -> 10.0.0.8:9200
2016-07-12 13:28:05 DEBUG: Begin 10.0.0.50:57618 -> 10.0.0.8:9200 -> 10.0.0.53:9200
2016-07-12 13:28:05 INFO : read tcp 10.0.0.8:58063->10.0.0.53:9200: use of closed network connection
2016-07-12 13:28:05 DEBUG: End 10.0.0.50:57618 -> 10.0.0.8:9200 -> 10.0.0.53:9200
2016-07-12 13:28:07 DEBUG: Accepted 10.0.0.50:57619 -> 10.0.0.8:9200
2016-07-12 13:28:07 DEBUG: Begin 10.0.0.50:57619 -> 10.0.0.8:9200 -> 10.0.0.51:9200
2016-07-12 13:28:07 INFO : read tcp 10.0.0.8:51749->10.0.0.51:9200: use of closed network connection
2016-07-12 13:28:07 DEBUG: End 10.0.0.50:57619 -> 10.0.0.8:9200 -> 10.0.0.51:9200
2016-07-12 13:28:07 DEBUG: Accepted 10.0.0.50:57620 -> 10.0.0.8:9200
2016-07-12 13:28:07 DEBUG: Begin 10.0.0.50:57620 -> 10.0.0.8:9200 -> 10.0.0.54:9200
2016-07-12 13:28:07 INFO : read tcp 10.0.0.8:33657->10.0.0.54:9200: use of closed network connection
2016-07-12 13:28:07 DEBUG: End 10.0.0.50:57620 -> 10.0.0.8:9200 -> 10.0.0.54:9200
2016-07-12 13:28:07 DEBUG: Accepted 10.0.0.50:57621 -> 10.0.0.8:9200
2016-07-12 13:28:07 DEBUG: Begin 10.0.0.50:57621 -> 10.0.0.8:9200 -> 10.0.0.55:9200
2016-07-12 13:28:07 INFO : read tcp 10.0.0.8:49360->10.0.0.55:9200: use of closed network connection
2016-07-12 13:28:07 DEBUG: End 10.0.0.50:57621 -> 10.0.0.8:9200 -> 10.0.0.55:9200
2016-07-12 13:28:07 DEBUG: Accepted 10.0.0.50:57622 -> 10.0.0.8:9200
2016-07-12 13:28:07 DEBUG: Begin 10.0.0.50:57622 -> 10.0.0.8:9200 -> 10.0.0.53:9200
2016-07-12 13:28:07 INFO : read tcp 10.0.0.8:58067->10.0.0.53:9200: use of closed network connection
2016-07-12 13:28:07 DEBUG: End 10.0.0.50:57622 -> 10.0.0.8:9200 -> 10.0.0.53:9200

seems like it is related to :

http://stackoverflow.com/questions/31385487/golang-http-having-use-of-closed-network-connection-error

panic: runtime error: invalid memory address or nil pointer dereference when no default section in config

./gobetween
2016-07-25 09:05:32 INFO : gobetween v0.2.0
2016-07-25 09:05:32 INFO : Using config file ./gobetween.toml
2016-07-25 09:05:32 INFO : Initializing...
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x464d22]

goroutine 6 [running]:
panic(0xa3ca00, 0xc82000a100)
/usr/lib/go/src/runtime/panic.go:464 +0x3e6
_/home/yyyar/workspace/gobetween/src/manager.prepareConfig(0xc8201ce184, 0x6, 0xc8201cd530, 0xc8201cd540, 0xc8201cdb90, 0xc8201cdb70, 0xc8201cc7c0, 0xc, 0xc8201cc838, 0x3, ...)
/home/yyyar/workspace/gobetween/src/manager/manager.go:269 +0x8b2
_/home/yyyar/workspace/gobetween/src/manager.Create(0xc8201ce184, 0x6, 0xc8201cd530, 0xc8201cd540, 0xc8201cdb90, 0xc8201cdb70, 0xc8201cc7c0, 0xc, 0xc8201cc838, 0x3, ...)
/home/yyyar/workspace/gobetween/src/manager/manager.go:116 +0xd7
_/home/yyyar/workspace/gobetween/src/manager.Initialize(0xc8201cc258, 0x5, 0xc8201cc320, 0x6, 0x1, 0xc8201cc4c0, 0x5, 0xc8201d2160, 0x0, 0x0, ...)
/home/yyyar/workspace/gobetween/src/manager/manager.go:46 +0x27b
created by main.main
/home/yyyar/workspace/gobetween/src/main.go:83 +0x4f3

I downloaded the binary package gobetween_0.2.0_linux_amd64.tar

Distributor ID: Ubuntu
Description: Ubuntu 14.04.4 LTS
Release: 14.04
Codename: trusty

uname -a
Linux 3.19.0-61-generic #69~14.04.1-Ubuntu SMP Thu Jun 9 09:09:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Add multy-endpoint support for docker discovery

For docker discovery need to add multi-endpoint functionality .

something like :

docker_endpoint  = [
    "http://localhost:2375",
    "http://1.2.3.4:2375"
]

Interval and timeout variable values should be applied to each endpoint in pool, not to whole servers pool.

In this case discovery will be asynchronous but it is only one possible deal to implement docker multi-endpoint(multi docker host) discovery. I think it will be okay for most cases.

Optional health checks

Provide config option that makes possible to disable health checks.
This makes sense for environments where heath checking is embedded or performed in service discovery tool (e.g. Consul)

Implement HA (Failover / Redundancy)

Hi, Is there any way I can achieve HA for the load balancer?
I would like to have two nodes (doesn't matter AA, or AP) and when a node goes down, the other keeps redirecting traffic.

Thanks.

Cоnsul service discovery intergration

Need to add consul discovery service integration.

       consul service discovery api calls :
       https://www.consul.io/docs/agent/http/catalog.html#catalog_services
api enteryponint - ip/hostname:port
api service enterypoint -  /v1/catalog/service/[servicename]
[{
    "Node": "ip-172-30-2-11",
    "Address": "172.30.2.11",
    "ServiceID": "ip-172-30-2-11:high_panini:8443",
    "ServiceName": "service1",
    "ServiceTags": ["tag1","tag2"],
    "ServiceAddress": "1.1.1.1",
    "ServicePort": 24338,
    "ServiceEnableTagOverride": false,
    "CreateIndex": 5550785,
    "ModifyIndex": 20803436
}, {
    "Node": "ip-172-30-2-12",
    "Address": "172.30.2.12",
    "ServiceID": "ip-172-30-2-12:insane_kirch:8443",
    "ServiceName": "service1",
    "ServiceTags": ["tag1","tag2"],
    "ServiceAddress": "2.2.2.2",
    "ServicePort": 24138,
    "ServiceEnableTagOverride": false,
    "CreateIndex": 5550914,
    "ModifyIndex": 20803449
}, {
    "Node": "ip-172-30-2-13",
    "Address": "172.30.2.13",
    "ServiceID": "ip-172-30-2-13:happy_shockley:8443",
    "ServiceName": "service1",
    "ServiceTags": ["tag1","tag2"],
    "ServiceAddress": "3.3.3.3",
    "ServicePort": 24338,
    "ServiceEnableTagOverride": false,
    "CreateIndex": 5550996,
    "ModifyIndex": 20803432
}]

Need to parse json output from consul and get pool of :
"ServiceAddress": "ip/hostname"
"ServicePort": port

from there.

Configuration options:

consul_endpoint=”some_host_name_or_ip:port”
consul_service_name=”service_name”
consul_service_tag=”tag” #optional
consul_api_service_endpoint=/v1/catalog/service/[servicename] #optional
consul_api_service_tls_enabled = false # (optional) enable client tls auth
consul_api_service_tls_cert_path = '/path/to/cert.pem'
consul_api_service_tls_key_path = '/path/to/key.pem'
consul_api_service_tls_cacert_path = '/path/to/cacert.pem'

healthcheck timeout should be necessarily defined and non-zero

in each healthcheck timeout should be defined and non-zero to prevent errors like this:

2016-07-14 20:29:08 INFO : Responce from exec [/home/ubuntu/lb/exec_healthcheck.sh 54.229.38.77 23756] is timed out. Killing process...
2016-07-14 20:29:08 INFO : Responce from exec [/home/ubuntu/lb/exec_healthcheck.sh 54.229.214.152 22399] is timed out. Killing process...
2016-07-14 20:29:08 INFO : Responce from exec [/home/ubuntu/lb/exec_healthcheck.sh 54.229.214.152 43999] is timed out. Killing process...
2016-07-14 20:29:08 INFO : Responce from exec [/home/ubuntu/lb/exec_healthcheck.sh 54.229.214.152 43989] is timed out. Killing process...
2016-07-14 20:29:08 INFO : Responce from exec [/home/ubuntu/lb/exec_healthcheck.sh 54.229.38.77 32399] is timed out. Killing process...

SNI support for proxying

Hi,
Is there a plan in the near future to implement SNI based routing?

Regards,
Shantanu

BTW:
Awesome software which I discovered only by accident!!!
The Consul discovery backend is truly great!!!
👍 👍

Implement TLS Termination

A TLS/SSL termination means handling incoming TLS connections, decrypting the TLS and passing on the unencrypted data to the backends by TCP.

protocol = "tls"

#  [servers.default.tls]             # (required) if protocol == "tls"
#  cert_path = "/path/to/file.crt"   # (required) path to crt file
#  key_path = "/path/to/file.key"    # (required) path to key file
#  min_version = "tls1"              # (optional) "ssl3" | "tls1" | "tls1.1" | "tls1.2" - minimum allowed tls version
#  max_version = "tls1.2"            # (optional) maximum allowed tls version
#  ciphers = []                      # (optional) list of supported ciphers. Empty means all supported. For a list see https://golang.org/pkg/crypto/tls/#pkg-constants
#  prefer_server_ciphers = false     # (optional) if true server selects server's most preferred cipher
#  session_tickets = true            # (optional) if true enables session tickets

Add support for multiple endpoints depending on labeling

Using Docker Discovery Backend, Is there any way to consume different services running same internal container port and different label for example, accessing from same server binded port?.

                    /--containerA:80 LABEL_A
CLIENT ---> HOST:3000
                    \--containerB:80 LABEL_B

Maybe this is not the purpose of gobetween.
Many Thanks and Regards,
Javier R

issue with "roundrobin" balancing type

with balance = "weight" everything works fine
but with balance = "roundrobin" gobetween won`t start.

please see config and error below:

[logging]
  level = "info"
  output = "/var/log/gobetween.log"

[api]
  enabled = true
  bind = "10.0.0.8:8888"

[defaults]
  max_connections = 0
  client_idle_timeout = "0"
  backend_idle_timeout = "0"
  backend_connection_timeout = "0"

[servers]
  [servers.es]
    max_connections = 10000
    client_idle_timeout = "10m"
    backend_idle_timeout = "10m"
    backend_connection_timeout = "2s"
    bind = "10.0.0.8:9200"
    protocol = "tcp"
    balance = "weight"
    [servers.es.discovery]
      kind = "static"
      failpolicy = "keeplast"
      interval = "0"
      timeout = "0"
      static_list = ["10.0.0.51:9200", "10.0.0.52:9200", "10.0.0.53:9200", "10.0.0.54:9200", "10.0.0.55:9200"]
    [servers.es.healthcheck]
      kind = "ping"
      interval = "1s"
      passes = 3
      fails = 3
      timeout = "500ms"
root@??:~# /usr/sbin/gobetween -c /etc/gobetween.toml
2016-07-10 14:36:47 [INFO ] (main): gobetween v0.2.0+dev
2016-07-10 14:36:47 [INFO ] (main): Using config file /etc/gobetween.toml
panic: interface conversion: *balance.RoundrobinBalancer is not balance.Balancer: missing method Elect

goroutine 6 [running]:
panic(0xa3b8e0, 0xc82012fe40)
    /usr/local/go/src/runtime/panic.go:464 +0x3e6
_/home/ubuntu/git_projects/gobetween/src/balance.New(0xc8201b4d80, 0xa, 0x0, 0x0)
    /home/ubuntu/git_projects/gobetween/src/balance/balancer.go:33 +0x17f
_/home/ubuntu/git_projects/gobetween/src/server.New(0xc8201b95c8, 0xd, 0xc8201b5bf0, 0xc8201b5bb0, 0xc8201c4240, 0xc8201c4260, 0xc8201b4c80, 0xd, 0xc8201b4d00, 0x3, ...)
    /home/ubuntu/git_projects/gobetween/src/server/server.go:74 +0x13a
_/home/ubuntu/git_projects/gobetween/src/manager.Create(0xc8201b95c8, 0xd, 0xc8201b5bf0, 0xc8201b5bb0, 0xc8201c4240, 0xc8201c4260, 0xc8201b4c80, 0xd, 0xc8201b4d00, 0x3, ...)
    /home/ubuntu/git_projects/gobetween/src/manager/manager.go:121 +0x160
_/home/ubuntu/git_projects/gobetween/src/manager.Initialize(0xc8201b4310, 0x4, 0xc8201b43d0, 0x7, 0x1, 0xc8201b4560, 0xd, 0xc8201b5a10, 0xc8201b59b0, 0xc8201b59d0, ...)
    /home/ubuntu/git_projects/gobetween/src/manager/manager.go:46 +0x27b
created by main.main
    /home/ubuntu/git_projects/gobetween/src/main.go:83 +0x4f3

In some cases SRV discovery returns backend with empty port

During SRV discovery one or more services have empty port key in backend definition:

{
  "host": "172.16.160.67",
  "port": "",
    "priority": 0,
    "stats": {
      "active_connections": 0,
      "live": false,
      "rx": 0,
      "rx_second": 0,
      "total_connections": 0,
      "tx": 0,
      "tx_second": 0
    },
    "weight": 0
}

Manual lookup returns correct SRV records:

;; ANSWER SECTION:
nginx.service.consul.   0   IN  SRV 1 1 9802 m-slave-01.ua.int.node.ua.consul.
nginx.service.consul.   0   IN  SRV 1 1 26661 m-slave-02.ua.int.node.ua.consul.
nginx.service.consul.   0   IN  SRV 1 1 57048 m-slave-04.ua.int.node.ua.consul.

;; ADDITIONAL SECTION:
m-slave-01.ua.int.node.ua.consul. 0 IN  A   172.16.160.50
m-slave-02.ua.int.node.ua.consul. 0 IN  A   172.16.160.60
m-slave-04.ua.int.node.ua.consul. 0 IN  A   172.16.160.143

Implement TLS Proxying

Allow to configure TLS intead of TCP while proxying traffic to backends.

Configuration options may be similar to nginx.

Which swarm version? / 1.12 have major implications

Does this project support 1.12+ ? As there is no mention of it, I guess it's not.

I'm still looking for a solid reverse-proxy under Docker Swarm 1.12+ (using 1.12.1 at the moment). Here some [solutions](Does this project support 1.12+ ?) meanwhile.

Cheers!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.