fabiolb / fabio Goto Github PK

View Code? Open in Web Editor NEW

7.3K 212.0 621.0 20.39 MB

Consul Load-Balancing made simple

Home Page: https://fabiolb.net

License: MIT License

Go 97.55% Shell 0.90% Makefile 1.17% Dockerfile 0.16% CSS 0.16% Vim Script 0.03% Perl 0.02%

go golang load-balancer consul vault docker

fabio's Introduction

Notes

From release 1.6.1 onward, the minimum golang version supported is 1.16.
From release 1.6.0 onward, metrics backend statsd is no longer supported. statsd_raw works similarly, though it actually resets counters appropriately. If you are using datadog, you should consider using the new dogstatsd backend, which has support for tags now. Graphite histogram functionality has changed slightly since switching to gokit framework, so something to be aware of. Prometheus functionality is now supported natively.
From release 1.5.15 onward, fabio changes the default GOGC from 800 back to the golang default of 100. Apparently this made some sense back in the golang 1.5 days, but with changes introduced with golang 1.12 and others, this is probably no longer a very good default. This is still configurable, as always, but the new default should make the most sense for most users.
From release 1.5.14, release hashes are signed with a new PGP key. See details here.
From release 1.5.14 onward, fabio binary releases are compiled with golang 1.15+.
This means that the fabio will no longer validate upstream https certificates that do not have SAN extensions matching the server name. This may be a concern if fabio is communicating with https backends with misconfigured certificates. If this is a problem, you can specify tlsskipverify=true on the route.

fabio is a fast, modern, zero-conf load balancing HTTP(S) and TCP router for deploying applications managed by consul.

Register your services in consul, provide a health check and fabio will start routing traffic to them. No configuration required. Deployment, upgrading and refactoring has never been easier.

fabio is developed and maintained by The Fabio Authors.

It powers some of the largest websites in Australia (gumtree.com.au) and Italy (www.kijiji.it). It delivers 23.000 req/sec every day since Sep 2015 without problems.

It integrates with Consul, Vault, Amazon ELB, Amazon API Gateway and more.

It supports (Full feature list)

TLS termination with dynamic certificate stores
Raw TCP proxy
TCP+SNI proxy for full end-to-end TLS without decryption
HTTPS+TCP+SNI proxy for TCP+SNI with HTTPS fallback
TCP dynamic proxy
HTTPS upstream support
Websockets and
SSE
Dynamic reloading without restart
Traffic shaping for "blue/green" deployments,
Prometheus,
Circonus,
Graphite,
StatsD,
DataDog for metrics,
WebUI and
Advertising BGP anycast addresses on non-windows platforms.

Watch Kelsey Hightower demo Consul, Nomad, Vault and fabio at HashiConf EU 2016.

The full documentation is on fabiolb.net

Getting started

Install from source, binary, Docker or Homebrew.

# go 1.15 or higher is required
go install github.com/fabiolb/fabio@latest          (>= go1.15)

brew install fabio                                  (OSX/macOS stable)
brew install --devel fabio                          (OSX/macOS devel)

docker pull fabiolb/fabio                           (Docker)

https://github.com/fabiolb/fabio/releases           (pre-built binaries)

Register your service in consul.

Make sure that each instance registers with a unique ServiceID and a service name without spaces.
Register a health check in consul as described here.

By default fabio only watches services which have a passing health check, unless overridden with registry.consul.service.status.
Register one urlprefix- tag per host/path prefix it serves, e.g.:

# HTTP/S examples
urlprefix-/css                                     # path route
urlprefix-i.com/static                             # host specific path route
urlprefix-mysite.com/                              # host specific catch all route
urlprefix-/foo/bar strip=/foo                      # path stripping (forward '/bar' to upstream)
urlprefix-/foo/bar proto=https                     # HTTPS upstream
urlprefix-/foo/bar proto=https tlsskipverify=true  # HTTPS upstream and self-signed cert

# TCP examples
urlprefix-:3306 proto=tcp                          # route external port 3306

Make sure the prefix for HTTP routes contains at least one slash (/).

See the full list of options in the Documentation.

Start fabio without a config file (assuming a running consul agent on localhost:8500) Watch the log output how fabio picks up the route to your service. Try starting/stopping your service to see how the routing table changes instantly.
Send all your HTTP traffic to fabio on port 9999. For TCP proxying see TCP proxy.
Done

Author and Founder

Frank Schroeder @magiconair

Maintainers

Contributors

This project exists thanks to all the people who contribute. [Contribute].

License

Contributions up to 14 Apr 2017 before 38f73da

MIT Licensed Copyright (c) 2017 eBay Software Foundation. All rights reserved.
Contributions after 14 Apr 2017 starting with 38f73da

MIT Licensed Copyright (c) 2017-2019 Frank Schroeder. All rights reserved.
Contributions after 22 Jan 2020 starting with 9da7b1b

MIT Licensed Copyright (c) 2020 Education Networks of America. All rights reserved.

See LICENSE for details.

fabio's People

Contributors

Stargazers

Watchers

Forkers

cinnabardk cinderalla creantan ad3n jeinwag eaglejava pnadig agilemobiledev geezer-workshop garybyrd alexzhang2015 gitssie sckevmit ryanbreen juliendsv mengjinglei yilab is00hcw dmeulen sydsvenskan gooops hidu handong890 junjie0510 langlearn gale320 thurday ligzy sinozozo emicklei bobbwu gitforks chenbk85 sxhao duanshuaimin jobell superseb muemich favoretti lanshan-studio hilerchyn ishawge celeskyking hafeez3000 doublerebel michaelandersen siathalysedi hinike simudream ruiaylin anilprasad jblackburn21 charloco twkun spierman zengzanda sguzwf volvoxjs huayl ashwanthkumar brewkode ccasado einyx gude devopstiger no2key jonmorehouse smancke ddelautre flyingliang sielaq ikerlan-digital md2k gthurm henrypfhu jsmancer robin-- alexsnet uoko-j-go zhangmuxi kmichaelkills jmheidly gavinhwa koulio libertyglobal bigr-lab dazer-chen devopsbox svdberg evandhoffman clearbit-boneyard cursesun shinexiao wsf1991 izogain maple603 kerasking fabiofumarola madeddie yexiaoming20140413

fabio's Issues

Externalize registry backends

Fabio is heavily reliant on Consul for service registration, discovery and heartbeat checking. This is a PULL situation. It would be nice if Fabio could complete it's RESTful API as a generic registry interface that could provide a PUSH function and remove any internal dependencies.
This is outside of API request in the Issues list of Fabio.

How would this work?

Let's go to the PowerPoint:

An external registry (whatever it might be) would call the fabio api and register a service. The request would contain all the useful information for the "Add route for service svc from src to dst and assign tags" command but in json format. Included in the json request would be an additional callback attribute string to either a URL or WS. For example:
"serviceCheck":"http://192.168.2.7:80/is_service_up/19382493"
or
"serviceCheck":"ws://192.168.2.7:1234/19382493"
or generically
"serviceCheck":"http://192.168.2.7:80/health"

Fabio doesn't care too much about the value.

Fabio would register this request in it's table.
Fabio would ws attach or use the serviceCheck URL to periodically see if the registry or service is still alive. (in case the registry itself went down or the registry failed in it's attempt to issue an unregister call to Fabio)
An external registry would call fabio occasionally to deregister a service that was no longer available. The request would contain all the useful information for the route del command.
Fabio would deregister this service from the Table.

"route weight" could also be provided as apart of the API.

This would provide a means for any external registry to utilize Fabio instead of having a dependency on only Consul or having to compile in another pkg with the type Backend interface to support an interface. Services themselves might even be able to register to Fabio directly without middleware as long as they met the API requirement.

Support custom 404/503 error pages

Hi again :)

It could be awesome to specify file or/and use consul key,
that will be used as an answer of 404 or 503 pages.

file or key value can be same as HAproxy do:

HTTP/1.0 404 NOK^M
Cache-Control: no-cache^M
Connection: close^M
Content-Type: text/html^M
^M
<html>
...
</html>

Case-insensitive path matching

I have added this route to test with, but the path matching appears to be case-sensitive.

route add orders /orders http://localhost:8080/ tags "urlprefix-/orders"

A request to this url works correctly: http://localhost:9999/orders

However, a request to this URL: http://localhost:9999/Orders
logs this warning: [WARN] No route for localhost:9999/Orders

Is it possible for the path matching to be case-insensitive? This would lessen the burden on the client to request the route with matching case.

server demo: Consul health check fails

Edit: Apparently the consul inside container cannot reach the outside...
When I let server listen on the docker host ip address seen from inside the consul docker container,
it works.

I run the server demo and let it connect to consul in a container:

./server -addr 127.0.0.1:5000 -name svc-a -prefix domain.com/foo -consul 127.0.0.1:8500

2015/12/07 16:17:49 Listening on 127.0.0.1:5000 serving domain.com/foo
2015/12/07 16:17:49 Registered service "svc-a" in consul with tags "urlprefix-domain.com/foo"

The new service is listed in consul web GUI,
however, although it listens on port 5000, it doesn't pass the consul health check,
therefore it is not listed in the routing table in fabio web GUI.

Service 'svc-a' check service:svc-a-127.0.0.1:5000 critical
NOTES

OUTPUT

Get http://127.0.0.1:5000/health: dial tcp 127.0.0.1:5000: connection refused

Show deleted routes

Currently if I have a service bla-bla-service which exposes route "bla-bla-path", if I remove it with route del bla-bla-service override, then it does not show on the Route page in UI, and then it is unclear whether it is not correctly exposed by the service or removed by the overrides.

I believe it would be more user friendly to show it with 0% weight, instead of hiding it alltogether.

Transparent urlprefix?

My understanding is that Fabio requires you to register URL prefixes that are actually present in the receiving services.
e.g.
if you have a service running on http://localhost:1234/foo then you can use urlprefix-/foo , correct?

Is there any chance that Fabio could support "transparent" prefixes?
A sort of prefix that is used to let Fabio resolve what service you want, but is not used when talking to the service.
lets say I make a request to Fabio on http://localhost:9999/OrderService/api/orders/123 that could be resolved to just http://localhost:1234/api/orders/123
That is the "OrderService" prefix is removed.

That way, I would not have to specify each and every path my service responds to, and it would avoid ambiguity between services, e.g. if two or more services both listen to /api but are different services.

I might have misunderstood something and this might not make any sense, but the current behavior seems to require me to register every path explicitly instead of registering capability/service name.

Support Forwarded and X-Forwarded-For headers

fabio currently supports only a simple version of the X-Forwarded-For headers which only contains the remote ip address. The X-Forwarded-For header and the newer Forwarded header have more complex formats and also require the detection of the local ip address.

I think it should be changed as follows:

When the proxy.header.clientip header value is set to X-Forwarded-For then it should behave like it, i.e. set it to <remote ip>, <local ip> if not set or add , <local ip> if already set.

In addition the Forwarded header should be honored every time, i.e. set it if it hasn't been set or augment it if was already set.

I'm still thinking whether a force option is necessary to prevent spoofing the remote ip.

The 100-Continue mechanism doesn't work correctly

The 100-Continue mechanism doesn't seem to work correctly when forwarding requests to the upstream server.

This is visible in particular in failure cases - e.g. 401 Unauthorized

Current behaviour

Client sends a POST request with a relatively big request body and the Expect: 100-continue header but with incorrect basic-auth credentials (Authorization header)
Fabio doesn't include the Expect header in the http request forwarded to the upstream server
Fabio sends back to the client HTTP/1.1 100 Continue before it gets any feedback from the upstream server
Client starts uploading the request body
Fabio starts streaming the request body to the upstream server
Upstream server responds with HTTP/1.1 401 Unauthorized
Fabio forwards the HTTP/1.1 401 Unauthorized back to the client

It "works" from the client perspective as finally it gets the HTTP/1.1 401 Unauthorized response.

The problem is that the client should never start uploading the request body in this scenario which is the main point of the 100-Continue mechanism.

Expected behaviour error case
As per https://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html (section 8.2.3 Use of the 100 (Continue) Status)

Client sends a POST request with a relatively big request body and the Expect: 100-continue header but with incorrect basic-auth credentials (Authorization header)
Fabio forwards the request to the upstream server including the Expect header
Upstream server responds with HTTP/1.1 401 Unauthorized
Fabio forwards the HTTP/1.1 401 Unauthorized back to the client

Expected behaviour success case

Client sends a POST request with a relatively big request body and the Expect: 100-continue header and with valid Authorization header
Fabio forwards the request to the upstream server including the Expect header
Upstream server responds with HTTP/1.1 100 Continue
Fabio forwards the HTTP/1.1 100 Continue back to the client
Client starts uploading the request body
Fabio starts streaming the request body to the upstream server
Upstream server responds with HTTP/1.1 201 Created
Fabio forwards the HTTP/1.1 201 Created back to the client

ui needs authentication

I am using fabio for public facing services therefore I would like the ui to live on https with some authentication scheme such as basic or digest.

Traffic shaping does not match on service name

The route weight command accepts a service name but does not use it when assigning weights to the targets. The current implementation matches on tags only.

The matching should happen as follows:

# assign weight w to src on service svc
route weight svc src weight w

# assign weight w to src on services with tags t1 and t2
route weight src weight w tags "t1,t2"

# assign weight w to src on service svc with tags t1 and t2
route weight svc src weight w tags "t1,t2"

Move dependencies to vendor path

Go 1.6 is around the corner and the dependencies should move from _third_party to vendor. I'll give the govendor tool a try for the heavy lifting.

TCP proxying support

This looks super cool. Wanted to send along a feature request that I think will make it even more useful.

At a previous job, we were using haproxy due to its ability to do not only straight TCP proxying, but straight TCP proxying while honoring the SNI header. This allows the TLS connection to be direct between a client and backend server, rather than having a proxy in the middle decrypt traffic and re-encrypt traffic to the backend. This direct connection is needed to avoid transitive trust issues, and can be especially important if you are required to adhere to various industry conformance standards like PCI.

So my feature request is: please allow straight TCP proxying while honoring the SNI header for routing.

Demo (shebang, documentation)

I installed golang on Debian Jessie (Package golang) and tried to run the server demo,
using this command as described in the docs:

$ cd demo/server
$ ./server

It is not interpreted as go:

./server.go: line 1: //: Is a directory
./server.go: line 2: //: Is a directory
./server.go: line 3: //: Is a directory
[...]

The problem can be fixed by prepending a shebang for go to the go files
(from http://stackoverflow.com/questions/7707178/whats-the-appropriate-go-shebang-line):

//usr/bin/go run $0 $@ ; exit
[...]

chmod +x may be needed, too to make it executable:

$ chmod +x ./demo/server/server.go
$ chmod +x ./demo/wsclient/wsclient.go

Also some go packages have to be installed before the demos can be tried,
this may be useful being documented, too:

(enter the ./demo directory first)
$ export GOPATH=$(pwd)

# for the server or wsclient demo
$ go get golang.org/x/net/websocket
$ mkdir -p src/github.com/eBay/fabio/_third_party
$ mv src/golang.org/ src/github.com/eBay/fabio/_third_party/


# specifically for the server demo
$ go get github.com/hashicorp/go-cleanhttp

$ go get github.com/hashicorp/consul
$ go get github.com/hashicorp/consul/api
$ mkdir -p src/github.com/eBay/fabio/_third_party/github.com
$ mv src/github.com/hashicorp/consul/ src/github.com/eBay/fabio/_third_party/github.com/hashicorp/

Direct server return?

Hi!
Does fabio support direct server return? If not, is this a feature that will be included later?

Detect when consul agent is down

Fabio does not detect when a consul agent is down properly only when a node or service is in maintenance mode. I've got a fix ready but need to test it first.

(Docker) Error initializing backend.

When starting fabio as docker container using the following command

docker run -d -p 9999:9999 -p 9998:9998 -v $PWD/fabio/fabio.properties:/etc/fabio/fabio.properties magiconair/fabio:1.0.6

the container will exit and I get this error in docker logs:

2015/12/06 21:42:01 [INFO] Version 1.0.6 starting
2015/12/06 21:42:01 [FATAL] Error initializing backend. Get http://localhost:8500/v1/agent/self: dial tcp [::1]:8500: getsockopt: connection refused

Include services with check ids other than 'service:*'

When you register a health check together with a service in consul the health check gets an id of service:name-node-port. Fabio currently relies on the service: prefix to tell service checks from node and maintenance checks but this is not reliable since you can register health checks for services separately and there you can specify your own check id.

If your service does not show up in fabio although you have a passing health check verify that the check id starts with service: until this is fixed. Run curl localhost:8500/v1/health/state/any to see all health checks.

Add/remove certificates using API

From what I understand, new certificates can be only set up using the fabio properties file.
When adding/removing apps or updating the PEM files, I would find it very helpful when fabio supports management of certificates using its API or even consul tags.

Support API Gateway for microservices

In order to serve microservices, it's better for API gateway to have such features:

Circuit breaker
Isolation to avoid of cascaded failures
Multiple protocol supports other than HTTP
Multiple API calls and aggregation in a single request
The existing alternative most close to above description is Netflix Zuul which has adopted Hystrix for the former two feature to provide resilent reverse proxy.

Support Google Compute Platform

@emicklei is working on a registry backend for GCP.

Allow configuration of serviceip used during consul registration

Hi,

currently, serviceRegistration is called with an addr equal to ui.addr, which defaults to ":9998". When not configured (i.e. as per default), the ipaddr part is filled in the the local ip (from config.LocalIP()), which, when running inside a container, means that fabio registers the container's IP in consul, which seems incorrect/unnecessary, as other hosts would not be able to get to that address.

The initial potential workaround seemed to me to change ui.addr to the externally-accessible IP, but that doesn't work as fabio attempts to bind to that IP, which is non-local within the container, and everything falls over (note I am trying to avoid running the container with net=host)

The best option seems to be to introduce a variable such as consul.register.ip (the partner to consul.register.name) and allow the registered/serviceip to be overridden that way. An alternative may also be to set this to the same value as proxy.localip.

As I need to get to a working configuration in this situation, I'm going to go ahead and take a stab at implementation if needed, so just checking what makes more sense, or if I'm missing something obvious.

Latest fabio docker image fails consul check

In previous version all worked fine. Now in atlas fabio at 9998 marked as unhealthy and i see this in logs:

2016/02/20 08:55:06 [WARN] agent: http request failed 'http://:9998/health': Get http://%3Cnil%3E:9998/health: dial tcp: lookup : invalid domain name

Allow configuration via env vars

Since fabio is designed to be near-zero configuration, I assumed there must be a way to set (at least the main) options with environment variables. So I went looking, and there is! Multiple command line flags are available: -addr, -consul, -name, -prefix, -proto, -token 1. However, these are not documented in the binary (in --help) or in the fabio docs.

I am happy to write the docs but wanted to leave a note here until then, if anyone else is looking for the same thing.

HTTP2 support with latest Go

So there is a Golang nightly image on Dockerhub and I tried it out and built fabio with it.

And indeed - HTTP2 support seems to work out of the box - except one issue with the routing (see end).

Here are the steps to build fabio with latest Go for HTTP2 support + testing it out.
(You need four terminals, e.g. tmux tabs for comfortably doing this, you can also see all the logs then):

1st step / 1st terminal:

# Run consul standalone container:
docker run -p 8500:8500  gliderlabs/consul-server:0.5 -data-dir /tmp/consul -bootstrap-expect 1 -server

2nd step / 2nd terminal:

# Create test folder
cd ~/src \
&& mkdir fabio-build && cd fabio-build

# Build fabio using latest Go (for HTTP2 support built in):
docker run --rm -v $(pwd):/root/src strarsis/golang-nightly:build-1.5.2-nightly-57c81ef2570ac896fa5d830effbf2494d5cde3e5 /bin/sh -c \
'cd /root/src && export GOPATH=$(pwd) && go get -u github.com/eBay/fabio'

# Generate dummy certificates for fabio HTTPS
# *Most if not all HTTP2 clients require working TLS/SSL.
mkdir tls && cd tls \
&& openssl genrsa -des3 -out self-ssl.key -passout pass:test 2048 \
&& openssl req -new -key self-ssl.key -out self-ssl.csr -subj "/C=/ST=/L=/O=/CN=test.com" -passin pass:test \
&& cp -v self-ssl.{key,original} \
&& openssl rsa -in self-ssl.original -out self-ssl.key -passin pass:test \
&& rm -v self-ssl.original \
&& openssl x509 -req -days 365 -in self-ssl.csr -signkey self-ssl.key -out self-ssl.crt

# Configure + start fabio
cd .. \
&& echo 'proxy.addr = :9999,:9993;./tls/self-ssl.crt;./tls/self-ssl.key' > ./fabio.properties \
&& ./bin/fabio -cfg fabio.properties

3rd step / 3rd terminal:

# Build demo server
cd ~/src/fabio-build \
&& docker run --rm -v $(pwd):/root/src strarsis/golang-nightly:build-1.6beta1-nightly-e2093cdeef8dcf0303ce3d8e79247c71ed53507d /bin/sh -c \
'cd /root/src && export GOPATH=$(pwd) && go build src/github.com/eBay/fabio/demo/server/server.go'

# Run demo server
./server -addr 172.17.0.1:5000 -prefix test.com/test

4th step / 4th terminal:

# Install + run nghttp (HTTP2 client)
sudo apt-get install -y nghttp2
nghttp --header='Host:test.com' https://127.0.0.1:9993/test

Note that 9993 is used as port for HTTPS so fabio can run as non-root,
it will still work the same as with port 443.

There is no error message from nghttp, so the HTTP2 request is indeed working,
just no response (or 404?) - fabio (in 2nd terminal) logs the following:

No route for 127.0.0.1:9993/test

Health check for the demo server is OK and fabio route is set,
probably some special case to take into account when processing HTTP2 requests?

Go 1.6 Milestone: https://github.com/golang/go/milestones/Go1.6
Or will HTTP2 be already included in some nearer Go 1.5.x release?

Support additional backends

It would be pretty great if this had a backend that could read from Apache Mesos (or alternatively, Marathon) in addition to consul. There's a great project in the space called Traefik which will do HTTP in a very similar fashion but the only solutions I've seen for straight TCP have revolved around scripted HAProxy which is pretty ugly.

Refactor listener config

The listener configuration in proxy.addr seemed like a good idea but wasn't. To provide parameters for read and write timeout and reuse the same configuration also for the UI (see issue #25) I'm planning to change the configuration of the listeners as follows:

# proxy.listen configures one or more listeners for the reverse proxy.
#
# The default is to create a single HTTP listener on :9999 without
# authentication. Listeners which have a valid certificate are on HTTPS. If the
# certificate file contains both the certificate and the private key then the
# 'key' parameter can be omitted. For HTTPS connections X509 client certificate
# authentication can be configured which is set to RequireAndVerifyCert when
# enabled.
#
# All certificates and keys must be in PEM format.
#
# To configure the listener you can set the following options:
#
# proxy.listen.<name>.addr; listen address
# proxy.listen.<name>.tlsCert: path to TLS certificate file
# proxy.listen.<name>.tlsKey: path to TLS key file
# proxy.listen.<name>.clientCert: path to client certficate file
# proxy.listen.<name>.readTimeout: read timeout
# proxy.listen.<name>.writeTimeout: write timeout

The same options will be available for the UI/API listener and the same code will start all listeners.

proxy.addr and ui.addr will be deprecated with a startup warning and removed in a later release.

Easy way of running fabio server in tests

When testing microservices, it would be nice to have an ability to run fabio server and register urls manually.

Is there any easy way doing it?

Support websockets

Some people have been asking about websocket support for fabio. I think it shouldn't be too hard to add but I need to see when I can get to it. Keeping this as a placeholder so that I don't forget.

Feature: Allow Wildcard Path on Host/path

Currently, as far as I can tell, routes need to be explicit for a given host, hence loading is very heavy because every route needs to be configured for a service.

What would be nice is a wildcard so only the host is taken into account and all traffic is redirected to that host:

For example, any of these requests:

http://fabio_server_ip:9999/foo.example.com/v1.0/about
http://fabio_server_ip:9999/foo.example.com/v1.0/login

would be redirected here:

foo.example.com/*
        foo_1                  10.0.0.2:32789
        foo_2                  10.0.0.3:32872

so foo_1 and 2 would receive this:

10.0.0.2:32789/v1.0/about
10.0.0.3:32872/v1.0/login

If there is a way to do this now, then I don't see it in the code or docs. Please explain.

I have to either be explicit or do this shit:

http://fabio_server_ip:9999/foo/v1.0/about
http://fabio_server_ip:9999/foo/v1.0/login

would be handled here:

/foo
        foo_1                  10.0.0.2:32789
        foo_2                  10.0.0.3:32872

and foo_1 and 2 receive this:

10.0.0.2:32789/foo/v1.0/about
10.0.0.3:32872/foo/v1.0/login

so now the server has to account for the hostname in the path which is BS.

Add support for Consul ACL token to demo server

Server in /demo/server needs support for Consul ACL tokens :)

Thanks!

Consul ACL Token

I failed to find how can i add consul token to fabio config/command line.

Is it somehow possible?

Deleted routes hide visible routes

All versions of fabio have the following bug:

If you delete all routes of a service with the route del service manual override then paths registered by this service are still considered for routing.

Given this routing table

route add serviceA /foo http://127.0.0.1:5000/
route add serviceB /foo/bar http://127.0.0.1:5001/
route del serviceB

the log will print the correct routing table

route add serviceA /foo http://127.0.0.1:5000/

but internally there is still a route for /foo/bar with no targets which is evaluated during lookup. Therefore, a request for /foo/bar will not be routed to serviceA but to nowhere since the dangling route has no targets.

The current workaround is to add a manual override for the specific route to serviceA, e.g.

route add serviceA /foo/bar http://127.0.0.1:5000/

Expand ${DC} to consul datacenter

Hi,

we need possibility to use variables like service_name.service.$DC.$DOMAIN in tag,
so it will solve issue when having same plans for multiple datacenters and using consul domains.

This can be solved also in other way around. Adding new fabio.property use.consul.domain or tag
(to use consul domains automatically).
for example:

input:
service: consul
datacenters: ams, fra
possible consul domains: consul.service.consul, consul.service.ams.consul, consul.service.fra.consul
tag: use_consul_domain
result:
fabio generate 3 routes:
consul.service.consul, consul.service.ams.consul, consul.service.fra.consul that routes to same consul service

Fabio not removing nodes with failed health check

Fabio doesn't seem to be removing nodes/routes which have a failed healthcheck. It seems to notice that the health of the service has changed, but keeps trying to send requests to the failed node (resulting in a 'connection refused').
Only when all nodes are failed, the routes are removed. Adding one of the nodes back, puts all nodes back in the routes, resulting in intermitted 'connection refused' responses again.
DNS lookups to consul return the expected result, which are just the passing nodes.

Support SSL/TLS client cert authentication

Add an option to support client cert authentication. This support authenticated calls from the Amazon API Gateway and other use cases.

Filter routing table not on tags

The routing table UI needs an overhaul for sure but what's bugging me most right now is that the filtering is not persisted during page reloads and that the filtering is on tags, too which makes it mostly useless. I'll plan to have a slightly improved version in 1.0.6 and a more useful UI in a later version.

Session persistence

In nginx Plus, one can use Session Persistence for load balancing.
For some apps, their data/state (session data) is not really separated
(e.g. PHP Sessions without special backend, legacy apps).
In that case, requests for a session should always go to the same backend (route).

Also when fabio instances are also used as backends (nested upstream),
Session Draining would be helpful, too, for the same kind of apps.

As session detection can be very different, some kind of go callback offered by fabio
would allow the admin to tailor it, like passing in a dump of HTTP headers and
the go callback would return a session ID for fabio to decide.

fabio route not removed after consul deregister

When running the demo server, it registers itself in consul, being listed by consul as healthy.

$ curl http://localhost:8500/v1/catalog/services

{"consul":[],"consul-server-8500":[],"fabio":[],"fabio-9998":[],"fabio-9999":[],"svc-a":["urlprefix-domain.com/foo"]}

$ curl http://localhost:8500/v1/catalog/service/svc-a

[{"Node":"74bd1b630c8d","Address":"172.17.0.2","ServiceID":"svc-a-172.17.0.1:5000","ServiceName":"svc-a","ServiceTags":["urlprefix-domain.com/foo"],"ServiceAddress":"172.17.0.1","ServicePort":5000}]

Fabio also correctly adds the route to routing table, also listed in fabio web gui.

1   svc-a   domain.com  /foo    http://172.17.0.1:5000/ 100%

When the demo server is shut down (Ctrl+c), it reports that it deregistered itself in consul:

2015/12/07 18:35:16 Deregistered service "svc-a" in consul

And indeed, consul doesn't list the demo server anymore:

$ curl http://localhost:8500/v1/catalog/services

{"consul":[],"consul-server-8500":[],"fabio":[],"fabio-9998":[],"fabio-9999":[]}

$ curl http://localhost:8500/v1/catalog/service/svc-a

[]

However, in fabio web gui, the route is still listed:

1   svc-a   domain.com  /foo    http://172.17.0.1:5000/ 100%

List among Consul community tools

Great software ⛵

This is a good fit for https://consul.io/downloads_tools.html.

Support Docker API

Investigate how complex support of the Docker API would be

routes not removed when passing empty string

When I send an empty string on the WatchServices channel, I expect all dynamic routes to be removed.

Build fails on master/last release tag

I try to build fabio from source but it fails on master and on the last release tag (v1.0.7).

Steps to (probably) reproduce this:

$ git clone https://github.com/eBay/fabio.git
$ cd fabio

$ git checkout v1.0.7
$ git reset --hard

$ export GOPATH=$(pwd) # is set!

$ go get github.com/eBay/fabio

During go get I get this error:

[...]
src/github.com/eBay/fabio/route/table.go:15: undefined: atomic.Value
[...]

When I don't run go get, and directly go build, I get the error that it cannot find sources,
hence I run go get first to fix this:

$ go build

main.go:10:2: cannot find package "github.com/eBay/fabio/admin" in any of:
        /usr/lib/go/src/pkg/github.com/eBay/fabio/admin (from $GOROOT)
        /home/build/src/fabio/src/github.com/eBay/fabio/admin (from $GOPATH)
init.go:10:2: cannot find package "github.com/eBay/fabio/config" in any of:
        /usr/lib/go/src/pkg/github.com/eBay/fabio/config (from $GOROOT)
        /home/build/src/fabio/src/github.com/eBay/fabio/config (from $GOPATH)
listen.go:16:2: cannot find package "github.com/eBay/fabio/exit" in any of:
        /usr/lib/go/src/pkg/github.com/eBay/fabio/exit (from $GOROOT)
        /home/build/src/fabio/src/github.com/eBay/fabio/exit (from $GOPATH)
init.go:11:2: cannot find package "github.com/eBay/fabio/metrics" in any of:
        /usr/lib/go/src/pkg/github.com/eBay/fabio/metrics (from $GOROOT)
        /home/build/src/fabio/src/github.com/eBay/fabio/metrics (from $GOPATH)
listen.go:17:2: cannot find package "github.com/eBay/fabio/proxy" in any of:
        /usr/lib/go/src/pkg/github.com/eBay/fabio/proxy (from $GOROOT)
        /home/build/src/fabio/src/github.com/eBay/fabio/proxy (from $GOPATH)
init.go:12:2: cannot find package "github.com/eBay/fabio/registry" in any of:
        /usr/lib/go/src/pkg/github.com/eBay/fabio/registry (from $GOROOT)
        /home/build/src/fabio/src/github.com/eBay/fabio/registry (from $GOPATH)
init.go:13:2: cannot find package "github.com/eBay/fabio/registry/consul" in any of:
        /usr/lib/go/src/pkg/github.com/eBay/fabio/registry/consul (from $GOROOT)
        /home/build/src/fabio/src/github.com/eBay/fabio/registry/consul (from $GOPATH)
init.go:14:2: cannot find package "github.com/eBay/fabio/route" in any of:
        /usr/lib/go/src/pkg/github.com/eBay/fabio/route (from $GOROOT)
        /home/build/src/fabio/src/github.com/eBay/fabio/route (from $GOPATH)

Also when running go build after go get:

[...]
$ go build
[...]

Same error message:

[...]
src/github.com/eBay/fabio/route/table.go:15: undefined: atomic.Value
[...]

No fabio binary is created.

Using go version go1.3.3 linux/amd64) from Debian Jessie stable.

Service is registered in Consul but does not show up in fabio

Hi,
fabio-example works perfectly but I can't register any other service in fabio.
I'm using /v1/agent/service/register HTTP API of Consul to register a service with tag "urlprefix-/hede". Consul says the service is registered but nothing happens at the fabio side.
What else should I check?
fabio v1.0.3, consul 0.5.2

Make read and write timeout configurable

The read and write timeout of the listeners should be configurable. Especially the read timeout should be configurable to prevent HTTP slow attacks.

web sockets and push

is there any support for http/2 yet ?

Why am i asking ?
HTTP/2 means we no longer have to try to push everything into a few http paylaods anymore.
So, it makes it possible to do web components as micro-services :)
Also i do versioning hashing this way, just to give you an idea:


module.exports = {
  baseUrl : 'https://mysite.com/static/',
  resources : {
    vendor : {
      version : 'vendor-d41d8cd98f.js',
      pushWith : [ 'scout' ]
    },
    application : {
      version : 'application-a32e3ec23d.js',
      pushWith : [ 'scout' ]
    },
    secondary : {
      version : 'secondary-e43b8ad12f.js',
      pushWith : [ ]
    }
  }
};

Documentation: make build before running ./fabio

fabio binary has to be built first after go get github.com/eBay/fabio using

make build

It may be useful in documentation.

Feature: Allow weight tag in Consul

"registrator" of paths into consul should also allow weight parameter to be passed so that traffic can be balanced to env needs. Some services may be running on nodes with less resources. Right now, rebalancing is distributed evenly.

Enhancement example:

docker run -d
-e "SERVICE_TAGS=urlprefix-/hello-world:0" 
--name hello_world_1  \

results in...

/hello-world
  /hello_world_1   100%

adding another specific weighted service...

docker run -d
-e "SERVICE_TAGS=urlprefix-/hello-world:20" 
--name hello_world_2  \

would result in...

/hello-world
  /hello_world_1   80%
  /hello_world_2   20%

Cleanup metrics for deleted routes

fabio continues to send metrics to Graphite for routes that have been deleted. It should probably remove them and report only metrics for active routes. It may be useful to consider whether they should be removed right away or through a cleanup process which could smooth over temporary removals during deployments for example.

Strip prefix in the forwarded request

Scenario:

I have fabio set up to listen to http://fabioserver/
I have a route configured for a service, with prefix /myservice

When making a request to http://fabioserver/myservice/index.html, the service currently receives the forwarded request from fabio to /myservice/index.html
I would like to have the option to strip the prefix in fabio, so that the service would receive a request to /index.html

I currently have a homemade go proxy that does that, using http.StripPrefix. I may have the time to prepare a PR for this at some point, but no promises...

Manage manual overrides via UI

Instead of relying on the Consul UI fabio should provide a UI for managing the manual overrides. Other backends may not provide such a UI or it works completely different.

fabiolb / fabio Goto Github PK

fabio's Introduction

Notes

Getting started

Author and Founder

Maintainers

Contributors

License

fabio's People

Contributors

Stargazers

Watchers

Forkers

fabio's Issues

Recommend Projects

Recommend Topics

Recommend Org