Vulcand

• HTTP proxy that uses Etcd as a configuration backend.
• Changes to configuration take effect immediately without restarting the service.

Status

Hardening: Testing, benchmarking. Not usable for production yet.

Discussions

Mailing list: https://groups.google.com/d/forum/vulcan-proxy

Installation

Deps: go>=1.2, Etcd

Install:

make install
make run

Concepts

Host

Incoming requests are matched by their hostname first. Hostname is defined by incoming 'Host' header. E.g. curl http://example.com/alice will be matched by the host example.com first.

Location

Hosts contain one or several locations. Each location defines a path - simply a regular expression that will be matched against request's url. Location contains link to an upstream and vulcand will use the endpoints from this upstream to serve the request.

E.g. location loc1 will serve the request curl http://example.com/search because it matches the path /search and host example.com:

Upstream

Upstream is a collection of endpoints. Upstream can be assigned to multiple locations at the same time. This is convenient as sometimes one endpoint serves multiple purposes and locations.

Endpoint

Endpoint is a final destination of the incoming request, each endpoint is defined by <schema>://<host>:<port>, e.g. http://localhost:5000

Middleware

Vulcand supports pluggable middlewares. Middlewares can intercept or transform the request to any location. Examples of the supported middlewares are rate limits and connection limits. You can add or remove middlewares using command line, API or directly via backends.

Etcd

Vulcan supports reading and updating configuration based on the changes in Etcd. Vulcans watch etcd prefix that is supplied when running an instance and configure themselves. Important All examples bellow assume that Vulcand is configured to listen on /vulcand prefix, which is a default prefix.

Upstreams and endpoints

# Upserts upstream and adds an endpoint to it
etcdctl set /vulcand/upstreams/up1/endpoints/e1 http://localhost:5000

Hosts and locations

# Upsert a host "localhost" and add a location to it that matches path "/home" and uses endpoints from upstream "up1"
etcdctl set /vulcand/hosts/localhost/locations/loc1/path "/home"
etcdctl set /vulcand/hosts/localhost/locations/loc1/upstream up1

The best part is that you can update upstream using the same command. Let's add another upstream and switch traffic to it:

# create a new upstream with endpoint http://localhost:5003
etcdctl set /vulcand/upstreams/up2/endpoints/e3 http://localhost:5003

# redirect the traffic of the location "loc1" to the endpoints of the upstream "up2"
etcdctl set /vulcand/hosts/localhost/locations/loc1/upstream up2

Note that you can add and remove endpoints to the existing upstream, and vulcan will start redirecting the traffic to them automatically:

# Add a new endpoint to the existing upstream
etcdctl set /vulcand/upstreams/up1/endpoints/e2 http://localhost:5001

Rate and connection limits

Vulcan supports setting rate and connection limits.

# Update or set rate limit the request to location "loc1" to 1 request per second per client ip with bursts up to 3 requests per second.
# Note the priority here - middlewares with lower priorities will be executed earlier.
etcdctl set /vulcand/hosts/localhost/locations/loc1/middlewares/ratelimit/rl1 '{"Type": "ratelimit", "Middleware":{"Requests":1, "PeriodSeconds":1, "Burst":3, "Variable": "client.ip"}}'

# Update or set the connection limit to 3 simultaneous connections per client ip at a time
# Note the priority here - middlewares with lower priorities will be executed earlier.
etcdctl set /vulcand/hosts/localhost/locations/loc1/middlewares/connlimit/rl1 '{"Type": "connlimit", "Middleware":{"Requests":1, "PeriodSeconds":1, "Burst":3, "Variable": "client.ip"}}'

Command line

Command line is the most convenient way to start working with vulcan, here are some examples.

Status

Displays the configuration and stats about the daemon

$ vulcanctl status

[hosts]
  │
  └host(name=localhost)
    │
    └location(id=loc1, path=/hello)
      │
      └upstream(id=u1)
        │
        └endpoint(id=e1, url=http://localhost:5001)

Host

Host command supports adding or removing host

# Add host with name 'example.com'
$ vulcanctl host add --name example.com

# Remove host with name 'example.com'
$ vulcanctl host rm --name example.com

Upstream

Upstream command adds or removes upstreams

# Add upstream  with id 'u1'
$ vulcanctl upstream add --id u1

# Adds upstream with auto generated id
$ vulcanctl upstream add 

# Remove upstream with id 'u1'
$ vulcanctl upstream rm --id u1

# "Drain" - wait till there are no more active connections from the endpoints of the upstream 'u1'
# or timeout after 10 seconds if there are remaining connections
vulcanctl upstream drain -id u1 -timeout 10

Endpoint

Endpoint command adds or removed endpoints to the upstream.

# add endpoint with id 'e2' and url 'http://localhost:5002' to upstream with id 'u1'
$ vulcanctl endpoint add --id e1 --up u1 --url http://localhost:5000 

# in case if id is omitted, etcd will auto generate it
$ vulcanctl endpoint add --up u1 --url http://localhost:5001 

# removed endpoint with id 'e1' from upstream 'u1'
$ vulcanctl endpoint rm --up u1 --id e1 

Location

Location adds or removes location to the host

# add location with id 'id1' to host 'example.com', use path '/hello' and upstream 'u1'
$ vulcanctl location add --host example.com --id loc1 --path /hello --up u1 

# add location with auto generated id to host 'example.com', use path '/hello2' and upstream 'u1'
$ vulcanctl location add --host example.com --path /hello2 --up u1 

# remove location with id 'loc1' from host 'example.com'
$ vulcanctl location rm --host example.com --id loc1 

# update upstream of the location 'loc1' in host 'example.com' to be 'u2'
# this redirects the traffic gracefully from endpoints in the previous upstream
# to endpoints of the upstream 'u2', see drain for connection draining
$ vulcanctl location set_upstream --host example.com --id loc1 --up u2

Rate limit

Rate add or removes rate limit restrictions on the location

# limit access per client ip to 10 requests per second in 
# location 'loc1' in host 'example.com'
$ vulcanctl ratelimit add --variable client.ip --host example.com --loc loc1 --requests 10

# limit access per custom http header value 'X-Account-Id' to 100 requests per second 
# to location 'loc1' in host 'example.com'
$ vulcanctl ratelimit add --variable request.header.X-Account-Id --host example.com --loc loc1 --requests 10

# remove rate limit restriction with id 'r1' from host 'example.com' location 'loc1'
$ vulcanctl ratelimit rm --id r1  --host example.com --loc 'loc1'

Connection limit

Control simultaneous connections for a location.

# limit access per client ip to 10 simultaneous connections for
# location 'loc1' in host 'example.com'
$ vulcanctl connlimit add --id c1 -host example.com -loc loc1 -connections 10

# limit access per custom http header value 'X-Account-Id' to 100 simultaneous connections
# to location 'loc1' in host 'example.com'
$ vulcanctl connlimit add --variable request.header.X-Account-Id --host example.com --loc loc1 --connections 10

# remove connection limit restriction with id 'c1' from host 'example.com' location 'loc1'
$ vulcanctl connlimit rm --id c1  --host example.com --loc 'loc1'

HTTP API

Vulcan's HTTP API is the best way to configure one or several instances of Vulcan at the same time.
Essentially it's a tiny wrapper around the Etcd backend, that writes to the Etcd. Multiple Vulcand instances listening to the same prefix would detect changes simultaneously and reload configuration.

Status

GET /v1/status

Check status of the Vulcan instance.

Returns:

200 OK
{
   "Status": "ok"
}

Host

GET /v1/hosts

Retrieve the existing hosts.

Example response:

{
  "Hosts": [
    {
      "Name": "localhost",
      "Locations": [
        {
          "Hostname": "localhost",
          "Path": "/home",
          "Id": "loc1",
          "Upstream": {
            "Id": "up1",
            "Endpoints": [
              {
                "Id": "e1",
                "Url": "http://localhost:5000",
                "UpstreamId": "up1",
                "Stats": {
                  "Successes": 0,
                  "Failures": 0,
                  "FailRate": 0,
                  "PeriodSeconds": 10
                }
              }
            ]
          },
          "Middlewares": [
            {
              "Id": "rl1",
              "Priority": 0,
              "Type": "ratelimit",
              "Middleware": {
                "PeriodSeconds": 1,
                "Burst": 3,
                "Variable": "client.ip",
                "Requests": 1
              }
            }
          ]
        }
      ]
    }
  ]
}

POST 'application/json' /v1/hosts

Add a host to the proxy.

{
  "Name": "localhost"
}

DELETE /v1/hosts/<name>

Delete a host.

Upstream

GET /v1/upstreams

Retrieve the existing upstreams. Example response:

{
  "Upstreams": [
    {
      "Id": "up1",
      "Endpoints": [
        {
          "Id": "e1",
          "Url": "http://localhost:5000",
          "UpstreamId": "up1",
          "Stats": null
        },
        {
          "Id": "e2",
          "Url": "http://localhost:5001",
          "UpstreamId": "up1",
          "Stats": null
        }
      ]
    }
  ]
}

POST 'application/json' /v1/upstreams

{"Id": "up1"}

Add upstream to the proxy

DELETE /v1/upstreams/<id>

Delete an upstream.

GET /v1/upstreams/drain?timeout=3

Wait till there are no more connections to any endpoints to the upstream.

Example response:

{
  "Connections": 0
}

Endpoint

GET /v1/upstreams/<id>/endpoints

Retrieve the endpoints of the upstream. Example response:

{
  "Endpoints": [
    {
      "Id": "e1",
      "Url": "http://localhost:5000",
      "UpstreamId": "up1"
    }
  ]
}

GET /v1/upstreams/<id>/endpoints/<endpoint-id>

Retrieve the particular endpoint with id endpoint-id

POST /v1/upstreams/<id>/endpoints

{
  "Id": "e4",
  "Url": "http://localhost:5004",
  "UpstreamId": "up1"
}

Add endpoint to the upstream

Example response:

{
  "Id": "e4",
  "Url": "http://localhost:5004",
  "UpstreamId": "up1",
  "Stats": null
}

DELETE /v1/upstreams/<id>/endpoints/<endpoint-id>

Delete an endpoint.

Location

GET /v1/hosts/<hostname>/locations

Retrieve the locations of the host.

Example response:

{
  "Locations": [
    {
      "Hostname": "localhost",
      "Path": "/home",
      "Id": "loc1",
      "Upstream": {
        "Id": "up1",
        "Endpoints": [
          {
            "Id": "e1",
            "Url": "http://localhost:5000",
            "UpstreamId": "up1",
            "Stats": null
          }
        ]
      },
      "Middlewares": []
    }
  ]
}

GET /v1/hosts/<hostname>/locations/<location-id>

Retrieve the particular location in the host hostname with id location-id

{
  "Hostname": "localhost",
  "Path": "/home",
  "Id": "loc1",
  "Upstream": {
    "Id": "up1",
    "Endpoints": [
      {
        "Id": "e1",
        "Url": "http://localhost:5000",
        "UpstreamId": "up1",
        "Stats": null
      },
      {
        "Id": "e2",
        "Url": "http://localhost:5001",
        "UpstreamId": "up1",
        "Stats": null
      },
      {
        "Id": "e3",
        "Url": "http://localhost:5003",
        "UpstreamId": "up1",
        "Stats": null
      },
      {
        "Id": "e4",
        "Url": "http://localhost:5004",
        "UpstreamId": "up1",
        "Stats": null
      }
    ]
  },
  "Middlewares": [
    {
      "Id": "rl1",
      "Priority": 0,
      "Type": "ratelimit",
      "Middleware": {
        "PeriodSeconds": 1,
        "Burst": 3,
        "Variable": "client.ip",
        "Requests": 1
      }
    },
    {
      "Id": "cl1",
      "Priority": 0,
      "Type": "connlimit",
      "Middleware": {
        "Connections": 3,
        "Variable": "client.ip"
      }
    }
  ]
}

POST 'application/json' /v1/hosts/<hostname>/locations

Add a location to the host

{
  "Id": "loc2",
  "Hostname": "localhost",
  "Path": "/home",
  "Upstream": {
    "Id": "up1"
  }
}

DELETE /v1/hosts/<hostname>/locations/<location-id>

Delete a location

PUT /v1/hosts/<hostname>/locations/<location-id>

Update location's upstream. Gracefully Redirects all the traffic to the endpoints of the new upstream.

Rate limit

GET /v1/hosts/<hostname>/locations/<location-id>/middlewares/ratelimit/<rate-id>

Retrieve the particular rate of location in the host hostname with id location-id and rate id rate-id

{
  "Id": "rl1",
  "Priority": 0,
  "Type": "ratelimit",
  "Middleware": {
    "PeriodSeconds": 1,
    "Burst": 3,
    "Variable": "client.ip",
    "Requests": 1
  }
}

POST 'application/json' /v1/hosts/<hostname>/locations/limits/rates

Add a rate limit to the location, will take effect immediately.

{
  "Id": "rl1",
  "Priority": 0,
  "Type": "ratelimit",
  "Middleware": {
    "PeriodSeconds": 1,
    "Burst": 3,
    "Variable": "client.ip",
    "Requests": 1
  }
}

DELETE /v1/hosts/<hostname>/locations/<location-id>/limits/rates/<rate-id>

Delete a rate limit from the location.

PUT /v1/hosts/<hostname>/locations/<location-id>/limits/rates/<rate-id>

Update location's rate limit. Takes effect immdediatelly.

{
  "Id": "rl1",
  "Priority": 0,
  "Type": "ratelimit",
  "Middleware": {
    "PeriodSeconds": 1,
    "Burst": 3,
    "Variable": "client.ip",
    "Requests": 1
  }
}

Connection limit

GET /v1/hosts/<hostname>/locations/<location-id>/middlewares/connlimit/<conn-id>

Retrieve the particular connection limit of location in the host hostname with id location-id and connection limit id conn-id

Example response:

{
  "Id": "cl1",
  "Priority": 0,
  "Type": "connlimit",
  "Middleware": {
    "Connections": 3,
    "Variable": "client.ip"
  }
}

POST 'application/json' /v1/hosts/<hostname>/locations/limits/connections

Add a connection limit to the location, will take effect immediately.

{
  "Id": "cl1",
  "Priority": 0,
  "Type": "connlimit",
  "Middleware": {
    "Connections": 3,
    "Variable": "client.ip"
  }
}

DELETE /v1/hosts/<hostname>/locations/<location-id>/middlewares/connlimit/<conn-id>

Delete a connection limit from the location.

PUT /v1/hosts/<hostname>/locations/<location-id>/limits/connections/<conn-id>

Update location's connection limit. Takes effect immdediatelly.

{
  "Id": "cl1",
  "Priority": 0,
  "Type": "connlimit",
  "Middleware": {
    "Connections": 3,
    "Variable": "client.ip"
  }
}

Docker

Here's how you build vulcan in Docker:

docker build -t mailgun/vulcan .

Starting the daemon:

docker run -p 8182:8182 -p 8181:8181 mailgun/vulcan /opt/vulcan/vulcand -apiInterface="0.0.0.0" -interface="0.0.0.0" --etcd=http://10.0.3.1:4001

Don't forget to map the ports and bind to the proper interfaces, otherwise vulcan won't be reachable from outside the container.

Using the vulcanctl from container:

docker run mailgun/vulcan /opt/vulcan/vulcanctl status  --vulcan 'http://10.0.3.1:8182'

Make sure you've specified --vulcan flag to tell vulcanctl where the running vulcand is. I've used lxc bridge interface in the example above.