eea / eea.docker.varnish Goto Github PK

VARNISH_HTTP_PORT - varnish port
VARNISH_HTTPS_PORT - varnish ssl port
VARNISH_SIZE - varnish cache size
AUTOKILL_CRON - Varnish re-create crontab, will force a recreation of the container. Uses UTC time, format is linux crontab - for example - 0 2 * * * is 02:00 UTC each day"
VARNISH_CFG_CONTENT - Multiline variable that will be written in the default.vcl file

Usage

Using `VARNISH_CFG_CONTENT`

See docker-compose.yml.

Extend the image with a custom varnish.vcl file

The default.vcl file provided with this image is bare and only contains the marker to specify the VCL version. If you plan on using a more elaborate base configuration in your container and you want it shipped with your image, you can extend the image in a Dockerfile, like this:

FROM eeacms/varnish
COPY varnish.vcl /etc/varnish/conf.d/

and then run

$ docker build -t varnish-custom /path/to/Dockerfile

How to add docker environment variables in varnish.vcl

Choose relevant variable name, starting with VARNISH_ - eg. VARNISH_EXAMPLE
Add default value in Dockerfile

ENV VARNISH_EXAMPLE="GET"
Add variable in <> in varnish.vcl

set req.http.X-Varnish-Routed = "<VARNISH_EXAMPLE>";
Add description in Readme.md

Rancher integration

Use dynamic.director to integrate varnish in rancher DNS - if a backend containers are changed, it knows to get the latest list of IPs automatically.

  new cluster = dynamic.director(port = "<VARNISH_BACKEND_PORT>", ttl = <VARNISH_DNS_TTL>);

Example:

You can use plone-varnish as an example of usage.

Upgrade

$ docker pull eeacms/varnish

Copyright and license

The Original Code is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Funding

European Environment Agency (EU)

eea.docker.varnish's People

Contributors

Stargazers

Watchers

eea.docker.varnish's Issues

Admin not running?

When I try: telnet localhost:6082 gives me "Connection refused".

Any idea?

Chaperone error on startup

Error:

May 18 15:48:15 77e569dc26e0 chaperone[1]: Switching all chaperone logging to /dev/log
May 18 15:48:15 77e569dc26e0 chaperone[1]: chaperone version 0.3.9, ready.
May 18 15:48:15 77e569dc26e0 chaperone[1]: system startup cancelled due to error: /track_hosts
May 18 15:48:15 77e569dc26e0 chaperone[1]: Request made to kill system.

chaperone.conf:

settings: {
  env_set: {

    # Priviledge separation user id
    _USER: "${PRIVILEDGED_USER:+-u ${PRIVILEDGED_USER}}",

    # Size of the cache storage
    CACHE_SIZE: "${CACHE_SIZE:-2G}",
    CACHE_STORAGE: "${CACHE_STORAGE:-malloc,${CACHE_SIZE}}",

    # Cache storage
    _STORAGE: "${CACHE_STORAGE:+-s ${CACHE_STORAGE}}",

    # Address:Port
    ADDRESS_PORT: "${ADDRESS_PORT:-:6081}",
    _ADDRESS: "${ADDRESS_PORT:+-a ${ADDRESS_PORT}}",

    # Admin:Port
    _ADMIN: "${ADMIN_PORT:+-T ${ADMIN_PORT}}",

    # Custom params
    PARAM_VALUE: "${PARAM_VALUE:--p default_ttl=3600 -p default_grace=3600}",
    _VALUE: "${PARAM_VALUE}",

    PARAMS: "${_USER} ${_STORAGE} ${_ADDRESS} ${_ADMIN} ${_VALUE}",

    _DNS_ENABLED: "${DNS_ENABLED:+true}",
    _DNS_DISABLED: "${DNS_ENABLED:+false}",
    _DNS_TTL: "*/${DNS_TTL:-1} * * * *"
  }
}

varnish.service: {
  service_groups: IDLE,
  command: "varnishd -F -f /etc/varnish/default.vcl ${PARAMS}",
  stdout: inherit,
  stderr: inherit,
}

agent.service: {
  service_groups: IDLE,
  command: "varnish-agent -K /etc/varnish/conf.d/secret",
  stdout: inherit,
  stderr: inherit,
  after: varnish.service,
}

dns.service: {
  type: cron,
  enabled: "${_DNS_ENABLED:-false}",
  command: "/track_dns",
  interval: "${_DNS_TTL}",
  service_groups: IDLE,
  after: varnish.service,
  stdout: inherit,
  stderr: inherit
}

hosts.service: {
  type: cron,
  enabled: "${_DNS_DISABLED:-true}",
  command: "/track_hosts",
  interval: "${_DNS_TTL}",
  service_groups: IDLE,
  after: varnish.service,
  stdout: inherit,
  stderr: inherit,
}

console.logging: {
  selector: '*.info',
  stdout: true
}

Don't force malloc storage backend

Perhaps NO_DEFAULT_CACHE=true or something...

Update documentation to reflect how to enable easy access to control terminal

Please update the documentation to reflect how to configure the Varnish control terminal running usually on port 6082:

Here's the snippet I am using:

    volumes:
      - ./varnish.secret:/etc/varnish/secret
    environment:
      PARAM_VALUE: "-S /etc/varnish/secret"
      ADMIN_PORT: ":6082"

Sample content of secret file:

a40ff8c617404b78d74-2a6707d565ef

PRIVILEDGED_USER does not seem to work

Docker logs:

varnish[12]: /usr/local/sbin/varnishd: invalid option -- 'u'

Current version being used (4.1) does not support that flag any more:
https://varnish-cache.org/docs/4.1/reference/varnishd.html

I cant use another backends except eeacms/hello

Hi, I'm running the following docker-compose file but the service has error, "No backends or directors found in VCL program, at least one is necessary."
I changed backend images and just eeacms/hello worked.
How can I use other images as varnish backend?

version: "2"
services:
web1:
image: alpine
environment:
PORT: "7000"
ports:
- 7000
web2:
image: python:3.5
ports:
- 7000
restart: on-failure
varnish:
image: eeacms/varnish
ports:
- "80:6081"
- "6085:6085"
depends_on:
- web1
- web2
environment:
BACKENDS: "web1 web2"
BACKENDS_PORT: "7000"
DNS_ENABLED: "true"
BACKENDS_PROBE_INTERVAL: "3s"
BACKENDS_PROBE_TIMEOUT: "1s"
BACKENDS_PROBE_WINDOW: "3"
BACKENDS_PROBE_THRESHOLD: "2"
DASHBOARD_USER: "admin"
DASHBOARD_PASSWORD: "admin"
DASHBOARD_SERVERS: "varnish"
DASHBOARD_DNS_ENABLED: "true"

Backend name too long

Hi,
when the dynamic backend is an AWS LB endpoint DNS name, which format is:

(internal-)?{lb_name}-{aws_account_id}.{aws_region}.elb.amazonaws.com

the Varnish backends' names will be:

"server_(internal_)?{lb_name}_{aws_account_id}_{aws_region}_elb_amazonaws_com_{ip_1}"
"server_(internal_)?{lb_name}_{aws_account_id}_{aws_region}_elb_amazonaws_com_{ip_2}"

e.g.:

backend server_internal_api_lb_123456789_eu_west_2_elb_amazonaws_com_172_1_12_123 {
[...]
}

backend server_internal_api_lb_123456789_eu_west_2_elb_amazonaws_com_172_1_12_345 {
[...]
}

but this eventually leads to a VLC compilation error, due to the backend name's constraints:

Message from VCC-compiler:
Name of backend too long (max 64, is 78):
[...]
Running VCC-compiler failed, exited with 2
VCL compilation failed

Any idea in order to resolve this issue?
Thank you

Statistics / vagent2

Hi there,

How do you suggest getting statistics or logs out of this container? I love the implementation - I have tried several Varnish Dockerfiles, and this one is one of the best. But it's not obvious what would be a sensible way to add vagent2, and vagent2 doesn't seem to have any independent Dockerfiles available.

How would I build vagent2 against the source for this container?
What do you use for monitoring varnish?
Is there anything I should be aware of?

It seems as if a sensible approach might be to fork and add straight to the docker-install.sh ?

/etc/varnish/default.vcl grows every time container is reloaded

In a production system, I've realized the mentioned file grows when service is reloaded. The problem seems to be at https://github.com/eea/eea.docker.varnish/blob/master/varnish/src/assemble_vcls.py#L6 because previous configuration is copied and every time this script runs, it adds a newline at https://github.com/eea/eea.docker.varnish/blob/master/varnish/src/assemble_vcls.py#L13

I'm forking the repo and I'll create a new PR

Thanks for your great work!

How do I access the admin console?

I set the ADMIN_PORT to 6082 but when I try to access nothing happens. Why?
This is the Varnish VAC port, or I am thinking wrong?

Permission error

syslog service cannot be started: [Errno 13] Permission denied: '/dev/log'
command service cannot be started: [Errno 13] Permission denied
Traceback (most recent call last):
  File "/add_backends.py", line 147, in <module>
    index=index
KeyError: 'director'
system startup cancelled due to error: setup.service failed on start-up with result '<ProcStatus exit_status=1>'
Request made to kill system.

static act purge IP adresses prevent successful purge

A backends.vcl file is generated with an static acl purge list - see: https://github.com/eea/eea.docker.varnish/blob/master/varnish/src/add_backends.py#L65

At least in my setup (nginx->eea varnish->eea haproxy->plone) plone cannot successfully purge content when it's outdated.

Instead of the hardcoded 172.17.0.0/16 and 10.42.0.0/16 a config option or broader netmaskj would be fine. In my case one of the Plone backends have a IP of 172.21.0.4.

simple way to include cookie rule missing

Currently there is no easy way to in configure a cookie rule. If cookies are present, the request bypasses the varnish cache completely. A rule snippet which we use for many projects is the following (actually generated by https://github.com/collective/plone.recipe.varnish ):

   /* cookies for pass */
    set req.http.UrlNoQs = regsub(req.url, "\?.*$", "");
    if (req.http.Cookie && req.http.Cookie ~ "__ac(|_(name|password|persistent))=") {
        if (req.http.UrlNoQs ~ "\.(js|css|kss|png|gif|jpg|pdf)$") {
            unset req.http.cookie;
            return(pipe);
        }
        return(pass);
    }

    /* Cookie whitelist, remove all not in there */
    if (req.http.Cookie) {
        set req.http.Cookie = ";" + req.http.Cookie;
        set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
        set req.http.Cookie = regsuball(req.http.Cookie, ";(statusmessages|cart|__ac|_ZopeId|__cp)=", "; \1=");
        set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
        set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
        if (req.http.Cookie == "") {
            unset req.http.Cookie;
        }
    }

    # Large static files should be piped, so they are delivered directly to the end-user without
    # waiting for Varnish to fully read the file first.
    if (req.url ~ "^[^?]*\.(mp3,mp4|rar|tar|tgz|gz|wav|zip)(\?.*)?$") {
        return(pipe);
    }

eea / eea.docker.varnish Goto Github PK

eea.docker.varnish's Introduction

Varnish Docker image

Supported tags and respective Dockerfile links

Stable and immutable tags

Changes

Base docker image

Source code

Installation

Variables