google / ngx_brotli Goto Github PK

NGINX module for Brotli compression

License: BSD 2-Clause "Simplified" License

C 86.45% Shell 13.55%

ngx_brotli's Introduction

ngx_brotli

Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression.

ngx_brotli is a set of two nginx modules:

ngx_brotli filter module - used to compress responses on-the-fly,
ngx_brotli static module - used to serve pre-compressed files.

Status
Installation
Configuration directives
Variables
- $brotli_ratio
Sample configuration
Contributing
License

Status

Both Brotli library and nginx module are under active development.

Installation

Statically compiled

Checkout the latest ngx_brotli and build the dependencies:

git clone --recurse-submodules -j8 https://github.com/google/ngx_brotli
cd ngx_brotli/deps/brotli
mkdir out && cd out
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_C_FLAGS="-Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_CXX_FLAGS="-Ofast -m64 -march=native -mtune=native -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections" -DCMAKE_INSTALL_PREFIX=./installed ..
cmake --build . --config Release --target brotlienc
cd ../../../..

$ cd nginx-1.x.x
$ export CFLAGS="-m64 -march=native -mtune=native -Ofast -flto -funroll-loops -ffunction-sections -fdata-sections -Wl,--gc-sections"
$ export LDFLAGS="-m64 -Wl,-s -Wl,-Bsymbolic -Wl,--gc-sections"
$ ./configure --add-module=/path/to/ngx_brotli
$ make && make install

This will compile the module directly into Nginx.

Dynamically loaded

$ cd nginx-1.x.x
$ ./configure --with-compat --add-dynamic-module=/path/to/ngx_brotli
$ make modules

You will need to use exactly the same ./configure arguments as your Nginx configuration and append --with-compat --add-dynamic-module=/path/to/ngx_brotli to the end, otherwise you will get a "module is not binary compatible" error on startup. You can run nginx -V to get the configuration arguments for your Nginx installation.

make modules will result in ngx_http_brotli_filter_module.so and ngx_http_brotli_static_module.so in the objs directory. Copy these to /usr/lib/nginx/modules/ then add the load_module directives to nginx.conf (above the http block):

load_module modules/ngx_http_brotli_filter_module.so;
load_module modules/ngx_http_brotli_static_module.so;

Configuration directives

`brotli_static`

syntax: brotli_static on|off|always
default: off
context: http, server, location

Enables or disables checking of the existence of pre-compressed files with.br extension. With the always value, pre-compressed file is used in all cases, without checking if the client supports it.

`brotli`

syntax: brotli on|off
default: off
context: http, server, location, if

Enables or disables on-the-fly compression of responses.

`brotli_types`

syntax: brotli_types <mime_type> [..]
default: text/html
context: http, server, location

Enables on-the-fly compression of responses for the specified MIME types in addition to text/html. The special value * matches any MIME type. Responses with the text/html MIME type are always compressed.

`brotli_buffers`

syntax: brotli_buffers <number> <size>
default: 32 4k|16 8k
context: http, server, location

Deprecated, ignored.

`brotli_comp_level`

syntax: brotli_comp_level <level>
default: 6
context: http, server, location

Sets on-the-fly compression Brotli quality (compression) level. Acceptable values are in the range from 0 to 11.

`brotli_window`

syntax: brotli_window <size>
default: 512k
context: http, server, location

Sets Brotli window size. Acceptable values are 1k, 2k, 4k, 8k, 16k, 32k, 64k, 128k, 256k, 512k, 1m, 2m, 4m, 8m and 16m.

`brotli_min_length`

syntax: brotli_min_length <length>
default: 20
context: http, server, location

Sets the minimum length of a response that will be compressed. The length is determined only from the Content-Length response header field.

Variables

`$brotli_ratio`

Achieved compression ratio, computed as the ratio between the original and compressed response sizes.

Sample configuration

brotli on;
brotli_comp_level 6;
brotli_static on;
brotli_types application/atom+xml application/javascript application/json application/vnd.api+json application/rss+xml
             application/vnd.ms-fontobject application/x-font-opentype application/x-font-truetype
             application/x-font-ttf application/x-javascript application/xhtml+xml application/xml
             font/eot font/opentype font/otf font/truetype image/svg+xml image/vnd.microsoft.icon
             image/x-icon image/x-win-bitmap text/css text/javascript text/plain text/xml;

Contributing

See Contributing.

License

Copyright (C) 2002-2015 Igor Sysoev
Copyright (C) 2011-2015 Nginx, Inc.
Copyright (C) 2015 Google Inc.
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this list of conditions and the following disclaimer in the
   documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.

ngx_brotli's People

Contributors

Stargazers

Watchers

Forkers

tedkornish wangfakang aslijiasheng badfiles othree duanshuaimin sdgdsffdsfff piotrsikora cuiwm kleopatra999 hsw theolodewijk edgesite wxfpdc fholzer maniacs-ops butch2k wpg liuliangsir slanterns-fork nobitanobi lloiacono danmallette caputomarcos cloudspeedy detailyang mikemaccana fluentdevelopment alimdd sunil-kumar3 exagone313 bartebor mqudsi maciekr 440hz eustas alsyundawy somebee kmalinich adslqa squallatf vadorovsky lemp-sh vinicius-ianni ryoh debdutbiswas dsimich06 quinndiggity johnyin123 beckyyala afcarl yourdigitalheart perire2017 thiagobarradas cnrting charlesqing windmgc mohnkhan regae liupeidong0620 nikolasmelui areebmoin westgen leehomewl martowu openmind-repo laikee99 wasas ncderek splitice henrywarne zoonru zclabc fluke667 prashanth-prabhu-p philipp-classen sterbur leex00g fishgege nonomal lianshitong monkey-wenjun jas0n99 scg82 sokratis1988 pkhanhao rohitub222 gencer kperrow mwangaben getpagespeed gayhub-dev amican jels0nb0nilla zuobeixing honsa w7374520 knoppixmeister leonardoalcantara hkraal

ngx_brotli's Issues

"error while loading shared libraries: libbrotlienc.so.1"

I get this error when installing on CentOS 6.5 with nginx 1.9.10, ngx_brotli and libbrotli.

[root@server nginx-1.9.10]# /software/nginx-1.9.10/sbin/nginx -h
/software/nginx-1.9.10/sbin/nginx: error while loading shared libraries: libbrotlienc.so.1: cannot open shared object file: No such file or directory

My guess is libbrotli is going into the wrong location and nginx can't find it. By default, it appears that installs into "/usr/local/". Where should it go for nginx to find it?

Thanks

Compiling nginx (1.11.3) with ngx_brotli failed

A same question like issue #32 , I do not know @Sunka how to fix it. So I submit a new issue here.

Environment: Centos 6.8 x64, gcc 4.4.7 20120313 (Red Hat 4.4.7-17), libstdc++-devel-4.4.7-17.

Nginx compiling stdout:

adding module in /data/src/ngx_brotli
checking for Brotli library ... not found
checking for Brotli library in /usr/local/ ... not found
checking for Brotli library in /usr/pkg/ ... not found
checking for Brotli library in /opt/local/ ... not found
./configure: error: ngx_brotli filter module requires Brotli library.

$cat objs/autoconf.err :

checking for Brotli library

In file included from objs/autotest.c:4:
/usr/local/include/brotli/enc/encode.h:12:18: error: string: No such file or directory
/usr/local/include/brotli/enc/encode.h:13:18: error: vector: No such file or directory
In file included from /usr/local/include/brotli/enc/./command.h:12,
                 from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././fast_log.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./command.h:13,
                 from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././prefix.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./command.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./hash.h:14:21: error: algorithm: No such file or directory
/usr/local/include/brotli/enc/./hash.h:15:19: error: cstring: No such file or directory
/usr/local/include/brotli/enc/./hash.h:16:18: error: limits: No such file or directory
In file included from /usr/local/include/brotli/enc/./hash.h:18,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././dictionary_hash.h:14: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:20,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././find_match_length.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:23,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././static_dict.h:14: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:24,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././transform.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./hash.h:27: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:16,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./ringbuffer.h:12:40: error: cstdlib: No such file or directory
In file included from /usr/local/include/brotli/enc/encode.h:16,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./ringbuffer.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:18,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./streams.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from objs/autotest.c:4:
/usr/local/include/brotli/enc/encode.h:21: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
----------

#include <sys/types.h>
#include <unistd.h>
#include <brotli/enc/encode.h>

int main(void) {
    BrotliEncoderCreateInstance(NULL, NULL, NULL);
    return 0;
}

----------
cc -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -o objs/autotest objs/autotest.c -ljemalloc -lrt -lbrotlienc
----------

----------------------------------------
checking for Brotli library in /usr/local/

In file included from objs/autotest.c:4:
/usr/local/include/brotli/enc/encode.h:12:18: error: string: No such file or directory
/usr/local/include/brotli/enc/encode.h:13:18: error: vector: No such file or directory
In file included from /usr/local/include/brotli/enc/./command.h:12,
                 from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././fast_log.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./command.h:13,
                 from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././prefix.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./command.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./hash.h:14:21: error: algorithm: No such file or directory
/usr/local/include/brotli/enc/./hash.h:15:19: error: cstring: No such file or directory
/usr/local/include/brotli/enc/./hash.h:16:18: error: limits: No such file or directory
In file included from /usr/local/include/brotli/enc/./hash.h:18,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././dictionary_hash.h:14: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:20,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././find_match_length.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:23,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././static_dict.h:14: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:24,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././transform.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./hash.h:27: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:16,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./ringbuffer.h:12:40: error: cstdlib: No such file or directory
In file included from /usr/local/include/brotli/enc/encode.h:16,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./ringbuffer.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:18,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./streams.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from objs/autotest.c:4:
/usr/local/include/brotli/enc/encode.h:21: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
----------

#include <sys/types.h>
#include <unistd.h>
#include <brotli/enc/encode.h>

int main(void) {
    BrotliEncoderCreateInstance(NULL, NULL, NULL);
    return 0;
}

----------
cc -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -I /usr/local/include -o objs/autotest objs/autotest.c -ljemalloc -lrt
 -L/usr/local/lib -lbrotlienc
----------

----------------------------------------
checking for Brotli library in /usr/pkg/

In file included from objs/autotest.c:4:
/usr/local/include/brotli/enc/encode.h:12:18: error: string: No such file or directory
/usr/local/include/brotli/enc/encode.h:13:18: error: vector: No such file or directory
In file included from /usr/local/include/brotli/enc/./command.h:12,
                 from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././fast_log.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./command.h:13,
                 from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././prefix.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./command.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./hash.h:14:21: error: algorithm: No such file or directory
/usr/local/include/brotli/enc/./hash.h:15:19: error: cstring: No such file or directory
/usr/local/include/brotli/enc/./hash.h:16:18: error: limits: No such file or directory
In file included from /usr/local/include/brotli/enc/./hash.h:18,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././dictionary_hash.h:14: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:20,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././find_match_length.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:23,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././static_dict.h:14: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:24,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././transform.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./hash.h:27: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:16,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./ringbuffer.h:12:40: error: cstdlib: No such file or directory
In file included from /usr/local/include/brotli/enc/encode.h:16,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./ringbuffer.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:18,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./streams.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from objs/autotest.c:4:
/usr/local/include/brotli/enc/encode.h:21: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
----------

#include <sys/types.h>
#include <unistd.h>
#include <brotli/enc/encode.h>

int main(void) {
    BrotliEncoderCreateInstance(NULL, NULL, NULL);
    return 0;
}

----------
cc -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -I /usr/pkg/include -o objs/autotest objs/autotest.c -ljemalloc -lrt -
L/usr/pkg/lib -lbrotlienc
----------

----------------------------------------
checking for Brotli library in /opt/local/

In file included from objs/autotest.c:4:
/usr/local/include/brotli/enc/encode.h:12:18: error: string: No such file or directory
/usr/local/include/brotli/enc/encode.h:13:18: error: vector: No such file or directory
In file included from /usr/local/include/brotli/enc/./command.h:12,
                 from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././fast_log.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./command.h:13,
                 from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././prefix.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:14,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./command.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./hash.h:14:21: error: algorithm: No such file or directory
/usr/local/include/brotli/enc/./hash.h:15:19: error: cstring: No such file or directory
/usr/local/include/brotli/enc/./hash.h:16:18: error: limits: No such file or directory
In file included from /usr/local/include/brotli/enc/./hash.h:18,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././dictionary_hash.h:14: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:20,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././find_match_length.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:23,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././static_dict.h:14: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/./hash.h:24,
                 from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/././transform.h:16: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:15,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./hash.h:27: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:16,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./ringbuffer.h:12:40: error: cstdlib: No such file or directory
In file included from /usr/local/include/brotli/enc/encode.h:16,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./ringbuffer.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from /usr/local/include/brotli/enc/encode.h:18,
                 from objs/autotest.c:4:
/usr/local/include/brotli/enc/./streams.h:17: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
In file included from objs/autotest.c:4:
/usr/local/include/brotli/enc/encode.h:21: error: expected '=', ',', ';', 'asm' or '__attribute__' before 'brotli'
----------

#include <sys/types.h>
#include <unistd.h>
#include <brotli/enc/encode.h>

int main(void) {
    BrotliEncoderCreateInstance(NULL, NULL, NULL);
    return 0;
}

----------
cc -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -I /opt/local/include -o objs/autotest objs/autotest.c -ljemalloc -lrt
 -L/opt/local/lib -lbrotlienc
----------

I saw that have two files could not include in /usr/local/include/brotli/enc/encode.h.

error: string: No such file or directory 
error: vector: No such file or directory

$cat /usr/local/include/brotli/enc/encode.h(Line 9~19) :

#ifndef BROTLI_ENC_ENCODE_H_
#define BROTLI_ENC_ENCODE_H_

#include <string>
#include <vector>
#include "./command.h"
#include "./hash.h"
#include "./ringbuffer.h"
#include "./static_dict.h"
#include "./streams.h"
#include "./types.h"

How can i fix that?

Thank you.

mtime comparison checks for statically pre-compressed content?

Is there any way to include mtime comparisons if brotli_static is set to auto such that if the .br file is older than the original, it isn't used?

We've developed an open-source tool to aid in static-compression of web content, (github link) and we've found that setting the mtime on the .br or .gz copy to match the source mtime is a good way of keeping cached content up-to-date without re-caching everything. It would be great if, should one update a resource and forget to update the cached copy, ngx_brotli did not serve the outdated version.

Thanks!

Yum requires libbrotlienc.so.1 even though the library installed already

Hi,

I installed libbrotli from source successfully, and compiled nginx with param --add-module=[Path/to/ngx_brotli] without any problems.

But when trying to install the result package with yum, I get the error like this:

--> Processing Dependency: libbrotlienc.so.1()(64bit) for package: nginx-1.10.1-1.el6.ngx.x86_64
--> Finished Dependency Resolution
Error: Package: nginx-1.10.1-1.el6.ngx.x86_64 (/nginx-1.10.1-1.el6.ngx.x86_64)
Requires: libbrotlienc.so.1()(64bit)
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest

In fact libbrotlienc.so is already there in the system

$ ls -l /usr/local/lib/libbrotlienc*

-rw-r--r-- 1 root root 7876000 Jun 21 05:53 /usr/local/lib/libbrotlienc.a
-rwxr-xr-x 1 root root 955 Jun 21 05:53 /usr/local/lib/libbrotlienc.la
lrwxrwxrwx 1 root root 21 Jun 21 05:53 /usr/local/lib/libbrotlienc.so -> libbrotlienc.so.1.0.0
lrwxrwxrwx 1 root root 21 Jun 21 05:53 /usr/local/lib/libbrotlienc.so.1 -> libbrotlienc.so.1.0.0
-rwxr-xr-x 1 root root 3691524 Jun 21 05:53 /usr/local/lib/libbrotlienc.so.1.0.0

Please help.

Need a release

I want to help this module easier to install on OSX using homebrew.
But before I can do it.
I will need a release like this.

Is there any plan for a release?

Thanks

Brotli v0.6.0 - Ngx_brotli module compatibility

Seems as the Ngx_brotli module isn't compatible with Brotli v0.6.0 yet.
It would be wonderful if Ngx_brotli could be compatible in the near feature.
Thanks.

Difference between gzip_static and brotli_static

I have a test setup with a file.js, file.js.gz and file.js.br. If add gzip_static on; to the nginx.conf, the server serves file.js.gz, and the Content-Encoding header is gzip.

If I replace that by brotli on; brotli_types application/javascript;, the server serves the on the fly compressed version of file.js, and the Content-Encoding header is br.

However, when I replace that by brotli_static on;, the server doesn't serve the file.js.br, and there is no Content-Encoding header.

Is there any difference between gzip_static and brotli_static?

This isn’t compiling with nginx 1.11.6

cc1: all warnings being treated as errors
objs/Makefile:1297: recipe for target 'objs/addon/src/ngx_http_brotli_filter_module.o' failed
make[1]: *** [objs/addon/src/ngx_http_brotli_filter_module.o] Error 1
make[1]: Leaving directory '/root/lnmp1.3/src/nginx-1.11.6'
Makefile:8: recipe for target 'build' failed
make: *** [build] Error 2

Some one with 1.9.11 #16

Support dynamic module

Nginx 1.9.11 supports dynamic module.

This requires a new way to compile modules:
https://www.nginx.com/resources/wiki/extending/converting/

It would be great if you could also offer pre compiled binaries.

cannot load libbrotlienc.so.1

nginx 1.10.0
Debian Jessie
ngx_brotli filter module

I've installed libbrotli and this nginx module.

In /usr/local/lib I can see the file:

lrwxrwxrwx  1 root   staff               21 May 29 xx:yy libbrotlidec.so -> libbrotlidec.so.1.0.0
lrwxrwxrwx  1 root   staff               21 May 29 xx:yy libbrotlidec.so.1 -> libbrotlidec.so.1.0.0
-rwxr-xr-x  1 root   staff           287052 May 29 xx:yy libbrotlidec.so.1.0.0

The module builds and I can install nginx, but when I try to execute it/reload the config I get this error message:

nginx: error while loading shared libraries: libbrotlienc.so.1: cannot open shared object file: No such file or directory

can't compiling on nginx 1.10.3 stable

Hello i was try compiling ngx_brotli and get error :

-Wl,-z,relro -Wl,--as-needed -ldl -lpthread -lcrypt /root/coba/ngx_pages                 peed-1.12.34.2-beta/psol/lib/Release/linux/x64/pagespeed_automatic.a -lstdc++ -l                 rt -pthread -lm -lm -lpcre /root/coba/openssl-1.0.2k/.openssl/lib/libssl.a /root                 /coba/openssl-1.0.2k/.openssl/lib/libcrypto.a -ldl -lz \
        -Wl,-E
objs/addon/enc/backward_references.o: In function `SearchInStaticDictionary':
/root/coba/ngx_brotli/deps/brotli/enc/././hash.h:200: undefined reference to `kS                 taticDictionaryHash'
/root/coba/ngx_brotli/deps/brotli/enc/././hash.h:200: undefined reference to `kS                 taticDictionaryHash'
/root/coba/ngx_brotli/deps/brotli/enc/././hash.h:200: undefined reference to `kS                 taticDictionaryHash'
/root/coba/ngx_brotli/deps/brotli/enc/././hash.h:200: undefined reference to `kS                 taticDictionaryHash'
/root/coba/ngx_brotli/deps/brotli/enc/././hash.h:200: undefined reference to `kS                 taticDictionaryHash'
objs/addon/enc/backward_references.o:/root/coba/ngx_brotli/deps/brotli/enc/././h                 ash.h:200: more undefined references to `kStaticDictionaryHash' follow
objs/addon/enc/encode.o: In function `EncodeData':
/root/coba/ngx_brotli/deps/brotli/enc/encode.c:905: undefined reference to `Brot                 liCreateHqZopfliBackwardReferences'
/root/coba/ngx_brotli/deps/brotli/enc/encode.c:898: undefined reference to `Brot                 liCreateZopfliBackwardReferences'
objs/addon/enc/encode.o: In function `BrotliCompressBufferQuality10':
/root/coba/ngx_brotli/deps/brotli/enc/encode.c:1093: undefined reference to `Bro                 tliInitZopfliNodes'
/root/coba/ngx_brotli/deps/brotli/enc/encode.c:1096: undefined reference to `Bro                 tliZopfliComputeShortestPath'
/root/coba/ngx_brotli/deps/brotli/enc/encode.c:1120: undefined reference to `Bro                 tliZopfliCreateCommands'
/usr/bin/ld: objs/nginx: hidden symbol `BrotliZopfliCreateCommands' isn't define                 d
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make[1]: *** [objs/nginx] Error 1
make[1]: Leaving directory `/root/coba/nginx-1.10.3'
make: *** [build] Error 2

anyone can help me? i'm not understand why bad value

question about test case

hi Piotr,
I found there is no test case included into this repo. Is there any test cases (e.g. t/brotli.t) within google?

What could cause the module not to load?

Hi,
Thanks to you, I got nginx to work with this module on one host. However, it won't work on another host with nginx 1.8.0 or 1.9.6:

nginx: [emerg] unknown directive "brotli_static"

Here is my configure line:

./configure --with-cc-opt=-I/usr/local/include --with-ld-opt=-L/usr/local/lib --with-cpu-opt=amd64 --sbin-path=/usr/local/sbin --with-http_ssl_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_gunzip_module --with-http_stub_status_module --with-http_gzip_static_module --add-module=/usr/local/src/ngx_brotli-557486 --conf-path=/etc/nginx/nginx.conf
checking for OS
 + Linux 3.10.23-xxxx-std-ipv6-64 x86_64
checking for C compiler ... found
 + using GNU C compiler
 + gcc version: 4.7.2 (Debian 4.7.2-5)

configuring additional modules
adding module in /usr/local/src/ngx_brotli-557486
checking for Brotli library ... found
 + ngx_brotli was configured

Once the build is complete, the "nginx -t" command works fine, but on startup I get an error on brotli_static directive. For the moment I'll only be using this directive inside a location block (serving files pre-compressed with the bro utility), like this:

brotli_static on;

So I should not need badger's libbrotli, but I built it anyway:

ldd /usr/local/sbin/nginx
        linux-vdso.so.1 =>  (***)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (***)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (***)
        libbrotlienc.so.1 => /usr/local/lib/libbrotlienc.so.1 (***)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (***)
        libpcre.so.1 => /usr/local/lib/libpcre.so.1 (***)
        libssl.so.1.0.0 => /usr/local/lib/libssl.so.1.0.0 (***)
        libcrypto.so.1.0.0 => /usr/local/lib/libcrypto.so.1.0.0 (***)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (***)
        libz.so.1 => /usr/local/lib/libz.so.1 (***)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (***)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (***)
        /lib64/ld-linux-x86-64.so.2 (***)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (***)

Do you know how I can debug this? I'm sure it has nothing to do with your module but most probably something is broken on my setup.
Thanks,

Brotli doesn't work with Firefox and SPDY

Hi,
Could you please provide a sample configuration to serve brotli files? Here are my test files:

static# ls -alh www-en.css*
-rw-r--r-- 1 nginx www-data 212K 2015-10-15 19:51 www-en.css
-rw-r--r-- 1 nginx www-data  28K 2015-10-15 19:51 www-en.css.br
-rw-r--r-- 1 nginx www-data  35K 2015-10-15 19:51 www-en.css.gz
static# nginx -V
nginx version: nginx/1.8.0
built by gcc 4.7.2 (Debian 4.7.2-5)
built with OpenSSL 1.0.2d 9 Jul 2015
TLS SNI support enabled
configure arguments: --with-cc-opt=-I/usr/local/include --with-ld-opt=-L/usr/local/lib --with-cpu-opt=amd64 --sbin-path=/usr/local/sbin --with-http_ssl_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_gunzip_module --with-http_stub_status_module --with-http_gzip_static_module --with-http_spdy_module --conf-path=/etc/nginx/nginx.conf --add-module=/usr/local/src/ngx_brotli-557486

Here is the relevant part of my config (this server block listens on "443 ssl spdy") :

    location ~ ^/static/.* {
        gzip_static off;
        gzip off;
        gzip_proxied off;
        brotli_static on;
        add_header Cache-Control "public";
    }

But it does not seem to catch the .br file:

Accept-Ranges: bytes
Cache-Control: max-age=31536000, public
Content-Length: 216945
Content-Type: text/css
Date: Fri, 16 Oct 2015 06:14:29 GMT
Etag: "561fe7a5-34f71"
Expires: Sat, 15 Oct 2016 06:14:29 GMT
Last-Modified: Thu, 15 Oct 2015 17:51:33 GMT
Server: nginx
X-Firefox-Spdy: 3.1

However, with "gzip_static on" it does serve the .gz file:

Cache-Control: max-age=31536000, public
Content-Encoding: gzip
Content-Length: 34836
Content-Type: text/css
Date: Fri, 16 Oct 2015 06:23:30 GMT
Etag: "561fe7a5-8814"
Expires: Sat, 15 Oct 2016 06:23:30 GMT
Last-Modified: Thu, 15 Oct 2015 17:51:33 GMT
Server: nginx
X-Firefox-Spdy: 3.1

What am I doing wrong? Or is my .br file broken? I checked it with a command like this and I got the expected CSS rules, so I guess the compressed file is ok:

cat www-en.css.br | /usr/local/src/brotli-0.2.0/tools/bro --verbose --decompress | tail

Error while compiling in Debian 8

Hello,

I am using deb-src from the Nginx mainline repo. (nginx is compiling correctly)
I installed libbrotli, but when doing dpkg-buldpackage -b, i get this :

make[1]: Entering directory '/opt/nginx/1.9.10/nginx-1.9.10'
dh_strip --dbg-package=nginx-dbg
make[1]: Leaving directory '/opt/nginx/1.9.10/nginx-1.9.10'
   dh_makeshlibs
   dh_shlibdeps
dpkg-shlibdeps: erreur: pas d'information de dépendance trouvée pour /usr/local/lib/libbrotlienc.so.1 (utilisé par debian/nginx/usr/sbin/nginx)
Indication: vérifiez que la bibliothèque provienne bien d'un paquet.
dh_shlibdeps: dpkg-shlibdeps -Tdebian/nginx.substvars debian/nginx/usr/sbin/nginx-debug debian/nginx/usr/sbin/nginx returned exit code 2
debian/rules:11: recipe for target 'binary' failed
make: *** [binary] Error 2
dpkg-buildpackage: erreur: debian/rules binary a produit une erreur de sortie de type 2

Document whether `ngx_http_brotli_filter_module` caches or not

A lot of the decision between using ngx_http_brotli_filter_module or ngx_http_brotli_static_module depends on whether the former has an in-memory-cache or not. If ngx_http_brotli_filter_module is throwing away previous results then there's a latency hit for every time a resource is requested, if not then it's only for the first time a given resource is requested.

I can't see a cache in ngx_http_brotli_filter_module (not a C developer) and I can't see any ngionx directives to control cache size, so I suspect the answer is 'there is no cache right now' but it would be nice to have confirmation either way.

Thanks!

undefined symbol: _ZN6brotli16 BrotliCompressorD1Ev ?

Every come across this error ? I haven't seen this before

CentOS 7 system

nginx -t                                                                                                                                            
nginx: [emerg] dlopen() "/usr/local/nginx/modules/ngx_http_brotli_filter_module.so" failed (/usr/local/nginx/modules/ngx_http_brotli_filter_module.so: undefined symbol: _ZN6brotli16
BrotliCompressorD1Ev) in /usr/local/nginx/conf/dynamic-modules.conf:4                                                                                                              
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed

./configure --with-ld-opt="-lrt -ljemalloc -lpcre -Wl,-z,relro -Wl,-rpath,/usr/local/lib" --with-cc-opt="-m64 -march=native -g -O3 -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations" --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --add-module=../nginx-module-vts --with-libatomic --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_pagespeed-release-1.11.33.4-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.13 --add-module=../ngx_http_redis-0.3.7 --add-module=../lua-nginx-module-0.10.7 --add-module=../memc-nginx-module-0.17 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.32 --with-pcre=../pcre-8.39 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../libressl-2.4.4
checking for OS

Linux 3.10.0-327.28.3.el7.x86_64 x86_64
checking for C compiler ... found

using GNU C compiler

gcc version: 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC)
checking for gcc -pipe switch ... found
checking for --with-ld-opt="-lrt -ljemalloc -lpcre -Wl,-z,relro -Wl,-rpath,/usr/local/lib" ... found
checking for -Wl,-E switch ... found
checking for gcc builtin atomic operations ... found
checking for C99 variadic macros ... found
checking for gcc variadic macros ... found
checking for gcc builtin 64 bit byteswap ... found
checking for unistd.h ... found
checking for inttypes.h ... found
checking for limits.h ... found
checking for sys/filio.h ... not found
checking for sys/param.h ... found
checking for sys/mount.h ... found
checking for sys/statvfs.h ... found
checking for crypt.h ... found
checking for Linux specific features
checking for epoll ... found
checking for EPOLLRDHUP ... found
checking for EPOLLEXCLUSIVE ... not found
checking for O_PATH ... found
checking for sendfile() ... found
checking for sendfile64() ... found
checking for sys/prctl.h ... found
checking for prctl(PR_SET_DUMPABLE) ... found
checking for sched_setaffinity() ... found
checking for crypt_r() ... found
checking for sys/vfs.h ... found
checking for nobody group ... found
checking for poll() ... found
checking for /dev/poll ... not found
checking for kqueue ... not found
checking for crypt() ... not found
checking for crypt() in libcrypt ... found
checking for F_READAHEAD ... not found
checking for posix_fadvise() ... found
checking for O_DIRECT ... found
checking for F_NOCACHE ... not found
checking for directio() ... not found
checking for statfs() ... found
checking for statvfs() ... found
checking for dlopen() ... not found
checking for dlopen() in libdl ... found
checking for sched_yield() ... found
checking for SO_SETFIB ... not found
checking for SO_REUSEPORT ... found
checking for SO_ACCEPTFILTER ... not found
checking for SO_BINDANY ... not found
checking for IP_BIND_ADDRESS_NO_PORT ... not found
checking for IP_TRANSPARENT ... found
checking for IP_BINDANY ... not found
checking for IP_RECVDSTADDR ... not found
checking for IP_PKTINFO ... found
checking for IPV6_RECVPKTINFO ... found
checking for TCP_DEFER_ACCEPT ... found
checking for TCP_KEEPIDLE ... found
checking for TCP_FASTOPEN ... found
checking for TCP_INFO ... found
checking for accept4() ... found
checking for eventfd() ... found
checking for int size ... 4 bytes
checking for long size ... 8 bytes
checking for long long size ... 8 bytes
checking for void * size ... 8 bytes
checking for uint32_t ... found
checking for uint64_t ... found
checking for sig_atomic_t ... found
checking for sig_atomic_t size ... 4 bytes
checking for socklen_t ... found
checking for in_addr_t ... found
checking for in_port_t ... found
checking for rlim_t ... found
checking for uintptr_t ... uintptr_t found
checking for system byte ordering ... little endian
checking for size_t size ... 8 bytes
checking for off_t size ... 8 bytes
checking for time_t size ... 8 bytes
checking for AF_INET6 ... found
checking for setproctitle() ... not found
checking for pread() ... found
checking for pwrite() ... found
checking for pwritev() ... found
checking for sys_nerr ... found
checking for localtime_r() ... found
checking for posix_memalign() ... found
checking for memalign() ... found
checking for mmap(MAP_ANON|MAP_SHARED) ... found
checking for mmap("/dev/zero", MAP_SHARED) ... found
checking for System V shared memory ... found
checking for POSIX semaphores ... not found
checking for POSIX semaphores in libpthread ... found
checking for struct msghdr.msg_control ... found
checking for ioctl(FIONBIO) ... found
checking for struct tm.tm_gmtoff ... found
checking for struct dirent.d_namlen ... not found
checking for struct dirent.d_type ... found
checking for sysconf(_SC_NPROCESSORS_ONLN) ... found
checking for openat(), fstatat() ... found
checking for getaddrinfo() ... found
configuring additional modules
adding module in ../nginx-module-vts

ngx_http_vhost_traffic_status_module was configured
adding module in ../ngx_cache_purge-2.3

ngx_http_cache_purge_module was configured
adding module in ../ngx_devel_kit-0.3.0

ngx_devel_kit was configured
adding module in ../set-misc-nginx-module-0.31
found ngx_devel_kit for ngx_set_misc; looks good.

ngx_http_set_misc_module was configured
adding module in ../echo-nginx-module-0.60

ngx_http_echo_module was configured
adding module in ../redis2-nginx-module-0.13

ngx_http_redis2_module was configured
adding module in ../ngx_http_redis-0.3.7

ngx_http_redis_module was configured
adding module in ../lua-nginx-module-0.10.7
checking for LuaJIT library in /usr/local/lib and /usr/local/include/luajit-2.1 (specified by the LUAJIT_LIB and LUAJIT_INC env, with -ldl) ... found
checking for export symbols by default (-E) ... found
checking for export symbols by default (--export-all-symbols) ... not found
checking for SO_PASSCRED ... found
checking for mmap(sbrk(0)) ... found
checking for attribute(constructor) ... found
checking for malloc_trim ... found

ngx_http_lua_module was configured
adding module in ../memc-nginx-module-0.17

ngx_http_memc_module was configured
adding module in ../srcache-nginx-module-0.31

ngx_http_srcache_filter_module was configured
adding module in ../headers-more-nginx-module-0.32

ngx_http_headers_more_filter_module was configured
configuring additional dynamic modules
adding module in ../ngx_brotli
checking for Brotli library ... found

ngx_brotli was configured
adding module in ../ngx_pagespeed-release-1.11.33.4-beta
mod_pagespeed_dir=../ngx_pagespeed-release-1.11.33.4-beta/psol/include
build_from_source=false
checking for psol ... found
List of modules (in reverse order of applicability): ngx_http_write_filter_module ngx_http_header_filter_module ngx_http_chunked_filter_module ngx_http_v2_filter_module ngx_http_range_header_filter_module ngx_http_gzip_filter_module ngx_http_postpone_filter_module ngx_http_ssi_filter_module ngx_http_charset_filter_module ngx_http_sub_filter_module ngx_http_addition_filter_module ngx_http_userid_filter_module ngx_http_headers_filter_module
checking for psol-compiler-compat ... found

ngx_pagespeed was configured
adding module in ../ngx-fancyindex-0.4.0

ngx_http_fancyindex_module was configured
checking for zlib library ... found
checking for GD library ... found
checking for GD WebP support ... not found
checking for GeoIP library ... found
checking for GeoIP IPv6 support ... found
checking for atomic_ops library ... found
creating objs/Makefile

Configuration summary

using threads

using PCRE library: ../pcre-8.39

using OpenSSL library: ../libressl-2.4.4

using system zlib library

using system libatomic_ops library

nginx path prefix: "/usr/local/nginx"
nginx binary file: "/usr/local/sbin/nginx"
nginx modules path: "/usr/local/nginx/modules"
nginx configuration prefix: "/usr/local/nginx/conf"
nginx configuration file: "/usr/local/nginx/conf/nginx.conf"
nginx pid file: "/usr/local/nginx/logs/nginx.pid"
nginx error log file: "/usr/local/nginx/logs/error.log"
nginx http access log file: "/usr/local/nginx/logs/access.log"
nginx http client request body temporary files: "client_body_temp"
nginx http proxy temporary files: "proxy_temp"
nginx http fastcgi temporary files: "fastcgi_temp"
nginx http uwsgi temporary files: "uwsgi_temp"
nginx http scgi temporary files: "scgi_temp"

Inconsistency content-encoding header if both brotli and gzip was enabled

Hello!

I got unexpected behavior when enable this directive:

... [truncated] ...
    brotli on;
    brotli_types text/css text/javascript application/json application/javascript;
    brotli_comp_level 9;

    gzip  on;
    gzip_vary  on;
    gzip_static on;
    gzip_comp_level 6;
    gzip_proxied any;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";
    ignore_invalid_headers on;
    gzip_types application/xml+rss;
    gzip_types application/ecmascript;
    gzip_types application/json;
    gzip_types application/javascript;
    gzip_types application/pdf;
    gzip_types application/postscript;
    gzip_types application/x-javascript;
    gzip_types image/svg+xml;
    gzip_types text/javascript;
    gzip_types text/css;
    gzip_types text/csv;
    gzip_types text/plain;
    gzip_types text/xml;
    gzip_http_version 1.1;
... [truncated] ...

Clients can got br or gzip randomly, but it's OK with Google Chrome and Mozilla Firefox. But fails against OSX Safari and Mobile Safari (iOS Device).

Brotli response header was

cache-control:max-age=300
content-encoding:br
content-type:text/css
date:Thu, 13 Apr 2017 07:55:51 GMT
expires:Thu, 13 Apr 2017 08:00:35 GMT
last-modified:Wed, 01 Mar 2017 08:30:42 GMT
server:dmnQ
status:200
vary:Accept-Encoding
x-cache:HIT
x-view:desktop

Gzip response header was

cache-control:max-age=300
content-encoding:gzip
content-type:text/css
date:Thu, 13 Apr 2017 07:58:09 GMT
expires:Thu, 13 Apr 2017 08:02:56 GMT
last-modified:Wed, 01 Mar 2017 08:30:42 GMT
server:dmnQ
status:200
vary:Accept-Encoding
x-cache:HIT
x-view:desktop

Is it normal?

brotli config not recognized

My Nginx has the brotli module compiled in as a static module (with NGX_BROTLI_STATIC_MODULE_ONLY):

# /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.9.5
built by gcc 5.1.0 (Ubuntu 5.1.0-0ubuntu11~14.04.1) 
built with OpenSSL 1.0.1f 6 Jan 2014 (running with OpenSSL 1.0.2d 9 Jul 2015)
TLS SNI support enabled
configure arguments: --add-module=ngx_brotli --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_gzip_static_module --with-http_geoip_module --with-ipv6 --with-cc-opt='-fstack-protector-all -Wl,-z,relro -Wl,-z,now -fPIE -pie -D_FORTIFY_SOURCE=2 -O4 -flto -march=x86-64' --with-ld-opt='-fstack-protector-all -Wl,-z,relro -Wl,-z,now -fPIE -pie -O4 -flto' --with-cc=gcc-5

But the directive is not recognized in config:

root@99ab84326f5b:/webcookies# /usr/local/nginx/sbin/nginx -t -c /nginx/nginx.conf 
nginx: [emerg] unknown directive "brotli" in /nginx/nginx.conf:24
nginx: configuration file /nginx/nginx.conf test failed

The relevant portion of nginx.conf is:

http {
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        server_tokens off;

        include /usr/local/nginx/conf/mime.types.default;
        default_type application/octet-stream;

        access_log /nginx/logs/access.log;
        error_log /nginx/logs/error.log;

        brotli on;
        gzip on;
        gzip_disable "msie6";

possible leak of br encoding to non supported browsers IE/Edge ?

Is it possible for ngx_brotli to incorrectly serve br encoded assets to non supported browsers like IE/Edge ? Seems I am experiencing this at https://community.centminmod.com/posts/41211/ but not sure if it's my Sucuri Cloudproxy's Nginx server in front of my ngx_brotli enabled Nginx server that is causing problems ?

nginx -V
nginx version: nginx/1.11.6
built by gcc 5.2.1 20150902 (Red Hat 5.2.1-2) (GCC)
built with OpenSSL 1.0.2j 26 Sep 2016
TLS SNI support enabled
configure arguments: --with-ld-opt='-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -DTCP_FASTOPEN=23 -g -O3 -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --add-module=../nginx-module-vts --with-libatomic --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_pagespeed-release-1.11.33.4-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-stream_geoip_module=dynamic --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.13 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.17 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.32 --with-pcre=../pcre-8.39 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2j

This isn’t compiling with nginx 1.9.11

The end of the output for make:

    -o objs/addon/src/ngx_http_brotli_filter_module.o \
    /home/jonny/server-files/nginx/ngx_brotli/src/ngx_http_brotli_filter_module.cc
/home/jonny/server-files/nginx/ngx_brotli/src/ngx_http_brotli_filter_module.cc:173:1: error: deprecated conversion from string constant to ‘char*’ [-Werror=write-strings]
 };
 ^
cc1plus: all warnings being treated as errors
objs/Makefile:1244: recipe for target 'objs/addon/src/ngx_http_brotli_filter_module.o' failed
make[1]: *** [objs/addon/src/ngx_http_brotli_filter_module.o] Error 1
make[1]: Leaving directory '/home/jonny/server-files/nginx/nginx-1.9.11'
Makefile:8: recipe for target 'build' failed
make: *** [build] Error 2

[Error] Dynamic Module

Hi, Frienda.

Compiler Dynamic Module, error (module "/etc/nginx/modules/ngx_http_brotli_filter_module.so" version 1010001 instead of 1011002 in /etc/nginx/nginx.conf:1)

Precendence over gzip

My Firefox browser is set to Accept-Encoding: br, gzip, deflate. I have both gzip and brotli turned on on the server, but gzip seems to take precedence over brotli when I view Content-Encoding for the response. If I turn off gzip on the server or take gzip off the Accept-Encoding from the browser's request, I'll get "br" as the Content-Encoding. Is there a way to change this?

Latest pull on brotli & ngx_brotli, nginx 1.9.7.rc1 (openresty)
Firefox v45.0a

modules

Performance measurements or config recommendations available?

Hi,

I'm working on wrapping Brotli into a Java library to make it available to Java developers.
My goal is similar to what ngx_brotli does: I want to enable such compression in Java based HTTP servers.

I'm curios if someone of you did performance measurements?
Or maybe can recommend certain config settings (e.g. buffer and window size) for ngx_brotli.

Background information:
My current benchmarks on canterburry corpus (cp.hml) shows
that gzip (default level) is roughly twice as fast as brotli (level 5).
(See https://github.com/nitram509/jbrotli).

ngx_brotli uses deprecated Brotli API.

installing ngx_brotli as dynamic module on CentOS 6.8 64bit with devtoolset-3 GCC 4.9.2 get following error during nginx make stage

                -o objs/addon/src/ngx_http_brotli_filter_module.o \
                ../ngx_brotli/src/ngx_http_brotli_filter_module.c
../ngx_brotli/src/ngx_http_brotli_filter_module.c: In function ‘ngx_http_brotli_body_filter’:
../ngx_brotli/src/ngx_http_brotli_filter_module.c:272:9: error: ‘BrotliEncoderInputBlockSize’ is deprecated (declared at /usr/local/include/brotli/encode.h:87) [-Werror=deprecated-declarations]
         ctx->brotli_ring = BrotliEncoderInputBlockSize(ctx->encoder);
         ^
../ngx_brotli/src/ngx_http_brotli_filter_module.c: In function ‘ngx_http_brotli_filter_add_data’:
../ngx_brotli/src/ngx_http_brotli_filter_module.c:498:5: error: ‘BrotliEncoderCopyInputToRingBuffer’ is deprecated (declared at /usr/local/include/brotli/encode.h:95) [-Werror=deprecated-declarations]
     BrotliEncoderCopyInputToRingBuffer(ctx->encoder, size, b->pos);
     ^
../ngx_brotli/src/ngx_http_brotli_filter_module.c: In function ‘ngx_http_brotli_filter_process’:
../ngx_brotli/src/ngx_http_brotli_filter_module.c:534:5: error: ‘BrotliEncoderWriteData’ is deprecated (declared at /usr/local/include/brotli/encode.h:109) [-Werror=deprecated-declarations]
     if (!BrotliEncoderWriteData(ctx->encoder, ctx->last, ctx->flush, &size,
     ^
../ngx_brotli/src/ngx_http_brotli_filter_module.c: At top level:
cc1: error: unrecognized command line option "-Wno-c++11-extensions" [-Werror]
cc1: all warnings being treated as errors
make[1]: *** [objs/addon/src/ngx_http_brotli_filter_module.o] Error 1
make[1]: Leaving directory `/svr-setup/nginx-1.11.5'
make: *** [build] Error 2

the configure options with nginx 1.11.5

./configure --with-ld-opt="-ljemalloc -lpcre -Wl,-z,relro -Wl,-rpath,/usr/local/lib" --with-cc-opt="-m64 -march=native -g -O3 -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2" --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --add-module=../nginx-module-vts --with-libatomic --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_pagespeed-release-1.11.33.4-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-stream_geoip_module --with-stream_realip_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-dynamic-module=../set-misc-nginx-module-0.31 --add-dynamic-module=../echo-nginx-module-0.60 --add-dynamic-module=../redis2-nginx-module-0.13 --add-module=../ngx_http_redis-0.3.7 --add-module=../lua-nginx-module-0.10.6 --add-dynamic-module=../memc-nginx-module-0.17 --add-dynamic-module=../srcache-nginx-module-0.31 --add-dynamic-module=../headers-more-nginx-module-0.31 --with-pcre=../pcre-8.39 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2j

Add option to compress proxied requests

Gzip nginx module supports gzip_proxied option that

Enables or disables gzipping of responses for proxied requests

Likewise it was useful to have a brotli one (brotli_proxied) :)

recompiling nginx (1.11.3) for newest Brotli failed (ngx_brotli filter module requires Brotli library)

`configuring additional modules
adding module in ../nginx-module-vts

ngx_http_vhost_traffic_status_module was configured
adding module in ../ngx_cache_purge-2.3
ngx_http_cache_purge_module was configured
adding module in ../ngx_devel_kit-0.3.0
ngx_devel_kit was configured
adding module in ../set-misc-nginx-module-0.31
found ngx_devel_kit for ngx_set_misc; looks good.
ngx_http_set_misc_module was configured
adding module in ../echo-nginx-module-0.60
ngx_http_echo_module was configured
adding module in ../redis2-nginx-module-0.13
ngx_http_redis2_module was configured
adding module in ../ngx_http_redis-0.3.7
ngx_http_redis_module was configured
adding module in ../memc-nginx-module-0.17
ngx_http_memc_module was configured
adding module in ../srcache-nginx-module-0.31
ngx_http_srcache_filter_module was configured
adding module in ../headers-more-nginx-module-0.31
ngx_http_headers_more_filter_module was configured
configuring additional dynamic modules
adding module in ../ngx_brotli
checking for Brotli library ... not found
checking for Brotli library in /usr/local/ ... not found
checking for Brotli library in /usr/pkg/ ... not found
checking for Brotli library in /opt/local/ ... not found
./configure: error: ngx_brotli filter module requires Brotli library.

Sun Aug 21 22:43:22 CEST 2016
Error: 1, Nginx configure failed

Total Nginx Upgrade Time: 162.504159814 seconds`

# nginx -V nginx version: nginx/1.11.3 built by clang 3.4.2 (tags/RELEASE_34/dot2-final) built with LibreSSL 2.4.2 TLS SNI support enabled configure arguments: --with-ld-opt='-lrt -ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -mtune=native -mfpmath=sse -g -O3 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-sign-compare -Wno-string-plus-int -Wno-deprecated-declarations -Wno-unused-parameter -Wno-unused-const-variable -Wno-conditional-uninitialized -Wno-mismatched-tags -Wno-c++11-extensions -Wno-sometimes-uninitialized -Wno-parentheses-equality -Wno-tautological-compare -Wno-self-assign -Wno-deprecated-register -Wno-deprecated -Wno-invalid-source-encoding -Wno-pointer-sign -Wno-parentheses -Wno-enum-conversion -Wno-c++11-compat-deprecated-writable-strings -Wno-write-strings' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --with-openssl-opt=enable-tlsext --add-module=../nginx-module-vts --with-libatomic --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.30 --add-module=../echo-nginx-module-0.59 --add-module=../redis2-nginx-module-0.13 --add-module=../ngx_http_redis-0.3.7 --add-module=../memc-nginx-module-0.17 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.30 --with-pcre=../pcre-8.39 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../libressl-2.4.2

Ngx_brotli module with a bundled encoder

Is it an idea to deliver the Ngx_brotli module standalone with a bundled encoder?
So you get a well tested complete ''all in'' package without external dependencies.
Just like the Cloudflare brotli Nginx module.

Just because:
a lot of different and possibly untested libbrotli's used with Ngx_brotli
Only distro Ubuntu delivers Brotli with their latest release but no one delivers libbrotlienc.
So you need to maintain another package yourself.
Many reported issues or uncertainties are all about loading or linking problems with libbrotlienc or compiling issues.

ngx_brotli can be a nginx dynamic module?

Can ngx_brotli be compiled as a dynamic module?

https://www.nginx.com/resources/wiki/extending/converting/

Ngx_brotli compatibility with Brotli 0.4.0

Brotli 0.4.0 is released.
Is Ngx_brotli compatible with Brotli 0.4.0?
(I know it depends on libbrotli, which has been updated to version 0.40 a few minutes ago).

How to compile with both the static brotli module and the dynamic one?

How can I use both the static and the dynamic brotli module?
I mean of course I have both static files (which might be well-served pre-compressed) and other files which I might want to dynamically compress...

invalid number of arguments in "brotli_static" directive

Hi,

I just built rev-557486 as a module on an nginx-1.8.0, then added the following line inside a server context (ssl):

brotli_static on

nginx stop then nginx start gives me the following:

nginx: [emerg] invalid number of arguments in "brotli_static" directive in ...

What am I doing wrong?
Thanks,

libbrotli include files path change

It seems that libbrotli changed the include files path(removed the enc dir) and I can't compile ngx_brotli with nginx anymore.

bagder/libbrotli@5a62fd9
bagder/libbrotli@ff849d1

symlinks not followed

brotli_static does not follow symlinks like gzip_static does.

I'm not sure what's going on here. If the request path needs to follow a symlink, .br files are never picked up, not even with always. .gz files are used without a problem. If i place all files directly in the root folder, not following any symlinks, brotli_static starts to work fine.

I should probably stop using symlinks. Still an odd behaviour.

ngx_brotli + ngx_pagespeed

ngx_brotli might have broken ngx_pagespeed support over https apache/incubator-pagespeed-ngx#1021 ?

Seeing as both Google projects, hope to see them working nicely together :)

BR content-encoding only works with static files like .css .js

But not working on full html pages.

My config:

brotli on;
brotli_static on;
brotli_min_length 1000;
brotli_buffers 32 8k;
brotli_comp_level 5;
brotli_types text/plain text/css text/xml application/javascript application/x-javascript application/xml application/xml+rss application/ecmascript application/json image/svg+xml;

Cannot compile with nginx 1.9.12

Hello,

I am on Ubuntu 14.04.4 LTS (GNU/Linux 2.6.32-042stab108.6 x86_64) and used mainline version of nginx source (apt-get source nginx).
The steps that i did:

Added "--add-module=/opt/ngx_brotli" to rules file in "nginx/debian" folder.
Installed libbrotli from https://github.com/bagder/libbrotli
When I run the "dpkg-buildpackage -b", the compilation is stop at about 99% and gives this error:

...
make[1]: Leaving directory `/opt/nginx-1.9.12'
dh_makeshlibs
dh_shlibdeps
dpkg-shlibdeps: error: no dependency information found for /usr/local/lib/libbrotlienc.so.1 (used by debian/nginx/usr/sbin/nginx-debug)
dh_shlibdeps: dpkg-shlibdeps -Tdebian/nginx.substvars debian/nginx/usr/sbin/nginx debian/nginx/usr/sbin/nginx-debug returned exit code 2
make: *** [binary] Error 2
dpkg-buildpackage: error: debian/rules binary gave error exit status 2

Tried to use ""--add-dynamic-module=/opt/ngx_brotli", it made the compilation, but nginx isn't recognise the "brotli on;" command

Please help!

Thank You,
Mark

Dynamic modules do not compile

To compile dynamic modules I had to add rule

modules:
    $(MAKE) -f objs/Makefile modules

and issue make modules.
By default dynamic modules do not compile.

ngx_brotli + nginx SSI

Not sure if it's related but in Firefox 45 nightly browser with Nginx compiled with ngx_brotli support, all my nginx SSI include text and html files have not loaded over https HTTP/2 on my site at https://centminmod.com/ ?

i.e. SSI include files are used for nav menu and sidebar links https://centminmod.com/news.html they all show up in Chrome 46, Opera 33 but not in Firefox 45 nightly or Firefox 44 dev editions ?

my Nginx configuration

nginx -V
nginx version: nginx/1.9.6
built by clang 3.4.2 (tags/RELEASE_34/dot2-final)
built with LibreSSL 2.2.4
TLS SNI support enabled
configure arguments: --with-ld-opt='-lrt -ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -mtune=native -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations -Wno-unused-parameter -Wno-unused-const-variable -Wno-conditional-uninitialized -Wno-mismatched-tags -Wno-c++11-extensions -Wno-sometimes-uninitialized -Wno-parentheses-equality -Wno-tautological-compare -Wno-self-assign -Wno-deprecated-register -Wno-deprecated -Wno-invalid-source-encoding -Wno-pointer-sign -Wno-parentheses -Wno-enum-conversion' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --with-openssl-opt=enable-tlsext --add-module=../nginx-module-vts --with-libatomic --with-threads --with-stream --with-stream_ssl_module --with-http_gzip_static_module --add-module=../ngx_brotli --add-module=../ngx_pagespeed-release-1.9.32.10-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module --with-http_geoip_module --with-http_realip_module --add-module=../nginx-accesskey-2.0.3 --add-module=../nginx-http-concat-master --add-module=../ngx-fancyindex-ngx-fancyindex --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.2.19 --add-module=../set-misc-nginx-module-0.29 --add-module=../echo-nginx-module-0.58 --add-module=../redis2-nginx-module-0.12 --add-module=../ngx_http_redis-0.3.7 --add-module=../lua-nginx-module-0.9.16 --add-module=../lua-upstream-nginx-module-0.03 --add-module=../lua-upstream-cache-nginx-module-0.1.1 --add-module=../nginx_upstream_check_module-0.3.0 --add-module=../openresty-memc-nginx-module-4f6f78f --add-module=../openresty-srcache-nginx-module-ffa9ab7 --add-module=../headers-more-nginx-module-0.261 --with-pcre=../pcre-8.37 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../libressl-2.2.4

settings

brotli on;
brotli_static on;
brotli_min_length 1000;
brotli_buffers 32 8k;
brotli_comp_level 5;
brotli_types text/plain text/css text/xml application/javascript application/x-javascript application/xml application/xml+rss application/ecmascript application/json image/svg+xml;

Vary Accept-Encoding header

Hi
Can this module add an "vary: accept-encoding" same as gzip module do? Can this be considered an enhancement? Is this feasible?

Thx in advance

nginx: error while loading shared libraries

Hello

I get this error after compiling Nginx 1.9.12 on Debian 8 :

root@server:~# nginx -V
nginx: error while loading shared libraries: libbrotlienc.so.1: cannot open shared object file: No such file or directory

I am using this script to install nginx + brotli : https://github.com/Angristan/nginx-autoinstall/blob/master/nginx-brotli.sh

What am I doing wrong ?

Sharing MIME type list using GZIP and BROTLI in NGINX

I want to enable GZIP and Brotli compression using NGINX. I have to supply each with their own list of MIME types like so in my nginx.conf:

gzip_types   text/plain
             text/css
             ...etc;

brotli_types text/plain
             text/css
             ...etc;

How can I create a single list of MIME types that can be used by both settings? Also posted on StackOverflow.

How to use Brotli on the CLI for pre-compiling files to use with brotli_static?

As you offer server admins to pre-compile files with Brotli and use them with brotli_static it would be nice if you could also explain how to use Brotli on the CLI to pre-compress the files.
Do we need to recompile libbrotli and if so how? /cc @bagder

Build failure on Debian

In 28th line of auto/feature there is ==. I guess one = is better.

php text/html files no longer br encoded ?

Previous ngx_brotli versions showed br encoding for php files with text/html mime types. But latest with Nginx 1.9.12-1.9.13 seem to do br encoding only for static files except php files ?

nginx -V
nginx version: nginx/1.9.13
built by gcc 4.9.1 20140922 (Red Hat 4.9.1-10) (GCC)
built with LibreSSL 2.3.3
TLS SNI support enabled
configure arguments: --with-ld-opt='-lrt -ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -mtune=native -mfpmath=sse -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --with-openssl-opt=enable-tlsext --add-module=../nginx-module-vts --with-libatomic --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_pagespeed-release-1.10.33.6-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_realip_module --add-module=../ngx-fancyindex-0.3.6 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0rc1 --add-module=../set-misc-nginx-module-0.30 --add-module=../echo-nginx-module-0.58 --add-module=../redis2-nginx-module-0.12 --add-module=../ngx_http_redis-0.3.7 --add-module=../nginx_upstream_check_module-0.3.0 --add-module=../openresty-memc-nginx-module-4f6f78f --add-module=../srcache-nginx-module-0.30 --add-module=../headers-more-nginx-module-0.29 --with-pcre=../pcre-8.38 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../libressl-2.3.3

i am currently using older ngx_brotli version and php files get working br encoding https://community.centminmod.com/

git log

git log -1
commit 86998c61fab726d4ee21fdf7e96f80e0cffbc71a
Author: Piotr Sikora <>
Date:   Tue Feb 9 19:34:36 2016 -0800

    Brotli: add support for dynamic modules.

    Fixes build with nginx-1.9.11+.

    Signed-off-by: Piotr Sikora <>

Infinite for loop when upstream partially failing

Hi,

when the network is not 100% reliable and part of the response from upstream is lost then the loop in ngx_http_brotli_body_filter seems to be infinite. This happened when the connection to upstream was successful, the upstream returned some data but then it stopped responding and nginx's proxy_read_timeout has run out.

According to gdb ngx_http_brotli_filter_add_data returns NGX_OK and then ngx_http_brotli_filter_process returns NGX_AGAIN. This consumes all the resources dedicated to given worker process and stays like this forever. Stracing such process shows no system calls, which supports the idea of infinite loop in user space. Below are few lines from gdb when breakpoint was set at ngx_http_brotli_body_filter and then the code was observed using next/nexti/step/stepi/finish & bt gdb commands.

`
(gdb) bt
#0 ngx_http_brotli_body_filter (r=0x1fbfd70, in=0x7fffd1099500) at ../modules/ngx_brotli-module/src/ngx_http_brotli_filter_module.cc:324
#1 0x00000000004a5e0f in ngx_http_postpone_filter (r=0x1fbfd70, in=0x7fffd1099500) at src/http/ngx_http_postpone_filter_module.c:82
#2 0x00000000004a6448 in ngx_http_ssi_body_filter (r=0x1fbfd70, in=0x7fffd1099500) at src/http/modules/ngx_http_ssi_filter_module.c:411
#3 0x00000000004ac609 in ngx_http_charset_body_filter (r=0x1fbfd70, in=0x7fffd1099500) at src/http/modules/ngx_http_charset_filter_module.c:557
#4 0x00000000004e36c3 in ngx_http_mp4_body_filter (req=0x1fbfd70, in=0x7fffd1099500) at ../modules/ngx_mp4_filter_module/ngx_http_mp4_filter_module.c:190
#5 0x000000000054572d in ngx_http_lua_capture_body_filter (r=0x1fbfd70, in=0x7fffd1099500) at ../modules/lua-nginx-module/src/ngx_http_lua_capturefilter.c:133
#6 0x000000000042876a in ngx_output_chain (ctx=0x197abb0, in=0x7fffd1099500) at src/core/ngx_output_chain.c:74
#7 0x00000000004b1684 in ngx_http_copy_filter (r=0x1fbfd70, in=0x7fffd1099500) at src/http/ngx_http_copy_filter_module.c:152
#8 0x00000000004a33e8 in ngx_http_range_body_filter (r=0x1fbfd70, in=0x7fffd1099500) at src/http/modules/ngx_http_range_filter_module.c:623
#9 0x00000000004675e8 in ngx_http_output_filter (r=0x1fbfd70, in=0x7fffd1099500) at src/http/ngx_http_core_module.c:1978
#10 0x00000000004765ef in ngx_http_send_special (r=0x1fbfd70, flags=2) at src/http/ngx_http_request.c:3346
#11 0x000000000048fbf8 in ngx_http_upstream_finalize_request (r=0x1fbfd70, u=0x315b868, rc=-1) at src/http/ngx_http_upstream.c:4107
#12 0x000000000048ef9a in ngx_http_upstream_process_request (r=0x1fbfd70, u=0x315b868) at src/http/ngx_http_upstream.c:3704
#13 0x000000000048ecbb in ngx_http_upstream_process_upstream (r=0x1fbfd70, u=0x315b868) at src/http/ngx_http_upstream.c:3628
#14 0x0000000000489865 in ngx_http_upstream_handler (ev=0x7f69659cf250) at src/http/ngx_http_upstream.c:1111
#15 0x000000000044c383 in ngx_event_expire_timers () at src/event/ngx_event_timer.c:94
#16 0x000000000044a21e in ngx_process_events_and_timers (cycle=0x18ee660) at src/event/ngx_event.c:272
#17 0x00000000004550ef in ngx_worker_process_cycle (cycle=0x18ee660, data=0x0) at src/os/unix/ngx_process_cycle.c:765
#18 0x0000000000451f71 in ngx_spawn_process (cycle=0x18ee660, proc=0x455027 <ngx_worker_process_cycle>, data=0x0, name=0x5a49b8 "worker process", respawn=-3)

at src/os/unix/ngx_process.c:202

#19 0x00000000004542aa in ngx_start_worker_processes (cycle=0x18ee660, n=1, type=-3) at src/os/unix/ngx_process_cycle.c:366
#20 0x0000000000453a14 in ngx_master_process_cycle (cycle=0x18ee660) at src/os/unix/ngx_process_cycle.c:138
#21 0x000000000042140c in main (argc=1, argv=0x7fffd1099c08) at src/core/nginx.c:367

`
#0 ngx_http_brotli_body_filter (r=0x1fbfd70, in=0x7fffd1099500) at ../modules/ngx_brotli-module/src/ngx_http_brotli_filter_module.cc:324

    rc = 0
    flush = 0
    cl = 0x197ab20
    ctx = 0x197a610

(gdb) print *in
$14 = {buf = 0x197ab60, next = 0x0}
(gdb) print *in->buf
$16 = {pos = 0x0, last = 0x0, file_pos = 0, file_last = 0, start = 0x0, end = 0x0, tag = 0x0, file = 0x0, shadow = 0x0, temporary = 0, memory = 0, mmap = 0, recycled = 0,
in_file = 0, flush = 1, sync = 0, last_buf = 0, last_in_chain = 0, last_shadow = 0, temp_file = 0, num = 0}

Code of libbrotli/brotli/ngx_brotli is current as of today and we are able to replicate this w/o any problem so if you need anything else to debug this, just let me know.

Thanks,
Tomas

Allow for libbrotli to be compiled directly into ngx_brotli

Compiling from source.

Would be nice if there is an option to compile libbrotli right into the ngx_brotli module which is in turn compiled into nginx.

Reason: Ubuntu 16.04 does not ship with libbrotli

Latest brotli bundle library not compatible with ngx_brotli

Hello!

Trying to update brotli sub-module to latest brotli library, but it was failed. Is there any change log or is there any reason that we should stay or update to latest bundle?

Many thanks.

configure: error: Brotli library is missing from the ../ngx_brotli/deps/brotli directory ?

CentOS 7.2 with Nginx 1.11.6 getting compile error

configure: error: Brotli library is missing from the ../ngx_brotli/deps/brotli directory

and seems to prompt for the following, is this a new procedure needed for updates ?

cd ../ngx_brotli && git submodule update --init && cd /svr-setup/nginx-1.11.6

configure options

./configure --with-ld-opt="-lrt -ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib" --with-cc-opt="-m64 -march=native -g -O3 -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations" --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --add-module=../nginx-module-vts --with-libatomic --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.13 --add-module=../ngx_http_redis-0.3.7 --add-module=../nginx-sticky-module-master --add-module=../memc-nginx-module-0.17 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.32 --with-pcre=../pcre-8.39 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../libressl-2.4.4

checking for OS
 + Linux 4.7.5-1.el7.elrepo.x86_64 x86_64
checking for C compiler ... found
 + using GNU C compiler
 + gcc version: 5.2.1 20150902 (Red Hat 5.2.1-2) (GCC) 
checking for gcc -pipe switch ... found
checking for --with-ld-opt="-lrt -ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib" ... found
checking for -Wl,-E switch ... found
checking for gcc builtin atomic operations ... found
checking for C99 variadic macros ... found
checking for gcc variadic macros ... found
checking for gcc builtin 64 bit byteswap ... found
checking for unistd.h ... found
checking for inttypes.h ... found
checking for limits.h ... found
checking for sys/filio.h ... not found
checking for sys/param.h ... found
checking for sys/mount.h ... found
checking for sys/statvfs.h ... found
checking for crypt.h ... found
checking for Linux specific features
checking for epoll ... found
checking for EPOLLRDHUP ... found
checking for EPOLLEXCLUSIVE ... not found
checking for O_PATH ... found
checking for sendfile() ... found
checking for sendfile64() ... found
checking for sys/prctl.h ... found
checking for prctl(PR_SET_DUMPABLE) ... found
checking for sched_setaffinity() ... found
checking for crypt_r() ... found
checking for sys/vfs.h ... found
checking for nobody group ... found
checking for poll() ... found
checking for /dev/poll ... not found
checking for kqueue ... not found
checking for crypt() ... not found
checking for crypt() in libcrypt ... found
checking for F_READAHEAD ... not found
checking for posix_fadvise() ... found
checking for O_DIRECT ... found
checking for F_NOCACHE ... not found
checking for directio() ... not found
checking for statfs() ... found
checking for statvfs() ... found
checking for dlopen() ... not found
checking for dlopen() in libdl ... found
checking for sched_yield() ... found
checking for SO_SETFIB ... not found
checking for SO_REUSEPORT ... found
checking for SO_ACCEPTFILTER ... not found
checking for SO_BINDANY ... not found
checking for IP_BIND_ADDRESS_NO_PORT ... not found
checking for IP_TRANSPARENT ... found
checking for IP_BINDANY ... not found
checking for IP_RECVDSTADDR ... not found
checking for IP_PKTINFO ... found
checking for IPV6_RECVPKTINFO ... found
checking for TCP_DEFER_ACCEPT ... found
checking for TCP_KEEPIDLE ... found
checking for TCP_FASTOPEN ... found
checking for TCP_INFO ... found
checking for accept4() ... found
checking for eventfd() ... found
checking for int size ... 4 bytes
checking for long size ... 8 bytes
checking for long long size ... 8 bytes
checking for void * size ... 8 bytes
checking for uint32_t ... found
checking for uint64_t ... found
checking for sig_atomic_t ... found
checking for sig_atomic_t size ... 4 bytes
checking for socklen_t ... found
checking for in_addr_t ... found
checking for in_port_t ... found
checking for rlim_t ... found
checking for uintptr_t ... uintptr_t found
checking for system byte ordering ... little endian
checking for size_t size ... 8 bytes
checking for off_t size ... 8 bytes
checking for time_t size ... 8 bytes
checking for AF_INET6 ... found
checking for setproctitle() ... not found
checking for pread() ... found
checking for pwrite() ... found
checking for pwritev() ... found
checking for sys_nerr ... found
checking for localtime_r() ... found
checking for posix_memalign() ... found
checking for memalign() ... found
checking for mmap(MAP_ANON|MAP_SHARED) ... found
checking for mmap("/dev/zero", MAP_SHARED) ... found
checking for System V shared memory ... found
checking for POSIX semaphores ... not found
checking for POSIX semaphores in libpthread ... found
checking for struct msghdr.msg_control ... found
checking for ioctl(FIONBIO) ... found
checking for struct tm.tm_gmtoff ... found
checking for struct dirent.d_namlen ... not found
checking for struct dirent.d_type ... found
checking for sysconf(_SC_NPROCESSORS_ONLN) ... found
checking for openat(), fstatat() ... found
checking for getaddrinfo() ... found
configuring additional modules
adding module in ../nginx-module-vts
 + ngx_http_vhost_traffic_status_module was configured
adding module in ../ngx_cache_purge-2.3
 + ngx_http_cache_purge_module was configured
adding module in ../ngx_devel_kit-0.3.0
 + ngx_devel_kit was configured
adding module in ../set-misc-nginx-module-0.31
found ngx_devel_kit for ngx_set_misc; looks good.
 + ngx_http_set_misc_module was configured
adding module in ../echo-nginx-module-0.60
 + ngx_http_echo_module was configured
adding module in ../redis2-nginx-module-0.13
 + ngx_http_redis2_module was configured
adding module in ../ngx_http_redis-0.3.7
 + ngx_http_redis_module was configured
adding module in ../nginx-sticky-module-master
 + ngx_http_sticky_module was configured
adding module in ../memc-nginx-module-0.17
 + ngx_http_memc_module was configured
adding module in ../srcache-nginx-module-0.31
 + ngx_http_srcache_filter_module was configured
adding module in ../headers-more-nginx-module-0.32
 + ngx_http_headers_more_filter_module was configured
configuring additional dynamic modules
adding module in ../ngx_brotli

./configure: error: Brotli library is missing from the ../ngx_brotli/deps/brotli directory.

Please make sure that the git submodule has been checked out:

    cd ../ngx_brotli && git submodule update --init && cd /svr-setup/nginx-1.11.6

google / ngx_brotli Goto Github PK

ngx_brotli's Introduction

ngx_brotli

Table of Contents

Status

Installation

Statically compiled

Dynamically loaded

Configuration directives

brotli_static

brotli

brotli_types

brotli_buffers

brotli_comp_level

brotli_window

brotli_min_length