Git Product home page Git Product logo

zstd-nginx-module's Introduction

Name

zstd-nginx-module - Nginx module for the Zstandard compression.

Table of Content

Status

This Nginx module is currently considered experimental. Issues and PRs are welcome if you encounter any problems.

Synopsis

# specify the dictionary
zstd_dict_file /path/to/dict;

server {
    listen 127.0.0.1:8080;
    server_name localhost;

    location / {
        # enable zstd compression
        zstd on;
        zstd_min_length 256; # no less than 256 bytes
        zstd_comp_level 3; # set the level to 3

        proxy_pass http://foo.com;
    }
}

server {
    listen 127.0.0.1:8081;
    server_name localhost;

    location / {
        zstd_static on;
        root html;
    }
}

Installation

To use theses modules, configure your nginx branch with --add-module=/path/to/zstd-nginx-module. Several points should be taken care.

  • You can set environment variables ZSTD_INC and ZSTD_LIB to specify the path to zstd.h and the path to zstd shared library represently.
  • static library will be tried prior to dynamic library, since this Nginx module uses some advanced APIs where static linking is recommended.
  • System's zstd bundle will be linked if ZSTD_INC and ZSTD_LIB are not specified.
  • Both ngx_http_zstd_static_module and ngx_http_zstd_filter_module will be configured.

Directives

ngx_http_zstd_filter_module

The ngx_http_zstd_filter_module module is a filter that compresses responses using the "zstd" method. This often helps to reduce the size of transmitted data by half or even more.

zstd_dict_file

Syntax: zstd_dict_file /path/to/dict;
Default: -
Context: http

Specifies the external dictionary.

WARNING: Be careful! The content-coding registration only specifies a means to signal the use of the zstd format, and does not additionally specify any mechanism for advertising/negotiating/synchronizing the use of a specific dictionary between client and server. Use the zstd_dict_file only if you can insure that both ends (server and client) are capable of using the same dictionary (e.g. advertise with a HTTP header). See #2 for the details.

zstd

Syntax: zstd on | off;
Default: zstd off;
Context: http, server, location, if in location

Enables or disables zstd compression for response.

zstd_comp_level

Syntax: zstd_comp_level level;
Default: zstd_comp_level 1;
Context: http, server, location

Sets a zstd compression level of a response. Acceptable values are in the range from 1 to ZSTD_maxCLevel().

zstd_min_length

Syntax: zstd_min_length length;
Default: zstd_min_length 20;
Context: http, server, location

Sets the minimum length of a response that will be compressed by zstd. The length is determined only from the "Content-Length" response header field.

zstd_types

Syntax: zstd_types mime-type ...;
Default: zstd_types text/html;
Context: http, server, location

Enables ztd of responses for the specified MIME types in addition to "text/html". The special value "*" matches any MIME type.

zstd_buffers

Syntax: zstd_buffers number size;
Default: zstd_buffers 32 4k | 16 8k;
Context: http, server, location

Sets the number and size of buffers used to compress a response. By default, the buffer size is equal to one memory page. This is either 4K or 8K, depending on a platform.

ngx_http_zstd_static_module

The ngx_http_zstd_static_module module allows sending precompressed files with the ".zst" filename extension instead of regular files.

zstd_static

Syntax: zstd_static on | off | always;
Default: zstd_static off;
Context: http, server, location

Enables ("on") or disables ("off") checking the existence of precompressed files. The following directives are also taken into account: gzip_vary.

With the "always" value, "zsted" file is used in all cases, without checking if the client supports it.

Variables

ngx_http_zstd_filter_module

$zstd_ratio

Achieved compression ratio, computed as the ratio between the original and compressed response sizes.

Author

Alex Zhang (张超) zchao1995@gmail, UPYUN Inc.

License

This Nginx module is licensed under BSD 2-Clause License.

zstd-nginx-module's People

Contributors

drawing avatar eilandert avatar hanadalee avatar juliyvchirkov avatar supersandro2000 avatar tokers avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zstd-nginx-module's Issues

Issues with compression not working with browsers

I have the module set up with nginx, compiled statically and running locally with the config here for testing. It appears that even with small content like this i am running into issues with zstd not being supported in most browsers. That said it works fine in other utilities such as curl. Firefox gives a generic "Unsupported compression method" error page, and chromium-based browsers show a blank page with a net::ERR_ZSTD_WINDOW_SIZE_TOO_BIG error. Are there any ways currently to fix this issue? Lowering the compression level to 19 seems to be a solution, but i would love to be able to go all the way up to 22 assuming my server has the capacity in the future.

I'd be happy to debug more as well and/or provide more info if asked. If it's unavoidable with the higher compression levels i might recommend just adding a note in the readme that higher levels may cause issues with browsers, in case others run into this as well :)

Support for dynamic-module build

./configure --add-dynamic-module=/zstd
[   20s] configuring additional dynamic modules
[   20s] adding module in /zstd
[   20s]     ./configure: error: ngx_http_zstd_filter_module can only be linked as an addon.
[   20s] checking for ZStandard static library ... not found
[   20s] checking for ZStandard dynamic library ... found

Would it be possible to support dynamic mode? ngx_brotli for example can do dynamic, so it seems that filters don't have a restriction to be hard-builtin.

Error compilation

Compilation error on latest version nginx 1.25.1 gcc 10. debian 12
changing compilation options -Werror -Wno-deprecated-declarations -Wno-deprecated, had no effect

gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 10.2.1-6' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-mutex
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.2.1 20210110 (Debian 10.2.1-6)

--

-o objs/addon/filter/ngx_http_zstd_filter_module.o
/usr/local/src/nginx/modules/zstd-nginx-module/filter/ngx_http_zstd_filter_module.c
cc -c -DZSTD_STATIC_LINKING_ONLY -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -Wno-ignored-qualifiers -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /usr/local/src/nginx/modules/nginx-rtmp-module -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I -DZSTD_STATIC_LINKING_ONLY -I -DZSTD_STATIC_LINKING_ONLY
-o objs/addon/static/ngx_http_zstd_static_module.o
/usr/local/src/nginx/modules/zstd-nginx-module/static/ngx_http_zstd_static_module.c
/usr/local/src/nginx/modules/zstd-nginx-module/filter/ngx_http_zstd_filter_module.c: In function ‘ngx_http_zstd_filter_create_cstream’:
/usr/local/src/nginx/modules/zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:606:9: error: ‘ZSTD_initCStream_usingCDict’ is deprecated [-Werror=deprecated-declarations]
606 | rc = ZSTD_initCStream_usingCDict(cstream, zlcf->dict);
| ^~
In file included from /usr/local/src/nginx/modules/zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:11:
/usr/local/include/zstd.h:2548:8: note: declared here
2548 | size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
cc -c -DZSTD_STATIC_LINKING_ONLY -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -Wno-ignored-qualifiers -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /usr/local/src/nginx/modules/nginx-rtmp-module -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I -DZSTD_STATIC_LINKING_ONLY -I -DZSTD_STATIC_LINKING_ONLY
-o objs/addon/nginx-rtmp-module/ngx_rtmp.o
/usr/local/src/nginx/modules/nginx-rtmp-module/ngx_rtmp.c
cc1: all warnings being treated as errors
make[1]: *** [objs/Makefile:1466: objs/addon/filter/ngx_http_zstd_filter_module.o] Ошибка 1
make[1]: *** Ожидание завершения заданий…
make[1]: выход из каталога «/usr/local/src/nginx/nginx-1.25.1»
make: *** [Makefile:10: build] Ошибка 2
make -f objs/Makefile install
make[1]: вход в каталог «/usr/local/src/nginx/nginx-1.25.1»
cc -c -DZSTD_STATIC_LINKING_ONLY -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -Wno-ignored-qualifiers -I src/core -I src/event -I src/event/modules -I src/event/quic -I src/os/unix -I /usr/local/src/nginx/modules/nginx-rtmp-module -I /usr/include/libxml2 -I objs -I src/http -I src/http/modules -I -DZSTD_STATIC_LINKING_ONLY -I -DZSTD_STATIC_LINKING_ONLY
-o objs/addon/filter/ngx_http_zstd_filter_module.o
/usr/local/src/nginx/modules/zstd-nginx-module/filter/ngx_http_zstd_filter_module.c
/usr/local/src/nginx/modules/zstd-nginx-module/filter/ngx_http_zstd_filter_module.c: In function ‘ngx_http_zstd_filter_create_cstream’:
/usr/local/src/nginx/modules/zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:606:9: error: ‘ZSTD_initCStream_usingCDict’ is deprecated [-Werror=deprecated-declarations]
606 | rc = ZSTD_initCStream_usingCDict(cstream, zlcf->dict);
| ^~
In file included from /usr/local/src/nginx/modules/zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:11:
/usr/local/include/zstd.h:2548:8: note: declared here
2548 | size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors**
make[1]: *** [objs/Makefile:1466: objs/addon/filter/ngx_http_zstd_filter_module.o] Ошибка 1

Environment variables ZSTD_INC and ZSTD_LIB in compiling.

Hi,

I met some issues in nginx compilation with zstd-module when dealing with the ZSTD_* environment variables.

  1. zstd lib 1.5.6 was built with make lib-mt or make lib in path_to/build/zstd/ without error. Thus, zstd.h, libzstd.a and libzstd.so in path_to/build/zstd/lib.
  2. According to read me, ZSTD_INC set to path_to/build/zstd/lib/zstd.h and ZSTD_LIB set to path_to/build/zstd/lib.
  3. nginx was compiled with --add-module=path_to/zstd-nginx-module.

Then, nginx checking reported:

checking for ZStandard static library in path_to/build/zstd/lib/zstd.h and path_to/build/zstd/lib ... not found
checking for ZStandard dynamic library in path_to/build/zstd/lib/zstd.h and path_to/build/zstd/lib ... not found
            ./auto/configure: error: ngx_http_zstd_filter_module requires the ZStandard library, please be sure that "$ZSTD_INC" and "$ZSTD_LIB" are set correctly.

All paths are correct and exist.

After searching for the source code, I found the variables were used in:

ngx_feature_path=$ZSTD_INC
ngx_zstd_opt_I="-I$ZSTD_INC -DZSTD_STATIC_LINKING_ONLY"

ngx_zstd_opt_L="$ZSTD_LIB/libzstd.a"

They both should be folder path, right?

Then, I set:

export ZSTD_INC=path_to/build/zstd/lib
export ZSTD_LIB=path_to/build/zstd/lib

Nginx reported 'not found' again.

Then, I tried to set (which is obviously not correct):

export ZSTD_INC=path_to/build/zstd/lib/zstd.h
export ZSTD_LIB=path_to/build/zstd/lib/libzstd.a

Nginx reported:

checking for ZStandard static library in path_to/build/zstd/lib/zstd.h and path_to/build/zstd/lib/libzstd.a ... not found
checking for ZStandard dynamic library in path_to/build/zstd/lib/zstd.h and path_to/build/zstd/lib/libzstd.a ... found
checking for ZStandard static library in path_to/build/zstd/lib/zstd.h and path_to/build/zstd/lib/libzstd.a ... not found
checking for ZStandard dynamic library in path_to/build/zstd/lib/zstd.h and path_to/build/zstd/lib/libzstd.a ... found

So, static library was not found but dynamic library was found. How strange!

How could i properly set the environment variables? And how does nginx produce ngx_found?

Thx.

zstd cannot match when accept-encoding is gzip,zstd,br

cannot match zstd when accept-encoding is gzip,zstd,br.

static ngx_int_t
ngx_http_zstd_accept_encoding(ngx_str_t *ae)
{
    u_char  *p;

    p = ngx_strcasestrn(ae->data, "zstd", sizeof("zstd") - 1);
    if (p == NULL) {
        return NGX_DECLINED;
    }

    if (p == ae->data || (*(p - 1) == ',' || *(p - 1) == ' ')) {

        p += sizeof("zstd") - 1;

        if (p == ae->data + ae->len || *p == ',' || *p == ' ' || *p == ';') {
            return NGX_OK;
        }
    }

    return NGX_DECLINED;
}

u_char * ngx_strcasestrn(u_char *s1, char *s2, size_t n)

The argument n of ngx_strcasestrn must be length of the second substring - 1. But the length 4 is passed in the code.

Using a Dictionary is not Standard-Compliant

Hi! Thanks so much for writing this module! We're very excited to see the community build tooling to help drive adoption of Zstd as an encoding for HTTP.

Are you running this module on a server somewhere that we can point to as a reference server for clients to test against?

I want to point out an issue with the zstd_dict_file directive you've added. Certainly dictionaries are a very exciting technology (that I spend most of my time working on and around). And we're actively working to figure out how to bring dictionary-based compression to the web.

However, while zstd the reference implementation has extensive support for dictionaries, zstd the HTTP content-coding, as specified by RFC8478, does not. Now technically... reading the RFC very closely, it seems like we may not have explicitly disallowed the use of a dictionary under the zstd encoding. But because the content-coding registration only specifies a means to signal the use of the zstd format, and does not additionally specify any mechanism for advertising/negotiating/synchronizing the use of a specific dictionary between client and server (other than the dictionary id that is optionally written into the frame header), the content encoding as specified is not sufficient for two implementations, lacking any external synchronization, to be able to use dictionaries together.

Which is to say, use of dictionary-based compression with just the Content-Encoding: zstd header is dangerous and almost certainly wrong. So in order to preserve interoperability with other implementations of the zstd content-coding, I would suggest making at least one of these changes:

  • Add warning text to the description of the zstd_dict_file directive.
  • Suggest or require that use of zstd_dict_file also require changing the content-coding identifier this extension accepts/produces (e.g., "x-zstd-dict-<DICTID>").

Thanks!

requests with "Content-Type: application/json" can't get zstd compressed

When I:
curl ip:port/health_check/check -H 'Accept-Encoding: zstd' -v
I got the response:

< HTTP/1.1 200 OK
< Server: nginx
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Content-Encoding: zstd
<

But if I (-H "Content-Type: application/json" added):
curl ip:port/health_check/check -H 'Accept-Encoding: zstd' -H "Content-Type: application/json" -v
I got:

< HTTP/1.1 200 OK
< Server: nginx
< Content-Type: application/json
< Content-Length: 1183
< Connection: keep-alive
<

The response body was not compressed with zstd
Confused...

Build fails with warnings enabled

        -o objs/ngx_http_zstd_static_module_modules.o \
        objs/ngx_http_zstd_static_module_modules.c
zstd-nginx-module-0.1.0/filter/ngx_http_zstd_filter_module.c: In function 'ngx_http_zstd_filter_create_cstream':
zstd-nginx-module-0.1.0/filter/ngx_http_zstd_filter_module.c:606:9: error: 'ZSTD_initCStream_usingCDict' is deprecated (declared at /usr/include/zstd.h:2530) [-Werror=deprecated-declarations]
         rc = ZSTD_initCStream_usingCDict(cstream, zlcf->dict);
         ^
cc1: all warnings being treated as errors
         ^

Fails to build the module on RHEL 7, 8, 9 if "warnings as errors" are enabled (the default). While it can be bypassed, RPM builds require more strict compilation settings.

It's no working for nginx-1.27.0,Compilation trapped in a dead loop.

engines/e_padlock.c:199:19: note: at offset [12, 16] into destination object 'iv' of size 16 engines/e_padlock.c:199:19: note: at offset [12, 16] into destination object 'iv' of size 16 engines/e_padlock.c:199:19: note: at offset [12, 16] into destination object 'iv' of size 16 engines/e_padlock.c:199:19: note: at offset [12, 16] into destination object 'iv' of size 16 engines/e_padlock.c:199:19: note: at offset [12, 16] into destination object 'iv' of size 16 engines/e_padlock.c:393:22: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=] 393 | *ivp = *(out_arg++) = *(in_arg++) ^ *ivp; | ^ engines/e_padlock.c:199:19: note: at offset [13, 16] into destination object 'iv' of size 16 199 | unsigned char iv[AES_BLOCK_SIZE]; /* Initialization vector */ | ^ engines/e_padlock.c:199:19: note: at offset [13, 16] into destination object 'iv' of size 16 engines/e_padlock.c:199:19: note: at offset [13, 16] into destination object 'iv' of size 16 engines/e_padlock.c:199:19: note: at offset [13, 16] into destination object 'iv' of size 16 engines/e_padlock.c:199:19: note: at offset [13, 16] into destination object 'iv' of size 16 engines/e_padlock.c:199:19: note: at offset [13, 16] into destination object 'iv' of size 16 engines/e_padlock.c:393:22: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=] 393 | *ivp = *(out_arg++) = *(in_arg++) ^ *ivp; | ^ engines/e_padlock.c:199:19: note: at offset [14, 16] into destination object 'iv' of size 16 199 | unsigned char iv[AES_BLOCK_SIZE]; /* Initialization vector */ |

endless loop

@tokers
你好, 有个问题请教下, 使用这个模块在做压测时偶尔会重现死循环, 请帮忙看看什么原因, 具体worker进程堆栈如下:

gdb /export/servers/nginx-1.10.3/sbin/nginx 180995
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /home/export/servers/nginx-1.10.3/sbin/nginx...done.
Attaching to program: /home/export/servers/nginx-1.10.3/sbin/nginx, process 180995
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libpcre.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libpcre.so.0
Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libssl.so.10
Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libcrypto.so.10
Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libfreebl3.so
Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgssapi_krb5.so.2
Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done.
Loaded symbols for /lib64/libk5crypto.so.3
Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libselinux.so.1
Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libnss_files.so.2

warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7fff9f9ec000
0x00000000004e8f4c in ZSTD_compressStream_generic ()
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.el6_6.9.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-37.el6_6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 nss-softokn-freebl-3.14.3-22.el6_6.x86_64 openssl-1.0.1e-30.el6.11.x86_64 pcre-7.8-6.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0 0x00000000004e8f4c in ZSTD_compressStream_generic ()
#1 0x00000000004eb560 in ZSTD_compressStream ()
#2 0x00000000004df0a1 in ngx_http_zstd_filter_compress (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:541
#3 ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:426
#4 0x000000000041892f in ngx_output_chain (ctx=0x21a6240, in=0x7fff9f9d74d0) at src/core/ngx_output_chain.c:74
#5 0x000000000046bcaf in ngx_http_copy_filter (r=0x2237dd0, in=0x7fff9f9d74d0) at src/http/ngx_http_copy_filter_module.c:152
#6 0x000000000046197d in ngx_http_range_body_filter (r=0x2237dd0, in=) at src/http/modules/ngx_http_range_filter_module.c:623
#7 0x000000000043f941 in ngx_http_output_filter (r=, in=) at src/http/ngx_http_core_module.c:1970
#8 0x00000000004470e0 in ngx_http_send_special (r=0x2237dd0, flags=2) at src/http/ngx_http_request.c:3336
#9 0x0000000000456ab3 in ngx_http_upstream_finalize_request (r=0x2237dd0, u=0x21a53c0, rc=-1) at src/http/ngx_http_upstream.c:4229
#10 0x0000000000457462 in ngx_http_upstream_process_request (r=0x2237dd0, u=0x21a53c0) at src/http/ngx_http_upstream.c:3830
#11 0x0000000000458c01 in ngx_http_upstream_process_upstream (r=0x2237dd0, u=0x21a53c0) at src/http/ngx_http_upstream.c:3733
#12 0x0000000000457165 in ngx_http_upstream_handler (ev=) at src/http/ngx_http_upstream.c:1117
#13 0x000000000042eca2 in ngx_event_process_posted (cycle=, posted=0xc042f0) at src/event/ngx_event_posted.c:33
#14 0x000000000042eb1f in ngx_process_events_and_timers (cycle=0x21a03d0) at src/event/ngx_event.c:259
#15 0x0000000000434a9c in ngx_worker_process_cycle (cycle=0x21a03d0, data=) at src/os/unix/ngx_process_cycle.c:753
#16 0x0000000000433387 in ngx_spawn_process (cycle=0x21a03d0, proc=0x434a11 <ngx_worker_process_cycle>, data=0x1, name=0x543d05 "worker process", respawn=1) at src/os/unix/ngx_process.c:198
#17 0x00000000004352a6 in ngx_reap_children (cycle=0x21a03d0) at src/os/unix/ngx_process_cycle.c:621
#18 ngx_master_process_cycle (cycle=0x21a03d0) at src/os/unix/ngx_process_cycle.c:174
#19 0x00000000004159d1 in main (argc=, argv=) at src/core/nginx.c:367
(gdb) b ngx_http_zstd_filter_module.c:428
Breakpoint 1 at 0x4df28f: file ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c, line 428.
(gdb) p ctx->last
No symbol "ctx" in current context.
(gdb) s
Single stepping until exit from function ZSTD_compressStream_generic,
which has no line number information.
0x00000000004eb560 in ZSTD_compressStream ()
(gdb) s
Single stepping until exit from function ZSTD_compressStream,
which has no line number information.
ngx_http_zstd_filter_compress (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:546
546 if (ZSTD_isError(rc)) {
(gdb) p ctx->last
$1 = 0
(gdb) p ctx->in->next
Cannot access memory at address 0x8
(gdb) p ctx->in
$2 = (ngx_chain_t *) 0x0
(gdb) p ctx->in_buf->last
$3 = (u_char *) 0x0
(gdb) p ctx->in_buf
$4 = (ngx_buf_t *) 0x21a61f0
(gdb) p ctx->in
$5 = (ngx_chain_t *) 0x0
(gdb) p ctx->out
$6 = (ngx_chain_t *) 0x0
(gdb) p ctx->out_buf
$7 = (ngx_buf_t *) 0x21a62b8
(gdb) p ctx->buffer_in
$8 = {src = 0x0, size = 0, pos = 0}
(gdb) p ctx->buffer_out
$9 = {dst = 0x2234510, size = 4096, pos = 0}
(gdb) s
560 ctx->in_buf->pos += ctx->buffer_in.pos - pos_in;
(gdb)
561 ctx->out_buf->last += ctx->buffer_out.pos - pos_out;
(gdb)
562 ctx->redo = 0;
(gdb)
564 if (rc > 0) {
(gdb)
565 if (ctx->action == NGX_HTTP_ZSTD_FILTER_COMPRESS) {
(gdb)
566 ctx->action = NGX_HTTP_ZSTD_FILTER_FLUSH;
(gdb)
569 ctx->redo = 1;
(gdb)
583 if (ngx_buf_size(ctx->out_buf) == 0) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:406
406 rc = ngx_http_zstd_filter_add_data(r, ctx);
(gdb)
ngx_http_zstd_filter_add_data (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:627
627 if (ctx->buffer_in.pos < ctx->buffer_in.size
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:416
416 rc = ngx_http_zstd_filter_get_buf(r, ctx);
(gdb)
ngx_http_zstd_filter_get_buf (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:677
677 if (ctx->buffer_out.pos < ctx->buffer_out.size) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:426
426 rc = ngx_http_zstd_filter_compress(r, ctx);
(gdb)
ngx_http_zstd_filter_compress (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:504
504 pos_in = ctx->buffer_in.pos;
(gdb)
505 pos_out = ctx->buffer_out.pos;
(gdb)
507 switch (ctx->action) {
(gdb)
519 rc = ZSTD_flushStream(ctx->cstream, &ctx->buffer_out);
(gdb) p ctx->buffer_out
$10 = {dst = 0x2234510, size = 4096, pos = 0}
(gdb) p ctx->buffer_in
$11 = {src = 0x0, size = 0, pos = 0}
(gdb) p ctx->out_buf
$12 = (ngx_buf_t *) 0x21a62b8
(gdb) p ctx->in_buf
$13 = (ngx_buf_t *) 0x21a61f0
(gdb) p ctx->in
$14 = (ngx_chain_t *) 0x0
(gdb) p ctx->out
$15 = (ngx_chain_t ) 0x0
(gdb) s
546 if (ZSTD_isError(rc)) {
(gdb)
560 ctx->in_buf->pos += ctx->buffer_in.pos - pos_in;
(gdb)
561 ctx->out_buf->last += ctx->buffer_out.pos - pos_out;
(gdb)
562 ctx->redo = 0;
(gdb)
564 if (rc > 0) {
(gdb)
571 } else if (ctx->last && ctx->action != NGX_HTTP_ZSTD_FILTER_END) {
(gdb)
580 ctx->action = NGX_HTTP_ZSTD_FILTER_COMPRESS; /
restore /
(gdb)
583 if (ngx_buf_size(ctx->out_buf) == 0) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:406
406 rc = ngx_http_zstd_filter_add_data(r, ctx);
(gdb)
ngx_http_zstd_filter_add_data (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:627
627 if (ctx->buffer_in.pos < ctx->buffer_in.size
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:416
416 rc = ngx_http_zstd_filter_get_buf(r, ctx);
(gdb)
ngx_http_zstd_filter_get_buf (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:677
677 if (ctx->buffer_out.pos < ctx->buffer_out.size) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:426
426 rc = ngx_http_zstd_filter_compress(r, ctx);
(gdb)
ngx_http_zstd_filter_compress (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:504
504 pos_in = ctx->buffer_in.pos;
(gdb)
505 pos_out = ctx->buffer_out.pos;
(gdb)
507 switch (ctx->action) {
(gdb)
541 rc = ZSTD_compressStream(ctx->cstream, &ctx->buffer_out,
(gdb)
546 if (ZSTD_isError(rc)) {
(gdb)
560 ctx->in_buf->pos += ctx->buffer_in.pos - pos_in;
(gdb)
561 ctx->out_buf->last += ctx->buffer_out.pos - pos_out;
(gdb)
562 ctx->redo = 0;
(gdb)
564 if (rc > 0) {
(gdb)
565 if (ctx->action == NGX_HTTP_ZSTD_FILTER_COMPRESS) {
(gdb)
566 ctx->action = NGX_HTTP_ZSTD_FILTER_FLUSH;
(gdb)
569 ctx->redo = 1;
(gdb)
583 if (ngx_buf_size(ctx->out_buf) == 0) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:406
406 rc = ngx_http_zstd_filter_add_data(r, ctx);
(gdb)
ngx_http_zstd_filter_add_data (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:627
627 if (ctx->buffer_in.pos < ctx->buffer_in.size
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:416
416 rc = ngx_http_zstd_filter_get_buf(r, ctx);
(gdb)
ngx_http_zstd_filter_get_buf (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:677
677 if (ctx->buffer_out.pos < ctx->buffer_out.size) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:426
426 rc = ngx_http_zstd_filter_compress(r, ctx);
(gdb)
ngx_http_zstd_filter_compress (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:504
504 pos_in = ctx->buffer_in.pos;
(gdb)
505 pos_out = ctx->buffer_out.pos;
(gdb)
507 switch (ctx->action) {
(gdb)
519 rc = ZSTD_flushStream(ctx->cstream, &ctx->buffer_out);
(gdb)
546 if (ZSTD_isError(rc)) {
(gdb)
560 ctx->in_buf->pos += ctx->buffer_in.pos - pos_in;
(gdb)
561 ctx->out_buf->last += ctx->buffer_out.pos - pos_out;
(gdb)
562 ctx->redo = 0;
(gdb)
564 if (rc > 0) {
(gdb)
571 } else if (ctx->last && ctx->action != NGX_HTTP_ZSTD_FILTER_END) {
(gdb)
580 ctx->action = NGX_HTTP_ZSTD_FILTER_COMPRESS; /
restore */
(gdb)
583 if (ngx_buf_size(ctx->out_buf) == 0) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:406
406 rc = ngx_http_zstd_filter_add_data(r, ctx);
(gdb) set ctx->last=1
(gdb) s
ngx_http_zstd_filter_add_data (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:627
627 if (ctx->buffer_in.pos < ctx->buffer_in.size
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:416
416 rc = ngx_http_zstd_filter_get_buf(r, ctx);
(gdb)
ngx_http_zstd_filter_get_buf (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:677
677 if (ctx->buffer_out.pos < ctx->buffer_out.size) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:426
426 rc = ngx_http_zstd_filter_compress(r, ctx);
(gdb)
ngx_http_zstd_filter_compress (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:504
504 pos_in = ctx->buffer_in.pos;
(gdb)
505 pos_out = ctx->buffer_out.pos;
(gdb)
507 switch (ctx->action) {
(gdb)
541 rc = ZSTD_compressStream(ctx->cstream, &ctx->buffer_out,
(gdb)
546 if (ZSTD_isError(rc)) {
(gdb)
560 ctx->in_buf->pos += ctx->buffer_in.pos - pos_in;
(gdb)
561 ctx->out_buf->last += ctx->buffer_out.pos - pos_out;
(gdb)
562 ctx->redo = 0;
(gdb)
564 if (rc > 0) {
(gdb)
565 if (ctx->action == NGX_HTTP_ZSTD_FILTER_COMPRESS) {
(gdb)
566 ctx->action = NGX_HTTP_ZSTD_FILTER_FLUSH;
(gdb)
569 ctx->redo = 1;
(gdb)
583 if (ngx_buf_size(ctx->out_buf) == 0) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:406
406 rc = ngx_http_zstd_filter_add_data(r, ctx);
(gdb)
ngx_http_zstd_filter_add_data (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:627
627 if (ctx->buffer_in.pos < ctx->buffer_in.size
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:416
416 rc = ngx_http_zstd_filter_get_buf(r, ctx);
(gdb)
ngx_http_zstd_filter_get_buf (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:677
677 if (ctx->buffer_out.pos < ctx->buffer_out.size) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:426
426 rc = ngx_http_zstd_filter_compress(r, ctx);
(gdb)
ngx_http_zstd_filter_compress (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:504
504 pos_in = ctx->buffer_in.pos;
(gdb)
505 pos_out = ctx->buffer_out.pos;
(gdb)
507 switch (ctx->action) {
(gdb)
519 rc = ZSTD_flushStream(ctx->cstream, &ctx->buffer_out);
(gdb)
546 if (ZSTD_isError(rc)) {
(gdb)
560 ctx->in_buf->pos += ctx->buffer_in.pos - pos_in;
(gdb)
561 ctx->out_buf->last += ctx->buffer_out.pos - pos_out;
(gdb)
562 ctx->redo = 0;
(gdb)
564 if (rc > 0) {
(gdb)
571 } else if (ctx->last && ctx->action != NGX_HTTP_ZSTD_FILTER_END) {
(gdb)
572 ctx->redo = 1;
(gdb)
573 ctx->action = NGX_HTTP_ZSTD_FILTER_END;
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:406
406 rc = ngx_http_zstd_filter_add_data(r, ctx);
(gdb)
ngx_http_zstd_filter_add_data (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:627
627 if (ctx->buffer_in.pos < ctx->buffer_in.size
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:416
416 rc = ngx_http_zstd_filter_get_buf(r, ctx);
(gdb)
ngx_http_zstd_filter_get_buf (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:677
677 if (ctx->buffer_out.pos < ctx->buffer_out.size) {
(gdb)
ngx_http_zstd_body_filter (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:426
426 rc = ngx_http_zstd_filter_compress(r, ctx);
(gdb)
ngx_http_zstd_filter_compress (r=0x2237dd0, in=) at ../zstd-nginx-module/filter/ngx_http_zstd_filter_module.c:504
504 pos_in = ctx->buffer_in.pos;
(gdb)
505 pos_out = ctx->buffer_out.pos;
(gdb)
507 switch (ctx->action) {
(gdb)
529 rc = ZSTD_endStream(ctx->cstream, &ctx->buffer_out);
(gdb) set ctx->last=1

这里一直重复compress->flush->compress, 设置ctx->last可以终止循环,终止循环worker没有异常。

use of dynamic libzstd.so

It should be possible to use a dynamic library (libzstd.so) instead of including the static libzstd.a into the module, it's a lot easier to maintain/upgrade libzstd.so when a new version comes out instead of recompiling nginx.

I tried to create PR but didn't succeed yet.

(this had nothing to do with compiling the dynamic modules for nginx)

text/html is not included by default when specifying zstd_types

When serving content-type text/html: this module does not encode zstd by default. It must be added to zstd_types manually. This goes against the expected behaviors like the gzip and brotli modules (when specifying zstd_types. The module serves zstd for text/html when zstd_types is not specified). Expected behavior is mentioned in the documentation, "types in addition to text/html"

ab test failed

Hi, I tried to replace gzip with zstd. So I did an ab test.
This is for gzip
ab -s 120 -c 1000 -n 200000 -H "Accept-Encoding: gzip" -T "application/json" -p test1.json ip:port/health_check/check
I got my ab reports finnally.
And then I tried this:
ab -s 120 -c 1000 -n 200000 -H "Accept-Encoding: zstd" -T "application/json" -p test1.json ip:port/health_check/check
ab exited with this:
apr_socket_recv: Connection reset by peer (104)
Total of 2317 requests completed
I checked the nginx error.log, found some wokers exited like this:
2023/03/01 20:10:26 [alert] 12898#0: worker process 12917 exited on signal 9
Any advices will be helpful, thx!

BREACH concern: 'text/html' is always compressed

By having text/html "always compressed" (as it seems zstd_types is going to do no matter what I put on that command line) you introduce a persistent BREACH risk to your data.

It would be better if you have text/html as the 'default' option for zstd_types, but allow people to override it to exclude text/html MIME types.

By forcing text/html to be compressed all the time, a compressed HTML body containing some secret information could be vulnerable to BREACH. Adjusting the defaults to allow zstd_types to be overridden to exclude text/html would help to mitigate this risk.

(Brotli is also affected by this, as is GZip and other compression modules)

Fails to compile with zstd 1.5.2

nginx> sed -e "s|%%PREFIX%%|/nix/store/kcqvlq3iaqbizbdh3h1yi81lwia4kwnv-nginx-1.23.3|" \
nginx>  -e "s|%%PID_PATH%%|/var/log/nginx/nginx.pid|" \
nginx>  -e "s|%%CONF_PATH%%|/nix/store/kcqvlq3iaqbizbdh3h1yi81lwia4kwnv-nginx-1.23.3/conf/nginx.conf|" \
nginx>  -e "s|%%ERROR_LOG_PATH%%|/var/log/nginx/error.log|" \
nginx>  < man/nginx.8 > objs/nginx.8
nginx> /nix/store/jz2zs0bdi1lgzg2lwjmm9zp6cy0480in-zstd/filter/ngx_http_zstd_filter_module.c: In function 'ngx_http_zstd_filter_create_cstream':
nginx> /nix/store/jz2zs0bdi1lgzg2lwjmm9zp6cy0480in-zstd/filter/ngx_http_zstd_filter_module.c:611:9: error: 'ZSTD_initCStream_usingCDict' is deprecated [-Werror=deprecated-declarations]
nginx>   611 |         rc = ZSTD_initCStream_usingCDict(cstream, zlcf->dict);
nginx>       |         ^~
nginx> In file included from /nix/store/jz2zs0bdi1lgzg2lwjmm9zp6cy0480in-zstd/filter/ngx_http_zstd_filter_module.c:11:
nginx> /nix/store/aa63djs8jjy64zc7jm145d3kr49jm8jl-zstd-1.5.2-dev/include/zstd.h:2256:8: note: declared here
nginx>  2256 | size_t ZSTD_initCStream_usingCDict(ZSTD_CStream* zcs, const ZSTD_CDict* cdict);
nginx>       |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~
nginx> cc1: all warnings being treated as errors
nginx> make[1]: *** [objs/Makefile:1906: objs/addon/filter/ngx_http_zstd_filter_module.o] Error 1
nginx> make[1]: *** Waiting for unfinished jobs....
nginx> make[1]: Leaving directory '/build/nginx-1.23.3'
nginx> make: *** [Makefile:10: build] Error 2

https://github.com/facebook/zstd/blob/v1.5.2/lib/zstd.h#L2247-L2277

ERR_HTTP2_PROTOCOL_ERROR when zstd module enabled

Hi all!

I'm facing this error with zstd module enabled and zstd browser:

ERR_HTTP2_PROTOCOL_ERROR

Web page it's not loaded at all, not any single png, or text, the response it's just empty.

Error_1

Tried with different parameters and mime types, without effects, activated nginx debug without any significant info.

Can anybody point me in any direction to solve this?

NGINX info:
nginx version: nginx/1.25.4 (https://deb.myguard.nl/nginx-modules/)
built with OpenSSL 3.0.13+quic 30 Jan 2024
TLS SNI support enabled
configure arguments: --conf-path=/etc/nginx/nginx.conf --error-log-path=stderr --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --modules-path=/usr/lib/nginx/modules --pid-path=/run/nginx.pid --prefix=/usr/share/nginx --with-compat --with-file-aio --with-threads --with-cc-opt='-g -ffile-prefix-map=/build/nginx-1.25.4=. -fstack-protector-strong -Wformat -Werror=format-security -Ofast -flto -DNGX_HTTP_HEADERS -DNGX_ZLIB_NG=1 -w -DTCP_FASTOPEN=23 -fomit-frame-pointer -pipe -fPIC -Wdate-time -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -fPIC' --with-pcre-jit --with-http_dav_module --with-http_ssl_module --with-http_v2_module --with-http_v3_module --with-stream=dynamic --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --build=https://deb.myguard.nl/nginx-modules/ --with-http_addition_module --with-http_auth_request_module --with-http_degradation_module --with-http_flv_module --with-http_geoip_module=dynamic --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_stub_status_module --with-http_sub_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-stream_geoip_module=dynamic

NGINX module loaded:
load_module modules/ngx_http_zstd_filter_module.so;
load_module modules/ngx_http_zstd_static_module.so;

ZSTD module parameters:
zstd on;
zstd_min_length 256;
zstd_comp_level 3;
zstd_buffers 32 8k;
zstd_types application/atom+xml application/geo+json application/javascript application/x-javascript application/json application/ld+json application/manifest+json application/rdf+xml application/rss+xml application/vnd.ms-fontobject application/wasm application/x-web-app-manifest+json application/xhtml+xml application/xml application/xml+rss font/eot font/otf font/ttf image/bmp image/svg+xml image/vnd.microsoft.icon image/x-icon text/cache-manifest text/calendar text/css text/javascript text/markdown text/plain text/xml text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy;

NGINX debug:
2024/04/06 09:37:31 [alert] 55733#55733: *56 the http output chain is empty while sending to client, client: xx.xxx.xxx.xx, server: proseg.es, request: "GET / HTTP/2.0", upstream: "fastcgi://unix:/var/lib/php8.2-fpm/web13.sock:", host: "www.proseg.es"

Thank you!

How to enable zstd in ingress controller

I really want to enable the zstd module to my nginx_ingress_controller. that I downloaded and installed from kubernetes/nginx_ingress. I doubt what step I took and the results failed me.

what I did:

  1. download your dynamic script in one group directory, and put all the files together in dir /my directory/ingress nginx/zstd-nginx-module/
    config
    ngx_http_zstd_static_module.c

  2. I edited my daemonset/deployment file and add extra line to load the file.
    - --add-module=/my directory/ingress nginx/zstd-nginx-module/

  3. I log in to my ingress kubectl exec -it ingress-nginx-controller-name -n namespace -- sh -ls

  4. after login inside one of the nginx-controller , I could check the Nginx modules, and it is located inside the container

/etc/nginx/modules$ 
ngx_http_auth_digest_module.so    ngx_http_brotli_static_module.so  ngx_http_influxdb_module.so       ngx_http_opentracing_module.so

I did not find the zstd module and all the extension file is .so

  1. I edited my configmap data:

allow-snippet-annotations: "true"

i tried this , it is acceptable
zstd: "on"
zstd_comp_level: "3"
zstd_min_length: "256"

but this code is not working
zstd on;
zstd_comp_level 3;

I belive I dis big mistake and I don't understand enough, thank you for the help, I hope you got my point

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.