I follow the steps on this doc
https://web.dev/how-to-set-up-signed-http-exchanges/#step-3-option-2
When I run make after config nginx-sxg-module it returns below error

-o objs/addon/nginx-sxg-module/ngx_sxg_utils.o \ ../nginx-sxg-module/ngx_sxg_utils.c In file included from src/os/unix/ngx_process.h:12, from src/core/ngx_core.h:55, from ../nginx-sxg-module/ngx_sxg_utils.c:20: src/os/unix/ngx_setaffinity.h:16:9: **error: unknown type name 'cpu_set_t'** 16 | typedef cpu_set_t ngx_cpuset_t; | ^~~~~~~~~ make[1]: *** [objs/Makefile:1234: objs/addon/nginx-sxg-module/ngx_sxg_utils.o] Error 1

I am using alpine:3.12 and nginx 1.19.2

From discussion at https://github.com/google/nginx-sxg-module/pull/37/files#r430855638
nginx running in threaded mode, handling subresource may happen simultaneously. We need mutex to support module context structure.

As spec requires

   When served over HTTP, a response containing an "application/signed-
   exchange" payload MUST include at least the following response header
   fields, to reduce content sniffing vulnerabilities (Section 6.8):

   o  Content-Type: application/signed-exchange;v=_version_

   o  X-Content-Type-Options: nosniff

from discussion in https://github.com/google/nginx-sxg-module/pull/37/files

If a subrequest response is larger than subrequest_output_buffer_size, the main response is empty. Instead, that particular preload URL should be omitted (since it wasn't possible to compute header-integrity), and the rest of processing should continue.

This function blocks outside the event loop.

https://github.com/google/libsxg/blob/f4c01347798da7f32791a569006bde9f2ad8a482/src/sxg_cert_chain.c#L196-L211

https://github.com/google/libsxg/blob/f4c01347798da7f32791a569006bde9f2ad8a482/src/sxg_cert_chain.c#L145-L165

can we avoid blocking on the refreshing certificate?

In end-to-end-test.yml:

1. apt install valgrind into the container
2. run something like valgrind --exit-on-first-error=yes --error-exitcode=1 nginx -g 'daemon off;' & instead of nginx
3. add some mild stress testing, e.g. for ((x=0; x<100; x++)); do curl http://localhost:8080/certs/cert.cbor >/dev/null 2>&1 & done (and ditto for an SXG)

Optionally, research non-default args to valgrind, or other dynamic analysis tools.

Hello,

i'm getting an error "debug: content has ingestion error: SXG validation failure: Certificate not found" even if the cert_url is valid

how can i solve this problem ?

Just tried building this with nginx but it appears that the build failed with libsxg.h missing.

Was this missed in the initial commit?

Main SXG response is timeout when subresources are stored to the file in NGINX server with proxy_cache setting. This timeout issue only happens when there is a cache file of subresource in NGINX server and the modules sends a subrequest.

How to reproduce:

1. Prepare the nginx conf which enables proxy_cache for subresources (location /static)

proxy_cache_path /tmp/cache levels=1:2 keys_zone=nodejs:100m max_size=2g inactive=60m;

upstream node {
	server app:3000;
}

server {
  listen 80;
  listen 443 ssl;
  ssl_certificate /etc/nginx/server.crt;
  ssl_certificate_key /etc/nginx/server.key;
  root /var/www/html;
  index index.html index.htm index.nginx-debian.html;
  server_name example.com;

  sxg on;
  sxg_certificate     /etc/nginx/cert.pem;
  sxg_certificate_key /etc/nginx/priv.key;
  sxg_cert_url        https://sxg-sandbox.firebaseapp.com/cert.cbor;
  sxg_validity_url    https://example.com/validity/resource.msg;

  location = / {
    proxy_cache nodejs;
    proxy_cache_key $host$uri;
    proxy_cache_valid 200 60m;
    add_header X-Content-Type-Options nosniff;
    add_header Vary Accept;
    add_header Cache-Control "public, max-age=30";
    proxy_pass http://node;
  }

  location /static {
    proxy_cache nodejs;
    proxy_cache_key $host$uri;
    proxy_cache_valid 200 60m;
    add_header X-Content-Type-Options nosniff;
    add_header Vary Accept;
    proxy_pass http://node;
  }
}

2. Setup upstream server which returns subresoruces with cachablable header setting.

node.js server example:

const express = require('express');
const path = require('path')

const PORT = 3000;
const HOST = '0.0.0.0';

const app = express();

// maxAge option is required because express server sets "Cache-Control: public, max-age=0" as a default, which leads NGINX not to cache response.
app.use('/static', express.static('public', {maxAge: 1000 * 60}))

app.get('/', (req, res) => {
  res.append('Link', '</static/style.css>;rel="preload";as="style"')
  res.sendFile(path.join(__dirname,'./public/index.html'), {cacheControl: false})
});

app.listen(PORT, HOST);
console.log(`Running on http://${HOST}:${PORT}`);

3. Run nginx and upstream application server
4. Store subresources to NGINX proxy cache.

// on your local machine
curl 'https://example.com/static/style.css'
// on NGINX server. Check if the cache file exists 
cat /tmp/cache/e/99/abda96163e15bae457c85f803036699e  | less

5. Send a request for SXG, the response won't be returned and timeout.

curl -H 'Accept: application/signed-exchange;v=b3' 'https://example.com' --output - -v

Chromium will recursively prefetch any links from their origin unless allowed-alt-sxg and alternate are also specified. Thus, such links impede the privacy-preserving prefetch use case.

invoke_subrequests:

(I'm not sure the best way to do that. I assume if the modification happens inside subresource_fetch_handler, it'll need to be protected by a mutex.)

Hey guys, so we tried following the guide below using debian packages only (not installing from source), we got the DigiCert Certificate, installed everything properly, we already have SSL, the site is configured fine but we just wanted to test out the SXG Extension:

https://web.dev/how-to-set-up-signed-http-exchanges/

And when adding the sxg on; anywhere inside the server block, it shows:
nginx: [emerg] unknown directive "sxg" in /etc/nginx/sites-enabled/xxxx.conf
nginx: configuration file /etc/nginx/nginx.conf test failed

our main nginx conf file looks like this:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
}

http {



	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	# server_tokens off;
        client_max_body_size 100m;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;
	add_header  X-Content-Type-Options nosniff;

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;


	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;


	gzip on;


	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}

So the module is enabled, everything seems to be fine except for when adding that directive. I can also send the actual nginx conf if necessary.

The module should remove cache related headers in upstream response when it change the response by transforming to SXG. Otherwise static resources in a client cache wouldn't be updated even if SXG is out of date.

Static resources from upstream usually have cache related headers, such as ETag and Last-Modified. Those are useful to let a client know if the cache is still fresh or not. But so far the module doesn't change them when transforming the response to SXG. That causes expired SXG files (mainly static sub-resources) being stored for a long time unexpectedly.

At the moment, just removing cache related headers seems good way I think.

This is the diagram to illustrate what happens.

When nginx is misconfigured, finding startup message is good clue to debug.
There should some informational log message in the initializing sequence.

Approximately 50% of CBOR requests in our build will fail with an empty server reply, due to the worker process crashing:

Nginx log:

2021/11/01 02:08:32 [alert] 16#16: worker process 17 exited on signal 11
2021/11/01 02:08:32 [alert] 17#17: *8 ignoring stale global SSL error (SSL: error:2706F06C:OCSP routines:OCSP_response_get1_basic:no response data), client: 95.217.86.147, server: www.xxxx.com, request: "GET /xxxx.com.cert.cbor HTTP/1.0", host: "1.2.3.4"

Stack trace:

Program received signal SIGSEGV, Segmentation fault.
0x00007f7863cbc18e in OCSP_response_get1_basic () from target:/lib/x86_64-linux-gnu/libcrypto.so.1.1
(gdb) bt
#0  0x00007f7863cbc18e in OCSP_response_get1_basic () from target:/lib/x86_64-linux-gnu/libcrypto.so.1.1
#1  0x000055925675be49 in refresh_if_needed ()
#2  0x000055925675c91b in ?? ()
#3  0x000055925669b540 in ngx_http_core_access_phase ()
#4  0x0000559256696e7d in ngx_http_core_run_phases ()
#5  0x00005592566a1edf in ?? ()
#6  0x00005592566a22d4 in ?? ()
#7  0x000055925668a183 in ?? ()
#8  0x0000559256680717 in ngx_process_events_and_timers ()
#9  0x00005592566882d0 in ?? ()
#10 0x00005592566867ed in ngx_spawn_process ()
#11 0x0000559256689019 in ngx_master_process_cycle ()
#12 0x0000559256660141 in main ()

I will attempt to reproduce on a debug build with source available and report back when available.

Versions:

FROM ubuntu:21.04 AS build


RUN wget https://github.com/cloudflare/quiche/raw/master/extras/nginx/nginx-1.16.patch

RUN cd nginx-module-vts && git checkout 3c6cf41315bfcb48c35a3a0be81ddba6d0d01dac
RUN cd nginx-rtmp-module && git checkout v1.2.2
RUN cd nginx-statsd && git checkout ef52718c0e1cf6b52899c89da89b28933eb11557
RUN cd ngx_brotli && git checkout 9aec15e2aa6feea2113119ba06460af70ab3ea62
RUN cd libsxg && git checkout 672b807e9a1a0032c73f1eeb7c9761d8cfd013d0
RUN cd nginx && git checkout release-1.16.1
RUN cd nginx-sxg-module && git checkout e748c0406bf0ab297a9dea97ea844fd1f75eefb0 # f742dad1a56e54c7af629f8a06efc3316d6700fa

RUN git clone https://github.com/openresty/headers-more-nginx-module.git
RUN cd headers-more-nginx-module && git checkout a4a0686605161a6777d7d612d5aef79b9e7c13e0

Configure:

RUN PATH=$HOME/.cargo/bin:$PATH ./auto/configure \
    --with-cc-opt='-O2 -fstack-protector-strong -Wformat -Wno-error=implicit-fallthrough -Werror=format-security -Werror=maybe-uninitialized -D_FORTIFY_SOURCE=2' \
    --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now' \
    --prefix=/usr \
    --conf-path=/etc/nginx/nginx.conf \
    --http-log-path=/var/log/nginx/access.log \
    --error-log-path=/var/log/nginx/error.log \
    --lock-path=/var/lock/nginx.lock \
    --pid-path=/var/run/nginx.pid \
    --modules-path=/usr/loacl/lib/nginx/modules \
    --http-client-body-temp-path=/var/lib/nginx/body \
    --http-fastcgi-temp-path=/var/lib/nginx/fastcgi \
    --http-proxy-temp-path=/var/lib/nginx/proxy \
    --http-scgi-temp-path=/var/lib/nginx/scgi \
    --http-uwsgi-temp-path=/var/lib/nginx/uwsgi \
    --with-compat \
    --with-pcre-jit \
    --with-ipv6 \
    --with-http_ssl_module \
    --with-http_stub_status_module \
    --with-http_image_filter_module \
    --with-http_realip_module \
    --with-http_auth_request_module \
    --with-http_ssl_module \
    --with-http_v2_module \
    --with-http_dav_module \
    --with-http_slice_module \
    --with-threads \
    --with-http_addition_module \
    --with-http_gunzip_module \
    --with-http_gzip_static_module \
    --with-http_sub_module \
    --with-stream \
    --with-stream_ssl_module \
    \
    --add-module=../nginx-module-vts \
    --add-module=../nginx-rtmp-module \
    --add-module=../nginx-statsd \
    --add-module=../ngx_brotli \
    --add-module=../njs/nginx \
    --add-module=../headers-more-nginx-module \
    --add-module=../nginx-sxg-module

The module has the feature that automatically includes signatures of subresources if the upstream response has the link header with rel=preload. However, subrequest isn't happened if the header is Link because the current implementation seems to be case-sensitive.

https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html

Each header field consists of a name followed by a colon (":") and the field value. Field names are case-insensitive.

This is relatively minor bug, but it should be fixed I think.

I initially installed libsxg to a custom path using -DCMAKE_INSTALL_PREFIX=$HOME/usr. I believe using -DCMAKE_PREFIX_PATH=$HOME/usr in this project should enable this to include/link with that location, but it didn't work. I believe some combination of find_library/target_link_libraries/etc. in CMakeLists.txt is necessary. Possible help from:

• https://mirkokiefer.com/cmake-by-example-f95eb47d45b1
• https://bastian.rieck.me/blog/posts/2018/cmake_tips/
• https://gitlab.kitware.com/cmake/community/-/wikis/doc/tutorials/How-To-Find-Libraries

When upstream response header is having the preload Link: in which the url is full path including domain name, the sxg filter makes an subrequest with the wrong URL, then a worker process terminates abnormally. It seems that ngx_http_subrequest expects only path string, not including domain. However, the current implementation set the URL which is passed from the link header value from upstream response as it is. That causes the issue.

I put logs below.

Response header from upstream:

Link: <https://example.com/static/style.css>;rel="preload";as="style"'

Error log:

2020/05/28 14:57:15 [debug] 8#8: *1 http upstream request: "/?"
2020/05/28 14:57:15 [debug] 8#8: *1 http upstream process header
2020/05/28 14:57:15 [debug] 8#8: *1 malloc: 00005648477A9F20:4096
2020/05/28 14:57:15 [debug] 8#8: *1 recv: eof:0, avail:1
2020/05/28 14:57:15 [debug] 8#8: *1 recv: fd:14 871 of 4096
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy status 200 "200 OK"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "X-Powered-By: Express"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "Link: <https://example.com/static/style.css>;rel="preload";as="style""
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "Accept-Ranges: bytes"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "Cache-Control: public, max-age=0"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "Last-Modified: Tue, 26 May 2020 08:54:35 GMT"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "ETag: W/"200-1725030ccf8""
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "Content-Type: text/html; charset=UTF-8"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "Content-Length: 512"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "Date: Thu, 28 May 2020 14:57:15 GMT"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header: "Connection: close"
2020/05/28 14:57:15 [debug] 8#8: *1 http proxy header done
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: param is: [<https://example.com/static/style.css>]
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: param is: [rel="preload"]
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: param is: [as="style"]
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: found as statement in link preload header: style
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: param is: [<https://example.com/static/style.css>]
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: param is: [rel="preload"]
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: param is: [as="style"]
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: found as statement in link preload header: style
2020/05/28 14:57:15 [debug] 8#8: *1 nginx-sxg-module: non preload header:
2020/05/28 14:57:15 [debug] 8#8: *1 invoke request for: https://example.com/static/style.css as style
2020/05/28 14:57:15 [debug] 8#8: *1 posix_memalign: 00005648477AAF30:4096 @16
2020/05/28 14:57:15 [debug] 8#8: *1 http subrequest "https://example.com/static/style.css?"
...
2020/05/28 14:57:15 [debug] 8#8: *1 http filename: "/var/www/htmlhttps://example.com/static/style.css"
2020/05/28 14:57:15 [debug] 8#8: *1 add cleanup: 00005648477ABE18
2020/05/28 14:57:15 [error] 8#8: *1 open() "/var/www/htmlhttps://example.com/static/style.css" failed (2: No such file or directory), client: 172.25.0.1, server: example.com, request: "GET / HTTP/1.1", subrequest: "https://example.com/static/style.css", host: "example.com"
2020/05/28 14:57:15 [debug] 8#8: *1 http finalize request: 404, "https://example.com/static/style.css?" a:1, c:2
2020/05/28 14:57:15 [alert] 1#1: worker process 8 exited on signal 11

It ends-up returning an empty response to a client.

$ curl --insecure -H 'Accept: application/signed-exchange;v=b3' https://example.com --output -
curl: (52) Empty reply from server

Of course a full URL is a valid format in Link header, so that would be great if the module supports it. Maybe the module should extract only a path string from the passed URL before making a subrequest? If it can't be supported soon, can we consider having the error handling for this issue as a quick fix? It can be critical for users.

I spotted this line cause this error.

I wasn't fully able to produce a CBOR file with the module yet (like mentioned in a previous issue), but in the current phase of my testing I want to mention two important topics.

First, the ngx_http_cert_chain_handler :

• It should be documented/mentioned this handler does the refreshing of the CBOR file only when a request to the CBOR file is issued. Which is logical, you only need to refresh it when it is requested. But it should be clear that this is not a periodic refresh or whatever. If no one is sending requests to that specific file given in sxg_cert_url, nothing touches that file. But I understand it, and it is logical.
• There is this string check in the cert_chain_handler:
  if (slc->cert_path.len <= 0 || req->uri.len != slc->cert_path.len || ngx_memcmp(req->uri.data, slc->cert_path.data, req->uri.len) != 0) { return NGX_OK; }
  As an explanation to the untrained eyes, this function makes sure that CBOR refresh code never runs, if the CBOR file is not the one being requested (as stated above). But it does so, by comparing the absolute path given in sxg_cert_path and request URI (if these strings are the same, it is considered a request to the CBOR file).

Something like this:

The absolute path of the file:
sxg_cert_path /mnt/server_documents/certificates/customer_id_346512/sxg/document.cbor

The way this module asks for it to be delivered.
www.website.com/mnt/server_documents/certificates/customer_id_346512/sxg/document.cbor

In other words, you have to put the entire directory structure in the link, and serve it that way. Although not impossible, I find this approach a bit forcing. It was also undocumented, and had to go through the code to understand why anything wasn't refreshed(or tried to be refreshed) and my SXG's were invalid after some time. I was just serving the CBOR file in a different URI. And the string comparison returning False, prevented the code from being executed. There could be many possible ways to improve this, so it's an open debate. Even without an improvement, documenting the current behavior would certainly clarify things.

Secondly, logging of refreshing:

• As an example, here:
  bool refreshed = refresh_if_needed(&slc->cert_chain); if (refreshed) { ngx_log_error( NGX_LOG_INFO, req->connection->log, 0, "nginx-sxg-module: OCSP Response in Certificate-Chain is refreshed"); }

It is logged only when CBOR refresh is needed and successful. A failure log is just as needed, with explaining the failure reason (OCSP error, file permission error etc...). Without fail log, the CBOR could be expiring without an alert.

The reason I primarily inspected sxg_cert_path is that it is such a fundamental feature of the module. SXG simply doesn't work without it. Even though there is an alternative for it (gen-certurl), it could be more appropriate for people to do without any 3rd party tool, which brings something other than nginx to maintain/monitor.

It also takes like seven days for the CBOR to expire, making the tests very slow, and failure unapparent (You don't know if it's failing). So at least the documentation is paramount.

Thanks for your time, and your good work. Have a nice day.

I think "#else" is correct.

I'm working on implementing signed exchanges for a website and I have a question and maybe a potential feature requests. Most of the HTML I'm serving has an expiration date of just a few days, however a lot of static content (hashed) uses 365 days. Signed exchanges can be generated for both, however in front of my Nginx servers I also have a load balancer with CDN capabilities. If it starts caching requests for signatures, then these will be stale. I couldn't find a way to differentiate expires and cache-control headers based on the fact if it's an SGX request or not. Should the module automatically set cache-control and expires to a reasonable value?

To make it more clear, that's my setup:

• Nginx servers in 12 geographical regions worldwide
• Google Cloud HTTPS load balancer
• Google Cloud CDN

Requests made from AMP caches would also end up on the load balancer and possibly be served by the CDN, unless there's a way I can prevent that from happening.

Serving static resources such as html, js, image etc from local, that is a common situation of nginx use case. But now the module expects they come only from upstream if subrequest is called, and cause the worker process error (this will be fixed in #46).

We can easily reproduce it in below situation.

If the response header of a document includes:

Link: <https://example.com/static/preload.js>;rel="preload";as="script"

nginx conf:

 location /static {
     root   /var/www/app;
     try_files $uri $uri/;
 }

Is is possible to support this use case? Anyway the current behavior is a little bit confusing, so let's have a documentation at the moment.

param_is_as function in ngx_sxg_utils.c is not tested by fuzzer.
Every parser which accepts wild strings from internet should heavily tested to prevent attacks.

    /setups/nginx-sxg-module/ngx_sxg_utils.c

/setups/nginx-sxg-module/ngx_sxg_utils.c: In function ‘get_term_length’:
/setups/nginx-sxg-module/ngx_sxg_utils.c:38:3: error: ‘for’ loop initial declarations are only allowed in C99 mode
for (size_t i = 0; i < len; ++i) {
^
/setups/nginx-sxg-module/ngx_sxg_utils.c:38:3: note: use option -std=c99 or -std=gnu99 to compile your code
/setups/nginx-sxg-module/ngx_sxg_utils.c: In function ‘sxg_qvalue_is_1’:
/setups/nginx-sxg-module/ngx_sxg_utils.c:165:3: error: ‘for’ loop initial declarations are only allowed in C99 mode
for (const char* const end = str + len; str < end;) {
^
/setups/nginx-sxg-module/ngx_sxg_utils.c: In function ‘check_refresh_needed’:
/setups/nginx-sxg-module/ngx_sxg_utils.c:365:3: error: ‘for’ loop initial declarations are only allowed in C99 mode
for (int i = 0; i < resps; ++i) {
^

tried compiling this module with ngnix getting above error

If i try CFLAGS="$CFLAGS -std=c99" getting error for u_char datatype

Please help me with this how to compile this module with Ngnix

Hello,

I inspect this url https://github.com/google/nginx-sxg-module/blob/master/packaging/build_deb but I can't find libnginx-mod.nginx.skeleton in related the path so I can't create deb file.

Testing subresource feature with a single line of Link header.

This doesn't work.

link: <https://example.com/static/style.css>; rel="preload"; as="style"

This works.

link: <https://example.com/static/style.css>;rel="preload";as="style"

The task extracting a url from the header might have some issues, but not sure.
So far I still don't test with multiple link headers, or comma separated values yet.

Hi,

I have SXGs enabled for my domain Mustakbil.com.

But when I search google.com for Mustakbil, and click the very first link which points to the home page, it opens a random wrong page of my website.

Here's the markup of the link from google search results:

<a href="https://www.mustakbil.com/" data-sxg-url="https://www-mustakbil-com.webpkgcache.com/doc/-/s/www.mustakbil.com/"></div></span><div><span class="VuuXrf">Mustakbil.com</span><div class="byrV5b"><cite class="tjvcx GvPZzd cHaqb" role="text" style="max-width:315px">https://www.mustakbil.com</cite></div></div></div></a>

The href is pointing correctly to https://www.mustakbil.com/

and SXGs url is also pointing correctly to https://www-mustakbil-com.webpkgcache.com/doc/-/s/www.mustakbil.com/

But when I click this link it opens wrong url. Which is a random page of my website but not the actual homepage.

Currently, lifespan of SXG files created by this module are limited just 1 day.
In spec, SXG allows lifespan 7 days in maximum, the lifespan should be configurable in nginx config files.

Hi! I'm leaving this issue here as there's a potential to improve the behavior of the module, however most importantly I wanted to leave somewhere a solution I found, so feel free to close if you don't see it relevant to fix. Other developers should be able to find it in closed issues as well.

A common way of implementing "clean URLs" (no trailing slashes) is to use try_files directive:

try_files $uri $uri/index.html =404;

The URL looks like this: /foo/bar, but the actual file lives in /foo/bar/index.html. When try_files is used alongside this module, the file path is in the signature instead of the requested URL: /foo/bar/index.html.

As a workaround, I'm now using location with a regular expression + alias:

location ~ ^/(.*) {
  rewrite ^/index(?:\.html|/)?$ / permanent;
  rewrite ^/(.*)/index(?:\.html|/)?$ /$1 permanent;
  rewrite ^/(.*)(?:\.html|/)$ /$1 permanent;

  alias /var/www/$1/index.html;
}

Result is the same when it comes to regular requests, however signatures contain a correct URL: /foo/bar with no /index.html.

Hello .
I'm new with nginx and accept my apologies if the question is obvious.
i have SXG certificate from DigiCert. but in the SXG config i should give cert.cbor and resource.msg path.
how can i generate these files? all i have is pem file.
thank you in advance.

        sxg on;
        sxg_certificate     /path/to/sxg/mySxg.pem;
        sxg_certificate_key /path/to/sxg/mySxg.key;
        sxg_cert_url        https://website.test/certs/cert.cbor;
        sxg_validity_url    https://website.test/validity/resource.msg;
        sxg_cert_path       /certs/cert.cbor;

make-debs.yml and end-to-end-test.yml are both failing with "The job was canceled because "ubuntu_eoan_eoan" failed." Details from the logs:

Step 5/12 : RUN apt-get update &&     apt-get install -y --no-install-recommends -q                     build-essential                     ca-certificates                     cmake                     debhelper                     devscripts                     dpkg-dev                     fakeroot                     git                     libssl-dev                     lsb-release                     moreutils                     wget
 ---> Running in caa6101fb843
Ign:1 http://security.ubuntu.com/ubuntu eoan-security InRelease
Ign:2 http://archive.ubuntu.com/ubuntu eoan InRelease
Err:3 http://security.ubuntu.com/ubuntu eoan-security Release
  404  Not Found [IP: 91.189.88.152 80]
Ign:4 http://archive.ubuntu.com/ubuntu eoan-updates InRelease
Ign:5 http://archive.ubuntu.com/ubuntu eoan-backports InRelease
Err:6 http://archive.ubuntu.com/ubuntu eoan Release
  404  Not Found [IP: 91.189.88.142 80]
Err:7 http://archive.ubuntu.com/ubuntu eoan-updates Release
  404  Not Found [IP: 91.189.88.142 80]
Err:8 http://archive.ubuntu.com/ubuntu eoan-backports Release
  404  Not Found [IP: 91.189.88.142 80]
Reading package lists...
E: The repository 'http://security.ubuntu.com/ubuntu eoan-security Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu eoan Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu eoan-updates Release' does not have a Release file.
E: The repository 'http://archive.ubuntu.com/ubuntu eoan-backports Release' does not have a Release file.
The command '/bin/sh -c apt-get update &&     apt-get install -y --no-install-recommends -q                     build-essential                     ca-certificates                     cmake                     debhelper                     devscripts                     dpkg-dev                     fakeroot                     git                     libssl-dev                     lsb-release                     moreutils                     wget' returned a non-zero code: 100

In current implementation, evaluation of qvalue is done with simple comparison of number.
When multiple candidates have the same qvalue, the plugin will give priority to one appeared first in the list.
As a natural motivation of the user, app/sxg header should have priority in spite of the order in the accept-header list when it have the same qvalue.
Then, add tie-break to prioritize app/sxg for such situation.

kSignedExchangeEnabledAcceptHeaderForPrefetch is used for <link rel=(preload|prefetch)> even outside of SXG context. This can interact with https://crbug.com/1180441 resulting in broken subresources for pages that use that tag and run this worker.

To avoid this, the module should serve SXG if and only if its q-value is 1, until https://crbug.com/1243065 is fixed.

nginx's subrequest has a limit (NGX_HTTP_MAX_SUBREQUESTS=50).

https://github.com/nginx/nginx/blob/0a683fdd9313b9796bf39442fd117beaa63a7157/src/http/ngx_http_request.c#L628

https://github.com/nginx/nginx/blob/0a683fdd9313b9796bf39442fd117beaa63a7157/src/http/ngx_http_request.h#L13

What should ngx_http_sxg_filter_module behave when it exceeds the limit of nginx's subrequest?

• raise the limit?
• cause an error?

If there are two server directives, SXG doesn't work. Normal HTML file is returned even if the client has Accept: application/signed-exchange;v=b3 header.

Here is the nginx conf file I tested.

/etc/nginx/sites-enabled/default

upstream node {
  server app:3000;
}

server {
  listen 80;
  # listen [::]:80 default_server;

  listen 443 ssl;
  # listen [::]:443 ssl default_server;

  ssl_certificate /etc/nginx/server.crt;
  ssl_certificate_key /etc/nginx/server.key;

  root /var/www/html;
  index index.html index.htm index.nginx-debian.html;

  server_name example.com;

  add_header X-Content-Type-Options nosniff;
  add_header Vary Accept;

  sxg on;
  sxg_certificate /etc/nginx/cert.pem;
  sxg_certificate_key /etc/nginx/priv.key;
  sxg_cert_url https://sxg-sandbox.firebaseapp.com/cert.cbor;
  sxg_validity_url https://example.com/validity/resource.msg;

  location / {
    proxy_pass http://node;
  }
}

server {
  server_name another-server;
  listen 8000;
  root /var/www/html;
  index index.html index.htm index.nginx-debian.html;
  location / {
    try_files $uri $uri/ =404;
  }
}

nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	sendfile on;
	tcp_nopush on;
	# tcp_nodelay on;
	keepalive_requests 100;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;
	subrequest_output_buffer_size 4096k;
	charset utf-8;

	ssl_protocols TLSv1.2; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log debug;

	gzip on;

	gzip_vary on;
	gzip_proxied any;
	gzip_comp_level 6;
	gzip_buffers 16 8k;
	gzip_http_version 1.1;
	gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}

https://tools.ietf.org/html/rfc7230#section-5.3 suggests that clients can make requests of the form GET http://host.example/path HTTP/1.1, though in practice they don't for typical clients/servers.

It seems that this line:

TODO:

• Verify this is a problem (maybe it's not because unparsed_uri isn't as unparsed as its name implies?).
• Fix it.

Given the rarity of this case, it seems low-priority, and I'd recommend the easiest solution, e.g. don't generate a SXG when unparsed_uri is absolute.

Change build_deb and libsxg's build_deb to have a non-default option to build with debug symbols and safety analysis. Propagate those settings down from end-to-end-test.yml

Candidates:

• https://github.com/google/sanitizers
• https://github.com/google/sanitizers/wiki
• https://clang.llvm.org/docs/index.html

Resolve this this line and pass the tests.
OWS(Optional White Space) is a sequence (more or equal to 0 length) of tabs (\t) or spaces.

The same problem is here too.

I'm not sure this is a bug or intentional behavior though, currently the module doesn't allow duplicated sxg directives and causes a startup error. So far we can't enable SXG only on a specific upstream, that makes A/B testing difficult to see how SXG performs better for loading performance.

  location = /bar/ {
    sxg on;
    proxy_pass http://bar;
  }
  location = /foo/ {
    sxg off;
    proxy_pass http://foo;
  }

Result

nginx: [emerg] "sxg" directive is duplicate in /etc/nginx/sites-enabled/default:38

When this plugin is enabled, the nginx will return SXG to the qvalue of the request header.
If there is any possibility of requested URL may returns SXG, Vary response header should always be appended.

The signed fallback URL does not include the ?foo from the original URL.

The construct_fallback_url function:

After this fix, I'm guessing that URLs with a ? at the end of them will still be signed improperly, but that's a relatively minor bug.