Git Product home page Git Product logo

libsxg's Introduction

Signed HTTP Exchange library

Build Status

Introduction

Signed HTTP Exchange (SXG) is file format which contains an HTTP exchange (request and response) signed by a publisher's origin. https://tools.ietf.org/html/draft-yasskin-http-origin-signed-responses-06

If the publisher creates an SXG file with a valid signature, it will be treated like a valid HTTP response regardless of the distribution channel.

This library is a minimal toolkit for handling SXG files.

Requirements

  • tested on OpenSSL 1.1.1c.

Build & Install

Simple cmake project.

$ mkdir build
$ cd build
$ cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=path/to/usr -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_INSTALL_BINDIR=bin
$ make sxg
$ sudo make install

Documentation

See docs.

Quickstart

#include <assert.h>
#include <libsxg.h>
#include <openssl/crypto.h>
#include <openssl/pem.h>
#include <stdbool.h>
#include <stdio.h>
#include <time.h>

int main() {
  // Load keys.
  char passwd[] = "";
  FILE* keyfile = fopen("ecdsa.privkey", "r");
  assert(keyfile != NULL);
  EVP_PKEY* priv_key = PEM_read_PrivateKey(keyfile, NULL, NULL, NULL);
  fclose(keyfile);

  FILE* certfile = fopen("ecdsa.cert", "r");
  assert(certfile != NULL);
  X509* cert = PEM_read_X509(certfile, 0, 0, passwd);
  fclose(certfile);

  // Initialize signers.
  time_t now = time(NULL);
  sxg_signer_list_t signers = sxg_empty_signer_list();
  if (!sxg_add_ecdsa_signer(
          "my_signer", now, now + 60 * 60 * 24,
          "https://original.example.com/resource.validity.msg",
           priv_key, cert, "https://yourcdn.example.test/cert.cbor",
           &signers)) {
    printf("Failed to append signer.\n");
    return 1;
  }

  // Prepare contents.
  sxg_raw_response_t content = sxg_empty_raw_response();
  if (!sxg_header_append_string("content-type", "text/html; charset=utf-8",
                                &content.header)) {
    printf("Failed to append content-type header.\n");
    return 1;
  }
  if (!sxg_write_string("<!DOCTYPE html><html><body>Hello Sxg!</body></html>\n",
                        &content.payload)) {
    printf("Failed to set payload.\n");
    return 1;
  }

  // Encode contents.
  sxg_encoded_response_t encoded = sxg_empty_encoded_response();
  if (!sxg_encode_response(4096, &content, &encoded)) {
    printf("Failed to encode content.\n");
    return 1;
  }

  // Generate SXG.
  sxg_buffer_t result = sxg_empty_buffer();
  if (!sxg_generate("https://original.example.com/index.html", &signers,
                    &encoded, &result)) {
    printf("Failed to generate SXG.\n");
    return 1;
  }

  // Save SXG as a file.
  FILE* fp = fopen("hello.sxg", "w");
  assert(fp != NULL);
  size_t wrote = fwrite(result.data, result.size, 1, fp);
  assert(wrote == 1);
  fclose(fp);

  // Release resouces.
  EVP_PKEY_free(priv_key);
  X509_free(cert);
  sxg_signer_list_release(&signers);
  sxg_raw_response_release(&content);
  sxg_encoded_response_release(&encoded);
  sxg_buffer_release(&result);
  return 0;
}

You can compile via:

$ gcc sxgsample.c -lsxg -lcrypto

libsxg's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libsxg's Issues

CMAKE error when following instructions in README.txt

even when specifying CMAKE_INSTALL_PREFIX, i get this error:

CMake Error at CMakeLists.txt:151 (install):
  install TARGETS given no LIBRARY DESTINATION for shared library target
  "sxg".

including a CMAKE_INSTALL_LIBDIR fixes, so it looks like the documentation change made in 1559f01 is innacturate.

Debian compilation instructions

It might help to include the packages necessary to compile libsxg. It needs at least libssl-dev and clang but I am missing the sanitizers:

fuzzer disabled because the compiler /usr/bin/c++ does not support [-fsanitize=address,fuzzer] option

Any suggestions?

use alternative methods for base64 encoding/decoding in order to support BoringSSL

sxg_codec.c uses BIO_f_base64 for base 64 encoding, but BoringSSL lists this function as deprecated:

// Deprecated functions.
  |  
  | // BIO_f_base64 returns a filter \|BIO\| that base64-encodes data written into
  | // it, and decodes data read from it. \|BIO_gets\| is not supported. Call
  | // \|BIO_flush\| when done writing, to signal that no more data are to be
  | // encoded. The flag \|BIO_FLAGS_BASE64_NO_NL\| may be set to encode all the data
  | // on one line.
  | //
  | // Use \|EVP_EncodeBlock\| and \|EVP_DecodeBase64\| instead.
  | OPENSSL_EXPORT const BIO_METHOD *BIO_f_base64(void);

Any chance we could replace the calls in libsxg with EVP_EncodeBlock/EVP_DecodeBase64 as the comments in BoringSSL suggest?

cc @twifkak @rgs1

length of signature is not consistent

we are seeing inconsistencies in the length of the signature, which is causing intermittent failures in CI. I was also able to reproduce in the libsxg build environment by building and running sxg_sig_test repeatedly with this script:

attempts=0
fails=0
for i in `seq 1 500`; do
  attempts=$((attempts+1));
  make sxg_sig_test && ./sxg_sig_test
  fails=$((fails+$?))
done
echo $attempts && echo $fails

In both environments, the signature is occasionally 4 bytes shorter than expected.

This is the output in our CI:

09:22:29 pinterest/filters/http/sxg/filter_test.cc:187: Failure
09:22:29 Expected equality of these values:
09:22:29   sxg.toString()
09:22:29     Which is: "sxg1-b3\0\0\x1Ehttps://example.org/hello.html\0\x1\x84\0\0\x84https://example.org/hello.html;cert-sha256=*unJ3rwJT2DwWlJAw1lfVLvPjeYoJh0+QUQ97zJQPZtc=*;cert-url=\"https://example.org/.well-known/amphtml/cert.cbor\";date=1610416640;expires=1611021440;integrity=\"digest/mi-sha256-03\";sig=*%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%*;validity-url=\"https://example.org/.well-known/amphtml/validity.msg\"\xA4" "FdigestX9mi-sha256-03=0x0E2wkWVYOJ7Gq8+Kfaiyjo3gYCyaijhGGgkzjPoTo=G:statusC200Lcontent-typeItext/htmlPcontent-encodingLmi-sha256-03\0\0\0\0\0\0\x10\0<html><body>hi!</body></html>\n"
09:22:29   result
09:22:29     Which is: "sxg1-b3\0\0\x1Ehttps://example.org/hello.html\0\x1\x80\0\0\x84https://example.org/hello.html;cert-sha256=*unJ3rwJT2DwWlJAw1lfVLvPjeYoJh0+QUQ97zJQPZtc=*;cert-url=\"https://example.org/.well-known/amphtml/cert.cbor\";date=1610416640;expires=1611021440;integrity=\"digest/mi-sha256-03\";sig=*%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%*;validity-url=\"https://example.org/.well-known/amphtml/validity.msg\"\xA4" "FdigestX9mi-sha256-03=0x0E2wkWVYOJ7Gq8+Kfaiyjo3gYCyaijhGGgkzjPoTo=G:statusC200Lcontent-typeItext/htmlPcontent-encodingLmi-sha256-03\0\0\0\0\0\0\x10\0<html><body>hi!</body></html>\n"

here is a sample of the output from building and running sxg_sig_test repeatedly:

...
[ RUN      ] SxgSig.ConstructAndRelease
[       OK ] SxgSig.ConstructAndRelease (0 ms)
[ RUN      ] SxgSig.MakeSignature
/home/papazian/code/libsxg/tests/sxg_sig_test.cc:69: Failure
Expected equality of these values:
  expected
    Which is: "testname;cert-sha256=*WrpTHrnR9I9Cj+cizXTozEZB+BnjQKkRe8kKgme4iLU=*;cert-url=\"https://cert.test/cert.cbor\";date=0;expires=0;integrity=\"digest/mi-sha256-03\";sig=*%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%*;validity-url=\"https://cert.test/validity.msg\""
  sxg_test::BufferToString(output)
    Which is: "testname;cert-sha256=*WrpTHrnR9I9Cj+cizXTozEZB+BnjQKkRe8kKgme4iLU=*;cert-url=\"https://cert.test/cert.cbor\";date=0;expires=0;integrity=\"digest/mi-sha256-03\";sig=*%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%*;validity-url=\"https://cert.test/validity.msg\""
[  FAILED  ] SxgSig.MakeSignature (1 ms)
[----------] 2 tests from SxgSig (1 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (1 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] SxgSig.MakeSignature

 1 FAILED TEST
[ 31%] Built target gtest
[ 81%] Built target sxg
[ 90%] Built target test_util
...
[ RUN      ] SxgSig.MakeSignature
[       OK ] SxgSig.MakeSignature (1 ms)
[----------] 2 tests from SxgSig (1 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (1 ms total)
[  PASSED  ] 2 tests.
[ 31%] Built target gtest
[ 81%] Built target sxg
[ 90%] Built target test_util
[100%] Built target sxg_sig_test
Running main() from /home/papazian/code/libsxg/build/third_party/gtest/googletest/src/gtest_main.cc
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from SxgSig
[ RUN      ] SxgSig.ConstructAndRelease
[       OK ] SxgSig.ConstructAndRelease (0 ms)
[ RUN      ] SxgSig.MakeSignature
[       OK ] SxgSig.MakeSignature (1 ms)
[----------] 2 tests from SxgSig (1 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 1 test suite ran. (1 ms total)
[  PASSED  ] 2 tests.
500
1

Is this expected? if so, can we update the libsxg tests to account for this inconsistency? that would provide a template for us to handle in our own tests.

fatal error: openssl/evp.h: No such file or directory

When I try to install libsxg i am getting the below error

In file included from /setups/libsxg-0.2/tests/test_util.cc:17:0:
/setups/libsxg-0.2/tests/test_util.h:17:25: fatal error: openssl/evp.h: No such file or directory
#include <openssl/evp.h>
^
compilation terminated.

my configuration command is

LDFLAGS="-L$AMPPS_PATH/lib -L/usr/local/apps/openssl-11/lib -lssl -lcrypto" CPPFLAGS="-I$AMPPS_PATH/include -I/usr/local/apps/openssl-11/include" cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local/apps/libsxg -DCMAKE_INSTALL_LIBDIR=/usr/local/apps/libsxg/lib -DCMAKE_INSTALL_BINDIR=/usr/local/apps/libsxg/bin -OPENSSL_INCLUDE_DIR=/usr/local/apps/openssl-11/include -DOPENSSL_ROOT_DIR=/usr/local/apps/openssl-11

No idea how to set the path for OpenSSL, I have OpenSSL installed in custom path /usr/local/apps/openssl-11

eliminate sxg_buffer_t from internal space

On making this library useful from Ruby or Python or C++, sxg_buffer_t is too closely bounded with internal implementation, we should divide it into pure pointer and length manipulation.

Provide build option to skip/disable sxg_cert_chain

We want to build a version of libsxg.a against BoringSSL so that we can integrate into our Envoy environment. However, BoringSSL does not provide openssl/ct.h, which breaks sxg_cert_chain. It looks like sxg_cert_chain is only needed for the gencertchain tool (not gensxg), which means we can build a version of libsxg that can generate SXGs with BoringSSL rather than full OpenSSL.

add SXG parser

SXG is a binary format and a bit complex for normal user.
As a SXG library, this project should support parsing existing SXG files and help extracting meaningful values from them.

Compilation with GCC

Presently libsxg compiles with clang but not gcc:

$ sudo make install
...
[ 50%] Building CXX object CMakeFiles/signer_fuzzer.dir/fuzzers/signer_fuzzer.cc.o
c++: error: unrecognized argument to -fsanitize= option: ‘fuzzer’
make[2]: *** [CMakeFiles/signer_fuzzer.dir/build.make:63: CMakeFiles/signer_fuzzer.dir/fuzzers/signer_fuzzer.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:337: CMakeFiles/signer_fuzzer.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

It appears that gcc does not support the fuzzer sanitizer. We should either document the clang dependency or change the Makefiles to not use the fuzzer.

Mixed sign comparison

As of #69 (in particular this change I requested), we get this warning when compiling against OpenSSL (not BoringSSL):

/usr/local/google/home/twifkak/devel/libsxg/src/sxg_codec.c: In function ‘sxg_base64encode_bytes’:
/usr/local/google/home/twifkak/devel/libsxg/src/sxg_codec.c:52:18: warning: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]
   52 |   if (out_length < length) return false;
      |                  ^

Fix this.

Add MacOS to CI

We should ensure the lib continues to work in MacOS (e.g. no regressions on the fix in #99).

The existing CI uses docker, which doesn't support MacOS as a guest OS (to my knowledge). However, GitHub Actions does, so we should add a workflow that builds libsxg directly, like #3 would've.

/cc @8W9aG

Error: Implicit declaration of function ‘EVP_DigestSign’

Hello all contributors.

I'm trying to install libsxg on Centos 7 but I'm having a bit of a problem building from source. I have the following versions installed:

  • cmake version 3.21.1
  • openssl version 1.1.1c

I refer to the install commands in this article: https://web.dev/how-to-set-up-signed-http-exchanges/ but it doesn't seem to work.

[root@centos build]# make
[ 6%] Building C object CMakeFiles/sxg.dir/src/sxg_buffer.c.o
[ 12%] Building C object CMakeFiles/sxg.dir/src/sxg_buffer_debug.c.o
[ 18%] Building C object CMakeFiles/sxg.dir/src/sxg_cbor.c.o
[ 25%] Building C object CMakeFiles/sxg.dir/src/sxg_codec.c.o
/root/libsxg/src/sxg_codec.c: In function ‘sxg_evp_sign’:
/root/libsxg/src/sxg_codec.c:247:7: warning: implicit declaration of function ‘EVP_DigestSign’ [-Wimplicit-function-declaration] (EVP_DigestSign(ctx, NULL, &sig_size, src->data, src->size) == 1) &&
[ 31%] Building C object CMakeFiles/sxg.dir/src/sxg_encoded_response.c.o
[ 37%] Building C object CMakeFiles/sxg.dir/src/sxg_generate.c.o
[ 43%] Building C object CMakeFiles/sxg.dir/src/sxg_header.c.o
[ 50%] Building C object CMakeFiles/sxg.dir/src/sxg_raw_response.c.o
[ 56%] Building C object CMakeFiles/sxg.dir/src/sxg_sig.c.o
/root/libsxg/src/sxg_sig.c: In function ‘sxg_sig_set_ed25519key’:
/root/libsxg/src/sxg_sig.c:71:3: warning: implicit declaration of function ‘EVP_PKEY_get_raw_public_key’ [-Wimplicit-function-declaration] return (EVP_PKEY_get_raw_public_key(public_key, NULL, &key_len) == 1) &&
[ 62%] Building C object CMakeFiles/sxg.dir/src/sxg_signer_list.c.o
[ 68%] Building C object CMakeFiles/sxg.dir/src/sxg_cert_chain.c.o
[ 75%] Linking C shared library libsxg.so
[ 75%] Built target sxg
[ 81%] Building C object CMakeFiles/gensxg.dir/src/gensxg.c.o
[ 87%] Linking C executable gensxg
libsxg.so.0.2: undefined reference to EVP_DigestSign' libsxg.so.0.2: undefined reference to EVP_PKEY_get_raw_public_key'
collect2: error: ld returned 1 exit status
make[2]: *** [gensxg] Error 1
make[1]: *** [CMakeFiles/gensxg.dir/all] Error 2
make: *** [all] Error 2

Looking forward to hearing from you. Thanks for looking at this issue.

Reenable nfail_malloc_test

Temporarily disabled in #79 because it fails to compile under boringssl. Hopefully it's an easy fix. If not, we can at least enable it only for the openssl build, by creating a new cmake option for it (default off), and overriding it in the openssl job of .travis.yml.

Build option list

It would be great if we see the list of build options, default values and option explanations.

  • CMAKE_BUILD_TYPE
  • CMAKE_INSTALL_LIBDIR
  • CMAKE_INSTALL_BINDIR
  • RUN_TEST

Any other options?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.