The basic hacl-star-raw source is in https://github.com/cryspen/hacl-packages/tree/main/ocaml.
This issue should make sure that the raw package is working as expected.

• #8
• #9
• without cmake mach fails with an unhelpful backtrace FileNotFoundError: [Errno 2] No such file or directory: 'cmake'
• running ./mach.py test on my M1 gives: FileNotFoundError: [Errno 2] No such file or directory: './blake2b'

• GCC-latest, clang-latest, gcc-minimum

• remove deprecated items
• remove unused APIs
• consistent naming of APIs (all hacl?)
• consistent API design (naming, parameter order, ...)
• documentation

• Check if everything we want is done.

• Only copy upstream files that we want https://github.com/cryspen/hacl/blob/f7851963ef02e227ffd4a617f91d8d13580d2971/tools/vcs.py#L44
• Copy ml files
• Copy karamel
• add info.txt
• #299

• https://github.com/google/benchmark
• https://github.com/polubelova/hacl-star-benchmarks

Tracking the update sub-command

• #76
• copy all needed files (msvc, c89)
• #12
• write out revision of all dependencies
• make snapshot of all dependencies in a docker image
• only copy what's really needed (using dependency analysis)

https://github.com/cryspen/hacl/blob/92293fdc43c6db96021ee725ddeb1a4c8493e1cb/CMakeLists.txt#L172

Tracking generation of snapshots (other dists)

• copy headers
• allow setting feature restrictions
• write out config.h

The build currently replicates what the upstream configure script. But that's not always correct or complete (see for example hacl-star/hacl-star#519 (comment)).

See https://github.com/cryspen/hacl/blob/main/cpu-features.md for full CPU feature matrix.

• Check for AES instructions
  • Lib_IntVector_Intrinsics_ni_aes_enc, Lib_IntVector_Intrinsics_ni_aes_enc_last, Lib_IntVector_Intrinsics_ni_aes_keygen_assist
  • x86: AES
• Check for clmul instructions
  • Lib_IntVector_Intrinsics_ni_clmul
  • x86: PCLMULQDQ
• Check for SSE2
  • Lib_IntVector_Intrinsics_vec128_xor, Lib_IntVector_Intrinsics_vec128_eq32, Lib_IntVector_Intrinsics_vec128_gt32, Lib_IntVector_Intrinsics_vec128_or, Lib_IntVector_Intrinsics_vec128_and, Lib_IntVector_Intrinsics_vec128_lognot, Lib_IntVector_Intrinsics_vec128_shift_left, Lib_IntVector_Intrinsics_vec128_shift_right, Lib_IntVector_Intrinsics_vec128_shift_left64, Lib_IntVector_Intrinsics_vec128_shift_right64, Lib_IntVector_Intrinsics_vec128_shift_left32, Lib_IntVector_Intrinsics_vec128_shift_right32, Lib_IntVector_Intrinsics_vec128_shuffle32, Lib_IntVector_Intrinsics_vec128_shuffle64, Lib_IntVector_Intrinsics_vec128_rotate_right_lanes32, Lib_IntVector_Intrinsics_vec128_rotate_right_lanes64, Lib_IntVector_Intrinsics_vec128_load32_le, Lib_IntVector_Intrinsics_vec128_load64_le, Lib_IntVector_Intrinsics_vec128_store32_le, Lib_IntVector_Intrinsics_vec128_store64_le, Lib_IntVector_Intrinsics_vec128_zero, Lib_IntVector_Intrinsics_vec128_add64, Lib_IntVector_Intrinsics_vec128_sub64, Lib_IntVector_Intrinsics_vec128_mul64, Lib_IntVector_Intrinsics_vec128_smul64, Lib_IntVector_Intrinsics_vec128_add32, Lib_IntVector_Intrinsics_vec128_sub32, Lib_IntVector_Intrinsics_vec128_load64, Lib_IntVector_Intrinsics_vec128_load64s, Lib_IntVector_Intrinsics_vec128_load32, Lib_IntVector_Intrinsics_vec128_load32s, Lib_IntVector_Intrinsics_vec128_interleave_low32, Lib_IntVector_Intrinsics_vec128_interleave_high32, Lib_IntVector_Intrinsics_vec128_interleave_low64, Lib_IntVector_Intrinsics_vec128_interleave_high64
• Check for SSE3
  • Lib_IntVector_Intrinsics_vec128_rotate_left32_8, Lib_IntVector_Intrinsics_vec128_rotate_left32_16, Lib_IntVector_Intrinsics_vec128_rotate_left32_24, Lib_IntVector_Intrinsics_vec128_rotate_left32, Lib_IntVector_Intrinsics_vec128_rotate_right32, Lib_IntVector_Intrinsics_vec128_load_be, Lib_IntVector_Intrinsics_vec128_load32_be, Lib_IntVector_Intrinsics_vec128_load64_be, Lib_IntVector_Intrinsics_vec128_store_be, Lib_IntVector_Intrinsics_vec128_store32_be, Lib_IntVector_Intrinsics_vec128_store64_be (all also SSE2)
• Check for SSE4.1
  • Lib_IntVector_Intrinsics_vec128_eq64, Lib_IntVector_Intrinsics_vec128_insert8, Lib_IntVector_Intrinsics_vec128_insert32, Lib_IntVector_Intrinsics_vec128_insert64, Lib_IntVector_Intrinsics_vec128_extract8, Lib_IntVector_Intrinsics_vec128_extract32, Lib_IntVector_Intrinsics_vec128_extract64, Lib_IntVector_Intrinsics_vec128_mul32, Lib_IntVector_Intrinsics_vec128_smul32 (also sse2)
• Check for SSE4.2
  • Lib_IntVector_Intrinsics_vec128_gt64,

Tracking build feature parity with the upstream hacl* dist directory.

Tracking implementations of benchmarks.

• #36
• #37
• Document benchmarking (book)
• Add benchmarks on CI (with downloadable performance artefacts)
• Add mach entry points
• Add comparison utilities to mach?
  • OpenSSL benchmarks
  • BoringSSL benchmarks
  • Modern Crypto benchmark comparison
• #197
• #198
• regression tests (https://github.com/google/benchmark/blob/main/docs/tools.md) #304
  • #330

Ensure that Rust exposes all APIs we have (advertise) in the C API.

• #54
• #55

• GCC-latest, clang-latest, gcc-minimum

• #28
• #69
• #70
• #77
• #90

Tier 1 Targets

Tier 1 targets are guaranteed to work. These targets have automated testing to ensure that changes do not break them.

• #17
  • GCC-latest, clang-latest, gcc-minimum
• #18
  • GCC-latest, clang-latest, gcc-minimum
• #19
• #20
• #21

Tier 2 Targets

Tier 2 targets are guaranteed to build. These targets have automated builds to ensure that changes do not break the builds. However, not all of them are always tested.

• #22
• #23
• #24
• #25

Tracking progress on mach for building

• allow disabling features
• selectively run tests
• connect to F*/kremlin config
• support GCC and clang

Document system and process

• #32
• #33
• #34
• #35

Tracking the cmake build

• Allow disabling hardware features
• setting compiler flags
• bundle files into units per hardware feature
• build and run tests
• build and run benchmarks
• include vale

To catch issues like: hacl-star/hacl-star#550

We should make sure this applies to the Windows and C89 flavors of our code;

https://github.com/cryspen/hacl/blob/f5fe166470347c14ca089cba3acc88269722452c/tools/vcs.py#L45

https://github.blog/2022-02-10-using-reusable-workflows-github-actions/

HACL

• AEAD
  • Chacha20-Poly1305
  • AES-GCM (only available through EverCrypt)
• Hash
  • Blake2
    • BLAKE2b
      • One-shot (32, 256)
        • Real-world comparison (OpenSSL, ...)
      • Streaming (32, 256) (#323)
        • Real-world comparison (OpenSSL, ...) (#329)
    • BLAKE2s
      • One-shot (32, 128)
        • Real-world comparison (OpenSSL, ...) (#329)
      • Streaming (32, 256) (#323)
        • Real-world comparison (OpenSSL, ...) (#329)
  • SHA-3
    • One-shot (224, 256, 384, 512, SHAKE128, SHAKE256) (#324)
      • Real-world comparison (OpenSSL, ...) (#329)
    • Streaming (224, 256, 384, 512, SHAKE128, SHAKE256) (#324)
      • Real-world comparison (OpenSSL, ...) (#329)
  • SHA-2 (TODO: all variants?)
    • One-shot
      • Real-world comparison (OpenSSL, ...)
    • Streaming
      • Real-world comparison (OpenSSL, ...)
  • SHA-1
    • One-shot (#327)
      • Real-world comparison (OpenSSL, ...)
    • Streaming (#327)
      • Real-world comparison (OpenSSL, ...)
• HPKE
• Signature
  • EdDSA
    • Ed25519 (TODO: All variants?)
      • One-shot (TODO: OpenSSL Verify)
        • Real-world comparison (OpenSSL, ...) (#334)
      • Precomputed (#334)
        • Real-world comparison (OpenSSL, ...) (#334)
  • RSAPSS (#340)
  • ECDSA
    • P-256
      • Real-world comparison (OpenSSL, ...)
    • K-256
      • Real-world comparison (OpenSSL, ...)
• Diffie-Hellman
  • Elliptic-Curve
    • Curve25519
      • Real-world comparison (OpenSSL, ...)
    • P-256
      • Real-world comparison (OpenSSL, ...)
    • K-256
      • Real-world comparison (OpenSSL, ...)
  • Finite Field
    • One-shot
    • Precomputed
• MAC
  • HMAC (#341)
• KDF
  • HKDF (#335)
• Randomness
  • DRBG (#336)
• Hazmat
  • Bignum
  • Montgomery Field Arithmetic
• NaCl
  • Public-key Authenticated Encryption
    • One-shot
      • Combined mode (#326)
      • Detached mode (#326)
    • Precomputed
      • Combined mode (#326)
      • Detached mode (#326)
  • Secret-key Authenticated Encryption
    • Combined mode (#326)
    • Detached mode (#326)

EverCrypt

• AEAD
• Hash
  • One-shot
  • Streaming
• Diffie-Hellman
  • Elliptic-Curve
    • Curve25519
• KDF
  • HKDF (#335)
• MAC
  • HMAC
• Randomness
  • DRBG

Variants

• Discuss if we also want to provide examples for variants, i.e., 32/256/512.

Building with ./mach build -a blake2 (or any other algorithm) disables evercrypt (the agile APIs).
While this is intentional, it's not great. It would be best to add a new well-designed, flexible API on top of HACL that would allow this.

https://github.com/cryspen/hacl/blob/4aa8d08b450f61c1e9b7e1ff9a89e4a8f1c439a1/tools/utils.py#L14

Improving mach's test capabilities.

• #38
• #39

- [ ] GCC 4.8
The c89 compatible code is broken. We are skipping this for now.

• Clang 7

Move https://github.com/hacl-star/hacl-star/tree/main/bindings/js to this repo and delete there.

• copy bindings into js
• update mach update to copy wasm as well
• add JS build/test to mach
• add bindings to CI node api_test.js && node test2.js && node test3.js
• publish new release
  • test packaging on CI
• publish documentation

Documentation only says SHA3 and blake2 but doesn't mention which versions are supported.

./mach update should take an argument where to find the hacl-star distribution.
This can be

• a directory
• a git repository

If the upstream is a git repository

• set revision to pull

The hacl-star source code is in https://github.com/cryspen/hacl-packages/tree/main/ocaml/hacl-star.
This issue should make sure that the opam packaging works as expected and it is easy to publish new packages.

• build opam package
• publish opam package

Handle bindings with mach

• #29
• #30
• #31