julienmalka / lila Goto Github PK

View Code? Open in Web Editor NEW

16.0 5.0 1.0 101 KB

Nix hash collection software, to aggregate build reports from several builders

License: European Union Public License 1.2

Nix 25.92% Python 53.21% Rust 18.02% Shell 2.86%

nix-ci

lila's Introduction

lila

Introduction

This repository aims to give a set of tools that can be used to create a hash collection mechanism for Nix. A hash collection infrastructure is used to collect and compare build outputs from different trusted builders.

This project is composed of two parts:

A post-build-hook, that his a software running after each of Nix builds and in charge to report the hashes of the outputs
A server to aggregate the results

Howto's

Keys

Set up your keys with:

nix key generate-secret --key-name username-hash-collection > secret.key

Server side

Create a user

Hashes reports are only allowed from trusted users, which are identified via a token. To generate a token run ./create_user "username"

Run the server

Run the server with uvicorn web:app --reload

Client side

  services.hash-collection = {
    enable = true;
    collection-url = "server url";
    tokenFile = "/token/path";
    secretKeyFile = "/secret/key/path";
  };

Reporting

At the time of writing only reports on run-time closures are supported. Reporting is experimental and still expected to evolve, change, and grow support for build-time closures as well.

Defining a report

You define a report by uploading a JSON CycloneDX SBOM as produced by nix-runtime-tree-to-sbom:

$ nix-store -q --tree $(nix-build '<nixpkgs/nixos/release-combined.nix>' -A nixos.iso_gnome.x86_64-linux) > tree.txt
$ cat tree.txt | ~/dev/nix-runtime-tree-to-sbom/tree-to-cyclonedx.py > sbom.cdx.json
$ export HASH_COLLECTION_TOKEN=XYX # your token
$ curl -X PUT --data @sbom.cdx.json "http://localhost:8000/reports/gnome-iso-runtime" -H "Content-Type: application/json" -H "Authorization: Bearer $HASH_COLLECTION_TOKEN"

Populating the report

If you want to populate the report with hashes from different builders (e.g. from cache.nixos.org and from your own rebuilds), use separate tokens for the different sources.

With hashes from cache.nixos.org

$ nix shell .#utils
$ export HASH_COLLECTION_TOKEN=XYX # your token for the cache.nixos.org import
$ ./fetch-from-cache.sh

This script is still very much WIP, and will enter an infinite loop retrying failed fetches.

By rebuilding

Make sure you have the post-build hook and diff hook configured as documented above.

TODO you have to make sure all derivations are available for building on your system - is there a smart way to do that?

$ export HASH_COLLECTION_TOKEN=XYX # your token for the cache.nixos.org import
$ ./rebuilder.sh

This script is still very much WIP, and will enter an infinite loop retrying failed fetches. You can run multiple rebuilders in parallel.

Related projects

nix-reproducible-builds-report aka r13y, which generates the reports at https://reproducible.nixos.org. Ideally the reporting feature can eventually replace the reports there.
rebuilderd provides distribution-agnostic container-based rebuild infrastructure. There is some preliminary Nix support but it is geared towards 'packages' rather than 'derivations' and that data model mismatch is somewhat awkward.
trustix has somewhat similar goals, but is more ambitious: nix-hash-collection only aims for something simple in the short term, just basically CRUD collection of hashes and some simple scripts around it. trustix has a more elaborate design with multiple transparency logs that are self-hosted by the attesters, and aims to support more advanced use cases, such as showing the aggregating system is not 'lying by omission' and perhaps showing that submitters aren't providing contradicting statements.

lila's People

Contributors

Stargazers

Watchers

Forkers

raboof

lila's Issues

model: maybe rename 'reports'?

I can imagine we'd like to be able to create 'reports' on package-sets later, so maybe we could find another name for individual collected hashes? maybe 'attestation'?

Epic: replace https://reproducible.nixos.org reports

It would be nice if nix-hash-collection reports could replace https://reproducible.nixos.org . Could be just a static download of the report page. For this to be possible, we'd need at least:

An MVP of #9
#14

And then maybe:

share rebuilds recording nondeterminism

When using nix-build --check or nix build --rebuild, successful rebuilds will be passed to the post-build-hook, but unsuccessful rebuilds are not.

Perhaps we could define a diff-hook that shares the rebuild result in this case?

Ingest build results from Hydra

Would be nice to write an ingestor that will be able to parse hydra results and add them to the database, or convince the infra team to add the post-build-hook to hydra's builders.

diff-hook create biaised output reproducibility statistics

The diff-hook is only called on the first non-reproducible output. To be fixed upstream probably.
See #32

coredump executing build-hook

Feb 26 11:31:00 rigter async-nix-post-build-hook-start[1169762]: terminate called after throwing an instance of 'nix::Error'
Feb 26 11:31:00 rigter async-nix-post-build-hook-start[1169762]:   what():  error: cannot determine user's home directory
Feb 26 11:31:00 rigter systemd[1]: Started Process Core Dump (PID 1169763/UID 0).
Feb 26 11:31:00 rigter systemd-timesyncd[1201]: Network configuration changed, trying to establish connection.
Feb 26 11:31:00 rigter systemd-timesyncd[1201]: Network configuration changed, trying to establish connection.
Feb 26 11:31:00 rigter systemd-timesyncd[1201]: Network configuration changed, trying to establish connection.
Feb 26 11:31:00 rigter systemd-timesyncd[1201]: Contacted time server 5.255.99.180:123 (0.nixos.pool.ntp.org).
Feb 26 11:31:00 rigter systemd-coredump[1169764]: [🡕] Process 1169762 (build-hook) of user 64608 dumped core.
                                                  
                                                  Module libattr.so.1 without build-id.
                                                  Module libkeyutils.so.1 without build-id.
                                                  Module libkrb5support.so.0 without build-id.
                                                  Module libcom_err.so.3 without build-id.
                                                  Module libk5crypto.so.3 without build-id.
                                                  Module libkrb5.so.3 without build-id.
                                                  Module libunistring.so.5 without build-id.
                                                  Module libxml2.so.2 without build-id.
                                                  Module libbz2.so.1 without build-id.
                                                  Module liblzma.so.5 without build-id.
                                                  Module libacl.so.1 without build-id.
                                                  Module libbrotlicommon.so.1 without build-id.
                                                  Module libaws-c-common.so.1 without build-id.
                                                  Module libaws-checksums.so.1.0.0 without build-id.
                                                  Module libaws-c-sdkutils.so.1.0.0 without build-id.
                                                  Module libaws-c-cal.so.1.0.0 without build-id.
                                                  Module libaws-c-compression.so.1.0.0 without build-id.
                                                  Module libs2n.so.1 without build-id.
                                                  Module libaws-c-io.so.1.0.0 without build-id.
                                                  Module libaws-c-http.so.1.0.0 without build-id.
                                                  Module libaws-c-auth.so.1.0.0 without build-id.
                                                  Module libaws-c-s3.so.0unstable without build-id.
                                                  Module libaws-c-event-stream.so.1.0.0 without build-id.
                                                  Module libaws-c-mqtt.so.1.0.0 without build-id.
                                                  Module libzstd.so.1 without build-id.
                                                  Module libgssapi_krb5.so.2 without build-id.
                                                  Module libpsl.so.5 without build-id.
                                                  Module libssh2.so.1 without build-id.
                                                  Module libidn2.so.0 without build-id.
                                                  Module libnghttp2.so.14 without build-id.
                                                  Module libz.so.1 without build-id.
                                                  Module libcpuid.so.16 without build-id.
                                                  Module libboost_context.so.1.81.0 without build-id.
                                                  Module libarchive.so.13 without build-id.
                                                  Module libbrotlidec.so.1 without build-id.
                                                  Module libbrotlienc.so.1 without build-id.
                                                  Module libseccomp.so.2 without build-id.
                                                  Module libaws-crt-cpp.so without build-id.
                                                  Module libaws-cpp-sdk-core.so without build-id.
                                                  Module libaws-cpp-sdk-s3.so without build-id.
                                                  Module libaws-cpp-sdk-transfer.so without build-id.
                                                  Module libgcc_s.so.1 without build-id.
                                                  Module libstdc++.so.6 without build-id.
                                                  Module build-hook without build-id.
                                                  Stack trace of thread 1169762:
                                                  #0  0x00007f5a796a407c __pthread_kill_implementation (libc.so.6 + 0x8d07c)
                                                  #1  0x00007f5a79654e06 raise (libc.so.6 + 0x3de06)
                                                  #2  0x00007f5a7963d8f5 abort (libc.so.6 + 0x268f5)
                                                  #3  0x00007f5a798acc0b _ZN9__gnu_cxx27__verbose_terminate_handlerEv.cold (libstdc++.so.6 + 0xacc0b)
                                                  #4  0x00007f5a798bc21a _ZN10__cxxabiv111__terminateEPFvvE (libstdc++.so.6 + 0xbc21a)
                                                  #5  0x00007f5a798bc285 _ZSt9terminatev (libstdc++.so.6 + 0xbc285)
                                                  #6  0x00007f5a798bc4d7 __cxa_throw (libstdc++.so.6 + 0xbc4d7)
                                                  #7  0x00007f5a7a539ea2 _ZN3nix9getHomeOfB5cxx11Ej.cold (libnixutil.so + 0x4dea2)
                                                  #8  0x00007f5a7a5b86ce _ZZN3nix7getHomeB5cxx11EvENKUlvE_clEv.isra.0 (libnixutil.so + 0xcc6ce)
                                                  #9  0x00007f5a7a5a9248 _ZN3nix7getHomeB5cxx11Ev (libnixutil.so + 0xbd248)
                                                  #10 0x00007f5a7a5ab2e0 _ZN3nix12getConfigDirB5cxx11Ev (libnixutil.so + 0xbf2e0)
                                                  #11 0x00007f5a7a5ac547 _ZN3nix13getConfigDirsB5cxx11Ev (libnixutil.so + 0xc0547)
                                                  #12 0x00007f5a79d740d1 _ZN3nix18getUserConfigFilesB5cxx11Ev (libnixstore.so + 0x1740d1)
                                                  #13 0x00007f5a79d753d0 _ZN3nix8SettingsC2Ev (libnixstore.so + 0x1753d0)
                                                  #14 0x00007f5a79cc306a _sub_I_65535_0.0 (libnixstore.so + 0xc306a)
                                                  #15 0x00007f5a7a6acfee call_init (ld-linux-x86-64.so.2 + 0x4fee)
                                                  #16 0x00007f5a7a6ad0dc _dl_init (ld-linux-x86-64.so.2 + 0x50dc)
                                                  #17 0x00007f5a7a6c3170 _dl_start_user (ld-linux-x86-64.so.2 + 0x1b170)
                                                  ELF object binary architecture: AMD x86-64

Seems to be related to the homedir not being defined for the service maybe?

more dummy-proof docs

Might be nice to add some more dummy-proof docs, like what to put in your system flake.nix and more step-by-step guide to putting the token on the fs and referring to it from the configuration.nix

reports: add link to attestation on each item

reports mention static files not produced by derivations

The reports generated as described in the readme include paths such as /nix/store/4rd3hzbjjiwpm91a87qb4qk2ppqsgnj9-relaxedsandbox.nix, which appear to be files that are not produced by derivations but included through string interpolation(?).

We should double-check that such paths are never substituted from binary caches. If that is indeed the case, they can be removed from the report.

Allow recording attestations without derivation

Binary caches can contain objects that are 'directly' part of a build, instead of being the result of a derivation: an example is https://cache.nixos.org/h9lc1dpi14z7is86ffhl3ld569138595.narinfo

Since systems can have run-time dependencies on such objects, and in those cases those objects may be fetched from the binary cache, it seems helpful to be able to collect hashes for such objects, too. This means we should be able to store attestations that 'this output path has the following nar_hash' without a reference to a derivation - i.e. making the drv_id field of Attestation nullable and allowing this in the upload model as well.

reporting: record derivation attribute paths

In the context of a build/report, some (but not all) derivations are reachable as an attribute of the build. Might make sense to record that for a nicer report presentation.

filter hashes that are unlikely to be useful

Some hashes will be unlikely to be useful to share, as they are specific to users' configurations.

Could we somehow identify those and avoid uploading them? Or perhaps periodically garbage-collect 'old' hashes that only appear once?

data dump

It would be nice if we could provide data dumps for people wanting to do reporting beyond what the tool provides.

The current /derivations/ can provide an entry point for that for now, but I bet that'll get too slow once the dataset becomes big.

This could be an API, but also an out-of-band way of giving people access to the data.

reproducibility reporting frontend

It would be nice to provide a front-end showing 'how reproducible are we?' reports, ideally replacing the 4 reports at https://reproducible.nixos.org/ .

Perhaps it would make sense to have an endpoint where trusted users can upload a hierarchy of outputs under some name, and then based on that name anyone could fetch a UI could show that hierarchy, perhaps annotated/filterable:

grey: no or only one collected hash for this output
red: more than one hash, none identical
orange: more than one hash, some of which are identical
green: more than one hash, all consistent

report mentions paths not found on cache.nixos.org

report definitions created as described in the ReadMe:

nix-store -q --tree $(nix-build '<nixpkgs/nixos/release-combined.nix>' -A nixos.iso_gnome.x86_64-linux)

... mention a few paths not found on cache.nixos.org. This is likely due to some parameters passed into the nix-build being different from what Hydra provides.

Difference with `trustix`?

i just came across this project (though NixOS/nix#10101 – which has a cool issue number btw), and I'm curious how it compare to https://github.com/nix-community/trustix which seems to have a similar goal of improving the trust on the Nix build packages

copy-from-cache: batch mode

Replacing fetch-from-cache.sh, remembering not to retry 404's and other nonrecoverable errors.

declaratively specify user accounts

it'd be nice if we could specify the user accounts/tokens in the configuration instead of managing them in the database

reporting: allow linking to diffoscope output

To be collected by #14 somehow - we probably don't want to store that in the database though.

Avoid duplicates from the same user

Should we allow the same user to submit the same hash twice? Maybe ignore those?

diff-hook: support async queuing

Using either https://github.com/nix-community/queued-build-hook or a variation.
See #32

reporting: allow recording links to GitHub issues

when an unreproducibility is already tracked in GitHub

model: record output path?

might be worth collecting as it's easy to determine when you've built the derivation, but not necessarily afterwards

sign hashes

might be nice to record the signature for the collected hashes, so they can be used to independently verify things.

Might make sense to record this in the same format as in the narinfo's in the binary cache, so those can be easily imported - e.g. https://cache.nixos.org/5lrsc92ci8mk4bm2g08x00lqb5siw26j.narinfo :

StorePath: /nix/store/5lrsc92ci8mk4bm2g08x00lqb5siw26j-linux-5.10.105
URL: nar/0pspdjxc0milrndyli2q0avd7xyb4va89ppl5yy2yvibzgqa6hgn.nar.xz
Compression: xz
FileHash: sha256:0pspdjxc0milrndyli2q0avd7xyb4va89ppl5yy2yvibzgqa6hgn
FileSize: 72738596
NarHash: sha256:0k16qm8j5np49idnq3g0js2brarwjbvqcl9mpn84qzky2hrb861h
NarSize: 83173784
References: 
Deriver: f82ng23ql4840mb66dr480wbiln2ckir-linux-5.10.105.drv
Sig: cache.nixos.org-1:rXoVjXzZxG8AfXEjnkZg0mRTjkw4phcVTnGaxd8cAfqNQ0+uw2lz9NXpfjow0A0FXbsaC3hgL/ZXYBteV1jXDA==

You can use nix key generate-secret to generate a key, but I'm not sure wheter nix store sign can sign a rebuild that's not in your store, and how to extract the signature.

If this works out we might want to provide the API to support nix store copy sigs, though.

The signature is a signed fingerprint of the 1;<storepath>;<narHash>;<narSize>;<references>, where the references are comma-separated. The signature consists of the key name, a :, and the base64-encoded detached ed25519 signature.