Git Product home page Git Product logo

trdl's Introduction

trdl

maintainability coverage github discussions coc

trdl (stands for "true delivery") is an Open Source solution providing a secure channel for delivering updates from the Git repository to the end user.

The project team releases new versions of the software and switches them in the release channels. Git acts as the single source of truth while Vault is used as a tool to verify operations as well as populate and maintain the TUF repository. The user selects a release channel, continuously receives the latest software version from the TUF repository, and uses it.

Scheme

We have been successfully using trdl to continuously deliver our werf CI/CD tool to CI runners and user hosts.

Architecture

trdl combines two key components: the server and the client.

trdl-server:

  • builds and releases software versions;
  • publishes the release channels (here is an example configuration from werf);
  • ensures the release and the publication security via verifying the minimal number of valid GPG signatures associated with an action;
  • ensures the object storage security via saving data signed by keys (no one has access to those keys) and continuously rotating TUF keys and metadata.

trdl-client:

  • manages software repositories;
  • updates software version within the selected release channel;
  • provides easy operation with software version artifacts in the shell session;
  • ensures safe communication via working with the TUF repository in a reliable fashion.

How it works

Releasing

Release

Publishing the channels

Publication

Installation

trdl-client

Download trdl client binaries from the GitHub Releases page, optionally verifying the binary with the PGP signature.

Documentation

Project's website is now available with more information (including developers quickstart) to follow soon.

Community & support

Please feel free to reach developers/maintainers and users via GitHub Discussions for any questions regarding trdl.

Your issues are processed carefully if posted to issues at GitHub.

License

Apache License 2.0, see LICENSE.

trdl's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

trdl's Issues

Error: unable to update tuf meta: tuf: failed to decode timestamp.json: expired at 2021-10-30 09:32:34 +0000 UTC

I am trying to install kubedog, but got an error: Error: unable to update tuf meta: tuf: failed to decode timestamp.json: expired at 2021-10-30 09:32:34 +0000 UTC

$ echo 'export PATH=$HOME/bin:$PATH' >> ~/.bash_profile
$ export PATH="$HOME/bin:$PATH"
$ curl -L "https://tuf.trdl.dev/targets/releases/0.1.3/linux-$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/')/bin/trdl" -o /tmp/trdl
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   162  100   162    0     0    177      0 --:--:-- --:--:-- --:--:--   177
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
100 8028k  100 8028k    0     0  3935k      0  0:00:02  0:00:02 --:--:-- 9206k
$ mkdir -p ~/bin
$ install /tmp/trdl ~/bin/trdl
$ trdl add kubedog https://tuf.kubedog.werf.io 1 2cc56abdc649a9699074097ba60206f1299e43b320d6170c40eab552dcb940d9e813a8abf5893ff391d71f0a84b39111ffa6403a3e038b81634a40d29674a531
$ source $(trdl use kubedog 0 stable)
Error: unable to update tuf meta: tuf: failed to decode timestamp.json: expired at 2021-10-30 09:32:34 +0000 UTC
Cleaning up file based variables
00:00
ERROR: Job failed: command terminated with exit code 1

Trdl client hanged when timeout has occured

First attempt to add trdl repo failed with timeout error:

$ curl -sSL https://werf.io/install.sh | bash -s -- --ci
[INPUT REQUIRED] Current login shell is "bash". Press ENTER to setup werf for this shell or choose another one.
[b]ash/[z]sh/[a]bort? Default: bash.
[INFO] Skipping trdl installation: already installed in "/home/gitlab-runner/bin/".
[INFO] Adding werf repo to trdl.
Error: unable to init repository "werf" client: unable to download "1.root.json": Get "https://tuf.werf.io/1.root.json": dial tcp 54.38.250.137:443: i/o timeout
[FATAL] Can't add "werf" repo to trdl.
[FATAL] Aborting.
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit status 1

— this process has failed with the error, but there also was a spawned background process, which hanged:

gitlab-+  137033  0.0  0.0   7232  1264 ?        S    03:46   0:00 bash -l
gitlab-+  137036  0.0  0.0   7800  3608 ?        S    03:46   0:00  \_ bash -s -- --ci
gitlab-+  137198  0.0  0.1 719784 10448 ?        Sl   03:46   0:01      \_ /home/gitlab-runner/bin/trdl add werf https://tuf.werf.io/ 1 b7ff6bcbe598e072a86d595a3621924c8612c7e6dc6a82e919abe89707d7e3f468e616b5635630680dd1e98fc362ae5051728406700e6274c5ed1ad92bea52a2

— this process held a file lock.

Next tries to run install script with trdl will always fail with file-lock timeout:

$ curl -sSL https://werf.io/install.sh | bash -s -- --ci
[INPUT REQUIRED] Current login shell is "bash". Press ENTER to setup werf for this shell or choose another one.
[b]ash/[z]sh/[a]bort? Default: bash.
[INFO] Skipping trdl installation: already installed in "/home/gitlab-runner/bin/".
[INFO] Adding werf repo to trdl.
Error: "/home/gitlab-runner/.trdl/.locks/2d478ff12948954c4b93c051a64d7374ee343545587b7059d75d5497bc7f37dc" file lock timeout 30s expired
[FATAL] Can't add "werf" repo to trdl.
[FATAL] Aborting.
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit status 1

It is required to manually kill hanged background process to make trdl work again.

Add Option for Specifying Different Metadata and Targets URLs

Currently there is no way to specify different metadata and target URLs for the same repository, this means that trdl can't handle repositories such as the Bottlerocket one. For reference this is how the same is handled in tuftool:

VERSION="v1.6.1"
VARIANT="vmware-k8s-1.24"
OVA="bottlerocket-${VARIANT}-x86_64-${VERSION}.ova"
OUTDIR="${VARIANT}-${VERSION}"

tuftool download "${OUTDIR}" --target-name "${OVA}" \
   --root ./root.json \
   --metadata-url "https://updates.bottlerocket.aws/2020-07-07/${VARIANT}/x86_64/" \
   --targets-url "https://updates.bottlerocket.aws/targets/"

Error: leveldb/table: corruption on data-block (pos=0)

$ source $(trdl use werf 1.2 beta)
Error: unable to init tuf client: unable to get meta from file local store: leveldb/table: corruption on data-block (pos=0): checksum mismatch, want=0xf6eefbb5 got=0xac0f3c59 [file=049347.ldb]
-bash: source: filename argument required
source: usage: source filename [arguments]

Site reorganization

  • Add Documentation section.
  • Move Quickstart, Security and Reference to Documentation.
  • Make Documentation/How it works from sections of the Home page.
  • Remove all sections except problems and benefits from the Home page.
  • Fix styles on the Home page.
  • Integrate How it works, Quickstart and Security into Documentation section.

MacOS: Self-update failed: unable to update tuf meta: tuf: no root keys found in local meta store

Recently I've got an error which I didn't remember. It was something about "database" and "file limit exceeded". Those time I've just removed folder that was mentioned in the error. Then I reinstalled trdl and now I'm getting the following error:

~ source $("$HOME/bin/trdl" use werf "1.2" "stable")
Previous run of trdl update in background generated following errors:
WARNING: Self-update failed: unable to update tuf meta: tuf: no root keys found in local meta store

I've looked through the instructions on werf documentation and trdl as well, but didn't found any idea how to fix that.

May be I need to completely remove trdl with all tuf metas and then reinstall trdl. But I don't know all paths that I have to remove. May be uninstallation sequence mentioned somewhere in documentation and I have missed it?

I need your advice, guys. For now each time I open the Terminal I'm getting that warning, its a bit annoying.
Thank you!

P.S. I'm running on MacOS Intel.

Restriction on re-release

It is necessary to remove the possibility of restarting a successful release. However, such an opportunity should be for a failed/broken release.

Redesign CLI

  • Divide commands into groups
  • Update layout of a command usage
  • Update command descriptions

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.