Git Product home page Git Product logo

github-ci's Introduction

MiLaboratories custom GitHub Actions and common workflows

To reduce number of repositories we keep all our custom actions in single repository (this repo) This allows us to split workflow into as small pieces, as it is comfortable for code deduplication and actions unit testing.

The drawback is that we can't control versions of particular actions: new 'v2' branch creates 'v2' version of all actions in repository, regardless the only one could have backwards incompatible changes.

Development

All production workflow versions are stored in special 'vN' branches: v1, v2, v3, ...

Version branches could get new features, but should keep backwards compatibility of workflow modifications. This means you can put backward-incompatible changes into some action and update workflows, so they can work with updated action.

Each 'vN' version branch has connected 'vN-beta' branch with not-tested-yet versions of actions and workflows. This 'vN-beta' branches can be used in action and workflow tests.

Testing

For actions and workflows testing the another GitHub repository was created: github-ci-tests. This repository contains lots of workflows for unit-testing particular actions and whole workflows. Use it to test your changes before putting them into vN branch.

v1

First version of all actions.

Most of 'v1' workflows initialize 'context' inside the workflow itself. This makes workflow usage simpler (you don't need to create two dependant jobs), but makes impossible to control workflow's parameters based on current context's state.

v2

All workflows here expect outer workflow to call 'context/init' action before starting. Without 'context/init' 'v2' workflows can't perform some of their jobs.

v3

All workflows here expect outer workflow to call 'context/init' action before starting. Without 'context/init' 'v3' workflows can't perform some of their jobs.

v4

All workflows here expect outer workflow to call 'context/init' action before starting. Without 'context/init' 'v3' workflows can't perform some of their jobs.

This version introduces the following new features:

Breaking changes:

  • in java-gradle s3-key and s3-secret have been removed in favor of aws iam assume role with oidc provider to generate aws credentials

Recent changes:

  • switch to node20 because node16 has reached end-of-life

github-ci's People

Contributors

denkoren avatar mike-ainsel avatar

Stargazers

 avatar

Watchers

 avatar  avatar

github-ci's Issues

Custom version sorting in detect-versions for latestVersionTag

When git repository faces both 'v-' prefixed version tags and regular tags, it splits them into two groups during sort even when we ask it to sort 'as version numbers'.
This causes wrong 'latest' version detection.

I tried to solve the problem by giving 'v-' prefixed tags higher priority than regular, but this logic sucks when we switch FROM v-prefixed style TO regular.

This problem should be solved with plain JS :(

Use local docker images for Jinja2 and Telegram in actions

Current implementation of Jinja2 templating action and Telegram send message action docker builds new image from Dockerfile EACH TIME they are used in new jobs (this causes pip install and other things to be performed). This consumes a lot of worker time when we talk about 100 of notifications.

We should build local docker image, store it in github-ci repository (or somewhere outside) and use prebuilt images in actions.

git-crypt support in workflows

  • Create actions for git-crypt init
  • Add inputs, secrets and steps to work with git-crypt in our standard workflows

From D.Bolotin:

  • на тачке должен стоять git-crypt и gpg
  • есть переменная в которой лежит ключ (сейчас называется GPG_KEY), чтобы из нее заимпортить ключ в локальную базу gpg надо сделать как-то так:
    echo "$GPG_KEY" > ./private_key.gpg
    gpg --batch --yes --pinentry-mode loopback --import private_key.gpg
    rm private_key.gpg
    
    не пробовал, наверно надо в каком-то докере проверить что это реально работает
    взял отсюда
  • потом надо сделать вот так: git crypt unlock

Fix tag fetch conflicts that break detect-version

Run milaboratory/github-ci/actions/detect-version@v1
with:
canonize: true
fetch-depth: 100
/usr/bin/git fetch --deepen=1 origin refs/tags/:refs/tags/
From https://github.com/milaboratory/redberry-pipe

  • [new tag] 1.0.0 -> 1.0.0
    ! [rejected] 1.1.0 -> 1.1.0 (would clobber existing tag)
  • [new tag] v0.9 -> v0.9
  • [new tag] v0.9.1 -> v0.9.1
  • [new tag] v0.9.2 -> v0.9.2
  • [new tag] v0.9.3 -> v0.9.3
  • [new tag] v0.9.4 -> v0.9.4
  • [new tag] v0.9.5 -> v0.9.5
  • [new tag] v0.9.6 -> v0.9.6
  • [new tag] v0.9.7 -> v0.9.7
    Error: command "git 'fetch' '--deepen=1' 'origin' 'refs/tags/:refs/tags/'" failed with code '1':

From https://github.com/milaboratory/redberry-pipe

Autodetect list of reviewers in environment.

We need an action to detect list of reviewers allowed to approve deployment to specific environment.

This will allow to mention responsible teammates in 'review is required' notifications.

Create draft releases _before_ release confirmation

Current workflow waits for release confirmation and then creates GitHub release along with S3 uploads and docker pushes.

It is much more convenient to have release draft created before confirmation, so developer can change description and add make other manual changes to release before it gets published.

Then, when release is confirmed in CI (environment deployment is approved), existing elease is published by CI automatically.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.