This allows config either to be set via config.yaml like this:

integrations:
  github:
    enable: true

Or through an environment variable:

ESCI_INTEGRATIONS_GITHUB_ENABLE=true

This helps in installing Estafette from scratch.

Default enabled to false

Support commands like

/estafette encrypt <secret>
/estafette release github.com/estafette/estafette-ci-web production 0.0.109
/estafette logs github.com/estafette/estafette-ci-web 0.0.109

The exact commands and structure still have to be fleshed out. The / at the beginning works for Slack slash commands and can be used to open a command line in the web interface. Perhaps in the web interface this will already show /estafette and a number of suggested commands.

This is still part of estafette-ci-api, but should run as a Kubernetes job separately when deployed.

This decouples the logic to handle events into small pieces. For example a CreateBuildCommand could kick off the following actions:

• store command in commands table
• create a build in builds table
• create a build job
• update computed tables used to serve quicker responses for the api endpoints

Like Google Inbox show a notification if a new frontend is available and the user needs to refresh to load it.

To avoid having to define roles in multiple locations reusing the permissions as set in github or bitbucket for a particular repository will be used as the source for who can view the pipelines (read permissions on the git repo) and deploy releases (write permissions on the git repo). Combine with the login system. Perhaps needed to link other types of login to an account, for example github logins.

It doesn't need to perform the scan itself but interface with another system to ask for it. Either clair or in case of Google Container Registry we can ask GCR for the security status. For Docker Hub it seems to be a paid feature.

Fixes

• dgrijalva/jwt-go#463
• dgrijalva/jwt-go#314

To avoid this warning and future failures:

batch/v1beta1 CronJob is deprecated in v1.21+, unavailable in v1.25+; use batch/v1 CronJob

In order to easily set up Bitbucket integration add a Bitbucket app descriptor endpoint

Configure private registries at estafette-ci-api level and make sure estafette-ci-builder authenticates itself for any private registries used by it's steps before starting execution of the build steps

Up until now each component - api, web, db-migrator, cron-event-sender - built, tested and pushed its own helm chart into the helm.estafette.io repository. By moving them into this repository as subcharts releasing changes in any of them becomes much simpler. Just updating the subchart version is enough to pick up changes in any of those components.

Estafette triggers build every-time a new branch is created, this is of no use as the branch is same as master.

Temporary workaround to address issue golang-jwt/jwt#98

By default it will keep the default images, but when clearDefaultTrustedImages: true is set it will only use the specified trusted images.

In order to run the web ui behind Identity Aware Proxy (IAP) adding the following values to the estafette-ci-api subchart:

# To put the admin interface behind Identity Aware Proxy (IAP) on Google Cloud Platform
iap:
  enabled: false
  # Create Google Cloud OAuth credentials and set client id and secret
  # clientId:
  # clientSecret:

With Estafette CI consisting of so many separate components a Helm chart to install all of them in one go is needed.

This should run as a cronjob and either directly access the database and clean up logs, etc. Or do this via the estafette-ci-api endpoints.

Using https://github.com/sethvargo/go-envconfig for allowing config to be initialized from environment variables led to pointers getting instantiated instead of remaining nil. This led to incorrect behaviour for affinity rules config for build and release jobs.

In order to standardize how work on features happens and how releases are created a git workflow has to be adopted that prescribes how to do this.

A list of well-known workflows:

• https://www.atlassian.com/git/tutorials/comparing-workflows#centralized-workflow
• https://www.atlassian.com/git/tutorials/comparing-workflows/feature-branch-workflow (current)
• https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow
• https://www.atlassian.com/git/tutorials/comparing-workflows/forking-workflow
• https://guides.github.com/introduction/flow/index.html
• https://docs.gitlab.com/ee/topics/gitlab_flow.html

What a workflow needs to do for Estafette CI is the following

• work in feature branches to make use of pull requests to have multiple pairs of eyes on code and working builds before merging
• create release branches to get semver version without label and be able to create a github release easily
• keep things simple and prefer rolling forward as much as possible

What's less important but some workflows do offer is

• being able to hotfix a released version while work on the next version is ongoing or even released

For now support at least Google login. Allow the login domain to be restricted. Make private repositories viewable to logged in users until role-based access is implemented.

Currently logs only show when a build has fully finished. Make this streaming to both the db and the web interface.

With Kubernetes signed certificates being deprecated in 1.22+ version 6.1.0 of the (Cockroachdb Helm Chart)[https://github.com/cockroachdb/helm-charts] introduces the self-signer. This also simplifies setup of estafette-ci since it no longer requires any csr approval.

Default enabled to false

Currently a Oauth provider config property clientSecret is empty error leads to the api container failing to start when config has no auth.google and auth.github properties.

To make configuring Estafette possible without redeploying any components it is useful to create Custom Resource Definitions (CRD) for various parts of configuration, like integrations, credentials and trustedImages in order to configure Estafette on the fly.

In order to make it easier to keep most of the default config, but add separate config for more localized config sections like credentials and possibly trustedImages splitting it into separate config files and merging those when reading them this is now feasible.

The best way of configuring an application I've used is that of grafana. It has a yaml based config file and allows you to override what's there with environment variables.

This config file should support estafette secrets so the secret values can be encrypted and be hidden in plain sight. The encryption key itself has be mounted as a secret into the estafette-ci-api.

During first install there might be a chicken and egg situation for using these secrets. Perhaps the future cli can assist in this by being able to install all estafette components and generate the encrypted secrets on the fly.

In order to simplify setup the secretDecryptionKey and jwtKey are automatically initialized on first install of the Helm chart, so you don't need to specify a value. You can set a value in the values with secret.secretDecryptionKey and secret.jwtKey if you want though.

Release targets could be a Kubernetes cluster, a package repository or container registry. Release targets defined at server level can be reused from individual application manifests to deploy / push to without knowing all secrets required to do so.

If we have cloning on a release enabled with

clone: true

And we try to release a build which was done on a feature branch, which since then has been merged and the branch was deleted, the cloning fails with this error:

Cloning git repository foo to branch bar and revision c8cbaf0c359e0fd605bd86a345fad34f0f6e5cd1 with shallow clone is true and depth 50...
Cloning into '/estafette-work'...
warning: Could not find remote branch bar to clone.
fatal: Remote branch bar not found in upstream origin

Is this a bug or as designed?
I'm not sure if this can be solved though, so when we merge a PR and check "Delete branch", whether that only removes the branch marker, or if it actually deletes all the commits in the branch. If it removes the actual commits, then we might not be able to solve it. In that case maybe we should ask the devs not to use the "Delete branch" option when they merge their PRs?