Git Product home page Git Product logo

github-action-atmos-component-updater's Introduction

github-action-atmos-component-updater

Latest ReleaseSlack Community

This is GitHub Action that can be used as a workflow for automatic updates via Pull Requests in your infrastructure repository according to versions in components sources.


Note

This project is part of Cloud Posse's comprehensive "SweetOps" approach towards DevOps.

Learn More

It's 100% Open Source and licensed under the APACHE2.

Introduction

This is GitHub Action that can be used as a workflow for automatic updates via Pull Requests in your infrastructure repository according to versions in components sources.

Key Features:

  • Selective Component Processing: Configure the action to exclude or include specific components using wildcards, ensuring that only relevant updates are processed.
  • PR Management: Limit the number of PRs opened at a time, making it easier to manage large-scale updates without overwhelming the system. Automatically close old component-update PRs, so they don't pile up.
  • Material Changes Focus: Automatically open pull requests only for components with significant changes, skipping minor updates to component.yaml files to reduce unnecessary PRs and maintain a streamlined system.
  • Informative PRs: Link PRs to release notes for new components, providing easy access to relevant information, and use consistent naming for easy tracking.
  • Scheduled Updates: Run the action on a cron schedule tailored to your organization's needs, ensuring regular and efficient updates.

Usage

Prerequisites

This GitHub Action once used in workflow needs permissions to create/update branches and open/close pull requests so the access token needs to be passed.

It can be done in two ways:

  • create a dedicated Personal Access Token (PAT)
  • use GITHUB_TOKEN

If you would like to use GITHUB_TOKEN make sure to set permissions in the workflow as follow:

permissions:
  contents: write
  pull-requests: write

Also, make sure that you set to Allow GitHub Actions to create and approve pull requests on both organization and repository levels:

  • https://github.com/organizations/YOUR-ORG/settings/actions
  • https://github.com/YOUR-ORG/YOUR-REPO/settings/actions

Workflow example

  name: "atmos-components"

  on:
    workflow_dispatch: {}

    schedule:
      - cron:  '0 8 * * 1'         # Execute every week on Monday at 08:00

  permissions:
    contents: write
    pull-requests: write

  jobs:
    update:
      runs-on: ubuntu-latest
      steps:
        - name: Checkout
          uses: actions/checkout@v3
          with:
            fetch-depth: 0

        - name: Update Atmos Components
          uses: cloudposse/github-action-atmos-component-updater@v2
          with:
            github-access-token: ${{ secrets.GITHUB_TOKEN }}
            max-number-of-prs: 5
            include: |
              aws-*
              eks/*
              bastion
            exclude: aws-sso,aws-saml

Using a Custom Atmos CLI Config Path (atmos.yaml)

If your atmos.yaml file is not located in the root of the infrastructure repository, you can specify the path to it using ATMOS_CLI_CONFIG_PATH env variable.

  # ...
  - name: Update Atmos Components
    uses: cloudposse/github-action-atmos-component-updater@v2
    env:
      # Directory containing the `atmos.yaml` file
      ATMOS_CLI_CONFIG_PATH: ${{ github.workspace }}/rootfs/usr/local/etc/atmos/
    with:
      github-access-token: ${{ secrets.GITHUB_TOKEN }}
      max-number-of-prs: 5

Customize Pull Request labels, title and body

  # ...
  - name: Update Atmos Components
    uses: cloudposse/github-action-atmos-component-updater@v2
    with:
      github-access-token: ${{ secrets.GITHUB_TOKEN }}
      max-number-of-prs: 5
      pr-title: 'Update Atmos Component \`{{ component_name }}\` to {{ new_version }}'
      pr-body: |
        ## what
        Component \`{{ component_name }}\` was updated [{{ old_version }}]({{ old_version_link }}) → [{{ old_version }}]({{ old_version_link }}).

        ## references
        - [{{ source_name }}]({{ source_link }})
      pr-labels: |
        component-update
        automated
        atmos

IMPORTANT: The backtick symbols must be escaped in the GitHub Action parameters. This is because GitHub evaluates whatever is in the backticks and it will render as an empty string.

For title template these placeholders are available:

  • component_name
  • source_name
  • old_version
  • new_version

For body template these placeholders are available:

  • component_name
  • source_name
  • source_link
  • old_version
  • new_version
  • old_version_link
  • new_version_link
  • old_component_release_link
  • new_component_release_link

FAQ

The action cannot find any components

You may see that the action returns zero components:

[06-03-2024 17:53:47] INFO    Found 0 components
[]

This is a common error when the workflow has not checked out the repository before calling this action. Add the following before calling this action.

- name: Checkout
  uses: actions/checkout@v4

The action cannot find the component.yaml file

You may see the action fail to find the component.yaml file for a given component as such:

FileNotFoundError: [Errno 2] No such file or directory: 'components/terraform/account-map/component.yaml'

This is likely related to a missing or invalid atmos.yaml configuration file. Set ATMOS_CLI_CONFIG_PATH to the path to your Atmos configuration file.

env:
  ATMOS_CLI_CONFIG_PATH: ${{ github.workspace }}/rootfs/usr/local/etc/atmos/

The action does not have permission to create Pull Requests

Your action may fail with the following message:

github.GithubException.GithubException: 403 {"message": "GitHub Actions is not permitted to create or approve pull requests.", "documentation_url": "https://docs.github.com/rest/pulls/pulls#create-a-pull-request"}

In order to create Pull Requests in your repository, we need to set the permissions for the workflow:

permissions:
  contents: write
  pull-requests: write

And you need to allow GitHub Actions to create and approve pulls requests in both the GitHub Organization and Repository:

  1. https://github.com/organizations/YOUR-ORG/settings/actions > Allow GitHub Actions to create and approve pull requests
  2. https://github.com/YOUR-ORG/YOUR-REPO/settings/actions > Allow GitHub Actions to create and approve pull requests

Inputs

Name Description Default Required
atmos-version Atmos version to use for vendoring. Default 'latest' latest false
dry-run Skip creation of remote branches and pull requests. Only print list of affected componented into file that is defined in 'outputs.affected-components-file' false false
exclude Comma or new line separated list of component names to exclude. For example: 'vpc,eks/*,rds'. By default no components are excluded. Default '' false
github-access-token GitHub Token used to perform git and GitHub operations ${{ github.token }} false
include Comma or new line separated list of component names to include. For example: 'vpc,eks/*,rds'. By default all components are included. Default '*' * false
infra-terraform-dirs Comma or new line separated list of terraform directories in infra repo. For example 'components/terraform,components/terraform-old. Default 'components/terraform' components/terraform false
log-level Log level for this action. Default 'INFO' INFO false
max-number-of-prs Number of PRs to create. Maximum is 10. 10 false
pr-body-template A string representing a Jinja2 formatted template to be used as the content of a Pull Request (PR) body. If not set template from src/templates/pr\_body.j2.md will be used false
pr-labels Comma or new line separated list of labels that will added on PR creation. Default: component-update component-update false
pr-title-template A string representing a Jinja2 formatted template to be used as the content of a Pull Request (PR) title. If not, set template from src/templates/pr\_title.j2.md will be used false
vendoring-enabled Do not perform 'atmos vendor component-name' on components that wasn't vendored true false

Outputs

Name Description
affected The affected components
has-affected-stacks Whether there are affected components

Related Projects

Check out these related projects.

References

For additional context, refer to some of these links.

✨ Contributing

This project is under active development, and we encourage contributions from our community. Many thanks to our outstanding contributors:

🐛 Bug Reports & Feature Requests

Please use the issue tracker to report any bugs or file feature requests.

💻 Developing

If you are interested in being a contributor and want to get involved in developing this project or help out with Cloud Posse's other projects, we would love to hear from you! Hit us up in Slack, in the #cloudposse channel.

In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow.

  1. Review our Code of Conduct and Contributor Guidelines.
  2. Fork the repo on GitHub
  3. Clone the project to your own machine
  4. Commit changes to your own branch
  5. Push your work back up to your fork
  6. Submit a Pull Request so that we can review your changes

NOTE: Be sure to merge the latest changes from "upstream" before making a pull request!

🌎 Slack Community

Join our Open Source Community on Slack. It's FREE for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build totally sweet infrastructure.

📰 Newsletter

Sign up for our newsletter and join 3,000+ DevOps engineers, CTOs, and founders who get insider access to the latest DevOps trends, so you can always stay in the know. Dropped straight into your Inbox every week — and usually a 5-minute read.

📆 Office Hours

Join us every Wednesday via Zoom for your weekly dose of insider DevOps trends, AWS news and Terraform insights, all sourced from our SweetOps community, plus a live Q&A that you can’t find anywhere else. It's FREE for everyone!

About

This project is maintained by Cloud Posse, LLC.

We are a DevOps Accelerator for funded startups and enterprises. Use our ready-to-go terraform architecture blueprints for AWS to get up and running quickly. We build it with you. You own everything. Your team wins. Plus, we stick around until you succeed.

Learn More

Your team can operate like a pro today.

Ensure that your team succeeds by using our proven process and turnkey blueprints. Plus, we stick around until you succeed.

📚 See What's Included
  • Reference Architecture. You'll get everything you need from the ground up built using 100% infrastructure as code.
  • Deployment Strategy. You'll have a battle-tested deployment strategy using GitHub Actions that's automated and repeatable.
  • Site Reliability Engineering. You'll have total visibility into your apps and microservices.
  • Security Baseline. You'll have built-in governance with accountability and audit logs for all changes.
  • GitOps. You'll be able to operate your infrastructure via Pull Requests.
  • Training. You'll receive hands-on training so your team can operate what we build.
  • Questions. You'll have a direct line of communication between our teams via a Shared Slack channel.
  • Troubleshooting. You'll get help to triage when things aren't working.
  • Code Reviews. You'll receive constructive feedback on Pull Requests.
  • Bug Fixes. We'll rapidly work with you to fix any bugs in our projects.

License

License

Preamble to the Apache License, Version 2.0

Complete license is available in the LICENSE file.

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.

Trademarks

All other trademarks referenced herein are the property of their respective owners.

Copyright © 2017-2024 Cloud Posse, LLC

README footer

Beacon

github-action-atmos-component-updater's People

Contributors

dudymas avatar goruha avatar milldr avatar zdmytriv avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

github-action-atmos-component-updater's Issues

Clean up State Branches

Have a question? Please checkout our Slack Community or visit our Slack Archive.

Slack Community

Describe the Feature

The component updater creates and orphans a lot of branches. Any PR that is not merged and is closed creates a stale branch.

Add an (optional) step to the component updater workflow to clean up stale branches

Expected Behavior

The component updater should include an option to regularly clean up stale branches in the same workflow

Use Case

Clean up stale branches every night

Describe Ideal Solution

No persistent stale branches created by this action

Alternatives Considered

Use another tool on a schedule to clean up stale branches. Examples

Additional Context

n/a

postgresql-user not up to date

Describe the Bug

Submodule modules/postgresql-user is not updated within aurora-postgres-resources.
The line

if not os.path.isfile(updated_file):
skip folders.
This could be the reason for skipping new versions for a "submodule".

Expected Behavior

Changes in a submodule only should be detected by the updater.

Steps to Reproduce

Steps to reproduce the behavior:

  1. Configure component.yaml with version 1.310.0
  2. I expect to get version 1.330.0
  3. Updater say that no changes are detected.

Screenshots

2023-10-26T10:19:11.1443389Z [26-10-2023 10:19:11] INFO    Processing component: aurora-postgres-resources
2023-10-26T10:19:11.6096994Z [26-10-2023 10:19:11] INFO    Looking for PR with branch: component-update/aurora-postgres-resources/1.330.0
2023-10-26T10:19:12.0148588Z [26-10-2023 10:19:12] INFO    Executing 'atmos vendor pull -c aurora-postgres-resources' for component version '1.310.0' ... 
2023-10-26T10:19:15.0423445Z [26-10-2023 10:19:15] INFO    Successfully vendored component: aurora-postgres-resources
2023-10-26T10:19:15.0428870Z [26-10-2023 10:19:15] INFO    Executing 'atmos vendor pull -c aurora-postgres-resources' for component version '1.330.0' ... 
2023-10-26T10:19:18.2087857Z [26-10-2023 10:19:18] INFO    Successfully vendored component: aurora-postgres-resources
2023-10-26T10:19:18.2125023Z [26-10-2023 10:19:18] INFO    Looking good. No changes found

Environment (please complete the following information):

Anything that will help us triage the bug will help. Here are some ideas:

  • github action

Embed Release Notes

Embed the release notes in a <details/> block for the current release. Bonos points for aggregating all release notes between versions.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.