Git Product home page Git Product logo

varunsridharan / action-github-workflow-sync Goto Github PK

View Code? Open in Web Editor NEW
60.0 4.0 15.0 494 KB

Github Action To Sync Github Action's Workflow Files Across Repositories

Home Page: https://github.com/marketplace/actions/github-workflow-sync

License: MIT License

JavaScript 100.00%
github-actions github-action github-action-docker github-workflow github-workflows github-action-workflow workflow workflows workflow-sync github-action-workflow-sync

action-github-workflow-sync's Introduction

Github Workflow Sync - Github Action

Github Action To Sync Github Action's Workflow Files Across Repositories

https://cdn.svarun.dev/gh/varunsridharan/action-github-workflow-sync/action-banner.jpg

Use Case ๐Ÿค” ?

This Github Action can come in handy when you have lot of projects like i do. where in some case certain projects users action workflow which are common across projects. Example : Project 1 & Project 2 it can be pain to keep all the workflow updated with Github Action's Module's version.

This also isn't limited to Github Action yaml files - another use case could be keeping the .editorconfig, LICENSE, tsconfig.json, tslint.json, .gitignore, etc. in sync across all your repositories.

Here where this action comes in and reduces your stress ๐Ÿ˜‰ it can update all your repository actions file based on the config provided

โš™๏ธ Configuration

Argument Defaults Description
GITHUB_TOKEN - Required Token to use to get repos and write secrets. ${{secrets.GITHUB_TOKEN}} will not work. instead Personal Access Token Required*
GIT_URL github.com URL for the instance of github, where repositories should be searched for. Change if using a GHES instance.
REPOSITORIES - Required New line deliminated regex expressions to select repositories. Repositires are limited to those in whcich the token user is an owner or collaborator.
WORKFLOW_FILES - Required New line deliminated regex expressions. workflow files to be copied to provided repositores
DRY_RUN false Run everything except for nothing will be pushed.
WORKFLOW_FILES_DIR workflows Local Path Where Common Workflow Files Are Located Eg : workflows
AUTO_CREATE_NEW_BRANCH false Auto create new brach in a repository if the branch dose not exists
COMMIT_EACH_FILE false if you need to keep track of each file's commit history separate then set it to true
PULL_REQUEST false Set to true if you want the changes to be pushed via pull request.
SKIP_CI false Set to true if you want skip all automations inside target repository.
COMMIT_MESSAGE false You can provide your custom commit message.
RETRY_MODE true Enable retry and throttling octokit plugins to avoid secondary rate limits on github content creation.

Personal Access Token Scope

Github Personal Token Is required with the below scope

https://cdn.svarun.dev/gh/varunsridharan/action-github-workflow-sync/scope.jpg

โ„น๏ธ Full Repo is only required when you need to update private repository if your are updating only public repository then just select public_repo inside repo scope

Click Here To Generate A Token


REPOSITORIES Configuration Examples

Repository With Default Brach
REPOSITORIES: |
    username/repo
    username/repo2
Repositry With Custom Branch
REPOSITORIES: |
    username/repo@dev
    username/repo1@dev2

You Can also have same repository multiple times if you provide different branch name


WORKFLOW_FILES Configuration Examples

  1. If you use = as a file seperator file1.md=myfile.md then file1 from the current repository will be copied to remote repository with the name of myfile.md
  2. If you use != as a file seperator file1.md!=myfile.md then file1 from the current repository will be copied to remote repository with the name of myfile.md only if myfile.md already not exists in the remote repository
Files - Source & Destination File Without Custom Name
WORKFLOW_FILES: |
    dependabot.yml
    .github/settings.yml

dependabot.yml will save in root folder in the repository

.github/settings.yml will save in .github in the repository

Files - Source File In Root & Destination File In Custom Location
WORKFLOW_FILES: |
    hello-bot.yml=.github/
    pr-bot.yml=.github/pull-request.yml

hello-bot.yml will save in .github in the repository with the same name

pr-bot.yml will save in .github in the repository with the name pull-request.yml

Folders - Source & Destination Folders Without Custom Name
WORKFLOW_FILES: |
    folder1
    .github/folder2

folder1 will save in root folder in the repository

.github/folder2 will save in .github in the repository

Folders - Source & Destination Folders With Custom Name
WORKFLOW_FILES: |
    folder1=./save-to-folder
    .github/folder2=custom-folder/save-to-folder2

folder1 will save inside REPOSITORY ROOT in the name of save-to-folder

.github/folder2 will save inside custom-folder in the name of save-to-folder2


How Files Sync Work ?

Before copying the WORKFLOW_FILES from the source to destination. this action will provide some flexibility. this searchs for a file in various locations for example lets take settings.yml as the file that you want to sync for multiple repository

Below are the locations that this action search for the file/folder

  • ./{OWNER}/{REPO_NAME}/workflows/{filename}
  • ./{OWNER}/workflows/{filename}
  • ./{WORKFLOW_FILES_DIR}/{filename}
  • ./.github/workflows/{filename}
  • ./{OWNER}/{REPO_NAME}/{filename}
  • ./{OWNER}/{filename}
  • ./{filename}

if the settings.yml is found inside workflows folder then the destination is automaitcally forced to .github/workflows in the destination repo

if the settings.yml is outside of workflows folder then the destination then its copied to the destination

How this can be useful ?

Lets assume that you want to maintain all the common github files in a single repository and suddenly a repository needs a single file to be changed in that case instead of editing the action yml file. you can just create a folder like {REPO_OWNER}/{REPO_NAME}/{FILE} to copy the overriden file to the destination

๐Ÿš€ Usage

Step 1

Create a New Repository or use our Repository Template

Step 2

if you have used our template repository then edit the file inside .github/workflows/workflow-sync.yml

OR

Create a new file in .github/workflows/ named workflow-sync.yml and copy & paste the below file content

workflow-sync.yml content

name: Workflow Sync

on:
  push:
    branches:
      - master
env:
  DRY_RUN: false
  REPOSITORIES: |
  
  WORKFLOW_FILES: |

jobs:
  Github_Workflow_Sync:
    runs-on: ubuntu-latest
    steps:
      - name: Fetching Local Repository
        uses: actions/checkout@master
      - name: Running Workflow Sync
        uses: varunsridharan/action-github-workflow-sync@main
        with:
          DRY_RUN: ${{ env.DRY_RUN }}
          REPOSITORIES: ${{ env.REPOSITORIES }}
          WORKFLOW_FILES: ${{ env.WORKFLOW_FILES }}
          GITHUB_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}

Troubleshooting

Spacing

Spacing around the equal sign is important. For example, this will not work:

WORKFLOW_FILES: |
  folder/file-sync.yml = folder/test.txt

It passes to the shell file 3 distinct objects

  • folder/file-sync.ymll
  • =
  • folder/test.txt

instead of 1 object

  • folder/file-sync.yml = folder/test.txt

and there is nothing I can do in code to make up for that

Slashes

You do not need (nor want) leading / for the file path on either side of the equal sign

The only time you need / trailing is for folder copies. While a file copy will technically still work with a leading /, a folder copy will not


๐Ÿ“ Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Checkout CHANGELOG.md

๐Ÿค Contributing

If you would like to help, please take a look at the list of issues.

๐Ÿ’ฐ Sponsor

I fell in love with open-source in 2013 and there has been no looking back since! You can read more about me here. If you, or your company, use any of my projects or like what Iโ€™m doing, kindly consider backing me. I'm in this for the long run.

  • โ˜• How about we get to know each other over coffee? Buy me a cup for just $9.99
  • โ˜•๏ธโ˜•๏ธ How about buying me just 2 cups of coffee each month? You can do that for as little as $9.99
  • ๐Ÿ”ฐ We love bettering open-source projects. Support 1-hour of open-source maintenance for $24.99 one-time?
  • ๐Ÿš€ Love open-source tools? Me too! How about supporting one hour of open-source development for just $49.99 one-time ?

๐Ÿ“ License & Conduct

๐Ÿ“ฃ Feedback

  • โญ This repository if this project helped you! ๐Ÿ˜‰
  • Create An ๐Ÿ”ง Issue if you need help / found a bug

Connect & Say ๐Ÿ‘‹


Built With โ™ฅ By Varun Sridharan ๐Ÿ‡ฎ๐Ÿ‡ณ


action-github-workflow-sync's People

Contributors

afirth avatar slevinwasalreadytaken avatar smlgbl avatar tdonaworth avatar varunsridharan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

action-github-workflow-sync's Issues

'License' and 'Code of Conduct' files missing, links are broken

First of all, thanks for this action.

I've noticed that though the README has a section about the code being GPL-licensed, there are no LICENSE files that I could find (at least in the main branch) and the links in the README are broken. This makes me a bit wary of using it in my projects and might discourage others from contributing.

Action Failure - Inputs for exiting non-zero

Is your feature request related to a problem? Please describe.
This action seems to always exit non-zero, which leads to false positives. The Github action frame-work already has the continue-on-error: true attribute which can be set for each step in a job. Additionally, if the action didn't do what it should do, which in this case is sync files, it should always return non-zero when it doesn't.

This sample output didn't actually make any changes.

...which for my implementation is good. In this case I want the workflow to repo 'success', but for other conditions I may want this to report as a 'failure':
image

Here's an example where the copy failed, but this action reports a success (???):

If I ask the action to copy files and they don't exist, this is a failure in my opinion. On the other side, If there are files but this action fails to perform the copy it should still report as a failure.
image

Here's an example where this action failed to create a branch (due to a URL unfriendly name), but instead exited to a success status:

image

Describe the solution you'd like
The approach which I'd recommend would be to have the additional inputs be verified for 'real' success and if this action is unable to perform them completely and successfully, then fast-fail. Example:

with:
  ..
  ...
  PULL_REQUEST: true # if for ANY reason this is unsuccessful, the action exits non-zero because I've asked this action to create a pull-request.
  AUTO_CREATE_NEW_BRANCH: true # if for ANY reason no branch was created or exists with the provided name, then exit non-zero
  WORKFLOW_FILES: |
    file1 # if this file exists in the source repo and doesn't after the commit step, fail the action
    folder2=folder1 # if content fails to copy, fail the action
  GITHUB_TOKEN: 90234560346523458234example # if unable to clone the source repo or push to the destination repo, again..fail this action.

If there is an issue with the release version, since this would probably be a breaking-change, then create a new input, but default it to true. When ready to release the need major version such as v4.0, then just make this default false.

with:
  ..
  ...
  INGNORE_FAILURES: true # this is defaulted to true, but can be set to `false` to get the behavior described above.

Describe alternatives you've considered
There are many ways to implement this, even not exiting non-zero, such as outputting the 'end-state' or 'exit status'. Here's an example:
*exit-status: "unknown_error"
*exit-status: "clone_failure"
*exit-status: "copy_failed"
*exit-status: "invalid_token"
*exit-status: "pull-request_failed"
*exit-status: "ect,ect."

With these statuses, a Final status for the github workflow/job can be determined in a subsequent step as to whether the action really did what the implementation needs.
Additional context
Add any other context or screenshots about the feature request here.

Contents is not getting updated once the files are synced and pushed to target repository

@varunsridharan Once the file is synced and updated in the target repository and when the changes are made in the script and updated and when the changes are pushed to the target repository it's not getting updated.

For example:
abc.sh in beginning
echo "hi"
abc.sh in target repo
echo "hi"

abc.sh after updating
echo "hello"
abc.sh in target repo after updating
echo "hi"

workflow file used

on:
  push:
    branches:
      - "main"
env:
  DRY_RUN: false
  REPOSITORIES: |
    QE/yarn-qe@github-actions-poc
  WORKFLOW_FILES: |
    yarn-qe/approval.yaml=.github/workflows/pr-approval.yaml
    yarn-qe/qe-check.yaml=.github/workflows/qe-check.yaml
    yarn-qe/qe-ci-gate-qe-gate.yaml=.github/workflows/qe-ci-gate-qe-gate.yaml
    .github/qaas-jobs/static-analysis.sh=.github/static-analysis.sh
    .github/qaas-jobs/commit_message_check.sh=.github/commit_message_check.sh
    .github/qaas-jobs/docker-build-system_test.sh=.github/docker-build-system_test.sh
    .github/qaas-jobs/docker-build-temp-image-system_test.sh=.github/docker-build-temp-image-system_test.sh
jobs:
  Github_Workflow_Sync:
    runs-on: [re-centos7]
    steps:
      - name: Fetching Local Repository
        uses: actions/checkout@v2
      - name: Running Workflow Sync
        uses: Kitchen/RE-thirdparty-actions/actions/workflow-sync@main
        with:
          DRY_RUN: ${{ env.DRY_RUN }}
          REPOSITORIES: ${{ env.REPOSITORIES }}
          WORKFLOW_FILES: ${{ env.WORKFLOW_FILES }}
          GITHUB_TOKEN: ${{ secrets.RE_JENKINS_GH_USER_TOKEN }}
          PULL_REQUEST: true
          GIT_URL: ***
          COMMIT_MESSAGE: "Sync workflow files | Triggered by Kitchen/RE-github-workflows"```

Automatic pull requests to default branch of touched repository

Is your feature request related to a problem? Please describe.
Some of the repos we would like to keep in sync customize the workflows to a small extent.
It would be nice to keep the customizability but have a common basis that is up to date.

Describe the solution you'd like
It would be cool if the sync could open PRs upon pushing the workflow files to a specified branch

Describe alternatives you've considered
Currently, we bypass the problem by having a workflow inside the target repo that creates the PR upon pushes to the "workflow-sync" branch

Does that make sense to you?

master branch EOL

Due to changes into the branches management, the master branch is not longer supported and will be removed in 1 month (EOL: 2020-11-29).

After a search, here is the list of projects who use master branch for this action inside their workflows.

Add a SKIP_CI parameter to modify the commit message and avoid triggering automation

Is your feature request related to a problem? Please describe.
I'd like to update a repository without kicking off that repo's build automation.

Describe the solution you'd like
Since Github Actions now supports adding [skip ci] to the commit message (https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/), I'd like to be able to add it to the commit message
by setting a parameter/input.
It could be something as simple as SKIP_CI: true
This would result in a commit message something like "Files Synced | [skp ci] | Runner ID : ! | โšก Triggered By Name/Repo"
Without that option, updating a config file could result in 100s of deployments kicking off at once.

Describe alternatives you've considered
A more flexible solution might be to edit part or the whole commit message by inputing a string.

Additional context
Might be useful to know that several CI engines support [skip ci]

https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/
https://circleci.com/docs/2.0/skip-build/
https://docs.travis-ci.com/user/customizing-the-build/#skipping-a-build

Also, really love your work here.
Great solution to a problem I've been trying to solve for far too long ๐Ÿ‘

[Feature request] Detect if target repo already has an open PR and append change, instead of opening new PR.

Is your feature request related to a problem? Please describe.
I use this action to sync files to a few repos by opening a PR, some of which are not maintained that often. For those repos every time a sync is started a new PR is opened while old ones are still open.

Describe the solution you'd like
Is it possible that this actions detects that there still is an open PR and just adds a new commit to that branch? (Maybe this needs to be configurable)

Describe alternatives you've considered
Just accept it and close 'old' PR's manually.

Add label to PR created by bot

Would be better if I can setup labels for PR created by bot

name: Workflow Sync

on:
  push:
    branches:
      - master
env:
  DRY_RUN: false
  CUSTOM_LABEL: workflow_sync_bot
  REPOSITORIES: |
  
  WORKFLOW_FILES: |

jobs:
  Github_Workflow_Sync:
    runs-on: ubuntu-latest
    steps:
      - name: Fetching Local Repository
        uses: actions/checkout@master
      - name: Running Workflow Sync
        uses: varunsridharan/action-github-workflow-sync@main
        with:
          DRY_RUN: ${{ env.DRY_RUN }}
          REPOSITORIES: ${{ env.REPOSITORIES }}
          WORKFLOW_FILES: ${{ env.WORKFLOW_FILES }}
          GITHUB_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
          CUSTOM_LABEL: ${{ env.CUSTOM_LABEL }}

add support for $default-branch placeholder expansion (to copy workflows from <ORG>/.github/workflow-templates)

Is your feature request related to a problem? Please describe.

I started my journey towards workflow sharing with starter workflows at https://docs.github.com/en/actions/using-workflows/creating-starter-workflows-for-your-organization, by creating files in the org .github repo's workflow-templates sub-directorywith the $default-branch placeholder.

Then later, I realized I can't use workflow-templates unless I switch to GitHub Enterprise Cloud. Which I'm not ready to do (yet).
And I only have private repos in my organization.

Then I found this project, and figured this would solve my immediate problem of copying workflow files to target repositories whenever a change is pushed.

Problem: The target repositories use a mix of master and main as default branch. And I cannot change that yet.

Describe the solution you'd like
Have action-github-workflow-sync replace $default-branch with the default branch name of the repository that the workflow file is being copied to. This way, I keep workflow-templates which is starter workflow friendly for the day I switch to Enterprise Cloud (which could be never), and yet the sync creates a workflow file that contains the correct default branch name on my behalf.

There is then only one source of truth for the workflows...

Describe alternatives you've considered
I've considered reusable workflows, but it is not possible to call a reusable workflow in a private repository from a workflow in another private repository.

Additional context
Organizations dealing only with private repositories are not well served by default when it comes to keeping workflows DRY. Making this change could help improve the situation.

regex support feature for adding branches

This is a nice feature, it would be beneficial for us if we could get regex support for adding branch names under the REPOSITORIES: section
Hoping to see this feature very soon!!!
Thanks!!

Expose the git_url as GIT_URL so that Github Enterprise instances can utilize this.

Is your feature request related to a problem? Please describe.
Currently the sync action only looks in github.com/{repository}, which does not work with Github Enterprise Servers.

Describe the solution you'd like
When assigning the git_url variable, use a new GIT_URL variable which would be defaulted to github.com but be over writable by users with GIT_URL

Describe alternatives you've considered
None

Additional context
Add any other context or screenshots about the feature request here.

Node.js 16 support

Is your feature request related to a problem? Please describe.

When using the action, the following warning shows up on the workflow run summary page under "Annotations":

Node.js 12 actions are deprecated. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/. Please update the following actions to use Node.js 16: varunsridharan/action-github-workflow-sync

Describe the solution you'd like
Please migrate to using node.js 16

Action fails - git configuration is incomplete

Describe the bug
Hi, thanks for providing this useful action.
Unfortunately it fails for us, because somehow the git user is not correctly set inside the action itself.

We are running the action inside our organization, the repos are not public.
Would you have an idea how to fix this?

Here is the log from the action

" "update-prod.yml
" "***" "false" "workflows" "false" "false"

๐Ÿ—ƒ Git Config
    > Name          : Github Actions Workflow Sync Bot
    > Email         : [email protected]

 
โš™๏ธ Basic Config
-------------------------------------------------------
  * DRY_RUN               : false
  * Total Repositories    : 1
  * Total Workflow Files  : 1
--------------------------------------------------------
 
๐Ÿ““  celonis/workflow-sync-test
  โš™๏ธ Repository Config
    Slug          : celonis/workflow-sync-test
    Branch        : default
    Url           : ***github.com/celonis/workflow-sync-test.git
    Local Path    : /workflow-sync/celonis/workflow-sync-test/default
  
  โœ”๏ธ Repository Cloned
  
  update-test.yml
    update-test.yml => update-test.yml
  
  Git Commit & Push Log
  ---------------------------------------------------
  
  *** Please tell me who you are.
  
  Run
  
    git config --global user.email "[email protected]"
    git config --global user.name "Your Name"
  
  to set your account's default identity.
  Omit --global to set the identity only in this repository.
  
  fatal: unable to auto-detect email address (got 'root@6a55273d2ba1.(none)')

Additional context
This seems to happen in the ghconfig function.
I couldn't find the source of /gh-toolkit/shell.sh where the function for setting up git is most likely defined.

Intermittent 'Repository Dose Not Exists !' error whilst trying to sync

Describe the bug
Intermittent 'Repository Dose Not Exists !' error whilst trying to sync

To Reproduce
Steps to reproduce the behavior:

  1. Run workflow for multiple repositories

Expected behavior
Workflows should sync across all repositories

Additional context
We have multiple repositories being used for syncing and we are constantly running into the Repository Dose Not Exists ! error. It does not happen all the time but intermittent

Ability to have custom destination paths

Would like to have the ability to have custom destination paths. For example:

WORKFLOW_FILES: |
  ./sync/dependabot.yml=./.github/dependabot.yml

I don't want the destination to be .github/workflows/dependabot.yml because then github thinks it's an action rather than a dependabot file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.