Git Product home page Git Product logo

tagbot's Introduction

Julia TagBot

TagBot creates tags, releases, and changelogs for your Julia packages when they're registered.

When we talk about tags and releases, we mean Git tags and GitHub releases, and not releases in a registry that allow the Julia package manager to install your package. TagBot does not register your package for you, see the documentation in General and Registrator for that. Instead, it reacts to versions of your packages that have been registered, making TagBot a set-and-forget solution to keep your repository in sync with your package releases. Tags and releases aren't at all necessary, but it's considered a good practice.

Other benefits of using TagBot include the ability for you and your users to browse package code at specific releases, and automatically-generated changelogs for each release that keep your users in the loop.

Setup

Create a file at .github/workflows/TagBot.yml with the following contents:

name: TagBot
on:
  issue_comment:
    types:
      - created
  workflow_dispatch:
    inputs:
      lookback:
        default: "3"
permissions:
  actions: read
  checks: read
  contents: write
  deployments: read
  issues: read
  discussions: read
  packages: read
  pages: read
  pull-requests: read
  repository-projects: read
  security-events: read
  statuses: read
jobs:
  TagBot:
    if: github.event_name == 'workflow_dispatch' || github.actor == 'JuliaTagBot'
    runs-on: ubuntu-latest
    steps:
      - uses: JuliaRegistries/TagBot@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          # Edit the following line to reflect the actual name of the GitHub Secret containing your private key
          ssh: ${{ secrets.DOCUMENTER_KEY }}
          # ssh: ${{ secrets.NAME_OF_MY_SSH_PRIVATE_KEY_SECRET }}

Check that workflows have the correct permissions on your repo.

Workflow permissions

Note

No further action should be required on your part, but if TagBot fails and you see 403: Resource not accessible by integration errors, try adding (or refreshing) an SSH key with the correct permissions. See the SSH Deploy Keys section below.

When you add a new release to a registry with Registrator, TagBot will create a GitHub release on your package's repository.

You may, however, want to customize the behaviour via the available configuration options.

For example, if you use GitHub Actions to build the documentation for your package, you will find that with the default TagBot configuration, your documentation build is not triggered when a new tag is created. In this case, you will need to use SSH Deploy Keys.

Read on for a full description of all of the available configuration options.

Table of Contents

Basic Configuration Options

SSH Deploy Keys

Sometimes, instead of using the default secrets.GITHUB_TOKEN, it is better to use an SSH deploy key. The most notable reason is that the default token does not have permission to trigger events for other GitHub Actions, such as documentation builds for new tags.

To use an SSH deploy key:

  • Create an SSH key and add it to your repository by following the instructions here. Make sure to give it write permissions.
  • Create a repository secret by following the instructions here. Use whatever you like as the name, such as SSH_KEY. Use the private key contents as the value.
  • Add the ssh input:
with:
  token: ${{ secrets.GITHUB_TOKEN }}
  ssh: ${{ secrets.SSH_KEY }}

If you already have a Base64-encoded deploy key and matching repository secret for Documenter, you can reuse it instead of creating a new one.

If your key is password-protected, you'll also need to include the password in another repository secret (not Base64-encoded):

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  ssh: ${{ secrets.SSH_KEY }}
  ssh_password: ${{ secrets.SSH_PASSWORD }}

Changelogs

TagBot creates a changelog for each release based on the issues that have been closed and the pull requests that have been merged. Additionally, you can write custom release notes in the same place that you register your packages. See Registrator or PkgDev for specifics.

The changelog is completely customizable with the Jinja templating engine. To supply your own template, use the changelog input:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  changelog: |
    This is release {{ version }} of {{ package }}.
    {% if custom %}
    Here are my release notes!
    {{ custom }}
    {% endif %}

The data available to you looks like this:

{
  "compare_url": "https://github.com/Owner/Repo/compare/previous_version...current_version (or null for first release)",
  "custom": "your custom release notes",
  "issues": [
    {
      "author": {
        "name": "Real Name",
        "url": "https://github.com/username",
        "username": "their login"
      },
      "body": "issue body",
      "labels": ["label1", "label2"],
      "closer": {"same format as": "issue author but sometimes null"},
      "number": 123,
      "title": "issue title",
      "url": "https://github.com/Owner/Repo/issues/123"
    }
  ],
  "package": "PackageName",
  "previous_release": "v1.1.2 (or null for first release)",
  "pulls": [
    {
      "author": {"same format as": "issue author"},
      "body": "pull request body",
      "labels": ["label1", "label2"],
      "merger": {"same format as": "issue author"},
      "number": 123,
      "title": "pull request title",
      "url": "https://github.com/Owner/Repo/pull/123"
    }
  ],
  "sha": "commit SHA",
  "version": "v1.2.3",
  "version_url": "https://github.com/Owner/Repo/tree/v1.2.3"
}

You can see the default template in action.yml.

Issues and pull requests with specified labels are not included in the changelog data. By default, the following labels are ignored:

  • changelog skip
  • duplicate
  • exclude from changelog
  • invalid
  • no changelog
  • question
  • wont fix

To supply your own labels, use the changelog_ignore input:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  changelog_ignore: ignore this label, ignore this label too

White-space, case, dashes, and underscores are ignored when comparing labels.

Custom Registries

If you're using a custom registry, add the registry input:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  registry: MyOrg/MyRegistry

If your registry is public, this is all you need to do. If your registry is private, you'll need to configure access to it via one of two options.

The first option is to change the token input to a PAT that has access to both your package repository and the registry. Take a look at the warnings about PATs if you choose this option.

The other option is to use the registry_ssh input, like so:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  registry: MyOrg/MyRegistry
  registry_ssh: ${{ secrets.REGISTRY_SSH_KEY }}

Here, the REGISTRY_SSH_KEY is the private key portion of a read-only SSH deploy key for your registry. By using this method, the only extra access that TagBot receives is read access to your private registry, not to any other repositories in your user/organization account.

Advanced Configuration Options

Self-Hosted GitHub

If your company is running their own GitHub server, then you'll need to update the relevant GitHub URL inputs, github and github_api:

with:
  github: git.corp.com
  github_api: api.git.corp.com

Git Configuration

By default, the Git tags that TagBot creates are authored by a user called github-actions[bot], whose email is 41898282+github-actions[bot]@users.noreply.github.com. If you want to use your own Git author info, you can set the user and email inputs:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  user: My Name
  email: [email protected]

Draft Releases

If you'd prefer to create a draft GitHub release instead of a full Git tag + GitHub release, use the draft input:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  draft: true

GPG Signing

If you want to create signed tags, you can supply your own GPG private key. Your key can be exported with gpg --export-secret-keys --armor <ID>, and optionally Base64-encoded. Create the repository secret, then use the gpg input:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  gpg: ${{ secrets.GPG_KEY }}

If your key is password-protected, you'll also need to include the password in another repository secret (not Base64-encoded):

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  gpg: ${{ secrets.GPG_KEY }}
  gpg_password: ${{ secrets.GPG_PASSWORD }}

It's also recommended to set your Git email address to one that is attached to the GPG key (see Git Configuration). If you fail to do so, your tags will be marked "Unverified" in the GitHub UI.

Lookback Period

By default, TagBot checks for new releases that are at most 3 days old. Therefore, if you only run TagBot every five days, it might miss some releases. To fix this, you can specify a custom number of days to look back in time with the lookback input:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  lookback: 14

An extra hour is always added, so if you run TagBot every 5 days, you can safely set this input to 5.

Personal Access Tokens (PATs)

We highly recommend that you use an SSH deploy key instead of a personal access token (PAT).

Please keep in mind that there are security concerns with using a PAT. For example, a PAT has access to all of your repositories. To reduce the consequences of a secret being leaked, we recommend that you instead use an SSH deploy key that only has permissions for a single repository.

To use a PAT:

  • Create a PAT by following the instructions here. Make sure that it has the repo scope.
  • Create a repository secret by following the instructions here. Use whatever you like as the name, such as TAGBOT_PAT. Use the new PAT as the value.
  • Replace the token input's value with the name of your secret, like so:
with:
  token: ${{ secrets.TAGBOT_PAT }}

Pre-Release Hooks

If you want to make something happen just before releases are created, you can do so with the dispatch input:

with:
  token: ${{ secrets.TAGBOT_PAT }}
  dispatch: true

When you enable this option, a repository dispatch event is created before releases are created. This means that you can set up your own actions that perform any necessary pre-release tasks.

The payload is an object mapping from version to commit SHA, which can contain multiple entries and looks like this:

{
  "v1.2.3": "abcdef0123456789abcdef0123456789abcdef01"
}

These actions will have 5 minutes to run by default, but you can customize the number of minutes with the dispatch_delay input:

with:
  token: ${{ secrets.TAGBOT_PAT }}
  dispatch: true
  dispatch_delay: 30

Avoid setting a delay longer than the interval between TagBot runs, since your dispatch event will probably be triggered multiple times and the same release will also be attempted more than once.

To use the dispatch feature, you must provide your own personal access token. For more details, see Personal Access Tokens (PATs).

Release Branch Selection

If you use a non-standard Git workflow where your default branch is not the main development branch, you may want to set the branch input to the name of your preferred release branch:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  branch: release

The branch you set will be the first one searched for a commit to tag, and releases will be anchored to that branch when possible.

Release Branch Management

If you're using PkgDev to release your packages, TagBot can manage the merging and deletion of the release branches that it creates. To enable this feature, use the branches input:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  branches: true

Subpackage Configuration

If your package is not at the top-level of your repository, you should set the subdir input:

with:
  token: ${{ secrets.GITHUB_TOKEN }}
  subdir: path/to/SubpackageName.jl

Version tags will then be prefixed with the subpackage's name: {PACKAGE}-v{VERSION}, e.g., SubpackageName-v0.2.3. (For top-level packages, the default tag is simply v{VERSION}.)

Note: Using TagBot with a non-empty subdir will only work for Julia package versions registered using the official Registrator (see also #281 and #282).

To tag releases from a monorepo containing multiple subpackages and an optional top-level package, set up a separate step for each package you want to tag. For example, to tag all three packages in the following repository,

.
├── SubpackageA.jl
│   ├── Package.toml
│   └── src/...
├── path
│   └── to
│       └── SubpackageB.jl
│           ├── Package.toml
│           └── src/...
├── Package.toml
└── src/...

the action configuration should look something like

    steps:
      - name: Tag top-level package
        uses: JuliaRegistries/TagBot@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          # Edit the following line to reflect the actual name of the GitHub Secret containing your private key
          ssh: ${{ secrets.DOCUMENTER_KEY }}
          # ssh: ${{ secrets.NAME_OF_MY_SSH_PRIVATE_KEY_SECRET }}
      - name: Tag subpackage A
        uses: JuliaRegistries/TagBot@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          # Edit the following line to reflect the actual name of the GitHub Secret containing your private key
          ssh: ${{ secrets.DOCUMENTER_KEY }}
          # ssh: ${{ secrets.NAME_OF_MY_SSH_PRIVATE_KEY_SECRET }}
          subdir: SubpackageA.jl
      - name: Tag subpackage B
        uses: JuliaRegistries/TagBot@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          # Edit the following line to reflect the actual name of the GitHub Secret containing your private key
          ssh: ${{ secrets.DOCUMENTER_KEY }}
          # ssh: ${{ secrets.NAME_OF_MY_SSH_PRIVATE_KEY_SECRET }}
          subdir: path/to/SubpackageB.jl

Generated tags will then be v0.1.2 (top-level), SubpackageA-v0.0.3, and SubpackageB-v2.3.1. As an alternative to the automatic tag prefixing, you can manually specify a different tag prefix as an input:

    steps:
      - name: Tag subpackage A
        uses: JuliaRegistries/TagBot@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          # Edit the following line to reflect the actual name of the GitHub Secret containing your private key
          ssh: ${{ secrets.DOCUMENTER_KEY }}
          # ssh: ${{ secrets.NAME_OF_MY_SSH_PRIVATE_KEY_SECRET }}
          subdir: SubpackageA.jl
          tag_prefix: MyOwnTagPrefix

In this case, the tag for SubpackageA.jl will be MyOwnTagPrefix-v0.0.3.

If you want to disable tag prefixes for subdirectory packages altogether, you can set the tag_prefix to NO_PREFIX. Note that this is only recommended if you only have a single Julia package in the repository.

ℹ️ Monorepo-specific changelog behavior

Each subpackage will include all issues and pull requests in its changelogs, such that a single issue will be duplicated up in all of the repository's subpackages' release notes. Careful changelog_ignore and/or custom changelog settings on a per-subpackage basis can mitigate this duplication.

Local Usage

There are some scenarios in which you want to manually run TagBot. Perhaps TagBot failed for some reason, or GitHub's service was down, or you just set up TagBot but would like to fill in tags for old releases. The simplest way to run TagBot manually is through Docker and the tagbot.local module.

$ docker run --rm ghcr.io/juliaregistries/tagbot python -m tagbot.local --help
Usage: __main__.py [OPTIONS]

Options:
  --repo TEXT        Repo to tag
  --version TEXT     Version to tag
  --token TEXT       GitHub API token
  --github TEXT      GitHub URL
  --github-api TEXT  GitHub API URL
  --changelog TEXT   Changelog template
  --registry TEXT    Registry to search
  --subdir TEXT      Subdirectory path in repo
  --tag-prefix TEXT  Prefix for version tag
  --help             Show this message and exit.

$ docker run --rm ghcr.io/juliaregistries/tagbot python -m tagbot.local \
    --repo Owner/Name \
    --token <TOKEN> \
    --version v1.2.3

Only the repo, version, and token options are required, and you will be prompted if you don't provide them. For instructions on how to obtain a token, see Personal Access Tokens (PATs).

You can also run the code outside of Docker, but you'll just need to install Poetry first, and ensure that you have Python 3.8.

$ git clone https://github.com/JuliaRegistries/TagBot  # Consider --branch vA.B.C
$ cd TagBot
$ poetry install
$ poetry run python -m tagbot.local --help

Troubleshooting tips

I am seeing some kind of permissions error

  • Check that your configuration matches the one shown in Setup, especially the permissions block
  • Try using an ssh deploy key even if you aren't using Documenter or otherwise need to trigger workflows from TagBot-generated tags

I am missing old tags

If you have missed tags due to now-fixed errors, you can manually trigger TagBot with a longer "lookback" period (days) in order to try to find them (assuming your workflow has been configured as shown in Setup with a workflow_dispatch trigger). See the Github docs for more on manually running workflows.

tagbot's People

Contributors

alecloudenback avatar christopher-dg avatar cormullion avatar dependabot[bot] avatar dilumaluthge avatar earlmilktea avatar ericphanson avatar fonsp avatar fredrikekre avatar github-actions[bot] avatar hannahilea avatar ianbutterworth avatar oxinabox avatar racinmat avatar rfourquet avatar sloede avatar thazhemadam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tagbot's Issues

Don't create annotated tags

I've kind of wanted to get rid of annotated tags for a while because it makes things a lot simpler (we get to skip the entire tag creation step), but I think this is an actual good reason:

JuliaRegistries/General#5999 (comment)

Any time that a release contains modified GitHub Actions files, I'm pretty sure that the tag will fail. Of course, we could just ignore the failure if we wanted.

The only benefit of annotated tags is the option to supply a GPG key, but I anticipate that to be a seldom used feature. It was nice to have when it was easy to centrally set up, but I don't think we really get much out of it seeing as the tags aren't actually used for anything important and exist only as a convenience now.

Are labels like `no changelog` supported in GitHub Action?

Old TagBot had this list of issue/PR labels for excluding items from the changelog:

When using the automatic changelog, you can ensure that certain issues or pull requests are not included.
These might include usage questions or typo fixes that aren't worth mentioning.
To exclude an issue or PR, add a label to it with one of the following values:

  • changelog skip
  • duplicate
  • exclude from changelog
  • invalid
  • no changelog
  • question
  • wont fix

You can spell these in a few different ways.
For example, no changelog could be nochangelog, no-changelog, no_changelog, No Changelog, NoChangelog, No-Changelog, or No_Changelog.

However, IIUC, it looks like the latest TagBot supports only changelog-skip:

{% if "changelog-skip" not in issue.labels %}

Can you add back all the previously supported labels?

Issue with custom registry

I have a custom registry (TrebstRegistry) and am trying to enable TagBot for a package registered therein (TestPackage2.jl).

My TagBot.yml workflow looks like this:

name: TagBot
on:
  schedule:
    - cron: 0 * * * *
jobs:
  TagBot:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - uses: JuliaRegistries/TagBot@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          registry: https://github.com/crstnbr/TrebstRegistry

Note the extra registry entry which I added as explained in the TagBot readme.

However, when this workflow runs, I get an error:

fatal: empty string is not a valid pathspec. please use . instead if you meant to match all paths

Check yourself here: https://github.com/crstnbr/TestPackage2.jl/commit/c3a932d42f94c3a592ee2e9c1b0f7d7c15c4bb17/checks?check_suite_id=303890250

I don't know how to debug this. Is this a TagBot issue, one with my registry, or just me wrongly setting up things?

Any help is very much appreciated!
Carsten

Squashed PRs do not show up in release notes

Only PRs which have been merged will show up in the release notes. It would be good to also include squashed or rebased PRs which should be possible to detect by looking for r"\(#\d+\)$" in the commit message.

Error handling

Today was an unfortunate day.

Moral of the story: Users should not have to care about TagBot failures, that's my job.

What to do:

  • Wrap everything in a try and stop changing the exit code on errors (always exit 0)
  • On unexpected errors, automatically open an issue on this repo for me to deal with.

Inputs with dashes aren't passed into the container

GitHub Discourse thread here.

In case you're wondering why your GPG key isn't being picked up, or your self-hosted GitHub settings aren't working.

I could change the names to have underscores, but I do want it to be consistent with the rest of the GA syntax (kebab-case).

How do I use TagBot?

I see that TagBot "Creates tags, releases, and changelogs for your Julia packages". I also see the detailed instructions for setting up TagBot. But how do I use it – how do I actually create a release? That information seems to be missing from the readme.

Does it depend on tags (Github "releases") I put into the repo (probably not)? Or does it look at the version number I set via Pkg (probably)? Does it wait for all tests to pass before it creates a release? What if I tag or increase the version number on a branch – will it wait until the branch has been merged into master? What events should I expect to happen, with what kind of typical delay? What do I do if things don't work, how do I trigger TagBot to have another look? Can TagBot also archive the repo somewhere, e.g. on Zenodo?

TagBot workflow does not trigger CI, preventing docs for tags

This is probably known, but I didn't see an issue. The problem, as @SaschaMann pointed me to, is that it's hard to get workflows to trigger other workflows. So when using Github Actions for CI and for tagbot, even adding an on: release trigger to the CI workflow does not cause the tagbot workflow's tag to cause CI to run (I know this because I tried just now with the repo https://github.com/ericphanson/UnbalancedOptimalTransport.jl). Hence, the docs don't get built for the tag, and one doesn't get release docs (only dev docs).

I'll switch back to the bot version of tagbot instead of the workflow (which also has the advantage of being instant instead of waiting for a tag). But maybe this issue is something to consider re #39.

Show PRs first in release notes

Currently the automatically generated release notes show the list of commits before the list of merged PRs. It would be nice to see the list of PRs first as this is typically what humans will be looking for.

Migrate to GitHub Actions

I've just spent the evening playing around with GitHub Actions and I think it could make things a lot nicer. It's been discussed a bit on Slack too.

Early idea stage, this is an ugly wall of text.

### Step 1 (Registrator)
Registrator comments on a commit in the package repo when it merges a PR into General. This means no more awkward listening for events in a separate repo.
The comment contains at least the version and commit sha to be tagged.
The commit that gets the comment doesn't really matter as long as the comment itself contains enough information.
I guess it can decide whether or not to do this by looking for a GitHub workflow file in the target repo.
Even just this step would make TagBot a lot less janky.

### Step 2 (parse + validate)
At this point we're in a workflow in the package repo that responds to commit comments.
Parse and validate the comment etc., then pass along the commit sha + tag name to be created.
This is code that would belong in this repo, since it's Julia-specific.

Step 1 (Registrator)

Registrator sends a repo dispatch event to the repository whose package was just merged.
The payload is JSON, and the event name identifies it as a registration event, so no validation or special parsing is required.

Grrrrrr, this strategy requires the sender of that dispatch event (aka Registrator or something associated with General) to have repo write permissions, so we're not doing that. Back to the old plan.
I also don't think GitHub Actions has a trigger for commit comments, so a workaround would be to create and immediately close an issue, and always comment on that issue. Feature request for on: commit_comment here.

Step 2 (make tag)

Super easy if you disregard GPG signing, I'm not quite sure how that would work now, but I'm sure it's doable. This can be something generic, nothing is Julia-specific. It just needs options to set the tag name and commit to be tagged (since the commit that was commented on is not necessarily the one that we should tag).

Step 3 (create changelog)

Run the changelog generator, and extract out the section for the newly created tag. I've started looking into this here, since it's also not Julia-specific at all.
This job could just dump the results to a file rather than creating the release, and pass along the filename.

Step 4 (create release)

Easy, and not specific to Julia at all. Just need to be able to configure the release body, name, and sha.


While most of the pieces can be useful to anyone, I'm not sure if we'll find pre-made stuff that fits perfectly since it needs to be configurable in just the right ways.

It would kind of be a shame if #29 went to waste, because I'm quite proud of that code tbh. At least some of it can be reused, even if it's in other repos. But if it amounts to a nicer system for the community, it's worth it.

Oh and... we could write this code in Julia 🙃


The above sketch outlines a modular system, but as an initial draft I think I'm just going to combine everything into the same action.

Easier manual usage

If I break something and TagBot goes stops working for a day or whatever, people might miss releases and want to trigger them manually but still get the changelog, ensure that the tag is on the right commit, etc.
It would be cool to be able to easily run TagBot locally without having to clone the TagBot repo or know an arcane docker run -e ... incantation. It might be fun to spin up a web service where you can authenticate with GitHub and run TagBot on a repo of your choosing. 🙃.

Changelog memory bump + error reporting

A changelog function call failed with a cryptic error, and the max memory usage is reported as 1024MB (the max memory allocation for the function), so I'm assuming it ran out of memory (on DiffEqBase).

/var/runtime/bootstrap: line 23: 8 Killed /var/runtime/lib/runtime.rb

Presumably Amazon kills containers that try to exceed their memory allocation.

One fix would be to simply max out the memory allocation for the changelog function, but I'd rather not do that since that amount of memory isn't needed most of the time.

A better fix would be to retry with max memory only if a message ends up in the dead letter queue.

Also, if that retry fails then it should a) delete the WIP message from the release and b) reply to the registry PR indicating it failed to generate a changelog.

Provide optional release branch merging option

Would you be open to add another optional feature that does the following: whenever it finds a new release and tags it, it also looks for a branch with the name release-1.2.3, and if that exists try to merges that into master, but only if it can be done with a fast-forward merge. If that succeeded, it also deletes the branch. If the ff merge didn't work, it opens a PR to merge that branch into master. The whole thing could be opt-in via a config option for the action.

This is the backstory for this feature: I have a rewrite of PkgDev.jl that provides a tag function. When you call PkgDev.tag("MyPkg"), the following things happen:

  1. it figures out what the next release version shall be (you can also specify that in the call to tag)
  2. it will create a new branch release-1.2.3 for that version
  3. it sets the version field in Project.toml to that version and commits
  4. it modifieds the version field in Project.tomlto1.2.4-DEV` and commits
  5. it pushes that branch to origin
  6. it then uses Registrator.jl to open a PR against the registry where this package is registered (with support for public and private registries) that registers the new version

At that point it stops.

If TagBot would do the additional things that I mentioned above, then we would have a complete, super easy package tagging story, IMO. Essentially as a pkg dev I would call Pkg.tag("MyPackage"), tne then the bots would do everything else, from modifying the Project.toml, to opening PRs that trigger various checks.

I'd be happy to help write the code, just wanted to check whether this might be acceptable in this repo first :)

Integration tests for Git + filesystem + Docker

Right now pretty much everything GitHub/Git/FS/-related is mocked out during tests. I'm not really interested in doing VCR testing for the GitHub API, but I would definitely like to run some tests that make sure that our Git commands and other filesystem manipulations are actually doing the things we want them to.
Also, I want to be able to run the tests in a freshly built Docker image too.

App is installed for user but the repository is not enabled

Hello!

I tried to move some of the repos I work on to use the Github workflows which I think is the recommended way now but I keep getting errors like: JuliaRegistries/General#9002 (comment)

However on the README it says specifically that the only thing to do is to add the TagBot.yml file which I have done: https://github.com/alan-turing-institute/MLJModelInterface.jl/blob/master/.github/workflows/TagBot.yml

Is there an additional step that must be done that would not be documented?

Thanks!

Rewrite TagBot in Julia (long-term goal)

Currently, TagBot is written in Python.

It would be great to eventually rewrite TagBot in pure Julia, for the following reasons:

  • We might as well "eat our own dog food" when developing our own tools.
  • It is always great to showcase the use of Julia as a general purpose programming language (as opposed to just a scientific computing language).

I don't have the time right to do this, and I'm guessing that @christopher-dG doesn't either. But I figure we leave this issue open (maybe label it as Help Wanted), and eventually maybe someone will do it.

Feature request: GPG-sign the tag that TagBot creates

Currently, I manually create tags on the command line, which allows me to sign the tags I create with my personal GPG key.

It would be great if TagBot had the ability to GPG-sign the tags that it creates (using its own GPG key, presumably).

GitHub App deprecation plan

ref: #38

So here's what I'm thinking:

We'll announce the deprecation on Discourse and tell everyone to move to GitHub Actions. I'm not sure when this will be, I need to improve the new version and add tests and stuff first, I just haven't had much time to work on it.
The deprecation period can be a few months, there's no real rush. During this period, the GitHub App should:

  • Completely ignore repos that already have TagBot set up as a GitHub Action
  • Create pull requests to enable TagBot in all other enabled repos that are Julia packages (this can happen in bulk right at the beginning)
  • When writing the final release notification comment, add a deprecation notice with reference to the Discourse post and the PR that was previously created.

After the deprecation period, we can either shut everything off, or keep it running but with no functionality except for a "Move to GitHub Actions now" message.

SSH/GPG keys with passwords

Currently there's nothing to input a password to GPG, and this is blocking me from using my own GPG key. I've tried a few things but I need to do more digging.

Feature request: tag message & release notes

With GitHub releases, I often added a description of the release using the web interface.
For example, this release.

With JuliaRegistrator+TagBot, it looks like the release notes are copied from the tag message (specified with -m if created manually), as can be seen here (manually created tag with a re-triggered release via TagBot). But there's no way to specify that message, and manually creating the tag + retriggering TagBot circumvents most of the bot's functionality.

It would be convenient if TagBot would look at the comment/issue that invokes JuliaRegistrator to pick up on a message, and use that for the tag message and/or the release notes (having a tag message makes it useful for git users too, and not only GitHub users).

Some possibly syntaxes:

@JuliaRegistrator register()

<message>
@JuliaRegistrator register()

Release notes:

```
<message>
```
@JuliaRegistrator register()

@TagBot message

```
<message>
```

Find commits on non master branches

When I tag a new release, I typically create a branch called release-1.2.3, then update the version field in Project.toml. with a commit on that branch, push that branch, trigger registrator on that commit on that branch.

So when things are merged in the registry, the commit that corresponds to the tag will not be on master yet. It seems that tagbot then doesn't find the commit? Is it only looking in the history of master for commits that might correspond to a new tag?

The process I described above is also what PkgDev.tag does.

I could see two solutions: 1) just check commits in all branches, or 2) also look for commits in branches that are named release-1.2.3 specifically.

Suggested logo tweak

Brilliant job all round on this bot, it works great! I just have one suggested very small tweak— the logo is a bit dark and low contrast. How about making the tags white?

Screenshot 2019-04-29 at 09 38 27

Run cancelled for master

TagBot is emitting Run cancelled for master for NIDAQ.jl, VennEuler.jl, and Morton,jl, but not, for example, for Gadfly.jl and Compose.jl. The only difference between the broken and working TagBot installations that i can see is the inclusion of - uses: actions/checkout@v1 in TagBot.yml in those that work. Don't know where that line came from, but it's not in the TagBot installation instructions in the README. should it be? or is something else wrong?? thanks.

Close the issue

This is a great bot! Thanks for doing this. Do you think tag-bot could also close the issue initially opened with @JuliaRegistrator register()?

Execution pipeline

There are a number of cases in which the webhook handler leaves a comment on a registry PR indicating something went wrong, with instructions to trigger a retry. In some of these cases (5xx from GitHub, git error), an automatic retry would be better.
We could take the same approach as the changelog generator, where instead of doing real work in the webhook handler, we just dump a message into SQS since it has better retry logic. In that case we just need to define which errors are retriable and which are not.

update: this is now basically a design refactor. sqs is not really cost-effective and sns works essentially the same so the plan is to push some context message to sns topic(s) between steps. that way every step can have its own retry policy. Each step either has some side effect or

  1. webhook from github: validate and parse. keep computation minimal, since this can't retry.
  2. notify: "starting", collect whatever github data will be needed (can retry, but should check for existing comments in case of retry)
  3. git tag: skipped if a tag already exists, retries if necessary, absolute failure is ok too since an existing tag is not required
  4. changelog generation: custom release notes are used if present, otherwise use the generator. need the --future-release option since a release does not yet exist (i think? i don't know if a tag works without a release) failure = empty release notes + notification
  5. github release: can retry, failure is actual failure
  6. notify: success or failure, can retry

Tags the wrong commit

For example, I instructed Registrator to register this commit. It did that properly. But then TagBot tagged this commit instead.

I assume that the problem here is that I'm asking Registrator to register a commit that is a) on a branch that is not master, and b) not at the tip of that branch. Still, it seems to me it should handle that properly.

I have to fix the tag in that repo now because it is messing up our dev flow, so you won't be able to see the incorrect tag for more than a few seconds, until I fix it.

Workaround for "n commits to master since this release" not appearing

GitHub seems to have a bug where if you specify a commit hash as the target_commitish when creating a release, the number of commits since the release never appears in GitHub's UI.

The workaround is to check that the commit being registered is the same as the commit on the tip of the branch being registered. In that case, we can set target_commitish to be the branch name and everything works. Most of the time, the commit being registered is the tip of master branch so it's a common case.

I've sent in a support ticket so I'll wait for a response before implementing this, in case I'm doing something wrong.

TagBot does not work when release commit is not in master branch

I noticed that a release I made via JuliaFolds/Transducers.jl#233 is not tagged by TagBot. It's been failing with

No matching commit was found for version v0.4.20 (c4f4cbdb7023e09acd0faebca628d386f1aaab98)

e.g. https://github.com/tkf/Transducers.jl/actions/runs/45228286

However, as you can see in JuliaRegistries/General#10043, the related commit is bf3a4db8ec49754e0a841db90e95a16d635a8667. I'm not sure where c4f4cbdb7023e09acd0faebca628d386f1aaab98 is comming from. It does not exist in the repository JuliaFolds/Transducers.jl@c4f4cbd (shows 404).

I used register(branch=release) when invoking JuliaRegistrator. Can it be relevant?

Feature request: Customize the "pre-release hook delay" (instead of it always being 5 minutes)

The section in README.md on "Pre-Release Hooks" says:

These actions will have 5 minutes to run.

Perhaps we could make this value customizable. That way, if someone wants to make a hook and they know the hook will take 10 minutes to run, they can set a "pre-release hook delay" of 15 minutes or 20 minutes or whatever.

Since public GitHub Actions can run for up to 6 hours, it wouldn't be a problem for TagBot to sleep for a period of time longer than 5 minutes.

Of course, it is reasonable to keep 5 minutes as the default value.

Feature request: after creating a tag, make a pull request to bump the version number

When I used to manually create tags, after I created the tag I would bump the version number in Project.toml. For example, after making tag v1.0.0, I would change the version in Project.toml from version = "0.1.0" to version = "0.2.0-DEV".

Now that TagBot automatically creates tags for me, I find myself forgetting to bump the version number in Project.toml to the next -DEV version.

Would it be possible for TagBot to automatically open a pull request to bump the version number?

Custom release notes in addition to autogenerated ones

Would be nice to be able to combine both custom release notes and the generated ones to include a sort of summary:

Registrator register()

Release notes:

This release has 9999 bug fixes.

{{changelog}}

This comes at the end. Thanks for reading?

Might be nice to insert anything above {{changelog}} after the generated header, i.e.:


v0.1.1 (2019-01-01)

Diff since v0.1.0

Summary

This release has 9999 bug fixes.

Closed issues:

Merged pull requests:

This comes at the end. Thanks for reading?

Move Docker images to GitHub Packages

No big benefits except I get to stop worrying about my personal Docker Hub account/credentials. Maybe marginally faster downloads? In any case it seems pretty simple to migrate.

Deploy guide

Deployment is not super trivial anymore, especially not with the upcoming #12, so a thorough guide is in order.

Handle yanked releases

When a release is yanked (marked as uninstallable) from a registry, its entry in the Versions.toml has yanked = true inserted. That make new yanks easily detectable by TagBot. But what's the right thing to do with a yanked release?

Options would appear to be:

  1. Delete it and the Git tag (not a fan, tags are supposed to be immutable)
  2. Delete it but leave the Git tag intact (seems fine but might leave people wondering "hey why does this tag have no release")
  3. Delete it and replace the Git tag with an identical one, but annotated and with a message like "this release has been yanked" (seems ideal and most similar to what is happening in the registry, but technically violates the "tags are immutable" thing)
  4. Only update the changelog (seems cool if we want to be safe and not touch Git tags)

Personally I think option 3 seems best. When replacing a tag with git push origin --tags --force, I don't think there is any chance of the tag being deleted and not replaced.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.