Git Product home page Git Product logo

github-release-retry's Introduction

github-release-retry

License

github-release-retry is a tool for creating GitHub Releases and uploading assets reliably. It differs from other tools because it uploads assets reliably by verifying that the asset exists, and retries (deleting partial assets) if not.

This is not an officially supported Google product.

Install

Requires Python 3.6+.

To ensure you use the pip module associated with your preferred Python 3.6+ binary:

python3 -m pip install --user github-release-retry
# where `python3` is your preferred Python 3.6+ binary.
# Omit `--user` to install for all users.

Or just:

pip3 install --user github-release-retry
# where `pip3` is your version of pip for Python 3.6+.
# Omit `--user` to install for all users.

Usage

If your Python user scripts directory is not on your PATH, you can use:

python3 -m github_release_retry.github_release_retry
# where `python3` is your preferred Python 3.6+ binary.

Otherwise:

$ github-release-retry -h
usage: github-release-retry [-h] --user USER --repo REPO --tag_name TAG_NAME
                            [--target_commitish TARGET_COMMITISH]
                            [--release_name RELEASE_NAME]
                            (--body_string BODY_STRING | --body_file BODY_FILE)
                            [--draft] [--prerelease]
                            [--github_api_url GITHUB_API_URL]
                            [--retry_limit RETRY_LIMIT]
                            [files [files ...]]

Creates a GitHub release (if it does not already exist) and uploads files to the release.
Please set the GITHUB_TOKEN environment variable.
EXAMPLE:
github-release-retry \
  --user paul \
  --repo hello-world \
  --tag_name v1.0 \
  --target_commitish 448301eb \
  --body_string "My first release." \
  hello-world.zip RELEASE_NOTES.txt

positional arguments:
  files                 The files to upload to the release. (default: None)

optional arguments:
  -h, --help            show this help message and exit
  --user USER           Required: The GitHub username or organization name in
                        which the repo resides. (default: None)
  --repo REPO           Required: The GitHub repo name in which to make the
                        release. (default: None)
  --tag_name TAG_NAME   Required: The name of the tag to create or use.
                        (default: None)
  --target_commitish TARGET_COMMITISH
                        The commit-ish value where the tag will be created.
                        Unused if the tag already exists. (default: None)
  --release_name RELEASE_NAME
                        The name of the release. Leave unset to use the
                        tag_name (recommended). (default: None)
  --body_string BODY_STRING
                        Required (or use --body_file): Text describing the
                        release. Ignored if the release already exists.
                        (default: None)
  --body_file BODY_FILE
                        Required (or use --body_string): Text describing the
                        release, which will be read from BODY_FILE. Ignored if
                        the release already exists. (default: None)
  --draft               Creates a draft release, which means it is
                        unpublished. (default: False)
  --prerelease          Creates a prerelease release, which means it will be
                        marked as such. (default: False)
  --github_api_url GITHUB_API_URL
                        The GitHub API URL without a trailing slash. (default:
                        https://api.github.com)
  --retry_limit RETRY_LIMIT
                        The number of times to retry creating/getting the
                        release and/or uploading each file. (default: 10)

Development

Optional: if you have just done git pull and Pipfile.lock was updated, you can delete .venv/ to start from a fresh virtual environment.

On Windows, you can use the Git Bash shell, or adapt the commands (including those inside dev_shell.sh.template) for the Windows command prompt.

Clone this repo and change to the directory that contains this README file. Execute ./dev_shell.sh.template. If the default settings don't work, make a copy of the file called dev_shell.sh and modify according to the comments before executing. pip must be installed for the version of Python you wish to use. Note that you can do e.g. export PYTHON=python3 first to set your preferred Python binary. We currently target Python 3.6+.

Pip for Python 3.6 may be broken on certain Debian distributions. You can just use the newer Python 3.7+ version provided by your distribution. Alternatively, see "Installing Python" below if you want to use Python 3.6.

The script generates a Python virtual environment (located at .venv/) with all dependencies installed. Activate the Python virtual environment via:

  • source .venv/bin/activate (on Linux)
  • source .venv/Scripts/activate (on Windows with the Git Bash shell)
  • .venv/Scripts/activate.bat (on Windows with cmd)

Presubmit checks

  • Execute ./check_all.sh to run various presubmit checks, linters, etc.
  • Execute ./fix_all.sh to automatically fix certain issues, such as formatting.

PyCharm

Use PyCharm to open the directory containing this README file. It should pick up the Python virtual environment (at .venv/) automatically for both the code and when you open a Terminal or Python Console tab.

Install and configure plugins:

  • File Watchers (may already be installed)
    • The watcher task should already be under version control.
  • Mypy: the built-in PyCharm type checking uses Mypy behind-the-scenes, but this plugin enhances it by using the latest version and allowing the use of stricter settings, matching the settings used by the ./check_all.sh script.

Add dictionary.dic as a custom dictionary (search for "Spelling" in Actions). Do not add words via PyCharm's "Quick Fixes" feature, as the word will only be added to your personal dictionary. Instead, manually add the word to dictionary.dic.

Terminal

The Terminal tab in PyCharm is useful and will use the project's Python virtual environment.

Installing Python

To manually install Python on your Linux distribution, you can use pyenv.

https://github.com/pyenv/pyenv#basic-github-checkout

In summary:

  • Install the required packages recommended here.

  • Then:

git clone https://github.com/pyenv/pyenv.git ~/.pyenv

# Add the following two lines to your ~/.bashrc file.
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"

# In a new terminal:
eval "$(pyenv init -)"
pyenv install 3.6.9
pyenv global 3.6.9

# Now execute the development shell script, as usual.
export PYTHON="python"
./dev_shell.sh.template

You can reactivate the virtual environment later using source .venv/bin/activate, without having to re-execute the above pyenv commands.

github-release-retry's People

Contributors

paulthomson avatar stasfilin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

github-release-retry's Issues

New GitHub node id format is not supported

It seems that GitHub recently introduced a new format for their node ids. Specifically, release asset node ids used to just be a base64 encoded string. Newer node ids look like this: "RA_[base64 encoded bytes]". It looks like the asset id is still encoded within the node id, but the format is different.

We use the ability to extract the REST v3 API "id" from the GraphQL v4 API "node id", even though this is undocumented.

Have an optional checksum check mode

To ensure that all the files in a release are correct, I would like to have a checksum check mode.
This would work from a file or directory, then all files would be hashed locally (say a sha256) and then each released file would be downloaded and its checksum computed and verified against the local file.

Also support whole directories upload

It would useful to be able to point to a directory (as opposed to one or more files) to upload its content recursively.
And thanks ++ for this tool!
We are using github-release-retry in scancode release handling to maintain a our own "PyPI" like repo of pre-built wheels at https://github.com/nexB/thirdparty-packages/releases/tag/pypi
@Abhishek-Dev09 worked out a simple wrapper to walk a directory at https://github.com/nexB/scancode-toolkit/blob/develop/etc/scripts/github_release.py#L72 but it would much simpler to have this baked in upstream here.
Do you want a PR?

Optimize re-uploading large releases

When I upload a large number of files as release assets, I sometimes want to reupload later when I have only a few new files added to a whole directory.
In this case each file is processed one by one.
I feel it may be possible to do something to optimize the case where there are many files uploaded at once.

  1. fetch all existing assets data at once
  2. compare this with the list of files to upload
  3. selectively work on things that need to be added/modified but do not do any further API call for each file

This is likely to speed things up as there would be one bigger API call at the start and then fewer API work after that.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.