From the README:

Gitlab's $CI_BUILD_TOKEN infrastructure doesn't support accessing artifacts, so a private token must be used. This is very unfortunate and kludgey. This might be fixed in future Gitlab releases (if I bug them hard enough).

This seems to have been resolved on Gitlab's end since setting the request header JOB-TOKEN to $CI_JOB_TOKEN allows you to query the Gitlab artifacts api.

https://github.com/JonathonReinhart/scuba/blob/516a513c63d466ee8dddffb85c397b1e42dec35c/setup.py#L51

Gitlab includes filesystem permissions in its artifacts zip files, however gitlab-art doesn't preserve those permissions when installing artifacts.

1c0dfde changed ZipFile iteration to use ZipInfo.is_dir which was introduced in Python 3.6.

I have not yet upgraded everything to Debian Buster, so some machines are still running 3.5.

Can we have a statement in the README about supported python versions?

Project doesn't work with GitLab API v4, API v3 was removed since GitLab 11.0.

Even when the gitlab artifact includes empty directories the install step skips them and doesn't ever create them. It should reproduce the artifact as-is.

In the example below, if the artifact zip contains the directories and files below the directory bar will not ever be installed.

$ tree 
.
├── bar
├── foo
│   └── hello.txt
└── hello.txt

See e.g. https://github.com/JonathonReinhart/scuba/blob/master/CHANGELOG.md

Currently, art uses AppDirs.user_data_dir to store its config file:
https://github.com/kosma/art/blob/9a417871e8b3d390b5edd2625778feb4ed9e811f/art/_paths.py#L11

On my Debian system this resolves to /home/$USER/.local/share/art

It would probably be better to use AppDirs.user_config_dir which resolves to /home/$USER/.config/art.

Sometimes, we want to not only use artifacts from another project, but also files out of the source tree. This can be worked around by collecting those files as artifacts, but it might be helpful to pull them right from the repository instead.

The hardest part of this would be adapting the YML schema to accommodate.

@xanarin pointed out that Git allows a tag and a branch to have the same name. So the fix in #16 introduces a potential ambiguity, by no longer allowing tags/v1.2.3.

The problem is that the GitLab list project pipelines API just says ref, and I've proven that you cannot pass tags/foo:

>>> p.pipelines.list(ref='v8.4.8-1')
[<ProjectPipeline id:20692>, <ProjectPipeline id:20685>, <ProjectPipeline id:20639>, <ProjectPipeline id:20638>, <ProjectPipeline id:20478>]
>>> p.pipelines.list(ref='tags/v8.4.8-1')
[]

The API does however provide scope:

So perhaps we could look at the ref to see if tags/ is present, and if so, strip it, and then pass scope=tags.

We should use fnmatch here:

The install command builds a list of file operations, but it never checks that every operation is performed. It's very common for a source file path to change, resulting in no match when iterating archive members.

One solution is to remove source files from the list when they are matched. An error should be raised if the installs list is not empty after processing the entire archive.

Currently art resolves ref to its commit hash and then queries jobs using the sha field.

This can result in incorrect results. In particular, consider a policy of only retaining artifacts for tags and latest:

in this case, fetching aritfacts for ref: v1.0.0 results in resolving v1.0.0 to abcdefdeadbeef and then getting the last successful job, (2), whose artifacts have been deleted.

If, on the other hand, the job was queried by ref, it would have returned 1.

If the art install command is run without art download or if an archive file is removed from the cache directory, the command raises a KeyError exception and prints a traceback.

$ python3 -m art install
Traceback (most recent call last):
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/.cache/art/project/499184.zip'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "kosma/gitlab-art/art/_cache.py", line 35, in get
    raise KeyError(filename)

Run against multiple python versions: https://github.com/JonathonReinhart/scuba/blob/516a513c63d466ee8dddffb85c397b1e42dec35c/.github/workflows/build-test.yml#L14

There are several common exceptions raised by GitLab that should be expected and reported to the user in a clearer way.

1. Exceptions related to connecting to GitLab (bad url, no network)
2. Problems with the TLS certificate (expired cert, untrusted cert)
3. Authentication issues (no token, expired token)

Unfortunately there is already a project named art on PyPI: https://pypi.org/project/art/

This project could be renamed to e.g. gitlab-art and deployed to PyPI.