jonasbb / podman-gitlab-runner Goto Github PK
View Code? Open in Web Editor NEWUse Podman as a custom executor for your Gitlab CI
License: MIT License
Use Podman as a custom executor for your Gitlab CI
License: MIT License
It looks like cache isn't working as expected with gitlab-runner custom executor to podman.
With the following .gitlab-ci.yml (that works with a plain docker executor), Job B fails because cat
command cannot find hello.txt
file that sould have been restored from cache.
default:
image: debian:buster
tags:
- podman
stages:
- build
- test
job A:
stage: build
script:
- mkdir -p vendor/
- echo "build" > vendor/hello.txt
cache:
key: build-cache
paths:
- vendor/
job B:
stage: test
script:
- cat vendor/hello.txt
cache:
key: build-cache
paths:
- vendor/
policy: pull
Here is (part of) my config.toml
[[runners]]
name = "pleiades-ci podman runner"
url = "https://[masked]"
token = "[masked]"
executor = "custom"
builds_dir = "/builds"
cache_dir = "/cache"
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.custom]
prepare_exec = "/home/me/bin2/prepare.sh"
prepare_exec_timeout = 1800
run_exec = "/home/me/bin2/run.sh"
cleanup_exec = "/home/me/bin2/cleanup.sh"
cleanup_exec_timeout = 300
graceful_kill_timeout = 60
force_kill_timeout = 180
Here is the output of the failling job :
Setting up git-lfs (2.7.1-1+deb10u1) ...
Preparing environment 00:00
Running on 7a72d7ad609c...
Getting source from Git repository 00:01
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/DEC/pleiades/demo-ci-cd/.git/
Created fresh repository.
Checking out 5c308f92 as test-cache-podman...
Skipping Git submodules setup
Restoring cache 00:00
Checking cache for build-cache...
Runtime platform arch=amd64 os=linux pid=3635 revision=58ba2b95 version=14.2.0
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
Executing "step_script" stage of the job script 00:00
WARNING: Starting with version 14.0 the 'build_script' stage will be replaced with 'step_script': https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26426
$ cat vendor/hello.txt
cat: vendor/hello.txt: No such file or directory
Cleaning up file based variables 00:01
ERROR: Job failed: exit status 1
Am i missing something ?
Is there a way to set up "podman as a custom executor" in the image docker.io/gitlab/gitlab-runner:alpine?
Something such as: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-the-docker-executor-with-the-docker-image-docker-in-docker
Besides the currently supported DOCKER_AUTH_CONFIG
there are multiple other ways to authenticate against the Gitlab registry. The ways are listed here.
The script should try all logins until one is working. The order should be:
DOCKER_AUTH_CONFIG
This is a very specific configuration not only for the Gitlab Registry but for all, thus it should have the highest priority.CI_DEPLOY_USER
Deploy tokens do not exist by default, thus if they are created manually they should have a high priority.CI_JOB_USER
Provided automatically, thus low priority.CI_REGISTRY_USER
Provided automatically, outdated version of CI_JOB_USER
thus lowest priority.Quack,
Thanks for you nice work. I'm using a custom running to run tests that require an init system. It would be handy if we could override the command in custom_base.sh
. I could prepare a PR if you're ok with this proposal.
Regards.
\_o<
podman-gitlab-runner/prepare.sh
Lines 71 to 76 in 02d3f9e
I think the install_command test could be improved as the packages listed under the dependencies don't all contain binaries that are equal to the name of the package.
The ca-certificates
package for example does not contain any binary called ca-certificates
(at least not on archlinux/debian/redhat) which causes the ca-certificates
package to be installed in every job.
This can count for a significant amount of time, especially on yum based distros that have a lot of repos enabled.
Possible solutions:
Hi, I am using this executor but it seems to be impossible to change behaviour of exit code. The problem I am facing is that when I execute som .sh script on runner which should exit with status code 22 the final status code is always 1 anyway.
In this case i am not able to use allow_failure in my .gitlab-ci.yml file.
allow_failure:
exit_codes:
- 22
I found that there is variable "$BUILD_FAILURE_EXIT_CODE" in run.sh script but where is this coming from ?
Is there any possibility to change this behaviour ?
Thank you very much.
I'm trying to contribute a pull if-not-present policy for this runner similar to this docker runner option, but I can't stop podman from pulling.
In prepare.sh
#### Attempting to change this to enable the same as a "pull if-not-present" policy
podman image ls # shows that image is available
# podman pull --authfile "$CACHE_DIR"/_authfile_"$CONTAINER_ID" "$IMAGE"
# rm "$CACHE_DIR"/_authfile_"$CONTAI
podman run \
--detach \
--name "$CONTAINER_ID" \
--volume "$CACHE_DIR:/home/user/cache":Z \
--pull=missing \ ## Added this line although missing is the default
"${PODMAN_RUN_ARGS[@]}" \
"$IMAGE"\
sleep 999999999
In cleanup.sh:
# Try to remove all old containers, images, networks, and volumes
## this is killing the cache, need some way to maintain a cache. For now prune manually.
#podman system prune --force --volumes
But when I run another job, podman reaches out and pulls the container again. What am I missing? Is this a feasible contribution?
Podman 1.9 fixed the stdin truncation bug. This code can now be changed to be more idiomatic
podman exec "$CONTAINER_ID" /bin/bash < "$1"
Lines 11 to 12 in b659aca
Preparing the "custom" executor
Using Custom executor...
Running in runner-152-project-2683-concurrent-0-66548
Login to with CI_REGISTRY_USER
Error: authenticating creds for "": error pinging docker registry : Get "https:///v2/": http: no Host in request URL
ERROR: Preparation failed: exit status 2
You can see that a podman login
is tried with an empty registry address. I don't know why CUSTOM_ENV_CI_REGISTRY_USER
and CUSTOM_ENV_CI_REGISTRY_PASSWORD
are set while CUSTOM_ENV_CI_REGISTRY
is not, but that's what my university's GitLab instance is dealing me.
A pull request is on the way.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.