Comments (8)
from gitlab-runner-operator.
@gbonnefille
I've checked the docs and it looks like you are after this scenario: https://docs.gitlab.com/runner/fleet_scaling/#intermediate-configuration-one-runner-multiple-workers
concurrent = 3
[[runners]]
name = "instance_level_shell_001"
url = ""
token = ""
executor = "shell"
[[runners]]
name = "instance_level_shell_002"
url = ""
token = ""
executor = "shell"
[[runners]]
name = "instance_level_shell_003"
url = ""
token = ""
executor = "shell"
It might be achievable with something like
apiVersion: gitlab.k8s.alekc.dev/v1beta1
kind: MultiRunner
metadata:
name: multi-runner
spec:
runners:
- registration_config:
token: "xxx"
tag_list:
- group1
- log_level: debug
executor_config:
image: "debian:slim"
memory_limit: "150Mi"
memory_request: "150Mi"
volumes:
config_map:
- mount_path: /cm/
name: test-config
secret:
- mount_path: /secrets/1/
name: test-secret
- mount_path: /secrets/2/
name: test-secret-2
registration_config:
token: "xxx"
tag_list:
- advanced-runner
I can definitely see advantages to this, however I need to think about it (due to a lack of personal free time atm sadly).
Biggest catches to look for would be:
- dealing with a situation where one of the regs is not succesful (since the registrations would have to be done one by one)
- removal of multiple runners on crd deletion.
from gitlab-runner-operator.
I can definitely see advantages to this, however I need to think about it (due to a lack of personal free time atm sadly).
No problem, the most important is that the idea is acceptable, so I can start looking how I can contribute to that.
from gitlab-runner-operator.
Here are some though after reading code.
The most challenging part is, of course, to take into account any changes on the config.
The gitlab-runner is able to reload automatically its config if a change occurs. So, no need to look for change in the executor_config
, simply rewrite again and again the configuration file.
The gitlab-runner-controler has to detect changes on registration_config
. As a registration token is certainly uniq, it could serves as a key for an internal map. Base on it, it would be possible to detect addition, change or deletion of runner. Such events trigger new registration/unregistration on the gitlab.
Note: if the user simply change the registration token, keeping everything else, this would be processed as a deletion+addition.
from gitlab-runner-operator.
@alekc I also started playing with the code (nothing functional yet).
Finally, I started looking at a different solution: extending the current CRD with a runners:
array.
#19
One idea is to stay as close as the Gitlab Runner configuration.
Another idea is to reuse the « root » configuration as a default configuration. When I set multiple configurations in runners:
tree, I can share some details by declaring a global registration_config
.
What do you think about such implementation direction?
from gitlab-runner-operator.
I am doing some work on it as well (you can see it in PR). I had to split Runner and Multirunner in 2 different CRD/controllers.
Main reason is: as right now, Runner saves the information about latest registration tag, registration and auth tokens as a single field in the status (they are needed because otherwise we do not know if the reg spec has been changed or not). That obviously won't work with the multiple runners approach because we need to keep track of this data for every runner specified in the multirunner section.
WHile I am at it, I am doing some minor/major refactoring around components and making them as generic as possible. You can expect a working testing demo in couple of days top.
I'd rather not to touch too much existing runner because its simple and already used in some environments, so I would like to keep the change as small as possible for legacy (well, I am going to drop the secret containing token, but thats another story).
from gitlab-runner-operator.
I understand you don't want to risk breaking the legacy.
But I have the feeling that we can create a solution keeping compatibility with previous logic. I will try to work on this.
from gitlab-runner-operator.
Hey @gbonnefille , sorry for the delay. Past days were a bit more challenging that expected.
You can try to build from the latest branch mentioned in the pr above.
so far in my tests given this object
apiVersion: gitlab.k8s.alekc.dev/v1beta1
kind: MultiRunner
metadata:
name: multirunner-simple
spec:
concurrent: 10
gitlab_instance_url: https://gitlab.com/
entries:
- name: runner 1
registration_config:
tag_list:
- dev1
- test-gitlab-runner
token: xxx
description: "Runner 1desc"
executor_config:
memory_request: "50Mi"
- name: runner 2
registration_config:
tag_list:
- dev2
- runner2
token: xxx
description: "Runner 2 desc"
A single runner (on kubernetes) is built and launched having following config:
apiVersion: v1
data:
config.toml: |
listen_address = ":9090"
concurrent = 10
check_interval = 3
log_level = "info"
[[runners]]
name = "runner 1"
limit = 10
url = "https://gitlab.com/"
token = "mSMjXSd-x"
executor = "kubernetes"
[runners.kubernetes]
host = ""
namespace = "default"
[[runners]]
name = "runner 2"
limit = 10
url = "https://gitlab.com/"
token = "xx"
executor = "kubernetes"
[runners.kubernetes]
host = ""
namespace = "default"
kind: ConfigMap
metadata:
creationTimestamp: "2022-10-19T19:49:50Z"
name: gitlab-mrunner-multirunner-simple
namespace: default
ownerReferences:
- apiVersion: gitlab.k8s.alekc.dev/v1beta1
controller: true
kind: MultirRunner
name: multirunner-simple
uid: 97867551-b586-4b4f-b2a3-249a00b17698
resourceVersion: "84100"
uid: 1ba25c67-0b60-4210-9dd0-402b658ff363
There are still a quiet bit of work to do before this sees the end release but, so far looks good for most aspects.
Todo list:
Check if there is a bug related to the generation of the config, in the example above, 50Mi should have been stored in the config somewherefixed- Due to how the logic is structured, multiple entries need to have unique tag. Otherwise there is a registration loop present. This has to be mentioned in the docs, and checked on admission controller/logic check version
- Since multirunner and single runner now share the same Business logic, they need to be unified
- Tests
- Documentation
- Examples
from gitlab-runner-operator.
Related Issues (7)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gitlab-runner-operator.