flux-local is a set of tools and libraries for managing a local flux gitops repository focused on validation steps to help improve quality of commits, PRs, and general local testing.
Home Page: https://allenporter.github.io/flux-local/
License: Apache License 2.0
![renovate[bot] avatar](https://avatars.githubusercontent.com/in/2740?v=4 "renovate[bot]")
A command like this:
$ flux build ks cluster-apps-kubernetes-dashboard --path tests/testdata/cluster2/apps/monitoring/kubernetes-dashboard/app --dry-run --kustomization-file tests/testdata/cluster2/apps/monitoring/kubernetes-dashboard/ks.yaml
can be used to build fluxtomizations rather than having them home grown and can better support flux features.
flux-local works perfectly for our use cases, but I cannot get flux-local build --enable-helm working when I have a helm chart which is oci hosted in my config.
Example:
I followed the install instructions for the weaveworks-gitops dashboard which leaves me with the following resources:
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
annotations:
metadata.weave.works/description: This is the source location for the Weave GitOps
Dashboard's helm chart.
labels:
app.kubernetes.io/component: ui
app.kubernetes.io/created-by: weave-gitops-cli
app.kubernetes.io/name: weave-gitops-dashboard
app.kubernetes.io/part-of: weave-gitops
name: ww-gitops
namespace: flux-system
spec:
interval: 1h0m0s
type: oci
url: oci://ghcr.io/weaveworks/charts
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
annotations:
metadata.weave.works/description: This is the Weave GitOps Dashboard. It provides
a simple way to get insights into your GitOps workloads.
name: ww-gitops
namespace: flux-system
spec:
chart:
spec:
chart: weave-gitops
sourceRef:
kind: HelmRepository
name: ww-gitops
interval: 1h0m0s
values:
adminUser:
create: true
passwordHash: <hash>
username: admin
with this HelmRelease in my flux kustomization
flux-local build ./clusters/<clustername> --enable-helm --skip-crds
failes with the following error:
flux-local error: Invalid <class 'flux_local.manifest.HelmChart'> missing sourceRef fields: {'apiVersion': 'helm.toolkit.fluxcd.io/v2beta1', 'kind': 'HelmRelease', 'metadata': {'name': 'ww-gitops', 'namespace': 'flux-system', 'annotations': {'config.kubernetes.io/index': '0', 'internal.config.kubernetes.io/index': '0'}}, 'spec': {'chart': {'spec': {'chart': 'weave-gitops', 'sourceRef': {'kind': 'HelmRepository', 'name': 'ww-gitops'}}}, 'interval': '1h0m0s', 'releaseName': 'ww-gitops'}}
It seems like this error happens because of a missing namespace key under spec.chart.spec.sourceRef.
When I add the namespace key like namespace: flux-system I get another error:
Command '(None) helm template ww-gitops flux-system-ww-gitops/weave-gitops --namespace flux-system --skip-tests --values /var/folders/tj/25qdsj_n1bqfr1bwvj9hjqxw0000gq/T/tmp7zjtbv7u/flux-system-ww-gitops-values.yaml --registry-config /dev/null --repository-cache /var/folders/tj/25qdsj_n1bqfr1bwvj9hjqxw0000gq/T/tmplbwf5vh5 --repository-config /var/folders/tj/25qdsj_n1bqfr1bwvj9hjqxw0000gq/T/tmp7zjtbv7u/repository-config.yaml' failed with return code 1
Error: chart "weave-gitops" matching not found in flux-system-ww-gitops index. (try 'helm repo update'): no chart name found
I don't know how to debug this further since flux-local seems to clean up my temp dir on the error.
With version v4 I noticed that the allenporter/flux-local/action/diff action is started to fail.
In the error there are errors regarding the generation of the kustomization.yaml e.g.:
DEBUG:flux_local.command:Command 'flux build ks crds-traefik --dry-run --kustomization-file /dev/stdin --path /Users/ansar/Projects/home-ops/clusters/home --namespace flux-system' failed with return code 1
✗ kustomize build failed: kustomization.yaml is empty
DEBUG:flux_local.command:Command 'flux build ks flux-system --dry-run --kustomization-file /dev/stdin --path /Users/ansar/Projects/home-ops/clusters/home --namespace flux-system' failed with return code 1
✗ failed to generate kustomization.yaml: failed to save original kustomization.yaml: open /Users/ansar/Projects/home-ops/clusters/home/kustomization.yaml: no such file or directory <nil> <nil>
As the error occurs in different places for each run my guess is that it's related to flux build generating the kustomization.yaml file but as multiple build commands are run parallel they will collide on the same file.
To verify I changed the concurrency level to 1 in here: https://github.com/allenporter/flux-local/blob/main/flux_local/command.py#L17
and it seems to "solved" the issue.
Currently you can only use single cluster flux-system repos.
Broken out into a separate issue from #35
Currently flux-local does not work like flux to generate a synthetic Kustomization if none is present on disk. This is not supported by kustomize build natively, as it prefers to have everything explicit.
Today flux works by actually writing to disk, which is not appropriate for a library/tool to update someones repo. Another option is to use some kind of overlay, which may be too heavy weight. Another option is to try to make a worktree we can write to which also seems too heavyweight for a readonly view (though seems appropriate for diff)
I actually did solve this in https://github.com/allenporter/flux-diff by making an in memory overlay to generate files in while runnign the go program. One option may be that the CLI needs to be entirely converted to golang.
Some Helm charts have required options in the templates which can cause the templating to fail if using Flux valuesFrom in Helm Releases. See this link for an example. I am able to install these because I use Flux's valuesFrom like in this HelmRelease, however helm template fails:
Error: execution error at (gha-runner-scale-set/templates/githubsecret.yaml:34:8): A valid .Values.githubConfigSecret is required for setting auth with GitHub server, provide .Values.githubConfigSecret.github_token or .Values.githubConfigSecret.github_app_id.
I am not sure if there is a way around this, because Helm doesn't provide an option to ignore these errors when using helm template. 😢 I guess we can leave this issue open for awareness if anything?
Incorporate the library https://github.com/willthames/kubernetes-validate to run against the built objects and ensure they are valid k8s objects against the schema.
This likely makes sense for the test command.
This may require specifying a different k8s version to validate against, and can allow users to validate in CI
The flux-local command line till will run the various tests in the library with the goal of meeting these use cases:
kustomize build on the cluster resources as a test to run before committing code, either as a pre-commit or a manually run tests ( example ), or run in CI ( example This will support overlays and multi-cluster repos like the existing tests for my k8s-gitops repo.HelmRelease for a specific chart version. The primary use case is to assist with HelmRelease version upgrades ( example ). This includes pulling in necessary HelmRepository objects from the cluster.kyverno cli policies to verify that cluster resources confirm to spec ( example )diff command on HelmRelease objects to support upgrades (example cubic3d's diff-hr-on-pr )You can do all of this today with the above references, so we're just pulling this together these various resources, but is complex to get to support all working together and supporting overlays, multi-cluster, or policies.
Example run:
https://github.com/onedr0p/home-ops/actions/runs/5595213825/jobs/10230874135
Many users reporting failures, like:
INFO:flux_local.git_repo:Unknown cluster source for OCIRepository flux: ./
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.10.12/x64/bin/flux-local", line 8, in <module>
sys.exit(main())
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/tool/flux_local.py", line 60, in main
asyncio.run(action.run(**vars(args)))
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/tool/diff.py", line 337, in run
await git_repo.build_manifest(
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 580, in build_manifest
clusters = await get_clusters(
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 470, in get_clusters
finished = await asyncio.gather(*tasks)
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 375, in kustomization_traversal
docs = await get_fluxtomizations(root_path_selector.root, path, build=build)
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 326, in get_fluxtomizations
docs = await cmd.objects()
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/kustomize.py", line 124, in objects
return [doc async for doc in self._docs()]
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/kustomize.py", line 124, in <listcomp>
return [doc async for doc in self._docs()]
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/kustomize.py", line 118, in _docs
out = await self.run()
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/kustomize.py", line 114, in run
return await run_piped(self._cmds)
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/command.py", line 104, in run_piped
result = await _run_piped_with_sem(cmds)
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/command.py", line 96, in _run_piped_with_sem
out = await asyncio.wait_for(cmd.run(stdin), _TIMEOUT)
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/command.py", line 69, in run
proc = await asyncio.create_subprocess_shell(
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/subprocess.py", line 205, in create_subprocess_shell
transport, protocol = await loop.subprocess_shell(
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/base_events.py", line 1648, in subprocess_shell
transport = await self._make_subprocess_transport(
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/unix_events.py", line 207, in _make_subprocess_transport
transp = _UnixSubprocessTransport(self, protocol, args, shell,
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/base_subprocess.py", line 36, in __init__
self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/unix_events.py", line 799, in _start
self._proc = subprocess.Popen(
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/subprocess.py", line 971, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/subprocess.py", line 1863, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: PosixPath('/home/runner/work/home-ops/home-ops/pr/deploy/controller')
Error: Process completed with exit code 1.
Error: Unable to process file command 'output' successfully.
Error: Invalid value. Matching delimiter not found '8b103dc866634c7a'
Last successful run: https://github.com/onedr0p/home-ops/actions/runs/5593429647/jobs/10227065727
It looks like the created worktree uses an absolute path (/private/var/..., [1]) whereas the process path is using the symlink path (/var/...).
This is probably hacky, but calling Path.resolve() in PathSelector.relative_to solves the issue:
return self.process_path.resolve().relative_to(self.root.resolve())
Stack trace:
❯ fl diff hr datalore -n datalore --path k8s/
Traceback (most recent call last):
File "/opt/homebrew/bin/flux-local", line 8, in <module>
sys.exit(main())
^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/flux_local/tool/flux_local.py", line 86, in main
asyncio.run(action.run(**vars(args)))
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/flux_local/tool/diff.py", line 308, in run
await git_repo.build_manifest(selector=query)
File "/opt/homebrew/lib/python3.11/site-packages/flux_local/git_repo.py", line 516, in build_manifest
clusters = await get_clusters(
^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/flux_local/git_repo.py", line 400, in get_clusters
kustomizations = await kustomization_traversal(path_selector)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/flux_local/git_repo.py", line 289, in kustomization_traversal
path_queue.put(path_selector.relative_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/flux_local/git_repo.py", line 141, in relative_path
return self.process_path.relative_to(self.root)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/pathlib.py", line 730, in relative_to
raise ValueError("{!r} is not in the subpath of {!r}"
ValueError: '/var/folders/24/vvv5svzx1xb2q1qpsq8zc8bw0000gn/T/tmpgjrnx0em/k8s' is not in the subpath of '/private/var/folders/24/vvv5svzx1xb2q1qpsq8zc8bw0000gn/T/tmpgjrnx0em' OR one path is relative and the other is absolute.[1] Under MacOS:
❯ ls -lsah /
total 10
0 drwxr-xr-x 20 root wheel 640B Jan 13 16:57 .
0 drwxr-xr-x 20 root wheel 640B Jan 13 16:57 ..
<...>
0 drwxr-xr-x 6 root wheel 192B Feb 19 23:41 private
0 lrwxr-xr-x@ 1 root wheel 11B Jan 13 16:57 tmp -> private/tmp
0 lrwxr-xr-x@ 1 root wheel 11B Jan 13 16:57 var -> private/varWhen testing flux-local in the repo https://github.com/xUnholy/k8s-gitops/tree/main it can't load some of the paths:
$ flux-local get hr --path kubernetes/namespaces/overlays/cluster-1
Command '(/home/allen/test/k8s-gitops/kubernetes/namespaces/overlays/cluster-1) kustomize build' failed with return code 1
Error: accumulating resources: accumulation err='accumulating resources from '../../base/actions-runner-system/actions-runner-controller/ks.yaml': security; file '/home/allen/test/k8s-gitops/kubernetes/namespaces/base/actions-runner-system/actions-runner-controller/ks.yaml' is not in or below '/home/allen/test/k8s-gitops/kubernetes/namespaces/overlays/cluster-1'': must build at directory: '/home/allen/test/k8s-gitops/kubernetes/namespaces/base/actions-runner-system/actions-runner-controller/ks.yaml': file is not directory
flux-local error: Error while building Kustomization 'None/kustomization' (path=None): Command '(/home/allen/test/k8s-gitops/kubernetes/namespaces/overlays/cluster-1) kustomize build' failed with return code 1
Error: accumulating resources: accumulation err='accumulating resources from '../../base/actions-runner-system/actions-runner-controller/ks.yaml': security; file '/home/allen/test/k8s-gitops/kubernetes/namespaces/base/actions-runner-system/actions-runner-controller/ks.yaml' is not in or below '/home/allen/test/k8s-gitops/kubernetes/namespaces/overlays/cluster-1'': must build at directory: '/home/allen/test/k8s-gitops/kubernetes/namespaces/base/actions-runner-system/actions-runner-controller/ks.yaml': file is not directory
A suggestion is to use --load-restrictor=LoadRestrictionsNone however this works fine in my own repo and is not needed:
$ flux-local get hr --path infrastructure/prod/ -A
NAMESPACE NAME REVISION CHART SOURCE
benji benji 2.0.0 benji-benji benji
kyverno kyverno 2.7.3 kyverno-kyverno kyverno
redis redis 17.11.2 redis-redis bitnami
rook-ceph rook-ceph v1.11.6 rook-ceph-rook-ceph rook-release
I got this error when running through github action:
flux-local error: Error building Fluxtomization in '/home/runner/work/home-cluster/home-cluster/pr' path 'cluster': Invalid <class 'flux_local.manifest.Kustomization'> missing metadata.namespace: {'apiVersion': 'kustomize.toolkit.fluxcd.io/v1', 'kind': 'Kustomization', 'metadata': {'name': 'flux-manifests', 'annotations': {'config.kubernetes.io/index': '165', 'config.kubernetes.io/path': 'base/flux-system/flux-manifests.yaml', 'internal.config.kubernetes.io/index': '165', 'internal.config.kubernetes.io/path': 'base/flux-system/flux-manifests.yaml'}}, 'spec': {'path': './', 'sourceRef': {'kind': 'OCIRepository', 'name': 'flux-manifests'}, 'interval': '30m', 'prune': True, 'wait': True, 'patches': [{'patch': '- op: add\n path: /spec/template/spec/containers/0/args/-\n value: --concurrent=8\n- op: add\n path: /spec/template/spec/containers/0/args/-\n value: --kube-api-qps=500\n- op: add\n path: /spec/template/spec/containers/0/args/-\n value: --kube-api-burst=1000\n- op: add\n path: /spec/template/spec/containers/0/args/-\n value: --requeue-dependency=5s\n', 'target': {'kind': 'Deployment', 'name': '(kustomize-controller|helm-controller|source-controller)'}}, {'patch': 'apiVersion: apps/v1\nkind: Deployment\nmetadata:\n name: all\nspec:\n template:\n spec:\n containers:\n - name: manager\n resources:\n limits:\n cpu: 2000m\n memory: 2Gi\n', 'target': {'kind': 'Deployment', 'name': '(kustomize-controller|helm-controller|source-controller)'}}]}}Try specifying another path within the git repo?
The error is gone when I add metadata.namespace to my Fluxtomization. This is expected in a logic viewpoint but flux in the cluster is not complaining so I think maybe this is something to be considered? Thank you for the great tool!
Hey @allenporter,
Seems like we can speed this up a bit :)
https://github.com/allenporter/flux-local/blob/main/action/diff/action.yml#L60
https://github.com/actions/setup-python?tab=readme-ov-file#caching-packages-dependencies
Flux-test should validate dependsOn rules to prevent mistakes referencing a helmrelease that doesn't exist.
I have "work in progress" .wip/ directories in my local repo that are included in my .gitignore. As these are work in progress, they may be in a state that cause flux-local to crash.
It would be really useful to be able to set flux-local (and the gh Action) to ignore directories that:
We use kustomize to generate configmaps from values.yaml files for better readability over the inline spec.values in HelmReleases and reference them in the spec.valuesFrom array.
These values will be ignored by flux-local on helm inflation because only the spec.values field is used (https://github.com/allenporter/flux-local/blob/main/flux_local/helm.py#L221).
I wonder if we could optionally (it is not guaranteed that the cm/secret is deployed via flux) pull in secret/cm values files if they exist in the content object.
I would like to try and implement this if it sounds like a valuable addition.
I will need some time to dig into the code though.
Example, this does not work at the moment:
example.yaml
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: example
namespace: example
spec:
interval: 5m
chart:
spec:
chart: example
version: "0.1.0"
sourceRef:
kind: HelmRepository
name: example
namespace: flux-system
interval: 1m
valuesFrom:
- kind: ConfigMap
name: example-values
optional: truekustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- example.yaml
configMapGenerator:
- files:
- values.yaml=example-values.yaml
name: example-values
namespace: example
generatorOptions:
disableNameSuffixHash: trueexample-values.yaml
image:
repository: "..."
tag: "..."Using 03ca70f (current HEAD), it looks as though flux-local doesn't support absolute paths within the kustomize.toolkit.fluxcd.io/v1 / Kustomization CRD. FluxCD does support absolute paths, where root would be the repository root.
I'm unable to share my flux repo directly, so hopefully the following information will suffice.
Repo structure, all flux related items are under /flux from the root of the repository:
$ tree
.
├── [...]
├── flux
│ ├── apps
│ │ └── [...]
│ ├── clusters
│ │ └── ovh
│ │ ├── bootstrap
│ │ │ ├── bootstrap.yaml
│ │ │ └── flux-system
│ │ │ ├── gotk-components.yaml
│ │ │ ├── gotk-sync.yaml
│ │ │ └── kustomization.yaml
│ │ └── deployments
│ │ └── kustomization.yaml
│ └── manifests
│ └── [...]
└── terraform
└── [...]flux/clusters/ovh/bootstrap/bootstrap.yaml contents, which looks to be the problem file:
$ cat flux/clusters/ovh/bootstrap/bootstrap.yaml
# Helm repositories
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: helm-repositories
namespace: flux-system
spec:
path: /flux/manifests/helm-repositories
prune: true
interval: 1h
timeout: 5m
retryInterval: 2m0s
sourceRef:
kind: GitRepository
name: flux-system
postBuild:
substitute: {}
substituteFrom:
- kind: ConfigMap
name: cluster-settings
decryption:
provider: sops
secretRef:
name: sops-keys
---
# [...]path: /flux/manifests/helm-repositories being the culprit.
Exception:
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/usr/local/lib/python3.10/dist-packages/_pytest/main.py", line 269, in wrap_session
INTERNALERROR> config.hook.pytest_sessionstart(session=session)
INTERNALERROR> File "/usr/local/lib/python3.10/dist-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR> return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR> File "/usr/local/lib/python3.10/dist-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> File "/usr/local/lib/python3.10/dist-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/usr/local/lib/python3.10/dist-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR> raise exc.with_traceback(exc.__traceback__)
INTERNALERROR> File "/usr/local/lib/python3.10/dist-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/src/flux_local/tool/test.py", line 280, in pytest_sessionstart
INTERNALERROR> asyncio.run(self.async_pytest_sessionstart(session))
INTERNALERROR> File "/usr/local/lib/python3.10/dist-packages/nest_asyncio.py", line 31, in run
INTERNALERROR> return loop.run_until_complete(task)
INTERNALERROR> File "/usr/local/lib/python3.10/dist-packages/nest_asyncio.py", line 99, in run_until_complete
INTERNALERROR> return f.result()
INTERNALERROR> File "/usr/lib/python3.10/asyncio/futures.py", line 201, in result
INTERNALERROR> raise self._exception.with_traceback(self._exception_tb)
INTERNALERROR> File "/usr/lib/python3.10/asyncio/tasks.py", line 232, in __step
INTERNALERROR> result = coro.send(None)
INTERNALERROR> File "/src/flux_local/tool/test.py", line 285, in async_pytest_sessionstart
INTERNALERROR> manifest = await git_repo.build_manifest(
INTERNALERROR> File "/src/flux_local/git_repo.py", line 533, in build_manifest
INTERNALERROR> results = await kustomization_traversal(selector.path)
INTERNALERROR> File "/src/flux_local/git_repo.py", line 387, in kustomization_traversal
INTERNALERROR> docs = await grep_fluxtomizations(selector.root, path)
INTERNALERROR> File "/src/flux_local/git_repo.py", line 315, in grep_fluxtomizations
INTERNALERROR> await kustomize.grep(f"kind={CLUSTER_KUSTOMIZE_KIND}", root / relative_path)
INTERNALERROR> File "/src/flux_local/kustomize.py", line 130, in objects
INTERNALERROR> return [doc async for doc in self._docs(target_namespace=target_namespace)]
INTERNALERROR> File "/src/flux_local/kustomize.py", line 130, in <listcomp>
INTERNALERROR> return [doc async for doc in self._docs(target_namespace=target_namespace)]
INTERNALERROR> File "/src/flux_local/kustomize.py", line 120, in _docs
INTERNALERROR> out = await self.run()
INTERNALERROR> File "/src/flux_local/kustomize.py", line 114, in run
INTERNALERROR> return await run_piped(self._cmds)
INTERNALERROR> File "/src/flux_local/command.py", line 117, in run_piped
INTERNALERROR> result = await _run_piped_with_sem(cmds)
INTERNALERROR> File "/src/flux_local/command.py", line 109, in _run_piped_with_sem
INTERNALERROR> out = await asyncio.wait_for(cmd.run(stdin), _TIMEOUT)
INTERNALERROR> File "/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
INTERNALERROR> return fut.result()
INTERNALERROR> File "/usr/lib/python3.10/asyncio/futures.py", line 201, in result
INTERNALERROR> raise self._exception.with_traceback(self._exception_tb)
INTERNALERROR> File "/usr/lib/python3.10/asyncio/tasks.py", line 232, in __step
INTERNALERROR> result = coro.send(None)
INTERNALERROR> File "/src/flux_local/command.py", line 82, in run
INTERNALERROR> proc = await asyncio.create_subprocess_shell(
INTERNALERROR> File "/usr/lib/python3.10/asyncio/subprocess.py", line 205, in create_subprocess_shell
INTERNALERROR> transport, protocol = await loop.subprocess_shell(
INTERNALERROR> File "/usr/lib/python3.10/asyncio/base_events.py", line 1648, in subprocess_shell
INTERNALERROR> transport = await self._make_subprocess_transport(
INTERNALERROR> File "/usr/lib/python3.10/asyncio/unix_events.py", line 207, in _make_subprocess_transport
INTERNALERROR> transp = _UnixSubprocessTransport(self, protocol, args, shell,
INTERNALERROR> File "/usr/lib/python3.10/asyncio/base_subprocess.py", line 36, in __init__
INTERNALERROR> self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
INTERNALERROR> File "/usr/lib/python3.10/asyncio/unix_events.py", line 799, in _start
INTERNALERROR> self._proc = subprocess.Popen(
INTERNALERROR> File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
INTERNALERROR> self._execute_child(args, executable, preexec_fn, close_fds,
INTERNALERROR> File "/usr/lib/python3.10/subprocess.py", line 1863, in _execute_child
INTERNALERROR> raise child_exception_type(errno_num, err_msg, err_filename)
INTERNALERROR> FileNotFoundError: [Errno 2] No such file or directory: PosixPath('/flux/manifests/helm-repositories')As for potential solutions, could likely check if the path is absolute, and if absolute, join <git worktree root> + <absolute path>. Where worktree root could be calculated from git rev-parse --show-toplevel or similar.
https://github.com/szinn/k8s-homelab/actions/runs/5206160235/jobs/9392423235
It seemed to generate a diff but showed the error in the PR - szinn/k8s-homelab#1905
The helm repository is an OCI repository - could that be it?
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
This repository currently has no open or pending branches.
.devcontainer/Dockerfile
ubuntu jammy-20240212docker.io/bitnami/kubectl 1.29.2registry.k8s.io/kustomize/kustomize v5.3.0ghcr.io/kyverno/kyverno-cli v1.11.4docker.io/alpine/helm 3.14.2ghcr.io/fluxcd/flux-cli v2.2.3Dockerfile
python 3.12-alpineghcr.io/fluxcd/flux-cli v2.2.3docker.io/alpine/helm 3.14.2docker.io/bitnami/kubectl 1.29.2registry.k8s.io/kustomize/kustomize v5.3.0ghcr.io/kyverno/kyverno-cli v1.11.4
.github/workflows/container-release.yaml
actions/checkout v4docker/metadata-action v5docker/setup-qemu-action v3docker/setup-buildx-action v3docker/login-action v3docker/build-push-action v5.github/workflows/container-test.yaml
actions/checkout v4docker/metadata-action v5docker/setup-qemu-action v3docker/setup-buildx-action v3docker/login-action v3docker/build-push-action v5.github/workflows/flux-local-diff.yaml
actions/checkout v4fluxcd/flux2 v2.2.3mshick/add-pr-comment v2.github/workflows/flux-local-test.yaml
actions/checkout v4fluxcd/flux2 v2.2.3.github/workflows/lint.yaml
actions/checkout v4chartboost/ruff-action v1.0.0ibiqlik/action-yamllint v3.github/workflows/pages.yaml
actions/checkout v4actions/setup-python v5actions/configure-pages v4actions/upload-pages-artifact v3actions/deploy-pages v4.github/workflows/python-package.yaml
actions/checkout v4fluxcd/flux2 v2.2.3actions/setup-python v5supplypike/setup-bin v3codecov/codecov-action v4.github/workflows/python-publish.yaml
actions/checkout v4actions/setup-python v5pypa/gh-action-pypi-publish v1.8.12action/diff/action.yml
actions/setup-python v5actions/checkout v4actions/checkout v4
requirements.txt
aiofiles ==23.2.1black ==24.2.0coverage ==7.4.3GitPython ==3.1.42mypy ==1.8.0nest_asyncio ==1.6.0pdoc ==14.4.0pip ==24.0pre-commit ==3.6.2pydantic ==2.6.3pytest ==8.0.2pytest-asyncio ==0.23.5pytest-cov ==4.1.0python-slugify ==8.0.4PyYAML ==6.0.1ruff ==0.3.0types-aiofiles ==23.2.0.20240106types-PyYAML ==6.0.12.12typing-extensions ==4.10.0types-python-slugify ==8.0.2.20240127wheel ==0.42.0yamllint ==1.35.1syrupy ==4.6.1
.pre-commit-config.yaml
pre-commit/pre-commit-hooks v4.5.0codespell-project/codespell v2.2.6charliermarsh/ruff-pre-commit v0.3.0adrienverge/yamllint v1.35.1psf/black 24.2.0
setup.cfg
aiofiles >=22.1.0nest_asyncio >=1.5.6pydantic >=2.5.2python-slugify >=8.0.0GitPython >=3.1.30PyYAML >=6.0pytest >=7.2.1pytest-asyncio >=0.20.3
Warning: The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Location: renovate.json5
Error type: Invalid JSON5 (parsing failed)
Message: JSON5.parse error: JSON5: invalid character '\"' at 5:5
A common case for version bumps is to see a diff like this repeated many times:
---
@@ -110,10 +110,10 @@
name: kube-prometheus-stack-grafana
namespace: default
labels:
- helm.sh/chart: grafana-6.50.7
+ helm.sh/chart: grafana-6.51.2
app.kubernetes.io/name: grafana
app.kubernetes.io/instance: kube-prometheus-stack
- app.kubernetes.io/version: "9.3.6"
+ app.kubernetes.io/version: "9.3.8"
app.kubernetes.io/managed-by: Helm
type: Opaque
data:
with numerous copies of the same string over and over. It may be easier to wield as an object diff, something like:
MyObject
helm.sh/chart: grafana-6.50.7 -> grafana-6.51.2
app.kubernetes.io/version: "9.3.6" -> "9.3.8"
There is a bug with flux diff supporting multiples clusters. This is a questionable use case, but it should be possible forward to make it work consistently across the commands.
https://github.com/onedr0p/home-ops/actions/runs/7304107954/job/19905782487?pr=6596
Given a rep at /github/workspace/pull/ running the following command from /github/workspace will fail
$ flux-local --log-level DEBUG diff kustomization --unified 6 --path-orig default/kubernetes/main --path pull/kubernetes/main --strip-attrs "helm.sh/chart,checksum/config,app.kubernetes.io/version,chart" --limit-bytes 10000 --all-namespaces --sources "home-kubernetes" --output-file diff.patch
...
FileNotFoundError: [Errno 2] No such file or directory: PosixPath('/github/workspace/pull/pull/kubernetes/main')
This is likely because of the relative path logic here
flux-local/flux_local/git_repo.py
Line 182 in c570d5c
Workarounds are:
A manifest file a a yaml description of all cluster resources. This allows you to have a declaration of the resources you expect to exist in the cluster to use for further unit testing. e.g. If you have a simple kustomize build test it will tell you that the build works, but it may not contain all the resources you expect.
This is currently being used in my personal cluster and this tracks porting this to the tool.
Broken out from #35
I run into a problem when using flux-local build because some helm charts use .capabilities checks in the templates:
flux-local build flux/clusters/cloud-infra/ --enable-helm --skip-crds
Command '(None) helm template renovate renovate-renovate/renovate --namespace renovate --skip-crds --skip-tests --version 35.141.3 --values /tmp/tmpt7op_xuk/renovate-renovate-values.yaml --registry-config /dev/null --repository-cache /tmp/tmpy2tsl0ia --repository-config /tmp/tmpt7op_xuk/repository-config.yaml' failed with return code 1
Error: execution error at (renovate/templates/cronjob.yaml:8:6):
ERROR: You must have at least batch/v1beta1 to use CronJob
While using flux-local test we can pass -a "batch/v1/CronJob" to the command to work around this issue, is there any similar way for flux-local build?
In order to make flux-local diff hr work better on CI, we need to have a simpler way to expose changed resources only. Building the kustomizations is much more light weight and we can look for changed objects and output them from the flux-local diff ks step to inform the flux-local diff hr step.
Right now when running flux-local diff hr -A it walks everything and evaluates it. Maybe internally it should run a flux local diff ks first, find the diffs, then run the hr diff. If its manual via diff ks output or internal, either way we need the diff building of kustomizations to better understand the objects that have changed.
Potentially related to #225 && #483
My latest my PR flux diff checks are failing to find the potentially related HelmRepository.
Run docker://ghcr.io/allenporter/flux-local:main
/usr/bin/docker run --name ghcrioallenporterfluxlocalmain_ed81d7 --label 6ed130 --workdir /github/workspace --rm -e "INPUT_ARGS" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/homelab/homelab":"/github/workspace" ghcr.io/allenporter/flux-local:main diff helmrelease --unified 6 --path /github/workspace/pull/kubernetes/apps --path-orig /github/workspace/default/kubernetes/apps --strip-attrs "helm.sh/chart,checksum/config,app.kubernetes.io/version,chart" --limit-bytes 10000 --all-namespaces --sources "home-kubernetes" --output-file diff.patch
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
flux-local error: Unable to find HelmRepository for flux-system-prometheus-community/kube-prometheus-stack for HelmRelease kube-prometheus-stack
I've run your local debugging testing and see results that I believe look correct
/ # flux-local get cluster --path /homelab/kubernetes/
PATH KUSTOMIZATIONS
kubernetes 38
/ # flux-local get helmreleases --path /homelab/kubernetes --all-namespaces
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
NAMESPACE NAME REVISION CHART SOURCE
services atuin 2.4.0 services-app-template bjw-s
auth authelia 2.4.0 auth-app-template bjw-s
cert-manager cert-manager v1.13.3 cert-manager-cert-manager jetstack
kube-system cilium 1.14.5 kube-system-cilium cilium
networking cloudflared 2.4.0 networking-app-template bjw-s
database cloudnative-pg 0.20.0 database-cloudnative-pg cloudnative-pg
kube-system coredns 1.29.0 kube-system-coredns coredns
networking echo-server 2.4.0 networking-app-template bjw-s
networking external-dns-cloudflare 1.13.1 networking-external-dns external-dns
networking external-dns-pihole 1.13.1 networking-external-dns external-dns
kube-system external-secrets 0.9.11 kube-system-external-secrets external-secrets
kube-system onepassword-connect 2.4.0 kube-system-app-template bjw-s
monitoring gatus 2.4.0 monitoring-app-template bjw-s
monitoring grafana 7.0.19 monitoring-grafana grafana
monitoring kube-prometheus-stack 55.5.1 monitoring-kube-prometheus-stack prometheus-community
monitoring kubernetes-dashboard 6.0.8 monitoring-kubernetes-dashboard kubernetes-dashboard
auth lldap 2.4.0 auth-app-template bjw-s
kube-system local-path-provisioner None kube-system-./deploy/chart/local-path-provisioner local-path-provisioner
kube-system metrics-server 3.11.0 kube-system-metrics-server metrics-server
storage minio 2.4.0 storage-app-template bjw-s
networking nginx-external 4.9.0 networking-ingress-nginx ingress-nginx
networking nginx-internal 4.9.0 networking-ingress-nginx ingress-nginx
database redis 18.6.2 database-redis bitnami
kube-system reloader 1.0.60 kube-system-reloader stakater
default smtp-relay 2.4.0 default-app-template bjw-s
kube-system snapshot-controller 2.0.4 kube-system-snapshot-controller piraeus
monitoring snmp-exporter 2.4.0 monitoring-app-template bjw-s
kube-system synology-csi 0.9.7 kube-system-synology-csi synology-csi
monitoring unpoller 2.4.0 monitoring-app-template bjw-s
volsync volsync 0.8.0 volsync-volsync backube
flux-system weave-gitops 4.0.36 flux-system-weave-gitops weave-gitops
The flux-local get helmreleases even shows the kube-prometheus-stack helmrelease that the PR check is reporting missing.
FWIW, I pulled the latest main docker image and ran this test there.
What is maybe interesting is that I get a missing HelmRepository that seems to relate to the changing package in the PR. In another one of my PRs it was a missing redis HelmRepository
Any ideas?
Will add a new comment with output from a run with debug: true on the action.
flux-local test won't find my kustomizations unless it's in the flux-system namespace
Next steps: Add a test case to reproduce the issue.
This does not yet work with repos that use OCIRepository for the source e.g. https://github.com/xUnholy/k8s-gitops/blob/main/kubernetes/clusters/cluster-1/cluster.yaml
Example run: https://github.com/allenporter/k8s-gitops/actions/runs/5099732326/jobs/9167613119
The output is so large it fails the action. We should instead have an option to truncate the diff and return a partial result.
Hello,
I was initially trying to test your CLI in our gitops repository but ran into errors:
flux-local error: Error building Fluxtomization in '/Users/foo/gitops' \
path 'manifests/monitoring/monitoring-config': Specified path is not a directory: \
/Users/foo/gitops/manifests/monitoring/monitoring-config - \
Is a Kustomization pointing to a path that does not exist?
I believe it is because an external GitReoisitory source is unavailable locally.
We have quite a few of those external sources, it would be nice if you could support them at some point.
Here is a minimal example that runs into the error mentioned above.
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
name: flux-source
namespace: flux-system
spec:
ref:
tag: "v2.1.2"
interval: 120m
timeout: 60s
url: https://github.com/fluxcd/flux2.git
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: flux-dashboards
namespace: flux-system
spec:
interval: 10m0s
path: ./manifests/monitoring/monitoring-config
prune: true
sourceRef:
kind: GitRepository
name: flux-source
timeout: 10mNot referencing the files in the Kustomization files will result in successful commands but incomplete diffs.
Thank you
From discussion in k8s @ home discord, seems like targetNamespace in the fluxtomization isn't supported properly.
Example of failed run: https://github.com/onedr0p/home-ops/actions/runs/6923823571/job/18832218496?pr=6412
Example of PR to add this: onedr0p/home-ops@7e54458
As discussed in Discord, it would be great to have an arg to output to a file. My use case would be to use the new container we built in a Github workflow like this since redirecting to stdout won't work using the container.
- name: Diff Resources
uses: docker://ghcr.io/allenporter/flux-local:main
with:
args: >
diff ${{ matrix.resources }}
--unified 6
--path-orig live/${{ matrix.paths }}
--path pr/${{ matrix.paths }}
--strip-attrs "helm.sh/chart,checksum/config,app.kubernetes.io/version,chart"
--limit-bytes 10000
--all-namespaces
--sources "home-kubernetes"
--output-file diff.patch
- name: Generate Diff
id: diff
run: |
echo "diff<<EOF" >> $GITHUB_OUTPUT
cat diff.patch >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUTRight now the annotations added by the cfg step make diffs nearly unreadable. We need to strip them before computing diffs.
We can either rewrite all objects when visiting, or push the logic into the visitor and have it re-write the objects before diffing.
The actions in https://github.com/allenporter/flux-local-action fail trying to pull the requirements they need from the code owners repo rather than its own repo.
ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/home/runner/work/home-ops/home-ops/pr/requirements.txt'
Error: Process completed with exit code 1.
When in a subdirectory, running flux test will fail the local path check since it is not relative to the repo.
E flux_local.exceptions.InputException: Specified path is not a directory: clusters/prod
One problem with Helm is that it "hides" images from our GitOps repos, many times I've seen people getting burned by those images not being on the registry yet when they go to merged a PR and Flux tries to reconcile but the image is not available yet.
It would be great if there was a way to parse out the images so we can test if they are available in CI on a PR.
We have added a private helm repository to our flux deployment.
Flux gets its credentials from a secret in the kubernetes cluster (which is therefore of course not available to flux-local in the repo).
Example helm repository:
---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
[...]
url: https://gitlab.example.com/api/v4/projects/1234/packages/helm/stable
secretRef:
name: helm-pull-tokenAccordingly, a flux-local run in the CI pipeline terminates with the following error:
Command '(None) helm template clusterconfig flux-system-clusterconfig/clusterconfig --namespace clusterconfig --skip-crds --skip-tests --version 0.1.0 --registry-config /dev/null --repository-cache /tmp/tmp0nzo4u72 --repository-config /tmp/tmpiynunb2i/repository-config.yaml' failed with return code 1
Error: no cached repo found. (try 'helm repo update'): open /tmp/tmp0nzo4u72/flux-system-clusterconfig-index.yaml: no such file or directory
flux-local error: Command '(None) helm template clusterconfig flux-system-clusterconfig/clusterconfig --namespace clusterconfig --skip-crds --skip-tests --version 0.1.0 --registry-config /dev/null --repository-cache /tmp/tmp0nzo4u72 --repository-config /tmp/tmpiynunb2i/repository-config.yaml' failed with return code 1
Error: no cached repo found. (try 'helm repo update'): open /tmp/tmp0nzo4u72/flux-system-clusterconfig-index.yaml: no such file or directory
We would therefore need a way to also provide a secret via flux-local to helm, so that we could pull and diff the private helmrelease from within a ci pipeline.
In the #80 PR the implicit handling was added but that did not handle if there are subdirectories that didn't contain a kustomisation file (this works in flux).
E.g.:
I have a kustomization set for the flux/apps folder which do not have a kustomization yaml file and it only contains subdirectories
flux/apps/database/postgresql
Checking the code: https://github.com/allenporter/flux-local/blob/6b8e0a8e8d2a017eedf5262fb972eb9b8711a9a2/flux_local/kustomize.py#L254C10-L254C10
now it only processes sub directories if it's explicitly kustomizable.
If that check is removed I was able to make it work locally
❯ flux-local get hr -A --path ./clusters/home
NAMESPACE NAME REVISION CHART SOURCE
database postgresql 12.7.1 database-postgresql bitnami-charts
without the change
❯ flux-local get hr -A --path ./clusters/home
no HelmRelease objects found in cluster
Is it possibly to remove this limitation and allow to process sub directories as well?
In the spec for the Kustomization kind in kustomize.toolkit.fluxcd.io/v1, the path field is optional:
(Optional)
Path to the directory containing the kustomization.yaml file, or the set of plain YAMLs a kustomization.yaml should be generated for. Defaults to ‘None’, which translates to the root path of the SourceRef.
However, when the test action is run against a repo using kustomizations with path unset, flux-local throws an input exception:
flux-local/flux_local/manifest.py
Lines 295 to 296 in 60be4ba
Example manifests which trigger this error:
---
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: GitRepository
metadata:
name: kube-prometheus-stack-source
namespace: flux-system
spec:
interval: 30m
url: https://github.com/prometheus-community/helm-charts.git
ref:
tag: kube-prometheus-stack-43.0.0
ignore: |
# exclude all
/*
# include deploy crds dir
!/charts/kube-prometheus-stack/crds
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: kube-prometheus-stack-crds
namespace: flux-system
spec:
interval: 15m
prune: false
sourceRef:
kind: GitRepository
name: kube-prometheus-stack-source
healthChecks:
- apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
name: alertmanagerconfigs.monitoring.coreos.com
- apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
name: alertmanagers.monitoring.coreos.com
- apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
name: podmonitors.monitoring.coreos.com
- apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
name: probes.monitoring.coreos.com
- apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
name: prometheuses.monitoring.coreos.com
- apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
name: prometheusrules.monitoring.coreos.com
- apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
name: servicemonitors.monitoring.coreos.com
- apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
name: thanosrulers.monitoring.coreos.comSource: https://github.com/rtrox/home-cluster/blob/main/cluster-cd/crds/kube-prometheus-stack/crds.yaml
Full Logs from run with traceback:
Run flux-local \
flux-local \
--log-level INFO \
test \
--enable-helm \
--no-enable-kyverno \
--api-versions "policy/v1/PodDisruptionBudget" \
--kustomize-build-flags="" \
--sources "" \
--path cluster-cd/clusters/chongus
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
env:
pythonLocation: /opt/hostedtoolcache/Python/3.10.12/x64
PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.10.12/x64/lib/pkgconfig
Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.12/x64
Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.12/x64
Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.10.12/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.10.12/x64/lib
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 368, in kustomization_traversal
INTERNALERROR> docs = await get_flux_kustomizations(root, path)
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 329, in get_flux_kustomizations
INTERNALERROR> return [
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 330, in <listcomp>
INTERNALERROR> Kustomization.parse_doc(doc)
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/manifest.py", line 296, in parse_doc
INTERNALERROR> raise InputException(f"Invalid {cls} missing spec.path: {doc}")
INTERNALERROR> flux_local.exceptions.InputException: Invalid <class 'flux_local.manifest.Kustomization'> missing spec.path: {'apiVersion': 'kustomize.toolkit.fluxcd.io/v1', 'kind': 'Kustomization', 'metadata': {'name': 'kube-prometheus-stack-crds', 'namespace': 'flux-system', 'annotations': {'config.kubernetes.io/index': '1', 'config.kubernetes.io/path': 'kube-prometheus-stack/crds.yaml', 'internal.config.kubernetes.io/index': '1', 'internal.config.kubernetes.io/path': 'kube-prometheus-stack/crds.yaml'}}, 'spec': {'interval': '15m', 'prune': False, 'sourceRef': {'kind': 'GitRepository', 'name': 'kube-prometheus-stack-source'}, 'healthChecks': [{'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'alertmanagerconfigs.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'alertmanagers.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'podmonitors.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'probes.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'prometheuses.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'prometheusrules.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'servicemonitors.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'thanosrulers.monitoring.coreos.com'}]}}
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/_pytest/main.py", line 267, in wrap_session
INTERNALERROR> config.hook.pytest_sessionstart(session=session)
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR> return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/tool/test.py", line 280, in pytest_sessionstart
INTERNALERROR> asyncio.run(self.async_pytest_sessionstart(session))
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/nest_asyncio.py", line 35, in run
INTERNALERROR> return loop.run_until_complete(task)
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/site-packages/nest_asyncio.py", line 90, in run_until_complete
INTERNALERROR> return f.result()
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/futures.py", line 201, in result
INTERNALERROR> raise self._exception.with_traceback(self._exception_tb)
INTERNALERROR> File "/opt/hostedtoolcache/Python/3.10.12/x64/lib/python3.10/asyncio/tasks.py", line 232, in __step
INTERNALERROR> result = coro.send(None)
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/tool/test.py", line 285, in async_pytest_sessionstart
INTERNALERROR> manifest = await git_repo.build_manifest(
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 656, in build_manifest
INTERNALERROR> clusters = await get_clusters(
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 534, in get_clusters
INTERNALERROR> kustomizations = await kustomization_traversal(path_selector)
INTERNALERROR> File "/home/runner/work/_actions/allenporter/flux-local/main/flux_local/git_repo.py", line 371, in kustomization_traversal
INTERNALERROR> raise FluxException(
INTERNALERROR> flux_local.exceptions.FluxException: Error building Fluxtomization in '/home/runner/work/home-cluster/home-cluster' path 'cluster-cd/crds': Invalid <class 'flux_local.manifest.Kustomization'> missing spec.path: {'apiVersion': 'kustomize.toolkit.fluxcd.io/v1', 'kind': 'Kustomization', 'metadata': {'name': 'kube-prometheus-stack-crds', 'namespace': 'flux-system', 'annotations': {'config.kubernetes.io/index': '1', 'config.kubernetes.io/path': 'kube-prometheus-stack/crds.yaml', 'internal.config.kubernetes.io/index': '1', 'internal.config.kubernetes.io/path': 'kube-prometheus-stack/crds.yaml'}}, 'spec': {'interval': '15m', 'prune': False, 'sourceRef': {'kind': 'GitRepository', 'name': 'kube-prometheus-stack-source'}, 'healthChecks': [{'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'alertmanagerconfigs.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'alertmanagers.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'podmonitors.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'probes.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'prometheuses.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'prometheusrules.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'servicemonitors.monitoring.coreos.com'}, {'apiVersion': 'apiextensions.k8s.io/v1', 'kind': 'CustomResourceDefinition', 'name': 'thanosrulers.monitoring.coreos.com'}]}}Is a Kustomization pointing to a path that does not exist?
Github Actions run: https://github.com/rtrox/home-cluster/actions/runs/5299408363/jobs/9592345745
It would be great if there was a way to remove the auto generated helm labels from the diff, see below for an example.
onedr0p/home-ops#5393 (comment)
I was doing something like this on my hacked xz version but you probably have more control here to omit certain labels and annotation (like helm.sh/chart and checksum/config) instead of all of them.
https://github.com/onedr0p/home-ops/blob/main/.github/scripts/helmReleaseDiff.mjs#L73-L81
We can add a separate flag to show the column with # of helm releases and repos, and possibly skip some of the building steps to speed up flux-local get ks
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.