validatedpatterns / multicloud-gitops Goto Github PK
View Code? Open in Web Editor NEWMulti-cloud Gitops validated pattern
License: Apache License 2.0
Multi-cloud Gitops validated pattern
License: Apache License 2.0
When running
./pattern.sh make install
per https://validatedpatterns.io/patterns/multicloud-gitops/mcg-getting-started/ I get
make -f common/Makefile operator-deploy
make[1]: Entering directory '/home/adelton/project/validatedpatterns/multicloud-gitops'
Checking prerequisites:
Check for 'git helm oc ansible': OK
Check for python-kubernetes: OK
Check for kubernetes.core collection: OK
Checking repository:
https://github.com/validatedpatterns/multicloud-gitops - branch main: Running inside a container: Skipping git ssh checks
Checking cluster:
cluster-info: OK
storageclass: OK
# Retry five times because the CRD might not be fully installed yet
for i in {1..5}; do \
helm template --include-crds --name-template multicloud-gitops common/operator-install/ -f values-global.yaml --set main.git.repoURL="https://github.com/validatedpatterns/multicloud-gitops" --set main.git.revision=main | oc apply -f- && break || sleep 10; \
done
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
[...]
Since I'm running this as an unprivileged user with rootless podman, I believe this warning comes from some container that gets used, so something that should be fixed in the container image.
test123
Hi,
this will likely be long lasting ticket but "As an admin I want to uninstall the applications and framework from a cluster".
Just wanted to write it down here. Currently I found it easier to rebuild the cluster than hunt down all the stuff operators leave behind. And that's a problem, not all operators clean themseves up nicely. It might require creating an ansible playbook to go and delete all the resources that are left behind, and naturally file tickets to operators while doing so. Hopefully the list would shorten while operators enhance.
Now if the Makefile would be ansible, it might be easier to just mark stuff with state: absent/present syntax, which then would help keeping track what was created and what needs to be removed. But to begin with, perhpaps having some sort of nuke_it_all.yaml, which would include individual playbooks to remove a given feature. Also perhaps having a directory where to drop a playbook to clean up a given application user might have added. E.g. call the relevant helm uninstalls.
I don't know how you handle tasks, but I suppose this could be epic, and then each component might be separate task to clean up.
Also workflow doc would need section on this.
Consider for the use case an admin tries this in some real environment. And wants to remove it, and there are tons of other real company test/dev environments there, one definately can't just nuke the clusters this framework installed stuff. There needs to be a way to uninstall this so one isn't doomed after trial/error.
Here is an example of k8s module in ansible creating/deleting resource using state. All resources in Fevermap OCP CI/CD were created with ansible, so it's not the best example for this as such, but it was 3.11 ocp time.
(I mistakenly created the ticket to docs site first, so i convert that just to docs ticket for the same issue)
I am trying to get an application entry working in values-hub.yaml that points back to our Enterprise GitHub for its definition.
Here is my values-hub.yaml declaration:
clusterGroup:
applications:
cost-collector:
name: cost-collector
namespace: cost-collector
project: hub
kustomize: true
repoURL: https://github.myenterprise.com/costcollector.git
path: https://github.myenterprise.com/costcollector/overlays/dev
targetRevision: main
Performing a . pattern.sh make preview-cost-collector
works without issue and is able to pull all the artifacts from the non-vp repo and runs them successfully through the VP preview pipeline.
However, when I deploy this to the hub cluster via Argo I get the following error:
Failed to load target state: failed to generate manifest for source 1 of 1: rpc error: code = Unknown desc = Manifest generation error (cached): https://github.myenterprise.com/costcollector/overlays/dev: app path does not exist
Instructions regarding secrets values are not completed. There is step to copy the values-secret template and warning "do not commit this file". In such situation if the Validated pattern is installed using operator from OperatorHub the secret needs to be created and added to vault manually. There is no information about it and instructions for this procedure.
Is there some prerequisites missing? I am running this from RHEL8.6.
$ make install
make -f common/Makefile deploy
make[1]: Entering directory '/home/jwilms/code/multicloud-gitops'
Checking repo https://github.com/jwilms1971/multicloud-gitops.git - branch main
https://github.com/jwilms1971/multicloud-gitops.git - main exists
helm upgrade --install multicloud-gitops common/install/ -f values-global.yaml --set main.git.repoURL="https://github.com/jwilms1971/multicloud-gitops.git" --set main.git.revision=main --set global.hubClusterDomain=apps.hub-cluster-1.aws.jwilms.net
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jwilms/clusters/hub-cluster-1/auth/kubeconfig
Release "multicloud-gitops" does not exist. Installing it now.
I0718 11:29:54.700962 120604 request.go:601] Waited for 1.179609528s due to client-side throttling, not priority and fairness, request: GET:https://api.hub-cluster-1.aws.jwilms.net:6443/apis/network.operator.openshift.io/v1?timeout=32s
NAME: multicloud-gitops
LAST DEPLOYED: Mon Jul 18 11:30:00 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
make[1]: Leaving directory '/home/jwilms/code/multicloud-gitops'
make[1]: Entering directory '/home/jwilms/code/multicloud-gitops'
make -f common/Makefile vault-init
make[2]: Entering directory '/home/jwilms/code/multicloud-gitops'
common/scripts/vault-utils.sh vault_init common/pattern-vault.init
PLAY [Vault initialization] ***********************************************************************************************************************************
TASK [vault_utils : include_tasks] ****************************************************************************************************************************
included: /home/jwilms/code/multicloud-gitops/common/ansible/roles/vault_utils/tasks/pre_check.yaml for localhost
TASK [vault_utils : Check if the kubernetes python module is usable from ansible] *****************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "cmd": ["/usr/bin/python3.8", "-c", "import kubernetes"], "delta": "0:00:00.011533", "end": "2022-07-18 11:30:03.800305", "msg": "non-zero return code", "rc": 1, "start": "2022-07-18 11:30:03.788772", "stderr": "Traceback (most recent call last):\n File "", line 1, in \nModuleNotFoundError: No module named 'kubernetes'", "stderr_lines": ["Traceback (most recent call last):", " File "", line 1, in ", "ModuleNotFoundError: No module named 'kubernetes'"], "stdout": "", "stdout_lines": []}
PLAY RECAP ****************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
make[2]: *** [common/Makefile:61: vault-init] Error 2
make[2]: Leaving directory '/home/jwilms/code/multicloud-gitops'
make[1]: *** [Makefile:10: vault-init] Error 2
make[1]: Leaving directory '/home/jwilms/code/multicloud-gitops'
make: *** [Makefile:13: install] Error 2
Somewhere in the last few months of VP commits a change was made that creates a Vault unseal CronJob in the imperative namespace by default. Since we do not use vault, this job perpetually fails.
Is there a switch to turn off this behavior?
The resource itself seems to be defined at common/clustergroup/templates/imperative/unsealjob.yaml
There is a toggle on line 1 that looks promising:
{{- if eq .Values.global.secretStore.backend "vault" | default "vault" }}
But looking at common/clustergroup/values.schema.json it does not seem to support a backend parameter:
"SecretStore": {
"type": "object",
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"description": "Name of the external secret backend",
"default": "vault-backend"
},
"kind": {
"type": "string",
"description": "Type of the external secret backend",
"default": "ClusterSecretStore"
}
},
"required": [
"name",
"kind"
],
"title": "SecretsStore"
},
Should the unseal job toggle be:
{{- if eq .Values.global.secretStore.kind "vault-backend" | default "vault-backend" }}
So that end users could turn off this behavior with a change to values-global.yaml:
---
global:
pattern: multicloud-gitops
secretStore:
kind: <something-else-here>
There is currently no linting in the repository and as we have more users we will need to make sure we have consistency in formatting. PR incoming.
refering to:
Note that this is a secure way of copying the secrets when using ACM 2.5. Prior to ACM 2.5 you would have likely used a Namespace Channel, but that feature is no longer available. The ACM 2.5 doc on securely copying hub data to managed clusters is here: https://access.redhat.com/documentation/en-us/red_hat_advanced_cluster_management_for_kubernetes/2.5/html/governance/governance#protect-function
I tried to push secret to vault, and run into problem secrets are read line by line, not as yaml key: value.
Example:
secrets:
ldap:
bindPassword: xxx
bindDN: xxx
url: xxx
groupsBaseDN: xxx
usersBaseDN: xxx
ca_crt: |
-----BEGIN CERTIFICATE-----
MIIEgjCCAuqgAwIBAgIBATANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQKDAhDT09M
LkxBQjEeMBwGA1UEAwwVQ2VydGlmaWNhdGUgQXV0aG9yaXR5MB4XDTIyMDEwNTEz
...
Error:
"rc": 127, "return_code": 127, "stderr": "Failed to parse K=V data: invalid key/value pair \"CERTIFICATE-----\": format must be key=value\nsh: line 1: MIIEgjCCAuqgAwIBAgIBATANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQKDAhDT09M: command not found\nsh: line 2: LkxBQjEeMBwGA1UEAwwVQ2VydGlmaWNhdGUgQXV0aG9yaXR5MB4XDTIyMDEwNTEz: command not found\nsh: line 3: MjA1MloXDTQyMDEwNTEzMjA1MlowMzERMA8GA1UECgwIQ09PTC5MQUIxHjAcBgNV: command not found\nsh: line 4: BAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTCCAaIwDQYJKoZIhvcNAQEBBQADggGP: command not found\nsh: line 5: ADCCAYoCggGBANhvBhWj5qQWpvO6h6yDvCKhc/Nj36xZN1990L2PGL6dJU2SoOuo: No such file or directory\n
Cloning latest repo and running make preview-all
fails with:
[multicloud-gitops]$ make preview-all
make -f common/Makefile preview-all
make[1]: Entering directory '/home/user/repos/GitHub/validatedpatterns/multicloud-gitops'
Error: open /home/user/repos/GitHub/validatedpatterns/multicloud-gitops/overrides/values-Azure.yaml: no such file or directory
Error: open /home/user/repos/GitHub/validatedpatterns/multicloud-gitops/overrides/values-Azure.yaml: no such file or directory
Error: open /home/user/repos/GitHub/validatedpatterns/multicloud-gitops/overrides/values-Azure.yaml: no such file or directory
Error: open /home/user/repos/GitHub/validatedpatterns/multicloud-gitops/overrides/values-Azure.yaml: no such file or directory
Error: open /home/user/repos/GitHub/validatedpatterns/multicloud-gitops/overrides/values-Azure.yaml: no such file or directory
common/scripts/preview.sh: eval: line 79: unexpected EOF while looking for matching "' common/scripts/preview.sh: eval: line 80: syntax error: unexpected end of file common/scripts/preview.sh: eval: line 79: unexpected EOF while looking for matching
"'
common/scripts/preview.sh: eval: line 80: syntax error: unexpected end of file
common/scripts/preview.sh: eval: line 79: unexpected EOF while looking for matching `"'
common/scripts/preview.sh: eval: line 80: syntax error: unexpected end of file
make[1]: *** [common/Makefile:59: preview-all] Error 2
make[1]: Leaving directory '/home/user/repos/GitHub/validatedpatterns/multicloud-gitops'
make: *** [Makefile:12: preview-all] Error 2
It seems secrets format has changed since I last used it. I didn't see it explained anywhere. So looking around I ended up with this:
- name: ldap
vaultPrefixes:
- global
fields:
- name: bindPassword
value: foobar
onMissingValue: generate
vaultPolicy: validatedPatternDefaultPolicy
which compalains:
TASK [vault_utils : Loads secrets file into the vault of a cluster] ***************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Secret has onMissingValue set to 'generate' but has a value set"}
Value is not missing, it should just ignore that generate field. Variable name "onMissingValue" would indicate it ignores this if value is set.
But alltogether, the format of secrets file should be described in the doc. As of current, it doesn't have "value:" in example.
When running source pattern.sh make preview-all
I am getting Error: Chart.yaml file is missing from certain applications.
I expect to see resource yaml output for each application declared in values-hub.yaml.
This occurs for any applications defined with kustomize: true
.
Since I was getting a bunch of these errors without insight as to origin I added a debug stmt to common/scripts/preview-all.sh line 13:
printf "# ------ Switching to app $app in cluster $cluster -------"
Resulting output from preview-all now looks like this.
# ------ Switching to app compliance-operator in cluster hub -------
Error: Chart.yaml file is missing
For this specific error my values-hub.yaml looks like this:
clusterGroup:
name: hub
isHubCluster: true
subscriptions:
compliance-operator:
name: compliance-operator
namespace: openshift-compliance
channel: stable
applications:
compliance-operator:
name: compliance-operator
namespace: openshift-compliance
project: ekho
kustomize: true
path: charts/all/compliance-operator/overlays/default
This page, https://redhat-gitops-patterns.io/multicloud-gitops/getting-started/
at the bottom links to https://hybrid-cloud-patterns.io/multicloud-gitops/managed-cluster/
as "For instructions on deploying the edge, please read the following document."
That target has broken images, but more importantly is a different site with no navigation back to the first site.
FIX
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.