Comments (15)
khuedoan
commented on June 11, 2024
Hmmm I can't reproduce this in my environment, can you please try again with this flag in the apply scripts to gather more info:
#!/bin/sh
+ set -exfrom homelab.
brimdor
commented on June 11, 2024
make[1]: Leaving directory '/home/brimdor/homelab/metal'
make -C bootstrap
make[1]: Entering directory '/home/brimdor/homelab/bootstrap'
kubectl create namespace argocd --dry-run=client --output=yaml \
| kubectl apply -f -
namespace/argocd created
cd argocd && ./apply.sh
+ helm template --dependency-update --include-crds --namespace argocd argocd .
+ kubectl apply -n argocd -f -
error: error validating "STDIN": error validating data: invalid object to validate; if you choose to ignore these errors, turn validation off with --validate=false
make[1]: *** [Makefile:11: argocd] Error 1
make[1]: Leaving directory '/home/brimdor/homelab/bootstrap'
make: *** [Makefile:18: bootstrap] Error 2
from homelab.
brimdor
commented on June 11, 2024
Now this is happening
ingress.networking.k8s.io/argocd-server created
resource mapping not found for name: "argocd-application-controller" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-repo-server" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-server" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-redis" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
make[1]: *** [Makefile:11: argocd] Error 1
make[1]: Leaving directory '/home/brimdor/homelab/bootstrap'
make: *** [Makefile:18: bootstrap] Error 2
from homelab.
brimdor
commented on June 11, 2024
I removed set -ex from both apply.sh files and I am able to get past the latest error.
from homelab.
brimdor
commented on June 11, 2024
I am regretting rebuilding to help troubleshoot this issue. Now Cloudflare is not populating the DNS records. It's only setting up the tunnel. When it runs the bootstrap/argocd/apply.sh, I get errors regarding the CRDs. The server is completely down and I have spent approx 10 hours trying to get it back up and running.
resource mapping not found for name: "argocd-application-controller" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-repo-server" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-server" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
resource mapping not found for name: "argocd-redis" namespace: "" from "STDIN": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
from homelab.
khuedoan
commented on June 11, 2024
Thanks for taking the time to rebuild, the ServiceMonitor warning is safe to ignore.
Do you have the logs for external-dns pod (which is responsible for updating DNS records)?
from homelab.
brimdor
commented on June 11, 2024
I don't. This is all new to me so I failed to collect certain items or even
know where to look when things go wrong. I eventually got the environment
back when I ran each piece manually.
make metal, then make bootstrap, then make external, etc
…On Sat, Jan 21, 2023, 22:59 Khue Doan ***@***.***> wrote:
Thanks for taking the time to rebuild, the ServiceMonitor warning is safe
to ignore.
Do you have the logs for external-dns pod (which is responsible for
updating DNS records)?
—
Reply to this email directly, view it on GitHub
<#102 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJFVD7GL77IDYB6GJID24TWTS5D3ANCNFSM6AAAAAAT5MGP4U>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
from homelab.
khuedoan
commented on June 11, 2024
I disabled ServiceMonitor on first apply to avoid the confusion caused by the warning cd39632
from homelab.
khuedoan
commented on June 11, 2024
You can use the k9s command included in the tools container to list the pods (it has vim bindings).
from homelab.
brimdor
commented on June 11, 2024
FYI: I had to rebuild and I get this same issue even with the latest pull of your respository.
I have some hardware coming in couple days that will allow me to do more troubleshooting without taking down my entire server build every time. Unfortunately, the issue is persistent and it seems if I run 'make' again after that, it goes through no problem. Same as before.
from homelab.
khuedoan
commented on June 11, 2024
You can test without additional hardware by using a virtual machine, here's what I usually do:
- Create a new VirtualBox VM with UEFI boot and a bridged network
- Create a
stag.ymlinventory - Run
make - Boot to the boot menu and select network boot, it will boot and install normally
from homelab.
brimdor
commented on June 11, 2024
It seems to be a time issue. The time for the namespace to be created before it can move forward. I added 'sleep 10' then 'sleep 30' then 'sleep 60' to line 7 in bootstrap/Makefile, tabbed in line with the kubectl command and it worked when I did sleep 60. 30 and 10 were not long enough wait times.
from homelab.
khuedoan
commented on June 11, 2024
Hmmm that's strange, there's a wait command in the install script that supposed to wait for argocd to complete before continuing with bootstrap:
homelab/bootstrap/argocd/apply.sh
Lines 16 to 18 in 4b69ec2
Do you have any config related to parallel jobs (for example make -j 4)?
from homelab.
brimdor
commented on June 11, 2024
The only changes are from make configure and the sleep 60. Everything else is out of the box experience that you have in your repo.
from homelab.
brimdor
commented on June 11, 2024
excuse the sleep 60 option. I was able to fix it using a condition that allows it to fix it if the conditions aren't met. Here is the updated bootstrap/Makefile:
.POSIX:
default: namespace argocd root
namespace:
kubectl create namespace argocd --dry-run=client --output=yaml | kubectl apply -f -
.PHONY: argocd
argocd:
kubectl get customresourcedefinitions.apiextensions.k8s.io applications.argoproj.io applicationsets.argoproj.io > /dev/null || kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml && cd argocd && ./apply.sh
.PHONY: root
root:
cd root && ./apply.sh
And here are the results showing a positive outcome:
[nix-shell:/home/brimdor/homelab]# make bootstrap
make -C bootstrap
make[1]: Entering directory '/home/brimdor/homelab/bootstrap'
kubectl create namespace argocd --dry-run=client --output=yaml | kubectl apply -f -
namespace/argocd created
kubectl get customresourcedefinitions.apiextensions.k8s.io applications.argoproj.io applicationsets.argoproj.io > /dev/null || kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml && cd argocd && ./apply.sh
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "applications.argoproj.io" not found
Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "applicationsets.argoproj.io" not found
customresourcedefinition.apiextensions.k8s.io/applications.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/applicationsets.argoproj.io created
customresourcedefinition.apiextensions.k8s.io/appprojects.argoproj.io created
serviceaccount/argocd-application-controller created
serviceaccount/argocd-applicationset-controller created
serviceaccount/argocd-dex-server created
serviceaccount/argocd-notifications-controller created
serviceaccount/argocd-redis created
serviceaccount/argocd-repo-server created
serviceaccount/argocd-server created
role.rbac.authorization.k8s.io/argocd-application-controller created
role.rbac.authorization.k8s.io/argocd-applicationset-controller created
role.rbac.authorization.k8s.io/argocd-dex-server created
role.rbac.authorization.k8s.io/argocd-notifications-controller created
role.rbac.authorization.k8s.io/argocd-server created
clusterrole.rbac.authorization.k8s.io/argocd-application-controller created
clusterrole.rbac.authorization.k8s.io/argocd-server created
rolebinding.rbac.authorization.k8s.io/argocd-application-controller created
rolebinding.rbac.authorization.k8s.io/argocd-applicationset-controller created
rolebinding.rbac.authorization.k8s.io/argocd-dex-server created
rolebinding.rbac.authorization.k8s.io/argocd-notifications-controller created
rolebinding.rbac.authorization.k8s.io/argocd-redis created
rolebinding.rbac.authorization.k8s.io/argocd-server created
clusterrolebinding.rbac.authorization.k8s.io/argocd-application-controller created
clusterrolebinding.rbac.authorization.k8s.io/argocd-server created
configmap/argocd-cm created
configmap/argocd-cmd-params-cm created
configmap/argocd-gpg-keys-cm created
configmap/argocd-notifications-cm created
configmap/argocd-rbac-cm created
configmap/argocd-ssh-known-hosts-cm created
configmap/argocd-tls-certs-cm created
secret/argocd-notifications-secret created
secret/argocd-secret created
service/argocd-applicationset-controller created
service/argocd-dex-server created
service/argocd-metrics created
service/argocd-notifications-controller-metrics created
service/argocd-redis created
service/argocd-repo-server created
service/argocd-server created
service/argocd-server-metrics created
deployment.apps/argocd-applicationset-controller created
deployment.apps/argocd-dex-server created
deployment.apps/argocd-notifications-controller created
deployment.apps/argocd-redis created
deployment.apps/argocd-repo-server created
deployment.apps/argocd-server created
statefulset.apps/argocd-application-controller created
networkpolicy.networking.k8s.io/argocd-application-controller-network-policy created
networkpolicy.networking.k8s.io/argocd-applicationset-controller-network-policy created
networkpolicy.networking.k8s.io/argocd-dex-server-network-policy created
networkpolicy.networking.k8s.io/argocd-notifications-controller-network-policy created
networkpolicy.networking.k8s.io/argocd-redis-network-policy created
networkpolicy.networking.k8s.io/argocd-repo-server-network-policy created
networkpolicy.networking.k8s.io/argocd-server-network-policy created
Error from server (NotFound): ingresses.networking.k8s.io "argocd-server" not found
error: error validating "STDIN": error validating data: invalid object to validate; if you choose to ignore these errors, turn validation off with --validate=false
customresourcedefinition.apiextensions.k8s.io/applications.argoproj.io condition met
customresourcedefinition.apiextensions.k8s.io/applicationsets.argoproj.io condition met
cd root && ./apply.sh
Error from server (NotFound): namespaces "gitea" not found
applicationset.argoproj.io/bootstrap created
applicationset.argoproj.io/system created
applicationset.argoproj.io/platform created
applicationset.argoproj.io/apps created
make[1]: Leaving directory '/home/brimdor/homelab/bootstrap'
from homelab.
Related Issues (20)
- Failed smoke tests HOT 2
- Random OutOfSync with ArgoCD server-side apply HOT 1
- 「automatic_upgrade: Install packages for automatic upgrade」error: Could not import the dnf python module using /usr/bin/python3 HOT 3
- zerotier HOT 9
- Unable to access via internet / cloudflared tunnel HOT 3
- DNS config was wrong except `argocd.xx.xx` HOT 6
- Terraform Error during make HOT 9
- minor and major k3 api not working issue HOT 10
- Nix image update breaks 'make tools' HOT 2
- Support IPv6 HOT 3
- ExternalSecret could not get secret data from provider HOT 3
- cloudflare module error in make process and cloudflared error HOT 9
- new fresh deployment only those pods run HOT 3
- Upon rebuilding, first node in inventory.yaml doesn't rejoin existing cluster. HOT 3
- Switching chart provider HOT 7
- vault version HOT 2
- Question: Partitioning HOT 2
- Tekton doesn't seem to work or be configured by default HOT 3
- Development sandbox make fails HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from homelab.