janus-idp / backstage-operator Goto Github PK

License: Apache License 2.0

Dockerfile 9.16% Makefile 47.52% Go 43.31%

backstage-operator's Introduction

Hybrid Helm/Golang operator for Backstage prototype

Warning

This project is no longer actively maintained. The repository has been archived, and its contents are available for historical and reference purposes only.

Reconciler logic: https://github.com/operator-framework/helm-operator-plugins/blob/main/pkg/reconciler/reconciler.go Hybrid operators lacks documentation, see:

Setup

make init
make install

Run

Containerized

export IMG=quay.io/<foo>/<bar>:latest
make podman-build
make podman-push
make deploy

Inside OpenShift cluster using OperatorHub

1. build and push all images

VERSION=0.0.1-dev1 IMG=quay.io/<username>/janus-operator:$VERSION IMAGE_TAG_BASE=quay.io/<username>/janus-operator make build-push-all

2. add CatalogSource to OperatorHub

./bin/kustomize build config/operator-hub/ | oc apply -f -

3. Install RHDH-dev from OperatorHub using OpenShift Web Console

Locally

export WATCH_NAMESPACE=baz
make run

in VSCode:

Edit namespace in .vscode/launch.json
CTRL+SHIFT+D, run Launch Backstage Operator

Known issues

After first sync/install (before any upgrade call or reconcile), we need to set .upstream.postgresql.auth.existingSecret

Extra features on top of the Helm chart

global.clusterRouterBase is automaticaly populated with the cluster's ingress domain.

backstage-operator's People

Contributors

Stargazers

Watchers

Forkers

nickboldt zaperex kadel tiagodolphine schultzp2020 cgruver pataknight acornett21 tumido kim-tsao

backstage-operator's Issues

Deprecation notice: Sqlite-based catalogs

while building catalog image with make catalog-build

WARN[0000] DEPRECATION NOTICE:
Sqlite-based catalogs and their related subcommands are deprecated. Support for
them will be removed in a future release. Please migrate your catalog workflows
to the new file-based catalog format.

Fix operator reconcile loop - pass existing PostgreSQL credentals

Backstory

Backstage chart uses bitnami/postgresql chart as a DB provider. When this chart is initially installed it generates credentials for the new database. New credentials are generated, passed to the database pod, and also stored in a Kubernetes Secret. This Secret is in turn mounted to the Backstage pod. Backstage then uses these credentials to access the DB.

When a helm upgrade is triggered (the operator does reconcile through helm upgrade), given credentials are regenerated, new password is created for the secret. However, the DB is already seeded with the original credentials so the new password is ignored by the DB. Unfortunately, that's not the case for the Backstage pod, which receives the new credentials which are invalid for the DB.

Solution

Implement a logic that would check for DB credentials Secret in the target namespace and if found, make it enforce this Secret as .upstream.postgresql.auth.existingSecret value. This would ensure new credentials are not generated and "old" password remains in use by the Backstage pod.

Operator CrashLoopBackOff due to Liveness probe error

Describe the bug

In the events we see the liveness probe fail:

Liveness probe error: Get "http://10.128.2.97:8081/healthz": dial tcp 10.128.2.97:8081: connect: connection refused body:

This is the operator logs:

1
I1012 15:04:24.582908       1 request.go:682] Waited for 1.032624601s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/submariner.io/v1?timeout=32s
2
2023-10-12T15:04:33Z	INFO	controller-runtime.metrics	Metrics server is starting to listen	{"addr": "127.0.0.1:8080"}
3
I1012 15:04:34.915972       1 request.go:682] Waited for 1.03513177s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/cluster.open-cluster-management.io/v1alpha1?timeout=32s
4
2023-10-12T15:04:44Z	INFO	controllers.Helm	Watching resource	{"group": "charts.janus-idp.io", "version": "v1alpha1", "kind": "Backstage"}
5
2023-10-12T15:04:44Z	INFO	setup	configured watch	{"gvk": "charts.janus-idp.io/v1alpha1, Kind=Backstage", "chartPath": "helm-backstage/charts/backstage", "maxConcurrentReconciles": 10, "reconcilePeriod": "1m0s"}
6
2023-10-12T15:04:44Z	INFO	setup	starting manager
7
2023-10-12T15:04:44Z	INFO	Starting server	{"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"}
8
2023-10-12T15:04:44Z	INFO	Starting server	{"kind": "health probe", "addr": "[::]:8081"}
9
I1012 15:04:44.171868       1 leaderelection.go:248] attempting to acquire leader lease openshift-operators/rhdh-operator...
10
I1012 15:05:01.818264       1 leaderelection.go:258] successfully acquired lease openshift-operators/rhdh-operator
11
2023-10-12T15:05:01Z	DEBUG	events	rhdh-operator-7b9b9d8cb8-9m428_4eaec52b-e57a-43c4-a4fb-f59e78b20421 became leader	{"type": "Normal", "object": {"kind":"Lease","namespace":"openshift-operators","name":"rhdh-operator","uid":"fe6d8060-900d-41fc-a23b-0988b039bf51","apiVersion":"coordination.k8s.io/v1","resourceVersion":"4475768571"}, "reason": "LeaderElection"}
12
2023-10-12T15:05:01Z	INFO	Starting EventSource	{"controller": "backstage-controller", "source": "kind source: *unstructured.Unstructured"}
13
2023-10-12T15:05:01Z	INFO	Starting EventSource	{"controller": "backstage-controller", "source": "kind source: *v1.Secret"}
14
2023-10-12T15:05:01Z	INFO	Starting Controller	{"controller": "backstage-controller"}

Expected Behavior

Pod starts and runs without liveness checks failing.

What are the steps to reproduce this bug?

Create Subscription for operator
Wait for operator pod running
CrashLoopBackOff after several restarts due to liveness probe

Versions of software used and environment

OCP 4.13.14
rhdhoperator.v1.0.0

Override used values for deployed image

What do you want to improve?

We want to enforce certain image repo/name/tag in the operator. So no matter what values the user inputs in the CR (aka values.yaml), the image used always resolves to given image.

What is the current behavior?

User is able to change the image, otherwise the helm default values.yaml is used

What is the new behavior?

Operator consumes configuration via environment variables and overrides the values

Add repo to sonarcloud for scanning

What needs to be done?

Although there isn't much code in this repo, just the main.go as far as I can see, we should get this scanned for coverage.

Operator doesn't ignore Helm tests on deployment/reconcile

It seems like the operator is attempting to install/deploy helm tests on reconcile, please

2023-09-16T16:35:51Z DEBUG controllers.Helm Starting install
2023-09-16T16:35:53Z DEBUG controllers.Helm Install failed
2023-09-16T16:35:53Z ERROR Reconciler error {"controller": "backstage-controller", "object": {"name":"backstage-sample","namespace":"openshift-operators"}, "namespace": "openshift-operators", "name": "backstage-sample", "reconcileID": "eabef921-feae-4fd9-bc27-68fd486b198e", "error": "template: backstage/templates/tests/test-connection.yaml:4:12: executing \"backstage/templates/tests/test-connection.yaml\" at <include \"common.names.fullname\" .>: error calling include: template: no template \"common.names.fullname\" associated with template \"gotpl\""}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/remote-source/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/remote-source/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/remote-source/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235
2023-09-16T16:35:53Z DEBUG controllers.Helm Reconciliation triggered {"backstage": "openshift-operators/backstage-sample"}
2023-09-16T16:35:53Z INFO controllers.Helm map[global:map[clusterRouterBase:lol host:] route:map[annotations:map[] enabled:%!s(bool=true) host:{{ .Values.global.host }} path:/ tls:map[caCertificate: certificate: destinationCACertificate: enabled:%!s(bool=true) insecureEdgeTerminationPolicy:Redirect key: termination:edge] wildcardPolicy:None] upstream:map[backstage:map[appConfig:map[app:map[baseUrl:https://{{- include "janus-idp.hostname" . }}] backend:map[baseUrl:https://{{- include "janus-idp.hostname" . }} cors:map[origin:https://{{- include "janus-idp.hostname" . }}] database:map[connection:map[password:${POSTGRESQL_ADMIN_PASSWORD} user:postgres]]]] command:[] extraEnvVars:[map[name:POSTGRESQL_ADMIN_PASSWORD valueFrom:map[secretKeyRef:map[key:postgres-password name:{{ .Release.Name }}-postgresql]]]] image:map[registry:quay.io repository:janus-idp/backstage-showcase tag:latest]] ingress:map[host:{{ .Values.global.host }}] nameOverride:backstage postgresql:map[auth:map[secretKeys:map[adminPasswordKey:post...

Operator is deployment-ready and reconciles changes properly

Goal

Bring the operator to a releasable, usable state where it offers a comparable stable experience as installing via helm chart.

Acceptance criteria

Operator is able to deploy Backstage
Operator can reconcile changes
Operator can enforce Janus IDP image
Operator can enforce proper routing

Requirements

Test plan
Documentation

Issues in Epic

Notes

Additional context
Add any other context or screenshots about the epic here.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

chore(deps): update helm-backstage digest to 31cb686
fix(deps): update module github.com/stretchr/testify to v1.8.4
fix(deps): update kubernetes packages to v0.28.3 (k8s.io/apimachinery, k8s.io/client-go)
fix(deps): update module github.com/go-logr/logr to v1.3.0
fix(deps): update module helm.sh/helm/v3 to v3.13.1
fix(deps): update module sigs.k8s.io/controller-runtime to v0.16.3
Click on this checkbox to rebase all open PRs at once

Detected dependencies

dockerfile

Dockerfile

registry.access.redhat.com/ubi9/go-toolset 1.19.13-4.1697647145

registry.access.redhat.com/ubi9/ubi-micro 9.2-15.1696515526

git-submodules

.gitmodules

helm-backstage main@a02298470db63d63019946dd4ccc92550da47ba9

github-actions

.github/workflows/add-to-project.yaml

.github/workflows/pr-checks.yaml

actions/checkout v4

actions/setup-go v4

azure/setup-helm v3

gomod

go.mod

go 1.19

github.com/go-logr/logr v1.2.3

github.com/operator-framework/helm-operator-plugins v0.0.11

github.com/stretchr/testify v1.8.1

helm.sh/helm/v3 v3.11.1

k8s.io/apimachinery v0.26.0

k8s.io/client-go v0.26.0

sigs.k8s.io/controller-runtime v0.14.1

kustomize

config/operator-hub/kustomization.yaml

Check this box to trigger a request for Renovate to run again on this repository

Provide values override for `global.clusterRouterBase`

In the helm chart install instructions we require the following step:

https://github.com/janus-idp/helm-backstage/tree/main/charts/backstage#installing-from-the-chart-repository

When installed on OpenShift via this operator, this should be automated:

Make the operator fetch oc get ingresses.config/cluster -o jsonpath={.spec.domain}
Provide this via reconciler.WithOverrideValues

Please make the operator to always enforce this value to be in sync with the cluster - The operator doesn't need to watch the ingresses.config/cluster resource, but it should fetch the correct value on every helm upgrade.

unable to use operator in-cluster (x509: certificate signed by unknown authority)

Describe the bug

When running operator in cluster I get the following errors

 status:
    conditions:
    - lastTransitionTime: "2023-10-13T13:38:01Z"
      status: "False"
      type: Deployed
    - lastTransitionTime: "2023-10-13T13:38:01Z"
      status: "True"
      type: Initialized
    - lastTransitionTime: "2023-10-13T13:38:26Z"
      message: |-
        values don't meet the specifications of the schema(s) in the following chart(s):
        upstream:
        Get "https://raw.githubusercontent.com/bitnami/charts/main/bitnami/postgresql/values.schema.json": x509: certificate signed by unknown authority
      reason: ReconcileError
      status: "True"
      type: Irreconcilable
    - lastTransitionTime: "2023-10-13T13:38:01Z"
      message: |-
        values don't meet the specifications of the schema(s) in the following chart(s):
        upstream:
        Get "https://raw.githubusercontent.com/bitnami/charts/main/bitnami/postgresql/values.schema.json": x509: certificate signed by unknown authority
      reason: InstallError
      status: "True"
      type: ReleaseFailed

What are the steps to reproduce this bug?

build and push operator, bundle and catalog images
add CatalogSource
create a backstage instance using sample

Versions of software used and environment

tested on crc and 4.13 cluster on GCP both with the same problem

Support air-gapped environments

What do you want to improve?

Operator should support air-gapped environment (besides the obvious - bring our own image)

What is the current behavior?

In airgapped environments, operator complains that it can't fetch JSON schemas for helm.

What is the new behavior?

Operator is capable to process any schema validation locally and deploy all required resources.

Reconciliation initiated without a valid cause

Describe the bug

It looks like the operator is stuck in some kind of reconcile loop

2023-09-08T12:53:58+02:00	DEBUG	controllers.Helm	Reconciliation triggered	{"backstage": "backstage-operator/backstage-sample"}
2023-09-08T12:53:59+02:00	DEBUG	predicate	Reconciling due to dependent resource update	{"name": "openshift-master-controllers", "namespace": "openshift-controller-manager", "apiVersion": "v1", "kind": "ConfigMap"}
2023-09-08T12:53:59+02:00	DEBUG	predicate	Reconciling due to dependent resource update	{"name": "cluster-policy-controller-lock", "namespace": "openshift-kube-controller-manager", "apiVersion": "v1", "kind": "ConfigMap"}
2023-09-08T12:54:01+02:00	DEBUG	controllers.Helm	preparing upgrade for backstage-sample
2023-09-08T12:54:06+02:00	DEBUG	controllers.Helm	performing update for backstage-sample
2023-09-08T12:54:06+02:00	DEBUG	controllers.Helm	dry run for backstage-sample
2023-09-08T12:54:06+02:00	DEBUG	predicate	Reconciling due to dependent resource update	{"name": "backstage-sample", "namespace": "backstage-operator", "apiVersion": "apps/v1", "kind": "Deployment"}
2023-09-08T12:54:06+02:00	DEBUG	predicate	Reconciling due to dependent resource update	{"name": "backstage-sample", "namespace": "backstage-operator", "apiVersion": "apps/v1", "kind": "Deployment"}
2023-09-08T12:54:06+02:00	INFO	controllers.Helm	Release reconciled	{"backstage": "backstage-operator/backstage-sample", "name": "backstage-sample", "version": 200}

Expected Behavior

Shouldn't perform reconciliation if there are no changes made to Backstage CR or related resources

Add generic github issue templates to the backstage-operator repository

Description

Add generic github issue templates to the backstage-operator repository to adhere to a standard generic issue template for the janus-idp organization.

Add workflow to automatically add newly created issues in backstage-operator into the github project

What needs to be done?

Add workflow to automatically add newly created issues in the backstage-operator repository into the organization project

janus-idp / backstage-operator Goto Github PK

backstage-operator's Introduction

Hybrid Helm/Golang operator for Backstage prototype

Setup

Run

Containerized

Inside OpenShift cluster using OperatorHub

1. build and push all images

2. add CatalogSource to OperatorHub

3. Install RHDH-dev from OperatorHub using OpenShift Web Console

Locally

in VSCode:

Known issues

Extra features on top of the Helm chart

backstage-operator's People

Contributors

Stargazers

Watchers

Forkers

backstage-operator's Issues

Backstory

Solution

Describe the bug

Expected Behavior

What are the steps to reproduce this bug?

Versions of software used and environment

What do you want to improve?

What is the current behavior?

What is the new behavior?

What needs to be done?

Goal

Acceptance criteria

Requirements

Issues in Epic

Notes

Open

Detected dependencies

Describe the bug

What are the steps to reproduce this bug?

Versions of software used and environment

What do you want to improve?

What is the current behavior?

What is the new behavior?

Describe the bug

Expected Behavior

Description

What needs to be done?

Recommend Projects

Recommend Topics

Recommend Org