projectsyn / component-cert-manager Goto Github PK

Context

We should upgrade to the latest cert-manager version.

Upgrading to cert-manager 1.6 is arguable a breaking change, as it deprecates old cert-manager resources. There could be issues for clusters that ran cert-manager < 1.0.

Context

In #134 we ensure that cert-manager metrics are scraped by Prometheus on OpenShift 4 and clusters using component prometheus. However, the cert-manager Helm chart version that we're using doesn't deploy any alerting rules.

We should invest some time to determine what metrics are exposed by cert-manager and define reasonable alerting rules for cert-manager.

Out of scope

• Update cert-manager
• Update helm chart version

Task deliverables

• Component deploys some actionable alerting rules
• Runbooks for the alerts are part of the component documentation

The current cert-manager version 1.5.4 seems not being compatible with IngressClasses.

cert-manager/cert-manager#4537

Steps to Reproduce the Problem

1. cert-manager/cert-manager#4537

Actual Behavior

The component version 2.5 uses v1.5.4.

Expected Behavior

Upgraded to v1.5.5+, v1.6.2+ or 1.7.1+. https://cert-manager.io/docs/installation/upgrading/ingress-class-compatibility/

Context

Most if not all DNS01 solvers require some sort of credentials. Those credentials are supposed to be put into a Kubernetes Secret which then gets referenced in the solvers section of the (Cluster)Issuer. In order to do so, the component should allow to confiugre a list of arbitrary Secretes which then can be referenced in solvers.

cert-manager:
  secrets:
    acmedns:
      stringData:
        acmedns.json: ?{vaultkv:${cluster:tenant}/${cluster:name}/acmedns}

  solvers:
    dns01:
      acmeDNS:
        accountSecretRef:
          name: acmedns
            key: acmedns.json
          host: auth.example.com

Alternatives

Allow the credentials to be configured as part of the solver data structure and convert it to a Secret within Jsonnet.
This favours the users perspective (simpler to configure). From a maintainers prespective, this is less favourlable as we would have to cover each and every solver (each does things a bit different) and we would have to actively support new ones.

AKS is adding an extra matchExpression to ValidatingWebhookConfiguration.

https://docs.microsoft.com/en-us/azure/aks/faq#can-admission-controller-webhooks-impact-kube-system-and-internal-aks-namespaces

To protect the stability of the system and prevent custom admission controllers from impacting internal services in the kube-system, namespace AKS has an Admissions Enforcer, which automatically excludes kube-system and AKS internal namespaces. This service ensures the custom admission controllers don't affect the services running in kube-system.

The cert-manager helm chart object is patched continuously by the AKS K8s Admissions Enforcer.

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: cert-manager-webhook
webhooks:
  - name: webhook.cert-manager.io
    namespaceSelector:
      matchExpressions:
        - key: control-plane
          operator: DoesNotExist

Currently the helm chart does not support configuring it https://github.com/jetstack/cert-manager/blob/master/deploy/charts/cert-manager/templates/webhook-validating-webhook.yaml#L27 statically for AKS.

Steps to Reproduce the Problem

1. Take any AKS cluster
2. Configure the component cert-manager v2.0.0
3. Check ArgoCD for unsychronized items

Actual Behavior

ArgoCD is continuously removing the cert-manager application matchExpression. The AKS Admissions Enforcer is continuously adding it.

Expected Behavior

The additional matchExpression can be configured.

Context

The current used version 0.15.1 does not support the API version cert-manager.io/v1. This causes as long as we don't upgrade, all new objects use an outdated API version and have to be migrated later. Therefore it would make much sense to rise the version to 1.0+, which makes the current API version available cert-manager/cert-manager#3177.

Alternatives

I think there is not an alternative to it long term.

This issue provides visibility into Renovate updates and their statuses. Learn more

This repository currently has no open or pending branches.

• Check this box to trigger a request for Renovate to run again on this repository

Context

To use DNS01 validation with an RFC-2136 compliant nameserver, a secret holding the TSIG key has to be created (see: https://cert-manager.io/docs/configuration/acme/dns01/rfc2136/#configuration-step-2---set-up-cert-manager).
Further, a check like the acme-dns check should be added using nsupdate (see: https://cert-manager.io/docs/configuration/acme/dns01/rfc2136/#using-nsupdate).

Alternatives

Not providing full integration for DNS validation using RFC-2136 in this component

This creates a ServiceMonitor but the annotation SYNMonitoring=main is not set on the namespace and therefore will not get picked up by prometheus.

Actual Behavior

Annotation not set.

Expected Behavior

Annotation SYNMonitoring=main set.

When updating component-cert-manager from v3.3.0 to v3.3.1 the deployment gets stuck because of the introduction of prorityClasses in #149.

  Warning  FailedCreate  13m (x19 over 35m)  replicaset-controller  Error creating: insufficient quota to match these scopes: [{PriorityClass In [system-node-critical system-cluster-critical]}]

Similar issues in other projects:

• cilium/cilium#13852
• open-policy-agent/gatekeeper#1120

By default there is a ResourceQuata for namespace "kube-system":

apiVersion: v1
kind: ResourceQuota
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
  name: gcp-critical-pods
  namespace: kube-system
spec:
  hard:
    pods: 1G
  scopeSelector:
    matchExpressions:
    - operator: In
      scopeName: PriorityClass
      values:
      - system-node-critical
      - system-cluster-critical

I think having a similar quota in the namespace where cert-manager is deployed should solve this issue.

Steps to Reproduce the Problem

1. Update cert-manager to v3.3.1 on a GKE cluster

Actual Behavior

Rollout gets stuck

Expected Behavior

Successful update of cert-manager

Context

The current version of v1.4 still uses deprecated APIs unavailable in K8s 1.22. Please upgrade to >= v1.5 to support K8s 1.22. "If you intend to upgrade to Kubernetes 1.22, you must upgrade to cert-manager 1.5." https://cert-manager.io/docs/release-notes/release-notes-1.5/#api-deprecation

Alternatives

N/A

Context

In some environments it is required for outgoing connections to pass through an HTTP proxy.
The component should allow the configuration of the helm chart HTTP proxy variables (see: https://github.com/jetstack/cert-manager/blob/master/deploy/charts/cert-manager/values.yaml#L171).

Alternatives

Using the extraEnv variable of the chart with something like projectsyn/component-steward@1c6f4fc

Context

cert-manager should configure default requests and limits to have a better QoS class than BestEffort. On some clusters there are default resource requests and limits (usually 250/250), which cert-manager doesn't consume.

Same as projectsyn/component-argocd#41

Alternatives

n/a