Git Product home page Git Product logo

kapp-controller's People

Contributors

100mik avatar aaronshurley avatar benmoss avatar carvel-bot avatar clebs avatar conzetti avatar cppforlife avatar danielhelfand avatar dennisdenuto avatar dependabot[bot] avatar ewrenn8 avatar gcapizzi avatar gcheadle-vmware avatar imusmanmalik avatar jeremy-boyle avatar joaopapereira avatar joe-kimmel-vmw avatar kumaritanushree avatar mamachanko avatar manning-ncsa avatar neil-hickey avatar phil9909 avatar pivotaljohn avatar praveenrewar avatar prembhaskal avatar rcmadhankumar avatar rohitagg2020 avatar sethiyash avatar thomasvitale avatar vijaykatam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kapp-controller's Issues

optional paths in template steps.

Describe the problem/challenge you have
Right now every path under ytt or similar template steps in the App CR is required, and the App CR will fail if it's not present:

  template:
    error: 'Templating dir: exit status 1'
    exitCode: 1
    stderr: |
      ytt: Error: Checking file '/etc/kappctrl-mem-tmp/kapp-controller-fetch-template-deploy695734710/config/aws': lstat /etc/kappctrl-mem-tmp/kapp-controller-fetch-template-deploy695734710/config/aws: no such file or directory
    updatedAt: "2020-12-14T16:27:39Z"

Describe the solution you'd like
We have a standardized way we write out App CRs, providing an easy to override mechanism for different environments. Not all App CRs need that overriding capability, but having the folders there reduces complexity. If there was a way to say "this path is optional, don't fail if it's not there, just proceed without it" that would reduce the need to populate a bunch of folders with a .gitkeep file or similar.

Anything else you would like to add:
Not a high priority, as almost 99% of our apps do need some type of configuration per environment, just wanted to track it for posterity.

introduce AppDefinition CRD

AppDefinition allows to expose App creation as a dedicated CRD so that "configuration details" are hidden from the App creator. For example:

AppDefinition named my-app specifies how to create MyApp/v1alpha1 by defining App template (represented via App CRD's spec section). Users that want to create new instance of MyApp use create MyApp CR and fill it out with data values (if any required).

apiVersion: kappctrl.k14s.io/v1alpha1
kind: AppDefinition
metadata:
  name: myapp
spec:
  crd:
    apiGroup: apps.co.com
    version: v1alpha1
    name: MyApp
  defaults: # <-- ytt data values?
    ...
  template:
    spec: # App CRD spec
      fetch:
      - git: ...
      template:
      - ytt: ...
      deploy:
      - kapp: ...
apiVersion: apps.co.com/v1alpha1
kind: MyApp
metadata:
  name: app1
  namespace: apps
spec:
  host: blah.com
status:
  conditions:
  - type: Reconciling
  - type: ReconcileSucceeded
  - type: ReconcileFailed
    reason: Invalid
    message: ...

Support apiextensions.k8s.io/v1 App CRD

When using kubectl 1.19.1 to install kapp-controller in a Kubernetes v1.19.1 cluster, we got this warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition. Using kubectl 1.18.x doesn’t get the warning.

Shall we provide the apiextensions.k8s.io/v1 App CRD? Thanks.

Reference: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#create-a-customresourcedefinition

Hashicorp Vault Templating

Describe the problem/challenge you have
I would like to use Vault to provide my secrets for my YTT templates. To achieve this, one can encrypt the secrets YAML with the Vault Transit engine and store in their repository.

Describe the solution you'd like
A templater in kapp controller that supports Hashicorp Vault's transit engine to decrypt files similar to the SOPS templater. This way I can decrypt secrets, template with ytt, and deploy with kapp.

Additional Details
It is also possible to use Vault to inject secrets, you just would not be able to use those secrets with ytt. See here and examples

Make `kubectl explain` work with kapp-controller CRDs

The kubectl explain command is a great way to get documentation about resources.

Currently, the App CRD does not provide documentation via kubectl explain:

$ kubectl explain app.spec
KIND:     App
VERSION:  kappctrl.k14s.io/v1alpha1

DESCRIPTION:
     <empty>

Docker powered templates

It would be nice to be able to run any docker image as a template engine.

Configuration may looks like this:

# ...
spec:
  # ...
  template:
    - docker:
        image: my-renderer:v1
        values:
          some-value: 123
        valuesFrom:
          - secretRef:
              name: secret-name
          - configMapRef:
              name: configmap-name

Have you considered something like this?

Update module name to reflect new repository

With the project being moved to a new location, I don't believe you will be able to do a go get github.com/vmware-tanzu/carvel-kapp-controller. It'll fail because the module declared in go.mod is github.com/k14s/kapp-controller. In the short term, folks can still do a go get github.com/k14s/kapp-controller. Perhaps this could be highlighted in the readme.

sops templater does not support encrypted private key

Hi,

we tried out the sops approach according to your documentation but got the following error:

template:
          error: 'Building config paths: Generating secring.gpg: Serializing pk: openpgp:
            invalid argument: unknown private key type'
          exitCode: -1

We have created the PGP private key with "gpg (GnuPG) 2.2.23".

gpg --list-secret-keys --keyid-format LONG
/Users/d040602/.gnupg/pubring.kbx
---------------------------------
sec   rsa4096/8C321E51694F531E 2020-10-09 [SC] [expires: 2022-10-09]
      F3599FD662F75135637911E88C321E51694F531E
uid                 [ultimate] ...

The secret with the private key looks as follows (only used for testing :-)):

apiVersion: v1
kind: Secret
metadata:
  creationTimestamp: "2020-10-08T07:56:53Z"
  name: pot-gitops-sops-key
  namespace: garden-hubtest
  resourceVersion: "1343617448"
  selfLink: /api/v1/namespaces/garden-hubtest/secrets/pot-gitops-sops-key
  uid: d6135486-1cc0-4c91-88fe-84e3bb62515b
data:
  my.pk: apiVersion: v1
data:
  my.pk: LS0tLS1CRUdJTiBQR1AgUFJJVkFURSBLRVkgQkxPQ0stLS0tLQoKbFFkR0JGOS8rcEFCRUFDOEcwVkNaZWZ2blJJdnJCM2FpTUtra3ptc0grUmErQzJUZnF6QXFEcmNJNG9RNzhmNQpLOTJTenpqeTcycHQwUlJrNVhqRXZSTHJJS2pUamJJUHQ2ZER1OXNxSnZ6dzJ6bzg5UVJHVFlxbG9tTnY1U2ZmClZPdEM5MFpVOEJFYXNjWTlycDZneTlBVGdOS09aTGNDenpmcjY0U2x4Y25Ka2pkb0hJWmFXUXplRE9TdWNJK3YKaitiWHkzRlAxUy9COTN2ZzJ0MVBtMVhTcHlXV1ZMbTRrQjlhVkU3Y1lrTUNxTklUV1VOM2JSV1o2SmJZNU85QgpuL292LzVSUnM0N25Zc2dRbE9CMlh1bGtFdzBITXNuTS8rOENyaDYwSkhPVVQxNDdNeWFRUXBqT3huRGhqK3E5Ci9FUmtTcW5PQkQ4Q09UWDYrZFZrVXpDTTMvdTJ2a3VIaHJjN2V3SUhNVFNSbFVjS1dST1VLaVE4MzFlRGt2V0YKbDF6VkQwVnZveTRDMmF2R3psbFdLVU1McytLUlN1Ym8ydU0zRllwcFRtV1dUNTA0QnVHUTVnUDhFUWRZb0E4NwplSXZ3ZkFmYi8zVzZoKzJSb3Q3YzhYZDlRZVNYZXRRS1U0RmdYTGV6VzhaZFNnMWJHUWF5dmtGS1J6SVk4RndXCmhSWStGRXEzbll2b1plY3c0ck1ycFM1VitDQWZmTDdKZWlsMk0wL0QzU054VldETDhkbStVOWZFUmU1SkVxSGsKQ2ZqT3Y2bk5MeDBYT2pvT1FDbjJsdlB0YStqNm43RENJNzFNYkJ0MEdDTEVTbDFJQmkvSFF6WWpQcitQNElLVgoyWmtycnU1MHQ2Q0phbE5Zd0JWRFNKc25zeEJZaGJGaWw0OFNRVVR4T2JqUmFVeXM2MTdJQVEwRjJ3QVJBUUFCCi9nY0RBaDhobDRoVWE5WWE3WHdlZnJvcUZPQkhkQ0hRbmU5a2pReGZoV0d2YXJQM1lxTjZGZ041SkNqY2lrUEcKOXBmSk8xTTlyTG0yVXRVRjdBSHRBL3F1MG1qSzF6eDROSU9oaWxVRzEwUkFkVGlha01VT2ppUW9zZTB5N0N1QgpJK0lFbHVlMldMZXdZOFdZNFBEaFVqOFEzZ0xpbm9FYTVoUWU3S3g4aXVEQ00xaGhOdGlHNVA4UWhwaWZQMmZTCkViOWRjME05WUcwZ0tXelZISUlCUDJjN09HNmdqMjE1V3ExQ0N5NE1Dbm5zSVpUWTV2ZGR6MExwcTRxVVRkOFMKdEdWTmRTU3doNGh5RzNVR0kwcE9DQW1jMkNwME9CSnhLT2xVd0JHc0ZIMVhpaENjd3k2b2FwTjBucTdsZXA2bgpnUjk4TmdjaE9FV0NVMVFkRzhRNHJockZ6c24xTlkyRnVUL2RnOUE4MnVpNW5DMzBxRTNwbkYwVkVMQVgvZDZtCnhSWUtIK2Q5MFhBNXZvUUMxelVOdDI5UmdXSkdCZmFNMjkrKzJvQnFEbFhISFpXYVZsV1BJYWFHMkxtNUI5WFMKRlpvcU9KbFhSVWlRbzVyaXFKdUFnT3dFbjYwNlFvQTBCK1dUek9hdWFsVERTa0pEdnRjSjZnaEhHTTJCMUplOQoxeXF0VDFOOHl1RURVcGNDNVlVeHpqQXdMNTd5V0lSM3Y2SUkrT3hmcDVaMENNeG9LVVVtUmY3MVRCaUIwMzQwCnB1QWY4SFU4UXhmblhqZDFXV0pBdVNyMjBnK1M2aVMvZTVlOFhSV3ZQN21FazdwNnBxeWVJSHl6K2NlSVdhTW0KZVJURXJMVVBIeDBLNXl3YnBRbWphTGVZMkt5MC9EYmwvU2FpMlB6L1liSFZYREE5cXpxRTFBdXFrQ0YyNldLNgo1dlk4ZE1YQmhaM0U1czBuVTcyT0VUcjFkcFJSVXNmRUlFT2w3MVJmRGNVR0gwR2lNZEhIZ2l5ZStORXhRZVl3ClBielVjcWFaZFJJY2R3M2pNaUx2KzJ5bjB6a3d1dVlpSUFLcmFGaUx5MEtLZXhjeXVRY0lTaEg4R1pNUW1ydVMKWEx6VkgwME8waW5LaTRKajRWZkhoWUtxZmRHYzJIbU1OSFRic3ByRlVOdjdQM002NXhjK3gzVGRNdVcxc3BKOQpUdVJVYWZURy9MUFM5cE1lZDFILzUxV0EwYldBZmkzZENtblRyMGZCUk1pdmNWYllpRXpZNnFHcTF3SjJqNDhTCm9HYUt1bnVPeW9WbitmWkpvakdROU1hcjE3amZJRkc2RitVY3o4RGowdkJHb1RqNDNXeDJHZ0hYYVYzdGRYWHEKUWR4VUxyaXVKYnFnNzJjb3hyNzk5ZjgrdGJTWkpNMmp4NjdHUURYNGFoazZhTzBuQVI1bkozZTdTL2FMYVl5UQpua25Ib0dJaEUrMWh0c2xEczhKLzFxSTZ3UDllTFYxTDVKcVNLWGFYRnRIYmVTVHlnUnJzekVUQ3lXSHV4engvCnhlNTcxSE0zM1VvRTNDTndOQVR2aFIzdjlUNi9lU1c4YzRETmtQSHcvOE5RUFI1SmFDRTE3MmgrTHdhSnp6bEgKUElsV3M1WmFPcmFYWnR2dXZwSkwzcCtPd1pIc2ExWHc1bjVMWnB2TmxWNzdIclVSY0JCbGVVc2pRUEIwMGNRLwpIVGRHWjlxTmtlRDhUUmZsZExLQTVnU1RUTEVlWVo5TGsrR0ZpL1YzbDhSRVVSME5HVmI3N3psSFcxQ2N3MXJqCmY4aGlhUFJ5cGdFbi9zV0VFNGlUN3NBRWkyVWVmck1idk9Iazhlamx5dnA0YnJqbXFFT0xwQkx4Tnc4Q05DWWIKSG1hWS9jQmF6WXdZcXozSkVWN3JHdVpDTnMyT0NjL2xKS3dEYUk2OVZJVnYyMWtpaTVhZDJyaTdkL1VFYmFkMQppNlQwUWhVcGJSWUFJdEd5RjNsMFRQcE42R3lTSW9HcGJYOVpxM3ZxV1l6dUFGRkgvUVE0Ly9TTGMweGtpckpQCmdHSDdWaThiVlk4eVpNVHp3U05uSnZtdzlxMlBGcmJuOFdqRS9YQ2JEM2N2dmY0T3Fiek55S3VZZ2VkYTlRc08KbzNXNjR5WTRFYWZiWEV4ZmJSaEpMcVRtMS9hQlFOd3U2ZXJYN2Qvd201bHh1YVVJWVp1U2cxVmMxRVppTEl1MwpxU1ZVNldlazI0VnRJSGh6RldXTnoyY0V2aTJwSkVsdFFBU0dhcFFCMEtQQ3ZuZ29SbFc4Vy8rM29xY0gwRW91CnhodGVKbElBV2FaVDBlL0VTZmlDRktIOWo1eUpBNXRibnM1VHB3azFXR2RCc0xqSjFQbDBkU2EwSEdGamFHbHQKSUR4aFkyaHBiUzUzWldsblpXeEFjMkZ3TG1OdmJUNkpBbFFFRXdFQ0FENFdJUVR6V1ovV1l2ZFJOV041RWVpTQpNaDVSYVU5VEhnVUNYMy82a0FJYkF3VUpBOEpuQUFVTENRZ0hBZ1lWQ2drSUN3SUVGZ0lEQVFJZUFRSVhnQUFLCkNSQ01NaDVSYVU5VEhoZ3pELzlhSWFCZmJhR1czdG9RWW1yYW5Za3lMRnlJcm15SVBPTVJKRUorbDJHWUkxRVcKWVVhSm5hb0RvcmlNa1hZTG9iRjNCcUhneCtlcS9vZ3F6RkxncTJlWVJvRitkVkY4bkVFUHYxTUxtdTR2YzVTTwp6YUNqZmU2cDRrK0laYWYybXQva0I1NENqSVk4LzhRa2RBTHRyRlZJSlhaM1ZOdm5aR2Z2MmNwdURKSEVpYzVmCkd2SWFBVzlwVlgrZW10b3p0TTJnOHBpa0pCYmhvUWpMMUdIUWExSEJmRFVtb1JSdzNIUEMrRndDeGtNNWZ6QWEKWUlHNFpVM2JMRTJPZ0ZCK1FOYmJTeFFoSWdmbnkveDFGYXovZnp5NHNzeHVoRlN1bmJuTmdDQng1ZmlqYTlkZwp3c0xiNS9Qb1UxVm50SGlkRElJajJXdGREWFVLMlc5SnlDd1NvSldRc1V1T29lSW9FZXdXeEpFZzJQSWlSSEFLCm5neVgyMjZNUjArMWRyTFVmWWhraFlCZlBFWi9VZHF5UmRVdEF3RUxkd1Y3M0VXVVlzK2xadWdoLy9qOUYrMEgKRlRORXFlckx6eElaTE9vNEtNNW5tOGloekxleXc2Zy9yZ3pubE15L25NVUhRUVlnL1kvR2pWUkJyakNnN1dwZgpXbUJkdW11dXAvemE3R1FqZXlUamhQZlZweStFT2hEeTRiUDkvbHByM3lTWG5jM3djS3dHbnRwZVhKZkVBbFJzCkJNazhGbml3Rml4ZndKLzdnNWxUeHBGWE5VTDRYNHhxYVg4TFFVRGE3ZklmcnM3MTlYcTZzeEt4cnFud1VralMKZlRaS29aR3FmaitvZHpJeUJweWNySmY4L2hJdUxtLzNaTWxvYnBWWXBNUGFweDBlc1FDSnhUQ0ZCcDJ3UGc9PQo9QWg0QwotLS0tLUVORCBQR1AgUFJJVkFURSBLRVkgQkxPQ0stLS0tLQo=
kind: Secret
metadata:
  name: pot-gitops-sops-key
  namespace: garden-hubtest
type: Opaque

type: Opaque

Do you perhaps any idea what is going wrong here?

Thank you very much in advance and best regards,

Achim

Typo in readme

What steps did you take:
I read the readme.md file

What happened:
I saw a typo.

What did you expect:
I expected words to be spelled correctly.

Anything else you would like to add:
In the sentence, "It will install, and continiously apply updates.", continiously should be continuously

Environment:
n/a

support path for helm template templater

proposed changes

  • add path to indicate where chart is
  • add ability to load values from a file path
  • raw options?
  - helmTemplate:
      name: external-dns
      path: helm/chart #! Tell helm template where to find the chart location
      valuesFrom:
      - fileRef: helm/external-dns-values.yaml #! Tell helm an additional file to pass as a -f option
      - fileRef: helm/secret-external-dns-values.yaml #! Tell helm an additional file to pass as a -f option
      - secretRef:
          name: secrets-that-were-setup-ahead-of-time-for-helm
      rawOptions:
      - "--namespace external-dns"
      - "--include-crds"

(https://raw.githubusercontent.com/voor/kapp-controller-helm-pipeline/main/external-dns/app/external-dns.yaml)

support sops non-pgp backends

figure out how to best connect AWS/AKS/etc auth to App CR's service account so that each App CR does not get "global" KMS auth.

**Explore:** Cannot delete App when reconciliation is stuck

What steps did you take:

  1. Create an App with the following definition:
apiVersion: kappctrl.k14s.io/v1alpha1
kind: App
metadata:
  name: simple-app
spec:
  syncPeriod: 10s
  serviceAccountName: default
  fetch:
    - inline:
        paths:
          deploy.yaml: |
            apiVersion: apps/v1
            kind: Deployment
            metadata:
              labels:
                app: bad
              name: bad
            spec:
              selector:
                matchLabels:
                  app: bad
              template:
                metadata:
                  labels:
                    app: bad
                spec:
                  containers:
                  - image: bad-image
                    name: bad-image
  template:
    - ytt: {}
  deploy:
    - kapp: {}
  1. App gets stuck reconciling because the image points to a non-existing container image
  2. Attempt to delete the app using kubectl delete app simple-app

What happened:
The app is stuck reconciling

What did you expect:
The app is deleted

Support the docker registry repo with self-signed certificate

  1. When executing 'imgpkg' on a local machine, we can add the self-signed CA certificate on the local machine to fix it.
  2. When executing 'imgpkg' in a container (e.g. kapp-controller), does it work? If not, we might need to add an option in App CR (around fetch.image.url) for specifying CA cert.

This is simliar to #42

kapp-controller crashes when the there is an unsupported templating method specified in the App CRD

What steps did you take:
Create an App CRD as follows:

apiVersion: kappctrl.k14s.io/v1alpha1
kind: App
metadata:
  name: test
  namespace: default
spec:
  fetch:
    - image:
        url: some-image-with-files
  template:
    - foo: {} 

What happened:
Kapp-controller panics with a nil pointer deref:

E0106 15:08:52.526837      15 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
goroutine 174 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic(0x14b6920, 0x236bbc0)
	k8s.io/[email protected]/pkg/util/runtime/runtime.go:74 +0xa3
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	k8s.io/[email protected]/pkg/util/runtime/runtime.go:48 +0x82
panic(0x14b6920, 0x236bbc0)
	runtime/panic.go:679 +0x1b2
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*App).template(0xc00043d680, 0xc0003aefa0, 0x46, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/app_template.go:50 +0x56c
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*App).reconcileFetchTemplateDeploy(0xc00043d680, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/app_reconcile.go:127 +0x48a
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*App).reconcileDeploy(0xc00043d680, 0x167d2f5, 0xe)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/app_reconcile.go:81 +0x1a6
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*App).Reconcile(0xc00043d680, 0xc000437400, 0x0, 0x0, 0x0)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/app_reconcile.go:44 +0x37f
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*CRDApp).Reconcile(...)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/crd_app.go:112
github.com/vmware-tanzu/carvel-kapp-controller/cmd/controller.(*AppsReconciler).Reconcile(0xc00025e580, 0xc000806788, 0x7, 0xc000806784, 0x4, 0xc00000d9c0, 0xc00046bba0, 0x1344e55, 0xc000364888)
	github.com/vmware-tanzu/carvel-kapp-controller@/cmd/controller/apps_reconciler.go:38 +0x353
github.com/vmware-tanzu/carvel-kapp-controller/cmd/controller.(*ErrReconciler).Reconcile(0xc00043b4c0, 0xc000806788, 0x7, 0xc000806784, 0x4, 0xc0008067f0, 0xc, 0xc00046bc30, 0x403744)
	github.com/vmware-tanzu/carvel-kapp-controller@/cmd/controller/err_reconciler.go:24 +0x177
github.com/vmware-tanzu/carvel-kapp-controller/cmd/controller.(*UniqueReconciler).Reconcile(0xc000364870, 0xc000806788, 0x7, 0xc000806784, 0x4, 0xc00046bcd8, 0xc0000d4360, 0xc0000d42d8, 0x186fd60)
	github.com/vmware-tanzu/carvel-kapp-controller@/cmd/controller/unique_reconciler.go:40 +0x19e
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00003a180, 0x15081c0, 0xc00000d840, 0x0)
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:256 +0x162
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00003a180, 0x0)
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232 +0xcb
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc00003a180)
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211 +0x2b
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc0002be280)
	k8s.io/[email protected]/pkg/util/wait/wait.go:152 +0x5e
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0002be280, 0x3b9aca00, 0x0, 0x1, 0xc00021c000)
	k8s.io/[email protected]/pkg/util/wait/wait.go:153 +0xf8
k8s.io/apimachinery/pkg/util/wait.Until(0xc0002be280, 0x3b9aca00, 0xc00021c000)
	k8s.io/[email protected]/pkg/util/wait/wait.go:88 +0x4d
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:193 +0x328
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x11ec9bc]

goroutine 174 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	k8s.io/[email protected]/pkg/util/runtime/runtime.go:55 +0x105
panic(0x14b6920, 0x236bbc0)
	runtime/panic.go:679 +0x1b2
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*App).template(0xc00043d680, 0xc0003aefa0, 0x46, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/app_template.go:50 +0x56c
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*App).reconcileFetchTemplateDeploy(0xc00043d680, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/app_reconcile.go:127 +0x48a
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*App).reconcileDeploy(0xc00043d680, 0x167d2f5, 0xe)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/app_reconcile.go:81 +0x1a6
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*App).Reconcile(0xc00043d680, 0xc000437400, 0x0, 0x0, 0x0)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/app_reconcile.go:44 +0x37f
github.com/vmware-tanzu/carvel-kapp-controller/pkg/app.(*CRDApp).Reconcile(...)
	github.com/vmware-tanzu/carvel-kapp-controller@/pkg/app/crd_app.go:112
github.com/vmware-tanzu/carvel-kapp-controller/cmd/controller.(*AppsReconciler).Reconcile(0xc00025e580, 0xc000806788, 0x7, 0xc000806784, 0x4, 0xc00000d9c0, 0xc00046bba0, 0x1344e55, 0xc000364888)
	github.com/vmware-tanzu/carvel-kapp-controller@/cmd/controller/apps_reconciler.go:38 +0x353
github.com/vmware-tanzu/carvel-kapp-controller/cmd/controller.(*ErrReconciler).Reconcile(0xc00043b4c0, 0xc000806788, 0x7, 0xc000806784, 0x4, 0xc0008067f0, 0xc, 0xc00046bc30, 0x403744)
	github.com/vmware-tanzu/carvel-kapp-controller@/cmd/controller/err_reconciler.go:24 +0x177
github.com/vmware-tanzu/carvel-kapp-controller/cmd/controller.(*UniqueReconciler).Reconcile(0xc000364870, 0xc000806788, 0x7, 0xc000806784, 0x4, 0xc00046bcd8, 0xc0000d4360, 0xc0000d42d8, 0x186fd60)
	github.com/vmware-tanzu/carvel-kapp-controller@/cmd/controller/unique_reconciler.go:40 +0x19e
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00003a180, 0x15081c0, 0xc00000d840, 0x0)
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:256 +0x162
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00003a180, 0x0)
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232 +0xcb
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc00003a180)
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211 +0x2b
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc0002be280)
	k8s.io/[email protected]/pkg/util/wait/wait.go:152 +0x5e
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc0002be280, 0x3b9aca00, 0x0, 0x1, 0xc00021c000)
	k8s.io/[email protected]/pkg/util/wait/wait.go:153 +0xf8
k8s.io/apimachinery/pkg/util/wait.Until(0xc0002be280, 0x3b9aca00, 0xc00021c000)
	k8s.io/[email protected]/pkg/util/wait/wait.go:88 +0x4d
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:193 +0x328
{"level":"error","ts":1609945732.5325596,"logger":"kc.init","msg":"Could not start controller","error":"exit status 2","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tgithub.com/go-logr/[email protected]/zapr.go:128\ngithub.com/vmware-tanzu/carvel-kapp-controller/cmd/controllerinit.Run\n\tgithub.com/vmware-tanzu/carvel-kapp-controller@/cmd/controllerinit/run.go:36\nmain.main\n\tgithub.com/vmware-tanzu/carvel-kapp-controller@/cmd/main.go:43\nruntime.main\n\truntime/proc.go:203"}

Rollback

kapp-controller could have the ability to perform a rollback. I'm currently unsure how this could be accomplished, but if the controller can somehow persist change requests, a previous one could be selected.

allow running kapp-controller in a CLI mode to deploy provided app crd

app.yml includes App CR describing what/how to deploy:

kappctrl deploy -a app1 -f app.yml
kappctrl deploy -a app1 -f app.yml

It might be worth adding "shortcut" commands

kappctrl deploy -a app1 --helm stable/postgres
kappctrl deploy -a app1 --git github.com/cloudfoundry/cf-for-k8s --ytt-...
  • deploy multiple CRs (recursively?)
  • show kapp output
  • provide a way to rerun continuously (ie --watch flag)
  • provide "shortcut" to install config via flags

related: carvel-dev/kapp#96

[fetch] Add config for fetching imgpkg bundles

Describe the problem/challenge you have
There is currently no way to fetch assets stored in an imgpkg bundle.

Describe the solution you'd like
A section of the AppCRD that contains info for fetching an imgpkg bundle.

spec:
  fetch:
    imgpkgBundle:
        # Image url; unqualified, tagged, or
        # digest references supported (required)
        url: host.com/username/image:v0.1.0
        # secret with auth details (optional)
        secretRef:
          name: secret-name
        # grab only portion of image (optional)
        subPath: inside-dir/dir2

This would allow users to leverage the benefits of imgpkg bundles over images, for example, using the ImagesLock file during the template stage for relocated assets .

improve error discovery for missing data values

Error message:

    Stderr:      kapp: Error: Validation errors:
- Expected 'kind' on resource '/ () cluster' to be non-empty (stdin doc 10)
- Expected 'apiVersion' on resource '/ () cluster' to be non-empty (stdin doc 10)
- Expected 'metadata.name' on resource '/ () cluster' to be non-empty (stdin doc 10)

Okay, this took a long time and was pretty much trial and error, would love to find some way to possibly make that easier. My secret that I was using for inline.pathsFrom with the ytt section didn't have #@data/values at the top.

how do we improve finding this kind of error? will schemas help with this?

issue with Contour app

What steps did you take:
[A clear and concise description steps that can be used to reproduce the problem.]

kubectl apply -f ingress/contour/namespace-role.yaml
kubectl create secret generic contour-data-values --from-file=values.yaml=ingress/contour/vsphere/contour-data-values.yaml -n tanzu-system-ingress-4
kubectl apply -f ingress/contour/contour-extension.yaml
kubectl get app contour -n tanzu-system-ingress

What happened:
[A small description of the issue]
root@tkg [ ~ ] tkg12workloadcluster-01-admin@tkg12workloadcluster-01:default) # k get app contour -n tanzu-system-ingress-4
NAME DESCRIPTION SINCE-DEPLOY AGE
contour Reconcile failed: Fetching (0): Fetching registry image: Imgpkg: exit status 1 (stderr: Error: Collecting images: Working with registry.tkg.vmware.run/tkg-extensions-templates:v1.2.0_vmware.1: Get https://registry.tkg.vmware.run/v2/: dial tcp: i/o timeout
) 64m

What did you expect:
[A description of what was expected]
NAME DESCRIPTION SINCE-DEPLOY AGE
contour Reconcile succeeded 15s 72s

Anything else you would like to add:
[Additional information that will assist in solving the issue.]

Environment:

  • kapp Controller version (execute kubectl get deployment -n kapp-controller kapp-controller -o yaml and the annotation is kbld.k14s.io/images):
  • Kubernetes version (use kubectl version)

[template] support helm v3

consider supporting include CRDs flag. still want to support helm v2. should we make this consistent with how vendir selects helm version?

determine better way to delete App CR when destination cluster is gone/disconnected

Describe the problem/challenge you have

in cases when App CR no longer has access to service account (e.g. remote cluster is deleted, disconnected), we dont have a natural way to delete App CRs. one "solution" today is to initiate deletion and then delete finalizer. this feels more like a hack.

Describe the solution you'd like

not sure. suggestions previously were:

  • add force delete flag of sorts (may be annotation, etc)

Anything else you would like to add:

this issue may arise also when service account resource is deleted before App CR (so not just in the cases of remote cluster).

[fetch] support http proxy configuration

seems like this should be per controller configuration? at the least, document how to configure env vars.

example env vars:

- name: http_proxy 
  value: http://proxy.corp.com:8888
- name: HTTP_PROXY
  value: http://proxy.corp.com:8888
- name: https_proxy 
  value: http://proxy.corp.com:8888
- name: HTTPS_PROXY
  value: http://proxy.corp.com:8888
- name: no_proxy
  value: 127.0.0.1,localhost,kubernetes.default.svc,.svc,.corp.com
- name: NO_PROXY
  value: 127.0.0.1,localhost,kubernetes.default.svc,.svc,.corp.com
- name: PROXY_ENABLED
  value: "yes"

Bump client-go dependency to at least v0.19.0

The kubernetes project started using all lower-case import strings. Because the version of client-go that we are using (v0.17.2) uses an upper case version of an import string, if we ever need to add a dependency, or bump one of the other k8s dependencies, that uses that same package, we will get the following go mod error: case-insensitive import collision: "github.com/googleapis/gnostic/openapiv2" and "github.com/googleapis/gnostic/OpenAPIv2".

Huge amount of retries in case of failures

Describe the problem/challenge you have
Assume we define an App CR to deploy some artefacts on some target cluster. If the deployment fails, e.g. if the target cluster is hibernated and not reachable, the deployment is retried every few seconds without any deceleration. This results in a flood of retries if there are more than one such App CR.

Describe the solution you'd like
Perhaps it would be a good idea to slow down the retries perhaps using the standard strategy of the controller framework. Not sure if this should be configurable?

Pin binary dependency versions in source

Right now, kapp-controller builds package the latest version of kapp. This makes it impossible to conclude the kapp version from kapp-controller's version.

Should we consider pinning kapp's version per kapp-controller release?

tagging my pair @Haegi

[fetch] support verifying signed git commits

It would be really cool if the git source could verify a commit was signed with a GPG signature. This seems like a nice middle ground between not really wanting to give our GitLab access to a serviceAccount with really broad permissions, but also not wanting to set up really fine grained deployment roles per application.

consider blocking service account deletion on app crs

currently if you deploy app cr and associated service account and delete them together, there is a race between kapp controller deleting workload and k8s deleting associated service account. once service account is deleted, kapp controller of course cannot delete app workload.

currently if you are deploying appcr+sa with kapp, you can add order rule; however, in a more basic cases (using kubectl for example), we should probably enforce "dependency" in api server via finalizer on service account and associated secrets. to support usage of same service account by multiple app crs, it's probably needed to add one finalizer per app cr to service account.

add condition to indicate constant reconciliation drift

it might be helpful to indicate that resources are consistently drifting. it should probably kick in if we see that X number of consecutive reconciliations happened without external change (though how does one no what is external...).

Allow kapp-controller to change context to another cluster

/kind feature

As a user, I would like to concile an App CR deployed in Cluster A into Cluster B, to support a few use cases:

  1. I run a central shared services cluster (A) and want to deploy agents into Cluster B
  2. I am running a Cluster API provider in cluster (A) and want all of the managed clusters (B... C...) to have applied to them specific capabilities after they are created.

Given a secret of the form:

apiVersion: v1
data:
  value: YXBpVmVyc... (base64 encoded kubeconfig file)
kind: Secret
metadata:
  name: another-kubeconfig
  namespace: some-namespace
type: Opaque

I can put into the App CR:

apiVersion: kappctrl.k14s.io/v1alpha1
kind: App
metadata:
  name: my-app-to-another
  namespace: some-namespace
spec:
  context: 
   -  secretRef:
      name: another-kubeconfig
      subPath: value
   -  secretRef:
      matchLabels:
        somelabel: on-mysecrets
      subPath: value
  fetch:
  - git:
...
  template:
  - ytt: {}
  deploy:
  - kapp: {}

Update 1: removed namespace reference so that you can only reference a secret in the same namespace.
Update 2: change context to an array allowing you to define multiple contexts

If kapp can never connect to remote cluster it can never delete App CR

Recreate:

Create a kubeconfig that is valid but points to an invalid endpoint (like api.example.com). On first deployment you will get something similar to this:

Status:
  Conditions:
    Message:  Deploying: exit status 1
    Status:   True
    Type:     ReconcileFailed
  Deploy:
    Error:       Deploying: exit status 1
    Exit Code:   1
    Finished:    true
    Started At:  2020-07-04T12:08:10Z
    Stderr:      kapp: Error: Creating app:
  Post https://api.example.com:6443/api/v1/namespaces/kube-system/configmaps: dial tcp 10.234.2.190:6443: i/o timeout
    Stdout:      Target cluster 'https://api.example.com:6443'
    Updated At:  2020-07-04T12:09:10Z
  Fetch:
    Exit Code:           0
    Started At:          2020-07-04T12:08:10Z
    Updated At:          2020-07-04T12:08:10Z
  Friendly Description:  Reconcile failed: Deploying: exit status 1
  Inspect:
    Error:      Inspecting: exit status 1
    Exit Code:  1
    Stderr:     kapp: Error: Getting app:
  Get https://api.example.com:6443/api/v1/namespaces/kube-system/configmaps/a-postcreate-ctrl: dial tcp 10.234.0.52:6443: i/o timeout
    Stdout:             Target cluster 'https://api.example.com:6443'
    Updated At:         2020-07-04T12:10:11Z
  Observed Generation:  1
  Template:
    Exit Code:   0
    Updated At:  2020-07-04T12:08:10Z

App CR will not sit around indefinitely, because it cannot ever get the status of the app.

git checkout reference is not a tree

We are using the following configuration to install cf-for-k8s:

apiVersion: kappctrl.k14s.io/v1alpha1
kind: App

metadata:
  name: cf

spec:
  cluster:
    kubeconfigSecretRef:
      name: "cf4k8s-kapp.kubeconfig"
      key: "kubeconfig"
  fetch:
    - git:
        url: https://github.tools.sap/cki/cf-for-k8s-scp.git
        ref: origin/master
        secretRef:
          name: github-tools-sap-token
        subPath: config

  template:
    - ytt:
        inline:
          pathsFrom:
            - secretRef:
                name: cf-values

  deploy:
    - kapp:
        inspect:
          rawOptions: ["--json=true"]

When we change ref: origin/master to ref: 11c6cd21ff7100de5bf208aa391912f38c830b32 we get the following error:

Reconcile failed: Fetching (0): Fetching git repo: Git [checkout 11c6cd21ff7100de5bf208aa391912f38c830b32 --recurse-submodules .]: exit status 128 (stderr: fatal: reference is not a tree: 11c6cd21ff7100de5bf208aa391912f38c830b32
)

although the commit 11c6cd21ff7100de5bf208aa391912f38c830b32 exists in the repo.

git checkout 11c6cd21ff7100de5bf208aa391912f38c830b32 --recurse-submodules . on the command line works fine.

Running

cd /tmp
rm -rf cf-for-k8s
mkdir cf-for-k8s
cd cf-for-k8s/
git init .
git remote add origin https://github.tools.sap/cki/cf-for-k8s-scp.git
git fetch origin
git checkout 11c6cd21ff7100de5bf208aa391912f38c830b32

inside the kapp-controller docker container also works.

introduce AppSet CRD

AppSet is meant to represent same App deployed in several variations (it's analogous to Pod to Deployment relation where each Pod gets a unique name):

  • across a set of clusters (based on collection of kubeconfig secrets)
  • across a set of namespaces (may be based on collection of service accounts?)
  • with different data values (with different config maps)
  • with different source configuration (e.g. different git branches)

Install Script Does Not Work with kubectl Ordering

Following the install instructions for kapp-controller, there is no mention of needing to create the kapp-controller namespace before using kapp or kubectl.

One approach to resolving this would be to actually include the creation of the kapp-controller namespace as part of the script. An alternative is to update the documentation so users are aware of the step before installing.

EDIT

This issue is actually about how kubectl needs the namespace resource to come before all other dependent resources in the release script as shown here. The kapp install process should still work as expected.

Don't run kapp inspect if we haven't successfully deployed

What steps did you take:

  1. Deploy an app that has a failing fetch stage:
apiVersion: kappctrl.k14s.io/v1alpha1
kind: App
metadata:
  name: simple-app
  namespace: default
spec:
  serviceAccountName: default-ns-sa
  fetch:
  - http:
      url: i-dont-exist
  template:
  - ytt: {}
  deploy:
  - kapp: {}
  1. kubectl get apps -n <namespace you deployed to> -oyaml
  2. see that status has an inspect stage showing an error

What happened:
Seeing the inspect error, which isnt the actual error causing reconciliation to fail, is misleading

    fetch:
      error: 'Fetching resources: exit status 1'
      exitCode: 1
      startedAt: "2021-01-27T19:33:30Z"
      stderr: |
        Error: Syncing directory '0': Syncing directory '.' with HTTP contents: Downloading URL: Initiating URL download: Get i-dont-exist: unsupported protocol scheme ""
      updatedAt: "2021-01-27T19:33:30Z"
    friendlyDescription: 'Reconcile failed: Fetching resources: exit status 1'
    inspect:
      error: 'Inspecting: exit status 1'
      exitCode: 1
      stderr: 'kapp: Error: App ''simple-app-ctrl'' (namespace: default) does not
        exist: configmaps "simple-app-ctrl" not found'
      stdout: Target cluster 'https://10.96.0.1:443'
      updatedAt: "2021-01-27T19:33:30Z"

What did you expect:
To just see the error that relates to why the app is failing (in this case the fetch)

We should not run inspect if the app has never been deployed.

Support a ytt inline values option for ease of use

I really like doing the following fairly often:

  template:
  - ytt:
      ignoreUnknownComments: true
      inline:
        paths:
          somemorevalues.yaml: |
            #@data/values
            ---
            stuff: that
            can: go
            here: true

Unfortunately you lose the power of native yaml syntax highlighting and other convenience stuff. If you added in support for just inline.values you could instead have the following:

  template:
  - ytt:
      ignoreUnknownComments: true
      inline:
        values:
          stuff: that
          can: go
          here: true

It's more of a convenience thing, but it also lowers the barrier because someone would need to know the little hack to include a data/values file directly.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.