crossplane-contrib / function-go-templating Goto Github PK

What happened?

During the access to the list item by index or by the first function inside the template, I got the error:

      at <index .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs
      0>: error calling index: index of untyped nil

log data

2024-03-18T20:45:49.992Z	DEBUG	fn/fn.go:60	template	{"template": "---\napiVersion: example.acme.io/v1beta1\nkind: Debug\nmetadata:\n  name: test-debug\n  annotations:\n    gotemplating.fn.crossplane.io/composition-resource-name: test-debug\nspec:\n  ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip  }}\n"}
2024-03-18T20:45:49.993Z	DEBUG	fn/fn.go:74	constructed request map	{"request": {"context":{"apiextensions.crossplane.io/environment":{}},"desired":{"composite":{"resource":{"apiVersion":"example.acme.io/v1beta1","kind":"XR"}},"resources":{"pod":{"resource":{"apiVersion":"kubernetes.crossplane.io/v1alpha2","kind":"Object","spec":{"forProvider":{"manifest":{"apiVersion":"v1","kind":"Pod","metadata":{"name":"nginx","namespace":"default"},"spec":{"containers":[{"image":"nginx:1.14.0","name":"nginx"}]}}},"providerConfigRef":{"name":"kubernetes-provider"}}}}}},"input":{"apiVersion":"gotemplating.fn.crossplane.io/v1beta1","inline":{"template":"---\napiVersion: example.acme.io/v1beta1\nkind: Debug\nmetadata:\n  name: test-debug\n  annotations:\n    gotemplating.fn.crossplane.io/composition-resource-name: test-debug\nspec:\n  ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip  }}\n"},"kind":"GoTemplate","source":"Inline"},"observed":{"composite":{"resource":{"apiVersion":"example.acme.io/v1beta1","kind":"XR","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"example.acme.io/v1beta1\",\"kind\":\"XR\",\"metadata\":{\"annotations\":{},\"name\":\"test\"},\"spec\":{\"data\":\"top-secret\"}}\n"},"creationTimestamp":"2024-03-18T20:44:44Z","finalizers":["composite.apiextensions.crossplane.io"],"generation":3,"labels":{"crossplane.io/composite":"test"},"managedFields":[{"apiVersion":"example.acme.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:finalizers":{".":{},"v:\"composite.apiextensions.crossplane.io\"":{}},"f:labels":{".":{},"f:crossplane.io/composite":{}}},"f:spec":{"f:compositionRef":{".":{},"f:name":{}},"f:compositionRevisionRef":{".":{},"f:name":{}}}},"manager":"crossplane","operation":"Update","time":"2024-03-18T20:44:44Z"},{"apiVersion":"example.acme.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:status":{".":{},"f:conditions":{".":{},"k:{\"type\":\"Synced\"}":{".":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}},"manager":"crossplane","operation":"Update","subresource":"status","time":"2024-03-18T20:44:44Z"},{"apiVersion":"example.acme.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{".":{},"f:compositionUpdatePolicy":{},"f:data":{}}},"manager":"kubectl-client-side-apply","operation":"Update","time":"2024-03-18T20:44:44Z"}],"name":"test","resourceVersion":"654357","uid":"1aa8b1d7-b30f-4d3b-be9f-05f8192232fa"},"spec":{"compositionRef":{"name":"function-go-template"},"compositionRevisionRef":{"name":"function-go-template-1737ec3"},"compositionUpdatePolicy":"Automatic","data":"top-secret"},"status":{"conditions":[{"lastTransitionTime":"2024-03-18T20:44:44Z","message":"cannot compose resources: pipeline step \"go-templating\" returned a fatal result: cannot execute template: template: manifests:9:10: executing \"manifests\" at <first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs>: error calling first: runtime error: invalid memory address or nil pointer dereference","reason":"ReconcileError","status":"False","type":"Synced"}]}}}}}}

How can we reproduce it?

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: function-go-template
spec:
  compositeTypeRef:
    apiVersion:  example.acme.io/v1beta1
    kind: XR
  mode: Pipeline
  pipeline:
    - step: patch-and-transform
      functionRef:
        name: function-patch-and-transform
      input:
        apiVersion: pt.fn.crossplane.io/v1beta1
        kind: Resources
        resources:
          - name: pod
            base:
              apiVersion: kubernetes.crossplane.io/v1alpha2
              kind: Object
              spec:
                providerConfigRef:
                  name: kubernetes-provider
                forProvider:
                  manifest:
                    apiVersion: v1
                    kind: Pod
                    metadata:
                      name: nginx
                      namespace: default
                    spec:
                      containers:
                        - name: nginx
                          image: nginx:1.14.0
    - step: go-templating
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            ---
            apiVersion: example.acme.io/v1beta1
            kind: Debug
            metadata:
              name: test-debug
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: test-debug
            spec:
              ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip  }}

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xrs.example.acme.io
spec:
  group: example.acme.io
  names:
    kind: XR
    plural: xrs
  versions:
    - name: v1beta1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                data:
                  type: string

---
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: debugs.example.acme.io
spec:
  group: example.acme.io
  names:
    kind: Debug
    plural: debugs
  versions:
    - name: v1beta1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                ip:
                  type: string

apiVersion: example.acme.io/v1beta1
kind: XR
metadata:
  name: test
spec:
  data: top-secret

What environment did it happen in?

• function-go-templating v0.4.1
• Crossplane v0.15.1

What problem are you facing?

Currently, we can access composition environment variables like:

env: {{ (index .context "apiextensions.crossplane.io/environment").key1 }} 

However, this is too verbose and can be simplified with a helper function.

How could this Function help solve your problem?

To make it easier to access composition environment variables, we can have a helper function called getCompositionEnvVar. The interface might be like:

env: {{ getCompositionEnvVar . "key1" }}

What happened?

When adding annotation to make resource ready (based on README example):

 gotemplating.fn.crossplane.io/ready: True

I am getting an error:

crossplane: error: cannot render composite resource: pipeline step "render-templates" returned a fatal result: "User" template is missing required "gotemplating.fn.crossplane.io/composition-resource-name" annotation

This is misleading because I have setgotemplating.fn.crossplane.io/composition-resource-name annotation, and the real error is caused by an invalid format of ready annotation (boolean instead of string). It works when I change it to:

 gotemplating.fn.crossplane.io/ready: "True"

How can we reproduce it?

What environment did it happen in?

Function version: 0.3.0

What problem are you facing?

Currently, the function can get template inputs as inline or from a file system directory. As a next step, we can support getting them from OCI images. In this way, users may easily decide which version of the templates to use in their composition.

How could this Function help solve your problem?

Following API is suggested by @turkenh in the comment:

      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Image
        image:
          repo: turkenh/my-templates
          tag: v0.1.0

In this case, templates are expected in a fixed path like /templates.

So, trying to define a composition of a multiple resources where one depends on another but does not implement selectors. Probably most typical use case is kind: CompositeConnectionDetails but also things like a IAM policy template that uses ARN of the previous resource.

Since there is a dependency, template would fail at the composition and nothing will happen.

One way to workaround that as far as I understood from the documentation is by using getResourceCondition function. So I ended up with something like this:

  - step: create-iam-policy
    functionRef:
      name: function-go-templating
    input:
      apiVersion: gotemplating.fn.crossplane.io/v1beta1
      kind: GoTemplate
      source: Inline
      inline:
        template: |
          {{- if ne .observed.resources nil }}
          {{- if eq (getResourceCondition "Ready" ( index .observed.resources "kms-key" )).Status "True" }}
          {{- if eq (getResourceCondition "Ready" ( index .observed.resources "s3-bucket" )).Status "True" }}

          apiVersion: iam.aws.upbound.io/v1beta1
          kind: Policy
          metadata:
            annotations:
              gotemplating.fn.crossplane.io/composition-resource-name: "iam-role-policy"
          spec:
            forProvider:
              policy: >-
                {
                  "Version": "2012-10-17",
                  "Statement": [
                    {
                      "Action": [
                        "kms:Encrypt",
                        "kms:Decrypt",
                        "kms:DescribeKey",
                        "kms:GenerateDataKey",
                        "kms:GenerateDataKeyWithoutPlaintext",
                        "kms:GenerateDataKeyPair",
                        "kms:GenerateDataKeyPairWithoutPlaintext",
                        "kms:GenerateMac",
                        "kms:GetPublicKey"
                      ],
                      "Effect": "Allow",
                      "Resource": [
                        "{{ ( index .observed.resources "kms-key" ).resource.status.atProvider.arn }}"
                      ]
                    },
                    {
                      "Action": [
                        "s3:ListBucket",
                        "s3:GetBucketLocation",
                        "s3:ListBucketMultipartUploads"
                      ],
                      "Effect": "Allow",
                      "Resource": [
                        "{{ ( index .observed.resources "s3-bucket" ).resource.status.atProvider.arn }}"
                      ]
                    },
                    {
                      "Action": [
                        "s3:PutObject",
                        "s3:GetObject",
                        "s3:DeleteObject",
                        "s3:ListMultipartUploadParts",
                        "s3:AbortMultipartUpload"
                      ],
                      "Effect": "Allow",
                      "Resource": [
                        "{{ ( index .observed.resources "s3-bucket" ).resource.status.atProvider.arn }}/*"
                      ]
                    }
                  ]
                }
            providerConfigRef:
              name: provider-aws-local-account

          {{- end }}
          {{- end }}
          {{- end }}

  - step: store-status-and-connection-details
    functionRef:
      name: function-go-templating
    input:
      apiVersion: gotemplating.fn.crossplane.io/v1beta1
      kind: GoTemplate
      source: Inline
      inline:
        template: |
          apiVersion: meta.gotemplating.fn.crossplane.io/v1alpha1
          kind: CompositeConnectionDetails
          {{- if eq .observed.resources nil }}
          data: {}
          {{- else }}
          data:

          {{- with ( index .observed.resources "kms-key" ) }}
          {{- if eq (getResourceCondition "Ready" .).Status "True" }}
            AWS_KMS_KEY_ARN: {{ .resource.status.atProvider.arn | b64enc }}
          {{- end }}
          {{- end }}

          {{- with ( index .observed.resources "s3-bucket" ) }}
          {{- if eq (getResourceCondition "Ready" .).Status "True" }}
            AWS_REGION: {{ .resource.status.atProvider.region | b64enc }}
            AWS_BUCKET_NAME: {{ .resource.status.atProvider.id | b64enc }}
            AWS_BUCKET_ARN: {{ .resource.status.atProvider.arn | b64enc }}
          {{- end }}
          {{- end }}

          {{- with ( index .observed.resources "iam-role" ) }}
          {{- if eq (getResourceCondition "Ready" .).Status "True" }}
            AWS_ROLE_ARN: {{ .resource.status.atProvider.arn | b64enc }}
          {{- end }}
          {{- end }}

          {{- end }}

And then it got me thinking... what if for whatever reason one of these resources become not ready? Composition will void dependent resources from the output and consequentially Crossplane will try to delete them. Am I missing something or doing something wrong? Any better ways to handle these dependencies? Am I supposed to just make sure that if dependencies are not ready, at least some incomplete version of the resource is still being produced? That doesn't solve the connection details though, as keys may become temporarily corrupted...

What happened?

The function errors out when gotemplating.fn.crossplane.io/ready is not a string.
the error we currently see is:

returned a fatal result: "ProviderConfig" template is missing required "gotemplating.fn.crossplane.io/composition-resource-name" annotation

How can we reproduce it?

define a managed resource and add the gotemplating.fn.crossplane.io/ready: True annotation.

What environment did it happen in?

Function version: v0.4.1

What to expect:

an error message similar to:

returned a fatal result: invalid function input: invalid "gotemplating.fn.crossplane.io/ready" annotation value "foobar": must be True, False, or Unspecified

maybe something like:

invalid function input: invalid "gotemplating.fn.crossplane.io/ready" annotation value is a boolean: must be True, False, or Unspecified as a string

What problem are you facing?

When using filesystem source, it's not possible as stated in the README to use crossplane beta render.

At this point the only workaround is to use inline source.

How could this Function help solve your problem?

It would be nice to allow mounting a volume (bind mount?) to access the actual template files or alternatively read from a YAML file containing a configMap.

What problem are you facing?

I'd like to be able to implement secretRef lookups against secrets in the crossplane-system namespace to include the resulting key in my compositions as templated values.

{{ secretRef "secret-name" "key" }}

This would of course incur some kubernetes ServiceAccount privileges to read secrets, but these should be optional and this additional templating function should fail with an error if the sufficient privileges aren't configured.

I'm not sure if this is even possible.

How could this Function help solve your problem?

This would allow me to use external secrets operator to go get some secrets from AWS secrets manager and have them accessible in my compositions, or pre-seed the cluster via terraform creating secrets.

Currently the only alternative I can think of is EnvironmentConfig objects but they are closer to a configMap than secret per se, and even then I can't see them reliably in Terraform because Crossplane is installed by ArgoCD out of band and the CRD for EnvironmentConfigs won't exist yet.

Notes

I appreciate this might be out of scope for the go-templating function, and I'm not even sure if functions can be configured for external connectivity - I know they're designed to assume they don't have it, but I'm under the impression there's work being done for Resource Lookups in a function, to reference Crossplane Compositions/MRs natively in a pipeline, which itself would require some RBAC type permissions.

edit: Reference to design proposal for network enabled functions: https://github.com/crossplane/crossplane/blob/master/design/design-doc-observe-only-resources.md#querying-and-filtering

apiVersion: apiextensions.crossplane.io/v2alpha1
kind: Composition
metadata:
  name: example
spec:
  compositeTypeRef:
    apiVersion: database.example.org/v1alpha1
    kind: XPostgreSQLInstance
  functions:
    - name: query-aws
      type: Container
      container:
        image: xkpg.io/query-aws:0.1.0
        # We need to access AWS API to make the queries. 
        network: Accessible
      config:
        apiVersion: query.aws.upbound.io/v1alpha1
        kind: VPC
        metadata:
          name: find-default-vpc
        spec:
          region: us-east-1
          default: true

What happened?

I have a parameter in my claim that lets users define a dynamic number of kafka clusters to be composed:

spec:
  parameters:
    clusters:
      - name: main
      - name: shared

Each cluster the user defines in the claim spec, the xfn will compose a kafka cluster and associated API key for it. Each API key has an associated connection detail.

I am aggregating all of the connection details into the XR/XRC connection detail via the go templating function:

            apiVersion: meta.gotemplating.fn.crossplane.io/v1alpha1
            kind: CompositeConnectionDetails
            {{ if eq $.observed.resources nil }}
            data: {}
            {{ else }}
            data:
            {{- range $i, $cluster := $.observed.composite.resource.spec.parameters.clusters }}
              {{ $clusterCompositionResourceName := printf "cluster-%d" $i  }}
              {{ $apiKeyCompositionResourceName := printf "apikey-%d" $i }}
              {{ $apiKeyMRConnectionDetails := (index $.observed.resources $apiKeyCompositionResourceName).connectionDetails }}
              {{ $clusterBootstrapEndpoint := (index $.observed.resources $clusterCompositionResourceName).resource.status.atProvider.bootstrapEndpoint | default "" }}

              {{ $cluster.name }}_api_key_id: {{ $apiKeyMRConnectionDetails.api_key_id }}
              {{ $cluster.name }}_api_key_secret: {{ $apiKeyMRConnectionDetails.api_key_secret }}
              {{ $cluster.name }}_bootstrap_endpoint: {{ b64enc $clusterBootstrapEndpoint }}
            {{- end }} #! end of nested range loop inside CompositeConnectionDetails.data
            {{ end }} #! end of if-else block

This works great, but if the user modifies their claim to remove a cluster, ie:

spec:
  parameters:
    clusters:
      - name: main

The keys from the shared cluster still persist in the connection detail in Vault:

My main question is:

1. It feels a bit messy to be aggregating multiple MR connection details into a single claim connection detail. Is there a way to create multiple connection details for a single claim? Maybe I should just separate each cluster into a separate XRC...

How can we reproduce it?

Follow the above steps

What environment did it happen in?

Function version: v0.3.0
Crossplane version: v1.14.0

What problem are you facing?

Currently, we don't have a way to calculate CIDR subnets in a composition. This can be achieved by a specific CIDR calculator function, or the logic can be implemented inside the go-templating function as a helper function.

Both of these options have their own advantages and disadvantages. Having several small-scoped functions makes the compositions harder to write and adds a function maintenance burden. However, implementing cloud-specific logic inside a generic function like go-templating makes the scope of the function larger.

I believe, from the user interface POV, it's better to add cloud-specific logic in the function or support a way to define additional helper functions to be used in the templates.

How could this Function help solve your problem?

Add a helper function to calculate subnet address like Terraform.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update module github.com/crossplane/crossplane-runtime to v1.15.1
• chore(deps): update dependency go to v1.22.2
• fix(deps): update module github.com/alecthomas/kong to v0.9.0
• fix(deps): update module google.golang.org/protobuf to v1.34.0
• fix(deps): update module k8s.io/apimachinery to v0.30.0
• fix(deps): update module sigs.k8s.io/controller-tools to v0.15.0
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Replacing crossplane/crossplane#5250.

What happened?

crossplane-cli-v1.14.5 beta render cluster/local/composition-render/mxp-control-plane/kube-control-plane/vcluster/xr.yaml cluster/xpkgs/mxp-control-plane/kube-control-plane/vcluster/composition.yaml cluster/local/composition-render/functions.yaml -o cluster/local/composition-render/mxp-control-plane/kube-control-plane/vcluster/observed.yaml

gives this output:

crossplane: error: cannot render composite resource: pipeline step "render-composed-resources" returned a fatal result: invalid function input: cannot parse the provided templates: template: manifests:606: bad character U+005B '['

The output does not say which manifest.

How can we reproduce it?

With the commands above and some template with a go-template error.

What environment did it happen in?

Crossplane version: 1.14.5

What problem are you facing?

We manage infrastructure with crossplane. Our compositions refer to some observe only resources (via the new beta style management policies). As of today, we need to create observe-only resources for each composition. Those resources could be global networks, flavors/hardware templates, or other resources that are commonly provided by the cloud provider.

How could this Function help solve your problem?

With the new feature of requesting extra resources in Crossplane 1.15, go template could fetch those extra resources and provide them as variables in the go template via the RunFunction request. This would have the following benefits:

• Less observed resources needed (1 per XRD/Composition, instead of 1 per XR) -> therefore, there is less load on the provider
• Faster composition creations. We do not need to wait until the observe resources are ready.

What happened?

I'm pretty new to use go-templating-function for composition. What I want to achieve is to pass result of provisioned managed resource to another managed resource in compostion. Let's say I want to create AWS security group and security group rule associated with group I created like:

{{- $xr := .observed.composite.resource }}
apiVersion: ec2.aws.upbound.io/v1beta1
kind: SecurityGroup
metadata:
     name: security-group
     annotations:
         crossplane.io/composition-resource-name: security-group
spec:
    forProvider:
         region: {{ $xr.spec.region }}
         vpcId: {{ $xr.spec.vpcId }}
         name: {{ $xr.spec.name }}-security-group

apiVersion: ec2.aws.upbound.io/v1beta1
kind: SecurityGroupRule
metadata:
     name: security-group-rule
     annotations:
         crossplane.io/composition-resource-name: security-group-rule
spec:
    forProvider:
         region: {{ $xr.spec.region }}
         fromPort: {{ $xr.spec.fromPort}}
         toPort: {{ $xr.spec.toPort}}
         protocol: TCP
         type: ingress
         securityGroupId: ???

so, I just want to use security group id created by security group in security group rule. If I use patch way, normally I can use ToCompositeFieldPath patch in security group resource to write security group id to XR , then use FromCompositeFieldPath patch in security group rule resource to read security group id from XR.

But I don't know how to achieve that in go-templating function?

How can we reproduce it?

No need

What environment did it happen in?

Function version: v0.4.1

What happened?

When creating a managed resource using the patch&transform method, the annotation for the composition resource name is different from that we using here in this function.

Current Annotation (patch&transform):
crossplane.io/composition-resource-name: workspace

Annotation used by this function (go-templating):
gotemplating.fn.crossplane.io/composition-resource-name: workspace

Consistency in the annotation format for the composition resource name between the old patch&transform method and this function go-templating

How can we reproduce it?

What environment did it happen in?

Function version:

What happened?

Status propagation works

            apiVersion: example.crossplane.io/v1beta1
            kind: XR
            status:
              dummy: cool-status

But metadata propagation does not.

            apiVersion: example.crossplane.io/v1beta1
            kind: XR
            metadata:
                  annotation:
                      cool: annotation

How can we reproduce it?

Try to set custom annotation for XR as in the example above. You will notice that annotation is not propagated to the XR

What environment did it happen in?

Function version: v0.4.1

What problem are you facing?

I don't understand the context, and content of the documented examples on the readme:

    {{ .observed.composite.resource.metadata.name }}
    {{ .desired.composite.resource.status.widgets }}
    {{ (index .desired.composed "resource-name").resource.spec.widgets }}
    {{ index .context "apiextensions.crossplane.io/environment" }}

What is the difference between observed/desired? Is this a naming convention?

How could this Function help solve your problem?

Document the go-template variables I can use, how they come to exist, and what can I do with them.

Essentially some more detailed usage examples would be welcome.

Thanks!

...like P&T function is published now.

And if so, is there a timeline for it?

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: Invalid configuration option: crossplane, packageRules:
You have included an unsupported manager in a package rule. Your list: crossplane.
Supported managers are: (ansible, ansible-galaxy, argocd, asdf, azure-pipelines, batect, batect-wrapper, bazel, bazel-module, bazelisk, bicep, bitbucket-pipelines, buildkite, bun, bundler, cake, cargo, cdnurl, circleci, cloudbuild, cocoapods, composer, conan, cpanfile, deps-edn, docker-compose, dockerfile, droneci, fleet, flux, fvm, git-submodules, github-actions, gitlabci, gitlabci-include, gomod, gradle, gradle-wrapper, helm-requirements, helm-values, helmfile, helmsman, helmv3, hermit, homebrew, html, jenkins, jsonnet-bundler, kotlin-script, kubernetes, kustomize, leiningen, maven, maven-wrapper, meteor, mint, mix, nix, nodenv, npm, nuget, nvm, osgi, pep621, pip-compile, pip_requirements, pip_setup, pipenv, poetry, pre-commit, pub, puppet, pyenv, ruby-version, sbt, setup-cfg, swift, tekton, terraform, terraform-version, terragrunt, terragrunt-version, tflint-plugin, travis, velaci, woodpecker, regex).

I'm very new to Crossplane and Composition Functions - this is probably the result of a typo or a trivial misunderstanding, but I've double-checked my work and can't see anything wrong with it, so I'm hoping that the very act of asking will help me realize my mistake 🫠

What happened?

Where I've used {{ .desired.composite.resource.spec.<field name> }} in Composition definitions, expecting the field to be plumbed through into the resulting Composite Resource, instead the string <no value> [sic] is inserted.

(Only tested so far on Vault Policies as that is the only Provider I've installed and tested. I can confirm that creating Vault Policies via direct application of a manifest works as expected)

How can we reproduce it?

Full reproduction instructions from scratch are in this repo. Assuming you have a cluster set up with Crossplane, Providers, etc. all set up (and, in particular, with a ProviderConfig named vault-provider-config for the Vault Provider), do the following:

$ kubectl apply -f -<<- EOF
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xapplicationvaultpolicies.crossplane-demo.legalzoom.com
spec:
  group: crossplane-demo.legalzoom.com
  names:
    kind: XApplicationVaultPolicy
    plural: xapplicationvaultpolicies
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                serviceName:
                  type: string
EOF

$ kubectl apply -f - <<- EOF
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: vault-policy-example
spec:
  compositeTypeRef:
    apiVersion: crossplane-demo.legalzoom.com/v1alpha1
    kind: XApplicationVaultPolicy
  mode: Pipeline
  pipeline:
    - step: create-policies
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            apiVersion: vault.vault.upbound.io/v1alpha1
            kind: Policy
            metadata:
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: dev-team-policy
            spec:
              forProvider:
                name: "dev-team-policy-for-{{- .desired.composite.resource.spec.serviceName -}}"
                policy: "path \"secret/{{- .desired.composite.resource.spec.serviceName -}}\" { capabilities = [\"update\"] }"
              providerConfigRef:
                name: vault-provider-config
    - step: automatically-detect-ready-composed-resources
      functionRef:
        name: function-auto-ready
EOF

$ kubectl apply -f - <<- EOF
apiVersion: crossplane-demo.legalzoom.com/v1alpha1
kind: XApplicationVaultPolicy
metadata:
  name: application-vault-policy-for-service-1
spec:
  serviceName: my-service-1
EOF

(I've tried the templating both with and without the leading and trailing dashes - i.e. {{- .desired.[...] -}} and {{ .desired.[...] }} - same outcome both ways)

Expectation - Creation of:

• A xapplicationvaultpolicies.crossplane-demo.legalzoom.com named application-vault-policy-for-service-1
• A policies.vault.vault.upbound.io named dev-team-policy-for-my-service-1
• A Vault Policy named dev-team-policy-for-my-service-1, with policy text path "secret/my-service-1" { capabilities = ["update"] }

Actuality - Resultant Vault Policy is named dev-team-policy-for-<no value>, with policy text path "secret/<no value>" { capabilities = "update"] }

What environment did it happen in?

• Function version: v0.3.0
• Running on kind local Kubernetes cluster
• kubectl version --short:

Client Version: v1.27.3
Kustomize Version: v5.0.1
Server Version: v1.27.3

• OS: MacOS 13.6.2
• Kernel: Darwin ML-LWW96P4V61 22.6.0 Darwin Kernel Version 22.6.0: Thu Nov 2 07:43:57 PDT 2023; root:xnu-8796.141.3.701.17~6/RELEASE_ARM64_T6000 arm64

What happened?

have one problem with source:FileSystem cannot parse the provided templates: template: manifests:33: template: multiple definition of template "common-labels

kubectl describe xtest

Events:
  Type     Reason                   Age               From                                                             Message
  ----     ------                   ----              ----                                                             -------
  Normal   CompositionUpdatePolicy  57s               defined/compositeresourcedefinition.apiextensions.crossplane.io  Default composition update policy has been selected
  Normal   SelectComposition        57s               defined/compositeresourcedefinition.apiextensions.crossplane.io  Successfully selected composition: test
  Normal   SelectComposition        57s               defined/compositeresourcedefinition.apiextensions.crossplane.io  Selected composition revision: test-ed16775
  Warning  ComposeResources         0s (x7 over 57s)  defined/compositeresourcedefinition.apiextensions.crossplane.io  cannot compose resources: pipeline step "render-templates" returned a fatal result: invalid function input: cannot parse the provided templates: template: manifests:33: template: multiple definition of template "common-labels"

when using the same with source:Inline its working:

crossplane beta render examples/xr.yaml apis/inline_composition.yaml examples/functions.yaml

---
apiVersion: haarchri.io/v1alpha1
kind: XTest
metadata:
  name: test
---
apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
metadata:
  annotations:
    crossplane.io/composition-resource-name: test
  generateName: test-
  labels:
    crossplane.io/composite: test
    haarchri.io/account-name: test
    haarchri.io/controlplane-name: test
  ownerReferences:
  - apiVersion: haarchri.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: XTest
    name: test
    uid: ""
spec:
  forProvider:
    manifest:
      apiVersion: v1
      data:
        test: |
          spec:
            resources: []
      kind: ConfigMap
      metadata:
        labels:
          haarchri.io/account-name: test
          haarchri.io/controlplane-name: test
        name: test-001
        namespace: upbound-system
  providerConfigRef:
    name: default

How can we reproduce it?

https://github.com/haarchri/issue-go-function

What environment did it happen in?

Function version: v0.4.0 / v1.14.5

What problem are you facing?

We want to template desired resources dependant on conditions of other composed resources.

How could this Function help solve your problem?

Currently, we can implement it like this via native template tools:

{{ $networkStatus := "Unknown" }}
{{ if .observed.resources }}
  {{- with (index .observed.resources "network").resource.status.conditions }}
    {{ range . }}
      {{ if eq .type "Ready" }}
        {{ $networkStatus = .status }}
      {{ end }}
    {{ end }}
  {{ end }}
{{ end }}

{{ if eq $networkStatus "True" }}
  {{/* Render resource */}}
{{ end }}

This works, but it is quite verbose. I like to propose a builtin function to get a resource condition inline in the template.

func getResourceCondition(resource: string, condition: string) string

{{ if eq (getResourceCondition "network" "Ready") "True" }}
  {{/* Render resource */}}
{{ end }}

Usecase Example

• Resource2 requires Resource1 (needs a Output value that is in place when resource1's condition Ready=True)
• Resource1 is always templated as desired resource
• Resource2 is only templated as desired resource, when observed Resource1's condition Ready=True

This way, we get no "false positive" events from Resource2 regarding missing parameters. Also, all resources that are created could become ready as all their depencies are met.

What do you think about such a function?

https://pkg.go.dev/vuln/GO-2024-2687

The version of golang.org/x/net can be updated to >= v0.23.0 to mitigate this CVE.

What happened?

Excluding Sprig's functions related to accessing the pod/host environment could enhance the security posture.

The current approach integrates all of Sprig's offerings, notably the "env" and "expandenv" functions. However, it's worth noting that other projects leveraging Sprig, such as Helm and ArgoCD, consciously omit these particular functions.

The Sprig documentation itself cautions against the potential for information leakage through these functions, as detailed here: Sprig Documentation.

Considering that typical use cases involving GoTemplate doesn't require fetching runtime environment variables, it seems prudent to reconsider including these capabilities.

I'd appreciate your thoughts on this matter. Thank you!

What problem are you facing?

Currently this function allows reading from the Context:

{{- $environmentConfig := index .context "apiextensions.crossplane.io/environment" }}

But one can not write to the context.

How could this Function help solve your problem?

Adding a Context resource similar to what already done with CompositeConnectionDetails, could be an option.

apiVersion: meta.gotemplating.fn.crossplane.io/v1alpha1
kind: Context
data:
  apiextensions.crossplane.io/environment: ....
  another.key: ...

TBD how to handle it w.r.t. the input Context, we could replace it altogether, but merging could be a little bit hard with this function, probably it's the best thing to do though.

What problem are you facing?

It would be nice to have the include function added.

How could this Function help solve your problem?

include function lets you pipe the output of the template to other functions.

I think this function helps in reducing the great amount of boilerplate currently needed in crossplane and these changes would IMO make it even better.

What problem are you facing?

"FileSystem" Templates loaded from configmaps is a wonky user experience which makes them not as extendable as possible.

How could this Function help solve your problem?

If you support go's embed it will allow templates to actually be loaded from filesystem and embedded into the function as a virtual file system which is available at runtime.

https://pkg.go.dev/embed

What problem are you facing?

function-go-templating requires a special annotation gotemplating.fn.crossplane.io/composition-resource-name in the resource manifests to identify composed resources. Instead of writing the annotation, we may simply define a helper function and use it in our templates.

How could this Function help solve your problem?

Define setName helper function with the input name as a string to set the required annotation.

apiVersion: iam.aws.upbound.io/v1beta1
kind: User
metadata:
  annotations:
    {{ setName "my-user" }}
  labels:
    testing.upbound.io/example-name: my-user
spec:
  forProvider: {}

What happened?

I defined the following CompositeConnectionDetails resource

apiVersion: meta.gotemplating.fn.crossplane.io/v1alpha1
kind: CompositeConnectionDetails
{{ if eq $.observed.resources nil }}
data: {}
{{ else }}
data:
  {{ $redisCacheMR := get .observed.resources "redis-cache" | default dict }}
  {{ $connectionString := dig "connectionDetails" "attribute.primary_connection_string" "" $redisCacheMR }}
  {{ $host := dig "status" "atProvider" "hostname" " " $redisCacheMR }}
  connection_string: {{ $connectionString }}
  host: {{ $host | b64enc }}
{{ end }}

When all the MR's successfully reached a ready state, the claim would fail with the following error.

cannot propagate connection details from composite: cannot establish
control of existing connection secret

This error was resolved by adding quotes to the host host parameter.

...
  host: {{ $host | quote | b64enc }}
 ...

This error was extremely misleading and I'm assuming thats because there's no error handling in the portion of this function which sets connection details. I'm unsure if its feasible but It might be useful to make this a little more robust by automatically quoting / encode values which aren't already in the correct format or outputting an error message.

How can we reproduce it?

set

What environment did it happen in?

Function version: v0.3.0
crossplane: v1.14.2
k8s: 1.26.9

A CVE with a moderate severity was published.
GHSA-8r3f-844c-mc37

The protobuf dependency needs to be updated to v1.33.0 or newer to mitigate this CVE.

What problem are you facing?

Right now, we can access composed resources with the following syntax:

{{ ( index $.observed.resources ( print "test-user-" $i ) ) }}

And we can access composite resources like:

{{  .observed.composite.resource }} 

We can define helper functions to make it less verbose and easier to use.

How could this Function help solve your problem?

Here are the possible solutions with different parameters:

1. We can define a getComposedResource function, which gets the run function request map and resource name as inputs and returns the observed composed resource:

{{ $composed := getComposedResource . "my-user" }}

The same logic can be implemented for the composite resource like:

{{ $composite := getCompositeResource . }}

2. We can define a more generic getComposedResource function interface which may also support accessing desired composed resources:

{{ $observedComposed := getComposedResource .observed "my-user" }}
{{ $desiredComposed := getComposedResource .desired "my-user" }}

Same can be achieved for composite resource like:

{{ $observedComposite := getCompositeResource .observed }}
{{ $desiredComposite := getCompositeResource .desired }}