Git Product home page Git Product logo

devops-toolkit-crossplane's People

Contributors

10hin avatar pjanouse avatar vfarcic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

devops-toolkit-crossplane's Issues

Adapt ytt syntax to leverage K8S CR completion during authoring

Thanks for sharing the great work with ytt for reducing verbosity.

I had worked on similar approach using ytt and kappcontroller for deployment in a gitops maneer without local preprocessing.

My initial goal was not that much reducing verbosity but leveraging IDE built-in code assistance for authoring crossplane XRD and compositions (see crossplane/crossplane#3197 (comment) for screenshot)

I see that in the ytt resources such as the following, the IDE support can't be leveraged

- name: nodepool
base:
apiVersion: container.gcp.crossplane.io/v1beta1
kind: NodePool
spec:
forProvider:
locations:
- us-east1-b

Here is code snippet of the yaml ytt templating logic I've used to leverage IDE coding assistance support to author XRD, and Composition

composition.yaml

#@ load("/cloudsql-resource-data.lib.yaml", "resource")
#@ load("/cloudsql-others-data.lib.yaml", "others")
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: xpostgresqlinstances.gcp.database.orange.com
  labels:
    provider: gcp
spec:
  writeConnectionSecretsToNamespace: 75-crossplane-gcp-cnx-secrets
  compositeTypeRef:
    apiVersion: database.orange.com/v1alpha1
    kind: XPostgreSQLInstance
  #@ resource_yaml_fragment = resource()
  #@ resource = { "base": dict(**resource_yaml_fragment) }
  #@ other_elements = dict(**others()["spec"]["resources"][0])
  #@ other_elements.pop("base")
  #@ resource.update(other_elements)
  #@ resources = [ resource ]
  resources: #@ resources

cloudsql-resource-data.lib.yaml:

#! To be excluded from output, this lib have no yaml document --- header
#@ load("@ytt:data", "data")
#@ def resource():
apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
metadata:
  name: cloudsql
spec:
  forProvider:
    databaseVersion: POSTGRES_14
    region: #@ data.values.external_gcp_region
    settings:
      tier: db-f1-micro
      #!              tier: db-custom-1-3840 # 1 CPU, 3840 MB RAM. See https://cloud.google.com/sql/docs/postgres/create-instance#machine-types
      availabilityType: "ZONAL" #! for HA db use REGIONAL
      backupConfiguration: { #! for HA db ie. REGIONAL set both following to true
        binaryLogEnabled: false,
        enabled: false
      }
      
      #!      dataDiskType: PD_SSD
      dataDiskType: PD_HDD #! cheaper
      ipConfiguration:
        ipv4Enabled: false
        privateNetwork: #@ "projects/{}/global/networks/{}".format(data.values.external_gcp_project_id, data.values.external_gcp_poc_openshift_cluster_vpc_network)
        requireSsl: false
      databaseFlags: #! Orange security requirement
        - name: "log_checkpoints"
          value: "on"
        - name: "log_connections"
          value: "on"
        - name: "log_disconnections"
          value: "on"
        - name: "log_lock_waits"
          value: "on"
        - name: "log_temp_files"
          value: "10000" #! 10,000 KB. The security spec isn't providing a value and suggest "0" as default value:  If all temporary files are not logged, it may be more difficult to identify potential performance issues that may be due to either poor application coding or deliberate resource starvation attempts.

  writeConnectionSecretToRef:
    name: default-name-overriden-by-patch
    namespace: 75-crossplane-gcp-cnx-secrets
#@ end

cloudsql-others-data.lib.yaml:

#! To be excluded from output, this lib have no yaml document --- header
#@ def others():
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
spec:
  resources:
    - patches:
        - type: FromCompositeFieldPath
          fromFieldPath: "metadata.uid"
          toFieldPath: "spec.writeConnectionSecretToRef.name"
          transforms:
            - type: string
              string:
                fmt: "%s-postgresql"
        #!        - fromFieldPath: "spec.parameters.storageGB"
        #!          toFieldPath: "spec.forProvider.settings.dataDiskSizeGb"

        #! Try to implement the service binding spec
        #! https://github.com/servicebinding/spec#provisioned-service
        - type: ToCompositeFieldPath
          fromFieldPath: metadata.labels[crossplane.io/claim-name]
          toFieldPath: status.binding.name

        #!  To facilitate discoverability, it is RECOMMENDED that a CustomResourceDefinition exposing a Provisioned Service add servicebinding.io/provisioned-service: "true" as a label.
        - type: ToCompositeFieldPath
          toFieldPath: metadata.labels[servicebinding.io/provisioned-service]
          fromFieldPath: status.atProvider.settingsVersion #! try to wait for a field which is only available when the resource is ready
          transforms:
            #! first convert int to string
            - type: convert
              convert:
                toType: string
            #! then format string with zero length
            - type: string
              string:
                fmt: "true%.0s"
                #! constant string %.0s prints the string with a max zero width. Otherwise gofmt reports
                #! an error about the argument not being used.
                #! EXTRA string
                #! more at https://pkg.go.dev/fmt
                #! https://github.com/golang/go/issues/8151
                #! interactive test https://go.dev/play/
      connectionDetails:
      - type: FromConnectionSecretKey
        name: host
        fromConnectionSecretKey: privateIP
      - type: FromConnectionSecretKey
        name: username
        fromConnectionSecretKey: username
      - type: FromConnectionSecretKey
        name: password
        fromConnectionSecretKey: password
      - type: FromValue
        name: port
        value: "5432"
      - type:  FromValue
        name: type
        value: "postgresql"
      - type:  FromValue
        name: provider
        value: "gcp cloudsql"
      - type:  FromValue
        name: database #! required by spring-clod-bindings when jdbc-url is missing, see https://github.com/spring-cloud/spring-cloud-bindings#postgresql-rdbms
        value: ""



      base: {}

  compositeTypeRef:
    apiVersion: dummy
    kind: dummy

#@ end

I'm still looking for a solution to reduce proportion of imperative syntax, and reduce starlak editions when adding new resources. The overlays is something I need to explore.

I wonder whether this is something you would be interested in exploring ?

Usage

Hey @vfarcic, please, what is the purpose for your repository ? Is it only targeting your youtube channel and all the things around it or you expect at some point external user to consume it as a devops-toolkit :-) ?

I'm bootstrapping an IDP for my team, and the tooling I want to use is really close to the one you are building.

Do you think it makes sense trying to be with you on this or it doesn't worth it as the purpose would be different ?

Orphan managed resources when deleting GitOpsClaim

Hi @vfarcic ,
how do you manage Crossplane managed resources deletion dependencies?

In my setup I have a cluster created with vcluster and I install a XR called GitOps which does exactly the same as your GitOpsClaim (install ArgoCD, install some projects, some applications...)

As far as I can see when you delete the GitOps XR Crossplane leaves orphaned object resources for ArgoCD Application objects like this.

Is there a way to get rid of them?

Cannot resolve package dependencies

Hi, I'm new to the whole Crossplane ecosystem, and I'm attempting to explore your dot-gitops Configuration.

I see that this configuration depends on the provider-helm provider being installed:

https://github.com/vfarcic/devops-toolkit-crossplane/blob/master/packages/gitops/crossplane.yaml#L13-L14

Which I have installed in my k8s cluster:

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-helm
spec:
  package: xpkg.upbound.io/crossplane-contrib/provider-helm:v0.17.0
$ kubectl get provider.pkg/provider-helm
NAME            INSTALLED   HEALTHY   PACKAGE                                                    AGE
provider-helm   True        True      xpkg.upbound.io/crossplane-contrib/provider-helm:v0.17.0   7m26s

However, when I proceed to install your configuration using the following:

apiVersion: pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: dot-gitops
spec:
  package: xpkg.upbound.io/devops-toolkit/dot-gitops:v0.2.14

I'm getting this error for the config:

Status:
  Conditions:
    Last Transition Time:  2024-03-10T15:27:06Z
    Message:               cannot resolve package dependencies: missing dependencies: [crossplane/provider-helm]
    Reason:                UnknownPackageRevisionHealth
    Status:                Unknown
    Type:                  Healthy
    Last Transition Time:  2024-03-10T15:27:04Z
    Reason:                ActivePackageRevision
    Status:                True
    Type:                  Installed
  Current Identifier:      xpkg.upbound.io/devops-toolkit/dot-gitops:v0.2.14
  Current Revision:        dot-gitops-00637ccddc9b
Events:
  Type     Reason                  Age              From                                      Message
  ----     ------                  ----             ----                                      -------
  Warning  InstallPackageRevision  7s (x6 over 9s)  packages/configuration.pkg.crossplane.io  current package revision health is unknown

I'm not sure how to effectively troubleshoot this issue. Any guidance would be greatly appreciated.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.