vfarcic / devops-toolkit-crossplane Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Thanks for sharing the great work with ytt for reducing verbosity.
I had worked on similar approach using ytt and kappcontroller for deployment in a gitops maneer without local preprocessing.
My initial goal was not that much reducing verbosity but leveraging IDE built-in code assistance for authoring crossplane XRD and compositions (see crossplane/crossplane#3197 (comment) for screenshot)
I see that in the ytt resources such as the following, the IDE support can't be leveraged
composition.yaml
#@ load("/cloudsql-resource-data.lib.yaml", "resource")
#@ load("/cloudsql-others-data.lib.yaml", "others")
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: xpostgresqlinstances.gcp.database.orange.com
labels:
provider: gcp
spec:
writeConnectionSecretsToNamespace: 75-crossplane-gcp-cnx-secrets
compositeTypeRef:
apiVersion: database.orange.com/v1alpha1
kind: XPostgreSQLInstance
#@ resource_yaml_fragment = resource()
#@ resource = { "base": dict(**resource_yaml_fragment) }
#@ other_elements = dict(**others()["spec"]["resources"][0])
#@ other_elements.pop("base")
#@ resource.update(other_elements)
#@ resources = [ resource ]
resources: #@ resources
cloudsql-resource-data.lib.yaml:
#! To be excluded from output, this lib have no yaml document --- header
#@ load("@ytt:data", "data")
#@ def resource():
apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
metadata:
name: cloudsql
spec:
forProvider:
databaseVersion: POSTGRES_14
region: #@ data.values.external_gcp_region
settings:
tier: db-f1-micro
#! tier: db-custom-1-3840 # 1 CPU, 3840 MB RAM. See https://cloud.google.com/sql/docs/postgres/create-instance#machine-types
availabilityType: "ZONAL" #! for HA db use REGIONAL
backupConfiguration: { #! for HA db ie. REGIONAL set both following to true
binaryLogEnabled: false,
enabled: false
}
#! dataDiskType: PD_SSD
dataDiskType: PD_HDD #! cheaper
ipConfiguration:
ipv4Enabled: false
privateNetwork: #@ "projects/{}/global/networks/{}".format(data.values.external_gcp_project_id, data.values.external_gcp_poc_openshift_cluster_vpc_network)
requireSsl: false
databaseFlags: #! Orange security requirement
- name: "log_checkpoints"
value: "on"
- name: "log_connections"
value: "on"
- name: "log_disconnections"
value: "on"
- name: "log_lock_waits"
value: "on"
- name: "log_temp_files"
value: "10000" #! 10,000 KB. The security spec isn't providing a value and suggest "0" as default value: If all temporary files are not logged, it may be more difficult to identify potential performance issues that may be due to either poor application coding or deliberate resource starvation attempts.
writeConnectionSecretToRef:
name: default-name-overriden-by-patch
namespace: 75-crossplane-gcp-cnx-secrets
#@ end
cloudsql-others-data.lib.yaml:
#! To be excluded from output, this lib have no yaml document --- header
#@ def others():
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
spec:
resources:
- patches:
- type: FromCompositeFieldPath
fromFieldPath: "metadata.uid"
toFieldPath: "spec.writeConnectionSecretToRef.name"
transforms:
- type: string
string:
fmt: "%s-postgresql"
#! - fromFieldPath: "spec.parameters.storageGB"
#! toFieldPath: "spec.forProvider.settings.dataDiskSizeGb"
#! Try to implement the service binding spec
#! https://github.com/servicebinding/spec#provisioned-service
- type: ToCompositeFieldPath
fromFieldPath: metadata.labels[crossplane.io/claim-name]
toFieldPath: status.binding.name
#! To facilitate discoverability, it is RECOMMENDED that a CustomResourceDefinition exposing a Provisioned Service add servicebinding.io/provisioned-service: "true" as a label.
- type: ToCompositeFieldPath
toFieldPath: metadata.labels[servicebinding.io/provisioned-service]
fromFieldPath: status.atProvider.settingsVersion #! try to wait for a field which is only available when the resource is ready
transforms:
#! first convert int to string
- type: convert
convert:
toType: string
#! then format string with zero length
- type: string
string:
fmt: "true%.0s"
#! constant string %.0s prints the string with a max zero width. Otherwise gofmt reports
#! an error about the argument not being used.
#! EXTRA string
#! more at https://pkg.go.dev/fmt
#! https://github.com/golang/go/issues/8151
#! interactive test https://go.dev/play/
connectionDetails:
- type: FromConnectionSecretKey
name: host
fromConnectionSecretKey: privateIP
- type: FromConnectionSecretKey
name: username
fromConnectionSecretKey: username
- type: FromConnectionSecretKey
name: password
fromConnectionSecretKey: password
- type: FromValue
name: port
value: "5432"
- type: FromValue
name: type
value: "postgresql"
- type: FromValue
name: provider
value: "gcp cloudsql"
- type: FromValue
name: database #! required by spring-clod-bindings when jdbc-url is missing, see https://github.com/spring-cloud/spring-cloud-bindings#postgresql-rdbms
value: ""
base: {}
compositeTypeRef:
apiVersion: dummy
kind: dummy
#@ end
I'm still looking for a solution to reduce proportion of imperative syntax, and reduce starlak editions when adding new resources. The overlays is something I need to explore.
I wonder whether this is something you would be interested in exploring ?
Hey @vfarcic, please, what is the purpose for your repository ? Is it only targeting your youtube channel and all the things around it or you expect at some point external user to consume it as a devops-toolkit
:-) ?
I'm bootstrapping an IDP for my team, and the tooling I want to use is really close to the one you are building.
Do you think it makes sense trying to be with you on this or it doesn't worth it as the purpose would be different ?
Hi @vfarcic ,
how do you manage Crossplane managed resources deletion dependencies?
In my setup I have a cluster created with vcluster and I install a XR called GitOps which does exactly the same as your GitOpsClaim (install ArgoCD, install some projects, some applications...)
As far as I can see when you delete the GitOps XR Crossplane leaves orphaned object resources for ArgoCD Application objects like this.
Is there a way to get rid of them?
Hi, I'm new to the whole Crossplane ecosystem, and I'm attempting to explore your dot-gitops Configuration.
I see that this configuration depends on the provider-helm
provider being installed:
Which I have installed in my k8s cluster:
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-helm
spec:
package: xpkg.upbound.io/crossplane-contrib/provider-helm:v0.17.0
$ kubectl get provider.pkg/provider-helm
NAME INSTALLED HEALTHY PACKAGE AGE
provider-helm True True xpkg.upbound.io/crossplane-contrib/provider-helm:v0.17.0 7m26s
However, when I proceed to install your configuration using the following:
apiVersion: pkg.crossplane.io/v1
kind: Configuration
metadata:
name: dot-gitops
spec:
package: xpkg.upbound.io/devops-toolkit/dot-gitops:v0.2.14
I'm getting this error for the config:
Status:
Conditions:
Last Transition Time: 2024-03-10T15:27:06Z
Message: cannot resolve package dependencies: missing dependencies: [crossplane/provider-helm]
Reason: UnknownPackageRevisionHealth
Status: Unknown
Type: Healthy
Last Transition Time: 2024-03-10T15:27:04Z
Reason: ActivePackageRevision
Status: True
Type: Installed
Current Identifier: xpkg.upbound.io/devops-toolkit/dot-gitops:v0.2.14
Current Revision: dot-gitops-00637ccddc9b
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning InstallPackageRevision 7s (x6 over 9s) packages/configuration.pkg.crossplane.io current package revision health is unknown
I'm not sure how to effectively troubleshoot this issue. Any guidance would be greatly appreciated.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.