Git Product home page Git Product logo

kustomize-plugin-merger's Introduction

CI CodeQL OpenSSF Best Practices OpenSSF Scorecard Go Report Card codecov GitHub Release Docker Go Reference Renovate PRs Welcome

๐Ÿ”€ Merger ๐Ÿ”€

A Kustomize generator plugin to merge YAML files seamlessly for real-world use cases.

Merger provides schemaless merge with different merge strategies (StrategicMerge).

Why

The main goal of the Merger plugin is to provide a more flexible way to merge YAML files and extend the strict merge capacities of Kustomize.

Kustomize's default patch strategy for the lists (arrays) is replace, and to change that for the Kubernetes Custom Resource, you must provide the OpenAPI schema of that custom resource, which is only helpful if the lists of that CR have a unique id.

Many people are asking for such functionalities as easy merging CRs, creating a ConfigMap from arbitrary YAML files, and for more details on the challenge of providing OpenAPI schema to merge files, please read the following post: Set OpenAPI patch strategy for Kubernetes Custom Resources.

Features

  • Generate multiple resources/manifests from a single base without copying the resources multiple times.
  • Merge any manifests (even CustomResources) without needing their OpenAPI schema.
  • Merge applications configuration YAML files into a ConfigMap or Secret.
  • Merge manifests with a list of maps without a unique identifier (when using x-kubernetes-patch-merge-key is not possible).
  • Merge YAML files with different merge strategies (StrategicMerge).

Options

---
apiVersion: generators.kustomize.devopshive.net/v1alpha1
kind: Merger
metadata:
  name: merge
    annotations:
      # Containerized KRM function.
      config.kubernetes.io/function: |
        container:
          image: ghcr.io/devopshivehq/kustomize-generator-merger
          mounts:
          - type: bind
            src: ./
            dst: /mnt
      # Exec KRM functions.
      # config.kubernetes.io/function: |
      #   exec:
      #     path: kustomize-plugin-merger
spec:
  resources:
  - name: example
    input:
      # Available options: overlay,patch.
      # - Overlay: Produce multiple outputs by merging each source with the destination.
      # - Patch: Produce a single output by merging all sources together then with the destination.
      method: overlay
      files:
        # The same as in the KRM container above, omit it if Exec KRM is used.
        root: /mnt
        sources:
        - src01.yaml
        - src02.yaml
        destination: dst.yaml
    merge:
      # Available options: replace,append,combine.
      # - Replace: All keys in source will merge and replace what's in the destination.
      # - Append: Maps from source merged with destination, but the lists will be appended from source to destination.
      # - Combine: Maps from source merged with destination, but the lists will be combined together.
      strategy: combine
    output:
      # Available options: raw,configmap,secret
      format: raw

Common use cases

This section shows a couple of use cases where Merger can help.

1. Generate multiple manifests from a single base

In this case, you have multiple CronJobs, all of which share the same body, but each has a different command or other config.

Read the full example.

2. Merge non-manifest files and store them into ConfigMap or Secret

No plans from Kustomize to support non-manifest files merge and storing them into ConfigMap or Secret. Using Merger you can merge any YAML files like application configuration.

Read the full example.

3. Merge lists in manifests without schema or a unique identifier

Currently, in Kustomize, it's not possible to merge resources without a unique identifier, even with Open API schema.

It's possible to do that using the merge strategy append in Merger (later on, combineWithKey will also be supported).

Read the full example.

4. Organize long manifests into smaller ones

In some use cases (e.g., Crossplane Compositions), you could have a really long YAML manifest, and it's hard to read. You can split that file and use the Merger patch input method to make it a single manifest again.

Read the full example.

TO-DO

  • Support combine merge strategy with an identifier key (similar to x-kubernetes-patch-merge-key).
  • Configure the output indentation.
  • Provide better docs for Merger options.

Project status

Please note that this project is still under development and could have breaking changes, but it will follow the SemVer convention.

Contributing

All feedback and contributions are welcome. If you find any issues or want to contribute, please feel free to fill an issue or create a PR.

License

Merger is an open-source software licensed using the Apache License 2.0. Please see LICENSE for details.

kustomize-plugin-merger's People

Contributors

aabouzaid avatar github-actions[bot] avatar renovate[bot] avatar step-security-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

kustomize-plugin-merger's Issues

Image is missing

Thanks for this plugin, it's exactly what I needed!

The container image seems to be missing, though:

$ docker run ghcr.io/devopshivehq/kustomize-generator-merger -v
Unable to find image 'ghcr.io/devopshivehq/kustomize-generator-merger:latest' locally
docker: Error response from daemon: denied.
See 'docker run --help'.

I have a token issued which should authenticate me to download the image.

And https://github.com/orgs/DevOpsHiveHQ/packages?repo_name=kustomize-plugin-merger shows no images.

I got it working by compiling the image and exec-ing locally, but this is not ideal for CI environments.

argocd support

Hi

This plugin can work with argocd ?

the krm function use docker for build manifests with embedded bin, but can't use docker in argocd

any suggestion ?

Thanks

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

  • chore(deps): update all non-major dependencies (GoogleCloudPlatform/release-please-action, actions/dependency-review-action, renovate/renovate)

Detected dependencies

github-actions
.github/workflows/deps-renovate.yaml
  • actions/checkout v4@b4ffde65f46336ab88eb53be808477a3936bae11
  • renovate/renovate slim@sha256:7a93004158a5e30f4eaee974d6acda5005f2a03d6e36a4157cc6b566cf792739
.github/workflows/deps-review.yml
  • step-security/harden-runner v2.6.0@1b05615854632b887b69ae1be8cbefe72d3ae423
  • actions/checkout v4.1.1@b4ffde65f46336ab88eb53be808477a3936bae11
  • actions/dependency-review-action v3.1.0@6c5ccdad469c9f8a2996bfecaec55a631a347034
.github/workflows/go-ci.yml
  • actions/checkout v4@b4ffde65f46336ab88eb53be808477a3936bae11
  • actions/setup-go v4@93397bea11091df50f3d7e59dc26a7711a8bcfbe
  • golangci/golangci-lint-action v3@3a919529898de77ec3da873e3063ca4b10e7f5cc
  • codecov/codecov-action v3@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d
  • actions/cache v3@704facf57e6136b1bc63b828d79edcd491f0ee84
  • actions/upload-artifact v3@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
  • actions/upload-artifact v3@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
.github/workflows/go-pre-release.yml
  • GoogleCloudPlatform/release-please-action v3@4c5670f886fe259db4d11222f7dff41c1382304d
.github/workflows/sec-codeql.yml
  • actions/checkout v4@b4ffde65f46336ab88eb53be808477a3936bae11
  • actions/setup-go v4@93397bea11091df50f3d7e59dc26a7711a8bcfbe
  • actions/cache v3@704facf57e6136b1bc63b828d79edcd491f0ee84
  • github/codeql-action v2@74483a38d39275f33fcff5f35b679b5ca4a26a99
  • github/codeql-action v2@74483a38d39275f33fcff5f35b679b5ca4a26a99
  • github/codeql-action v2@74483a38d39275f33fcff5f35b679b5ca4a26a99
.github/workflows/sec-scorecard.yml
  • actions/checkout v4.1.1@b4ffde65f46336ab88eb53be808477a3936bae11
  • ossf/scorecard-action v2.3.1@0864cf19026789058feabb7e87baa5f140aac736
  • actions/upload-artifact v3.1.3@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
  • github/codeql-action v2.22.5@74483a38d39275f33fcff5f35b679b5ca4a26a99
.github/workflows/tpl-packaging.yml
  • actions/checkout v4@b4ffde65f46336ab88eb53be808477a3936bae11
  • actions/setup-go v4@93397bea11091df50f3d7e59dc26a7711a8bcfbe
  • actions/cache v3@704facf57e6136b1bc63b828d79edcd491f0ee84
  • docker/login-action v3@343f7c4344506bcbf9b4de18042ae17996df046d
  • goreleaser/goreleaser-action v5@7ec5c2b0c6cdda6e8bbb49444bc797dd33d74dd8
  • actions/cache v3@704facf57e6136b1bc63b828d79edcd491f0ee84
gomod
go.mod
  • go 1.21
  • dario.cat/mergo v1.0.0
  • github.com/knadh/koanf/parsers/yaml v0.1.0
  • github.com/knadh/koanf/providers/file v0.1.0
  • github.com/knadh/koanf/v2 v2.0.1
  • github.com/stretchr/testify v1.8.4
  • k8s.io/apimachinery v0.28.3
  • k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00@2dd684a91f00
  • sigs.k8s.io/kustomize/kyaml v0.15.0

  • Check this box to trigger a request for Renovate to run again on this repository

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: .github/renovate.json5
Error type: The renovate configuration file contains some invalid settings
Message: Invalid schedule: 'Invalid schedule: "0 4 1,16 * *" has cron syntax, but doesn't have * as minutes'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.