vmware-labs / distribution-tooling-for-helm Goto Github PK

Describe the bug

This might be a intentional beheaviour and me simply using this plugin wrong. When I'm unwrapping a chart, I'm getting the following error:

Failed to push images: failed to verify Helm chart Images.lock: Images.lock does not validate:                                    
Helm chart "benthos": image "localhost:5000/jeffail/benthos:4.11.0": digests do not match:
- sha256:e2fa170d1d0c2e7bf5c89986b6e5de471fe6499f231d6f8e9c1f31c4350353b0
+ sha256:85689f09ddac1449188c9b9c50138027ff476bab81e39d4229f28eb72793314b
Helm chart "benthos": image "localhost:5000/jeffail/benthos:4.11.0": digests do not match:
- sha256:244e9c77b90ec18e692acf162c55060156b1d36a820d19a8606f0a98e55d4ee3
+ sha256:f794f3b17252736664244d508c6ffc10f05619cc6d09bf4c4d465d1815e0afa7

Reproduction steps

Setup a OCI-based registry locally using docker:

docker run -d -p 5000:5000 --restart=always --name registry registry:2

Run the following script

#!/bin/bash

helm repo add temp --force-update https://benthosdev.github.io/benthos-helm-chart/
helm repo update

helm pull temp/benthos --version 0.8.0 --untar --untardir .

# This helm chart isn't annotated by default, so I'm manually doing this here
images_override=$(cat <<-END
- name: benthos
  image: docker.io/jeffail/benthos:4.11.0
END
)
yq -i e ".annotations.images |= \"$images_override\"" ./benthos/Chart.yaml

helm dt wrap  ./benthos
helm dt unwrap  benthos-0.8.0.wrap.tgz oci://localhost:5000 --yes

After this script has ran, a new directory named benthos is present in the current working directory.

.
└── benthos/
│ ├── images/
│ │ ├── 244e9c77b90ec18...tar (linux/arm64)
│ │ └── e2fa170d1d0c2e7...tar (linux/amd64)
│ └── Images.lock
│ └── ....
└── benthos-0.8.0.wrap.tgz

When untaring benthos-0.8.0.wrap.tgz you get the same content as ./benthos.

Expected behavior

1. The helm chart and docker images is being relocated and pushed to the local registry

2. The benthos-0.8.0.wrap.tgz wound have the Image.lock relocated and Image.lock look like this:

apiVersion: v0
kind: ImagesLock
metadata:
  generatedAt: "2023-11-30T09:27:39.313962408Z"
  generatedBy: Distribution Tooling for Helm
chart:
  name: benthos
  version: 0.8.0
  appVersion: 4.11.0
images:
  - name: benthos
    image: localhost:5000/jeffail/benthos:4.11.0
    chart: benthos
    digests:
      - digest: sha256:e2fa170d1d0c2e7bf5c89986b6e5de471fe6499f231d6f8e9c1f31c4350353b0
        arch: linux/amd64
      - digest: sha256:244e9c77b90ec18e692acf162c55060156b1d36a820d19a8606f0a98e55d4ee3
        arch: linux/arm64

instead of this:

apiVersion: v0
kind: ImagesLock
metadata:
  generatedAt: "2023-12-05T10:32:00.611186491Z"
  generatedBy: Distribution Tooling for Helm
chart:
  name: benthos
  version: 0.8.0
  appVersion: 4.11.0
images:
  - name: benthos
    image: docker.io/jeffail/benthos:4.11.0
    chart: benthos
    digests:
      - digest: sha256:e2fa170d1d0c2e7bf5c89986b6e5de471fe6499f231d6f8e9c1f31c4350353b0
        arch: linux/amd64
      - digest: sha256:244e9c77b90ec18e692acf162c55060156b1d36a820d19a8606f0a98e55d4ee3
        arch: linux/arm64

Additional context

I'm able to produce the digests found in the error, when doing the following addtional steps(after the bash script previously mentioned):

> helm dt charts relocate benthos/ oci://localhost:5000
> helm dt images lock ./benthos
> cat Images.lock

apiVersion: v0
kind: ImagesLock
metadata:
  generatedAt: "2023-12-05T13:48:53.107936871Z"
  generatedBy: Distribution Tooling for Helm
chart:
  name: benthos
  version: 0.8.0
  appVersion: 4.11.0
images:
  - name: benthos
    image: localhost:5000/jeffail/benthos:4.11.0
    chart: benthos
    digests:
      - digest: sha256:85689f09ddac1449188c9b9c50138027ff476bab81e39d4229f28eb72793314b
        arch: linux/amd64
      - digest: sha256:f794f3b17252736664244d508c6ffc10f05619cc6d09bf4c4d465d1815e0afa7
        arch: linux/arm64

So this seems to me that the relocate changes the digest of the images. I'm assuming this is some metadata within the images that is changed.

However this change seems to be messing with the actual digest which is used when comparing.

Is your feature request related to a problem? Please describe.

When relocating a Helm Chart with one of more dependencies the repository for said dependencies is not updated to the new location in the parent Chart.yaml. Meaning when installing the newly relocated parent chart it still pulls the original dependencies that contain the original image references.

Describe the solution you'd like

When relocating/unwrapping to a new repository the dependency repository references in Chart.yaml are updated to the new location. Thus, referencing the relocated subcharts that have references to the relocated images.

Describe alternatives you've considered

Besides manual intervention, no other alternatives considered.

Additional context

No response

Is your feature request related to a problem? Please describe.

My personal usecase is a mirroring tool, that mirrors helm charts to an internal OCI registry. So fare this is done using custom bash script, which takes a yaml file that looks like the following:

charts:
  - name: redis
    type: oci
    repository: oci://registry-1.docker.io/bitnamicharts
    version: 18.1.5

  - name: benthos
    repository: https://benthosdev.github.io/benthos-helm-chart/
    version: 0.8.0
    images_override:
    - name: benthos
      image: docker.io/jeffail/benthos:4.11.0

This bash script will obviously wrap and unwrap the charts, to the target repository. However on top of that, I've decided to store the Images.lock files. Both the original Images.lock file (The source) and the relocated Images.lock file (The target).

This is stored within a repository, to help keep track of the mirroed helm charts.

.
└── charts/
    ├── benthos/
    │   └── 0.8.0/
    │       ├── Images.source.lock
    │       └── Images.target.lock
    └── redis/
        └── 18.1.5/
            ├── Images.source.lock
            └── Images.target.lock

The problem is that the generated Images.lock file, currently does not contain the souce of where the helm chart were pulled from. Which would be nice to locate it in the future.

Describe the solution you'd like

My suggestion is simply adding the repository to within the Images.lock like this:

apiVersion: v0
kind: ImagesLock
metadata:
  generatedAt: "2023-12-06T14:51:01.632471183Z"
  generatedBy: Distribution Tooling for Helm
chart:
  name: benthos
  version: 0.8.0
  appVersion: 4.11.0
  repository: https://benthosdev.github.io/benthos-helm-chart/ <--- Like this
images:
  - name: benthos
    image: docker.io/jeffail/benthos:4.11.0
    chart: benthos
    digests:
      - digest: sha256:e2fa170d1d0c2e7bf5c89986b6e5de471fe6499f231d6f8e9c1f31c4350353b0
        arch: linux/amd64
      - digest: sha256:244e9c77b90ec18e692acf162c55060156b1d36a820d19a8606f0a98e55d4ee3
        arch: linux/arm64

However a different name might be prefered, as one might not consider a OCI-registry a "repository". Can't up with a better term myself at this moment.

Describe alternatives you've considered

No response

Additional context

No response

Is your feature request related to a problem? Please describe.

When performing a charts relocate CHART_PATH OCI_URI only values.yaml is updated. This can be very handy to apply the relocation to all values_smth.yaml present in the chart. Charts can include various values_smth.yaml files that are used depending of the context.

Describe the solution you'd like

The cli could provide an extra flag like helm

 -f, --values strings                             specify values in a YAML file or a URL (can specify multiple) 

If no flag applied keep the current behaviour be applying update only to values.yaml.
If flag applied update only files pass as argument.

Describe alternatives you've considered

No response

Additional context

Changes in pkg/relocator/chart.go is needed to take in account additional files fortunately helm chart loader already load all canonical (Chart.yaml, values.yaml, values.schema.json, templates/, charts/ ) and other non canonical files already.