hayorov / helm-gcs Goto Github PK

View Code? Open in Web Editor NEW

278.0 7.0 63.0 19.59 MB

Manage Helm 3 repositories on Google Cloud Storage 🔐 **privately**

License: MIT License

Shell 5.68% Go 94.32%

helm helm-plugin google-cloud-storage kubernetes bucket gcs helm-gcs chart

helm-gcs's Introduction

helm-gcs

helm-gcs is a helm plugin that allows you to manage private helm repositories on Google Cloud Storage aka buckets.

Installation

Install the stable version:

$ helm plugin install https://github.com/hayorov/helm-gcs.git

Update to latest

$ helm plugin update gcs

Install a specific version:

$ helm plugin install https://github.com/hayorov/helm-gcs.git --version 0.4.0

Quick start

# Init a new repository
$ helm gcs init gs://bucket/path

# Add your repository to Helm
$ helm repo add repo-name gs://bucket/path

# Push a chart to your repository
$ helm gcs push chart.tar.gz repo-name

# Update Helm cache
$ helm repo update

# Fetch the chart
$ helm fetch repo-name/chart

# Remove the chart
$ helm gcs rm chart repo-name

Documentation

Authentification

To authenticate against GCS you can:

Use the application default credentials
Use a service account via export GOOGLE_APPLICATION_CREDENTIALS=credentials.json system variable
Use a temporary OAuth 2.0 access token via export GOOGLE_OAUTH_ACCESS_TOKEN=<MY_ACCESS_TOKEN> environment variable. When used, plugin will ignore other authentification methods.

See GCP documentation for more information.

Create a repository

First, you need to create a bucket on GCS, which will be used by the plugin to store your charts.

Then you have to initialize a repository at a specific location in your bucket:

$ helm gcs init gs://your-bucket/path

You can create a repository anywhere in your bucket.

This command does nothing if a repository already exists at the given location.

You can now add the repository to helm:

$ helm repo add my-repository gs://your-bucket/path

Push a chart

Package the chart:

$ helm package my-chart

This will create a file my-chart-<semver>.tgz.

Now, to push the chart to the repository my-repository:

$ helm gcs push my-chart-<semver>.tgz my-repository

Push the chart with additional option by providing metadata to the object :

$ helm gcs push my-chart-<semver>.tgz my-repository --metadata env=my-env,region=europe-west4

Push the chart with additional option by providing path inside bucket :

This would allow us to structure the content inside the bucket, and stores at gs://your-bucket/my-application/my-chart-<semver>.tgz

$ helm gcs push my-chart-<semver>.tgz my-repository --bucketPath=my-application

If you got this error:

Error: update index file: index is out-of-date

That means that someone/something updated the same repository, at the same time as you. You just need to execute the command again or, next time, use the --retry flag to automatically retry to push the chart.

Once the chart is uploaded, use helm to fetch it:

# Update local repo cache if necessary
# $ helm repo update

$ helm fetch my-chart

This command does nothing if the same chart (name and version) already exists.

Using --retry is highly recommended in a CI/CD environment.

Remove a chart

You can remove all the versions of a chart from a repository by running:

$ helm gcs remove my-chart my-repository

To remove a specific version, simply use the --version flag:

$ helm gcs remove my-chart my-repository --version 0.1.0

Don't forget to run helm repo up after you remove a chart.

Troubleshooting

You can use the global flag --debug, or set HELM_GCS_DEBUG=true to get more informations. Please write an issue if you find any bug.

Helm versions

Starting from 0.3 helm-gcs works with Helm 3, if you want to use it with Helm 2 please install the latest version that supports it

helm plugin install https://github.com/hayorov/helm-gcs.git --version 0.2.2 # helm 2 compatible

helm-gcs's People

Contributors

Stargazers

Watchers

helm-gcs's Issues

.helm\plugins\helm-gcs\scripts\pull.sh: %1 is not a valid Win32 application.

Hitting the following while adding repo:

helm repo add oceana-gcp gs://my-team-helm-repo --debug
Error: Looks like "gs://my-team-helm-repo" is not a valid chart repository or cannot be reached:
fork/exec C:\Users\me\.helm\plugins\helm-gcs\scripts\pull.sh: %1 is not a valid Win32 application.

$ ~/.helm/plugins/helm-gcs/bin/helm-gcs.exe version
version: 0.2.0
commit: 12854b9ec61b242dba21786181b9af5a4e7692c6
date: 2018-05-31T11:11:26Z

helm version
Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}

CVE-2020-29652 in golang.org/x/crypto with high severity

Version 0.3.11 has the following vulnerability:

Total: 1 (HIGH: 1, CRITICAL: 0)

+---------------------+------------------+----------+------------------------------------+------------------------------------+
|       LIBRARY       | VULNERABILITY ID | SEVERITY |         INSTALLED VERSION          |           FIXED VERSION            |
+---------------------+------------------+----------+------------------------------------+------------------------------------+
| golang.org/x/crypto | CVE-2020-29652   | HIGH     | v0.0.0-20201002170205-7f63de1d35b0 | v0.0.0-20201216223049-8b5274cf687f |
+---------------------+------------------+----------+------------------------------------+------------------------------------+

Can this module get updated?

Error: storage: object doesn't exist when installing/upgrading a chart

I went through all steps from the readme and it works till I try to install a chart.

Running

$ helm --namespace=dev upgrade --install --atomic --cleanup-on-fail --timeout 5m --verify --values=values.dev.yaml admin my-repo/admin --version=0.1.0
Error: storage: object doesn't exist
Usage:
  helm-gcs pull gs://bucket/path [flags]

Flags:
  -h, --help   help for pull

Global Flags:
      --debug                    activate debug
      --service-account string   service account to use for GCS

Error: failed to download "my-repo/admin" (hint: running `helm repo update` may help)

Though, I see the chart package downloaded to the helm cache folder.

$ helm version                                                                                                                                                                        
version.BuildInfo{Version:"v3.0.3", GitCommit:"ac925eb7279f4a6955df663a0128044a8a6b7593", GitTreeState:"clean", GoVersion:"go1.13.6"}
$ helm plugin list                           
NAME  VERSION DESCRIPTION                                
gcs     0.3.1   Manage repositories on Google Cloud Storage

Repo initialisation subcommand creates an index.yaml with a wrong Content-Type

When initialising a new chart repo the plugin creates the initial index.yaml file with a wrong Content-Type metadata attribute.

$ helm gcs init gs://bucket/path
$ gsutil ls -L gs://bucket/path
gs://bucket/path/index.yaml:
    Creation time:          Tue, 20 Jul 2021 00:00:08 GMT
    Update time:            Tue, 20 Jul 2021 00:00:08 GMT
    Storage class:          STANDARD
    Cache-Control:          no-cache, max-age=0, no-transform
    Content-Length:         73
    Content-Type:           text/plain; charset=utf-8
    Hash (crc32c):          heU+XQ==
    Hash (md5):             nS8RrwZ+mwH4lY4OjrqUqg==
    ETag:                   CMP0veWr8PECEAE=
    Generation:             1626739208845891
    Metageneration:         1
    ACL:                    []
TOTAL: 1 objects, 73 bytes (73 B)

The index.yaml Content-Type metadata attribute should be text/yaml or text/x-yaml (according to https://helm.sh/docs/topics/chart_repository/) instead of text/plain.

add "--dry-run" option for push

Can we have "helm gcs push" with "--dry-run" to check if version exists without actually pushing similar to hypnoglow/helm-s3#44?

Add flag to allow `helm gcs push` to fail silently if chart/version already exists

I'm pushing from a CI pipeline and don't want it to fail if the chart version has already been pushed.

Is this something you would consider adding support for?

Helm3 not detecting latest charts

I'm using 'version: "0.3.1' version of the hayorov/helm-gcs plugin. Using helm version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}.

Running "helm search repo my-gcp-helm-repo" returns:
NAME CHART VERSION APP VERSION DESCRIPTION my-gcp-helm-repo/credentials 2.2.0+12 2.2.0 A Helm chart for deploying configmaps and secrets

but 2.2.0+17 exists in the bucket. Why is that not seen as the latest version?

Error installing on Windows 10

Got the following error installing on Windows 10:

helm plugin install https://github.com/nouney/helm-gcs --version 0.1.4
Error: symlink C:\Users\XXX\.helm\cache\plugins\https-github.com-nouney-helm-gcs C:\Users\XXX\.helm\plugins\helm-gcs: A required privilege is not held by the client.

This was in GitBash.

Error when pushing packages to gcs repo

Hi,

We are currently receiving the following error when trying to run helm gcs push package repo, any ideas:

Usage:
  helm-gcs push [chart] [repository] [flags]

Flags:
  -h, --help   help for push

storage: object doesn't exist
Error: plugin "gcs" exited with error

Init removes everything from the bucket

I accidentally ran helm gcs init for a bucket that already was a helm repo, and found out that after that it was completely empty.

It would be great if the plugin could warn that the bucket is not empty before running this operation.

Support SemVer v2 for Chart version

Hello

I've successfully init my helm repo and pushed my chart in my repo.
When done, I can see my chart in the bucket with the associated index.yaml file.

But helm search repo my-repo returns:
No result found.

How can I debug what's wrong ?

helm version
version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}

Helm-gcs Plugin version : 0.3.1

Thanks for the help

Version 0.3.1 reports as like it is 0.3.0

➜ helm plugin install https://github.com/hayorov/helm-gcs --version 0.3.1
Installing helm-gcs 0.3.0 ...
x LICENSE
x README.md
x helm-gcs
helm-gcs 0.3.0 is correctly installed.

Init a new repository:
  helm gcs init gs://bucket/path

Add your repository to Helm:
  helm repo add repo-name gs://bucket/path

Push a chart to your repository:
  helm gcs push chart.tar.gz repo-name

Update Helm cache:
  helm repo update

Get your chart:
  helm fetch repo-name/chart

Installed plugin: gcs

I see no version update in two commits related to 0.3.1 release:
98dcdca
290becd

Installation failed ./scripts/install.sh: 20: ./scripts/install.sh: [[: not found

Hello,

First off, congrats on taking over the repo!

So my pipeline to deploy helm charts is breaking since helm-gcs 0.2.1 with the following error:

Step #3: Installing helm-gcs 0.2.1 ...
Step #3: ./scripts/install.sh: 20: ./scripts/install.sh: [[: not found
Step #3: 
Step #3: gzip: stdin: not in gzip format
Step #3: tar: Child returned status 1
Step #3: tar: Error is not recoverable: exiting now
Step #3: helm-gcs 0.2.1 is correctly installed.
Step #3: 
Step #3: Init a new repository:
Step #3:   helm gcs init gs://bucket/path
Step #3: 
Step #3: Add your repository to Helm:
Step #3:   helm repo add repo-name gs://bucket/path
Step #3: 
Step #3: Push a chart to your repository:
Step #3:   helm gcs push chart.tar.gz repo-name
Step #3: 
Step #3: Update Helm cache:
Step #3:   helm repo update
Step #3: 
Step #3: Get your chart:
Step #3:   helm fetch repo-name/chart
Step #3: 
Step #3: Installed plugin: gcs
Step #3: /builder/home/.helm/plugins/helm-gcs/scripts/pull.sh: 3: /builder/home/.helm/plugins/helm-gcs/scripts/pull.sh: /builder/home/.helm/plugins/helm-gcs/bin/helm-gcs: not found
Step #3: Error: Looks like "gs://my-charts-bucket" is not a valid chart repository or cannot be reached: plugin "scripts/pull.sh" exited with error
Step #3: Adding repo my-repo gs://my-charts-bucket
Step #3: in helmfile.yaml: helm exited with status 1:
Step #3:   /builder/home/.helm/plugins/helm-gcs/scripts/pull.sh: 3: /builder/home/.helm/plugins/helm-gcs/scripts/pull.sh: /builder/home/.helm/plugins/helm-gcs/bin/helm-gcs: not found
Step #3:   Error: Looks like "gs://my-charts-bucket" is not a valid chart repository or cannot be reached: plugin "scripts/pull.sh" exited with error
Finished Step #3

Any ideas?

[improvement] build a static binary

I was trying to run on Alpine Linux but helm-gcs is dynamically linked.
Would you consider a statically build binary?

bash-4.3# ls -la helm-gcs 
-rwxr-xr-x    1 jenkins  jenkins    9606656 Oct 31 14:59 helm-gcs
bash-4.3# ./helm-gcs --help
bash: ./helm-gcs: No such file or directory
bash-4.3# file helm-gcs 
helm-gcs: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, stripped
bash-4.3# ldd helm-gcs 
	/lib64/ld-linux-x86-64.so.2 (0x7fbec083b000)
	libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fbec083b000)
	libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fbec083b000)
bash-4.3# cat /etc/issue 
Welcome to Alpine Linux 3.6
Kernel \r on an \m (\l)

bash-4.3# ls -la /lib64/ld-linux-x86-64.so.2
ls: /lib64/ld-linux-x86-64.so.2: No such file or directory
bash-4.3#

Current workaround:

bash-4.3# wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.25-r0/glibc-2.25-r0.apk
Connecting to github.com (192.30.253.113:443)
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (52.216.16.120:443)
glibc-2.25-r0.apk    100% |****************************************************************************************|  2902k  0:00:00 ETA
bash-4.3# apk add --allow-untrusted glibc-2.25-r0.apk
WARNING: Ignoring APKINDEX.84815163.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.24d64ab1.tar.gz: No such file or directory
(1/1) Installing glibc (2.25-r0)
OK: 177 MiB in 72 packages
bash-4.3# ./helm-gcs --help
Manage Helm repositories on Google Cloud Storage

Usage:
  helm-gcs [command]

Available Commands:
  help        Help about any command
  init        Init a repository
  pull        pull a file from GCS
  push        Push a chart into a GCS repository
  remove      Remove a chart from a GCS repository
  version     Print current version

Flags:
  -h, --help   help for helm-gcs

Use "helm-gcs [command] --help" for more information about a command.

Helm 3 support adoption

I'm trying to push image to the repository using Helm v3, I was able to successfully run helm gcs init and add repository to my list of repos. But when I'm trying to do the push I see the following

Error: entry: load: Couldn't load repositories file (~/Library/helm/repository/repositories.yaml).
You might need to run `helm init` (or `helm init --client-only` if tiller is already installed)
Usage:
  helm-gcs push [chart.tar.gz] [repository] [flags]

Flags:
      --force   upload the chart even if already indexed
  -h, --help    help for push
      --retry   retry if the index changed

Global Flags:
      --debug                    activate debug
      --service-account string   service account to use for GCS

entry: load: Couldn't load repositories file (~/Library/helm/repository/repositories.yaml).
You might need to run `helm init` (or `helm init --client-only` if tiller is already installed)
Error: plugin "gcs" exited with error```

Remove only delete info on index but fetch always show them

I deploy my chart

helm gcs push my-chart-0.0.1.tgz my-repo

After that i fetch it

helm fetch my-repo/my-chart --version 0.0.1

All is good.

But now i want to delete it;

helm gcs remove my-repo/my-chart

On my index, it is removed but tgz file always exist.

And fetch always see it.

is it possible to fetch chart only if it is present on index?
And add option on remove to force delete tgz?

Version 0.3.9 is shown as 0.3.7

The two most recent releases (v0.3.8 and v0.3.9) have not had their version reflected in plugin.yaml. I suspect that is the reason for the following, unexpected behavior:

$ helm plugin install https://github.com/hayorov/helm-gcs.git --version 0.3.9
Installing helm-gcs 0.3.7 ...

Expected behavior:

$ helm plugin install https://github.com/hayorov/helm-gcs.git --version 0.3.9
Installing helm-gcs 0.3.9 ...

Error 403: Insufficient Permission - version

I'm facing Error 403: Insufficient Permission when I push the repo to gcs.
helm version is 2.9.1
helm-gs is 0.2.2

I tested with gsutil and it worked fine, so it isn't related to IAM permissions.

helm gcs push repo/*.tgz helm-charts --debug
DEBU[0000] helm home: /root/.helm                       
DEBU[0000] load index file "gs://helm-charts/repo/index.yaml" 
DEBU[0000] index file generation: 1574099372461929      
DEBU[0000] load chart "repo/api-1.0.536.tgz" (force=false, retry=false) 
DEBU[0000] chart loaded: api-1.0.536        
DEBU[0000] indexing chart 'api-1.0.536' as 'api-1.0.536.tgz' (base url: gs://helm-charts/repo) 
DEBU[0000] push index file                              
DEBU[0000] update condition: if generation = 1574099372461929 
Error: update index file: close: googleapi: Error 403: Insufficient Permission, insufficientPermissions
Usage:
  helm-gcs push [chart.tar.gz] [repository] [flags]

Cloud you please help?
Is there any compatibility matrix gcloud-sdk x helm x helm-gcs?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Other Branches

These updates are pending. To force PRs open, click the checkbox below.

chore(deps): update actions/checkout action to v4
chore(deps): update actions/setup-go action to v5
chore(deps): update github/codeql-action action to v3
chore(deps): update golangci/golangci-lint-action action to v6
chore(deps): update goreleaser/goreleaser-action action to v6

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

chore(deps): update dependency go to v1.22.5
fix(deps): update module github.com/spf13/cobra to v1.8.1
fix(deps): update module cloud.google.com/go/storage to v1.43.0
fix(deps): update module golang.org/x/oauth2 to v0.21.0
fix(deps): update module google.golang.org/api to v0.189.0
fix(deps): update module helm.sh/helm/v3 to v3.15.3
Click on this checkbox to rebase all open PRs at once

Detected dependencies

github-actions

.github/workflows/release.yml

actions/checkout v3

github/codeql-action v2

actions/setup-go v3

goreleaser/goreleaser-action v4

.github/workflows/test.yml

actions/checkout v3

github/codeql-action v2

actions/setup-go v3

golangci/golangci-lint-action v3

gomod

go.mod

go 1.21

go 1.22.4

cloud.google.com/go/storage v1.30.1

github.com/ghodss/yaml v1.0.0

github.com/pkg/errors v0.9.1

github.com/sirupsen/logrus v1.9.3

github.com/spf13/cobra v1.8.0

golang.org/x/oauth2 v0.10.0

google.golang.org/api v0.126.0

helm.sh/helm/v3 v3.14.2

Check this box to trigger a request for Renovate to run again on this repository

helm repo add command doesn't work on Windows

helm repo add command fails when I try to add one of my helm chart repo created in google bucket using my Windows 10 laptop.

Error: helm\plugins\helm-gcs\scripts\pull.sh: %1 is not a valid Win32 application.

I know this can be over come by using WSL or cygwin but we are going to deploy a solution for users where they might not have any of the WSL or cygwin installed on their systems.

helm gcs init intermittently fails with permission errors

About every third time that we call helm gcs init as part of our deploy pipeline, we receive the error message:

Error: googleapi: got HTTP response code 403 with body: <?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>gke-cluster@[MASKED].iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.</Details></Error>
Usage:
  helm-gcs init gs://bucket/path [flags]
Flags:
  -h, --help   help for init
Global Flags:
      --debug                    activate debug
      --service-account string   service account to use for GCS
googleapi: got HTTP response code 403 with body: <?xml version='1.0' encoding='UTF-8'?><Error><Code>AccessDenied</Code><Message>Access denied.</Message><Details>gke-cluster@[MASKED].iam.gserviceaccount.com does not have storage.objects.get access to the Google Cloud Storage object.</Details></Error>
Error: plugin "gcs" exited with error

We've made sure that the service account has the Editor and Owner IAM privileges, and we've made sure that the GCS bucket permissions allow Editors and Owners to be Storage Admins.

Is it necessary for the Go language code behind helm-gcs init to have some kind of retry mechanism?

Can't use --service-account flag with helm repo add

I can't figure out how to use the service-account flag with helm repo add. Is this possible?

If it's not possible, it makes sense why it would be difficult, but it should be more clearly documented in the README.

Publish with optimistic locking

As user,
I want to see optimistic locking error when I try to push when my repository index is not up-to-date
So that I do not overwrite changes in repo index made by someone else at the same time.

As user,
I want to optionally to reload index and retry push when optimistic locking error occurs
So that I can push in simultaneously run automated builds.

Design concept proposal:

when loading index from repo, get GCS $revision on index.yaml
when pushing, upload index first with conditional PUT: add header x-goog-if-generation-match:$revision
on error, to optionally retry, reload fresh index and run push again

See example: https://github.com/ausov/helm-gs/blob/3d994597fbddae7029ee0e3afad6d45796615761/bin/publish.sh#L72

can't install latest version

I'm trying to install the latest stable version (currently 0.3.1) but when I run this command

helm plugin install https://github.com/hayorov/helm-gcs

it seems to install version 0.3.0. This is the output I get

Installing helm-gcs 0.3.0 ...
x LICENSE
x README.md
x helm-gcs
helm-gcs 0.3.0 is correctly installed.

Init a new repository:
  helm gcs init gs://bucket/path

Add your repository to Helm:
  helm repo add repo-name gs://bucket/path

Push a chart to your repository:
  helm gcs push chart.tar.gz repo-name

Update Helm cache:
  helm repo update

Get your chart:
  helm fetch repo-name/chart

Installed plugin: gcs

The same thing happens if I try to install it specifying a version

helm plugin install https://github.com/hayorov/helm-gcs --version 0.3.1

Install fails on windows 10

The install script seems to identify windows by uname = MINGW. As of Win10 this has changed to MSYS_NT-10.0. This leads to the following output.

helm plugin install https://github.com/nouney/helm-gcs --debug

[debug] updating https://github.com/nouney/helm-gcs
[debug] symlinking C:\Users\Klaboe\.helm\cache\plugins\https-github.com-nouney-helm-gcs to C:\Users\Klaboe\.helm\plugins\helm-gcs
[debug] loading plugin from C:\Users\Klaboe\.helm\plugins\helm-gcs
[debug] running install hook: &{C:\Program Files\Git\usr\bin\sh.exe [sh -c cd $HELM_PLUGIN_DIR; ./scripts/install.sh] [] <nil> <nil> <nil> [] %!s(*syscall.SysProcAttr=<nil>) %!s(*os.Process=<nil>) <nil> <nil> <nil> %!s(bool=false) [] [] [] [] %!s(chan error=<nil>) %!s(chan struct {}=<nil>)} Installing helm-gcs 0.2.0 ...
tar.exe: Error opening archive: Failed to open 'helm-gcs_0.2.0_UNKNOWN:MSYS_NT-10.0_x86_64.tar.gz' helm-gcs 0.2.0 is correctly installed.

(...)

Installed plugin: gcs`

So it says that it is correctly installed, even though the un-tar'ing has failed for helm-gcs_0.2.0_UNKNOWN:MSYS_NT-10.0_x86_64.tar.gz.

So i propose 2 fixed:

include MSYS_NT-10.0 as a windows-identifyer, and
abort install if OS is unknown, as it seems unnecessary to try to un-tar an invalid file

Unable to fetch chart pushed after helm package

How to reproduce:

Install the plugin, init the repo and add it to your local Helm installation.
helm create foo.
helm package foo (this will create foo-0.1.0.tgz file).
helm gcs push foo-0.1.0.tgz <your-repo>.
helm repo update.
helm fetch <your-repo>/foo.

After that I get the following error:

2017/11/02 22:24:13 storage: object doesn't exist
panic: storage: object doesn't exist

goroutine 1 [running]:
log.Panic(0xc42016fcf8, 0x1, 0x1)
	/usr/lib/go/src/log/log.go:330 +0xc0
github.com/nouney/helm-gcs/helm-gcs/cmd.glob..func2(0x192f220, 0xc420045760, 0x1, 0x1, 0x0, 0x0)
	/home/nouney/work/go/src/github.com/nouney/helm-gcs/helm-gcs/cmd/pull.go:19 +0xae
github.com/nouney/helm-gcs/helm-gcs/vendor/github.com/spf13/cobra.(*Command).execute(0x192f220, 0xc420045730, 0x1, 0x1, 0x192f220, 0xc420045730)
	/home/nouney/work/go/src/github.com/nouney/helm-gcs/helm-gcs/vendor/github.com/spf13/cobra/command.go:698 +0x47a
github.com/nouney/helm-gcs/helm-gcs/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x192f880, 0xc42016ff30, 0xc42016ff38, 0x14ac23e)
	/home/nouney/work/go/src/github.com/nouney/helm-gcs/helm-gcs/vendor/github.com/spf13/cobra/command.go:783 +0x30e
github.com/nouney/helm-gcs/helm-gcs/vendor/github.com/spf13/cobra.(*Command).Execute(0x192f880, 0x0, 0x192faa0)
	/home/nouney/work/go/src/github.com/nouney/helm-gcs/helm-gcs/vendor/github.com/spf13/cobra/command.go:736 +0x2b
github.com/nouney/helm-gcs/helm-gcs/cmd.Execute()
	/home/nouney/work/go/src/github.com/nouney/helm-gcs/helm-gcs/cmd/root.go:17 +0x31
main.main()
	/home/nouney/work/go/src/github.com/nouney/helm-gcs/helm-gcs/main.go:6 +0x20

Expected behavoir: chart foo-0.1.0.tgz being fetched.

helm gcs fails to install on an armhf or arm64 system

I see this with a raspberry PI 3/4.

`Installing helm-gcs 0.3.18 ...
tar: This does not look like a tar archive

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
helm-gcs 0.3.18 is correctly installed.

Init a new repository:
helm gcs init gs://bucket/path

Add your repository to Helm:
helm repo add repo-name gs://bucket/path

Push a chart to your repository:
helm gcs push chart.tar.gz repo-name

Update Helm cache:
helm repo update

Get your chart:
helm fetch repo-name/chart

Installed plugin: gcs`

The plugin isn't installed, and when I try to invoke I get:

Error: fork/exec /home/user/.local/share/helm/plugins/helm-gcs.git/bin/helm-gcs: no such file or directory
I can workaround it by downloading the release manually and copying the binary to the above folder.

helm gcs push error: couldn't load repositories file (/Users/<username>/Library/Preferences/helm/repositories.yaml)

helm gcs push can-server-charts-repo/game-0.1.0.tgz can-server-charts Error: repo entry: load repo file: couldn't load repositories file (/Users/<username>/Library/Preferences/helm/repositories.yaml): open /Users/<username>/Library/Preferences/helm/repositories.yaml: no such file or directory

helm gcs version version: 0.3.0 commit: dac6f33092435ff5b96ebf75a3110af1129cd512 date: 2019-12-24T13:45:55Z

I'm unable to push helm packages to my helm repo. I read in a previous issue that this is due to the plugin supporting helm 2 instead of helm 3, but there was an update to fix the issue. This issue seems unresolved as I am on helm-gcs v 0.3.0 and I still have the problem.

Version 0.3.5 fails to generate auth token

In a GKE 1.14 cluster with with Workload Identity, 0.3.5 plugin fails to generate token for the serviceaccount:

Error: Get "https://storage.googleapis.com/xyz_helm3/index.yaml": compute: Received 500 Unable to generate token

Downgrade to 0.3.4 solves the issue.

Helm charts not listed after pushing new chart

I followed these instructions to setup a private repository: https://www.arthurkoziel.com/private-helm-repo-with-gcs-and-github-actions/

I have successfully published a chart to my bucket and can verify the index.yaml file is updated and my the chart tarball is uploaded. However, adding the helm repo and attempting to list my chart fails, showing I have no charts in my repo. I have access to the bucket, verified via gsutil. Any ideas how to troubleshoot why no chart is being pulled?

I'm using Helm 3.3.4 and Helm-gcs 0.3.18

╭─ ~/src ──────────────────────────────────────────────────────────────
╰─ helm repo list
NAME          	URL
kedacore      	https://kedacore.github.io/charts
astronomer    	https://helm.astronomer.io
rimusz        	https://charts.rimusz.net
ory           	https://k8s.ory.sh/helm/charts
airflow-stable	https://airflow-helm.github.io/charts
datadog       	https://helm.datadoghq.com
jenkins       	https://charts.jenkins.io
nginx-stable  	https://helm.nginx.com/stable
jetstack      	https://charts.jetstack.io
ko-helm-charts	gs://my-bucket-helmcharts/charts
╭─ ~/src ───────────────────────────────────────────────────────────────
╰─ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "rimusz" chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "kedacore" chart repository
...Successfully got an update from the "airflow-stable" chart repository
...Successfully got an update from the "ory" chart repository
...Successfully got an update from the "nginx-stable" chart repository
...Successfully got an update from the "datadog" chart repository
...Successfully got an update from the "astronomer" chart repository
...Successfully got an update from the "jenkins" chart repository
...Successfully got an update from the "ko-helm-charts" chart repository
Update Complete. ⎈Happy Helming!⎈
╭─ ~/src ──────────────────────────────────────────────────────────────────
╰─ helm search repo ko-helm-charts
No results found
╭─ ~/src ───────────────────────────────────────────────────────────────────
╰─ gsutil ls gs://my-bucket-helmcharts
gs://my-bucket-helmcharts/charts/
╭─ ~/src ───────────────────────────────────────────────────────────────────
╰─ gsutil ls gs://my-bucket-helmcharts/charts
gs://my-bucket-helmcharts/charts/my_app-0.0.1-rc1.tgz
gs://my-bucket-helmcharts/charts/index.yaml
╭─ ~/src ────────────────────────────────────────────────────────────────────
╰─ gsutil cp gs://my-bucket-helmcharts/charts/index.yaml .
Copying gs://my-bucket-helmcharts/charts/index.yaml...
/ [1 files][  417.0 B/  417.0 B]
Operation completed over 1 objects/417.0 B.
╭─ ~/src ────────────────────────────────────────────────────────────────────
╰─ cat index.yaml
apiVersion: v1
entries:
  my_app:
  - apiVersion: v2
    appVersion: 0.1.0
    created: "2021-12-01T20:22:42.71819179Z"
    description: A Helm chart for Kubernetes
    digest: c69d3c8e1d2d538ffd7daw4f855d3d0d46bc0d3059fe780bb4a36c261393097111
    name: my_app
    type: application
    urls:
    - gs://my-bucket-helmcharts/charts/my_app-0.0.1-rc1.tgz
    version: 0.0.1-rc1
generated: "2021-12-01T20:22:42.71819409Z"

requesting an updated release

Is it possible to get a release that includes #23 ? i have a use case for this feature but the current master version installs 0.2.0 and 0.2.1 does not include this change.

Error: unknown flag: --service-account

Trying to run
helm repo update --service-account <service-account.json>

got Error: unknown flag: --service-account

from help:

Usage:
  helm-gcs pull gs://bucket/path [flags]

Flags:
  -h, --help   help for pull

Global Flags:
      --debug                    activate debug
      --service-account string   service account to use for GCS

`helm repo up` not working with helm-gcs

Example:

→ helm gcs push staging-model-template-0-1-0.tgz bi-staging --force

→ helm repo up
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "bi-staging" chart repository
Update Complete. ⎈Happy Helming!⎈

→ helm fetch bi-staging/staging-model-template-0-1-0
Error: chart "staging-model-template-0-1-0" matching  not found in bi-staging index. (try 'helm repo update'): no chart name found

→ helm gcs push staging-model-template-0-1-0.tgz bi-staging
Error: chart staging-model-template-0-1-0 already indexed. Use --force to still upload the chart

→ helm search repo  bi-staging
No results found

See - it is indexed when you try to push, but it is not when you try to fetch/update/search.

When viewing the GCS bucket, the .tgz is there, along with an index.yaml as follows:

Note: Just for privacy on a public post, I removed the digest variable and changed the bucket name to gs://HIDDEN. The information is correct on the real bucket

apiVersion: v1
entries:
  staging-model-template:
  - apiVersion: v2
    appVersion: 0-1-0
    created: "2020-09-29T12:14:32.763916-05:00"
    description: Model description goes here
    icon: https://i.imgur.com/lc9OKZr.png
    name: staging-model-template
    type: application
    urls:
    - gs://HIDDEN/staging-model-template-0-1-0.tgz
    version: 0-1-0
generated: "2020-09-29T12:14:32.763918-05:00"

Is this a caching issue related to #44 ? What is happening to cause this? How would it be resolved?

Can't pull / install a chart after its been pushed to a bucket

Hello.
I have used the README to successfully push a chart into a bucket.
But when I get to the helm fetch repo-name/chart, I get multiple different errors instead of helm retrieving the chart.
I've tried the command as above:
Error: chart "chart" matching not found in repo-name index. (try 'helm repo update'): no chart name found
I've tried the command like this (without /chart): helm fetch repo-name
Error: non-absolute URLs should be in form of repo_name/path_to_chart, got repo_name
I'm not sure what to put for path_to_chart.

"gs://BUCKET-NAME" is not a valid chart repository or cannot be reached for Helm < 3.8.0

The following issue appears starting version 0.3.19

/home/runner/.local/share/helm/plugins/helm-gcs.git/scripts/pull.sh: 3: /home/runner/.local/share/helm/plugins/helm-gcs.git/bin/helm-gcs: not found
Error: looks like "gs://BUCKET-NAME" is not a valid chart repository or cannot be reached: plugin "scripts/pull.sh" exited with error

Helm version: 3.7.2,
might be related to this change: a837e6b

Checking now if works with Helm 3.8.0

exit code of tar not handled

The following error occurred in a CI Pipeline so I cannot reproduce it, but I it happen while GitHub had issues with connections timing out.

Below some printout:

helm plugin install https://github.com/hayorov/helm-gcs
Installing helm-gcs 0.3.1 ...
tar: invalid magic
tar: short read
helm-gcs 0.3.1 is correctly installed.
...
/root/.local/share/helm/plugins/helm-gcs/scripts/pull.sh: line 3: /root/.local/share/helm/plugins/helm-gcs/bin/helm-gcs: not found
Error: looks like "gs://xxxxxxxxxxx/" is not a valid chart repository or cannot be reached: plugin "scripts/pull.sh" exited with error

The interesting part is tar: invalid magic. It looks like it coming from the following line

https://github.com/hayorov/helm-gcs/blob/master/scripts/install.sh#L43

Referenz: helm/helm#7973

Query: how to manage multiple repositories that exist in different GCP projects

Hi,
We have setup chart repositories (buckets) in two different accounts. I am setting up the environment variable, GOOGLE_APPLICATION_CREDENTIALS to the corresponding account to push/fetch the helm charts.
But the same thing will not work with "helm repo update". Is there a way that I can supply the service account (in fact multiple service accounts) to "helm repo update" through this plugin?
Or any other suggestion?

Thanks,
Sarma K

Whether supports helm3 dependency function in Chart.yaml

As the titile.
After I push the chart package to GCS, I find the Chart.yaml of package miss the dependency for Helm 3.
Any comments?
Thanks a lot!

gcs init fails

I am unable to get helm gcs init to work. Irrespective of which auth method i use, it invariably fails with--

ERROR: (gcloud.auth.application-default.login) Invalid file format. 
See https://developers.google.com/api-client-library/python/guide/aaa_client_secrets 
Expected a JSON object with a single property for a "web" or "installed" application

I have tried these (all env variables below are configured properly) :

With application-default

echo $GCLOUD_SERVICE_KEY > ${HOME}/account-auth-deploy.json
gcloud auth application-default login --client-id-file=${HOME}/account-auth-deploy.json
gcloud --quiet container clusters get-credentials $CLOUDSDK_CONTAINER_CLUSTER

With application default (alt)

gcloud auth application-default login --client-id-file=${HOME}/account-auth.json

With google app creds env var

export GOOGLE_APPLICATION_CREDENTIALS=${HOME}/account-auth-deploy.json

Neither of these seem to have any effect.

Using: helm 2.13.1, gcs plugin latest.

helm install error: split path: incorrect url

helm install can-server-charts/game --dry-run Error: split path: incorrect url, should be "gs://bucket/path"

helm gcs version version: 0.3.0 commit: dac6f33092435ff5b96ebf75a3110af1129cd512 date: 2019-12-24T13:45:55Z

After uploading a package and index.yaml to a gs bucket, and adding and updating the repo with helm, I am unable to install any chart from the repo.

Helm install only works if I use a direct path to the tar file; eg helm install gs://<bucket_name>/game-0.1.0.tgz.

Wrong version installed when use `--version 0.3.0`

General description

Wrong version of helm-gcs (0.2.2) is installed when --version 0.3.0 is provided.

Steps to reproduce

clean cache
rm -rf ~/.cache/helm/*
install plugin with --version added
helm plugin install https://github.com/hayorov/helm-gcs --version 0.3.0 --debug

Proof

# helm plugin install https://github.com/hayorov/helm-gcs --version 0.3.0 --debug
[debug] cloning https://github.com/hayorov/helm-gcs to /root/.cache/helm/plugins/https-github.com-hayorov-helm-gcs
[debug] setting version to "0.3.0"
[debug] symlinking /root/.cache/helm/plugins/https-github.com-hayorov-helm-gcs to /root/.local/share/helm/plugins/helm-gcs
plugin_install.go:73: [debug] loading plugin from /root/.local/share/helm/plugins/helm-gcs
plugin.go:60: [debug] running install hook: /bin/sh -c cd $HELM_PLUGIN_DIR; ./scripts/install.sh
Installing helm-gcs 0.2.2 ...
helm-gcs 0.2.2 is correctly installed.

Solution (workaround)

The only way to install 0.3.0 version is not using --version and use master branch as an installation source. This solution will not work when the version in the master branch will change.

# helm plugin install https://github.com/hayorov/helm-gcs --debug
[debug] cloning https://github.com/hayorov/helm-gcs to /root/.cache/helm/plugins/https-github.com-hayorov-helm-gcs
[debug] symlinking /root/.cache/helm/plugins/https-github.com-hayorov-helm-gcs to /root/.local/share/helm/plugins/helm-gcs
plugin_install.go:73: [debug] loading plugin from /root/.local/share/helm/plugins/helm-gcs
plugin.go:60: [debug] running install hook: /bin/sh -c cd $HELM_PLUGIN_DIR; ./scripts/install.sh
Installing helm-gcs 0.3.0 ...
helm-gcs 0.3.0 is correctly installed.

I can't push local chart to gcs repo

I build a local chart

 helm package -u . --version 0.2.0-alpah1-PR-6+0013
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
        Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused

...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "akeneo-charts-dev" chart repository
...Successfully got an update from the "akeneo-charts" chart repository
...Successfully got an update from the "coreos" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 3 charts
Downloading mysql from repo gs://akeneo-charts/
Downloading elasticsearch from repo gs://akeneo-charts/
Downloading nfs-server-provisioner from repo https://kubernetes-charts.storage.googleapis.com
Deleting outdated charts
Successfully packaged chart and saved it to: .../pim-0.2.0-alpah1-PR-6+0013.tgz

And want to push it to gcs

helm gcs push pim-0.2.0-alpah1-PR-6+0013.tgz akeneo-charts-dev
Error: chart pim-0.2.0-alpah1-PR-6+0013 already indexed. Use --force to still upload the chart
Usage:
  helm-gcs push [chart.tar.gz] [repository] [flags]

Flags:
      --force   upload the chart even if already indexed
  -h, --help    help for push
      --retry   retry if the index changed

Global Flags:
      --debug                    activate debug
      --service-account string   service account to use for GCS

chart pim-0.2.0-alpah1-PR-6+0013 already indexed. Use --force to still upload the chart
Error: plugin "gcs" exited with error

If i check with search, i have a local chart but no chart on gcs

helm search pim -l --version 0.2.0-alpah1-PR-6+0013
NAME                    CHART VERSION           APP VERSION     DESCRIPTION
akeneo-charts-dev/pim   0.2.0-alpah1-PR-6+0011                  Akeneo PIM package
local/pim               0.2.0-alpah1-PR-6+0013                  Akeneo PIM package

helm version
Client: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}

Is there a way to use a developer's existing gcp credentials?

Currently it seems to require the service-account or the environment variable being set to a credentialed file, which are good for the CICD setups.
But for our own developers, is there a way to just tell it to use our existing developer logins instead of needing to create the credentials file?

Thanks!

helm-gcs is incompatible with Landscaper

... or potentially anything that uses this Helm function - https://github.com/kubernetes/helm/blob/master/pkg/downloader/chart_downloader.go#L132

Please see this landscaper issue - Eneco/landscaper#117

Error: exec: "sh": executable file not found in %PATH%

Hi there,

When I try to run on windows 10 command: helm plugin install https://github.com/hayorov/helm-gcs.git
I get an error: Error: "exec: "sh": executable file not found in %PATH%"
Helm version: version.BuildInfo{Version:"v3.3.0", GitCommit:"8a4aeec08d67a7b84472007529e8097ec3742105", GitTreeState:"dirty", GoVersion:"go1.14.7"}

Add option to add http(s) URLs to index.yaml

I would like to use helm-gcs to publish my Helm charts, but let clients use http(s) to download the Helm charts.

Would it be possible to get a --url option to the helm gcs push command, which allows me to either add additional URLs (e.g. a HTTP or HTTPS URL for my GCS bucket), or override the default URL?

Thanks!

helm gcs rm not functioning properly

VERSION
helm-gcs = Version 0.3.0

COMMAND
helm gcs rm gcp-helm-project repo

ISSUE
RemoveChart Appears to remove the chart from the index.yaml, but the tgz remains in the bucket. I see the following error.

helm gcs remove gcp-helm-project repo --debug
Error: delete: storage: object doesn't exist
Usage:
  helm-gcs rm [chart] [repository] [flags]

Aliases:
  rm, remove

Flags:
  -h, --help             help for rm
      --retry            retry if the index changed
  -v, --version string   version of the chart to remove

Global Flags:
      --debug                    activate debug
      --service-account string   service account to use for GCS

delete: storage: object doesn't exist
Error: plugin "gcs" exited with error
helm.go:84: [debug] plugin "gcs" exited with error

index.yaml

apiVersion: v1
entries:
  gcp-helm-project:
  - apiVersion: v2
    appVersion: 0.1.0
    created: "2020-05-20T10:47:06.307038-04:00"
    description: A Helm chart.
    digest: 234254356fdtgdfgsdg334534534535
    name: gcp-helm-project
    type: application
    urls:
    - gs://[BUCKET]/platform/gcp-helm-project-0.1.0.tgz
    version: 0.1.0
generated: "2020-05-20T10:47:06.307042-04:00"

helm-gcs fails to install when curl is not present but the install script succeeds

Helm version: 2.13.0

Installing helm-gcs 0.2.0 ...
./scripts/install.sh: line 32: curl: not found
tar: can't open 'helm-gcs_0.2.0_Linux_x86_64.tar.gz': No such file or directory
helm-gcs 0.2.0 is correctly installed.

No curl in the container I am using so the script fails, however it seems to exit OK and report being installed correctly.

It would be great if the install script checked for curl and wget. I do have wget in the container.

helmfile with helm-gcs plugin gives error with"apply"

Failed to pull the images from the GCS repositories when using helmfile apply to deploy the helm charts:

Sample command that was generated:

exec: helm upgrade --install --reset-values sample-service-pre-qa sample-daily/sample-service --version ~1.2.84 --verify --wait --timeout 180 --atomic --namespace pre-qa --values /tmp/values205449575 --values /tmp/values081688986 --values /tmp/values212236337 --values /tmp/values592251868 --set replicas=1

err: release "sample-service-pre-qa" in "helmfile.yaml" failed: failed processing release sample-service-pre-qa: helm exited with status 1:
Error: storage: object doesn't exist
Usage:
helm-gcs pull gs://bucket/path [flags]

Flags:
-h, --help help for pull

Global Flags:
--debug activate debug
--service-account string service account to use for GCS

It works fine when we do a helm fetch.

Here are the versions used:

helm-gcs plugin version:

gcs 0.2.2 Manage repositories on Google Cloud Storage

helm version:

Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}

helmfile version:

helmfile version v0.90.5

Installation with option --version 0.2.0 fails

bash-4.3# helm plugin install https://github.com/nouney/helm-gcs --version 0.2.0 --debug
[debug] updating https://github.com/nouney/helm-gcs
[debug] setting version to "0.2.0"
[debug] symlinking /root/.helm/cache/plugins/https-github.com-nouney-helm-gcs to /root/.helm/plugins/helm-gcs
[debug] loading plugin from /root/.helm/plugins/helm-gcs
[debug] running install hook: &{/bin/sh [sh -c cd $HELM_PLUGIN_DIR; ./install.sh] []  <nil> <nil> <nil> [] %!s(*syscall.SysProcAttr=<nil>) %!s(*os.Process=<nil>) <nil> <nil> <nil> %!s(bool=false) [] [] [] [] %!s(chan error=<nil>) %!s(chan struct {}=<nil>)}
sh: ./install.sh: not found
Error: plugin install hook for "gcs" exited with error

The root cause is that plugin.yaml tagged with 0.2.0 contains invalid version and hooks:

version: "0.1.4"
...
hooks:
  install: "cd $HELM_PLUGIN_DIR; ./install.sh"
  update: "cd $HELM_PLUGIN_DIR; ./install.sh"