codacy / helm-ssm Goto Github PK

Injects values from AWS SSM parameters on the values.yaml file

License: MIT License

Makefile 9.47% Go 73.12% Shell 17.41%

hacktoberfest codacy injects-values helm-ssm helm-plugin helm kubernetes helm3 aws ssm

helm-ssm's Introduction

Helm SSM Plugin

This is a helm3 plugin to help developers inject values coming from AWS SSM parameters, on the values.yaml file. It also leverages the wonderful sprig package, thus making all its functions available when parsing.

Since helm2 is deprecated the current version of the plugin only supports helm3. The last version to support helm2 is v2.2.1. There will be no further patches or updates to this legacy version.

Usage

Loads a template file, and writes the output.

Simply add placeholders like {{ssm "path" "option1=value1" }} in your file, where you want it to be replaced by the plugin.

Currently the plugin supports the following options:

region=eu-west-1 - to resolve that parameter in a specific region
default=some-value - to give a default string value when the ssm parameter is optional. The plugin will throw an error when values are not defined and do not have a default.
prefix=/something - you can use this to specify a given prefix for a parameter without affecting the path. It will be concatenated with the path before resolving.

Values file

service:
ingress:
  enabled: false
  hosts:
    - service.{{ssm "/exists/subdomain" }}
    - service1.{{ssm "/empty/subdomain" "default=codacy.org" }}
    - service2.{{ssm "/exists/subdomain" "default=codacy.org" "region=eu-west-1" }}
    - service3.{{ssm "/subdomain" "default=codacy.org" "region=eu-west-1" "prefix=/empty" }}
    - service4.{{ssm "/securestring" }}

when you do not want a key to be defined, you can use a condition and an empty default value:

service:
ingress:
  enabled: false
  hosts:
    {{- with $subdomain := (ssm "/exists/subdomain" "default=") }}{{ if $subdomain }}
    - service.{{$subdomain}}
    {{- end }}{{- end }}

Command

$ helm ssm [flags]

Flags

  -c, --clean                   clean all template commands from file
  -d, --dry-run                 doesn't replace the file content
  -h, --help                    help for ssm
  -p, --profile string          aws profile to fetch the ssm parameters
  -t, --tag-cleaned string      replace cleaned template commands with given string
  -o, --target-dir string       dir to output content
  -f, --values valueFilesList   specify values in a YAML file (can specify multiple) (default [])
  -v, --verbose                 show the computed YAML values file/s

Example

Install

Choose the latest version from the releases and install the appropriate version for your OS as indicated below.

$ helm plugin add https://github.com/codacy/helm-ssm

Developer (From Source) Install

If you would like to handle the build yourself, instead of fetching a binary, this is how we recommend doing it.

Make sure you have Go installed.
Clone this project
In the project directory run

$ make install

What is Codacy

Codacy is an Automated Code Review Tool that monitors your technical debt, helps you improve your code quality, teaches best practices to your developers, and helps you save time in Code Reviews.

Among Codacy’s features

Identify new Static Analysis issues
Commit and Pull Request Analysis with GitHub, BitBucket/Stash, GitLab (and also direct git repositories)
Auto-comments on Commits and Pull Requests
Integrations with Slack, HipChat, Jira, YouTrack
Track issues in Code Style, Security, Error Proneness, Performance, Unused Code and other categories

Codacy also helps keep track of Code Coverage, Code Duplication, and Code Complexity.

Codacy supports PHP, Python, Ruby, Java, JavaScript, and Scala, among others.

Free for Open Source

Codacy is free for Open Source projects.

License

helm-ssm is available under the MIT license. See the LICENSE file for more info.

helm-ssm's People

Contributors

Stargazers

Watchers

helm-ssm's Issues

[CY-2991] Set content-type when uploading to Github Assets

Hi,

Newer versions of helm detects the mime type in the content-type header when installing plugins
https://github.com/helm/helm/blob/54ea21d24533d5fc10bc07fe1d4695d8383833a8/pkg/plugin/installer/http_installer.go#L65

Using the instructions to install the latest release:

$ helm plugin install https://github.com/codacy/helm-ssm/releases/download/latest/helm-ssm-linux.tgz

unfortunately, the mime-type is set to application/octet-stream which is not supported in the list. Thus the installer will assume it is a repository installation and results in an error

➜  helm plugin install https://github.com/codacy/helm-ssm/releases/download/latest/helm-ssm-linux.tgz
Error: Unable to get repository: Cloning into '/home/user/.cache/helm/plugins/https-github.com-codacy-helm-ssm-releases-download-3.0.1-helm-ssm-linux.tgz'...
remote: Not Found
fatal: repository 'https://github.com/codacy/helm-ssm/releases/download/3.0.1/helm-ssm-linux.tgz/' not found
: exit status 128

ghr does not seem to pass in the content type properly as well:
https://github.com/tcnksm/ghr/blob/master/github.go#L191

Feature Request: Decrypt SecureString parameters

Please add the ability to decrypt SecureString parameters from SSM. This is one of the key features of SSM and would be very useful.

[TS-166] Plugin installation clones repo instead of installing

When I successfully install this plugin, invoking the command to use it results in an error. It's trying to invoke the helm-ssm file inside the helm-ssm directory, but that file doesn't exist. Instead, the helm plugin add command seems to have cloned the helm-ssm repo into the plugins directory.

~ # helm plugin add https://github.com/codacy/helm-ssm
Installed plugin: ssm
~ # helm ssm
Error: fork/exec /root/.local/share/helm/plugins/helm-ssm/helm-ssm: no such file or directory
~ # cd /root/.local/share/helm/plugins
~/.local/share/helm/plugins # ls -al
total 16
drwxr-xr-x 4 root root 4096 Jan 10 21:21 .
drwxr-xr-x 3 root root 4096 Jan 10 04:47 ..
drwxr-xr-x 7 root root 4096 Jan 10 21:21 helm-ssm
~/.local/share/helm/plugins # cd helm-ssm
~/.local/share/helm/plugins/helm-ssm # ls -al
total 88
drwxr-xr-x 7 root root  4096 Jan 10 21:21 .
drwxr-xr-x 4 root root  4096 Jan 10 21:21 ..
drwxr-xr-x 2 root root  4096 Jan 10 21:21 .circleci
drwxr-xr-x 8 root root  4096 Jan 10 21:21 .git
drwxr-xr-x 3 root root  4096 Jan 10 21:21 .github
-rw-r--r-- 1 root root    47 Jan 10 21:21 .gitignore
-rw-r--r-- 1 root root   760 Jan 10 21:21 CHANGELOG.md
-rw-r--r-- 1 root root  1063 Jan 10 21:21 LICENSE
-rw-r--r-- 1 root root  1817 Jan 10 21:21 Makefile
-rw-r--r-- 1 root root  4404 Jan 10 21:21 README.md
drwxr-xr-x 2 root root  4096 Jan 10 21:21 cmd
-rw-r--r-- 1 root root   879 Jan 10 21:21 go.mod
-rw-r--r-- 1 root root 20971 Jan 10 21:21 go.sum
-rwxr-xr-x 1 root root  3340 Jan 10 21:21 install-binary.sh
drwxr-xr-x 2 root root  4096 Jan 10 21:21 internal
-rw-r--r-- 1 root root   225 Jan 10 21:21 plugin.yaml

As you can see, there's no additional helm-ssm file inside the /plugins/helm-ssm directory. But the helm ssm command is trying to find it.

This is using helm version:

~/.local/share/helm/plugins/helm-ssm # helm version
version.BuildInfo{Version:"v3.10.2", GitCommit:"", GitTreeState:"", GoVersion:"go1.19.3"}

[CY-5244] installing specific version is ignored by install-binary.sh

while trying to use helm plugin install with "--version" option, install-binary.sh always ignore it and install the latest version

getDownloadURL() {
  # Use the GitHub API to find the latest version for this project.
  local latest_url="https://api.github.com/repos/$PROJECT_GH/releases/latest

[CY-5182] Need of Indent(x)

Hi guys! I'm not a GO expert otherwise I'd just do a PR. I'm facing an issue while trying to inject a private key.
Since it's multiline, the yalm breaks. Could we add something like:

{{ssm 'some/parameter' | indent(20) }}

So you see what I mean, on the values.yaml I have:

privateKeySource:
  directEntry:
    privateKey: |
{{ssm "/secrets/ssh/private_key"}}

and it outputs:

privateKeySource:
   directEntry:
     privateKey: |
-----BEGIN RSA PRIVATE KEY-----
MIIJKAIBAAKCAgEA2Bh2okyf3wPfWNis2h5mcg1wOM+1sqNerncA1z3ES8E4hm4j
Line3 .. etc

I would like it to output:

privateKeySource:
   directEntry:
     privateKey: |
       -----BEGIN RSA PRIVATE KEY-----
       MIIJKAIBAAKCAgEA2Bh2okyf3wPfWNis2h5mcg1wOM+1sqNerncA1z3ES8E4hm4j
       ....

If you indent the parameter call it will only indent the first line, so to be able to send certificates we need the indent(x) feature.

[CY-5330] No prebuild binary for macos-arm64

Please add binary for Apple M1

[CY-5073] Is there any way of excluding non {{ssm variables?

Hi,

I'm trying to deploy a chart which contains more variables passed as config.
I get the error:

failed exit status 1: Error: template: ssmtpl:78: function "include" not defined

Is there any way in helm-ssm to force the parsing only for "{{ssm ....}}"?

>>Here<< is an example.

hub:
  image:
    registry: docker.io
  configuration: |
    Chart:
      Name: {{ .Chart.Name }}
      Version: {{ .Chart.Version }}
    Release:
      Name: {{ .Release.Name }}
      Namespace: {{ .Release.Namespace }}
      Service: {{ .Release.Service }}
    hub:
      config:
        JupyterHub:
          ...................
          Authenticator:
            admin_users:
              - {{ .Values.hub.adminUser }}
      ....................
      db:
        type: postgres
        url: postgresql://{{ ternary .Values.postgresql.postgresqlUsername .Values.externalDatabase.user .Values.postgresql.enabled }}@{{ ternary (include "jupyterhub.postgresql.fullname" .) .Values.externalDatabase.host .Values.postgresql.enabled }}:{{ ternary "5432" .Values.externalDatabase.port .Values.postgresql.enabled }}/{{ ternary .Values.postgresql.postgresqlDatabase .Values.externalDatabase.database .Values.postgresql.enabled }}
      ........................
    singleuser:
      podNameTemplate: {{ include "common.names.fullname" . }}-jupyter-{username}
      {{- if .Values.singleuser.tolerations }}
      extraTolerations: {{- include "common.tplvalues.render" ( dict "value" .Values.singleuser.tolerations "context" $) | nindent 4 }}
      {{- end }}
      {{- if .Values.singleuser.nodeSelector }}
      nodeSelector: {{- include "common.tplvalues.render" ( dict "value" .Values.singleuser.nodeSelector "context" $) | nindent 4 }}
      {{- end }}

Many thanks

Specify AWS Profile

I have multiple AWS accounts used for my dev and prod environments, so

$ aws ssm get-parameter --name "/my/secret" --query Parameter.Value

gives me different output than

$ aws ssm get-parameter --profile prod --name "/my/secret" --query Parameter.Value

Is there a way to achieve a similar thing with this plugin?

Helm 3 support

Since a lot of things have been changed in the recently released version 3 of Helm, it seems likely that work will be needed for the SSM plugin to support it. Please add support for Helm 3.

Also, perhaps you can join forces with https://github.com/totango/helm-ssm while you implement an SSM plugin that is compatible Helm 3? After all, both your projects and theirs serve the same purpose.

Thanks for considering. 🙂

Use git-version

Use https://github.com/codacy/git-version to bump versions

Support output_dir option

It would be nice to have the possibility of adding to .gitignore the outputs of helm ssm..

Proposal:
Add the options -o | --output_dir

Example usage:

helm ssm -f values.yaml -o helm_values

[CY-5433] Supporting multiline parameters

I want to put a private SSH key into SSM. However I got an error:

unescaped newline inside substitute pattern

What I can do with?

use variable in ssm path

Hi , I'm wondering is there anyway to use variable or parameter in path section.
For example:
value.yaml

env: dev
key1: {{ ssm /{{ .env }}/exists/subdomain" }}

Which will search the path /dev/exists/subdomain for ssm path.
This way we could switch value with env easily without changing the value.yaml.

Thanks in advance.

[CY-2854] add --target-file option to allow output to a different file name

Problem:
-f option specifies the support of multiple files, thus the -o (target-dir) option makes sense. However, it is also desirable in certain use cases, where the user wants the input files to be processed and saved into a single file with a different file name.
This is essentially what -v | tee <file> does, however there are caveats with piping the output of helm. The most annoying one is that helm prints warnings of various checks into the same stdout/stderr and will pollute the ssm -v output.

Request:
add --target-file option to allow output the processed values into a single file.

[CY-6152] How does plugin parse ssm paramters

Hi,

I wanted to umderstand how the plugin fetch and replace the parameter's value with the place holder variable? Does it store the fetched values from ssm parameter store in memory before replacing the placeholder variable?

How to extract values(json format) from parameter store

We have data available in JSON on parameter store

Example: Parameter: /helm_data/vpc_details
Value:
{
vpc_id: 123456
vpc_cidr_range: 10.0.0.0/16
vpc_private_subnets: 10.0.1.0/24
vpc_public_subnets: 10.0.2.0/24
}
Now, I wanted this to be extracted by helm ssm plugin eg:-

vpc_cidr: {{ ssm /helm_data/vpc_details/vpc_cidr_range region=us-east-1}}

Release version 1.0.0

[CY-5242] [Bug]install-binary.sh returning 2 URLs for OS=linux breaking 'helm plugin install' on linux

When this line is executed with OS=linux, a string containing 2 URLs are returned and then the download curl fails due to malformed URL:

helm-ssm/install-binary.sh

Line 62 in 7f64ef2

 DOWNLOAD_URL=$(curl -s $latest_url | grep $OS | awk '/"browser_download_url":/{gsub( /[,"]/,"", $2); print $2}') 

Here is what is returned:
https://github.com/codacy/helm-ssm/releases/download/3.1.9/helm-ssm-linux-arm.tgz https://github.com/codacy/helm-ssm/releases/download/3.1.9/helm-ssm-linux.tgz

Due to this helm plugin install always fails if run on OS that return linux to this line:

helm-ssm/install-binary.sh

Line 32 in 7f64ef2

OS=$(echo $(uname)|tr '[:upper:]' '[:lower:]')

Seems have been result of arm addition causing 2 entries in releases to match grep for 'linux'

[CY-5496] cannot install current rel

cannot download and install:

helm version
version.BuildInfo{Version:"v3.7.2", GitCommit:"663a896f4a815053445eec4153677ddc24a0a361", GitTreeState:"clean", GoVersion:"go1.17.4"}

$ helm plugin install https://github.com/codacy/helm-ssm
Downloading https://github.com/codacy/helm-ssm/releases/download/3.1.9/helm-ssm-linux-arm.tgz
https://github.com/codacy/helm-ssm/releases/download/3.1.9/helm-ssm-linux.tgz
curl: (3) URL using bad/illegal format or missing URL
Failed to install helm-ssm
For support, go to https://github.com/codacy/helm-ssm.
Error: plugin install hook for "ssm" exited with error

$ helm ssm
Error: fork/exec /home/kdm/.local/share/helm/plugins/helm-ssm/helm-ssm: no such file or directory

$ helm plugin list
NAME   	VERSION	DESCRIPTION                                     
cm-push	0.10.1 	Push chart package to ChartMuseum               
ssm    	dev    	Inject AWS SSM parameters into Helm values files

[CY-2537] Generate and Upload security values to SSM

When retrieving the values from SSM it would be nice to be able to generate and upload security credentials.

This would allow more automated deployments in new environments, with less human intervention.
By storing the keys in SSM the app from then on works and the keys are stored securely.

Feature request: return JSON instead of strings

https://gist.github.com/krogebry/60a889fa631fe84724f65eb609773b4c

I'm trying to figure this out on my own, but golang makes me want to throw things. Basically I want to store a sort of "state" of a thing in SSM as a JSON payload. So, for example, the idea would be maybe I have something like this:

{
  "version": 0.1.0
  "env": {
    "p1": 100,
    "p2": 200
  }
}

I want to pull that "state" down from SSM and then roll it out to the values file I'm using. Basically storing the canonical state of a thing in SSM. Then, of course the next thing to do would be to make a similar plugin for SecretsManager using the same idea.

{ 
  "aws_access_key": "blah",
  "aws_secret_access_key": "yada"
}

Does that make sense? How hard is that to pull off?

[CY-2538] prefix from $PREFIX

how i can add prefix=$PREFIX ???
i want add support env
something like

helm ssm --env=prod

in amazon ssm i create

/dev/param
/prod/param
/staging/param

how i can do this?...

[CY-4473] AWS SSO not supported

When using AWS with SSO configured, helm ssm fails - not being able to read credentials:

template: ssmtpl:24:18: executing "ssmtpl" at <ssm "/myparam">: error calling ssm: NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
Error: plugin "ssm" exited with error

I believe updating to AWS SDK 1.37+ should help based on this article:
https://aws.amazon.com/blogs/developer/aws-sso-support-in-the-aws-sdk-for-go/

[CY-4778] Please add ARM binaries

Please add binaries for arm. I am unable to install it using helm plugin add https://github.com/codacy/helm-ssm command
I am able to compile this by changing
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 ... string by replacing value to GOARCH=arm64

[CY-5957] Pass options when using `ssm` as a filter

When ssm is used as a filter, it doesn't order its arguments correctly (namely {{ parameter | ssm options}} becomes {{ ssm options parameter }} instead of {{ ssm parameter options}}.

This fails /w error calling ssm: Invalid option: parameter. Valid options: [required prefix region].

Chaining filters is a nice templating feature that we've used for other things (quoting, manipulation) and it would be nice to be able to use it consistently.

Downloading https://github.com/codacy/helm-ssm/releases/download/3.1.9/helm-ssm-linux-arm.tgz
https://github.com/codacy/helm-ssm/releases/download/3.1.9/helm-ssm-linux.tgz
curl: (3) URL using bad/illegal format or missing URL
Failed to install helm-ssm
For support, go to https://github.com/codacy/helm-ssm.
Error: plugin install hook for "ssm" exited with error```