Hey folks, do you think it makes sense to have a golang playground with sprig functions already loaded ? I think it will be nice to get a better feel of the functions that one can use, and how they behave.

Thanks for writing this! Great library

Just a quick note that took me a few minutes to track down... documentation for has is wrong.

Should be: has 4 $myList

instead of

has $myList 4

https://github.com/Masterminds/sprig/blob/master/crypto.go#L35
need change from:
return fmt.Sprintf("%s", uuid.NewV4())
=>
return fmt.Sprintf("%s", uuid.Must(uuid.NewV4()))

randAlphaNum and randAscii functions can be very helpful to generate random strings. You can choose the length of the string and the kind of characters to use but you can't ensure the string will have all the kind of characters.

There are certain scenarios when you need to ensure that the generated string has at least one character of each kind. E.g. "I want a password that has at least one number and one letter".

I can assume that using a long password both will be generated but they might be not. E.g. "randAlphaNum 10" could generate "AUIjLlmiOk" with no numbers.

What do I expect?

• randAlphaNum to include both letters and numbers.
• randAscii to include both letters, numbers and symbols.

It seems that all these functions defined at springs.go are based on randomstringutils.go which do not accept any parameter to modify this behaviour.

Hi,
I am trying to use derivePassword in loop but unfortunately counter is defined uint32 so it is hard (I didn't find any way to do it from HELM) to use it.
Thanks
N.B. The algorithm of this function moved to here

Given:

{{- define "foobar" }}
foo:
  bar: baz
{{- end }}

qux:
{{- template "foobar" . | indent 2 }}

I'd like to get:

qux:
  foo:
    bar: baz

Given the above, I get:

executing "test" at <2>: wrong type for value; expected string; got map[string]interface {}

Using {{- template "foobar" . | toString | indent 2 }} I get:

qux:
foo:
  bar: baz

Meaning the line(s) never get indented. Even doing {{ template "foobar" . | join "_" | indent 2 }} results in the above, so clearly piping after template isn't working as I hoped it would.

Are there any ways to solve this?

We are using indent function in helm templates to generate yaml files. In lots of places we need to indent stuff, but that breaks template indentations (see below).

  resources:
{{ .Values.resources | indent 4 }}

Jinja indent filter allows skipping the first line. I think that would be useful so that we can write templates as the following snippet which allows keeping the natural indentation in template files:

  resources:
    {{ .Values.resources | indent 4 true }}

eg. parse a JSON string and return a complex datatype (eg. for looping)

Hello!

While reviewing recent charts PRs (example: helm/charts#2004), I see this interesting pattern:

{{- if .Values.persistence.storageClass }}
{{- if (eq "-" .Values.persistence.storageClass) }}
  storageClassName: ""
{{- else }}
  storageClassName: "{{ .Values.persistence.storageClass }}"
{{- end }}
{{- end }}

The explanation is:

  ## mariadb data Persistent Volume Storage Class
  ## If defined, storageClassName: <storageClass>
  ## If set to "-", storageClassName: "", which disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
  ##   GKE, AWS & OpenStack)
  ##
  # storageClass: "-"

My question was, since the template-rendered YAML will either be a storage class name, an empty string (to disable dynamic provisioning), or not set at all, can't we just do this instead?

  ## mariadb data Persistent Volume Storage Class
  ## If defined, storageClassName: <storageClass>
  ## If set to empty string "", this disables dynamic provisioning
  ## If undefined (the default) or set to null, no storageClassName spec is
  ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
  ##   GKE, AWS & OpenStack)
  ##
  # storageClass: ""

Then the template logic could be simplified to:

{{- if .Values.persistence.storageClass }}
  storageClassName: "{{ .Values.persistence.storageClass }}"
{{- end }}

But this would only work if an empty string is different from not set (nil) when checking if .Values.some.value in go template. Can someone help clarify this for me, or point to the relevant part of the docs? I seem to not be able to find this info.

Thanks!

Recently I needed to work with string slices in templates and apply various functions to each element (title, lower, trim, etc). I wanted to avoid using loops and assignments, so I came up with eachTitle, eachLower, eachTrim, etc functions.

It's quite simple really:

// each applies a function to each string in a slice.
func each(s []string, fn func(string) string) []string {
	for key, value := range s {
		s[key] = fn(value)
	}

	return s
}

// eachFunc makes a simple string transformation function to an each function.
func eachFunc(fn func(string) string) func([]string) []string {
	return func(s []string) []string {
		return each(s, fn)
	}
}

Usage:

funcMap["eachTitle"] = eachFunc(strings.Title)

Do you think it would fit into this package? I could extend it into a solution that would work with functions that have more than one parameter.

Need a way to conditionally check a value against a regex.

{{- if match-regex $my_value '^[a-z]...'  }}

{{- end }}

or something like that.

For versioning, would it be possible to tag the releases with v#.#.#?

Doing a go get will pull a potentially unstable master branch, where using gopkg.in will retrieve the latest tagged release branch.

$ go get -d gopkg.in/Masterminds/sprig.v2
** this will fail

POC: I forked sprig and did the following which now works correctly

**fork sprig via github
$ git clone [email protected]:TerraTech/sprig
$ cd sprig
$ git tag v2.12.0 2.12.0
$ git push --tags
$ go get -d gopkg.in/TerraTech/sprig.v2
** correctly pulls the v2.12.0 tag

Hello, I spent some time understanding how to extract the element at index n of an Array

Es:

arr := [1,2,3]
I want to get arr[1] => 2

I finally found the index function, but is not documented in the docs and I did't find it's declaration in the code.

The usage is

{{ index arr 2 }}

and I found references in:

• list_test.go
• string_test.go

Where does it come from? Should be documented in the list.md doc file?

I'm sorry if it's something obvious, I'm new to go and sprig. I can provide a PR to the documentation if you point me in the right direction.

Thank you!

It would be nice to have strings.TrimSuffix:

{
  "trimSuffix":  strings.TrimSuffix
}

kebabcase "helloWorld" 

would both produce hello-world, similar to how camelcase and snakecase work.

Hello,

my go version is: go version go1.8.3 linux/amd64

When I want to use your library I get next message:

# github.com/Masterminds/sprig
../../Masterminds/sprig/crypto.go:35: multiple-value uuid.NewV4() in single-value context

Checking crypto.go file I have seen this:

// uuidv4 provides a safe and secure UUID v4 implementation
func uuidv4() string {
	return fmt.Sprintf("%s", uuid.NewV4())
}

But uuid.NewV4()' return two values, the UUIDand anerror`

I've send you one pull request #74.

Regards

The documentation doesn't mention many of the functions which are actually available:

• regex*
• b32*
• fail
• genCA/genSelfSignedCert/genSignedCert
• merge
• toJson/toPrettyJson
  more probably

I wanted to iterate over a value defined size list of integers, and skip integers matching a given value.

Ideally this would be doable via..

{{range $k,$v := without (until $max) $skipMe}}{{$.Values.service.name}}-{{$v}}{{end}}

Except until $max returns int[] and without expects interface[]

I tried a few other ways to build the list, and found that regexFindAll splitList and others also return primitive arrays that are incompatible with interface[]

The only way I could see to create a list of interface[] was via the list function, but that doesn't allow me to build my list from 0 to $max because I need to specify all the list members up front. (Couldn't build the list using range either, because that declared a new list in the range scope each time).

In the end, I found a workaround by going to/from json, with a bit of fudging, I ended up with ..

{{ $rawjson := ( until $max | toJson )}}{{ $jsonmap := cat "{\"values\":" $rawjson "}" }}{{ $newmap := replace "[" "[\"" $jsonmap }} {{ $newmap := replace "]" "\"]" $newmap }} {{ $newmap := replace "," "\",\"" $newmap}}{{ $currentasstr := cat $v }}{{ $parsedmap := fromJson $newmap }}{{ $filtered := (without $parsedmap.values $currentasstr) }}{{ range $o, $p := $filtered }}{{ if ne $o 0 }},{{end}}{{$.Values.service.name}}-{{$p}}{{end}}

Which is pretty nasty ! Ideally the methods returning arrays of things not compatible with the interface[] type expected by without etc would start returning compatible types.

Hi,

the doc to the default function states:

default: Give a default value. Used like this: trim " "| default "empty".
Since trim produces an empty string, the default value is returned. For
things with a length (strings, slices, maps), len(0) will trigger the default.
For numbers, the value 0 will trigger the default. For booleans, false will
trigger the default. For structs, the default is never returned (there is
no clear empty condition). For everything else, nil value triggers a default.

A common scenario is to template config files, where you want to define a default value if nothing is given by the user.

default true "my value"
# "my value"
default true ""
# true

If your default option needs to be true and the user supplies false to deactivate the option. The default function will use the default instead of the user supplied value.

default true false
# true

In my opinion this behavior is a pitfall and needs to be changed, that boolean true/false are passed as they are if the default is also boolean or needs to be more clear in the documentation.
You can circumvent the problem with the following template:

{{/*
defaulttrue: Give true as default value. Used like this: trim "   "| defaulttrue.
Since trim produces an empty string, the default value true is returned. For
things with a length (strings, slices, maps), len(0) will trigger the default value true.
For numbers, the value 0 will trigger the default value true. For booleans, the value is
passed as is. For structs, the default is never returned (there is no clear
empty condition). For everything else, nil value triggers the default value true.
*/}}
{{- define "defaulttrue" -}}
  {{- if typeIs "bool" . -}} 
    {{- . -}}
  {{- else -}}
    {{- default true . -}} 
  {{- end -}}
{{- end -}}

Just a suggestion: deprecate biggest and add max and min funcs.

Hi,

Why do you type juggle with template.FuncMap and html.FuncMap in the public functions instead of using only map[string]interface{} ?

Also, in this, i suspect it is not working on a copy, compare with this playground

btw, i noticed a typo at => tpl := template.New("foo").Funcs(sprig.FuncMap())

In #10 @technosophos said:

I have been thinking of adding some crypto functions to sprig for use with public key cryptography use cases.

I'd actually love if sprig supported generation of 1024 bit dhparams. This would be tremendously useful to anyone, for instance, using github.com/helm/helm to manage Nginx or other web servers.

fwiw, generating 2048 bit dhparams would be lunacy because it takes soooo long. But what would we think about 1024 bit dhparams?

This might be crazy.

It would be nice to add localization functions. Not sure how it would all be composed, but something like:

t "FR" "cheese" -> "fromage" would be nice.

It would be particularly nice to be able to set a "global" language and translation strings facility and then be able to just call t "cheese". This is how Drupal does it.

The backend would need something like:

type Translator interface {
  T(string) string
}

An easy map-based backend could satisfy that interface.

I don't think this was purposeful, but let me know why if it was a decision made.

Given the following go program:

package main
import (
        "fmt"
        "os"
        "text/template"
        "github.com/Masterminds/sprig"
)
func main() {
        tpl := `{{- $i := genSelfSignedCert "example.com" nil nil 365 }}{{ $i.Cert }}`
        t := template.Must(template.New("test").Funcs(sprig.TxtFuncMap()).Parse(tpl))
        if err := t.Execute(os.Stdout, nil); err != nil {
                fmt.Printf("Error during template execution: %s", err)
                return
        }
}

a certificate is printed to stdout. This certificate is a valid certificate readable by openssl, etc. However, given a verify command, openssl cannot verify this cert using itself:

# openssl verify -CAfile out.crt out.crt
test.crt: CN = example.com
error 20 at 0 depth lookup:unable to get local issuer certificate

It is a common use case of a self signed cert to use for a cert that is verified by itself to encrypt two systems, so I assume this should work for this use case.

The corresponding openssl commands being

# openssl req -x509 -nodes -batch -days 86400 -newkey rsa:2048 -subj "/CN=example.com" -out "./tls.crt" -keyout "./tls.key"
Generating a 2048 bit RSA private key
...+++
.............................+++
writing new private key to './tls.key'
-----
# openssl verify -CAfile tls.crt tls.crt
tls.crt: OK

I would note that making a sprig CA using genCA and using it in genSignedCert will allow you to verify the cert with the CA correctly - as expected.

EDIT:
adding x509.KeyUsageCertSign to template.KeyUsage in crypto.go::generateSelfSignedCertificate() here does fix this use case - not sure about all the implications.

This is suggested here:

http://stackoverflow.com/questions/18276173/calling-a-template-with-several-pipeline-parameters/18276968#18276968

It'd stand beside the tuple function, which is the equivalent for arrays.

This is really useful for writing templates that are encapsulated and don't depend on the full dot value.

The changelog (see version at time of issue) is from the 1.x version. We are now well into the 2.x version. It should be updated (from the release notes on GitHub?) or removed.

There are cases when errors are not handled but it could be useful for debugging compilation errors. See 1, 2, 3

In some cases it is pretty much all right to return default-ish values silently but in some cases it would be useful to fail and print the error message.

Proposal:

• extend the existing API and introduce {func}E versions of some funcs like it is done here https://github.com/spf13/cast

A function that can generate the htpasswd type hash from username & password would be great. In Kubernetes Helm charts, this would help in configuring basic auth for ingress resources.

Hi Everyone,

I am sorry for writing this here. But I cannot find the answer and really wondering reason why the sprig uses the $_ when using the set function. in the documentation it is like {{ $_ := set .... }}
Is $_ a special variable or something in sprig, please?

Thank you very much and looking forward to learn more on this.

Hello,

The title contains a very specific description but the tooling could be generic.
We would like a file mode filter that can convert from unix octal permissions 0755 to decimal. The reason being that json does not support octal and it makes writing and maintaining charts quite cumbersome.

See helm/helm#4378 (comment) for use case.

Thanks,

../../Masterminds/sprig/crypto.go:35:37: multiple-value uuid.NewV4() in single-value context

Wrap and WrapWith both lose the character at the wrap point

Test example :

import (
  "github.com/Masterminds/sprig"
  "html/template"
  "bytes"
)

func main() {
  // The template
  tpl := "Word Wrap Test\n{{.Name | wrap 11 }}\n"

  // The values to pass to the template
  vars := map[string]string{"Name": "123456789_123456789_123456789_123456789_1234567890"}

  // Compile the template
  t := template.Must(template.New("example").Funcs(sprig.FuncMap()).Parse(tpl))

  // Run the template, copying the output to the buffer.
  var b bytes.Buffer
  err := t.Execute(&b, vars)
  if err != nil {
      panic(err)
  }
  print(b.String())
}

Output showing issue - in the first split line 1 is lost and 2 starts the second line, and so on :

Word Wrap Test
123456789_
23456789_1
3456789_12
456789_123
567890

So I need a template function that will wrap double quotes around a provided string. So something like this: {{ stringify "this string" }}. Right now go templates will not enclose the string with quotes so I have to do something like this: `"{{"this string"}}". Which looks weird when Im using go templates to output configuration files.

When passed an array, only pass unique values.
http://stackoverflow.com/a/26875358

Something like:

{{range $index, $thing := unique $}}

Currently there is no way to add any key to already instantiated variable. It makes building large dictionaries difficult, as all key needs to be specified within a single {{ with }} command.
It would be great to be able to build dict variable by adding 1 key at the time.

example:
{{ $env := dict "blah-1" $blah-1 }}
{{ $env := dict "blah-2" $blah-2 }}
resulting $env having two pairs of keys.

I have a case where I want a template to take an input dictionary and produce a copy, omitting a few keys at various levels deep in the hierarchy (in this helm chart).

I ran into a snare, where I didn't realize that after merging two dictionaries, editing the merged dictionary can modify the originals:

{{- $values := pick .Values "auth" "other"}}
{{- /* trim secret values. Update here if new secrets are added! */ -}}
{{- /* make a copy of values.auth to avoid modifying the original */ -}}
{{- $_ := set $values "auth" (merge dict .Values.auth) }}
{{- $_ := set $values.auth "state" (omit $values.auth.state "cryptoKey") }}

Without that set $values "auth" (merge dict .Values.auth),
the subsequent set $values.auth "state" would modify the original .Values.auth.state, which I found surprising.

Is there an existing mechanism to produce a deep copy of a dict, to ensure the original doesn't get modified?

Most of the regex functions are unfortunately significantly broken when it comes to using them in pipelines. By broken I mean the functions technically work, but their usage I don't think would be intuitive to anyone, and that usage is difficult.

For example to use the regexReplaceAll function, you cannot do this:

"foo subject string" | regexReplaceAll "foo" "bar"

You instead have to do this:

"bar" | regexReplaceAll "foo" "foo subject string"

This means that if you are operating in a pipeline, you have to put your pipeline as the 2nd argument to the function, or store it in a variable first, which can be really messy and hard to maintain.

Golang templates pass the result of the previous item in the pipeline as the last argument to the function being called. Thus the arguments of all the regex functions need to be swapped around.

Unfortunately changing this is a breaking change, but I think this is a fairly major usage issue. And since these functions aren't even documented (ref #84), the impact might be minor.

It would be a nice utility to be able to pluralize a string. For example, {{ pluralize 1 "anchovy" "anchovies" }} would return anchovy, but ``{{ pluralize 2 "anchovy" "anchovies" }}would returnanchovies`

{{ $envDict := pick .Values.global .Values.global.env }}
{{ $chartDict := pick $.envDict .Chart.Name }}
{{ $imgDict := pick $.chartDict "image" }}
{{- define "app.release_labels" }}
app: {{ printf "%s-%s" .Release.Name .Chart.Name | trunc 63 }}
version: {{ .Chart.Version }}
release: {{ .Release.Name }}
{{- end }}
{{- define "app.full_name" -}}
{{- printf "%s-%s" .Release.Name .Chart.Name | trunc 63 -}}
{{- end -}}
{{- define "app.repository" -}}
{{- printf "%s" .imgDict.repository | trunc 63 -}}
{{- end -}}
{{- define "app.tag" -}}
{{- printf "%s" .imgDict.tag -}}
{{- end -}}
{{- define "app.replica_count" -}}
{{- printf "%s" $.chartDict.replicaCount-}}
{{- end -}}

I am getting an error: executing "app.replica_count" at <$.charDict.replicaCo...>: can't evaluate field replicaCount in type interface {}

I am using this as a helper file in my helm chart repo. The input file to this looks like this:

global:
env: "dev"
dev:
hello-world:
image:
repository: gcr.io/google-samples
tag: '1.0'
replicaCount: '1'
hello-kube:
image:
repository: gcr.io/google-samples
tag: '1.0'
replicaCount: '1'
stage:
hello-world:
image:
repository: gcr.io/google-samples
tag: '1.0'
replicaCount: '1'
hello-kube:
image:
repository: gcr.io/google-samples
tag: '1.0'
replicaCount: '1'

Can some one please help me with what I am doing wrong?..
Many thanks in advance.

The list function has is implemented as has needle haystack,
see https://github.com/Masterminds/sprig/blob/master/list.go#L241

The documentation at https://github.com/Masterminds/sprig/blob/master/docs/lists.md
on the other hand claims the reverse: has haystack needle.

Trying to follow the documentation, like has .Values.foo "x" will cause the error Cannot find has on type string.

I have a use for needing to easily serialize a map[string]string given a sep and assign. I need it for a helm chart where I want to serialize a yaml map into an env var of the form key:value,key2:value which will then be auto deserialized into a map with https://github.com/kelseyhightower/envconfig.

Do you think others might find this useful? I want to add it here because helm uses this lib for utility functions in templates.
If you're willing to add such a function here I'd be happy to create a PR.

I would make the implementation similar to the join (join: strings.Join, but as join SEP SLICE) function currently supported.
something like joinMap SEP ASSIGN MAP
and then joinMap "," ":" map[string][string]{"a":"b", "c", "d"} results in a:b,c:d

It's not clear what happens with div if the result is not an int. Need to be able to explicitly control if it gets rounded up or down.

Example use case: calculate the min number of nodes required to form a quorum, i.e. {{ replicas | div 2 | ceil}}. I'm not sure this will work even if we had ceil/floor functions because I'm guessing div is already performing an implicit ceil or floor.

I suppose if div is already doing an implicit floor then we can do {{ replicas | div 2 | add1}}.

Thoughts?

In many projects you still need sha1sum. In my example I want to write a Helm chart for Jenkins with automatic scriptApproval for my jobdsl scripts.

A very useful addition to the Path function class would be functions dealing with URLs, like extracting domain, path or a query argument from a URL.

The function Add, for example, takes two arguments. Why not take an arbitrary number? The language is prefix so it's easy to have multiple arguments:

{{Add 2 3 4}}

func Add(nums []int) int

You could make them even more general by accepting any numeric type and converting as necessary under the covers, but that's a lot more work.

I'm trying to split a string into parts and take the last element.

{{regexSplit "/src/" "foo/src/bar" -1 | last}}

Expecting "bar" but getting

 executing "" at <last>: wrong type for value; expected []interface {}; got []string

As a work around I have done this:

{{with $my:=regexSplit "/src/" "foo/src/bar" -1}}
     {{index $my 1}} # Can't do arithmetic in templates :-(
{{end}}

Which feels unreasonably cumbersome and still not quite what I wanted.

Thanks for building this lib! I really like it, but I'd like it even more if it haven't any dependency (outside the standard library)

Some of the current dependencies are to allow functions I don't care at all on templates, like UUID generation

For others, the following Go proverb applies:

"A little copying is better than a than a little dependency"

Rob Pike

For example, the xstrings package is imported just because of two functions

Proposal

Have a github.com/Masterminds/sprig/slim package that have all the functions that doesn't require importing external libraries

sprig is currently locked to version 0.2.2 of mergo in glide.lock. Version 0.3.0 added this fix for merging slices correctly (deeply). Any chance of updating to 0.3.0? Here is a comparison between those versions for reference.

I'd like to have requiredEnv which would work similar to this https://github.com/roboll/helmfile

The required_env function allows you to declare a particular environment variable as required for template rendering. If the environment variable is unset or empty, the template rendering will fail with an error message.