Git Product home page Git Product logo

northwood-labs / terraform-provider-corefunc Goto Github PK

View Code? Open in Web Editor NEW
16.0 1.0 1.0 3.89 MB

Utilities that should have been Terraform core functions.

Home Page: https://registry.terraform.io/providers/northwood-labs/corefunc

License: Apache License 2.0

HCL 23.89% Makefile 3.28% Go 69.43% Shell 2.62% Python 0.78%
go golang opentofu opentofu-provider terraform terraform-provider terratest bats data-sources functions

terraform-provider-corefunc's Introduction

Terraform Provider: Core Functions

Terraform Docs Library.tf Go Docs GitHub Go Report Card Open Source Insights GitHub issues GitHub contributors GitHub commit activity (branch) GitHub all releases GitHub Workflow Status (with event) GitHub Workflow Status (with event)

Overview

Utilities that should have been Terraform/OpenTofu core functions.

While some of these can be implemented in HCL, some of them begin to push up against the limits of Terraform and the HCL2 configuration language. We also perform testing using the Terratest framework on a regular basis. Exposing these functions as both a Go library as well as a Terraform/OpenTofu provider enables us to use the same functionality in both our Terraform/OpenTofu applies as well as while using a testing framework.

Note

While it’s common knowledge that Terraform is great at standing up and managing Cloud infrastructure, it’s also good at running anything with an API. People regularly manage code repositories, DNS records, feature flags, identity and access management, content delivery, passwords, monitoring, alerts, zero trust network access, cryptographic signatures, and can even order a pizza.

This provider is more analogous to HashiCorp’s utility providers such as local, external, and archive.

Since earlier versions of Terraform/OpenTofu didn't have the concept of user-defined functions, the next step to open up the possibilities was to write a custom Provider which has the functions built-in, using existing support for inputs and outputs.

This does not add new syntax or constructs. Instead it uses the existing concepts around Providers, Resources, Data Sources, Variables, Outputs, and Functions to expose new custom-built functionality.

The goal of this provider is not to call any APIs, but to provide pre-built functions in the form of Data Sources or Provider Functions.

Compatibility testing

  • We have automated testing that runs on every commit and every pull request.
  • We intend for the Go libraries to work with all non-EOL versions of Go (i.e., current, current-1).
  • Built using the Terraform Plugin Framework, which speaks Terraform Protocol v6.
Testing type Details Description
integration Terraform 1.0–1.8 Executes the provider with this release, pulling from registry.terraform.io.
integration OpenTofu 1.6–1.7 Executes the provider with this release, pulling from registry.opentofu.org.
unit Go 1.21–1.22 Tests using these versions.
mutation Go 1.21–1.22 Tests using these versions.
fuzz Go 1.21–1.22 Tests using these versions.
terratest Go 1.21–1.22 Tests using these versions.

Usage Examples

See the docs/ directory for user-facing documentation.

Documentation

Terraform Registry

If you are using this as a Terraform provider, see the documentation at registry.terraform.io.

Go Package

If you are using this as a Go library, see the documentation at pkg.go.dev.

More Information

After the provider is installed, you can run terraform-provider-corefunc on the CLI.

  • Install with either terraform init or make build.
  • The Go binary path (discovered by running ./find-go-bin.sh) is on your $PATH.

This will display the following text:

terraform-provider-corefunc
This binary is a plugin. These are not meant to be executed directly.
Please execute the program that consumes these plugins, which will
load any plugins automatically

However, by passing the --help flag, you can see the other options available, including a description of the software.

terraform-provider-corefunc --help

The provider has one primary sub-command: version. It includes long-form version information, including the build commit hash, build date, Go version, and external dependencies.

terraform-provider-corefunc version
BASIC
Version:    dev
Go version: go1.22.1
Git commit: 822858fc0c80b34eebf2a5ddd1b48684414d71b3
Build date: 2024-03-11T03:09:24Z
OS/Arch:    darwin/arm64
System:     macOS on Apple Silicon
CPU Cores:  10

DEPENDENCIES
github.com/bits-and-blooms/bitset                          v1.13.0
github.com/chanced/caps                                    v1.0.2
github.com/fatih/color                                     v1.16.0
github.com/golang/protobuf                                 v1.5.4
github.com/gookit/color                                    v1.5.4
github.com/hashicorp/go-hclog                              v1.6.2
github.com/hashicorp/go-plugin                             v1.6.0
github.com/hashicorp/go-uuid                               v1.0.3
github.com/hashicorp/terraform-plugin-framework            v1.6.1
github.com/hashicorp/terraform-plugin-framework-validators v0.12.0
[...snip...]

terraform-provider-corefunc's People

Contributors

dependabot[bot] avatar skyzyx avatar step-security-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

terraform-provider-corefunc's Issues

Implement `corefunc_list_push`

Go: corefunc.ListPush(arr, any)
TF: corefunc_list_push

Push a new item onto the end of a list, mutating the list along the way. Equivalent to Array.push() in JavaScript.

Implement `corefunc_strip_comments_and_spaces`

  • Go: corefunc.StripCommentsAndSpaces(string)
  • TF: corefunc_strip_comments_and_spaces

Idea is to have something that can strip all comments and inner-whitespace from a long, multi-line string. The original use-case is for enabling users to wrap and comment things like regular expressions, then use this to strip all of that stuff back out before passing it to regex(), regexall() or replace() in Terraform.

Regexes should use \s to match spaces instead of using a literal ASCII=20 space character.

Supported comments begin with a #// (space, octothorpe, slash, slash, space — making this up as I go along), and end with an LF or CRLF. This means that it will only support single-line, end-of-line comments.

The goal is not to validate or even execute the regex. The goal is simply compress it into a single line without whitespace or comments without messing it up, and without having to do list/join gymnastics.

Looking for examples both simple and gnarly.

Implement `corefunc_env_copy_to_var`

Go: corefunc.EnvCopyToVar(envName, tfVarName)
TF: corefunc_env_copy_to_var

Real-world use-case: When integrating PagerDuty and Datadog, need to read PagerDuty API key from PAGERDUTY_TOKEN, then pass it to a variable, so that the Datadog provider can read it without printing in text (which is insecure).

If a Terraform variable is called var.pagerduty_token, then this would read PAGERDUTY_TOKEN and then write that same value to TF_VARS_pagerduty_token so that Terraform will read it as the variable value.

variable "pagerduty_token" {
  description = "PagerDuty API token. Value is set via .env file."
  type        = string
}

data "corefunc_env_copy_to_var" "pagerduty_token" {
  name     = "PAGERDUTY_TOKEN"
  variable = "pagerduty_token"
}

Yes, you could do export TF_VARS_pagerduty_token=$PAGERDUTY_TOKEN, but:

  1. You'd be surprised how few people know about this.
  2. This (and corefunc_env_ensure) help make it an explicit and unmissable part of running the Terraform.

Implement `corefunc_list_pop`

Go: corefunc.ListPop(array)
TF: corefunc_list_pop_item

Remove and return the last item in a list, mutating the list along the way. Equivalent to Array.pop() in JavaScript.

[Feature]: `corefunc_semver_must_meet_constraint`

What functionality would you like to see?

Validates whether or not a version meets a constraint. If not, triggers an error.

data "corefunc_semver_constraint" "constraint" {
  version    = "1.2.3-beta.1+build345"
  constraint = ">= 1.2.3, < 2.0.0"
}
#=> Error

[Feature]: `corefunc_url_googlesafe_parse`

What functionality would you like to see?

Same as corefunc_url_parse, but with the stricter Google Safe Browsing profile applied instead.

data "corefunc_url_googlesafe_parse" "url" {
  url = "http://example.com:80/a?b#c"
}

[Feature]: `corefunc_github_assets`

What functionality would you like to see?

Download assets from GitHub.com/GitHub Enterprise Server.

corefunc_github_assets "trivy" {
  endpoint = "https://api.github.com" # Default value
  token    = var.ghes_token # Use GITHUB_TOKEN for GitHub.com, or GHE_TOKEN for GitHub Enterprise

  repository  = "aquasecurity/trivy"
  version     = "~> 0.40" # Default value is latest
  asset_match = "${OS}-${ARCH}.tar.gz$"
  decompress  = true # Default
  save_as     = "${path}/trivy.tar.gz" # Useful when download is uncompressed binary

  // If you want to simplify the path
  symlink = {
    from = "bin/trivy*",
    to   = "trivy",
  }

  // When current OS is… set ${OS} to…
  set_os_value_when = {
    linux   = "Linux"
    darwin  = "Darwin"
    windows = "Windows"
    solaris = "Solaris"
    freebsd = "FreeBSD"
    openbsd = "OpenBSD"
    netbsd  = "NetBSD"
  }

  // When current CPU architecture is… set ${ARCH} to…
  set_arch_value_when = {
    intel32 = "32bit"
    intel64 = "64bit"
    arm32   = "ARM"
    arm64   = "ARM64"
    ppc32   = "PPC"
    ppc64   = "PPC64LE"
    s390x   = "s390x"
  }
}

Implement `corefunc_str_constant`

Go: corefunc.StrConstant(string)
TF: corefunc_str_constant

A CONSTANT_CASE formatting option.

THIS_IS_CONSTANT_CASE

Is this the right name for this?

[Feature]: `corefunc_http_request`

What functionality would you like to see?

Something that allows users to make HTTP requests beyond simple GET requests.

corefunc_http_request "example" {
  method   = "POST"
  endpoint = "https://example.com/form.php"
  query    = ["q": 1]
  headers  = ["Content-Type": "text/plain"]
  body     = file("${path}/hello.txt")

  timeout_ms  = 2000     # Either this, or timout_sec
  timeout_sec = 2        # Either this, or timout_ms
  min_tls     = "tls1.2" # Default
  ca_cert     = file("${path}/ca-cert.pem")

  error_on_failure     = true            # Default
  success_status       = ">= 100, < 400" # Default
  enable_tracing       = false           # Default
  enable_debug_logging = false           # Default
  follow_redirects     = true            # Default
  retry_algorithm      = "default"       # Default
  retry_max            = 3               # Default

  save_output = "${path}/output.txt"
}

[Feature]: `corefunc_semver_coerce`

What functionality would you like to see?

Coerces a not-quite semantic version into a semantic version.

data "corefunc_semver_coerce" "coersion" {
  version = "1.6"
}
#=> 1.6.0

Implement `corefunc_println`

Go: corefunc.Println(string)
TF: corefunc_println

Print a custom line of text to the plan/apply output.

  • Add color support?
  • Would support heredocs, of course.

[Feature]: `corefunc_semver_meet_constraint`

What functionality would you like to see?

Validates whether or not a version meets a constraint.

data "corefunc_semver_constraint" "constraint" {
  version    = "1.2.3-beta.1+build345"
  constraint = ">= 1.2.3, < 2.0.0"
}
#=> false

[Feature]: `corefunc_markdown_render`

[Feature]: `corefunc_url_parse`

What functionality would you like to see?

Parses a URL according to the rules of the WHATWG URL Specification.

data "corefunc_url_parse" "url" {
  url = "http://example.com:80/a?b#c"
}
#=> {
#  scheme            = "http"
#  host              = "example.com"
#  port              = ""
#  pathname          = "/a"
#  hash              = "#c"
#  fragment          = "c"
#  search            = "?b"
#  query             = "b"
#  normalized        = "http://example.com/a?b#c"
#  normalized_nofrag = "http://example.com/a?b"
# }

[Feature]: `corefunc_semver_sort`

What functionality would you like to see?

Sorts a list of versions according to the rules of semver 2.0.

data "corefunc_semver_sort" "versions" {
  versions = ["1.2.3-beta.1+build345", "1.2.3", "0.0.1", "1.5"]
  reverse  = true
}
#=> ["1.5", "1.2.3", "1.2.3-beta.1+build345", "0.0.1"]
```]

[Feature]: `corefunc_semver_validate`

What functionality would you like to see?

Validates a version as a semantic version. An invalid version will respond with false.

data "corefunc_semver_validate" "coersion" {
  version = "1.6"
}
#=> false

Implement `corefunc_str_resolve_markers`

Go: corefunc.StrResolveMarkers(string)
TF: corefunc_str_resolve_markers

The original need was wanting to implement something similar to Late Static Binding where markers could be replaced with values, once those values were determined, as late as possible in the execution process. The replacement values would have been variable/local values.

Terraform does not support this natively.

Need to figure out if we can reflect the current state of all locals/variables in Terraform, and whether or not they are resolved values. If so, we could implement (the very edge-casey) LSB support for string replacements using locals/variables.

If we cannot, then this feature is dead in the water.

Implement `corefunc_list_unshift`

Go: corefunc.ListUnshift(arr)
TF: corefunc_list_unshift

Push a new item onto the front of a list, mutating the list along the way. Equivalent to Array.unshift() in JavaScript.

Implement `corefunc_list_shift`

Go: corefunc.ListShift(arr, any)
TF: corefunc_list_shift

Removes and returns the first item in a list, mutating the list along the way. Equivalent to Array.shift() in JavaScript.

[Feature]: `corefunc_homedir_get`

What functionality would you like to see?

Gets the home directory of the current user.

data "corefunc_homedir_get" "home" {}
#=> /Users/me

Implement `corefunc_dump`

  • Go: corefunc.Dump(any)
  • TF: corefunc_dump

Idea is to have something that can dump the contents of a variable. There are a few things that do this in Go, like gospew. However, I don't think that a straight-up spew of variable internals from Go is the right solution as HCL is a much simpler language with many-fewer constructs.

But I've not yet seen anything that provides this in Terraform other than, maybe, terraform console — which I've never used, but is supposed to be like a REPL for your Terraform state. This is meant to be something that works with Terraform plan, which more users are familiar/comfortable with.

The HCL language is strongly typed under the hood, and in the Terraform Plugin Framework. Not sure if this can be implemented without some sort of generics system, or by supporting any somehow.

Will need to investigate.

[Feature]: `corefunc_semver_parse`

What functionality would you like to see?

Parses a semantic version (coercing if necessary), and provides an object of parts.

data "corefunc_semver_parse" "parse1" {
  version = "1.2.3-beta.1+build345"
}

#=> {
#  major      = 1
#  minor      = 2
#  patch      = 3
#  prerelease = beta.1
#  metadata   = build345
# }

[Feature]: `corefunc_semver_must_validate`

What functionality would you like to see?

Strictly validates a version as a semantic version. An invalid version will trigger an error.

data "corefunc_semver_must_validate" "coersion" {
  version = "1.6"
}
#=> Error

Implement `corefunc_str_leftpad`

Go: corefunc.StrLeftpad(any, string, length)
TF: corefunc_str_leftpad

Pad the left side of a string with a specified character, a specified number of times.

[Feature]: `corefunc_homedir_expand`

What functionality would you like to see?

Resolves ~ in a path.

data "corefunc_homedir_expand" "home" {
  path = "~/.aws/"
}
#=> /Users/me/.aws/

Implement `corefunc_str_word`

Go: corefunc.StrWord(string)
TF: corefunc_str_word

The original idea is to take a string and strip-out any non-word characters (uppercase, lowercase, digits, underscores). Although this idea might need more batting around.

Implement `corefunc_str_studly`

Go: corefunc.StrStudly(string)
TF: corefunc_str_studly

A StudlyCase formatting option. Also known as "PascalCase" or "UpperCamelCase".

ThisIsStudlyCase

Is there a more gender-neutral name for this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.