Git Product home page Git Product logo

terraform-registry-address's Introduction

terraform-registry-address

This module enables parsing, comparison and canonical representation of Terraform Registry provider addresses (such as registry.terraform.io/grafana/grafana or hashicorp/aws) and module addresses (such as hashicorp/subnets/cidr).

Provider addresses can be found in

Module addresses can be found within source argument of module block in Terraform configuration (*.tf) and parts of the address (namespace and name) in the Registry API.

Compatibility

The module assumes compatibility with Terraform v0.12 and later, which have the mentioned JSON output produced by corresponding CLI flags.

We recommend carefully reading the ambigouous provider addresses section below which may impact versions 0.12 and 0.13.

Related Libraries

Other libraries which may help with consuming most of the above Terraform outputs in automation:

Usage

Provider

pAddr, err := ParseProviderSource("hashicorp/aws")
if err != nil {
	// deal with error
}

// pAddr == Provider{
//   Type:      "aws",
//   Namespace: "hashicorp",
//   Hostname:  DefaultProviderRegistryHost,
// }

Module

mAddr, err := ParseModuleSource("hashicorp/consul/aws//modules/consul-cluster")
if err != nil {
	// deal with error
}

// mAddr == Module{
//   Package: ModulePackage{
//     Host:         DefaultProviderRegistryHost,
//     Namespace:    "hashicorp",
//     Name:         "consul",
//     TargetSystem: "aws",
//   },
//   Subdir: "modules/consul-cluster",
// },

Other Module Address Formats

Modules can also be sourced from other sources and these other sources (outside of Terraform Registry) have different address formats, such as ./local or github.com/hashicorp/example.

This library does not recognize such other address formats and it will return error upon parsing these.

Ambiguous Provider Addresses

Qualified addresses with namespace (such as hashicorp/aws) are used exclusively in all recent versions (0.14+) of Terraform. If you only work with Terraform v0.14.0+ configuration/output, you may safely ignore the rest of this section and related part of the API.

There are a few types of ambiguous addresses you may comes accross:

  • Terraform v0.12 uses "namespace-less address", such as aws.
  • Terraform v0.13 may use - as a placeholder for the unknown namespace, resulting in address such as -/aws.
  • Terraform v0.14+ configuration still allows ambiguous providers through provider "<NAME>" {} block without corresponding entry inside required_providers, but these providers are always resolved as hashicorp/<NAME> and all JSON outputs only use that resolved address.

Both ambiguous address formats are accepted by ParseProviderSource()

pAddr, err := ParseProviderSource("aws")
if err != nil {
	// deal with error
}

// pAddr == Provider{
//   Type:      "aws",
//   Namespace: UnknownProviderNamespace,    // "?"
//   Hostname:  DefaultProviderRegistryHost, // "registry.terraform.io"
// }
pAddr.HasKnownNamespace() // == false
pAddr.IsLegacy() // == false
pAddr, err := ParseProviderSource("-/aws")
if err != nil {
	// deal with error
}

// pAddr == Provider{
//   Type:      "aws",
//   Namespace: LegacyProviderNamespace,     // "-"
//   Hostname:  DefaultProviderRegistryHost, // "registry.terraform.io"
// }
pAddr.HasKnownNamespace() // == true
pAddr.IsLegacy() // == true

However NewProvider() will panic if you pass an empty namespace or any placeholder indicating unknown namespace.

NewProvider(DefaultProviderRegistryHost, "", "aws")  // panic
NewProvider(DefaultProviderRegistryHost, "-", "aws") // panic
NewProvider(DefaultProviderRegistryHost, "?", "aws") // panic

If you come across an ambiguous address, you should resolve it to a fully qualified one and use that one instead.

Resolving Ambiguous Address

The Registry API provides the safest way of resolving an ambiguous address.

# grafana (redirected to its own namespace)
$ curl -s https://registry.terraform.io/v1/providers/-/grafana/versions | jq '(.id, .moved_to)'
"terraform-providers/grafana"
"grafana/grafana"

# aws (provider without redirection)
$ curl -s https://registry.terraform.io/v1/providers/-/aws/versions | jq '(.id, .moved_to)'
"hashicorp/aws"
null

When you cache results, ensure you have invalidation mechanism in place as target (migrated) namespace may change.

terraform provider

Like any other legacy address terraform is also ambiguous. Such address may (most unlikely) represent a custom-built provider called terraform, or the now archived hashicorp/terraform provider in the registry, or (most likely) the terraform provider built into 0.11+, which is represented via a dedicated FQN of terraform.io/builtin/terraform in 0.13+.

You may be able to differentiate between these different providers if you know the version of Terraform.

Alternatively you may just treat the address as the builtin provider, i.e. assume all of its logic including schema is contained within Terraform Core.

In such case you should construct the address in the following way

pAddr := NewProvider(BuiltInProviderHost, BuiltInProviderNamespace, "terraform")

terraform-registry-address's People

Contributors

bflad avatar dbanck avatar dependabot[bot] avatar hashicorp-copywrite[bot] avatar hashicorp-tsccr[bot] avatar radditude avatar radeksimko avatar xiehan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

terraform-registry-address's Issues

Functionality for Verifying Full Provider Addresses (hostname/namespace/type)

Use Case

Downstream in Terraform Provider SDKs such as terraform-plugin-go, terraform-plugin-framework, and terraform-plugin-sdk, it would be great if we could verify provider developer defined strings that should only contain a full provider address, e.g. "registry.terraform.io/hashicorp/example". We want to ensure a full provider address is submitted as this value can eventually wind up in Terraform reattach configurations.

This module currently provides the ParseAndInferProviderSourceString() and ParseRawProviderSourceString() functions, however they allow a few things we don't desire:

  • Missing hostname
  • Missing namespace
  • Namespace being - (which is valid for legacy reasons when the hostname is registry.terraform.io, but not something we want to allow for this use case)

Attempted Solutions

Manual logic:

// Validate a given provider address. This is only used for the Address field
// to preserve backwards compatibility for the Name field.
//
// This logic is manually implemented over importing
// github.com/hashicorp/terraform-registry-address as its functionality such as
// ParseAndInferProviderSourceString and ParseRawProviderSourceString allow
// shorter address formats, which would then require post-validation anyways.
func (opts ServeOpts) validateAddress(_ context.Context) error {
	addressParts := strings.Split(opts.Address, "/")
	formatErr := fmt.Errorf("expected hostname/namespace/type format, got: %s", opts.Address)

	if len(addressParts) != 3 {
		return formatErr
	}

	if addressParts[0] == "" || addressParts[1] == "" || addressParts[2] == "" {
		return formatErr
	}

	return nil
}

Proposal

Implement a function in this Go module similar to the above, potentially with better validation. 😄 e.g.

func ValidateFullProviderAddress(in string) error {
	addressParts := strings.Split(in, "/")
	formatErr := fmt.Errorf("expected hostname/namespace/type format, got: %s", in)

	if len(addressParts) != 3 {
		return formatErr
	}

	provider := Provider{
		Hostname:  addressParts[0],
		Namespace: addressParts[1],
		Type:      addressParts[2],
	}

	if provider.Hostname == "" || provider.Namespace == "" || provider.Type == "" {
		return formatErr
	}

	if provider.Namespace == "-" {
		return fmt.Errorf("expected defined namespace, the - namespace is reserved, got: %s", in)
	}

	return nil
}

References

Consider Tagged Release(s)

Use Case

Currently this Go module does not have any release tags, so consumers are stuck with commit-based references to the module:

github.com/hashicorp/terraform-registry-address v0.0.0-20210412075316-9b2996cce896

This also can make it more difficult for consumers can use Go module management tooling such as Dependabot for updates.

Proposal

Bootstrap a CHANGELOG.md and cut a v0.1.0 tag for starters. When later changes occur, update CHANGELOG.md and cut new semantic version tags.

Use mirror url in provider source

When setting a provider's source url to the artifactory registry mirror I receive this validation error:

The “source” attribute must be in the format “[hostname/][namespace/]name”

How can I bypass the validation?

The api artifactory url is:
https://instance-name.company-name.com/artifactory/api/terraform/virtual-repository-name/providers/"

Example url I've tried (although I haven't been able to confirm it works properly, due to the validation error):
instance-name.company-name.com/virtual-repository-name/newrelic/newrelic

Dots should be allowed in namespace for third-party module registry addresses

Use Case

If I am using a third-party module registry, I should be allowed to use dots in the namespace. My organisation is not able to use registry as module source because we have a dot in our namespace. It is not easy to change the namespace.

For instance, I should be allowed to use example.com/my.namespace/name/targetsystem as a module source.

When I try to use the above as module source, I get the following error from Terraform CLI:
source address must have three more components after the hostname: the namespace, the name, and the target system

I checked the tests and found this assertion for the above error.

"missing part with explicit hostname": {
	input:   `foo.com/var/baz`,
	wantErr: `source address must have three more components after the hostname: the namespace, the name, and the target system`,
},

However, in the above usecase, all three components are provided after the hostname example.com.

Proposal

Based on the test suite, I believe this is a buggy implementation which is mistakenly stopping a use-case it didn't intend to. We should add a test case for example.com/my.namespace/name/targetsystem and change the implementation to make it green.

Contribution

I am open to contributing a solution if you need help. I haven't used Golang before so I will need some time to implement the solution though.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.