1. make install should build the go binary and place it into the local 3rd-party TF plugins dir ~/.terraform.d/plugins

Some of our resources should support keepers to support re-creating resources periodically. For example, Service Account API Keys would benefit from this capability.

There are some examples from Terraform's official providers: https://github.com/hashicorp/terraform-provider-random/blob/27fa05e361c077b2da94d19353bf45fed11a29c9/internal/provider/resource_id.go#L58-L66

It would be great if you could deploy flows via this provider too.

We have a lot of AWS infrastructure that we deploy via a terraform module to match the needs of a flow. It would be awesome to add this provider in there to deploy the flow for us too - that way we could use the terraform module versioning as a way to version the flow and keep all our deployments in line.

We can add golangci-lint to this repository for static code analysis: https://golangci-lint.run/

The Terraform Plugin Framework includes a test harness for writing integration tests: https://developer.hashicorp.com/terraform/plugin/framework/acctests

Our automated tests would require an external endpoint and secrets, though we can optionally mock out the remote server or use Prefect server for some tests. The Terraform test framework does not support unit testing, so validating errors or warnings may be more difficult (it is not currently possible to check for warnings given a Terraform configuration/run; see: hashicorp/terraform-plugin-testing#69)

There is currently no test coverage for this provider and changes are tested manually on local development machines. It would be preferable to automate these tests, so that we can more easily review and accept pull request contributions.

• gofmt
• golangci-lint
• any TF provider pre-commit plugins that would be useful

version = "0.0.0-alpha5"

When attempting to create a variable with the following config, i receive the below error:

resource "prefect_variable" "example" {
  name  = "my_variable"
  value = "variable value goes here"
}

│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to prefect_variable.example, provider "provider[\"registry.terraform.io/prefecthq/prefect\"]" produced an unexpected new value:
│ .tags: was null, but now cty.ListValEmpty(cty.String).
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

The variable is created but when I run a subsequent plan/apply, I see this proposed change which results in the same error as listed above

  # prefect_variable.example is tainted, so must be replaced
-/+ resource "prefect_variable" "example" {
      ~ created = "2023-11-07T19:42:38Z" -> (known after apply)
      ~ id      = "afca6fee-8ff3-49e4-80bb-fd363902095f" -> (known after apply)
        name    = "my_variable"
      - tags    = [] -> null
      ~ updated = "2023-11-07T19:42:38Z" -> (known after apply)
        # (1 unchanged attribute hidden)

Note:

• we do not expose a create endpoint for account_memberships -- these are handled upon creation of the actors themselves (bots / users)
• However, we could still create a Terraform resource, but this would require that practitioners always import the resource via terraform import prefect_account_membership
• Another way to handle this could be that upon Create of the resource, we query for account_memberships based on the email attribute, and then hit the PATCH endpoint with the fetched membership ID. Update wouldn't be a problem, as we could just use the membership ID in state

This issue will be deprioritized for the initial provider launch

Each resource and data source currently inherits the provider-scoped account/workspace ID. We should allow these to be overridden at a per-resource or per-data source level, so that a provider can manage multiple workspaces in an account.

In most cases, we expect that users will have a single account and multiple workspaces, but it is possible to manage multiple workspaces and multiple accounts in a single Terraform module.

Update:
Let's patch the PATCH /bots endpoint to allow modifying the account_role_id. We'll abstract this behavior in the Terraform code....or just do it directly on the existing resource

depends on #78

the PAT endpoint doesn't accept changing the account_role_id, meaning we can't actually update whatever we set upon create. rather than leave this door open, we can rely on the default value that is set when null is passed on create (which is Member), and then use the newly created prefect_account_membership or prefect_account_access resource from #78 and use that as the binder

I've defined my resource as follows:

resource "prefect_workspace_role" "jamie_test" {
  name = "jamie-test"
  description = "hi"
  scopes = [
    "manage_blocks"
  ]
}

I then subsequently ran:

 terraform import prefect_workspace_role.jamie_test a94f185e-c416-4667-a60a-68109be63f53
prefect_workspace_role.jamie_test: Importing from ID "a94f185e-c416-4667-a60a-68109be63f53"...
prefect_workspace_role.jamie_test: Import prepared!
  Prepared prefect_workspace_role for import
prefect_workspace_role.jamie_test: Refreshing state... [id=a94f185e-c416-4667-a60a-68109be63f53]

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

but when I run a terraform plan without changing my defined resource, I receive this proposed change:

prefect_workspace_role.jamie_test: Refreshing state... [id=a94f185e-c416-4667-a60a-68109be63f53]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # prefect_workspace_role.jamie_test will be updated in-place
  ~ resource "prefect_workspace_role" "jamie_test" {
      ~ account_id  = "9a67b081-4f14-4035-b000-1f715f46231b" -> (known after apply)
      ~ created     = "2023-11-07T20:06:48Z" -> (known after apply)
        id          = "a94f185e-c416-4667-a60a-68109be63f53"
        name        = "jamie-test"
      + scopes      = [
          + "manage_blocks",
        ]
      ~ updated     = "2023-11-07T20:15:45Z" -> (known after apply)
        # (1 unchanged attribute hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

In order to make updates to the provider over time, we'll want to automate things using our OpenAPI spec to generate resources.

data "prefect_workspace" "production" {
  handle = "production"
}

prefect workspace names are unique (per account)

The Terraform plugin framework provides its own logging interface, and we can use this to propagate diagnostic information to Terraform.

https://developer.hashicorp.com/terraform/tutorials/providers-plugin-framework/providers-plugin-framework-logging

It would be convenient to be able to manage these resources like Automations and Service Accounts via the Terraform provider. Managing them via code as opposed to the UI (or even the API) would be nice because these kinds of resources are often:

1. Relatively static
2. Configured very similarly from one to the next, and so would lend themselves to being built out via custom Terraform modules

In the case of Service Accounts, these objects also generate sensitive output that needs to be pushed to other secrets stores like AWS Secrets Manager, for example. Making changes to these resources (e.g. refreshing API tokens) and having the changes instantly propagated to your secrets vault/manager of choice would remove some of the tedium of having to do this kind of stuff manually.

Instructions for publishing are in HashiCorp's documentation: https://developer.hashicorp.com/terraform/registry/providers/publishing#publishing-to-the-registry

Depends on #12

something in the configuration overrides is broken - it doesn't pick up on the api key from the env. looks like !config.APIKey.IsUnknown() is always true

	var apiKey string
	if !config.APIKey.IsUnknown() {
		apiKey = config.APIKey.ValueString()
	} else if key, ok := os.LookupEnv("PREFECT_API_KEY"); ok {
		apiKey = key
	}

allows us to query the Prefect Cloud API during acceptance tests, to ensure that the API resource is also created + created correctly

https://developer.hashicorp.com/terraform/plugin/testing/testing-patterns#basic-test-to-verify-attributes

Create a resource and data source for managing organizations (accounts).

resource "prefect_account" "foo" {
    name = "My test account"
    handle = "foo"
}

data "prefect_account" "foo" {
    id = "6afbd2d8-48c5-4adf-969b-f2b42dd6acc8"
}

to be read down + assigned specific workspace or account accesses via role attachment

Create a build pipeline that builds the provider: https://developer.hashicorp.com/terraform/registry/providers/publishing#creating-a-github-release

currently set up to query by name, but the read method expects to query by ID

Hello.

Im trying to run the terraform on the examples folder, but Im getting the following error message: "Could not create work pool, unexpected error: status code 403 Forbidden".

My provider initialization looks like this:

provider "prefect" {
endpoint = "https://api.prefect.cloud/api"
api_key = "{{api-key}}"
account_id = "{{account-id}}"
workspace_id="{{workspace-id}}"
}

And using prefect, I'm able to create a work pool resource by sending a post request to https://api.prefect.cloud/api/accounts/{{api-key}}/workspaces/{{workspace-id}}/work_pools/ with the {{api-key}} as a bearer token, so everything seems fine on the backend...

Any thoughts on how to troubleshoot this?

Best regards,
Carlos Jourdan

Users need a way to:

• Define the account (organization) that we are operating on, at the provider level
• Define the workspace that we are operating on, at the provider level

We expect that the most common use cases for this provider are:

• To manage service account keys corresponding to an account
• To manage resources in one or more workspaces (work pools, variables, etc)

It would also be convenient to allow people to override the provider-level defaults, so that we can support more comprehensive end-to-end scenarios, such as a configuration that creates a workspace and provisions other resources into that workspace (such as variables or work pools). We can also reuse this capability in automated tests, since we can set up and tear down a temporary workspace during our build.

The Terraform plugin framework supports generation of documentation based on descriptions defined in our code: https://developer.hashicorp.com/terraform/tutorials/providers-plugin-framework/providers-plugin-framework-documentation-generation

resource "prefect_work_pool" "kubernetes" {
  name              = "test-k8s-work-pool"
  type              = "Kubernetes"
  workspace_id      = prefect_workspace.internal["integration-tests"].id
  base_job_template = file("./job-template/base.json")
}

however, the update call is happening correctly, as I see the job template update correctly in the UI

• accounts
• workspaces
• variables
• works pools

Instead, we should specify that the practitioner should confirm that a workspaceID is being passed before considering that this is a bug on the provider side

Create a resource and data source for managing workspace-scoped roles:

resource "prefect_workspace_role" "foo" {
}

data "prefect_workspace_role" "foo" {
}

it's missing the api prefix

Create a resource and data source for managing service accounts.

resource "prefect_service_account" "foo" {
}

data "prefect_service_account" "foo" {
}

will unify workspace access operations for users, service accounts, and eventually teams

no datasource or resource import needed, as the primary write endpoint is an Upsert -- so, if a practitioner pulls in an existing service account and wants to use its workspace access, they can just define it in TF and it will overwrite any existing access links

Hello!

I'd love to include some Terraform code to create Infrastructure and Storage blocks in my Prefect v2 Cloud so both AWS cloud infrastructure and Prefect components remain tightly coupled.

Are there plans to update this provider and cover your cloud v2 APIs?

Our objects use a standard UUID, and implementing a custom type would make these easier to work with. We should apply validation and return a uuid.UUID (via the github.com/google/uuid module)

there is a lot of outdated information

also, making sure all structs / methods are godoc-ed

locally:

│ An unexpected error was encountered while verifying an attribute value matched its expected type to prevent unexpected behavior or panics. This is always an error in the provider. Please
│ report the following to the provider developer:
│
│ Expected framework type from provider logic: types.ListType[types.ObjectType["account_id":UUIDType, "base_job_template":basetypes.StringType, "concurrency_limit":basetypes.Int64Type,
│ "created":TimestampType, "default_queue_id":UUIDType, "description":basetypes.StringType, "id":UUIDType, "name":basetypes.StringType, "paused":basetypes.BoolType,
│ "type":basetypes.StringType, "updated":TimestampType, "workspace_id":UUIDType]] / underlying type: tftypes.List[tftypes.Object["account_id":tftypes.String,
│ "base_job_template":tftypes.String, "concurrency_limit":tftypes.Number, "created":tftypes.String, "default_queue_id":tftypes.String, "description":tftypes.String, "id":tftypes.String,
│ "name":tftypes.String, "paused":tftypes.Bool, "type":tftypes.String, "updated":tftypes.String, "workspace_id":tftypes.String]]
│ Received framework type from provider logic: types.ListType[types.ObjectType["base_job_template":basetypes.StringType, "concurrency_limit":basetypes.Int64Type, "created":TimestampType,
│ "default_queue_id":UUIDType, "description":basetypes.StringType, "id":UUIDType, "name":basetypes.StringType, "paused":basetypes.BoolType, "type":basetypes.StringType,
│ "updated":TimestampType]] / underlying type: tftypes.List[tftypes.Object["base_job_template":tftypes.String, "concurrency_limit":tftypes.Number, "created":tftypes.String,
│ "default_queue_id":tftypes.String, "description":tftypes.String, "id":tftypes.String, "name":tftypes.String, "paused":tftypes.Bool, "type":tftypes.String, "updated":tftypes.String]]
│ Path: work_pools

remote:

Error: Invalid Object Attribute Type
with data.prefect_work_pools.all
on outputs.tf line 57, in data "prefect_work_pools" "all":
data "prefect_work_pools" "all" {
While creating a Object value, an invalid attribute value was detected. A Object must use a matching attribute type for the value. This is always an issue with the provider and should be reported to the provider developers.

Object Attribute Name (default_queue_id) Expected Type: UUIDType
Object Attribute Name (default_queue_id) Given Type: basetypes.StringType

When creating a new work pool, we can pull a default base job template from the collections/views/aggregate-worker-metadata endpoint. This needs to be exposed as a data source to simplify the creation of work pools, or we need to change our server API to apply this as a default when creating the pool, if the base job template is not specified.

Create a resource and data source for managing account-scoped roles:

resource "prefect_account_role" "foo" {
}

data "prefect_account_role" "foo" {
}

The Go Client SDK included in this provider is not Terraform-specific, and we can refactor it into a separate Go module for use in other applications. We could then also simplify much of this code by using code generation, related to #13

cannot import by handle, as the Read() method only expects a workspace ID

$ PREFECT_API_KEY=<key> terraform import prefect_workspace.integration_tests name/integration-tests
╷
│ Error: Error parsing Workspace ID
│
│ Could not parse workspace ID to UUID, unexpected error: invalid UUID length: 0
╵

Our timestamps are RFC3339 timestamps, and implementing a custom type would make these easier to work with. We should apply validation and return a time.Time

When attempting to create a workspace role with the following configuration, I receive an error indicating that a description is a required parameter for the request, but the documentation lists it as optional (also in the UI it is optional)

resource "prefect_workspace_role" "example" {
  name = "Custom Workspace Role"
  scopes = [
    "manage_blocks",
    "see_flows"
  ]
}

│ Error: Error creating Workspace Role
│ 
│   with prefect_workspace_role.example,
│   on main.tf line 18, in resource "prefect_workspace_role" "example":
│   18: resource "prefect_workspace_role" "example" {
│ 
│ Could not create Workspace Role, unexpected error: status code 422 Unprocessable Entity, error={"exception_message":"Invalid request
│ received.","exception_detail":[{"loc":["body","description"],"msg":"none is not an allowed
│ value","type":"type_error.none.not_allowed"}],"request_body":{"name":"Custom Workspace
│ Role","description":null,"scopes":["manage_blocks","see_flows"],"inherited_role_id":null}}

Create a resource and data source for managing workspaces.

resource "prefect_workspace" "foo" {
    name = "My test workspace"
}

data "prefect_workspace" "foo" {
    id = "6afbd2d8-48c5-4adf-969b-f2b42dd6acc8"
}