All tables contain a start_line but there's no end_line. Also, the source of the block should also be included

References
Add any related links that will help us understand the resource, including vendor documentation, related GitHub issues, and Go SDK documentation.

Is your feature request related to a problem? Please describe.
I'd like to query resources with states as close to the deployed resources

Describe the solution you'd like
Parsing state files to get resource information would be helpful

Describe alternatives you've considered
Look at plan files or base TF config files

Additional context
N/A

Is your feature request related to a problem? Please describe.
When viewing config/terraform.spc (or the copy that gets created locally when installing the plugin), it's not clear I can insert GitHub or S3 URLs.

Describe the solution you'd like
It should include a brief mention and link to the Hub site (https://hub.steampipe.io/plugins/turbot/terraform#supported-path-formats).

Describe alternatives you've considered
Viewing https://hub.steampipe.io/plugins/turbot/terraform#supported-path-formats.

Additional context
Add any other context or screenshots about the feature request here.

Examples for secret scanning would be great, but there's currently a bug that prevents it from working (turbot/steampipe-postgres-fdw#347). Once this bug is fixed, or a workaround is made available, these examples can be re-added.

Is your feature request related to a problem? Please describe.
When running checks in the Terraform AWS Compliance mod, if I have resources from TF config files, plan files, and state files, checks for TF config and plan files work OK since the controls check in arguments, but state files have resource information in instances -> attributes.

Describe the solution you'd like
We could add a new column attributes that merges results from the arguments and instances -> attributes columns. This would allow users and mods to use a common column to check resource state.

I'm not sure if arguments and attribute property names are always the same and if all arguments are exported as attributes (I think this is the case).

Also, I'm not sure if instances -> attributes contains only the arguments passed in the resource's arguments, or if it contains all attributes.

Describe alternatives you've considered
Checking both columns in all controls.

Additional context
Add any other context or screenshots about the feature request here.

Describe the bug
The arguments column seems to be always null when querying a terraform module.

Steampipe version (steampipe -v)
v0.19.5

Plugin version (steampipe plugin list)

hub.steampipe.io/plugins/turbot/terraform@latest           | 0.5.0   | terraform

To reproduce

SELECT name, split_part(path, '/', -1), arguments FROM terraform_module ORDER BY name;

in a directory with your terraform code (assuming your paths are configured for that). For example, here's a snippet from my working directory:

+------------------------------------+------------------------------------+-----------+
| name                               | split_part                         | arguments |
+------------------------------------+------------------------------------+-----------+
| accelerate-demo                    | mongodbatlas--cluster.tf           | <null>    |
| ansible_playbook_bucket            | aws-schlage-wss.tf                 | <null>    |
| ansible_playbook_bucket_logs       | aws-schlage-wss.tf                 | <null>    |
| assets_auth_lambda_edge            | main.tf                            | <null>    |
| assets_auth_req_forwarder_lambda   | main.tf                            | <null>    |
| cdn                                | main.tf                            | <null>    |
| chained_descriptors                | descriptors.tf                     | <null>    |

This is what the assets_auth_lambda_edge module looks like in the terraform code:

module "assets_auth_lambda_edge" {
  source           = "../../../modules/lambda"
  name             = format("%s-%s", var.environment, "assets-auth-lambda-edge")
  s3_bucket_name   = var.aws_lambdas_s3_bucket_name
  s3_bucket_key    = format("%s/%s", var.environment, "assets-auth-lambda-edge.zip")
  publish          = true
  role_name_prefix = "assets-auth-lambda-edge-role-"
  owner            = var.owner
  project          = var.project
  environment      = var.environment
}

Expected behavior
The arguments column should contain JSONB data of the key => value mappings of argument keys and values, much like the terraform_resource table.

In the example above, the arguments column should look like

{"source": "../../../modules/lambda", "owner": "var.owner", }

etc.

Additional context
From my very quick and cursory glance at the source code, it seems Arguments is not defined in the terraformModule struct nor built up in the buildModule func, but I didn't dig in very deep.

Describe the bug
I installed the Terraform plugin, went to a blank directory, and then ran select * from terraform_resource and received the error Error: failed to get directory specified by the source tfplan.json: relative paths require a module with a pwd (SQLSTATE HV000)

Steampipe version (steampipe -v)
v0.20.9

Plugin version (steampipe plugin list)
v0.8.0

To reproduce
See above

Expected behavior
No rows should be returned

Additional context
Add any other context about the problem here.

Describe the bug
When querying any column from terraform_local, sometimes a conversion error is returned:

Error: Failed to convert value map[_kics__default:{16 []} _kics_array:{21 [map[_kics__default:{21 []}] map[_kics__default:{21 []}] map[_kics__default:{0 [map[_kics__default:{21 []}] map[_kics__default:{0 [map[_kics__default:{21 []}]]}]]}]]} _kics_faz:{17 []} _kics_faz1:{18 []} _kics_myInt:{19 []} _kics_myNum:{20 []} _kics_simple_array:{22 [map[_kics__default:{22 []}]]}] due to unknown type: map[string]model.LineObject (SQLSTATE HV000)

Steampipe version (steampipe -v)
v0.11.2

Plugin version (steampipe plugin list)
v0.0.2

To reproduce
Create a TF file with at least one locals block and run select * from terraform_local (may need to run multiple times, as this only occurs sometimes).

Expected behavior
Queries should not return an error

Additional context
Add any other context about the problem here.

Terraform users are used to the idea of running TF commands in the current working directory for their files. It's inconvenient and unintuitive that we require the paths to be configured first. Can we discover files in the current working directory by default?

The obvious snag is operation when in service mode - we'd need to think about that a bit more carefully as part of this.

Below results should have the type inside the arguments json

> select name,jsonb_pretty(arguments) as arguments from terraform_resource where name = 'aws_iam_regional_access_analyzer_source'
+-----------------------------------------+-------------------------------------------------------------------------------------------------------------->
| name                                    | arguments                                                                                                    >
+-----------------------------------------+-------------------------------------------------------------------------------------------------------------->
| aws_iam_regional_access_analyzer_source | {                                                                                                            >
|                                         |     "note": "AWS CIS v3.0.0 - Controls: 1.20",                                                               >
|                                         |     "value": "resource \"aws_accessanalyzer_analyzer\" \"cis_access_analyzer\" {\n  analyzer_name = \"access_>
|                                         |     "resource": "${turbot_smart_folder.aws_cis_v300_s1_iam.id}"                                              >
|                                         | }                                                                                                            >
+-----------------------------------------+-------------------------------------------------------------------------------------------------------------->

resource "turbot_policy_setting" "aws_iam_regional_access_analyzer_source" {
  resource = turbot_smart_folder.aws_cis_v300_s1_iam.id
  type     = "tmod:@turbot/aws#/policy/types/regionStackSource"
  note     = "AWS CIS v3.0.0 - Controls: 1.20"
  value    = <<-EOT
    resource "aws_accessanalyzer_analyzer" "cis_access_analyzer" {
      analyzer_name = "access_analyzer"
      type          = "ACCOUNT"
    }
    EOT
}

I'd like to query terraform variables using Steampipe!

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Is your feature request related to a problem? Please describe.
The paths configuration can include GitHub repository URLs as described in the documentation. However, the URLs will not be accessible if the repositories are private due to the call to GitHub being unauthenticated.

Describe the solution you'd like
Implement the optional use of GitHub PAT tokens akin to how it is performed in the Steampipe GitHub plugin. Either an explicit github_token config declaration or read from ENV VARs is sufficient for this use-case.

Describe alternatives you've considered
Not sure if it is a good solution, but consider using the authenticated session provided by GitHub cli aka. gh to authenticate when using the plugin. Or integrate with Steampipe GitHub plugin, but this I bet is less appealing.

Additional context
I want to access organizational private repositories, and I am not sure if such repositories differ from normal private repositories.

Describe the bug
A clear and concise description of what the bug is.

This is in the case when the Terraform file is referring to a data source, e.g. defining the policies in the aws_iam_policy_document, and the same is referred to in provisioning the aws_ecr_repository_policy.

Please look at the Note section below for other observations.

Control hangs

➜ steampipe check control.ecr_repository_policy_prohibit_public_access
⠴ Running 1 control. (0 complete, 1 running, 0 pending, 0 errors)
⠙ Running 1 control. (0 complete, 1 running, 0 pending, 0 errors)
⠴ Running 1 control. (0 complete, 1 running, 0 pending, 0 errors)
⠴ Running 1 control. (0 complete, 1 running, 0 pending, 0 errors)
CTRL + C
+ ECR repository policy should prohibit public access .......... 1 / 1 [          ]
  
  ERROR: execution cancelled

Steampipe version (steampipe -v)
Example: Steampipe v0.20.10

Plugin version (steampipe plugin list)
Example: 0.7.0 | terraform

To reproduce
Steps to reproduce the behaviour (please include relevant code and/or commands).

Create a TF file with the below.

### This works as no data source is provided to associate the policy (inbuilt)

resource "aws_ecr_repository_policy" "foobarworks" {
  repository = "foobar"

  policy = jsonencode(
    {
      Version = "2008-10-17",
      Statement = [
        {
          Effect    = "Allow",
          Principal = "*",
          Action = [
            "ecr:BatchGetImage",
            "ecr:BatchCheckLayerAvailability",
            "ecr:DescribeImages",
            "ecr:DescribeRepositories",
            "ecr:GetDownloadUrlForLayer",
            "ecr:ListImages"
          ],
          Condition = {
            "StringEquals" = {
              "aws:PrincipalOrgID" = "something"
            }
          }
        }
      ]
    }
  )
}

### CHECKING POLICY AS DATA INPUT (Fails when referring to the data source)
resource "aws_ecr_repository" "foo" {
  name = "bar"
}

data "aws_iam_policy_document" "foopolicy" {
  statement {
    sid    = "new policy"
    effect = "Allow"

    principals {
      type        = "AWS"
      identifiers = ["123456789012"]
    }

    actions = [
      "ecr:GetDownloadUrlForLayer",
      "ecr:BatchGetImage",
      "ecr:BatchCheckLayerAvailability",
      "ecr:PutImage",
      "ecr:InitiateLayerUpload",
      "ecr:UploadLayerPart",
      "ecr:CompleteLayerUpload",
      "ecr:DescribeRepositories",
      "ecr:GetRepositoryPolicy",
      "ecr:ListImages",
      "ecr:DeleteRepository",
      "ecr:BatchDeleteImage",
      "ecr:SetRepositoryPolicy",
      "ecr:DeleteRepositoryPolicy",
    ]
  }
}

resource "aws_ecr_repository_policy" "foopolicy" {
  repository = aws_ecr_repository.foo.name
  policy     = data.aws_iam_policy_document.foopolicy.json
}

The query to execute

with policy_statement as (
select
  distinct (type || ' ' || name ) as name
from
  terraform_resource ,
  jsonb_array_elements(
    case when ((arguments ->> 'policy') = '')
      then null
      else ((arguments ->> 'policy')::jsonb -> 'Statement') end
) as s
where
  type = 'aws_ecr_repository_policy'
  and (
    (s ->> 'Principal' = '*')
    and ((s ->> 'Condition') is null)
   )
)
select
  type || ' ' || r.name as resource,
  case
    when (arguments ->> 'policy') = ''  then 'ok'
    when s.name is null then 'ok'
    else 'alarm'
  end status,
  case
     when (arguments ->> 'policy') = '' then ' no policy defined'
    when s.name is null then ' not public'
    else ' public'
  end || '.' reason
from
  terraform_resource as r
  left join policy_statement as s on s.name = concat(r.type || ' ' || r.name)
where
  type = 'aws_ecr_repository_policy'

Note-

1. Interesting thing is when I execute .cache off & .cache clear, the query renders as expected
2. For the first time, the query works (not the control), but when we re-run it, it hangs.
3. Now same way if the control is hanging for this case, it halts the execution of entire controls across the services
4. Control & query with CTE block mostly hangs
5. It may be very much related to cache, as it works when we execute the query with .cache off. The moment the cache is on it hangs

Error from plugin log

2023-08-22 10:39:38.535 UTC [TRACE] steampipe-plugin-terraform.plugin: [TRACE] 1692700778380: IndexItem) SatisfiesRequest: satisfiedColumns true satisfiesLimit true satisfiesQuals true
2023-08-22 10:39:38.535 UTC [TRACE] steampipe-plugin-terraform.plugin: [TRACE] 1692700778380: found pending index item to satisfy columns for_each,depends_on,lifecycle,start_line,source,arguments,count,count_src,path,end_line,name,type,provider,_ctx, limit -1, quals: NONE (terraform-1692700778380)
2023-08-22 10:39:38.535 UTC [TRACE] steampipe-plugin-terraform.plugin: [TRACE] 1692700778380: getPendingResultItem returning &{0xc0006de8a0 <nil> terraform-1692700646836 0xc015043100}
2023-08-22 10:39:38.535 UTC [INFO]  steampipe-plugin-terraform.plugin: [INFO]  1692700778380: found pending item [terraform-1692700646836] - subscribing to its data (terraform-1692700778380)
2023-08-22 10:39:38.535 UTC [INFO]  steampipe-plugin-terraform.plugin: [INFO]  1692700778380: stream all data already cached

Expected behaviour
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Currently, the Terraform plugin only scans .tf files and pulls the data into the table format. But the parser can also parse the TF plan files, not just .tf files.

Describe the solution you'd like
Update the plugin to support Terraform plan parsing and pulling the data into the existing tables.

Is your feature request related to a problem? Please describe.
When using modules in my TF config files, the plugin does not parse them correctly

Describe the solution you'd like
Terraform plugin should know how to parse module blocks

Describe alternatives you've considered
None

Additional context
Module examples - https://www.terraform.io/language/modules/develop/composition

A number of the terraform stacks I work with are configured with azurerm backend values, which doesn't currently appear to be supported, based on documentation and examples.

I'd like to request support for azurerm-backed terraform be built in; there's already support for s3 at this point, so this seems like a logical step, and equivalent functionality would be a boon for those of us working in azure storage accounts & containers for tfstate management.

Thanks.

Describe the bug
A clear and concise description of what the bug is.
Continuous execution of any query on any TF plugin tables crashes
Error: rpc error: code = Unavailable desc = error reading from server: EOF (SQLSTATE HV000)

Steampipe version (steampipe -v)
Example: v0.12.2

Plugin version (steampipe plugin list)
Example: v0.0.3

To reproduce
Steps to reproduce the behavior (please include relevant code and/or commands).

Run multiple times the below query
> select * from terraform_resource
Error: rpc error: code = Unavailable desc = error reading from server: EOF (SQLSTATE HV000)
Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Given this TF:

variable "allowed_ssh_cidrs" {
  description = "List of CIDRs that can SSH into instances"
  type        = list(string)
  default     = ["10.0.0.1/32"]
}

resource "aws_security_group" "example_sg" {
  name        = "example_sg"
  description = "Example Security Group"

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = var.allowed_ssh_cidrs
  }
}

> select path from terraform_resource
+----------------------------------------------------+
| path                                               |
+----------------------------------------------------+
| /home/jon/terraform-examples/dynamic.tf |
+----------------------------------------------------+

Now:

terraform plan -var 'allowed_ssh_cidrs=["0.0.0.0/0"]' -out=plan.tfplan
terraform show -json plan.tfplan > plan.json

> select path from terraform_resource

Error: runtime error: invalid memory address or nil pointer dereference (SQLSTATE HV000)

+----------------------------------------------------+
| path                                               |
+----------------------------------------------------+
| /home/jon/terraform-examples/dynamic.tf |
+----------------------------------------------------+