[BUG] Terraform apply fails when index is created and configured to use ISM rollover policy about terraform-provider-opensearch HOT 13 OPEN

Hey @sharathganga I assume you are using the latest version of the provider https://registry.terraform.io/providers/opensearch-project/opensearch/2.0.0
Adding @rblcoder @afrodidact @premkirank
Thank you

from terraform-provider-opensearch.

@prudhvigodithi I'm using the provider version 1.0.0.

from terraform-provider-opensearch.

@sharathganga @prudhvigodithi once the rollover happens, the index created using terraform will no longer have is_write_index true, using plan and apply again will throw the more than one write index error. ignore_changes can be used to ignore the changes.
Or update the terraform code with the current write-enabled index.

from terraform-provider-opensearch.

Hey @rblcoder can you please take a look at the provider code to make this change? Please let me know I can assign this bug to you.
Thank you

from terraform-provider-opensearch.

@sharathganga @prudhvigodithi once the rollover happens, the index created using terraform will no longer have is_write_index true, using plan and apply again will throw the more than one write index error. ignore_changes can be used to ignore the changes. Or update the terraform code with the current write-enabled index.

@rblcoder, I've already tried using lifecycle policy with ignore_changes=all on the index resource but that didn't help either. This resource is using for_each argument so it wouldn't be realistic to get the current write enabled index as the indexes are at various states of rolled over index.

from terraform-provider-opensearch.

The following steps work for me -
Post creating the resources through terraform, and a rollover, terraform apply did not throw errors. This is using OpenSearch provider version = "2.0.0" and OpenSearch version 2.11.0

from terraform-provider-opensearch.

@rblcoder , I'm using AWS Opensearch 1.3 and provider version 1.0.0. When I tried using the provider version 2.0.0, it gave me a different error while creating opensearch_index_template resource.

╷
│ Error: elastic: Error 400 (Bad Request): [1:35] [index_template] unknown field [settings] [type=x_content_parse_exception]
│
│   with opensearch_index_template.index_template,
│   on main.tf line 60, in resource "opensearch_index_template" "index_template":
│   60: resource "opensearch_index_template" "index_template" {
│
╵

from terraform-provider-opensearch.

Can you please try with index specified as follows?

resource "opensearch_index" "index" {
  name = "test-logs-000001"
  number_of_shards = 3
  number_of_replicas = 1

  aliases = jsonencode(
    {
      "test-logs" = {
        "is_write_index" = true
      }
    }
  )

from terraform-provider-opensearch.

@rblcoder number_of_shards and number_of_replicas is being managed by the opensearch_index_template resource with rollover alias configured.

from terraform-provider-opensearch.

We've just migrated from phillbaker's project to this project and started facing the same issue. At phillbaker's library, this fix has been implemented for following the write index from ISM/ILM.
Maybe we need the same approach in this project?

from terraform-provider-opensearch.

@malaquf We have specified

Computed: true 

for number_of_replicas
https://github.com/opensearch-project/terraform-provider-opensearch/pull/123/files
The PR above is merged.
This fixed the error thrown when index configuration has no number_of_replicas specified.

from terraform-provider-opensearch.

Interesting, in our case, we do have number_of_replicas specified, and we still face the issue. We are on v2.2.1.

from terraform-provider-opensearch.

@malaquf I created a policy using Terraform code for the following example
https://opensearch.org/docs/latest/im-plugin/ism/policies/#sample-policy-with-ism-template-for-auto-rollover
Next I waited for the rollover to happen.
Applying the Terraform code again did not give errors.

Output of GET /log after rollover

{
  "log-000001": {
    "aliases": {
      "log": {
        "is_write_index": false
      }
    },
    "mappings": {
      "properties": {
        "contractor": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "name": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    },
    "settings": {
      "index": {
        "replication": {
          "type": "DOCUMENT"
        },
        "number_of_shards": "1",
        "plugins": {
          "index_state_management": {
            "rollover_alias": "log"
          }
        },
        "provided_name": "log-000001",
        "creation_date": "1713583424647",
        "number_of_replicas": "1",
        "uuid": "Jgntl4tARBahGdefUh08NQ",
        "version": {
          "created": "136337827"
        }
      }
    }
  },
  "log-000002": {
    "aliases": {
      "log": {
        "is_write_index": true
      }
    },
    "mappings": {},
    "settings": {
      "index": {
        "replication": {
          "type": "DOCUMENT"
        },
        "number_of_shards": "1",
        "plugins": {
          "index_state_management": {
            "rollover_alias": "log"
          }
        },
        "provided_name": "log-000002",
        "creation_date": "1713584187139",
        "number_of_replicas": "1",
        "uuid": "8BB_8wnfQgaRJeb3xzncFw",
        "version": {
          "created": "136337827"
        }
      }
    }
  }
}

from terraform-provider-opensearch.