Comments (9)
zli82016
commented on June 18, 2024
Hello, @richardmoe , can you please provide more information to help reproduce the issue? Thanks.
- The version of Terraform Google provider
- The complete configuration before and after updating
- The DEBUG logs when running the command
TF_LOG=DEBUG terraform apply. The current logs don't have so much information.
from terraform-provider-google.
richardmoe
commented on June 18, 2024
- The provider version is listed above: registry.terraform.io/hashicorp/google v5.25.0, but also tested with v5.26.0
- TF config is here: https://gist.github.com/richardmoe/5d232059fa74bf793d28c9050487ffeb. The only change is
image = "gke-on-prem-ubuntu-1.28.300-gke.123" -> "gke-on-prem-ubuntu-1.28.400-gke.75" - TF LOG is here. Only WARN level, since DEBUG contains secrets: https://gist.github.com/richardmoe/5428ff722283f25a0d08dc92156d0a9e
from terraform-provider-google.
zli82016
commented on June 18, 2024
Forwarding to the service team to check the error
│ Error: Error updating VmwareCluster "projects/***": googleapi: Error 400: Proto is invalid: [version_update: control plane upgrade disallowed with cluster config updates]
from terraform-provider-google.
NickElliot
commented on June 18, 2024
hi @richardmoe, we're having a hard time reproducing this. Would it be possible for you to share a scrubbed version of the PATCH request within the debug logs?
from terraform-provider-google.
naitianliu-google
commented on June 18, 2024
Hi @richardmoe, the error returned means it is not allowed the modify other fields such as image as mentioned when modifying the on_prem_version field. Could you please help confirm if only on_prem_version is modified to perform the upgrade operation?
from terraform-provider-google.
richardmoe
commented on June 18, 2024
Here is the relevant PATCH request @NickElliot
2024-05-08T11:48:30.603+0200 [DEBUG] module.gkee-plattform-lab-onprem.google_gkeonprem_vmware_cluster.gkee_onprem_cluster: applying the planned Update change
2024-05-08T11:48:30.611+0200 [DEBUG] provider.terraform-provider-google_v5.28.0_x5: 2024/05/08 11:48:30 [DEBUG] Updating VmwareCluster "projects/nrk-platt-gke-enterprise-test/locations/europe-west3/vmwareClusters/gkee-plattform-lab": map[string]interface {}{"annotations":map[string]string{"onprem.cluster.gke.io/block-docker-node-pools":"true", "onprem.cluster.gke.io/error-reason":"", "onprem.cluster.gke.io/gke-on-prem-last-update-start-time":"2024-04-29T10:29:09Z", "onprem.cluster.gke.io/operation":"upgrade", "onprem.cluster.gke.io/operation-id":"f8bcfef4-b117-4e22-8e2f-c9e2e448d614", "onprem.cluster.gke.io/source":"api-vmware", "onprem.cluster.gke.io/start-time":"2024-04-29T11:26:54Z", "onprem.cluster.gke.io/status":"started", "onprem.cluster.gke.io/user-cluster-resource-link":"//gkeonprem.googleapis.com/projects/464094393805/locations/europe-west3/vmwareClusters/gkee-plattform-lab"}, "antiAffinityGroups":map[string]interface {}{}, "authorization":map[string]interface {}{"adminUsers":[]interface {}{map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}, map[string]interface {}{"username":"[email protected]"}}}, "autoRepairConfig":map[string]interface {}{"enabled":true}, "controlPlaneNode":map[string]interface {}{"autoResizeConfig":map[string]interface {}{"enabled":true}, "cpus":4, "memory":8196, "replicas":3, "vsphereConfig":map[string]interface {}{"datastore":"IPLAB_NFSDATASTORE"}}, "dataplaneV2":map[string]interface {}{"advancedNetworking":true, "dataplaneV2Enabled":true, "windowsDataplaneV2Enabled":true}, "description":"Onprem cluster deployed in VmWare by Terraform", "disableBundledIngress":true, "enableControlPlaneV2":true, "loadBalancer":map[string]interface {}{"metalLbConfig":map[string]interface {}{"addressPools":[]interface {}{map[string]interface {}{"addresses":[]interface {}{"10.252.242.98-10.252.242.126"}, "avoidBuggyIps":true, "pool":"gke-usercluster1-lab-lb-pool"}}}, "vipConfig":map[string]interface {}{"controlPlaneVip":"10.252.242.97"}}, "networkConfig":map[string]interface {}{"controlPlaneV2Config":map[string]interface {}{"controlPlaneIpBlock":map[string]interface {}{"gateway":"10.252.242.1", "ips":[]interface {}{map[string]interface {}{"hostname":"gkee-plattform-lab-control-plane-01", "ip":"10.252.242.5"}, map[string]interface {}{"hostname":"gkee-plattform-lab-control-plane-02", "ip":"10.252.242.6"}, map[string]interface {}{"hostname":"gkee-plattform-lab-control-plane-03", "ip":"10.252.242.7"}}, "netmask":"255.255.255.128"}}, "dhcpIpConfig":map[string]interface {}{"enabled":true}, "hostConfig":map[string]interface {}{"dnsServers":[]interface {}{"10.252.243.2", "10.252.243.3"}, "ntpServers":[]interface {}{"ntp.uio.no"}}, "podAddressCidrBlocks":[]interface {}{"10.124.0.0/16"}, "serviceAddressCidrBlocks":[]interface {}{"192.168.20.0/22"}, "vcenterNetwork":"IP-LAB/gke-usercluster1-lab"}, "onPremVersion":"1.28.400-gke.75", "storage":map[string]interface {}{}, "upgradePolicy":map[string]interface {}{}, "vcenter":map[string]interface {}{"address":"malabvc01.vmware.nrk.cloud", "caCertData":"redacted, "cluster":"IP-LAB", "datacenter":"Marienlyst", "datastore":"IPLAB_NFSDATASTORE", "folder":"/Marienlyst/vm/IP-LAB", "resourcePool":"IP-LAB/Resources"}}
2024-05-08T11:48:30.612+0200 [DEBUG] provider.terraform-provider-google_v5.28.0_x5: PATCH /v1/projects/nrk-platt-gke-enterprise-test/locations/europe-west3/vmwareClusters/gkee-plattform-lab?alt=json&updateMask=onPremVersion%2CantiAffinityGroups%2Cstorage%2CupgradePolicy HTTP/1.1
2024-05-08T11:48:30.612+0200 [DEBUG] provider.terraform-provider-google_v5.28.0_x5: "onprem.cluster.gke.io/source": "api-vmware",
2024-05-08T11:48:30.612+0200 [DEBUG] provider.terraform-provider-google_v5.28.0_x5: "onprem.cluster.gke.io/user-cluster-resource-link": "//gkeonprem.googleapis.com/projects/464094393805/locations/europe-west3/vmwareClusters/gkee-plattform-lab"
2024-05-08T11:48:30.613+0200 [DEBUG] provider.terraform-provider-google_v5.28.0_x5: "address": "malabvc01.vmware.nrk.cloud",
2024-05-08T11:48:31.276+0200 [ERROR] provider.terraform-provider-google_v5.28.0_x5: Response contains error diagnostic: diagnostic_detail="" diagnostic_severity=ERROR tf_proto_version=5.4 tf_provider_addr=registry.terraform.io/hashicorp/google tf_req_id=7ddb4cd7-eeb7-af50-9eb5-2f670c49bab6 @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 @module=sdk.proto tf_resource_type=google_gkeonprem_vmware_cluster tf_rpc=ApplyResourceChange
| Error updating VmwareCluster "projects/nrk-platt-gke-enterprise-test/locations/europe-west3/vmwareClusters/gkee-plattform-lab": googleapi: Error 400: Proto is invalid: [version_update: control plane upgrade disallowed with cluster config updates]
2024-05-08T11:48:31.286+0200 [ERROR] vertex "module.gkee-plattform-lab-onprem.google_gkeonprem_vmware_cluster.gkee_onprem_cluster" error: Error updating VmwareCluster "projects/nrk-platt-gke-enterprise-test/locations/europe-west3/vmwareClusters/gkee-plattform-lab": googleapi: Error 400: Proto is invalid: [version_update: control plane upgrade disallowed with cluster config updates]```
from terraform-provider-google.
NickElliot
commented on June 18, 2024
could you share your terraform show output for the resource youre attempting to update, where specifically i want to know what it is showing as your read state for onPremVersion, antiAffinityGroups{}, storage{}, and upgradePolicy{}
from terraform-provider-google.
richardmoe
commented on June 18, 2024
output of terraform show: https://gist.github.com/richardmoe/a05dbb8f000bffbb5de57f0396a9092c
from terraform-provider-google.
NickElliot
commented on June 18, 2024
@naitianliu-google the cause of this issue is the surrounding objects of boolean fields set to false are being added to the update mask of a request because those objects mentioned in #17933 (comment) are included in the patch request as empty structs. This should be resolvable by adjusting the resource to not add these empty structs to the update mask, or not send the empty structs within each post request at all (if the API would allow for this).
Thanks for the detailed diagnostic info @richardmoe!
from terraform-provider-google.
Related Issues (20)
- Remove default value for google_artifact_registry_repository.remote_repository_config.*.public_repository
- Document how to use CustomizeDiff in MMv1
- Postgres IAM SQL User not able to authenticate when created through terraform only HOT 1
- IP collection field is missing in google_compute_forwarding_rule to support BYOIP IPv6 address
- environment variable name GCP_PROJECT is reserved by the system: it cannot be set by users HOT 5
- API response code 429 "RESOURCE_EXHAUSTED" incorrectly classified as retryable HOT 6
- Make iam_policy.custom_diff_suppress reference a function name instead of a template
- Allow full management of IAM parent fields
- Only add IAM condition fields to IAM resources if the API supports IAM conditions
- GCE instance left without boot disk HOT 1
- Allow specifying a validation function for IAM parent_resource_attribute field
- Unnecessary ForceNew On Container Node Pool / Cluster `gcfs_config` HOT 1
- Add tlsEarlyData field to TargetHttpsProxy.
- SECURITY RISK: Google Compute Regional Security Policy doesn't implement default deny HOT 9
- Error when applying changes to module google_bigquery_dataset_access.dset_access HOT 3
- google_gke_hub_feature: User cannot unset fleet default member config in TF
- Plan changes on every execution even though there are no changes to the code
- BadRequest error when updating google_spanner_instance with labels set at provider level
- Error waiting to create Gateway: Error waiting for Creating Gateway: Error code 13, message: an internal error has occurred
- Project tainted
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from terraform-provider-google.