amer / resinfra Goto Github PK

Issue: All entities created through terraform are assigned a name. These must be unique. If some infra is up and running that has been built with the terraform script, one cannot build additional infrastructure on the same terraform script, without either destroying the existing infra or renaming the entities.

Pot. solution: We should think of setting entity names (of the VM, ssh key, ...) in a more dynamic way (e.g. based on current timestamp).

Investigate why connection Azure <> Hetzner is constantly rekeyed. This means that it is more or less uninterrupted, but a new security association is renegotiated every few seconds. I assume this leads to decreased performance and packet loss, as also mentioned in #49.

when both the cockroach and the nodeexporter playbooks are executed on the same time, they are trying to run apt update at the same time, which will fail.

fatal: [10.3.0.4]: FAILED! => {"cache_update_time": 1611665481, "cache_updated": false, "changed": false, "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"      install 'docker-ce'' failed: E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", "rc": 100, "stderr": "E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", "stderr_lines": ["E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)", "E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?"], "stdout": "", "stdout_lines": []}

Because of changes introduced to the terraform scripts, the tf scripts in the benchmarking code directory are out of date.

Waiting for #45 before fixing.

From the StrongSWAN Wiki:

Break-before-make: This is default behavior of the IKE daemon when reauthenticating an IKEv2 SA. It means that all IKE and IPsec SAs are torn down before recreating them. This will cause some interruptions during which no IPsec SAs are installed.

• Change this, see this section
• Investigate if LibreSWAN does the same.

Cloned VMs on Proxmox are marked as protected and cannot be deleted:

This is not good:

data "hcloud_image" "deployer-snapshot" {
with_selector = "hetzner-benchmark"
most_recent = true
}

needs to be changed back to

data "hcloud_image" "deployer-snapshot" {
with_selector = "hetzner-deployer"
most_recent = true
}

in the main branch

We've got AWS, Proxmox, and Hetzner in terraform/<provider-name>, and Azure in az/terraform. This should be unified. I propose to follow the majority and move az/terraform to terraform/az.

When using Ansible to, for example deploy the gateway application, the script will stop because you need to manually accept the fingerprint. The visibility of that message is not good because the Terraform process will run in parallel and print a lot of other information in the console.

We should come up with a solution for this to reduce the manual interference as much as possible.

Low priority

The CHILD_SA cannot be rekeyed due to a proposal mismatch.

Feb 19 18:31:49 ri-hetzner-gateway-vm-bf581983 charon: 07[CFG] found matching child config "to-gcp" with prio 10
Feb 19 18:31:49 ri-hetzner-gateway-vm-bf581983 charon: 07[CFG] selecting proposal:
Feb 19 18:31:49 ri-hetzner-gateway-vm-bf581983 charon: 07[CFG]   no acceptable DIFFIE_HELLMAN_GROUP found
Feb 19 18:31:49 ri-hetzner-gateway-vm-bf581983 charon: 07[CFG] received proposals: ESP:AES_GCM_16_128/AES_GCM_16_256/AES_GCM_16_192/AES_CBC_128/AES_CBC_256/AES_CBC_192/HMAC_SHA2_256_128/HMAC_SHA2_512_256/HMAC_SHA1_96/MODP_2048/MODP_2048_224/MODP_2048_256/MODP_1536/MODP_3072/MODP_4096/MODP_8192/MODP_1024/MODP_1024_160/NO_EXT_SEQ
Feb 19 18:31:49 ri-hetzner-gateway-vm-bf581983 charon: 07[CFG] configured proposals: ESP:AES_CBC_128/AES_CBC_192/AES_CBC_256/HMAC_SHA2_256_128/HMAC_SHA2_384_192/HMAC_SHA2_512_256/HMAC_SHA1_96/AES_XCBC_96/NO_EXT_SEQ
Feb 19 18:31:49 ri-hetzner-gateway-vm-bf581983 charon: 07[IKE] no acceptable proposal found
Feb 19 18:31:49 ri-hetzner-gateway-vm-bf581983 charon: 07[IKE] failed to establish CHILD_SA, keeping IKE_SA

StrongSWAN wiki says:

Therefore, a proposal mismatch might not immediately be noticed when the SA is established,
but may later cause rekeying to fail.

Setting esp=aes128-sha1-modp1024 in ipsec.conf seems to work. I executed the benchmark with this setting, and it was not terminated because of network errors. However, Grafana still shows that some connections are unstable, briefly interrupted.

• investigate if this is really fixed
• investigate if proxmox is also affected
• investigate why connection Azure <> Hetzner is constantly rekeyed (which means that it is uninterrupted, but a new security association is renegotiated every few seconds. The connection is not interrupted, but I assume that this leads to decreased performance.)

When executing

terraform apply -target=module.hetzner -target=module.gcp -target=module.azure

I get this error message:

Error: Error Creating/Updating AzureRM Virtual Network Gateway "ri-network-gateway" (Resource Group "ri-multi-cloud-rg"): network.VirtualNetworkGatewaysClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="GatewayMustUseDynamicPublicIp" Message="Public IP /subscriptions/e6994910-2d0d-4220-ae62-73c0242d7d4d/resourceGroups/ri-multi-cloud-rg/providers/Microsoft.Network/publicIPAddresses/ri-public-gateway-ip-1b7ff5d0 reference by Virtual Network Gateway /subscriptions/e6994910-2d0d-4220-ae62-73c0242d7d4d/resourceGroups/ri-multi-cloud-rg/providers/Microsoft.Network/virtualNetworkGateways/ri-network-gateway must have PublicIPAllocationMethod as Dynamic." Details=[]

on modules/azure/main.tf line 160, in resource "azurerm_virtual_network_gateway" "main":
160: resource "azurerm_virtual_network_gateway" "main" {

When i try to change the public ip resource manually to dynamic, it will be changed to static on the next run producing the same error.

Changing the allocation_method to "Dynamic" in the azurerm_public_ip.gateway seems to fix this problem. At least for the spin up of the infrastructure.