The Azure and GCP demos generate a PEM file with a private key that is used to SSH into the VMs. The AWS demos should provide a similar flow for consistency, but the existing flow should also be kept.

Machine that is invoking terraform needs to have nomad binary installed, see:
cloud/shared/terraform/modules/shared-nomad-jobs/jobs.tf

resource "null_resource" "wait_for_nomad_api" {
  provisioner "local-exec" {
    command = "while ! nomad server members > /dev/null 2>&1; do echo 'waiting for nomad api...'; sleep 10; done"
    environment = {
      NOMAD_ADDR = var.nomad_addr
    }
  }
}

Should be added to the docs

Hello Team,
I would like to know if we have in premises datacenter and we have vcentre cluster where we have 3 nomad master and 3 worker nodes. the vagrant example works perfect but this is applicable only for a single node cluster where we can define easily. But let just say we have a cluster of the above situation the example does not work due to static configs.
So let me know how we can tackle that.

The horizontal application scaling and horizontal cluster scaling demos run hey as a load generator in the local control machine. It would be nice to be able to run hey as a Nomad dispatch job instead, reducing the number of required tools that need to be installed locally.

Demos require specific tools to be installed locally, and fail if they are missing. We could do a pre-flight check with a local-exec and fail early if a requirement is missing.

I did some (extremely basic) troubleshooting of failed server node scenarios. My test was: shutdown -h now on one of the nomad/consul server nodes. This results in the AWS autoscaler bringing a new node up to replace the shutdown one, but it doesn't join the consul and nomad clusters.

I'm confused why this happened. If the cluster is configured as it is in this demo to auto discover via AWS tags, why didn't the new node rejoin automatically? Should this have happened or is my assumption incorrect?

The Azure and GCP demos run Packer automatically along with Terraform. The AWS should do the same.

When the AWS AMI is created, a snapshot is also generated. We currently only provide a command to delete the AMI, leaving the snapshot behind.

Sample code:

function del_ami {
  AWS_PAGER=""
  IMAGE=$1
  SNAP=$(aws ec2 describe-images --image-id $IMAGE | jq -r '.Images[].BlockDeviceMappings[]| select(.Ebs !=null)|.Ebs.SnapshotId')
  aws ec2 deregister-image --image-id $IMAGE
  aws ec2 delete-snapshot --snapshot-id $SNAP
}

The demos do not currently work on M1/M2 laptops due to the requirement on Vagrant and VirtualBox. It might be possible to use LimaVM as an alternative, but we have experienced configuration challenges with ToxiProxy. This might be resolvable by using Intel based VM's rather than ARM, but needs investigation.

@Juanadelacuesta please feel free to add any notes or thoughts you have based on your experience.

According to the documentation you need TF 0.12.23 but one of the modules actually sets version to ">= 0.13", thus:

Error: Unsupported Terraform Core version

Module hashistack_jobs (from
"../../../shared/terraform/modules/shared-nomad-jobs") does not support
Terraform version 0.12.23.

i want use already exists project but not create a new project.so i make some update.
main...symsimmy:nomad-autoscaler-demos:main
then,it has some error occured.

can u help me fix this issue,how to use existed project rather than create new one?thx very much

The Packer command to build the AMI image uses created_email and created_name, while the Terraform config uses owner_name and owner_email.

Howdy, team! 👋🏼

When I try using the Azure example, I have the following issue with the Consul auto_join and clients_vmss resources:

Error: authorization.RoleAssignmentsClient#Delete: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailed" Message="The client 'xxx' with object id 'xxx' does not have authorization to perform action 'Microsoft.Authorization/roleAssignments/delete' over scope '/subscriptions/yyy' or the scope is invalid. If access was recently granted, please refresh your credentials."

Are there any special permissions or considerations I need to address when working with Azure?

I have set the following ENV vars locally:

• ARM_CLIENT_ID
• ARM_CLIENT_SECRET
• ARM_SUBSCRIPTION_ID
• ARM_TENANT_ID

When trying to exec into an allocation running the Terraform remote demo cluster, I get this error message:

failed to exec into task: Unexpected response code: 400 (Bad Request
failed to upgrade connection: websocket: the client is not using the websocket protocol: 'upgrade' token not found in 'Connection' header)

Can we do a demo on vagrant instead of cloud providers for horizontal cluster autoscaling

The existing Vangrant demo tries to mimic a real-world scenario where a robust monitoring solution is deployed and the scaling policy uses a more realist number of connections.

This is a good scenario, but it includes too many external components that are not necessary to get started with the Nomad Autoscaler.

A minimal demo would use a simple job being scaled using metrics from Nomad itself, so no extra decencies (such as Prometheus and Grafana) would be necessary.

Hello everyone!
I was following this tutorial: https://developer.hashicorp.com/nomad/tutorials/autoscaler/horizontal-cluster-scaling, and got stuck on terraform init step. Once I run terraform init, I get the following errors:

Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/nomad from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/template from the dependency lock file
- Reusing previous version of hashicorp/local from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/null v3.1.0...
- Installed hashicorp/null v3.1.0 (signed by HashiCorp)
- Installing hashicorp/local v2.1.0...
- Installed hashicorp/local v2.1.0 (signed by HashiCorp)
- Installing hashicorp/random v3.1.0...
- Installed hashicorp/random v3.1.0 (signed by HashiCorp)
╷
│ Error: Incompatible provider version
│
│ Provider registry.terraform.io/hashicorp/nomad v1.4.13 does not have a package available for your current platform, darwin_arm64.
│
│ Provider releases are separate from Terraform CLI releases, so not all providers are available for all platforms. Other versions of this provider may have different platforms
│ supported.
╵

╷
│ Error: Incompatible provider version
│
│ Provider registry.terraform.io/hashicorp/template v2.2.0 does not have a package available for your current platform, darwin_arm64.
│
│ Provider releases are separate from Terraform CLI releases, so not all providers are available for all platforms. Other versions of this provider may have different platforms
│ supported.
╵

╷
│ Error: Incompatible provider version
│
│ Provider registry.terraform.io/hashicorp/aws v2.70.0 does not have a package available for your current platform, darwin_arm64.
│
│ Provider releases are separate from Terraform CLI releases, so not all providers are available for all platforms. Other versions of this provider may have different platforms
│ supported.
╵

First I was following the tutorial on #learn branch (as instruted in the tutorial), after that I tried main branch, but the error still persists.

This is my terraform -v output:

Terraform v1.3.7
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v2.70.0
+ provider registry.terraform.io/hashicorp/local v2.1.0
+ provider registry.terraform.io/hashicorp/nomad v1.4.13
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/template v2.2.0

Thanks, and have a nice day!

The Vagrant and Terraform demos of the Nomad Autoscaler provide a quick way to deploy a sample scenario and see the Autoscaler in action.

But the documentation fails to properly explain all the components involved, how they interact with each other and how the Autoscaler makes use of them.

The documentation should answer questions such as:

1. What's the scenario being showcased.
2. Which components are being deployed.
3. How each component contributes to the scenario.
4. Which components are strictly necessary for the Autoscaler to function.
5. What will the policy do and how.

I'm using AWS. First time I terraform applied, it waited on this step for >10 minutes before I stopped it:

module.hashistack_jobs.null_resource.wait_for_nomad_api: Still creating... [10m40s elapsed]

Second run it seems to get there pretty quickly - within 2 minutes - and now it's just waiting again.

This is a placeholder item for work that needs to be done for Nomad 1.1.0