This is a project to provision a Lustre cluster as quickly as possible. All the Lustre setup scripting is taken from the AzureHPC but the difference in this project is the Lustre cluster is provisioned through an ARM template using a custom image.

This project includes the following:

• A packer script to build an image with the Lustre packages installed.
• An ARM template to deploy a Lustre cluster using the image.

The ARM template performs the installation with cloud init where the installation scripts are embedded. The azuredeploy.json includes the embedded scripts but the repo includes script to create this from the azuredeploy_template.json.

Check out the repository:

git clone https://github.com/Azure/azlustre

Packer is required for the build so download the latest version for your operating system from https://www.packer.io. It is distributed as a single file so just put it somewhere that is in your PATH. Go into the packer directory:

cd azlustre/packer

The following options are required to build:

These can be read by packer from a JSON file. Use this template to create options.json and populate the fields:

{
    "var_subscription_id": "",
    "var_tenant_id": "",
    "var_client_id": "",
    "var_client_secret": "",
    "var_resource_group": "",
    "var_image": "lustre-7.8-lustre-2.13.5"
}

Use the following command to build with packer:

packer build -var-file=options.json centos-7.8-lustre-2.12.5.json

Once this successfully completes the image will be available.

The "Deploy to Azure" button can be used once the image is available (alternatively the CLI can be used with az deployment group create). Below is a description of the parameters:

The additional parameters can be used to enable HSM for the Lustre deployment.

The additional parameters can be used to log metrics for the Lustre deployment.

When creating a Lustre configuration you pay attention to the following:

• The expected network bandwidth for the VM type
• The max uncached disk throughput when using managed disks
• The throughput for the managed disks

This section provides options for three types of setup:

1. Ephemeral This is the cheapest option and uses local disks to the VMs. This can also provide the lowest latency as the physical storage resides on the host. Any VM failure will result in data loss but is a good option for scratch storage.

   Size: 7.6 GB per OSS

   Expected performance: 1600 MB/s per OSS (limited by NIC on VM)

2. Persistent Premium This option uses premium disks attached to the VMs. A VM failing will not result in data loss.

   Size: 6 GB per OSS

   Expected performance: 1152 MB/s per OSS (limited by uncached disk throughput)

3. Persistent Standard This option uses standard disks attached to the VMs. This requires relatively higher storage per OSS since the larger disks are needed in order to maximise the bandwidth to storage for a VM.

   Size: 32 GB per OSS

   Expected performance: 1152 MB/s per OSS (limited by uncached disk throughput)

These are the parameters that can be used when deploying:

This is only required when making changes to the scripts.

The scripts are placed in a self-extracting compressed tar archive and embedded into the ARM template to be executed by cloud-init. The cloud-ci.sh script performs this step and the build.sh executes this with the parameters used for the currently distributed ARM template in the repository.

Note: The makeself tool is required for this step.