Is it possible to customize AMI? about aws-parallelcluster HOT 5 CLOSED

Currently, it is only possible to use AMIs that are provided as part of cfncluster. They have all the software pre-installed that is required for cfncluster such as a scheduler and bootstrap scripts. You could build custom AMI's if you install all of the same software, packages and scripts on your AMI.

You can also launch one of the AMI's found in amis.txt, customize that instance and then run /usr/local/sbin/ami_cleanup.sh before making an image.

Once you have a working custom AMI, you can simply specify that in the cfncluster config as custom_ami= setting.

With regards to the template URL, they are versioned. So the current us-east-1 template URL is https://s3.amazonaws.com/cfncluster-us-east-1/templates/cfncluster-0.0.17.cfn.json

from aws-parallelcluster.

Hi Dougal,

Thank you so much for your response! I tried to customize the AMI (us-west-2 ami-2b095a1b) from amis.txt. I found it is different from the AMI (Amazon Linux AMI 2014.09.1 (HVM) - ami-b5a7ea85). When I tried to install R and got the below error message,

sudo yum install R
Loaded plugins: fastestmirror, security
Setting up Install Process
Loading mirror speeds from cached hostfile

• base: mirrors.xmission.com
• extras: mirror.spro.net
• updates: linux.mirrors.es.net
  No package R available.
  Error: Nothing to do

Then I tried to install epel using the below command,
su -c 'rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm'

But it was still failed. I am wondering if you can provide an example to install a scheduler and bootstrap scripts on the Amazon Linux AMI 2014.09.1 (HVM) - ami-b5a7ea85 then I can launch a cluster with cnfcluster. Or could you tell me how to install R on your customized ami (us-west-2 ami-2b095a1b)?

Thanks a lot!

from aws-parallelcluster.

cfncluster AMIs are based on CentOS 6 and track errata with each update.

The easiest way to customize a cluster deployed with cfncluster, is to utilize pre/post install actions. I have written some documentation that describes this and provides an example of adding R to a cluster.

You can find the doc here - https://github.com/awslabs/cfncluster/blob/develop/docs/staging/pre_post-install-actions.txt

from aws-parallelcluster.

Hi, I'm having same issue configuring ubuntu os AMI and launch. I'm wondering what could be wrong? Thanks

from aws-parallelcluster.

@cgibioinformatics May I understand why you want to customized AMI ?

Suggestion:

1. Please look at pre_install/post_install option of cfncluster instead of creating custom AMI
2. If you still want to create AMI, Please use packer command to create ami (E.g. for ubuntu14 https://github.com/awslabs/cfncluster-cookbook/blob/develop/packer_ubuntu1404.json)

from aws-parallelcluster.