GoCD server plugin for bringing up Amazon EC2 instances as its agents on demand.
Compatible with version 5.0 of the elastic agent endpoint (GoCD server versions starting from 19.3.0).

• Installation
  • Amazon Machine Image
  • IAM Instance Profile
  • Security Groups
  • Subnets
  • AWS Authentication
• Building the code base
• Credits
• Disclaimer

Copy the file build/libs/gocd-ec2-elastic-agent-plugin-VERSION.jar to the GoCD server under ${GO_SERVER_DIR}/plugins/external and restart the server.

Prepare AMI, security groups and subnets for the agents.

Tested on GoCD server & agent versions 20.6.0.

This is the most important step, where you will prepare a base image for the agents. Create new clean EC2 instance and install there all the tools and configurations that your agents may need. This plugin in intended to be used with Amazon Linux 2 based agents, but you can also adapt it to run with other operating systems like Ubuntu.
After that, follow up the official guide to install Go-Agent. Do not connect it to the server yet, nor enable auto startup of go-agent.service! All this will be done by the plugin itself with the help of the user data scripts. Before stopping this instance perform cleanup with the following commands:

rm -rf /var/lib/cloud/*
rm -rf /var/log/cloud-init*
rm -rf /usr/share/go-agent/go-agent.pid
rm -rf /usr/share/go-agent/config/*
rm -rf /var/log/go-agent/*
rm -rf /var/lib/go-agent/config/*

Finally, create new Amazon Machine Image from your instance. Each Elastic Agent Profile can use different AMI to suit your needs.

AWS allows you to create an IAM Instance Profile which assigns an AWS IAM role to the EC2 instance(s), which can be used to authorize the instances to invoke the AWS API. Provide the IAM Instance profile name (not the Role name, or the ARN) to launch instances with the corresponding role associated with them. There is no validation of the IAM Instance Profile and providing an invalid profile name will result in instances not being deployed.

Note: in order to be able to assign instance profiles to your agents remember to assign proper policies to your GoCD server.

You will need to setup some connectivity between your GoCD server and elastic agents. You may also want to allow any other inbound/outbound traffic to the agents, like tunnels, version control systems or repositories. The most straightforward way to achieve this is with security groups.

Once you have all your security groups defined, put their identifiers into the Elastic Agent Profile and they will be automatically assigned to every newly created EC2 instance.

To be able to launch new agents you need to have at least 2 subnets in your VPC where you will put your newly created instances. You can define even more subnets (ideally in different availability zones) in the elastic agent profile and the plugin will choose randomly one of them each time it has to create new instance. If the chosen availability zone has run out of your requested instance type, the plugin will try to bring up instance in the next subnet.

Also, remember to enable auto-assign public IP address to the subnets.

If an AWS Access Key ID and AWS Secret Access Key is provided for a cluster profile, then those credentials will be used. Otherwise, if left blank, the Default Provider Credential Chain will be used. i.e. The default provider credentials will be resolved from the GoCD server environment (e.g. ~/.credentials file or Ec2 IAM Instance profiles from instance metadata).

To build the jar, run ./gradlew clean assemble

This project is fully based on GoCD Elastic agent plugin skeleton and GoCD Elastic agent plugin for Docker. The structure and some parts of code are taken directly from these projects.

The GoCD Elastic agent plugin for AWS EC2 is supplied "AS IS", use is at your own risk. Author and contributors expressly disclaim all warranties nor support of any kind.