Git Product home page Git Product logo

meltwater / terraform-aws-asg-dns-handler Goto Github PK

View Code? Open in Web Editor NEW
74.0 11.0 45.0 92 KB

Terraform module for dynamically setting hostnames following a pattern on instances in AWS Auto Scaling Groups

Home Page: https://registry.terraform.io/modules/meltwater/asg-dns-handler

License: Apache License 2.0

Python 42.34% HCL 36.84% Go 20.82%
terraform terraform-module asg-dns-handler asg team-a-team lifecycle-maintenance purpose-library usage-public automation-drone coe-universal-solutions

terraform-aws-asg-dns-handler's Introduction

ASG DNS handler | Build Status

This Terraform module sets up everything necessary for dynamically setting hostnames following a certain pattern on instances spawned by AWS Auto Scaling Groups (ASGs).

Learn more about our motivation to build this module in our blog post Dynamic Route53 records for AWS Auto Scaling Groups with Terraform.

Maintainers

This repository and the module it houses are maintained Foundation Missions A-Team. Should you encounter issues or require changes to code maintained in this repository, please reachout through an issue that is part of this project.

Requirements

How do I use it?

Create an ASG and set the asg:hostname_pattern tag for example like this:

asg-test-#instanceid.asg-handler-vpc.testing@Z3QP9GZSRL8IVA

#instanceid is converted by a Lambda function within this module to the actual AWS instance_id that corresponds to the launched instance. The @ symbol is used to split the FQDN from the Route 53 zone_id.

This could be interpolated in Terraform like this:

tag {
  key                 = "asg:hostname_pattern"
  value               = "${var.hostname_prefix}-#instanceid.${var.vpc_name}.testing@${var.internal_zone_id}"
  propagate_at_launch = true
}

Once you have your ASG set up, you can just invoke this module and point to it:

module "clever_name_autoscale_dns" {
  source  = meltwater/asg-dns-handler/aws"
  version = "~> 2.0"
  
  # use_public_ip      = true
  # route53_record_ttl = 300
  autoscale_handler_unique_identifier = "clever_name"
  autoscale_route53zone_arn           = "ABCDEFGHIJ123"
  vpc_name                            = "my_vpc"
}

How does it work?

The module sets up these things:

  1. A SNS topic
  2. A Lambda function
  3. A topic subscription sending SNS events to the Lambda function

The Lambda function then does the following:

  • Fetch the asg:hostname_pattern tag value from the ASG, and parse out the hostname and Route53 zone ID from it.
  • If it's an instance being created
    • Fetch internal IP from EC2 API
    • Create a Route53 record pointing the hostname to the IP
    • Set the Name tag of the instance to the initial part of the generated hostname
  • If it's an instance being deleted
    • Fetch the internal IP from the existing record from the Route53 API
    • Delete the record

Setup

Add initial_lifecycle_hook definitions to your aws_autoscaling_group resource , like so:

resource "aws_autoscaling_group" "my_asg" {
  name = "myASG"

  vpc_zone_identifier = var.aws_subnets

  min_size                  = var.asg_min_count
  max_size                  = var.asg_max_count
  desired_capacity          = var.asg_desired_count
  health_check_type         = "EC2"
  health_check_grace_period = 300
  force_delete              = false

  launch_configuration = aws_launch_configuration.my_launch_config.name

  lifecycle {
    create_before_destroy = true
  }

  initial_lifecycle_hook {
    name                    = "lifecycle-launching"
    default_result          = "ABANDON"
    heartbeat_timeout       = 60
    lifecycle_transition    = "autoscaling:EC2_INSTANCE_LAUNCHING"
    notification_target_arn = module.autoscale_dns.autoscale_handling_sns_topic_arn
    role_arn                = module.autoscale_dns.agent_lifecycle_iam_role_arn
  }

  initial_lifecycle_hook {
    name                    = "lifecycle-terminating"
    default_result          = "ABANDON"
    heartbeat_timeout       = 60
    lifecycle_transition    = "autoscaling:EC2_INSTANCE_TERMINATING"
    notification_target_arn = module.autoscale_dns.autoscale_handling_sns_topic_arn
    role_arn                = module.autoscale_dns.agent_lifecycle_iam_role_arn
  }

  tag {
    key                 = "asg:hostname_pattern"
    value               = "${var.hostname_prefix}-#instanceid.${var.vpc_name}.testing@${var.internal_zone_id}"
    propagate_at_launch = true
  }
}

module "autoscale_dns" {
  source  = "meltwater/asg-dns-handler/aws"
  version = "2.1.7"

  autoscale_handler_unique_identifier = "my_asg_handler"
  autoscale_route53zone_arn           = var.internal_zone_id
  vpc_name                            = var.vpc_name
}

Developers Guide / Contributing

Please read CONTRIBUTING.md to understand how to submit pull requests to us, and also see our Code of Conduct.

Difference between Lifecycle action

Lifecycle_hook can have CONTINUE or ABANDON as default_result. By setting default_result to ABANDON will terminate the instance if the lambda function fails to update the DNS record as required. Complete_lifecycle_action in lambda function returns LifecycleActionResult as CONTINUE on success to Lifecycle_hook. But if lambda function fails, Lifecycle_hook doesn't get any response from Complete_lifecycle_action which results in timeout and terminates the instance.

At the conclusion of a lifecycle hook, the result is either ABANDON or CONTINUE. If the instance is launching, CONTINUE indicates that your actions were successful, and that the instance can be put into service. Otherwise, ABANDON indicates that your custom actions were unsuccessful, and that the instance can be terminated.

If the instance is terminating, both ABANDON and CONTINUE allow the instance to terminate. However, ABANDON stops any remaining actions, such as other lifecycle hooks, while CONTINUE allows any other lifecycle hooks to complete.

License and Copyright

This project was built at Meltwater. It is licensed under the Apache License 2.0.

terraform-aws-asg-dns-handler's People

Contributors

apoorva-marisomaradhya avatar bdebyl avatar beateguia avatar cmckeen avatar dafyddcrosby avatar feraudet avatar hikerspath avatar jcaffet avatar jimsheldon avatar meltwater-ateam avatar seanturner83 avatar spier avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

terraform-aws-asg-dns-handler's Issues

Using the same DNS name for all instances in an ASG?

Could we omit instance id from DNS name template, and use this module just as a way to create a DNS record to access nodes in ASG with stable DNS name? For example,

asg-test-someuniquename.asg-handler-vpc.testing@Z3QP9GZSRL8IVA

instead of

asg-test-#instanceid.asg-handler-vpc.testing@Z3QP9GZSRL8IVA

Since all nodes in ASG has the same name, this could also be used as a less expensive alternative to load balancers. It's useful at least for development environments, where long DNS propagation time don't matter much.

set identifier could be used to create/delete multiple A records for the same DNS name.

[Bug] Broken Lambda in v2.1.8

Observed Behaviour

Updating to v2.1.8 causes the Lambda to fail with the following error:


[ERROR] ParamValidationError: Parameter validation failed: Invalid type for parameter ChangeBatch.Changes[0].ResourceRecordSet.TTL, value: 300, type: <class 'str'>, valid types: <class 'int'> Traceback (most recent call last):   File "/var/task/autoscale.py", line 144, in lambda_handler     process_record(record)   File "/var/task/autoscale.py", line 136, in process_record     process_message(json.loads(record['Sns']['Message']))   File "/var/task/autoscale.py", line 132, in process_message     update_record(zone_id, ip, hostname, operation)   File "/var/task/autoscale.py", line 87, in update_record     route53.change_resource_record_sets(   File "/var/runtime/botocore/client.py", line 530, in _api_call     return self._make_api_call(operation_name, kwargs)   File "/var/runtime/botocore/client.py", line 919, in _make_api_call     request_dict = self._convert_to_request_dict(   File "/var/runtime/botocore/client.py", line 990, in _convert_to_request_dict     request_dict = self._serializer.serialize_to_request(   File "/var/runtime/botocore/validate.py", line 381, in serialize_to_request     raise ParamValidationError(report=report.generate_report())
--

Feels like this line is missing integer conversion.

Version(s) Affected

v2.1.8

Steps to Reproduce

  1. Deploy v2.1.8.
  2. Use it.

Expected Behaviour

No response

[Feature] Allow users to set TTL on route53 records

Feature Request

Hello,

Currently the TTL set for the route53 records is hardcoded to 300s. I would like to be able to configure this to 60s. My use case is for a single EC2 instance in an autoscaling group with a public IP to have a static name, e.g. web.example.com. When the instance is replaced and the record updated I don't want to have to wait 5 minutes before being able to reconnect.

I'm happy to submit a PR for this.

Version(s) Affected

v2.1.7

Value Provided

Ability to customize something that otherwise is hard-coded for no apparent reason.

Documentation on registry.terraform.io outdated

Documentation on terraform registry page contain outdated code:

module "clever_name_autoscale_dns" {
  source  = "meltwater/asg-dns-handler/aws"
  version = "x.y.z"

  autoscale_update_name     = "clever_name"
  autoscale_group_names     = "${aws_autoscaling_group.my_asg.name}"
  autoscale_route53zone_arn = "${var.zone_to_manage_records_in}"
}

In README.md there's autoscale_handler_unique_identifier instead of autoscale_update_name.

Feature idea: maintain a sequence of CNAMEs per ASG

This module looks great and we're looking into using it, thanks for the great work!

It would be nice to have a way to maintain an ordered list of CNAME records pointing to the unique records that contain the instance IDs, like in the below example for an ASG with two instances i-foo and i-bar:

When instances are added to the group, the function will create N.prefix records for each of them, as they are launched.

1.prefix -> i-foo
2.prefix -> i-bar

Later, when maybe the instance i-foo is terminated for whatever reason, the logic would delete its CNAME record.

Once the missing instance is eventually replaced by i-baz the logic would detect the free 1.prefix and would re-create it to point to i-baz.

IndexError: list index out of range

The lambda function is unable to create new Route53 records for launching instances and kills them over and over. I had to remove the lifecycle hooks to get it to stop killing my servers.

This is the error in the log that is printed when it fails to create a Route53 record for a launching instance:

list index out of range: IndexError
Traceback (most recent call last):
  File "/var/task/autoscale.py", line 138, in lambda_handler
    process_record(record)
  File "/var/task/autoscale.py", line 130, in process_record
    process_message(json.loads(record['Sns']['Message']))
  File "/var/task/autoscale.py", line 120, in process_message
    private_ip = fetch_private_ip_from_ec2(instance_id)
  File "/var/task/autoscale.py", line 23, in fetch_private_ip_from_ec2
    ip_address = ec2_response['Reservations'][0]['Instances'][0]['NetworkInterfaces'][0]['PrivateIpAddress']
IndexError: list index out of range

Add possibility to use public IP

Adding this issue for visibility. We'd like to be able to set records pointing to public IP addresses of instances.
There's PR implementing this feature: #26

[ERROR] InvalidChangeBatch: An error occurred (InvalidChangeBatch) when calling the ChangeResourceRecordSets operation: [RRSet with DNS name ........ is not permitted in zone ......]

Hello tried the code you provided as example but records are not created. Lambda logs:

[ERROR] InvalidChangeBatch: An error occurred (InvalidChangeBatch) when calling the ChangeResourceRecordSets operation: [RRSet with DNS name . is not permitted in zone ...]
Traceback (most recent call last):
  File "/var/task/autoscale.py", line 144, in lambda_handler
    process_record(record)
  File "/var/task/autoscale.py", line 136, in process_record
    process_message(json.loads(record['Sns']['Message']))
  File "/var/task/autoscale.py", line 132, in process_message
    update_record(zone_id, ip, hostname, operation)
  File "/var/task/autoscale.py", line 87, in update_record
    route53.change_resource_record_sets(
  File "/var/runtime/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/runtime/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)

code:

module "tomcat_asg" {
  for_each = { for k, v in var.tomcat_instances : k => v if var.tomcat_instances[k]["enabled"] == "true" }
  source   = "./terraform-aws-autoscaling"

  name = each.key 
  # Launch configuration
  use_lc               = true
  create_lc            = false
  launch_configuration = aws_launch_configuration.lc_conf[each.key].name
  vpc_zone_identifier = ["${data.terraform_remote_state.basic_network.outputs.basic_network.private_subnets}"]
  health_check_type = "EC2"
  min_size          = var.tomcat_instance_count
  max_size          = var.tomcat_instance_count
  desired_capacity  = var.tomcat_instance_count
  target_group_arns = [aws_lb_target_group.tomcat_tg[each.key].arn]

  initial_lifecycle_hooks = [{
    name                    = "lifecycle-launching"
    default_result          = "CONTINUE"
    heartbeat_timeout       = 60
    lifecycle_transition    = "autoscaling:EC2_INSTANCE_LAUNCHING"
    notification_target_arn = module.autoscale_dns.autoscale_handling_sns_topic_arn
    role_arn                = module.autoscale_dns.agent_lifecycle_iam_role_arn
    },
    {

      name                    = "lifecycle-terminating"
      default_result          = "CONTINUE"
      heartbeat_timeout       = 60
      lifecycle_transition    = "autoscaling:EC2_INSTANCE_TERMINATING"
      notification_target_arn = module.autoscale_dns.autoscale_handling_sns_topic_arn
      role_arn                = module.autoscale_dns.agent_lifecycle_iam_role_arn
    }
  ]
  tags = [
    {
      key                 = "asg:hostname_pattern"
      value               = "${each.key}@${data.terraform_remote_state.R53.outputs.r53_zone_id}" 
      propagate_at_launch = true
    }
  ]
}


module "autoscale_dns" {
  source  = "meltwater/asg-dns-handler/aws"
  version = "2.1.2"

  autoscale_handler_unique_identifier = "tomcat_dns"
  autoscale_route53zone_arn           = data.terraform_remote_state.R53.outputs.r53_zone_id
  vpc_name                            = data.terraform_remote_state.basic_network.outputs.basic_network.name
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.