Git Product home page Git Product logo

sprinkle_dns's Introduction

SprinkleDNS logo

SprinkleDNS

A diff-based way of managing DNS for people with lots of domains for AWS Route53.

How

Use plain old Ruby to define your DNS configuration:

require 'sprinkle_dns'

client = SprinkleDNS::Route53Client.new(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
sdns   = SprinkleDNS::Client.new(client)

sdns.entry('A', 'www.billetto.com', '88.80.188.142', 360)
sdns.entry('A', 'staging.billetto.com', '88.80.188.143', 360)

sdns.sprinkle!

Or a more advanced example using loops and interpolation:

require 'sprinkle_dns'

client = SprinkleDNS::Route53Client.new(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
sdns   = SprinkleDNS::Client.new(client)

domains = ['billetto.dk', 'billetto.co.uk', 'billetto.com',  'billetto.se']

domains.each do |domain|
  sdns.entry('A', domain, '88.80.188.142', 360)
  sdns.entry('A', "www.#{domain}", '88.80.188.142', 360)

  s.entry("CNAME", "docs.#{domain}",  'ghs.googlehosted.com', 43200)
  s.entry("CNAME", "mail.#{domain}",  'ghs.googlehosted.com', 43200)
  s.entry("CNAME", "drive.#{domain}", 'ghs.googlehosted.com', 43200)

  s.entry("MX",    domain, ['1 aspmx.l.google.com',
                            '5 alt1.aspmx.l.google.com',
                            '5 alt2.aspmx.l.google.com',
                            '10 aspmx2.googlemail.com',
                            '10 aspmx3.googlemail.com'], 60)
end

# Overwrite one of the domains, to test our new loadbalancer:
sdns.entry('A', 'billetto.com', '89.81.189.143', 360)

sdns.sprinkle!

Configuration

You can configure the SprinkleDNS::Client like so:

client = SprinkleDNS::Route53Client.new(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
sdns = SprinkleDNS::Client.new(client,
  dry_run: false,
  diff: true,
  show_untouched: false,
  force: true,
  delete: false,
  interactive_progress: true,
  create_hosted_zones: false,
)

Here is a table that shows the different configuration options:

Name Description Default value
dry_run Do not make any changes, just compare and exit, useful with diff: true. true
diff Prints a diff to list the changes that are going to be made. true
show_untouched Specifies whether or not the diff should show untouched records or not. false
force Do not ask before changes are made, just apply. false
delete Specifies whether unreferenced entries should be deleted. false
interactive_progress Shows interactive progress whilst changes are being applied, nice for your terminal, not for your CI-job. true
create_hosted_zones Specifies whether or not hosted zones should be created. false

dry_run and diff

dry_run is useful combined with diff because it will let you see the changes in a safe manner without any changes being applied:

dry_run and diff

force: false

With force being set to false you will be asked whether or not you want to apply the changes:

force set to false

delete: true

With delete being set to true SprinkleDNS will delete any entries not being referenced, these will also show up in the diff (if it is enabled):

delete true shows up in diffs

create_hosted_zones: true

With create_hosted_zones set to true, SprinkleDNS will create a hosted zone if not existing, it requires the route53:CreateHostedZone permission.

Support for ALIAS-records

Route53 supports ALIAS-records to achieve CNAME-flattening, SprinkleDNS also supports that, here we point our root domain to an ELB:

require 'sprinkle_dns'

client = SprinkleDNS::Route53Client.new(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
sdns   = SprinkleDNS::Client.new(client)

sdns.alias('A', 'billetto.com', 'Z215JYRZR1TBD5', 'dualstack.mothership-test-elb-546580691.eu-central-1.elb.amazonaws.com')

sdns.sprinkle!

Amazon policy

This gem uses the following permissions to manage hosted zones:

  • route53:ListHostedZones, for getting the list of hosted zones.
  • route53:ListResourceRecordSets, to read the records for a hosted zone.
  • route53:ChangeResourceRecordSets, to change records for a hosted zone.
  • route53:GetChange, for reading when a change have been applied.

Additionally, you can consider adding the following permissions:

  • route53:CreateHostedZone, for allowing the gem to create hosted zones.

You can allow it for all of your hosted zones:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "route53:ListResourceRecordSets",
                "route53:ChangeResourceRecordSets",
                "route53:GetChange",
                "route53:ListHostedZones"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

For a more "locked down" policy you can use this (remember to update the resource array):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "route53:ListResourceRecordSets",
                "route53:ChangeResourceRecordSets"
            ],
            "Resource": [
                "arn:aws:route53:::hostedzone/Z3EATJAGJWXQE8"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "route53:GetChange",
                "route53:ListHostedZones"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Obtain certificates with LetsEncrypt

Not everyone is aware of it, but LetsEncrypt allows for a DNS-challenge, this means that if you want to have a certificate for billetto.com you can ask certbot to use the DNS-challenge, and run a script:

certbot --preferred-challenges dns --manual-auth-hook "bash run_my_dns_script.sh"

The script run_my_dns_script.sh will then recieve two ENV-variables, one for CERTBOT_DOMAIN which in our example is billetto.com and CERTBOT_VALIDATION which is a value that needs to be set in the DNS, so in order to prove to LetsEncrypt that we manage the domain we have to set the following:

TXT   _acme-challenge.ENV['CERTBOT_DOMAIN']   ENV['CERTBOT_VALIDATION']

Instead of a bash-script, we can use Ruby and SprinkleDNS like so:

#!/usr/bin/env ruby
require 'sprinkle_dns'
require_relative '../includes/access_keys'

raise 'ENV-variable CERTBOT_DOMAIN is not supplied' if ENV['CERTBOT_DOMAIN'].nil?
raise 'ENV-variable CERTBOT_VALIDATION is not supplied' if ENV['CERTBOT_VALIDATION'].nil?

c = SprinkleDNS::Route53Client.new(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
s = SprinkleDNS::Client.new(c, interactive_progress: false, diff: false, force: true, delete: false, create_hosted_zones: false)
s.entry('TXT', "_acme-challenge.#{ENV['CERTBOT_DOMAIN']}", %Q{"#{ENV['CERTBOT_VALIDATION']}"}, 60)
s.sprinkle!

Save it as dns_auth.rb, and remember to chmod it: chmod +x dns_auth.rb.

Now you can start on the main script ssl_certbot.rb:

#!/usr/bin/env ruby
require 'open3'
require 'fileutils'

EMAIL = '[email protected]'
MAIN_DOMAIN = 'billetto.com'
DOMAINS = ['billetto.dk', 'billetto.co.uk', 'billetto.com']

def run_command(command)
  puts("+: #{command}")

  Open3.popen2e(command) do |stdin, stdout_stderr, wait_thread|
    Thread.new do
      stdout_stderr.each {|l| puts l }
    end
    wait_thread.value
  end
end

def print_guide
  puts "Congratulations, you have a new certificate!"
  puts "----------------------------------------------------------------"
  puts "CERTIFICATE: #{Dir.pwd}/config/live/billetto.com/cert.pem"
  puts "KEY:         #{Dir.pwd}/config/live/billetto.com/privkey.pem"
  puts "CHAIN:       #{Dir.pwd}/config/live/billetto.com/chain.pem"
end

letsencrypt_dirs = ['config', 'work', 'logs']
previous_letsencrypt_run = letsencrypt_dirs.all?{|dir| Dir.exist?(dir)}

case ARGV[0]
when 'create'
  certbot_commands = []
  certbot_commands << "certbot certonly"
  certbot_commands << "--manual --manual-public-ip-logging-ok --agree-tos"
  certbot_commands << "--email #{EMAIL} --update-registration --no-eff-email"
  certbot_commands << "--non-interactive --preferred-challenges dns"
  certbot_commands << "--manual-auth-hook \"bundle exec #{Dir.pwd}/dns_auth.rb\""
  certbot_commands << "--config-dir config --work-dir work --logs-dir logs"
  certbot_commands << "--cert-name #{MAIN_DOMAIN}"
  DOMAINS.each do |domain|
    certbot_commands << "-d #{domain} -d www.#{domain}"
  end
  certbot_commands = certbot_commands.join(" ")

  letsencrypt_dirs.select{|dirname| Dir.exists?(dirname)}.map{|dirname| FileUtils.remove_dir(dirname)}
  run_command("mkdir -p #{letsencrypt_dirs.join(' ')}")
  stdout, stdeerr, status = run_command(certbot_commands)

  print_guide
when 'renew'
  if previous_letsencrypt_run
    certbot_commands =  []
    certbot_commands << "certbot renew"
    certbot_commands << "--manual --manual-public-ip-logging-ok --agree-tos"
    certbot_commands << "--email #{EMAIL} --update-registration --no-eff-email"
    certbot_commands << "--non-interactive --preferred-challenges dns"
    certbot_commands << "--manual-auth-hook \"bundle exec #{Dir.pwd}/dns_auth.rb\""
    certbot_commands << "--config-dir config --work-dir work --logs-dir logs"
    certbot_commands << "--cert-name #{MAIN_DOMAIN}"
    certbot_commands = certbot_commands.join(" ")
    stdout, stdeerr, status = run_command(certbot_commands)

    print_guide
  else
    puts "It seems like there are no files from a previous LetsEncrypt run, exiting!"
    exit 1
  end
else
  puts "Usage:"
  puts "bundle exec ruby ssl_certbot.rb COMMAND"
  puts
  puts "Commands:"
  puts "create - Request a new certificate from LetsEncrypt, should only be used on the first run, or if you have modified the list of domains."
  puts "renew  - Renew an already created certificate"
end

You can update the variables in top of the script, and then you can run bundle exec ruby ssl_certbot.rb create, and everytime you need to renew the certificate you can run bundle exec ruby ssl_certbot.rb renew.

You will need to run the create if your list of domains have changed.

sprinkle_dns's People

Contributors

dependabot[bot] avatar kaspergrubbe avatar paneq avatar swistak35 avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sprinkle_dns's Issues

Octal domains

We fixed it for c83ebbd

But there might be more, like:

a-z
0-9
- (hyphen)
! " # $ % & ' ( ) * + , - / : ; < = > ? @ [ \ ] ^ _ ` { | } ~ .

And:

If your domain name contains any of the following characters, you must specify the characters by using escape codes in the format \three-digit octal code:

- Characters 000 to 040 octal (0 to 32 decimal, 0x00 to 0x20 hexadecimal)
- Characters 177 to 377 octal (127 to 255 decimal, 0x7F to 0xFF hexadecimal)
- . (period), character 056 octal (46 decimal, 0x2E hexadecimal), when used as a character in a domain name. When using . as a delimiter between labels, you do not need to use an escape code.
For example, to create a hosted zone for exämple.com, you specify ex\344mple.com.

But the most valuable insight is this:

If the domain name includes any characters other than a to z, 0 to 9, - (hyphen), 
or _ (underscore), Amazon Route 53 API actions return the characters as escape codes.

Todo

  • Make a test for exämple.com, also known as ex\344mple.com.

Sources:

http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/DomainNameFormat.html
https://forums.aws.amazon.com/thread.jspa?threadID=113183
boto/boto#1216

Prepended dot in TXT-records

# certbot
s.entry('CNAME', '_acme-challenge.nav.billetto.com', 'yaOgKnUdbBSL48lsyoaPk0OusZualdls8rZvUVEDnHw', 360)

Becomes:

screenie_1487597704_976727

Setting CNAME to integer instead of string throws error

require 'sprinkle_dns'
require_relative 'test_perms'

client = SprinkleDNS::Route53Client.new(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
sdns   = SprinkleDNS::Client.new(client)

sdns.entry("A", "beta.test.billetto.com", '88.80.188.143', 60, 'test.billetto.com')
sdns.entry("TXT", "test.test.billetto.com", Time.now.to_i, 60, 'test.billetto.com')

sdns.sprinkle!

Error:

Traceback (most recent call last):
	15: from example.rb:10:in `<main>'
	14: from /Users/kaspergrubbe/projects/sprinkle_dns/lib/sprinkle_dns/client.rb:72:in `sprinkle!'
	13: from /Users/kaspergrubbe/projects/sprinkle_dns/lib/sprinkle_dns/providers/route53_client.rb:65:in `change_hosted_zones'
	12: from /Users/kaspergrubbe/projects/sprinkle_dns/lib/sprinkle_dns/providers/route53_client.rb:65:in `each'
	11: from /Users/kaspergrubbe/projects/sprinkle_dns/lib/sprinkle_dns/providers/route53_client.rb:67:in `block in change_hosted_zones'
	10: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-route53-1.21.0/lib/aws-sdk-route53/client.rb:1035:in `change_resource_record_sets'
	 9: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/seahorse/client/request.rb:70:in `send_request'
	 8: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/seahorse/client/plugins/response_target.rb:23:in `call'
	 7: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
	 6: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
	 5: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
	 4: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
	 3: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
	 2: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/aws-sdk-core/plugins/param_validator.rb:23:in `call'
	 1: from /Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/aws-sdk-core/param_validator.rb:13:in `validate!'
/Users/kaspergrubbe/.rbenv/versions/2.6.2/lib/ruby/gems/2.6.0/gems/aws-sdk-core-3.48.3/lib/aws-sdk-core/param_validator.rb:33:in `validate!': expected params[:change_batch][:changes][0][:resource_record_set][:resource_records][0][:value] to be a String, got value 1556036081 (class: Integer) instead. (ArgumentError)

Support for CAA-records

Today I was trying out setting a CAA-record for my domain, and it turned out SprinkleDNS doesn't support that option.

Create specs before 1.0.0

Specs to add

  • Give a nice error when a hosted zone isn't created at the provider (308a0b9)
  • Add a way to configure whether or not the client should delete unreferenced records (strategies: list changes, force changes, ask).
  • Test Route53 interactions with missing permission errors
  • Test Route53 interactions with a full working run

Support Route53 ALIAS

If you're using an Amazon ELB on your apex domain you will have to use an alias. The issue is that A-records does only support IP addresses, and ELBs are only able to live on CNAMEs, and you aren't able to attach an IP to an Amazon ELB.

Solution

To support CNAME flattening (that's the branded Cloudflare PR name for it) we need to do the following change to our Route53 DNS:

ALIAS billetto.tld      www.billetto.tld
CNAME www.billetto.tld  foobar-ingress-elb-1293773007.eu-central-1.elb.amazonaws.com

API-format

Right now we use the following format for updates:

change_batch_options << {
  action: 'CREATE',
  resource_record_set: {
    name: entry.name,
    type: entry.type,
    ttl: entry.ttl,
    resource_records: entry.value.map{|a| {value: a}},
  },
}

And the changed format is looking like this (notice that resource_records and ttl is missing):

change_batch_options << {
  action: 'CREATE',
  resource_record_set: {
    name: entry.name,
    type: entry.type,
    alias_target: {
      hosted_zone_id: "string",
      dns_name: "string",
      evaluate_target_health: false
    }
  },
}

(There is a full example of the supported values here for reference: https://docs.aws.amazon.com/cli/latest/reference/route53/change-resource-record-sets.html#options)

SprinkleDNS API-change

This is how it works right now:

require 'sprinkle_dns'
require_relative 'test_perms'

client = SprinkleDNS::Route53Client.new(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
sdns   = SprinkleDNS::Client.new(client)

sdns.entry('A',     'kaspergrubbe.com',        '88.80.188.142',    360)
sdns.entry('CNAME', 'beta.kaspergrubbe.com',   'kaspergrubbe.com', 360)
sdns.entry('A',     'assets.kaspergrubbe.com', '88.80.188.142',    360)
sdns.entry('MX',    'mail.kaspergrubbe.com',   ['10 mailserver.example.com', '20 mailserver2.example.com'], 300)

sdns.sprinkle!

The way sdns.entry(type, name, value, ttl) is constructed is a bit limited since it doesn't allow for using aliases.

Proposal 1

A way we could solve it could be to create an extra argument like this:

sdns.entry('A', 'kaspergrubbe.com', '88.80.188.142', 360, alias: ['hosted_zone_id', 'dns_name'])

And then remove the value and ttl if an alias is present, so this becomes valid:

sdns.entry('A', 'kaspergrubbe.com', nil, nil, alias: ['hosted_zone_id', 'dns_name'])

Proposal 2

Another way could be to build a new construct like this:

sdns.alias('A', 'kaspergrubbe.com', alias: ['hosted_zone_id', 'dns_name'])

Or maybe:

sdns.alias('A', 'kaspergrubbe.com', 'hosted_zone_id', 'dns_name')

Things to test for

Example 1

elb_hostname = 'dualstack.foobar-ingress-elb-1293773007.eu-central-1.elb.amazonaws.com'
sdns.entry('CNAME', 'www.kaspergrubbe.com', elb_hostname, 360)
sdns.entry('A', 'kaspergrubbe.com', nil, nil, alias: ['Z3RGKRXYUSHK13', elb_hostname])

Example 2

sdns.entry('A', 'kaspergrubbe.com', '88.80.188.142', 360)
sdns.entry('A', 'kaspergrubbe.com', '88.80.188.142', 360, alias: ['hosted_zone_id', 'dns_name'])
# Blow up!

(Actual behaviour might be to overwrite, we need to verify that)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.