hsdp-task-streaming-backup's Introduction

hsdp-task-streaming-backup

A Docker image that you can schedule on HSDP IronIO to perform PostgreSQL backups to an S3 bucket

Features

Streaming backups so not dependent on runner disk storage
Compresses backups

Usage

module "siderite_backend" {
  source = "philips-labs/siderite-backend/cloudfoundry"

  cf_region   = "eu-west"
  cf_org_name = "hsdp-demo-org"
  cf_user     = var.cf_user
  iron_plan   = "medium-encrypted-4GB"
}

resource "hsdp_function" "psql_backup" {
  name         = "psql_backup"
  docker_image = "philipslabs/hsdp-task-streaming-backup:v0.2.0"
  command      = ["/app/backup.sh"]

  environment = {
    DB_HOST               = "postgres-xxx.eu-west-1.rds.amazonaws.com"
    DB_PORT               = "5432"
    DB_USER               = "[REDACTED]"
    DB_PASSWORD           = "[REDACTED]"
    DB_NAME               = "mydb"

    BACKUP_NAME           = "backups/mydb"
    S3_ENDPOINT           = "s3-eu-west-1.amazonaws.com"
    S3_BUCKET             = "cf-s3-xxx"
    AWS_ACCESS_KEY_ID     = "[REDACTED]"
    AWS_SECRET_ACCESS_KEY = "[REDACTED]"
  }

  schedule {
    run_every = "7d"
  }

  backend {
    credentials = module.siderite_backend.credentials
  }
}

Bucket lifecycle policy

It is advised to set a S3 Bucket lifecycle policy. A good practice is to move your database backups to the GLACIER storage class after a couple of days and to set a expiration date to automatically delete older backups. The below policy moves dumps to CLACIER after 7 days and deletes them after 6 months (180 days)

[
  {
    "Expiration": {
      "Days": 180
    },
    "ID": "Move to Glacier and expire after 6 months",
    "Prefix": "",
    "Status": "Enabled",
    "Transitions": [
      {
        "Days": 7,
        "StorageClass": "GLACIER"
      }
    ]
  }
]