Git Product home page Git Product logo

our-postgresql-setup's Introduction

our-postgresql-setup

ARCHIVED: This configuration is no longer used by GoCardless, see the replacement at https://github.com/gocardless/stolon-pgbouncer

Overview

This repo is an extracted version of how we run PostgreSQL clusters at GoCardless.

It helps you quickly spin up a 3-node cluster of PostgreSQL, managed by Pacemaker, and proxied by PgBouncer.

It's intended as a playground for us, and a learning resource that we wanted to share with the community.

You can hear more about how the cluster works in our talk - Zero-downtime Postgres upgrades.

What's in the cluster?

When you start the cluster, you get 3 nodes, each running:

  • PostgreSQL
  • Pacemaker
  • PgBouncer

All packages are from Ubuntu 14.04, except for PostgreSQL itself, which is at version 9.4.

The cluster is configured with a single primary, one synchronous replica, and one asynchronous replica.

Dependencies

  1. Virtualbox
  2. Vagrant
  3. git clone https://github.com/gocardless/our-postgresql-setup.git
  4. [Recommended] tmux

Getting started

With tmux (recommended)

  1. ./tmux-session.sh start

By hand

  1. On 3 separate windows:
  2. vagrant up pg01 && vagrant ssh pg01
  3. vagrant up pg02 && vagrant ssh pg02
  4. vagrant up pg03 && vagrant ssh pg03

Viewing cluster status

You can run crm_mon -Afr on any node to see the current state of the cluster and all resources in it. Press ^c to quit.

Connecting to PostgreSQL

Once the cluster is up, you have two options:

  • Connect directly to Postgres on the PostgresqlVIP at 172.28.33.10
  • Connect via PgBouncer at 172.28.33.9

Note: The migrator.py script will only give you zero-downtime migrations if you connect via PgBouncer.

Running a zero-downtime migration

  1. Ensure clients are connected to the PgBouncerVIP.
  2. Run /vagrant/migrator.py on the node that has the PgBouncerVIP (you can find out where the PgBouncerVIP is by viewing the cluster status).
  3. Follow the prompts.
    1. It is safe to ignore the Make sure you have the following command ready... prompt. This is aimed at cases where you'd want to quickly re-enable traffic, and doesn't matter when running locally.
  4. Assuming everything went well, the primary will migrate to the synchronous replica, and the clients won't have received any connection resets.

I have a question!

We're happy to receive questions as issues on this repo, so don't be shy!

It's hard to know exactly what documentation/guidance is useful to people, so we'll use the questions we answer to improve this README and link out to more places you can read up on the technologies we're using.

Further reading

References

our-postgresql-setup's People

Contributors

benwh avatar nexusventuri avatar nlopes avatar sinjo avatar stevedomin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

our-postgresql-setup's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.