Git Product home page Git Product logo

lambda-mongocluster-s3's Introduction

Back up MongoDB (Atlas) to S3 through Lambda

Back up a MongoDB database to AWS S3 through a simple AWS Lambda function by using the mongodump binary. Result is a ZIP archive with .bson and .metadata.json files for each collection.

For a MongoDB Atlas cluster database backup, specify the URI command option like this:

--uri "mongodb+srv://[user]:[pass]@[host]/[name]"

Adapted from alonhar/lambda-mongodb-s3-backup.

mongodump binary is version 100.1.1 (mongodb-database-tools-amazon2-x86_64-100.1.1)


Setup instructions

  1. Clone repository and run npm install
  2. ZIP contents of the folder (not the folder itself)
  3. Create an AWS Lambda function
    • Select 'Author from scratch', enter your function name and select Node.js 12.x
    • Choose an existing role or create a new one and make sure it has a policy with s3:PutObject and s3:ListBucket permissions for the S3 bucket that you want to back up to, as well as the AWSLambdaBasicExecutionRole policy
    • Upload the ZIP file (no need to upload it to S3 first, as it should be just below 10MB so that it can be uploaded directly in Lambda)
    • Set environment variables (see table below)
    • Increase timeout from 3sec to 30sec
    • Configure a trigger. For instance, with CloudWatch Event rules (or Amazon EventBridge as it is now called), you can set up a cron schedule

Environment variables

Variable Description Required?
MONGODUMP_OPTIONS Your mongodump command options separated by a space (without mongodump at the beginning), for instance --uri "mongodb+srv://[user]:[pass]@[host]/[name]" Refer to the mongodump docs for a list of available options. Important: do not include the --out or -o option. Yes
S3_BUCKET Name of the S3 bucket Yes
S3_STORAGE_CLASS S3 storage class for the backup. Refer to the S3 SDK docs for a list of available options. No. Default is STANDARD
ZIP_FILENAME Name of the ZIP archive No. Default is mongodb_backup
DATE_FORMAT Will be appended to ZIP_FILENAME with a _ separator. Refer to the DayJS docs for a list of available formatting options. No. Default is YYYYMMDD_HHmmss

Changelog

v1.2.0

  • Updated mongodump from 4.0.5 to 100.1.1
  • The Mongo connection string is no longer fixed at a certain format. You now have the flexibility to specify the URI or host in the format you need, as well as other mongodump command options.
  • There is a new env variable through which you can specify the ZIP filename (it is no longer the database name)
  • The uploaded archive is now automatically AES256 encrypted on S3
  • The S3 storage class is now defined on the file level instead of the bucket level
  • Replaced 'zip-a-folder' by 'archive'
  • Updated docs

lambda-mongocluster-s3's People

Contributors

arackaf avatar dependabot[bot] avatar llangit avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.