benjamin-maynard / kubernetes-cloud-mysql-backup Goto Github PK

The in-house scripts we're moving from dump everything on an mysql server except the following list of databases:

• mysql
• sys
• tmp
• information_schema
• performance_schema

I have some bash for this and will raise a PR in the next few days.

Hi Benjamin,

I like your tool and will use it on my gke-k8s to backup my gcp-mysql server to gcs.
If you hadn't written the script, I probably would have had to....
I use postgresql on gcp in addition to mysql.

Would you accept a patch that adds support for Postgres?

Best regards
Volker

mysqldump: Got error: 1045: "Access denied for user 'root'@'192.168.8.44' (using password: YES)" when trying to connect

There is currently no support for ARM devices like RPI 4.

I am getting the following error message when running the container:

Database backup failed to upload for *** at 26-11-2020 23:30:08. Error: Traceback (most recent call last):
  File "/usr/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/lib/python3.8/site-packages/awscli/clidriver.py", line 17, in <module>
    import botocore.session
  File "/usr/lib/python3.8/site-packages/botocore/session.py", line 29, in <module>
    import botocore.configloader
  File "/usr/lib/python3.8/site-packages/botocore/configloader.py", line 19, in <module>
    from botocore.compat import six
  File "/usr/lib/python3.8/site-packages/botocore/compat.py", line 26, in <module>
    from dateutil.tz import tzlocal
  File "/usr/lib/python3.8/site-packages/dateutil/tz/__init__.py", line 2, in <module>
    from .tz import *
  File "/usr/lib/python3.8/site-packages/dateutil/tz/tz.py", line 19, in <module>
    import six
ModuleNotFoundError: No module named 'six'
kubernetes-cloud-mysql-backup encountered 1 or more errors. Exiting with status code 1.

Hi Everyone, I wanted to try this tool and I followed the instructions, but unfortunately I have already tried everything, but unfortunately I do not know where the error is.

ERROR: (gcloud.auth.activate-service-account) Could not read json file /root/gcloud.json: Expecting value: line 1 column 1 (char 0)

According to the instructions, I downloaded the json key and used base64 to encrypt and paste it into the secret. But unfortunately it doesn't work.

Thank you for your help

Great tool !

Any plans to add custom certificate when connecting to private Minio host ?

Hi there,
your tool sounds very promising! Until now, I'm missing any kind of housekeeping. For example, keeping only the last X backups. Are there any plans to integrate that soon?

Thanks for this project. It's handy! Are there any plans to offer encryption prior to upload? I know the buckets can be encrypted, but uploading encrypted files helps with our compliance requirements.

BACKUP_COMPRESS don't work, still upload uncompress file to GC

spec: template: metadata: annotations: cattle.io/timestamp: "2020-07-10T13:36:12Z" creationTimestamp: null spec: containers: - env: - name: BACKUP_COMPRESS value: "true" - name: BACKUP_PROVIDER value: gcp ... image: gcr.io/maynard-io-public/kubernetes-cloud-mysql-backup imagePullPolicy: Always

I have setup cloud mysql backup in our environment which consists of develop (on our own k8s), staging/production on digital ocean.

On our k8s, it works like a treat!
On digital ocean, I am getting this error: "Database backup FAILED for at . Error: mysqldump: Got error: 2002: 'Can't connect to MySQL server on '' (115)' when trying to connect".

Have you run into an issue like this before?

Thank you!

Hey! 👋 Thanks for this project. It looks like a really simple solution to backing up mysql instances. I've ran into a problem when triying to use this though..

Database backup FAILED for corteza at 03-01-2020 22:25:29. Error: mysqldump: Got error: 1045: 'Plugin caching_sha2_password could not be loaded: Error loading shared library /usr//usr/lib/mariadb/plugin/caching_sha2_password.so: No such file or directory' when trying to connect

$ kubectl describe cronjobs -n sales mysql-backup 
Name:                          mysql-backup
Namespace:                     sales
Labels:                        <none>
Annotations:                   kubectl.kubernetes.io/last-applied-configuration:
                                 {"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"annotations":{},"name":"mysql-backup","namespace":"sales"},"spec":{"jobTemplat...
Schedule:                      * * * * *
Concurrency Policy:            Allow
Suspend:                       False
Successful Job History Limit:  3
Failed Job History Limit:      1
Starting Deadline Seconds:     <unset>
Selector:                      <unset>
Parallelism:                   <unset>
Completions:                   <unset>
Pod Template:
  Labels:  <none>
  Containers:
   mysql-backup:
    Image:      gcr.io/maynard-io-public/kubernetes-cloud-mysql-backup:v2.2.0
    Port:       <none>
    Host Port:  <none>
    Environment:
      GCP_GCLOUD_AUTH:           <set to the key 'gcp_gcloud_auth' in secret 'mysql-secrets'>  Optional: false
      BACKUP_PROVIDER:           gcp
      GCP_BUCKET_NAME:           moov-production-mysql-backups
      GCP_BUCKET_BACKUP_PATH:    sales/mysql/
      TARGET_DATABASE_HOST:      mysql.sales.svc.cluster.local
      TARGET_DATABASE_PORT:      3306
      TARGET_DATABASE_NAMES:     <set to the key 'database' in secret 'mysql-secrets'>  Optional: false
      TARGET_DATABASE_USER:      <set to the key 'username' in secret 'mysql-secrets'>  Optional: false
      TARGET_DATABASE_PASSWORD:  <set to the key 'password' in secret 'mysql-secrets'>  Optional: false
      BACKUP_TIMESTAMP:          _%Y_%m_%d
      SLACK_ENABLED:             true
      SLACK_CHANNEL:             #alerts
      SLACK_WEBHOOK_URL:         <set to the key 'slack_webhook_url' in secret 'mysql-secrets'>  Optional: false
    Mounts:                      <none>
  Volumes:                       <none>
Last Schedule Time:              <unset>
Active Jobs:                     <none>
Events:                          <none>

I have encoded the base64 string using the command

base64 -i gke_key.json -o k.txt

I added the text string in the kubernetes yaml file as a secret. What is the best option to encode the json file?

Hello,
Thank you for the great work.
Does this package handle saving to an SFTP inside or outside the cluster?
Thanks.

Hey ! I use this project since 2 years and this is very helpful.
I recently upgrade my database to mysql 8 with caching_sha2_password Authentication Plugin to improve security.

And now i have error on backup :

Database backup FAILED for api_hobbies at 26-11-2022 10:54:46. Error: mysqldump: Got error: 1045: 'Plugin caching_sha2_password could not be loaded: Error loading shared library /usr/lib/mariadb/plugin/caching_sha2_password.so: No such file or directory' when trying to connect

Is there any plan to support caching_sha2_password authentication plugin ?

Thanks a lot ! 🙏

Thank you for the great tooling here already, including the documentation. Worked our right of the box.

One thing i wonder is, why is there no retention configuration - seems like this is somehow not important to you?
Could you explain a bit how you deal with keeping only a hand full of backups over time?

Are you using Lifecycle management on those s3 storage?

Thanks!

I'm working on moving from our in-house scripts to yours, as they looked much further developed. However we're missing the ability to add custom parameters like --single-transaction. I'll raise a PR for this in the next few days.

Seems I'm unable to upload the created backup using a non-direct aws service like wasabi.com ... :( can you help.
'm always running into the following issue as secrets and access keys are not processed within your script(s), this container is pretty much useless:

Database backup successfully completed for app at 21-11-2020 08:02:19.
Database backup failed to upload for app at 21-11-2020 08:02:20. Error: upload failed: tmp/app_2020_11_21.sql to s3://dbbackup//app_2020_11_21.sql An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.
kubernetes-s3-mysql-backup encountered 1 or more errors. Exiting with status code 1.

apiVersion: v1
kind: Secret
metadata:
  name: database-backup
type: Opaque
data:
  aws_secret_access_key: $somehiddenvalue
  database_password: $somehiddenvalue
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: database-backup
spec:
  schedule: "*/2 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          initContainers:
          - name: wait-for-mysql
            image: busybox:latest
            command: [ 'sh', '-c', "until nslookup mysql.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for mysql; sleep 2; done" ]
          containers:
          - name: database-backup
            image: gcr.io/maynard-io-public/kubernetes-cloud-mysql-backup
            imagePullPolicy: Always
            env:
              - name: AWS_ACCESS_KEY_ID
                value: "somehiddenvalue"
              - name: AWS_SECRET_ACCESS_KEY
                valueFrom:
                   secretKeyRef:
                     name: database-backup
                     key: aws_secret_access_key
              - name: AWS_S3_ENDPOINT
                value: "s3.eu-central-1.wasabisys.com"
              - name: AWS_DEFAULT_REGION
                value: "eu-central-1"
              - name: AWS_BUCKET_NAME
                value: "dbbackup"
              - name: AWS_BUCKET_BACKUP_PATH
                value: "/"
              - name: TARGET_DATABASE_HOST
                value: "mysql"
              - name: TARGET_DATABASE_PORT
                value: "3306"
              - name: TARGET_DATABASE_NAMES
                value: "app"
              - name: TARGET_DATABASE_USER
                value: "root"
              - name: TARGET_DATABASE_PASSWORD
                valueFrom:
                   secretKeyRef:
                     name: mysql-root-password
                     key: password
              - name: BACKUP_TIMESTAMP
                value: "_%Y_%m_%d"
          restartPolicy: OnFailure

error from server (Invalid): error when creating "backups.yaml": Secret "AWS_SECRET_ACCESS_KEY" is invalid: metadata.name: Invalid value: "AWS_SECRET_ACCESS_KEY": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end wit

YOU MUST SEPARATE each module with "---" in your example you put "--"

An error occurred (SignatureDoesNotMatch) when calling the PutObject operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I am using minio; access_key and secret_key are correct.