greenmask restore fails without error messages or exiting about greenmask HOT 14 CLOSED

Hi!

Could you provide the Greenmask version? Are you using S3 storage or directory?

Try to execute Greenmask in debug mode. Maybe you will find something that you can share.

Let's gather more information

from greenmask.

I am using v0.1.6. The storage is a directory. I ran greenmask restore in debug mode. Once again, it stopped working, with no exit and no error-level logs. It stops with the following debug message:

2024-03-14T17:46:57Z DBG ../greenmask/internal/db/postgres/pgrestore/pgrestore.go:44 > pg_restore: /usr/local/pgsql/bin/pg_restore --dbname host=db.host.name port=5432 user=postgres dbname=db_name password=super_secret_password --jobs 1 --section post-data --username postgres /tmp/1710436063879645860
 pid=145520

from greenmask.

Try to restore dump directly via pg_reatore

pg_restore --verbose -U postgres -h localhost /storage_path/1708598401362987000

Replace your connection string and actual backup id (1708598401362987000)

In case it will stop unexpectedly, there is going to be bug in ported library.

from greenmask.

I suspect there is bug in TOC library. Add --verbose parameter in pg_restore command

from greenmask.

Provide PostgreSQL version as well, please

from greenmask.

I ran pg_restore as directed. No errors, and it exited as expected.

I am using Postgres 13.13.

from greenmask.

Does Greenmask return non-zero exit code?

I suspect it is a schema restoration issue, could you two times run Greenmask with the next parameters:

1. greenmask --config config.yml restore --schema-only=true --verbose=true latest
2. greenmask --config config.yml restore --verbose=true latest

The things that I will be happy if you provide:

• is the schema and data restored in the database (even partially if greenmask fails)?
• is Greenmask fails on the index or constraint restoration?
• could you check the PostgreSQL log as well, maybe there you will find a log like with the reason
• Could you check that the greenmask was not killed by OOM Killer? You can find this log via dmesg -t

from greenmask.

I get the following with verbose flag enabled:

$ greenmask --config config.yml restore --schema-only --verbose latest
{"level":"fatal","error":"1 error(s) decoding:\n\n* cannot parse 'restore.pg_restore_options.schema-only' as bool: strconv.ParseBool: parsing \"--verbose\": invalid syntax","time":"2024-03-15T17:55:14Z"}

from greenmask.

I am also getting the following with just --schema-only enabled, but not --verbose:

$ greenmask --config config.yml restore --schema-only latest
{"level":"fatal","error":"1 error(s) decoding:\n\n* cannot parse 'restore.pg_restore_options.schema-only' as bool: strconv.ParseBool: parsing \"latest\": invalid syntax","time":"2024-03-15T18:00:24Z"}

from greenmask.

That's my fault. Try to use

greenmask --config config.yml restore --verbose=true --schema-only=true latest

from greenmask.

Your first command ran with no issues. So did your second command, except that it took forever to restore the indexes. Maybe restoring indexes was where it stopped working before?

from greenmask.

Thank you for the details. I was not able to reproduce your case.

Could you try to restore the DB with two steps:

• greenmask --config config.yml restore --schema-only=true --verbose=true latest --log-level=debug
• greenmask --config config.yml restore --data-only=true --verbose=true latest --log-level=debug

This command splits the restoration into two stages:

• Schema will be restored in the first command. Grenmask internally delegates it to pg_restore
• The data will be restored in the second command. Greenmask restores the data by itself

If there is a stuck in the second command It is supposed to be a bug in data restoration. Check the log output maybe there will be something that can help me to find the issue

from greenmask.

@jeffsutherland-bluesky has this problem been solved? I could not reproduce your case

from greenmask.

@jeffsutherland-bluesky I am closing this issue. Feel free to re-open this or create a new issue. Thank you!

from greenmask.