This task will partition an Amazon S3 bucket by the number of specified AWS list max keys. Each partition will be a list of S3 objects that may need to be remediated by renaming its key.

The remediation process:

1. parse root (URN NSS) path from object key
2. reverse URN NSS
3. prepend two additional "directory", ///some/path
4. copy object from source key to renamed destination key
5. delete source object

Each partition will be provided to a process task along with a object store to be queued in a process queue until all objects have been processed.

Each object remediated will result in a row in the remediation log. The remediation log will be appended on subsequent executions.

Remediation log:

source object key, destination object key, object eTag, object size in bytes, result of rename, ellapsed time in milliseconds

The result flag:

-2 copy incorrect etag
-1 s3 client/server error
 0 success
 1 skipped due to multipart/threshold
 2 skipped due to unsupported key
 3 skipped due to verified rename
 4 skipped due to verify only and not renamed

Build

mvn clean package

Run

java -jar target\drs-remediate-jar-with-dependencies.jar

Build

docker build --no-cache -t drs-remediate .

Run

docker run -v /c/drs-remediation/external:/external --env-file=.env drs-remediate

Build

docker-compose build --no-cache

Run

docker-compose up

If using docker, do not forget to copy .env.example to .env and update accordingly.