See: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/SSECustomerKey.html

This would likely need to provided as an argument pointing to a local file.

Currently s3rename cannot rename empty folders (as folder keys are ignored to avoid creating dummy files when renamed).

At the moment in main() we .await on each Future:

This means we can run asynchronously but only on one thread.

Instead we should spawn these futures and await their handles like we do with the destructor handles.

i.e. something like:

    let mut futs = FuturesUnordered::new();
    futs.push(tokio::spawn(task("task1", now.clone())));
    futs.push(tokio::spawn(task("task2", now.clone())));
    futs.push(tokio::spawn(task("task3", now.clone())));
    while let Some(_handled) = futs.next().await {}

It will be interesting to see if the nested .spawn() calls work okay for the destructors still.

Note this means all arguments and return values need to be Send and Sync so this will be quite involved.

Currently there is no support for the y/ translate command i.e. y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ for translating upper-case text to lower-case.

This should first be patched in the sedregex crate, see this issue.

If that is not possible, this could be implemented separately (i.e. its own function).

See: https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context

Not clear when this is needed, as renames of KMS-encrypted objects seem to work okay at the moment.

We could add subcommands for deleting matching keys, or changing their properties (tags, permissions, storage class, etc.).

First we would need to be able to parse 'g/' commands which just try to match with the given sedregex. If there is a match we apply the operation to that key, otherwise it is ignored.

This should first be patched to the sedregex crate.

If that is not possible it could be implement with regex directly.

Add interactive mode --interactive, -i

Allow user to decide whether to carry out key renames and overwrites on a key by key basis in a TUI.

We want the rename operations to be atomic, that is the S3 bucket should never be left in an inconsistent state where we have copied some keys to their new names, but not deleted the original keys.

This a problem in the case that s3rename is suddenly terminated. To alleviate this we try to wrap the Copy in a struct that triggers the corresponding Delete when dropped - so if there is a panic (or an interrupt signal is received) then we would still trigger these deletes for the completed copies. See: https://github.com/jamesmcm/s3rename/blob/master/src/wrapped_copy.rs#L36

However, we don't want these deletions to block the OS thread they run on. In principle this is okay, since the DeleteObjectRequest returns a future, however the problem is that drop() (from Drop) is not async, and so we cannot .await inside it.

This means the destructors have to be synchronous and we would be required to block the threads, greatly reducing the throughput of key renames.

To bypass this, we instead tokio::spawn() a new task from inside the destructor, and then await those tasks back in our async main function. See: https://github.com/jamesmcm/s3rename/blob/master/src/main.rs#L122

However, this doesn't resolve the original issue of atomicity, because whilst the tasks will be spawned when the destructors are called, if s3rename receives an interrupt, those tasks will not have time to complete (and will not be awaited).

Unfortunately, Asynchronous Destructors are not yet supported in Rust. But perhaps there is a better work-around for now.

Also note that if we didn't care about atomicity, we could make a much smaller number of batched DeleteObjectsRequests which deletes many keys at a time.

Get the AWS environment from the local environment (e.g. ~/.aws/config ) if it is not provided, Rusoto probably has a function for this.

Only really matters for the initial get_bucket_location request, as after that we want to use region of the bucket itself.

Use log crate for debug messages, so we can add timestamps.

Also use log crate for standard messages (i.e. those filtered in quiet mode).

These are supported by the S3 API, would further filter keys matched by prefix (and the regular expression for renaming).

Would need to accept datetime arguments somehow.

Currently you cannot use anonymous capture groups like s/([A-Za-z]+)_([0-9]{4})/\2_\1/ to rename test_2019 to 2019_test. At the moment the capture groups must be named, i.e.: s/(?P<text>[A-Za-z]+)_(?P<year>[0-9]{4})/$year_$text

This should first be patched in the sedregex crate.

If that isn't possible, we can do the conversion from anonymous to named ourselves, with the same outcome for the user.

Add --no-overwrite flag skipping renames that would result in overwrites.

Could either:

• Make a head_object request for each of the new key names
• Pull all keys from the bucket with list_keys_v2 (without prefix filtering), and then check locally vs the HashSet of keys

Both of these should only be triggered when the --no-overwrite flag is used.

Currently s3rename does not handle Access Control List permissions resulting in publicly accessible objects becoming private when renamed.

See: https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html

See: https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lock-overview.html

Probably we cannot delete objects in compliance mode, however in governance mode we can set bypass_governance_retention = Some(true) in our request.

However, we can only do this for buckets with Object Lock enabled, otherwise we get a 400 error.

x-amz-bypass-governance-retention is only applicable to Object Lock enabled buckets.