• AWS: #119
• GCP

Edit 2020-06-04 [giovannt0]: GCP already offers this in CreateDiskCopy() with the zone parameter. As such, removing GCP label.

Similar to log2timeline/dftimewolf#176

• Change method names to CamelCase

• Add pylintrc to enforce style

When executing log queries it would be nice to have the option to have the output copied to a S3 or GCS storage bucket in a destination project.

Refactor how we read install/test_requirements in setup.py and do not use internal pip structures due to pypa/pip#8188

• AWS
• GCP

As discussed, let's break down the compute.py module into more granular modules so that we are closer to the orginal API: compute.disk, compute.instance ...

Factor out the bash script inside gcp.py into its own file.

https://docs.python.org/3/library/typing.html

See this PR for more details https://github.com/log2timeline/dftimewolf/pull/257/files

• Folder structure (unittests / e2e tests)
• Function to check validity of JSON file

#19

Explore possibilities of retrieving system logs (e.g. /var/log/syslog) and other comon logs directly from libcloudforensics

When creating a disk copy the generated snapshot is not deleted in the source project.

It'd be nice to be able to attach several disks in one go when creating the analysis VM. I.e. attach_disk should be replaced by a list of GoogleComputeDisk instead of a single item.

while not have_all_tokens: kinda loops, across gcp.py and aws.py

Sometimes the ec2 browser client fails to install, resulting in the machine being locked out from external connexions.

E.g. https://www.terraform.io/docs/providers/google/r/compute_snapshot.html

libcloudforensics should be able to read/write S3 and GCS buckets.

There's a bug in GoogleCloudProject.crreate_disk_from_snapshot() with the disk's name creation. If the project name and the instance name are long strings, then truncate_at becomes negative and the resulting disk_name that is created may end up being longer than 63 chars. Additionally, len(project_id) should not be taken out of truncate_at since the project id itself is not used in the disk name.

The disk name creation code should be factored out of this method, and unit tested separately.

YAPF should put every function parameter on a separate line for readability.
Add below to YAPF style file and create a separate re-style PR for all code.

SPLIT_ALL_COMMA_SEPARATED_VALUES = True

• Class to interact with AWSProjects
• Unit tests

The idea is to pass a list of strings to StartAnalysisVm() for the attach_volume / attach_disk parameter instead of an AWSVolume or a GoogleComputeDisk object. The list of strings would ideally be a list of volume IDs/names to attach. This will make StartAnalysisVm() more flexible (e.g. if wanting to use these methods in the CLI tool without defining volumes/disks in the first place)

Make sure try/except blocks only contain code useful in the context of the boto3 API calls and catch both client / waiter exceptions.

Hijack the plaso-ci platform maybe? And have a script to test the "making a copy of a live VM and attach it to a new forensics instance"

Have the ability to pass log queries in raw to be able to create very specific recipes.

Be able to pull cloud logs:

• pull logs of a specific host
• pull logs of the project in general
• pull logs related to specific areas (eg authentication)

With new capabilities brought by #70 and #119 and seeing how granular AWS IAM permissions can get when it comes to identifying the minimum set of required permissions to use the CreateVolumeCopy() functionality in different scenarios, we need more e2e tests that verify these scenarios:

• Boot volume copy
• Specific volume copy
• Bootstrap VM
• Volume copy in a different zone
• Encrypted volume copy
• Encrypted volume copy in a different zone

Make sure docstrings are on par with https://github.com/log2timeline/l2tdocs/blob/master/process/Style-guide.md

See #28 (comment)

https://www.chargedretail.co.uk/2020/05/11/lidl-owner-launching-its-own-rival-to-amazon-web-services/

Maybe use Sphinx? https://www.sphinx-doc.org/en/master/