Comments (9)
Prototype:
@cli.command()
@click.argument("bucket")
@common_boto3_options
def list_bucket(bucket, access_key, secret_key, session_token, endpoint_url):
"List content of bucket"
s3 = make_client("s3", access_key, secret_key, session_token, endpoint_url)
paginator = s3.get_paginator("list_objects_v2")
keys = []
for page in paginator.paginate(Bucket=bucket):
for row in page["Contents"]:
click.echo(json.dumps(row, indent=4, default=str))
from s3-credentials.
Related:
from s3-credentials.
Looks like I accidentally checked in the implementation of this in f6fd163 - but it still needs documentation and tests.
s3-credentials/s3_credentials/cli.py
Lines 668 to 677 in 75ce3b2
from s3-credentials.
Spotted a bug - throws an error if run against a bucket that contains no keys:
% s3-credentials list-bucket pysqlite3-wheels
Traceback (most recent call last):
File "/Users/simon/.local/share/virtualenvs/s3-credentials-J8M1ChYK/bin/s3-credentials", line 33, in <module>
sys.exit(load_entry_point('s3-credentials', 'console_scripts', 's3-credentials')())
File "/Users/simon/.local/share/virtualenvs/s3-credentials-J8M1ChYK/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/Users/simon/.local/share/virtualenvs/s3-credentials-J8M1ChYK/lib/python3.10/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/Users/simon/.local/share/virtualenvs/s3-credentials-J8M1ChYK/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/simon/.local/share/virtualenvs/s3-credentials-J8M1ChYK/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/simon/.local/share/virtualenvs/s3-credentials-J8M1ChYK/lib/python3.10/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/Users/simon/Dropbox/Development/s3-credentials/s3_credentials/cli.py", line 676, in list_bucket
for row in page["Contents"]:
KeyError: 'Contents'
from s3-credentials.
Also it currently outlines not-quite-newline-delimited-JSON (it's newline delimited but also pretty printed, which I don't think will work) - need to design better default and output options for it.
% s3-credentials list-bucket static.niche-museums.com | head -n 20
{
"Key": "Griffith-Observatory.jpg",
"LastModified": "2020-01-05 16:51:01+00:00",
"ETag": "\"a4cff17d189e7eb0c4d3bf0257e56885\"",
"Size": 3360040,
"StorageClass": "STANDARD"
}
{
"Key": "IMG_0353.jpeg",
"LastModified": "2019-10-25 02:50:49+00:00",
"ETag": "\"d45bab0b65c0e4b03b2ac0359c7267e3\"",
"Size": 2581023,
"StorageClass": "STANDARD"
}
from s3-credentials.
I'm going to steal some of the output option ideas from sqlite-utils
- in particular I will default to streaming regular JSON that looks like this:
% sqlite-utils rows ../datasette/fixtures.db searchable
[{"pk": 1, "text1": "barry cat", "text2": "terry dog", "name with . and spaces": "panther"},
{"pk": 2, "text1": "terry dog", "text2": "sara weasel", "name with . and spaces": "puma"}]
Then offer --nl
and --csv
options too.
from s3-credentials.
I just spotted list-buckets
has the same not-quite-newline-delimited JSON output format, which is a bad default. I should fix that too.
from s3-credentials.
This is looking good, needs documentation.
from s3-credentials.
Documentation: https://github.com/simonw/s3-credentials/blob/07fa2073c784c69f2eb4245a664df161e933012f/README.md#list-bucket
from s3-credentials.
Related Issues (20)
- Way to make an existing bucket public or private HOT 1
- Convert README into documentation website HOT 3
- Make it easier to add extra policy statements HOT 10
- Provide a `--profile` option to allow AWS profile selection HOT 3
- Using --policy should imply --user-permissions-boundary=none HOT 2
- s3-credentials.AmazonS3FullAccess has MaxSessionDuration 3600, should be 12 hours HOT 5
- KeyError if listing bucket with no items returned
- s3-credentials list-buckets --details should show region and website URL, if configured HOT 2
- `s3-credentials get-objects` command HOT 7
- `get-objects/put-objects` `--skip` and `--skip-hash` options HOT 1
- Add the options to add tags to the created resources HOT 3
- `set-public-policy` command HOT 5
- Add s3:PutObjectAcl to write policies HOT 3
- `s3-credentials delete-objects` command HOT 11
- Mysterious test failure in `test_put_objects` HOT 4
- debug-bucket command HOT 3
- Command to make a bucket public HOT 4
- `s3-credentials create name-of-bucket --create-bucket --public` fails with error HOT 4
- `s3-credentials list-bucket --urls` option HOT 1
- CI failures, including ImportError: cannot import name 'mock_s3' from 'moto' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from s3-credentials.