Git Product home page Git Product logo

gs's Introduction

GS: A minimalistic Google Storage client

gs is a command line interface (CLI) and Python library that provides a set of essential commands for Google Cloud Storage. It is modeled after the AWS CLI's aws s3 command. Its features are:

  • Python 3 compatibility
  • A minimalistic set of dependencies
  • A tiny footprint
  • Intuitive convention-driven configuration of API credentials without browser login prompts
  • Checksum validation to ensure end-to-end data integrity in uploads and downloads
  • Progress bars for long-running upload and download operations
  • Resumable uploads and downloads
  • Multithreaded directory sync and batch delete, capable of handling large numbers of objects
  • An attractive paging and table layout interface
  • A JSON object metadata output mode for feeding data to other utilities

Installation

pip install gs

Synopsis

Usage:
gs [OPTIONS] COMMAND [ARGS]...
Options:
--version Show the version and exit.
--help Show this message and exit.
Commands:
gs configure Set gs config options, including the API key.
gs ls List buckets or objects in a bucket/prefix.
gs cp Copy files to, from, or between buckets.
gs mv Move files to, from, or between buckets.
gs mb Create a new Google Storage bucket.
gs rb Permanently delete an empty bucket.
gs rm Delete objects (files) from buckets.
gs sync Sync a directory of files with bucket/prefix.
gs api Use httpie to perform a raw HTTP API request.
gs presign Get a pre-signed URL for accessing an object.

Run gs configure to configure Google service account access credentials that will be used by the gs command. You can create a new service account key at https://console.cloud.google.com/iam-admin/serviceaccounts.

Credentials

Before making API calls, gs ingests API credentials in the following order of priority:

  • First, gs checks if a GOOGLE_APPLICATION_CREDENTIALS environment variable is set. If so, it attempts to load and use credentials from the service account credentials filename referenced by the variable.
  • If that varible is not set, gs attempts to load service account credentials previously configured with gs configure (stored in ~/.config/gs/config.json).
  • If that fails, gs attempts to load a service account API token from Google instance metadata.
  • If that fails, gs prints a warning and attempts to make API requests anonymously.

Using the Python library interface

from gs import GSClient
client = GSClient()
object_meta = client.get("b/my-bucket/o/my-object")
with client.get("b/my-bucket/o/my-object", params=dict(alt="media"), stream=True) as res:
    object_bytes = res.raw.read()
presigned_url = client.get_presigned_url("my-bucket", "my-object", expires_at=time.time()+3600)

Authors

  • Andrey Kislyuk

Links

Bugs

Please report bugs, issues, feature requests, etc. on GitHub.

License

Licensed under the terms of the MIT License.

https://travis-ci.com/chanzuckerberg/gs.png

gs's People

Contributors

kislyuk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

tengle1080

gs's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.