Comments (4)
I believe gs:// is more popular amongst the general public.
but not among our user base, for obvious reasons.
Would you be prepared to switch the order of the gcsfs.core.GCSFileSystem.protocol
for yourself at runtime? You could propose this as a PR and we can maybe get a feeling for whether this is disruptive for people here.
from gcsfs.
At the point of running unstrip, we no longer know what protocol prefix was used in the original string, of course. "gcs" and "gs" are both allowed by GCSFS, and the former is by far the more popular with users (see also examples in the documentation). The aim of unstrip is to produce a URL which fsspec will recognise, so either is "valid".
I believe our use of full URLs of the style "gcs://bucket/path/file" may predate google's.
from gcsfs.
Thanks for your reply!
"gcs" and "gs" are both allowed by GCSFS, and the former is by far the more popular with users
How are you measuring that? I believe gs://
is more popular amongst the general public.
- Search results on GitHub for
gcs://
: 10.3k results - Search results on GitHub for
gs://
: 473k results
...(see also examples in the documentation)
Do you mean the fsspec docs or the Google docs? I can't find a single mention of gcs://
in the Google docs.
The aim of unstrip is to produce a URL which fsspec will recognise
I believe the aim should also be to produce URIs that are digestible by other tools. Currently, fsspec produces bespoke non-standard URIs not recognized by other tools, including Google's official gsutil
CLI:
$ gsutil ls gcs://my-bucket
InvalidUrlError: Unrecognized scheme "gcs".
I believe our use of full URLs of the style "gcs://bucket/path/file" may predate google's.
That may be true, and I apologize for not having the context on fsspec. I'm just a new client of the library trying to print out URIs which can be consumed by others on my team.
I should say that fsspec is an absolutely lovely library to work with and I'm such a fan. That's why I so badly want this little kink to be ironed out. Thanks for your hard work!
from gcsfs.
Sure, PR is here: #620
from gcsfs.
Related Issues (20)
- Strange behavior with `HTTPError` and multiprocessing HOT 3
- isdir/info method works incorrectly HOT 20
- Clarify how to pass JSON credentials HOT 2
- Is it needed to be so strict about dependency on fsspec HOT 1
- Release 2023.10.0 and consider relaxing fsspec dependency HOT 3
- Error when listing large directory with versions=True
- Request: add chmod
- Issues when using identity_pool.Credentials for connecting GCSFileSystem HOT 2
- Strange error message when using cp instead of put HOT 2
- Filename with slashes in the path are getting URL encoded, causing them to fail HOT 2
- Pin generation on open for version aware file system HOT 4
- asyncio exception while writing to zarr store HOT 5
- Add API reference for gcsfs.mapping to docs HOT 2
- `fs.isdir` latency 200x slower beginning with version 2023.09.01 HOT 1
- Error introduced in 2024.3.0 HOT 2
- FileNotFoundError since 2024.3.1 HOT 5
- Missing 'name' attribute in 'GCSFile' object when accessing PDF files HOT 1
- Invalid multipart request on retry HOT 6
- Question: aiohttp vs. gRPC API HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gcsfs.