🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

As a user of fs-azureblob, I want to consult documentation about which aspects of pyfilesystem are supported by the library, as well as which aspects of blob storage are supported.

Describe your proposed implementation

• Add docstrings where needed
• Publish docs to the website, or readthedocs
• We only support BlockBlobs - either mention that explicitly or see if support for other blob types is feasible
• Find and mention any caveats related to directory support
• Document test setup - could a user without access to our account easily run all tests or write new ones?
• Document development workflow (editable vs normal pip install)
• etc

🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

As a user of the fs-azureblob extension, I want to know what errors could be raised and prefer they are consistent with possible errors from other filesystems.

Describe your proposed implementation

Pyfilesystem provides a bunch of built in errors that can represent underlying errors from the os, sdk, etc. We should handle any internal errors by converting to an fs error and re-raising if needed.

🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

I want to see badges when I browse the github repo

🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

Enable consumption of the extension by standard means

Describe your proposed implementation

Steps to create an extension from the docs. The main points are:

• Add the entry_points to setup.py
• Implement Opener subclass to handle custom url protocol
• Add pypi token to github secrets
• Add release.yml workflow to publish when new tags are created
• Create git tag to trigger workflow

Describe alternatives you've considered, if relevant

n/a

Additional context

Technically, we could use the extension without doing this, but it make the package consistent with other extensions and possibly worth listing as an "official" 3rd party extension

🚀

• Is your feature request essential for your project?
  No, but it would be pretty nice.

Describe the workflow you want to enable

I want progress bars for upload/download.

Describe your proposed implementation

Write a custom download method instead of delegating to the base FS implementation. The download method allows for additional kwargs, one of which should be a callback, so that the exact progress bar implementation is left up to consumers.

Note: the BlobFile implementation first downloads the blob to a temp file before any read operations occur. We should consider providing some context to the user since the progress bar function below is delayed until the initial download is complete. Alternatively, the behavior could be optimized, but this might be difficult to do while preserving the file-like-object api.

Describe alternatives you've considered, if relevant

Not having progress bars.

Additional context

Refer to the similar customization done for the ssh fs

🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

Some of the tests require credentials to our storage account. It would be nice if an arbitrary developer could run these without needing access.

Describe your proposed implementation

There is a storage emulator which could be run in a container. The emulator supports authentication, so it should suffice for testing error handling or any credential related functionality.

Update: the storage emulator is now deprecated and being replaced by azurite

🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

The smart_open package implements a file api on top of various storage backends. It could potentially replace the BlobFile implementation which does not stream efficiently.

Describe your proposed implementation

This should provide a description of the feature request, e.g.:

• Delete blob_file.py
• Change openbin to simply call smart_open

Additional context

When I get a chance, I will test this with the progress bar to see if the behavior is intuitive. Currently, a blob must be downloaded before the progress bar would start, which for large enough files is quite apparent.

🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

Use blob storage as a pyfilesystem backend

Describe your proposed implementation

Inherit from the base FS class and implement the essential methods for the filesystem to work.

Describe alternatives you've considered, if relevant

n/a

Additional context

n/a

🚀

• Is your feature request essential for your project?

Describe the workflow you want to enable

Make sure any relevant test cases pass.

Describe your proposed implementation

Follow the testing guide from the docs. We should look into using the existing tests, but at the very least define our own test cases specific to blob storage. Not all features provided by other filesystems will apply here (e.g. symlinks, device files, other os stuff). One example is that blob storage doesn't have a concept of directories, but blobs can have paths. So [blob_container]/path/to/file.txt would be a single blob named /path/to/file.txt, and ls /path/to doesn't work unless we fake it (which we do, at least tentatively).

Describe alternatives you've considered, if relevant

n/a

Additional context

n/a