avalentino / asfsmd Goto Github PK

The httpio package is very small seems to fit quite well the needs of this project but:

• it seems to be a not-active project (last commit 2019)
• it is necessary to have a specialisation of the httpio.SyncHTTPIOFile class to be able to use this package in asfsmd

Other alternatives exists that could replace httpio as shall be assessed:

• RemoteZip

  Very interesting, it can be used out of the box and allows to simplify the codebase quite a lot.
  The performance to get the list of files included in the ZIP archive seems to be better (x2) than the ones that one can get with httpio but the test consisting in the download of annotations of 5 S1 products seems to be 12% slower than httpio.

• pyremotezip

  Not maintained upstream, Python2 only.

• smart-open

  Interesting project bur slower (x3-x4) than httpio to retrieve the list of files contained in the ZIP archive.

• fsspec

  Extremely interesting and feature complete. Used also in Dask.
  Direct use of the ZipFileSystem is very slow. The use of the HttpFileSystem + zipfile from the Python stdlib seems to work fine, but it is 33% slower than httpio