Scalable and easy WARC records storage on S3.

Install the package from PyPI:

pip install warc-s3

TODO

To build this package and contribute to its development you need to install the build, and setuptools and wheel packages:

pip install build setuptools wheel

(On most systems, these packages are already pre-installed.)

Install package and test dependencies:

pip install -e .[tests]

Verify your changes against the test suite to verify.

flake8 warc_s3  # Code format
mypy warc_s3    # Static typing
pylint warc_s3  # LINT errors
bandit -c pyproject.toml -r warc_s3  # Security
pytest warc_s3  # Unit tests

Please also add tests for your newly developed code.

Wheels for this package can be built with:

python -m build

If you hit any problems using this package, please file an issue. We're happy to help!

This repository is released under the MIT license.