shaomeng / sperr Goto Github PK

SPERR (pronounced like spur) is a lossy compressor for scientific data (2D or 3D floating-point data, mostly produced by numerical simulations). SPERR has one of the highest coding efficiencies among popular lossy compressors, meaning that it usually uses the least amount of storage to satisfy a prescribed error tolerance (e.g., a maximum point-wise error tolerance).

Under the hood, SPERR uses wavelet transforms, SPECK coding, and a custom outlier coding algorithm in its compression pipeline. This combination gives SPERR flexibility to compress targetting different quality controls, namely 1) bit-per-pixel (BPP), 2) peak signal-to-noise ratio (PSNR), and 3) point-wise error (PWE). The name of SPERR stands for SPeck with ERRor bounding.

SPERR documentation is hosted on Github Wiki pages. To get started, one might want to build SPERR from source and explore compression and decompression utilities for 3D and 2D inputs. One may also want to pull a docker image which contains SPERR in a complete development environment, or use spack to install SPERR by one command spack install sperr. Finally, a collection of canonical scientific data sets is available at SDRBench for testing and evaluation purposes.

SPERR also provides programming API in C++ and C.

If SPERR benefits your work, please kindly cite this publication:

@INPROCEEDINGS{10177487,
  author={Li, Shaomeng and Lindstrom, Peter and Clyne, John},
  booktitle={2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS)}, 
  title={Lossy Scientific Data Compression With SPERR}, 
  year={2023},
  pages={1007-1017},
  doi={10.1109/IPDPS54959.2023.00104}}