Git Product home page Git Product logo

simd-adler32's Introduction

simd-adler32

docs.rs badge crates.io badge mit license badge

A SIMD-accelerated Adler-32 hash algorithm implementation.

Features

  • No dependencies
  • Support no_std (with default-features = false)
  • Runtime CPU feature detection (when std enabled)
  • Blazing fast performance on as many targets as possible (currently only x86 and x86_64)
  • Default to scalar implementation when simd not available

Quick start

Cargo.toml

[dependencies]
simd-adler32 = "*"

example.rs

use simd_adler32::Adler32;

let mut adler = Adler32::new();
adler.write(b"rust is pretty cool, man");
let hash = adler.finish();

println!("{}", hash);
// 1921255656

Support

CPU Features

impl arch feature
x86, x86_64 avx512
x86, x86_64 avx2
x86, x86_64 ssse3
x86, x86_64 sse2
🚧 arm, aarch64 neon
wasm32 simd128

MSRV 1.36.0**

Minimum supported rust version is tested before a new version is published. [**] Feature const-generics needs to disabled to build on rustc versions <1.51 which can be done by updating your dependency definition to the following.

Cargo.toml

[dependencies]
simd-adler32 = { version "*", default-features = false, features = ["std"] }

Performance

Benchmarks listed display number of randomly generated bytes (10k / 100k) and library name. Benchmarks sources can be found under the bench directory. Crates used for comparison are adler and adler32.

Windows 10 Pro - Intel i5-8300H @ 2.30GHz

name avg. time avg. thrpt
10k/simd-adler32 212.61 ns 43.805 GiB/s
10k/wuffs 3843 ns 2.63 GiB/s*
10k/adler32 4.8084 us 1.9369 GiB/s
10k/adler 17.979 us 530.43 MiB/s
----------------------- --------------- ------------------
100k/simd-adler32 2.7951 us 33.320 GiB/s
100k/wuffs 34733 ns 2.6814 GiB/s*
100k/adler32 48.488 us 1.9207 GiB/s
100k/adler 178.36 us 534.69 MiB/s

* wuffs ran using mingw64/gcc, ran with wuffs bench -ccompilers=gcc -reps=1 -iterscale=300 std/adler32.

MacBookPro16,1 - Intel i9-9880H CPU @ 2.30GHz

name avg. time avg. thrpt
10k/simd-adler32 200.37 ns 46.480 GiB/s
10k/adler32 4.1516 us 2.2433 GiB/s
10k/adler 10.220 us 933.15 MiB/s
----------------------- --------------- ------------------
100k/simd-adler32 2.3282 us 40.003 GiB/s
100k/adler32 41.130 us 2.2643 GiB/s
100k/adler 83.776 us 534.69 MiB/s

Safety

This crate contains a significant amount of unsafe code due to the requirement of unsafe for simd intrinsics. Fuzzing is done on release and debug builds prior to publishing via afl. Fuzzy tests can be found under fuzz the directory.

Resources

Credits

Thank you to the contributors of the following projects.

Contributing

Feel free to submit a issue or pull request. 😄

simd-adler32's People

Contributors

alextmjugador avatar cryze avatar fintelia avatar mcountryman avatar web-flow avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

simd-adler32's Issues

Rolling hash support

A requirement of support rolling hashes is the ability to remove old bytes.
Can this crate in any way provide that? Are the SIMD implementations prohibiting this? Is it just faster to re-calculate the entire hash instead?
Thanks for this incredible code!

Fails to build on the latest nightly when using the `nightly` feature

Hi, thanks for your work on this useful library! 😄

As stated in the issue title, simd-adler32 fails to build on the latest nightly (rustc 1.78.0-nightly (d44e3b95c 2024-02-09)) due to an unstable feature that was removed:

Build error

This is a fairly recent regression: nightlies older than a week or so are unaffected.

The Rust tracking issue for the removed stdsimd feature has already been referenced by other affected projects: rust-lang/rust#48556

misaligned pointer dereference

Using Rust 1.71 on a Wasm target, I am getting the following error in debug mode:

book_web_advanced.js:470 panicked at 'misaligned pointer dereference: address must be a multiple of 0x10 but is 0x3cabc8',
.cargo/registry/src/index.crates.io-6f17d22bba15001f/simd-adler32-0.3.5/src/imp/wasm.rs:103

I am using simd-adler32 indirectly through several dependencies of the image crate (for example png), and I am compiling the wasm with the following flag: RUSTFLAGS="-C target-feature=+simd128" .

My understanding is that we are hitting the recently-merged (1.70) safety check: rust-lang/rust#98112

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.