Git Product home page Git Product logo

noble-hashes's Introduction

noble-hashes

Audited & minimal JS implementation of SHA, RIPEMD, BLAKE, HMAC, HKDF, PBKDF, Scrypt & Argon2.

  • ๐Ÿ”’ Audited by an independent security firm
  • ๐Ÿ”ป Tree-shakeable: unused code is excluded from your builds
  • ๐ŸŽ Fast: hand-optimized for caveats of JS engines
  • ๐Ÿ” Reliable: chained / sliding window / DoS tests and fuzzing ensure correctness
  • ๐Ÿ” No unrolled loops: makes it easier to verify and reduces source code size up to 5x
  • ๐Ÿข Scrypt supports N: 2**22, while other implementations are limited to 2**20
  • ๐Ÿฆ˜ SHA3 supports Keccak, cSHAKE, KangarooTwelve, MarsupilamiFourteen and TurboSHAKE
  • ๐Ÿชถ 89KB (17KB gzipped) for everything, 10KB (2.5KB gzipped) for single-hash build

The library's initial development was funded by Ethereum Foundation.

For discussions, questions and support, visit GitHub Discussions section of the repository.

This library belongs to noble cryptography

noble cryptography โ€” high-security, easily auditable set of contained cryptographic libraries and tools.

Usage

npm install @noble/hashes

We support all major platforms and runtimes. For Deno, ensure to use npm specifier. For React Native, you may need a polyfill for getRandomValues. A standalone file noble-hashes.js is also available.

// import * from '@noble/hashes'; // Error: use sub-imports, to ensure small app size
import { sha256 } from '@noble/hashes/sha2'; // ECMAScript modules (ESM) and Common.js
// import { sha256 } from 'npm:@noble/[email protected]/sha2'; // Deno
console.log(sha256(new Uint8Array([1, 2, 3]))); // Uint8Array(32) [3, 144, 88, 198, 242...]
// you could also pass strings that will be UTF8-encoded to Uint8Array
console.log(sha256('abc')); // == sha256(new TextEncoder().encode('abc'))

Implementations

All hash functions:

  • receive Uint8Array and return Uint8Array
  • may receive string, which is automatically converted to Uint8Array via utf8 encoding (not hex)
  • support little-endian and big-endian architectures
  • can hash up to 4GB per chunk, with any amount of chunks
function hash(message: Uint8Array | string): Uint8Array;
hash(new Uint8Array([1, 3]));
hash('string') == hash(new TextEncoder().encode('string'));

All hash functions can be constructed via hash.create() method:

  • the result is Hash subclass instance, which has update() and digest() methods
  • digest() finalizes the hash and makes it no longer usable
hash
  .create()
  .update(new Uint8Array([1, 3]))
  .digest();

Some hash functions can also receive options object, which can be either passed as a:

  • second argument to hash function: blake3('abc', { key: 'd', dkLen: 32 })
  • first argument to class initializer: blake3.create({ context: 'e', dkLen: 32 })
sha2: sha256, sha384, sha512 and others
import { sha256, sha384, sha512, sha224, sha512_256, sha512_384 } from '@noble/hashes/sha2';
// also available as aliases:
// import ... from '@noble/hashes/sha256'
// import ... from '@noble/hashes/sha512'

// Variant A:
const h1a = sha256('abc');

// Variant B:
const h1b = sha256
  .create()
  .update(Uint8Array.from([1, 2, 3]))
  .digest();

for (let hash of [sha384, sha512, sha224, sha512_256, sha512_384]) {
  const res1 = hash('abc');
  const res2 = hash.create().update('def').update(Uint8Array.from([1, 2, 3])).digest();
};

See RFC 4634 and the paper on truncated SHA512/256.

sha3: FIPS, SHAKE, Keccak
import {
  sha3_224,
  sha3_256,
  sha3_384,
  sha3_512,
  keccak_224,
  keccak_256,
  keccak_384,
  keccak_512,
  shake128,
  shake256,
} from '@noble/hashes/sha3';
const h5a = sha3_256('abc');
const h5b = sha3_256
  .create()
  .update(Uint8Array.from([1, 2, 3]))
  .digest();
const h6a = keccak_256('abc');
const h7a = shake128('abc', { dkLen: 512 });
const h7b = shake256('abc', { dkLen: 512 });

See FIPS PUB 202, Website.

Check out the differences between SHA-3 and Keccak

sha3-addons: cSHAKE, KMAC, K12, M14, TurboSHAKE
import {
  cshake128,
  cshake256,
  kmac128,
  kmac256,
  k12,
  m14,
  turboshake128,
  turboshake256,
  tuplehash128,
  tuplehash256,
  parallelhash128,
  parallelhash256,
  keccakprg,
} from '@noble/hashes/sha3-addons';
const h7c = cshake128('abc', { personalization: 'def' });
const h7d = cshake256('abc', { personalization: 'def' });
const h7e = kmac128('key', 'message');
const h7f = kmac256('key', 'message');
const h7h = k12('abc');
const h7g = m14('abc');
const h7t1 = turboshake128('abc');
const h7t2 = turboshake256('def', { D: 0x05 });
const h7i = tuplehash128(['ab', 'c']); // tuplehash(['ab', 'c']) !== tuplehash(['a', 'bc']) !== tuplehash(['abc'])
// Same as k12/blake3, but without reduced number of rounds. Doesn't speedup anything due lack of SIMD and threading,
// added for compatibility.
const h7j = parallelhash128('abc', { blockLen: 8 });
// pseudo-random generator, first argument is capacity. XKCP recommends 254 bits capacity for 128-bit security strength.
// * with a capacity of 254 bits.
const p = keccakprg(254);
p.feed('test');
const rand1b = p.fetch(1);
  • Full NIST SP 800-185: cSHAKE, KMAC, TupleHash, ParallelHash + XOF variants
  • Reduced-round Keccak:
    • ๐Ÿฆ˜ K12 aka KangarooTwelve
    • M14 aka MarsupilamiFourteen
    • TurboSHAKE
  • KeccakPRG: Pseudo-random generator based on Keccak
ripemd160
import { ripemd160 } from '@noble/hashes/ripemd160';
// function ripemd160(data: Uint8Array): Uint8Array;
const hash8 = ripemd160('abc');
const hash9 = ripemd160
  .create()
  .update(Uint8Array.from([1, 2, 3]))
  .digest();

See RFC 2286, Website

blake2b, blake2s, blake3
import { blake2b } from '@noble/hashes/blake2b';
import { blake2s } from '@noble/hashes/blake2s';
import { blake3 } from '@noble/hashes/blake3';

const h10a = blake2s('abc');
const b2params = { key: new Uint8Array([1]), personalization: t, salt: t, dkLen: 32 };
const h10b = blake2s('abc', b2params);
const h10c = blake2s
  .create(b2params)
  .update(Uint8Array.from([1, 2, 3]))
  .digest();

// All params are optional
const h11 = blake3('abc', { dkLen: 256, key: 'def', context: 'fji' });

See RFC 7693, Website.

sha1: legacy hash

SHA1 was cryptographically broken, however, it was not broken for cases like HMAC.

See RFC4226 B.2.

Don't use it for a new protocol.

import { sha1 } from '@noble/hashes/sha1';
const h12 = sha1('def');
hmac
import { hmac } from '@noble/hashes/hmac';
import { sha256 } from '@noble/hashes/sha2';
const mac1 = hmac(sha256, 'key', 'message');
const mac2 = hmac
  .create(sha256, Uint8Array.from([1, 2, 3]))
  .update(Uint8Array.from([4, 5, 6]))
  .digest();

Matches RFC 2104.

hkdf
import { hkdf } from '@noble/hashes/hkdf';
import { sha256 } from '@noble/hashes/sha';
import { randomBytes } from '@noble/hashes/utils';
const inputKey = randomBytes(32);
const salt = randomBytes(32);
const info = 'abc';
const dkLen = 32;
const hk1 = hkdf(sha256, inputKey, salt, info, dkLen);

// == same as
import * as hkdf from '@noble/hashes/hkdf';
import { sha256 } from '@noble/hashes/sha2';
const prk = hkdf.extract(sha256, inputKey, salt);
const hk2 = hkdf.expand(sha256, prk, info, dkLen);

Matches RFC 5869.

pbkdf2
import { pbkdf2, pbkdf2Async } from '@noble/hashes/pbkdf2';
import { sha256 } from '@noble/hashes/sha2';
const pbkey1 = pbkdf2(sha256, 'password', 'salt', { c: 32, dkLen: 32 });
const pbkey2 = await pbkdf2Async(sha256, 'password', 'salt', { c: 32, dkLen: 32 });
const pbkey3 = await pbkdf2Async(sha256, Uint8Array.from([1, 2, 3]), Uint8Array.from([4, 5, 6]), {
  c: 32,
  dkLen: 32,
});

Matches RFC 2898.

scrypt
import { scrypt, scryptAsync } from '@noble/hashes/scrypt';
const scr1 = scrypt('password', 'salt', { N: 2 ** 16, r: 8, p: 1, dkLen: 32 });
const scr2 = await scryptAsync('password', 'salt', { N: 2 ** 16, r: 8, p: 1, dkLen: 32 });
const scr3 = await scryptAsync(Uint8Array.from([1, 2, 3]), Uint8Array.from([4, 5, 6]), {
  N: 2 ** 22,
  r: 8,
  p: 1,
  dkLen: 32,
  onProgress(percentage) {
    console.log('progress', percentage);
  },
  maxmem: 2 ** 32 + 128 * 8 * 1, // N * r * p * 128 + (128*r*p)
});

Conforms to RFC 7914, Website

  • N, r, p are work factors. To understand them, see the blog post.
  • dkLen is the length of output bytes
  • It is common to use N from 2**10 to 2**22 and {r: 8, p: 1, dkLen: 32}
  • onProgress can be used with async version of the function to report progress to a user.

Memory usage of scrypt is calculated with the formula N * r * p * 128 + (128 * r * p), which means {N: 2 ** 22, r: 8, p: 1} will use 4GB + 1KB of memory. To prevent DoS, we limit scrypt to 1GB + 1KB of RAM used, which corresponds to {N: 2 ** 20, r: 8, p: 1}. If you want to use higher values, increase maxmem using the formula above.

Note: noble supports 2**22 (4GB RAM) which is the highest amount amongst JS libs. Many other implementations don't support it. We cannot support 2**23, because there is a limitation in JS engines that makes allocating arrays bigger than 4GB impossible, but we're looking into other possible solutions.

argon2

Warning

Experimental: it may be removed at any time.

Argon2 RFC 9106 implementation.

import { argon2d, argon2i, argon2id } from '@noble/hashes/argon2';
const result = argon2id('password', 'salt', { t: 2, m: 65536, p: 1 });
utils
import { bytesToHex as toHex, randomBytes } from '@noble/hashes/utils';
console.log(toHex(randomBytes(32)));
  • bytesToHex will convert Uint8Array to a hex string
  • randomBytes(bytes) will produce cryptographically secure random Uint8Array of length bytes
All available imports
import { sha256, sha384, sha512, sha224, sha512_256, sha512_384 } from '@noble/hashes/sha2';
// prettier-ignore
import {
  sha3_224, sha3_256, sha3_384, sha3_512,
  keccak_224, keccak_256, keccak_384, keccak_512,
  shake128, shake256
} from '@noble/hashes/sha3';
// prettier-ignore
import {
  cshake128, cshake256,
  turboshake128, turboshake256,
  kmac128, kmac256,
  tuplehash256, parallelhash256,
  k12, m14, keccakprg
} from '@noble/hashes/sha3-addons';
import { ripemd160 } from '@noble/hashes/ripemd160';
import { blake3 } from '@noble/hashes/blake3';
import { blake2b } from '@noble/hashes/blake2b';
import { blake2s } from '@noble/hashes/blake2s';
import { hmac } from '@noble/hashes/hmac';
import { hkdf } from '@noble/hashes/hkdf';
import { pbkdf2, pbkdf2Async } from '@noble/hashes/pbkdf2';
import { scrypt, scryptAsync } from '@noble/hashes/scrypt';

import { sha1 } from '@noble/hashes/sha1'; // legacy

// small utility method that converts bytes to hex
import { bytesToHex as toHex } from '@noble/hashes/utils';
console.log(toHex(sha256('abc'))); // ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad

Security

The library has been independently audited:

It is tested against property-based, cross-library and Wycheproof vectors, and has fuzzing by Guido Vranken's cryptofuzz.

If you see anything unusual: investigate and report.

Constant-timeness

JIT-compiler and Garbage Collector make "constant time" extremely hard to achieve timing attack resistance in a scripting language. Which means any other JS library can't have constant-timeness. Even statically typed Rust, a language without GC, makes it harder to achieve constant-time for some cases. If your goal is absolute security, don't use any JS lib โ€” including bindings to native ones. Use low-level libraries & languages. Nonetheless we're targetting algorithmic constant time.

Memory dumping

The library shares state buffers between hash function calls. The buffers are zeroed-out after each call. However, if an attacker can read application memory, you are doomed in any case:

  • At some point, input will be a string and strings are immutable in JS: there is no way to overwrite them with zeros. For example: deriving key from scrypt(password, salt) where password and salt are strings
  • Input from a file will stay in file buffers
  • Input / output will be re-used multiple times in application which means it could stay in memory
  • await anything() will always write all internal variables (including numbers) to memory. With async functions / Promises there are no guarantees when the code chunk would be executed. Which means attacker can have plenty of time to read data from memory
  • There is no way to guarantee anything about zeroing sensitive data without complex tests-suite which will dump process memory and verify that there is no sensitive data left. For JS it means testing all browsers (incl. mobile), which is complex. And of course it will be useless without using the same test-suite in the actual application that consumes the library

Supply chain security

  • Commits are signed with PGP keys, to prevent forgery. Make sure to verify commit signatures.
  • Releases are transparent and built on GitHub CI. Make sure to verify provenance logs
  • Rare releasing is followed to ensure less re-audit need for end-users
  • Dependencies are minimized and locked-down:
    • If your app has 500 dependencies, any dep could get hacked and you'll be downloading malware with every install. We make sure to use as few dependencies as possible
    • We prevent automatic dependency updates by locking-down version ranges. Every update is checked with npm-diff
  • Dev Dependencies are only used if you want to contribute to the repo. They are disabled for end-users:
    • scure-base, scure-bip32, scure-bip39, micro-bmark and micro-should are developed by the same author and follow identical security practices
    • prettier (linter), fast-check (property-based testing) and typescript are used for code quality, vector generation and ts compilation. The packages are big, which makes it hard to audit their source code thoroughly and fully

Randomness

We're deferring to built-in crypto.getRandomValues which is considered cryptographically secure (CSPRNG).

In the past, browsers had bugs that made it weak: it may happen again. Implementing a userspace CSPRNG to get resilient to the weakness is even worse: there is no reliable userspace source of quality entropy.

Speed

Benchmarks measured on Apple M1 with macOS 12. Note that PBKDF2 and Scrypt are tested with extremely high work factor. To run benchmarks, execute npm run bench:install and then npm run bench

SHA256 32B x 1,219,512 ops/sec @ 820ns/op ยฑ 2.58% (min: 625ns, max: 4ms)
SHA384 32B x 512,032 ops/sec @ 1ฮผs/op
SHA512 32B x 509,943 ops/sec @ 1ฮผs/op
SHA3-256, keccak256, shake256 32B x 199,600 ops/sec @ 5ฮผs/op
Kangaroo12 32B x 336,360 ops/sec @ 2ฮผs/op
Marsupilami14 32B x 298,418 ops/sec @ 3ฮผs/op
BLAKE2b 32B x 379,794 ops/sec @ 2ฮผs/op
BLAKE2s 32B x 515,995 ops/sec @ 1ฮผs/op ยฑ 1.07% (min: 1ฮผs, max: 4ms)
BLAKE3 32B x 588,235 ops/sec @ 1ฮผs/op ยฑ 1.36% (min: 1ฮผs, max: 5ms)
RIPEMD160 32B x 1,140,250 ops/sec @ 877ns/op ยฑ 3.12% (min: 708ns, max: 6ms)
HMAC-SHA256 32B x 377,358 ops/sec @ 2ฮผs/op

HKDF-SHA256 32B x 108,377 ops/sec @ 9ฮผs/op
PBKDF2-HMAC-SHA256 262144 x 3 ops/sec @ 326ms/op
PBKDF2-HMAC-SHA512 262144 x 1 ops/sec @ 970ms/op
Scrypt r: 8, p: 1, n: 262144 x 1 ops/sec @ 616ms/op

Compare to native node.js implementation that uses C bindings instead of pure-js code:

SHA256 32B node x 1,302,083 ops/sec @ 768ns/op ยฑ 10.54% (min: 416ns, max: 7ms)
SHA384 32B node x 975,609 ops/sec @ 1ฮผs/op ยฑ 11.32% (min: 625ns, max: 8ms)
SHA512 32B node x 983,284 ops/sec @ 1ฮผs/op ยฑ 11.24% (min: 625ns, max: 8ms)
SHA3-256 32B node x 910,746 ops/sec @ 1ฮผs/op ยฑ 12.19% (min: 666ns, max: 10ms)
keccak, k12, m14 are not implemented
BLAKE2b 32B node x 967,117 ops/sec @ 1ฮผs/op ยฑ 11.26% (min: 625ns, max: 9ms)
BLAKE2s 32B node x 1,055,966 ops/sec @ 947ns/op ยฑ 11.07% (min: 583ns, max: 7ms)
BLAKE3 is not implemented
RIPEMD160 32B node x 1,002,004 ops/sec @ 998ns/op ยฑ 10.66% (min: 625ns, max: 7ms)
HMAC-SHA256 32B node x 919,963 ops/sec @ 1ฮผs/op ยฑ 6.13% (min: 833ns, max: 5ms)
HKDF-SHA256 32 node x 369,276 ops/sec @ 2ฮผs/op ยฑ 13.59% (min: 1ฮผs, max: 9ms)
PBKDF2-HMAC-SHA256 262144 node x 25 ops/sec @ 39ms/op
PBKDF2-HMAC-SHA512 262144 node x 7 ops/sec @ 132ms/op
Scrypt r: 8, p: 1, n: 262144 node x 1 ops/sec @ 523ms/op

It is possible to make this library 4x+ faster by doing code generation of full loop unrolls. We've decided against it. Reasons:

  • the library must be auditable, with minimum amount of code, and zero dependencies
  • most method invocations with the lib are going to be something like hashing 32b to 64kb of data
  • hashing big inputs is 10x faster with low-level languages, which means you should probably pick 'em instead

The current performance is good enough when compared to other projects; SHA256 takes only 900 nanoseconds to run.

Contributing & testing

  1. Clone the repository
  2. npm install to install build dependencies like TypeScript
  3. npm run build to compile TypeScript code
  4. npm run test will execute all main tests. See our approach to testing
  5. npm run test:dos will test against DoS; by measuring function complexity. Takes ~20 minutes
  6. npm run test:big will execute hashing on 4GB inputs, scrypt with 1024 different N, r, p combinations, etc. Takes several hours. Using 8-32+ core CPU helps.

Resources

Check out paulmillr.com/noble for useful resources, articles, documentation and demos related to the library.

License

The MIT License (MIT)

Copyright (c) 2022 Paul Miller (https://paulmillr.com)

See LICENSE file.

noble-hashes's People

Contributors

aulneau avatar ckniffen avatar jacogr avatar jeetiss avatar jonathan-albrecht-ibm avatar libitx avatar neil-yoga-crypto avatar paulmillr avatar pkieltyka avatar quixoten avatar systemcluster avatar webmaster128 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

noble-hashes's Issues

Please publish ts native version to jsr.io

Pretty please could we have a typescript native version on jsr.io ?

For those of us in modern js environments, this would be grand.

jsr.io has supply chain provenance from github actions as well, but you might be able to skip the build step perhaps, and webhook straight from github, putting even less stuff in the way of purity.

Thank you for the @noble libraries ๐Ÿ™

warning: Attempted to import the module

When noble-hashes are used with React Native

warning: Attempted to import the module "/Users/danielsteigerwald/dev/evolu/node_modules/@noble/hashes/crypto.js" which is not listed in the "exports" of "/Users/danielsteigerwald/dev/evolu/node_modules/@noble/hashes". Falling back to file-based resolution. Consider updating the call site or asking the package maintainer(s) to expose this API.

Ideas for next major update

  • Remove cryptoNode import! Requires to ditch support for nodejs v18 (EOL 30 Apr 2025)
  • Remove import maps. Change @noble/hashes/sha3 to @noble/hashes/sha3.js
    • Import maps are not browser-friendly etc
    • Before doing this, actually test in real browser
  • Test in different environments:
    • bundler-less browser
    • deno
    • bun
  • See how package can fit for JSR.io
  • Rename / refactor
    • crypto / cryptoNode => webcrypto (to match noble-ciphers)
    • sha256, sha512 => sha2? not sure yet
    • blake2b, blake2s, blake3 => blake?
    • ripemd160 => ripemd?
    • sha1 => legacy? and add md5??

Maybe also:

  • Make package ESM-only
  • Re-audit (need $ funds)

TextEncoder is not defined

I'm trying to use a library which depends on @paralleldrive/cuid2 which on itself depends on @noble/hashes. When I try to launch my Nuxt.js app on Node 16.19.1 with the new imported library I get this error:
image
I'm using the latest version for each library, but I've also tried to install version 1.1.5 for @noble/hashes (the earlies the dependency graph allows) but I still get the same error.
Any idea?

Jest TextEncoder missing, Uint8Array failing instanceof checks workaround

When using jsdom environment the tests sometimes fail with Uint8Array is not defined etc.

You can workaround this by setting a testEnvironment in your jest.config.js:

...
testEnvironment: `${__dirname}/commands/jest/customizedTestEnvironment.js`,
...

Impl a custom environment:

const { TestEnvironment } = require('jest-environment-jsdom')

/**
 * Jest creates a new VM context for each test and doesn't add in all 
 * the node globals. 
 * We rectify this here.
 */
class CustomizedTestEnvironment extends TestEnvironment {
  constructor(config, context) {
    super(config, context)
  }

  async setup() {
    await super.setup()

    // These seem to be needed for @noble/hashes
    this.global.TextEncoder = globalThis.TextEncoder
    this.global.Uint8Array = globalThis.Uint8Array
  }
}

module.exports = CustomizedTestEnvironment

It would be nice if didn't need to do this. May need to patch the jsdom environment?

In any case leaving this here in case it helps others.

Uint8Array check unreliable

Heyho! I came across an issue about the Uint8Array checks that are used in this library. TL;DR: instanceof Uint8Array is unrelyable in many cases.

There is a longer writeup about the problem in https://medium.com/@simonwarta/limitations-of-the-instanceof-operator-f4bcdbe7a400. A simple way to reproduce the problem is:

$ node -e 'require("repl").start().context.a = new Uint8Array([1, 2, 2, 1]);'
> a
Uint8Array(4) [ 1, 2, 2, 1 ]
> a instanceof Uint8Array
false

A more reliable way to check for Uint8Array can be found here: https://github.com/cosmos/cosmjs/blob/79396bfaa49831127ccbbbfdbb1185df14230c63/packages/utils/src/typechecks.ts#L14-L32. Wether Buffer is an Uint8Array is a bit controvesial as you can see in the code there. I think for our use case and for the sake of simplicity, Buffer should be considered an Uint8Array, right?

I'm happy to provide a PR to get this fixed.

Include JavaScript in repo, or build config that CDNs understand

I'd like to be able to use RIPEMD160 from a CDN like unpkg or cdnjs or GitHub Pages.

Would you be open to publishing a ./dist folder as part of an npm prepublish step, or something like that so that it's easier to test this out in a browser?

I see that GitHub Releases has the full bundle, but it's HUGE if all you need is one algo.

BigInt syntax errors in unsupported environments

... you are not going to like this one, bear with me :)

Ok, I started pulling this in. Basically in my case I still need to support some ancient environments as well, which is not that much of an issue, basically on my hashing wrappers I do the following -

import { blake2b as blake2bJs } from '@noble/hashes/lib/blake2b';

// wasm
import { blake2b as blake2bWasm } from '@polkadot/wasm-crypto'; 

export function blake2 (u8a: Uint8Array): Uint8Array {
  return typeof BigInt !== 'undefined'
    ? blake2bJs(u8a)
    : blake2bWasm(u8a);
}

So effectively there is some fallback in there. (The above is typed from memory as illustration, but shows the point. Normally when wasm is initialized, it actually is the preferred route in my case. The same applies to eg fallback to blake2-js when BigInt is not available)

Anyway, all good. Now however the issue is the following - on some older environments, such as for instance React Native, it even throws up when trying to parse the modules, e.g. this is problematic (although not used at runtime) https://github.com/paulmillr/noble-hashes/blob/main/src/_u64.ts#L1

I did a quick search for the 32n e.g. /\dn/ and there are actually not that many. So for instance, we could get around it with the following ...

// _64.ts 
// suggested adjustments as an example, some files will need adjusting 
// yes, it is, well ... horrid :(
export const _BigInt: (value: string | number | bigint | boolean) => bigint =
  typeof BigInt !== 'undefined'
    ? BigInt
    : () => undefined as unknown as bigint;

const U32_MASK64 = _BigInt(2 ** 32 - 1);

const _32n = _BigInt(32);

export function fromBig(n: bigint, le = false) {
  if (le) return { h: Number(n & U32_MASK64), l: Number((n >> _32n) & U32_MASK64) };
  return { h: Number((n >> _32n) & U32_MASK64) | 0, l: Number(n & U32_MASK64) | 0 };
}

The above is problematic, obviously since it will give undefined results in non-BigInt environments. In BigInt environments there is no extra overhead, well, maybe slightly at library load. However for non-BigInt-envs inside functions it does use BigInt which would throw up at runtime.

At this point the TL;DR is -

  1. I'm making sure that at runtime, the BigInt libs are not used (if not available)
  2. I still have support issue where library users now complain that it doesn't compile in their environments

So my options are -

  1. Having to debug all user environments to figure out why
  2. Documenting all of it
  3. Making a PR here with "some" hacks like above

None of the options are fun atm :) Other suggestions welcome.

Support big-endian hardware

Big-endian hardware is currently explicitly unsupported:

// big-endian hardware is rare. Just in case someone still decides to run hashes:
// early-throw an error because we don't support BE yet.
export const isLE = new Uint8Array(new Uint32Array([0x11223344]).buffer)[0] === 0x44;
if (!isLE) throw new Error('Non little-endian hardware is not supported');

As an indirect consumer of this package (via noble-curves), it would be great not have to worry whether consumers of my code want to run on big-endian hardware.

any plan to use webcrypto internally?

I love noble packages' api for their simplicity. But while I am using sha256 and pbkdf2, I notice that the source is self-implemented, not using webcrypto primitives.

Right now (May 2024), webcrypto api is adopted everywhere, especially both in browser and nodejs. Would you like to build the api based on webcrypto primitives? I am concerning the speed, and obviously using native webcrypto should be much faster than a pure js implementation.

Thanks!

Feature request: Add `md5` support

It would be nice to have a modern and dependency-free version of the md5 hash.

It is still required for non-cryptographic services such as Gravatar.

blake3 keyed hash blanks out the key uint8array

Unsure if this is intended - feels like a bug. When using blake3 in keyed hash mode, the key uint8array is blanked out. Eg:

const key = new Uint8Array([1,2,3,4,5,6,7,8,9... etc])
const hash = blake3(input, { key })
console.log(key) // => Uint8Array(32) [0,0,0,0,0,0,0,0,0... etc]

I currently work round this by copying the key each time I use it:

const hash = blake3(input, { key: new Uint8Array(key) })

npm is unable to install the package

I'm still a newbie in the JS ecosystem so please forgive me if I'm doing something dumb, but when I try to install the package I get an error:

$ npm i @noble-hashes
npm ERR! code EINVALIDTAGNAME
npm ERR! Invalid tag name "@noble-hashes" of package "@noble-hashes": Tags may not have any characters that encodeURIComponent encodes.

A different error with yarn

error Error: https://registry.yarnpkg.com/@noble-hashes: Request "https://registry.yarnpkg.com/@noble-hashes" returned a 405

Support deno

src files have imports such as import {sha2} from './sha2.js'. JS files can't be imported in Deno. TS files can't be imported in node/browser.

Another thing is crypto import in crypto.ts. Should be replaced with node:crypto - but that's easy.

Low Keccak Hash Performance

Hi Paul,
I am just testing out the Noble libraries to analyze for an eventual breaking release integration (through ethereum-cryptography).

On the hash function (keccak) I am seeing some serious performance degradation, which would be significant for us e.g. for the Trie + VM use case.

So here are some numbers I get:

The old keccak package using the Node bindings:

import { keccak224, keccak384, keccak256 as k256, keccak512 } from 'ethereum-cryptography/keccak'
console.time('p'); for (let i=1; i <= 100000; i++) { k256(Buffer.from([1,2,3])) }; console.timeEnd('p');
// p: 274.337ms

The old keccak packages using native JS (by manually switching to it in the index.js file):

console.log('k2')
module.exports = require('./js')
import { keccak224, keccak384, keccak256 as k256, keccak512 } from 'ethereum-cryptography/keccak'
console.time('p'); for (let i=1; i <= 100000; i++) { k256(Buffer.from([1,2,3])) }; console.timeEnd('p');
// p: 199.906ms

(astonishingly even faster, Apple MacBook Air M1)

The noble/hashes library:

import * as sha3_ from "@noble/hashes/sha3"
console.time('p'); for (let i=1; i <= 100000; i++) { sha3_.keccak_256(Uint8Array.from([1, 2, 3])) }; console.timeEnd('p');
// p: 568.131ms

So this would be roughly a 2x decrease and which would make it a hard decision for us to do a switch here, weighting the obvious benefits against this, since performance is also an extremely important factor for us, since we are a lot of hashing operations in the Trie library.

Am I doing everything correct here on the measuring side? I've also seen this README entry on performance, but can't easily put this into context and compare this with the existing library we are using right now.

Thanks a lot for the great work done here! ๐Ÿ™‚

tests failed with noble hashes 1.1.1

running the tests failed
this is my output of the tests on a Mac MacOSs 10.15.7

..............
โ˜† should Scrypt types:
โœ“ should Scrypt types

โ˜† should HKDF types:
โœ“ should HKDF types

โ˜† should hkdf(sha256) generator:
โ˜“ should hkdf(sha256) generator
(node:19204) UnhandledPromiseRejectionWarning: TypeError: crypto.hkdfSync is not a function
at /Users/pepsi/Downloads/noble-hashes-1.1.1/test/generator.js:73:16
at run (/Users/pepsi/Downloads/noble-hashes-1.1.1/node_modules/micro-should/index.js:14:24)
at /Users/pepsi/Downloads/noble-hashes-1.1.1/node_modules/micro-should/index.js:81:13
at processTicksAndRejections (internal/process/task_queues.js:95:5)
(Use node --trace-warnings ... to show where the warning was created)
(node:19204) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 542)
(node:19204) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
pepsis-iMac:noble-hashes-1.1.1 pepsi$

".js" imports are causing issues in Deno

For example, I am trying to use the blake2b hash in Deno by importing the following:

import { blake2b } from 'https://deno.land/x/[email protected]/blake2b.ts';

But I am getting the following issue:

error: Module not found "https://deno.land/x/[email protected]/_blake2.js".
    at https://deno.land/x/[email protected]/blake2b.ts:1:42

Changing the import statements on the first lines of blake2b.ts to use the ".ts" extension instead of ".js" would solve the issue.

typo in README

import { bytesToHex as toHex, randomBytes } from '@noble/hashes/scrypt';

should be..

import { bytesToHex as toHex, randomBytes } from '@noble/hashes/utils';

Module not found: Error: Can't resolve './_assert' ... Did you mean '_assert.js'?

When using

import { k12 } from '@noble/hashes/sha3-addons'

I ended up with the Module not found error. As a quick fix, I needed to change the file sha3-addons.js and add the .js file extension to the require statement. I'm writing this issue because I'm uncertain if this type of require without the extension is intentional or not.

Reduce Bundle Size

Hey Paul, love what you're doing here. I was hoping to switch from sha3 to this in-order to save on bundle size.

Unfortunately the keccak_256 import from noble appears to be 2x the size of the sha3 import from the sha3 library. Before I go into the code to investigate, do you know why this might be?

import { keccak_256 } from '@noble/hashes/sha3';

versus

import { Keccak } from 'sha3';

Update cause problems

After updating the @noble/hashes from 1.3.2 to 1.3.3 in our project, we had a problem with the fact that after the build, the product was detected as a virus (only on Windows systems). Rollback to 1.3.2 fixed this issue

`isPlainObject` check fails in serverless env

When trying to use some of these hashing functions in the context of a serverless environment, such as vercel or cloudfare, this function

const isPlainObject = (obj: any) =>

fails to accept this object {c:2048,dkLen:64} as a plain object.

I have followed it down to this line:

obj.constructor === Object never being true. I'd suggest something like this instead:

obj.constructor.name === 'Object'

Can't use shake256 as Keccak without casting

import { Keccak, shake256 } from '@noble/hashes/sha3'

import { PrngFn } from './types'

export function shakePrng(seed: Uint8Array): PrngFn {
  const prng = shake256.create({}) as Keccak
  prng.update(seed)
  return prng.xof.bind(prng)
}

Also a bit strange requiring empty object for the options

v1.3.0 doesn't build in React Gatsby

Description
I have a Gatsby/yarn v3 app and it doesn't build the latest bip32 and bip39 libraries.
The issue is with v1.3.0 of noble-hashes that they both depend on, more specifically with the ESM syntax import * as nc from 'node:crypto'.

image

Solution
The solution for me currently is

"resolutions": {
   "@noble/hashes": "1.2.0",
}

Add package-lock.json

Upon forking the repo, I noticed there is no package-lock.json.

All recent versions of npm generate this file and it's imperative for deterministic installs. A security recommendation for all JS projects is to include a package-lock.json [source. ]

Build error "Cannot find name 'Crypto'"

Heyho! Thank you for this initiative! It's amazing to see hashing implementations without Buffer dependencies.

I'd love to get this into CosmJS to replace first ripemd160 and maybe sha.js and js-sha3. When I tried doing this, I got the following compile error:

./../.yarn/cache/@noble-hashes-npm-0.4.1-4637b511e4-7d06789f41.zip/node_modules/@noble/hashes/lib/utils.d.ts:49:11 - error TS2304: Cannot find name 'Crypto'.

49     web?: Crypto;
             ~~~~~~


Found 1 error.

Seems like my tsconfig does not have the DOM lib. Is this enabled by default in your tsconfig?

Would it be possible to avoid the dependency on those types or pull them in explicitly?

Benchmarks are slowed down when other modules are imported

I've spotted this after benchmarks were transitioned from common.js to ESM in 88c1aa3. Check out main branch.

  • git clone [email protected]:paulmillr/noble-hashes.git && cd noble-hashes && npm install && npm run build && npm run bench:install
  • Comment-out lines 14-28 in benchmark/hashes.js, run again. sha3 would become 25% faster

We need to investigate:

  1. Why sha3 is 25% slower when other modules are imported
  2. Why sha2 is NOT slower when other modules are imported. What's special about sha3?
  3. Why everything was fine with common.js, but is slow with ESM?
  4. Bonus issue: re-exporting isBytes slows-down performance considerably. See branch https://github.com/paulmillr/noble-hashes/tree/import-perf. Probably related.

To investigate this, v8/node.js profiling, memory dumps, etc may help.

hmac check seems to not work correctly in extension background contexts

I've been playing around with using some of the noble libs in web extensions, specifically in the background context which is similar to service workers, and this line causes issues:

if (!(this.iHash instanceof Hash))

When I console.log the iHash this is what we get:

{
    "blockLen": 64,
    "outputLen": 32,
    "padOffset": 8,
    "isLE": false,
    "finished": true,
    "length": 128,
    "pos": 0,
    "destroyed": false,
    "buffer": {
       // removed for brevity
    },
    "view": {},
    "A": 0,
    "B": 0,
    "C": 0,
    "D": 0,
    "E": 0,
    "F": 0,
    "G": 0,
    "H": 0
}

When I patch-package the hmac file and remove the conditional check, it works as expected.

Curious what we could do to make it work in this context? webcrypto should be fully supported

Simple isomorphic wrapper ?

For things like ripple library where they are using createHash one needs to create little wrappers to replace existing functions, ala:

const HASH_BYTES = 32

const sha512Half = (() => {
  const isBrowser = typeof window !== 'undefined'

  if (isBrowser) {
    const { sha512 } = require('@noble/hashes/sha512')
    const { bytesToHex, hexToBytes } = require('@noble/hashes/utils')

    return function sha512Half(hex: string): string {
      const digestHalf = sha512(hexToBytes(hex)).slice(0, HASH_BYTES)
      return bytesToHex(digestHalf).toUpperCase()
    }
  } else {
    const crypto = require('crypto')

    return function sha512Half(hex: string): string {
      const hash = crypto.createHash('sha512')
      hash.update(Buffer.from(hex, 'hex'))
      const digestHalf = hash.digest().slice(0, HASH_BYTES)
      return digestHalf.toString('hex').toUpperCase()
    }
  }
})()

to replace:

const HASH_SIZE = 64

function sha512Half(hex: string): string {
  return createHash('sha512')
    .update(Buffer.from(hex, 'hex'))
    .digest('hex')
    .toUpperCase()
    .slice(0, HASH_SIZE)
}

This is annoying, and less than ideal because you have to mess with your webpack resolve.alias rather than rely upon a browser field in package.json.

I think a simple wrapper like this would go along way:

type CreateHashFn = (algo: 'sha512' | 'sha256' | ... ) => {
  update(buffer: Uint8Array): void
  digest(): Uint8Array
}

Are you interested in doing something like this under @noble/ ?

The new v0.4.2 seems to break imports from .mjs (Node.js ES module files)

I am running the latest version of Node (v17.2.0) on Intel Mac OS X.

I have an ES Node.js package (with the "type":"module" flag), which is equivalent to having the code in an .mjs file.

Up until version 0.4.1, I used to do the following to use this package :

import { keccak_256 } from '@noble/hashes/lib/sha3';
console.log(keccak_256(new Uint8Array([1, 2, 3])));

With version 0.4.2, I am now getting the following error :

import { keccak_256 } from '@noble/hashes/lib/sha3';
         ^^^^^^^^^^
SyntaxError: Named export 'keccak_256' not found. The requested module '@noble/hashes/lib/sha3' is a CommonJS module, which may not support all module.exports as named exports.
CommonJS modules can always be imported via the default export, for example using:

import pkg from '@noble/hashes/lib/sha3';
const { keccak_256 } = pkg;

    at ModuleJob._instantiate (node:internal/modules/esm/module_job:127:21)
    at async ModuleJob.run (node:internal/modules/esm/module_job:191:5)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:331:24)
    at async loadESM (node:internal/process/esm_loader:88:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12)

Node.js v17.2.0

I tried the suggestion :

import pkg from '@noble/hashes/lib/sha3';
const { keccak_256 } = pkg;
console.log(keccak_256(new Uint8Array([1, 2, 3])));

But am getting the following error :

(node:17801) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/Users/[redacted]/node_modules/@noble/hashes/lib/esm/sha3.js:1
import * as u64 from './_u64';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1026:15)
    at Module._compile (node:internal/modules/cjs/loader:1061:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1149:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:190:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:331:24)

Node.js v17.2.0

Changing my file extension to .cjs and using the old require() syntax works, but that somewhat defeats the purpose of trying to move to the new ES Module syntax.

This works (when running from .cjs file) :

const { keccak_256 } = require('@noble/hashes/lib/sha3');
console.log(keccak_256(new Uint8Array([1, 2, 3])));

Is there any possibilty to make this module work from an .mjs Node.js file ?

Package subpath './_assert not exposed in exports.

'./_assert' is not exposed in the package.json exports fields. This is causing js-ethereum-cryptography, which calls it here to fail on install in my docker. I'm guessing its just a matter of adding it back to the export field/reverting something, as js-ethereum-cryptography and noble-hashes were working properly until I reinstalled today.

The `noble/scrypt` may incompatible with the WEB3 SECRET STORAGE DEFINITION

noble-hashes/src/scrypt.ts

Lines 110 to 118 in ba9d92f

const blockSize = 128 * r;
const blockSize32 = blockSize / 4;
if (N <= 1 || (N & (N - 1)) !== 0 || N >= 2 ** (blockSize / 8) || N > 2 ** 32) {
// NOTE: we limit N to be less than 2**32 because of 32 bit variant of Integrify function
// There is no JS engines that allows alocate more than 4GB per single Uint8Array for now, but can change in future.
throw new Error(
'Scrypt: N must be larger than 1, a power of 2, less than 2^(128 * r / 8) and less than 2^32'
);
}

{
    "kdfparams": {
      "dklen": 32,
      "n": 262144,
      "p": 8,
      "r": 1,
      "salt": "ab0c7876052600dd703518d6fc3fe8984592145b591fc8fb5c6d43190334ba19"
    }
}

Test vector from the Ethereum developer community

the cost 262144 here is over 65535(2 ** (blockSize / 8)

Request for twox/xxhash

First off, this is impressive - I have early branches switching to this polkadot-js/common#1188 (also planning on doing secp256k1 & ed25519 - the first also has a PR, the second haven't gotten around to). Anyway, in that bracnch a couple of operations actually beat out Rust WASM. So certainly impressive.

One of the reasons for switching to the @noble family (provided I can get the BigInt story right, have a number of React Native users) is the drops in dependencies, size and speed. Finally a library that follows recent ES standards without resorting to Buffer everywhere.

One of my hashing requests would be for xxhash, I'm currently using https://www.npmjs.com/package/xxhashjs and overall quite happy, but once again pulls in yet-another-big-numbers library. I'm guessing this is quite niche for the target audience, however xxhash is used extensively in the Polkadot/Substrate ecosystem in the runtime, so it would certainly be a welcome addition from this side of the fence.

Lots of imports required to type a hash

From some code I was working on recently

import { sha512, SHA512 } from '@noble/hashes/sha512'
import { Hash } from '@noble/hashes/utils'

export default class Sha512 {
  hash: Hash<SHA512>

In this case could simply do hash = sha512.create() but
Maybe it's fine, but it seems a bit involved?

Thoughts?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.