Git Product home page Git Product logo

wycheproof's Introduction

Project Wycheproof

https://github.com/google/wycheproof

Project Wycheproof is named after Mount Wycheproof, the smallest mountain in the world. The main motivation for the project is to have a goal that is achievable. The smaller the mountain the more likely it is to be able to climb it.

[TOC]

Introduction

Project Wycheproof tests crypto libraries against known attacks. It is developed and maintained by members of Google Security Team, but it is not an official Google product.

At Google, we rely on many third party cryptographic software libraries. Unfortunately, in cryptography, subtle mistakes can have catastrophic consequences, and we found that libraries fall into such implementation pitfalls much too often and for much too long. Good implementation guidelines, however, are hard to come by: understanding how to implement cryptography securely requires digesting decades' worth of academic literature. We recognize that software engineers fix and prevent bugs with unit testing, and we found that cryptographic loopholes can be resolved by the same means.

These observations have prompted us to develop Project Wycheproof, a collection of unit tests that detect known weaknesses or check for expected behaviors of some cryptographic algorithm. Project Wycheproof provides tests for most cryptographic algorithms, including RSA, elliptic curve crypto and authenticated encryption. Our cryptographers have systematically surveyed the literature and implemented most known attacks. We have over 80 test cases which have uncovered more than 40 bugs. For example, we found that we could recover the private key of widely-used DSA and ECDHC implementations.

While we are committed to develop as many attacks as possible, Project Wycheproof is by no means complete. Passing the tests does not imply that the library is secure, it just means that it is not vulnerable to the attacks that Project Wycheproof tests for. Cryptographers are also constantly discovering new attacks. Nevertheless, with Project Wycheproof developers and users now can check their libraries against a large number of known attacks, without having to spend years reading academic papers or become cryptographers themselves.

For more information on the goals and strategies of Project Wycheproof, please check out our documentation.

Coverage

Project Wycheproof has tests for the most popular crypto algorithms, including

  • AES-EAX
  • AES-GCM
  • ChaCha20-Poly1305
  • DH
  • DHIES
  • DSA
  • ECDH
  • ECDSA
  • EdDSA
  • ECIES
  • HKDF
  • HMAC
  • RSA
  • X25519, X448

The tests detect whether a library is vulnerable to many attacks, including

  • Invalid curve attacks
  • Biased nonces in digital signature schemes
  • Of course, all Bleichenbacherโ€™s attacks
  • And many more -- we have over 80 test cases

Our first set of tests are written in Java, because Java has a common cryptographic interface. This allowed us to test multiple providers with a single test suite. While this interface is somewhat low level, and should not be used directly, we still apply a "defense in depth" argument and expect that the implementations are as robust as possible. For example, we consider weak default values to be a significant security flaw. We are converting as many tests into sets of test vectors to simplify porting the tests to other languages. We provide ready-to-use test runners for Java Cryptography Architecture providers such as Bouncy Castle, Spongy Castle, the Amazon Corretto Crypto Provider and the default providers in OpenJDK.

Usage

Setup

git clone https://github.com/google/wycheproof.git

Execute tests

  • To test latest stable version of Bouncy Castle:
bazel test BouncyCastleAllTests
  • To test other versions, e.g., v1.52:
bazel test BouncyCastleAllTests_1_52
  • To test all known versions (warning, will take a long time):
bazel test BouncyCastleAllTests_*
  • To test a local jar, set the WYCHEPROOF_BOUNCYCASTLE_JAR environment variable:
$ WYCHEPROOF_BOUNCYCASTLE_JAR=/path/to/bouncycastle
$ bazel test BouncyCastleTestLocal
$ bazel test BouncyCastleAllTestsLocal

Note: Bazel does not currently invalidate the build on environment changes. If you change the WYCHEPROOF_BOUNCYCASTLE_JAR environment variable, run bazel clean to force a rebuild:

$ WYCHEPROOF_BOUNCYCASTLE_JAR=/path/to/bouncycastle
$ bazel test BouncyCastleTestLocal
$ WYCHEPROOF_BOUNCYCASTLE_JAR=/path/to/other/jar
$ bazel clean
$ bazel test BouncyCastleTestLocal
  • To test Spongy Castle, replace BouncyCastle with SpongyCastle in your commands, for example:
bazel test SpongyCastleAllTests
bazel test AccpAllTests
  • To test a local jar for the Amazon Corretto Crypto Provider, set the WYCHEPROOF_ACCP_JAR environment variable:
$ WYCHEPROOF_ACCP_JAR=/path/to/accp
$ bazel test AccpTestLocal
$ bazel test AccpAllTestsLocal

Note: bazel does not currently invalidate the build on environment changes. If you change the WYCHEPROOF_ACCP_JAR environment variable, run bazel clean to force a rebuild:

$ WYCHEPROOF_ACCP_JAR=/path/to/accp
$ bazel test AccpTestLocal
$ WYCHEPROOF_ACCP_JAR=/path/to/other/jar
$ bazel clean
$ bazel test AccpTestLocal
  • To test your current installation of OpenJDK:
bazel test OpenJDKAllTests

Note that OpenJDKAllTests expects that OpenJDK is your default JDK, so it might refuse to run or its results might be incorrect if you are using some other JDK. If you downloaded your JDK from Oracle or https://java.com, you're probably using Oracle JDK, which should be compatible with OpenJDK, thus the tests should run correctly.

Some tests take a very long time to finish. If you want to exclude them, use BouncyCastleTest, SpongyCastleTest or OpenJDKTest -- these targets exclude all slow tests (which are annotated with @SlowTest).

Most test targets are failing, and each failure might be a security issue. To learn more about what a failed test means, you might want to check out our documentation or the comments on top of the corresponding test function and test class.

Hall of Bugs

Here are some of the notable vulnerabilities that are uncovered by Project Wycheproof:

  • OpenJDK's SHA1withDSA leaks private keys > 1024 bits

    • Test: testBiasSha1WithDSA in DsaTest.
    • This bug is the same as CVE-2003-0971 ("GnuPG generated ElGamal signatures that leaked the private key").
  • Bouncy Castle's ECDHC leaks private keys

    • Test: testModifiedPublic and testWrongOrderEcdhc in EcdhTest.

Maintainers

Project Wycheproof is maintained by:

  • Daniel Bleichenbacher
  • Thai Duong
  • Emilia Kasper
  • Quan Nguyen
  • Charles Lee

Contact and mailing list

If you want to contribute, please read CONTRIBUTING and send us pull requests. You can also report bugs or request new tests.

If you'd like to talk to our developers or get notified about major new tests, you may want to subscribe to our mailing list. To join, simply send an empty mail to [email protected].

wycheproof's People

Contributors

alex avatar angea avatar bdonlan avatar bleichen avatar brandonweeks avatar chuckx avatar criadoperez avatar cryptosubtlety avatar ekasper avatar filosottile avatar gusted avatar jeanpaulgalea avatar jordanmews avatar jurajsomorovsky avatar lgarron avatar peterdettman avatar reaperhulk avatar salusasecondus avatar thaidn avatar tienthanh411 avatar wwjd228 avatar xenide avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wycheproof's Issues

License Citation and Using Wycheproof Tests

If we wish to use the Wycheproof tests in another library, is it sufficient to create a README.md file in the directory containing the JSON files that includes the Apache2-required notice and copyright statement?

EdDSA tests

I would love to be able to run this against my Java EdDSA library, which implements the JCA and will have a Provider class in the next release. But we'd need the test cases first ๐Ÿ˜„

Test exists with a failure, but tests log has no details

Hi,
I ran a local test of a bouncycastle 1.56 jar with the command: bazel test BouncyCastleTestLocal.
But I'm unable to make any sense of the test log. Please can you shed some light on this:

exec ${PAGER:-/usr/bin/less} "$0" || exit 1
-----------------------------------------------------------------------------
JUnit4 Test Runner
..........

Use of deprecated "set" in "rules_closure"

I can't run the tests because of the following error:

> bazel test OpenJDKAllTests
Extracting Bazel installation...
......
ERROR: /home/bertrand/.cache/bazel/_bazel_bertrand/684ed2cea9e9fd42af2238d84707407e/external/io_bazel_rules_closure/closure/private/defs.bzl:27:16: The function 'set' has been removed in favor of 'depset', please use the latter. You can temporarily refer to the old 'set' constructor from unexecuted code by using --incompatible_disallow_uncalled_set_constructor=false
ERROR: error loading package '': Extension file 'closure/private/defs.bzl' has errors
ERROR: error loading package '': Extension file 'closure/private/defs.bzl' has errors
INFO: Elapsed time: 1.473s
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Couldn't start the build. Unable to run tests

Unfortunately, adding the --incompatible_disallow_uncalled_set_constructor=false option doesn't help.

> bazel version
Build label: 0.9.0
Build target: bazel-out/k8-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Dec 19 09:31:58 2017 (1513675918)
Build timestamp: 1513675918
Build timestamp as int: 1513675918

"Acceptable" inputs are difficult to assert on (missing flags?)

Wycheproof defines a tri-state result value, with "acceptable" alongside "valid" and "invalid". It would be nice to programmatically dispatch on these so implementations can test that they accept or reject them as they expect.

It looks like the flags field could be used here, if that was not already the intent. I started looking at using them in BoringSSL, and it's almost what we'd need, but some tests are under-annotated. The model I'm thinking is that our GetWycheproofResult function would take a list of flags we consider acceptable. If any (all?) of a test case's flags fall in that set, we'd map "acceptable" to "valid", otherwise "invalid".

Is this close to what you all had in mind? In that case, these are the "acceptable" inputs that lack flags:

  • aes_eax_test.json - The "small IV size" and "IV size == 0" tests
  • aes_gcm_test.json - The "small IV sizes" tests
  • kw_test.json - All "acceptable" tests
  • kwp_test.json - Ditto
  • x25519_test.json - Most tests do indeed have flags now, but the "public key >= p" and "private key == 1 (mod order) on twist" tests are missing flags. Additionally, while most "public key with low order" tests have the "Small public key" annotation, some are flags-less. ("Small public key" is also a different style from the other flag names. "SmallPublicKey", perhaps?)

logo design proposal

Hi, I've designed a logo for the project. If you liked it i will send you the files.

logo
wycheproof-01-01

logotype 1
wycheproof-02

logotype 2
wycheproof-03

Best regards,
Tobaloidee

Installation with OS

Installation and setup guide with debain ,CENTOS
any installation guide or video available to setup wycheproof?

Best Regards,
cheri3a

AES/CBC/PKCS5Padding tests

AES-CBC is still (unfortunately) very popular. What do people think about tests targeting AES/CBC/PKCS5Padding. Possible test cases would include:

  • Various bad paddings
    • > 16 bytes
    • Not all padding bytes the same
    • Last byte is 0
  • Non-constant time unpadding (how much do we care?)
  • Properly generates a random IV if no AlgorithmParameterSpec provided to init()
  • Properly rejects IVs of incorrect length
  • ???

ChaCha20-Poly1305 large test vectors

Performance optimized implementations of ChaCha20-Poly1305 tend to use conditional areas of code that use CPU intrinsics to operate on larger ciphertext streams.
(SSSE3 intrinsics for 256 byte blocks, AVX2 intrinsics for 512 byte blocks)

It would be good to see some large test vectors that will target this, at least one in the >=256 n <512 range, and at least one in the >=512 range. This will demonstrate any implementation bugs in these optimized paths.

How to run Javascript tests?

Hello! The README clearly states how I can run the Java tests with Bazel, but it does not say how to run the Javascript implementation of the tests.

So, how do I do it?

Thanks!

Zero-length KWP keys should set 'invalid' result

I noticed the KWP test vector file (kwp_test.json) has 3 test cases where key length (where key is 'key to be wrapped') is set to 0 and msg = ''. These cases are tcId 11, 86 and 171. I think these cases should be recorded with result set to 'invalid'. They currently have result set to 'acceptable'.

According to NIST SP 800-38F, length of the KWP key to be wrapped must be at least 1 byte. The language in SP 800-38F Sec 5.2 states this as: "KW-AE and TKW-AE are defined on two or more semiblocks. For KWP-AE, the domain of possible inputs is extended to nonempty octet strings." Wrapping a key with 0 length would be invalid.

Distinguish ECDSA malleability?

Libsecp256k1 implements a variant of ECDSA used by the Bitcoin system (and many other things now) which eliminates malleability by requiring s to be in the lower half of its range. Callers that need compatible (but vulnerable...) behaviour are instructed by the documentation to call a provided normalize function on signatures before validating them.

Someone opened an issue reporting that secp256k1 ecdsa sha256 test case 218 fails. I confirmed that this failure is due exclusively to this restriction.

You can expect other secp256k1 implementations to implement similar things as this behaviour is required for consensus consistency with Bitcoin, so it might be useful to avoid gratuitously triggering this incompatibility, and flag tests that necessarily do so (e.g. because they are intentionally trying to use high S values).

Thanks for the interesting work. Although we have very extensive testing already, I think a diverse test cases created by a different set of minds will be a valuable addition to our testing arsenal.

You may be interested in a different test strategy, implemented by our "exhaustive" tests based on the observation the virtually all code in an ECC library is independent of b in the curve equation (usually only point decompression/validation depends on it). We use this fact to allow a test harness to build an alternative version of the library that operates using a different curve equation on a small sgroup (e.g. of size 43) where we can do things like exhaustively test group operations. We've shown this test to be effective at detecting some real but otherwise incredibly hard to detect bugs, such as cases of incorrectly handled group law incompleteness. As a result it might be interesting to publish vectors for these easily tested degenerate groups.

JUnit3 runner is being used instead of JUnit4

The JUnit3 runner is being used instead of JUnit4.
This causes the annotations (Slow/Fast) to be ignored.
I have a fix and will submit a pull request shortly.

There was a timeout issue reported on the mailing list in Feb. It was identified then that SlowTest annotations were being ignored. I believe this was likely the cause of that issue.

Repro Steps:

  • In junit-4.12.jar!/org/junit/internal/builders/AllDefaultPossibilitiesBuilder.class, set a breakpoint in the return runner line of the runnerForClass method
  • Set a breakpoint inside of testWrappedAroundCounter (a SlowTest)
  • Run a test with debugger attached
    i.e. bazel test --java_debug --genrule_strategy=standalone --spawn_strategy=standalone BouncyCastleTest_1_57

Actual Result:

  • The breakpoint will show JUnit38ClassRunner being initialized
  • The breakpoint in testWrappedAroundCounter will be hit

Expected Result:

  • A JUnit4 runner is initialized
  • testWrappedAroundCounter does not run during a 'Fast' test

Fix:

  • Remove extends TestCase which triggers usage of the JUnit3 runner
  • Replace import static org.junit.Assert.assertArrayEquals; with import static org.junit.Assert.*;
  • Replace import junit.framework.TestCase; with import org.junit.Test;
  • Annotate all existing test methods (that are prefixed with 'test' and have no args) with @Test

Verify Fix:

  • Use same breakpoints as before in AllDefaultPossibilitiesBuilder and AesGcmTest.
  • Run the same 'Fast' test
  • The breakpoint in AllDefaultPossibilitiesBuilder.runnerForClass will show runner as BlockJunit4ClassRunner (instead of JUnit38ClassRunner)
  • The breakpoint in testWrappedAroundCounter will not be hit.

DsaTest.testTiming() could use a warmup

I observed a lot of variability in the results from this DSA timing test. Sigma values on the same machine could vary by over 2 sigma. When looking at the individual timing results over multiple runs, it was clear there is a startup cost that is interfering with the check.

The test starts off with getCurrentThreadCpuTime() differences of:
3496000
1320000
1220000
1171000
1054000
1065000
1009000
968000
987000
1094000
1173000
1171000

After the first 100, we see the difference has narrowed:
387000
381000
362000
338000
348000
334000
348000

After 260 iterations, the values finally reach their lowest level:
175000
177000
165000
167000
171000
164000
162000
163000
171000
177000
173000
164000
166000

Now we are getting the true timing of the operation. Removing just the first 300 iterations halved the sigma values returned by the test and it was more consistent run over run. It is my belief that the variability of sigma values is a result of larger K values being randomly chosen more in the early runs that skews the sigma. These timing differences are likely caused by transitioning to math intrinsics from java code and/or the large allocation/GC related to the 50k BigIntegers and 50k longs just before the testing loop. Like many performance tests, there is a warmup sequence to remove startup costs.

The simplest fix for this is to throw away the first 300 results.

This is starting at line 516 of wycheproof/java/com/google/security/wycheproof/testcases/DsaTest.java

+   // To reduce noise in this test, a large sample is taken and a warmup
+   // to eliminate startup costs.
-    // The timings below are quite noisy. Thus we need a large number of samples.

+   int loops = 50300;
     int samples = 50000;
     int j = 0;
     long[] timing = new long[samples];
     BigInteger[] k = new BigInteger[samples];
+     for (int i = 0; i < loops; i++) {
-     for (int i = 0; i < samples; i++) {
         long start = bean.getCurrentThreadCpuTime();
         signer.update(messageBytes);
         byte[] signature = signer.sign();
+        long end = bean.getCurrentThreadCpuTime();
+        // Skip first 300 for warmup
+        if (i >= 300) {
+            timing[j] = end - start;
+            k[j++] = extractK(signature, h, priv, false);
+        }
-       timing[i] = bean.getCurrentThreadCpuTime() - start;
-       k[i] = extractK(signature, h, priv, false);
     }

Flag "format" vs. "data" failures in testvectors

For algorithms which take in an encoded object (such as (EC)DSA signatures) it would be helpful to flag those test cases which are wrong because the encoding is wrong rather than because the encoded data is wrong. This is partially done with the "BER" flag, but that only covers alternate (but still vaguely sane) encodings.

The specific use-case is I have some (EC)DSA logic I wish to test which takes in raw (r,s) values and thus it makes sense for me to simply skip any cases which do not test the underlying values/math. Instead, the various encoding related tests simply test my test-harness' parsing logic, which isn't helpful.

Support for ChaCha20 testvectors?

Hi there!

I'm opening this issue to see if there's appetite for supporting ChaCha20 test vectors (i.e. just the block cipher, not a composed AEAD).

As justification:

  • ChaCha20 is specified in RFC 7539 with a 96/32 nonce/counter split;
  • ChaCha20 is also originally specified here, and doesn't directly specify a nonce/counter split; a common split is 64/64 nonce/counter.

The variance between the two has produced some discrepancies between major implementations of ChaCha20. From a survey:

  • OpenSSL takes a 32-bit counter as input, but rolls over to the next 32 bits (in the nonce space) on counter overflow. Source.
  • LibreSSL takes a 64-bit counter as input, and increments the high 32 bits on overflow of the lower bits. Source
  • BoringSSL takes a 32-bit counter as input, and only uses those 32-bits. On overflow the counter wraps around. Source

Of the three, BoringSSL's implementation is the only one that strictly follows the RFC.

Naively, this should only cause issues at the the 2^32 block boundary, assuming that the counter starts at 0 or 1. However, starting at 0 or 1 is not standardized, and some ChaCha20 APIs assume that the entire 16-byte nonce+counter input space is randomly initialized instead. This means that the initial counter value may be significantly closer to the rollover point than 2^32 blocks, causing the different implementations to diverge after fewer enciphered blocks than might otherwise be expected.

CC @davidben, who I've been nagging about this ๐Ÿ™‚

OpenJDK tests no longer run on latest JDK versions

The OpenJDK test setup code removes all security providers and then tries to re-add specific ones based on dynamic class loading. This no longer works with the latest JDKs because (a) some of the provider classes have moved/disappeared and (b) access to others is blocked by the module system.

Check for more kinds of timing correlation in (EC)DSA

The DSA timing test in DsaTest.testTiming() (and the equivalent ECDSA test) currently looks for small k values that are correlated with short timings. I would like to suggest the following enhancements:

  1. Look for correlation between large k values and longer timings. I could imagine countermeasures that introduce some artificial delay, but not enough to cover the time of the longest executions. If the test had another loop that looks at increasingly small groups of executions with large timings, then it would be able to identify problems like this.
  2. Look for inverse correlation between timing and the size of k. I have encountered countermeasures that introduce a large amount of delay, but only when k is small. So smaller k values (below some threshold) actually produce longer timings.

Separate tests by component

Not only separate tests by algorithm, but also by component. This will make it easier to test providers which only supply some Services (such as Cipher or Signature) but not others (such as KeyFactory).

BazelTestRunner exiting with a return value of 1

My system information:
fresh Ubuntu 16.10
I have been following these commands:

$ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer
$ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
$ curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
$ sudo apt-get update && sudo apt-get install bazel
$ sudo apt-get upgrade bazel
$ git clone https://github.com/google/wycheproof.git
$ cd wycheproof

fixed other errors (Java Security: Illegal key size or default parameters) by using https://stackoverflow.com/questions/6481627/java-security-illegal-key-size-or-default-parameters

Now I still get some errors. The full output of the logfile can be found here: http://pastebin.com/hEQAbrDi
Any idea's?

Please update ecdh.md

Hi,
First of all I want to commend the nice job you all are doing.
Just wanted to inform you that bouncy castle (at least the GitHub version) is no longer vulnerable to the ecdh bug (the one that has to do with multiplying keys without first verifying they are on the same curve).

You can refer to the source

https://github.com/bcgit/bc-java/blob/d159aa822b3325aa391380c1825e9a4deecbe629/core/src/main/java/org/bouncycastle/crypto/agreement/ECDHBasicAgreement.java#L57

Regards.

Repeated input values for primality test

I noticed some values are checked more than once by the primality test, and don't see any obvious reason for the test to repeat for just those inputs. The values 164280218643672633986221, 318665857834031151167461, and 360681321802296925566181 all appear multiple times in the test data in https://github.com/google/wycheproof/blob/master/java/com/google/security/wycheproof/testcases/BigIntegerTest.java

Maybe they can be removed, or maybe there should be a comment explaining the duplication.

Thanks for this great set of tests.

ecdsa_secp256r1_sha384_test.json, ecdsa_secp256r1_sha512_test.json, and ecdsa_secp384r1_sha256_test.json are missing

These unusual combinations of P-256 + {SHA-384, SHA-512} and P-384 + SHA-256 are supported in some crypto libraries, for backward compatibility with some TLS implementations. However, there are no test vectors for these combinations. It would be great to include the test vectors for these in the open source release. (P-256 + SHA-512 is arguably even a useful combination since SHA-512 is faster than SHA-256 on many platforms, and probably stronger, even/especially when truncated in half.)

A few KW tests in v1 folder marked "acceptable" violate spec for minimum plaintext length

I found 3 KW test cases in the newer "testvectors_v1" folder that need review of the "result" flag enumeration. These test cases are in testvectors_v1/aes_wrap_test.json, test cases tcId 12, 54 and 109. These test cases each violate the minimum required plaintext length for KW in SP 800-38F Sec 5.2, which states KW plaintext (i.e. the key to be wrapped) must be a minimum of 2 semiblocks in length (which is 16 bytes). I believe these test cases should update the result flag from 'acceptable' to 'invalid'.

Relevant snippet from SP 800-38F Sec 5.2: "KW-AE and TKW-AE are defined on two or more semiblocks."

No RsassaPkcs1Generate tests in testvectors_v1

I started looking at porting BorinSSL to the testvectors_v1 format, but it looks like there's no equivalent of rsa_sig_gen_misc_test.json (type RsassaPkcs1Generate) in the testvectors_v1 set. The only tests I can find are signature verification. Is that intentional?

Please export a jar

Please modify the build system (and ideally source layout) such that it is easy to build a JAR which other tests can easily depend on.

rsa_signature_tests.json uses 1024-bit keys for all tests

If the implementation properly rejects all small RSA keys then rsa_signature_tests.json is useless: all negative test vectors will pass since the key will be rejected for being too small, and both positive test vectors will fail for the same reason.

In the case of ring, 2048-bit keys are the minimum size accepted and I expect this to become more common.

Are tcIds stable?

Hi, are the tcId values within each file stable? For example, when I fix a bug in a library, would it make sense to reference the tcId and the filename of the test? Or do they get renumbered when they are updated?

Fix Bazel Dependency

I'm having trouble running wycheproof with a modern version of bazel as it complains that http_archive isn't defined. I ended up installing version 0.15 to get everything working.

Update public GitHub repo with latest version available

I notice that it has been almost two years since this repo had a commit added to it. Is it possible that there might be a non-public version of this repo that has been updated more recently, from which this public repo could be updated? I am particularly hoping there are new test vectors.

And/or, could you otherwise indicate whether (a) this is the latest version, (b) this repo is still supported? I noticed that there is one open approved PR that has been waiting to be merged for over a year.

X448 vectors?

Are there any plans to introduce test vectors for x448?

ECDSA test vectors and docs

ECDSA tests implemented so far do not cover different combinations of standard elliptic curves with hash functions. In particular, the code we would like to test uses SHA-384 with secp384r1.
There is also no documentation regarding how these test vectors were created and what sources were used.

Error: no such target '//:java/com/google/security/wycheproof/testcases/SpongyCastleTest':

Hi,

I am trying to setup wycheproof on windows 8 machine and getting below mentioned error:

ERROR: no such target '//:java/com/google/security/wycheproof/testcases/SpongyCastleTest':
target 'java/com/google/security/wycheproof/testcases/SpongyCastleTest' not declared in package '' defined by C:/Softy/wycheproof-master/wycheproof-master/BUILD.
INFO: Elapsed time: 0.707s
ERROR: Couldn't start the build. Unable to run tests.

Note:

  1. Tried navigating inside "\wycheproof-master\wycheproof-master\java\com\google\security\wycheproof>" but still not resolved
  2. Tried building bazel and got stuck with below data:
    **$ bash C:/Users/west/Downloads/bazel-0.4.3/bazel-0.4.3/compile.sh
    INFO: You can skip this first step by providing a path to the bazel binary as second argument:
    INFO: C:/Users/west/Downloads/bazel-0.4.3/bazel-0.4.3/compile.sh compile /pat h/to/bazel
    ๐Ÿƒ Building Bazel from scratch
    ERROR: Must specify PROTOC if not bootstrapping from the distribution artifact

NOTE: This failure is likely occuring if you are trying to bootstrap bazel from
a developer checkout. Those checkouts do not include the generated output of
the protoc compiler (as we prefer not to version generated files).

  • To build a developer version of bazel, do

    bazel build //src:bazel

  • To bootstrap your first bazel binary, please download a dist archive from our
    release page at https://github.com/bazelbuild/bazel/releases and run
    compile.sh on the unpacked archive.

The full install instructions to install a release version of bazel can be found
at https://bazel.build/versions/master/docs/install.html#compiling-from-source
For a rationale, why the bootstrap process is organized in this way, see
https://bazel.build/designs/2016/10/11/distribution-artifact.html
--------------------------------------------------------------------------------**
3. Tried this also: C:\Users\west\Downloads\bazel-0.4.3\bazel-0.4.3>bash ./compile.sh
INFO: You can skip this first step by providing a path to the bazel binary as se
cond argument:
INFO: ./compile.sh compile /path/to/bazel
โ–’โ–’โ–’โ–’ Building Bazel from scratchFile not found - *.jar
File not found - *.jar
no error prone jar

Please suggest how to resolve above issue or any other guided way to start using wycheproof for analyzing crypto libraries

Best Regards,
Sunil

DsaTest.testModifiedSignatures doesn't report errors

DsaTest.testModifiedSignatures calls testVectors (source) with isValidDER: false and isValidBER: true. In such a case, errors++ will only be called if verifier.verify raises an exception different from SignatureException. As a result, non-DER signatures that are accepted by the provider (Sun or BouncyCastle) are not reported by this Wycheproof test.

For instance, the last signature in MODIFIED_SIGNATURES (source) seems to still be accepted as valid by the Sun provider on Java 1.8.0u162.

Can you confirm this? It looks like the if-else structure in testVectors (source) could be extended to account for those non-DER signatures that are accepted and verified by the provider.

Minor feature request: unify JWK representations in JSON test vectors

I have recently been importing a subset of the Wycheproof test vectors into the test suite for our in-house JWT library. In doing so, the JWK key format has been the most useful for me as we already have utilities to parse these and we can then obtain the relevant JWA algorithm identifiers from them (thank you so much for that!). However, a really very minor grumble is that there is some inconsistency across the test suites around the names used for these JWK format keys that makes it just very slightly harder to write generic code to extract them:

  • Some tests don't have a JWK version of the key at all
  • When there is one key, some tests use the field name "jwk", others use "keyJwk", and others again use "privateKeyJwk"
  • When a public key is also present, then the naming switches to "public" and "private", but the format of these could be a JWK or could be a hex-encoded ASN.1 key or a PEM. This seems to depend on the naming of the test vector file.

I appreciate this is some seriously pedantic nitpicking, but I wonder if you'd be open to a pull request that attempts to unify the use of JWKs across the test suites?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.