Please make it possible to run tests on local jars about wycheproof HOT 13 CLOSED

I think that far more people would like/need to test the stuff they have locally, compared to those who want to test what's in the global repo. Especially since it's trivial to download and point wycherproof at the downloaded stuff for the case when the "official" distro needs testing.

Thanks!

from wycheproof.

Emilia, could you please take a look? Not urgent, but when you have cycles.

from wycheproof.

In the absence of an actual test target, you can at least edit the WORKSPACE file to configure a jar as a local file:

http_jar(
    name = "bouncycastle_1_56",
    url = "file:///Users/peter/Downloads/bcprov-jdk15on-156.jar"
)

You could just use an existing target and point its jar locally as above, or edit build_defs.bzl to add a separate target referencing the jar.

from wycheproof.

Aw
aresome

from wycheproof.

I must be dense...

I've edited WORKPSPACE file (just removing the 1.48-1.54 related stuff at the top):

#maven_jar(
#    name = "bouncycastle_1_55",
#    artifact = "org.bouncycastle:bcprov-jdk15on:1.55",
#)

http_jar(
    name = "bouncycastle_1_56",
    url = "file:///Library/Java/Extensions/bcprov-ext-jdk15on-156.jar"
)
. . . . .

My attempt to do bazel test BouncyCastleAllTests produced:

$ bazel test BouncyCastleAllTests
Warning: ignoring http_proxy in environment.
..............
ERROR: /Users/ur20980/Src/wycheproof/BUILD:33:1: no such package '@bouncycastle_1_55//jar': error loading package 'external': The repository named 'bouncycastle_1_55' could not be resolved and referenced by '//:BouncyCastleAllTests'.
ERROR: Analysis of target '//:BouncyCastleAllTests' failed; build aborted.
INFO: Elapsed time: 4.864s
ERROR: Couldn't start the build. Unable to run tests.
$ 

Frankly, I am not aspiring to become a wycherproof wizard - being able to (smoothly) run it as a black box tool over my BouncyCastle jars (and later - over C++ crypto libraries like Crypto++) is all I hope for.

Update
Fooling around with build_defs.bzl and changing its top lines to

#bouncycastle_versions = range(49, 56)
bouncycastle_versions = 56
. . . . .

produced this:

$ bazel test BouncyCastleAllTests
Warning: ignoring http_proxy in environment.
ERROR: /Users/ur20980/Src/wycheproof/BUILD:33:1: Traceback (most recent call last):
	File "/Users/ur20980/Src/wycheproof/BUILD", line 33
		bouncycastle_all_tests(size = "large", srcs = ["java/com/..., <2 more arguments>)
	File "/Users/ur20980/Src/wycheproof/build_defs.bzl", line 9, in bouncycastle_all_tests
		for version in bouncycastle_versions: ...
type 'int' is not iterable.
ERROR: /Users/ur20980/Src/wycheproof/BUILD:75:1: Traceback (most recent call last):
	File "/Users/ur20980/Src/wycheproof/BUILD", line 75
		bouncycastle_tests(size = "large", srcs = ["java/com/..., <2 more arguments>)
	File "/Users/ur20980/Src/wycheproof/build_defs.bzl", line 36, in bouncycastle_tests
		for version in bouncycastle_versions: ...
type 'int' is not iterable.
ERROR: no such target '//:BouncyCastleAllTests': target 'BouncyCastleAllTests' not declared in package '' (did you mean 'SpongyCastleAllTests'?) defined by /Users/ur20980/Src/wycheproof/BUILD.
INFO: Elapsed time: 0.166s
ERROR: Couldn't start the build. Unable to run tests.
$

from wycheproof.

Update2 - got something to run

1. Did not remove the already-defined BouncyCastle targets - added http_jar for 1.56.
2. Edited build_defs.bzl changing in bouncycastle_versions = range(49, 56) "56" to "57" (yeah, obvious).
3. Ran bazel test BouncyCastleTest_1_56

And it failed:

$ bazel test BouncyCastleTest_1_56
Warning: ignoring http_proxy in environment.
INFO: Found 1 test target...
FAIL: //:BouncyCastleTest_1_56 (see /private/var/tmp/_bazel_uri/e4d57cbfa3d72fc2e017b4127feb30c0/execroot/wycheproof/bazel-out/local-fastbuild/testlogs/BouncyCastleTest_1_56/test.log).
Target //:BouncyCastleTest_1_56 up-to-date:
  bazel-bin/BouncyCastleTest_1_56.jar
  bazel-bin/BouncyCastleTest_1_56
INFO: Elapsed time: 21.920s, Critical Path: 16.34s
//:BouncyCastleTest_1_56                                                 FAILED in 14.1s
  /private/var/tmp/_bazel_uri/e4d57cbfa3d72fc2e017b4127feb30c0/execroot/wycheproof/bazel-out/local-fastbuild/testlogs/BouncyCastleTest_1_56/test.log

Executed 1 out of 1 test: 1 fails locally.
$ 

Here's the log:
wycherproof-1.log.txt

@peterdettman what's your take on this?

from wycheproof.

@mouse07410 Please see #8 for our proposed changes for BC 1.56, which addresses several of the failures (spurious ones I think). The remaining failures (after PR applied) are all related to CipherInputStream/CipherOutputStream behaviour with AEAD modes. I believe this is down to a disagreement over expected API behaviour (particularly when the MAC check fails).

from wycheproof.

It's surprisingly non-trivial to supply a custom jar from the command line. I got something to work though it's fairly hackish - see #10

If we accept that users will have to edit the BUILD then we can trivially use bazel's java_import - but I'd rather they didn't have to touch the source.

from wycheproof.

It's surprisingly non-trivial to supply a custom jar from the command line.......see #10

I can sympathize... And thank you for #10!

If we accept that users will have to edit the BUILD then we can trivially use bazel's java_import - but I'd rather they didn't have to touch the source.

I agree. But I suspect that 99.99% of the entire population couldn't care less about how well BouncyCastle-1.49 behaves under test (and those people who have to use it, don't have any choice anyway - so why feed them correct but useless data?). I'd expect the majority to want to know how the current Github master holds up (granted, they'd need to be able to sign JCE provider), and how the latest-stable release does under test... From that point of view, I consider most of the BUILD to be of rather limited use.

So your #10 solution (or maybe a perfection/improvement of it) seems like the best way forward, IMHO.

P.S. I hope #8 and #10 get merged rather sooner than later. ;-)

from wycheproof.

Thanks ekasper! mouse07410 please see if you can test your local jars now and reopen this ticket if you can't.

from wycheproof.

Yes, I can run the tests on the local JAR now - thank you!

from wycheproof.

One problem is now bazel build ... stops working. I got this error

no such target '@local//:bouncycastle_jar': target 'bouncycastle_jar ' not declared in package '' defined by BUILD and referenced by '//:BouncyCastleTestLocal'.

from wycheproof.

Can you try adding tags=["manual"] to the local rules?

Targets with tags=["manual"] will not be included in wildcard target patterns (..., :*, :all, etc). You should specify such test targets with explicit target patterns on the command line if you want Bazel to build/test them.

https://bazel.build/versions/master/docs/bazel-user-manual.html#target-patterns

from wycheproof.