compute a hash of user code about caliper HOT 14 CLOSED

what i want is something that stashes a patch relative to head (and enough
information for me to know what "head" was), so i can just be hacking away and
rerunning my benchmarks and then come back later and match performance changes 
to
specific code changes. i'm sort of seeing running the benchmarks as something 
i'd do
at least as frequently as running the unit tests. (and, when i'm doing 
performance
work, that's already the case.)

i could also imagine having my continuous build using something similar, so i 
can
look at the long-term graph of my performance and click to find out why 
NumberFormat
suddenly started to suck or why Formatter suddenly got faster. (but maybe that 
isn't
very similar at all, implementation-wise, just conceptually.)

but, yeah, even just a hash is something: "did the code change between these 
runs, or
are they just 'duplicate' data points?" is still useful.

I planned to submit a different issue about allowing the user to specify a 
-sourcePath 
on the command line, and what extra cool stuff we can do if we are permitted to 
make 
use of that source code.  I think it opens up lots of amazing possibilities and 
also 
many cans of worms.

Does this issue serve as a feature request for attaching source code to the web 
UI runs?

Another thing we may want to hash is the benchmark itself. Two runs are not 
really comparable if the benchmark has changed between when they were run, and 
it would be nice to show that somehow.

I would say that this issue does not include anything related to uploading 
actual source code.  That should be filed separately if it isn't already.  It 
has security implications.

The idea is only to capture a hash code so that we can track when 
implementation A becomes implementation B while still leaving it up to the user 
to fill in a description if they want.

And this value should incorporate benchmark code AND tested code.  One 
possibility would be run md5 over every .class and .jar file in the user's 
CLASSPATH.  It might as well even include caliper classes in that.  Another 
possibility would be to only cover the classes that actually get loaded in the 
course of benchmark execution.  We could make a ChecksummingClassLoader.  I 
don't know how crazy these ideas are.

It's a little tricky to do this with the ClassLoader approach because of all 
the subprocesses used, but probably possible. 

Probably shouldn't use loaded classes because benchmarkA() and benchmarkB() may 
cause different classes to be loaded.

Could just look at the benchmark class, every class referred to by it, and 
maybe just one level deeper?

Checksumming the entire classpath might be too slow?  But should be pretty 
solid?

Or we could forget all this and just have the user enter an optional 
description on the command line and leave it at that.  (Actually even if we do 
this, the description is probably a good idea anyway; it can appear in place of 
the checksum bits in the web display..)