notbenh / euler_bench Goto Github PK

currently we are treating each interp as the language, what the config really should be doing is listing all known interps and then each interp should point to an array of 'languages' (I cant really think of why I have this feeling that it should be an array but its future proof that way I guess).

because the config does not have haskell listed, haskell does not get picked up as a proper language thus it's tossed in to the opts bucket and then all known languages are run.

we should first check the config languages, then the solution languages, leftovers then should go to opt.

we should have a problem 000 that is the most basic thing that you can hand to the interpreter in the hope to get a vague number of what the 'startup' time is.

Then once we have that we should lop that off the top of the results of the runs for the actual solutions, or at least make it an option.

http://search.cpan.org/~dapm/perl-5.10.1/lib/FindBin.pm

this should likely test -x for the given string and if fails complain.

currently 3 spaces for indentation is not going over well.

what we should do here though is run all the perl6 interps as you asked for a language.

otherwise you get a bunch of warnings....

Currently this is done with File::Fu and File::Spec::Functions, It's a bit messy and could use a cleaning.

Mmmmmm, docs.

Needs to be smart enough to skip languages that are not installed.

For example if some one wanted to compare perl5.8 and perl5.10, currently there is no way to do a direct comparison as we only accept one value for 'where does perl live'

Perl 5.8 or Perl 5.10 ?

I (dukeleto) lean heavily towards 5.10, but making it 5.8 would mean that we could benchmark perl 5.8 against 5.10 .

This should allow any one to build eternal visualizations of the data.

There are multiple implementations of Perl 6 (like Rakudo, SMOP and Pugs), so we should use the name "rakudo" to refer to Perl 6 on Parrot.

compare_problem_number and compare_implementations fail to notice if a language implementation is not found. Instead it just looks like that language runs really fast. E.g,

Benchmarking EP#002 with 100 iterations
            (warning: too few iterations for a reliable count)
                  Rate perl5/002/01.pl perl6/002/01.pl
perl5/002/01.pl 49.3/s              --            -79%
perl6/002/01.pl  233/s            372%              --

http://headrattle.blogspot.com/2009/10/termprogressbar.html

Currently we are using ENV to store the locations of all interpreters for solutions. It has been suggested that moving forward a config file would be a more flexible solution.

will need the ability to take an array of intrripts.

We currently like the cmp method of specifying both lang and problems to mix and match your results, but it could use some work.

We should capture output and only display it if asked for, thus requiring an option to toggle this setting.

./bin/bench 003:02 parrot --count=1000

would be cool. It would run only the 02 implementation of Euler Problem 003.

Chromatic said that a persistent parrot interpreter is possible.

Or we could just subtract a measured interpreter startup time.

it's nice to have local comparisons, but if we start to do run-to-run comparisons then we should have a way to start to standardize things so that we don't skew results to favor those run on faster boxes.

ya get cracking.

We can safely assume we have perl5 (though checking versions might not be a bad idea). Will need to also install some CPAN modules.

Ask if the user wants to install Rakudo(parrot), ruby(?), and any other language that have solutions for.