A collection of "Best in Class" modules. This means we promote solid browser/environment support, ES5 precedents, unit testing, and plenty of documentation.
John-David Dalton | Benjamin Tan | Kit Cambridge | Mathias Bynens |
A benchmarking library. As used on jsPerf.com.
Home Page: https://benchmarkjs.com/
License: Other
A collection of "Best in Class" modules. This means we promote solid browser/environment support, ES5 precedents, unit testing, and plenty of documentation.
John-David Dalton | Benjamin Tan | Kit Cambridge | Mathias Bynens |
Instead of passing the benchmark as a second argument as
suite.on('cycle', function(event, bench) {
console.log(bench);
});
we should use
suite.on('cycle', function(event) {
console.log(event.target);
});
and also use event.currentTarget
to reference the suite
instance.
The Benchmark.platform object needs a dependency on platform.js in order to work. Make note of this in the documentation to avoid confusion.
I can't find the library size on your homepage or README - care to add it? Would be nifty if you could read the result from closure compiler and write it into the README
Remove Math.random
from the example in /example/jsperf/index.html
.
Do the Benchmark.setup and onStart methods support async methods ?
The code would be something like
new Benchmark.Suite({}).add({
'setup' : function(){
window.setTimeout(function(){
var i = 10;
},1000);
},
defer: true
'fn': function(def) {
window.setTimeout(function() {
console.log(i); // Do somethign with 'i', but only when it is defined by setup
def.resolve();
}, 100);
}
});
The setTimeout will be replaced with an async method.
P.S: Not able to find a google groups for questions, so asking here
Here is a sample benchmark I am trying to make work:
"use strict";
var Benchmark = require("benchmark");
function makeArray(length) {
var array = new Array(length);
for (var i = 0; i < length; ++i) {
array[i] = 1;
}
}
var suite = new Benchmark.Suite("Loops with arrays");
var stepValues = [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024];
stepValues.forEach(function (stepValue) {
suite.add(stepValue + " at a time", function () {
var array = makeArray(64 * 1024 * 1024);
for (var i = 0; i < array.length; ++i) {
array[i] *= 3;
}
});
});
suite.on("complete", function () {
console.log(Object.getPrototypeOf(this));
this.forEach(function (result) {
if (result.error) {
console.error(result.name + ": ", result.error);
} else {
console.log(result.name + ": ", result.stats);
}
});
});
suite.run();
This doesn't work because of the function decompilation; it gives a bunch of ReferenceError: makeArray is not defined
. OK, but how can I make this work? I tried attaching to this
inside the setup
option, but that was a no-go.
Are there any plans to distribute benchmark.js in bower and other package managers? Likewise you are doing with lodash.
var Bench = require('benchmark')
Bench.support.timeout = false
function each(fns, index, cb) {
var sync = true
for (var i = index; i < fns.length; i++) {
var done = false
fns[i](function() {
done = true
if (sync) return
each(fns, i + 1, cb)
})
sync = done
if (!sync) return
}
cb()
}
function noop(cb) {
cb()
}
function asyncNoop(cb) {
setImmediate(cb)
}
var sync = [noop, noop, noop, noop, noop]
var async = [noop, noop, asyncNoop, noop, noop]
var suite = new Bench.Suite
suite.add('sync', function() {
each(sync, 0, function() {})
})
suite.add('async', function(deferred) {
each(async, 0, function() {
deferred.resolve()
})
}, {
defer: true
})
suite.add('setImmediate()', function(deferred) {
setImmediate(function() {
deferred.resolve()
})
}, {
defer: true
})
suite
.on('cycle', function(event) {
console.log(String(event.target));
})
.run({async: true})
Output
sync x 5,450,974 ops/sec ±0.25% (99 runs sampled)
async x 38,051 ops/sec ±2.67% (74 runs sampled)
setImmediate() x 305,534 ops/sec ±0.24% (85 runs sampled)
We see just a huge difference between async
and setImmediate()
for no reason. Actually neither benchmarking via Http server nor via adhoc script do not show such difference.
something kills Opera Mini on its proxy's side.
Add Benchmark#clone unit tests.
If you have for example console.log('$') in a benchmark code, it fails to run the benchmark suite.
cf. https://developer.mozilla.org/en/docs/JavaScript/Reference/Global_Objects/String/replace
To fix this, apply the following patch. Thanks.
--- benchmark.js.bak 2013-01-03 13:06:16.739109681 +0900
+++ benchmark.js 2013-01-03 15:06:32.846285741 +0900
@@ -1630,7 +1630,7 @@
function interpolate(string, object) {
forOwn(object, function(value, key) {
// escape regexp special characters in `key`
- string = string.replace(RegExp('#\\{' + key.replace(/([.*+?^=!:${}()|[\]\/\\])/g, '\\$1') + '\\}', 'g'), value);
+ string = string.replace(RegExp('#\\{' + key.replace(/([.*+?^=!:${}()|[\]\/\\])/g, '\\$1') + '\\}', 'g'), value.replace(/\$/g, '$$$$'));
});
return string;
}
A regression in Benchmark.js 0.3.0 was introduced.
While synchronous cycles, the default, is supported asynchronous cycles are not.
We should have the fallback avoid use of the Function constructor so we don't throw a JavaScript security error in AIR.
I'm giving a 6 minute talk on jsPerf in a few days, & like to know how is "hot" code prevented. please? Seems that FireFox has several optimization layers, & I'd expect tight loops will make FF jump to optimize early.
From your talks I see you use several loops of different lengths, so my brief bullet points would be:
Do you think this is fair/accurate please? BTW, I'm not saying to never test on tiny routines, but my ASM programming experience & lessons on ninja JS engine optimization tricks, indicates code context is important.
I could use such an option when testing a snippet of JS that has to run on a large DOM structure. In those cases, the DOM tree must be cloned, appended to the document, and only then can the timed code run once on the clone. After it has completed, the modified tree must be discarded.
Currently, if the DOM tree is too large, the majority of time is spent appending it to the document, so the amount of time spent executing the JS which modifies the tree pales in comparison to the amount of time spent actually attaching the tree.
Hi. I've just started using this library and it's been great. I was just wondering, is it possible to set a specific number of times an operation should run, then calculate the time taken? Sorry if it's written in the documentation; I wasn't sure where to look.
Istanbul ftw.
var Benchmark = require('benchmark');
console.log(Benchmark.add);
console.log(Benchmark.Benchmark.add);
console.log(Benchmark().add);
Add Benchmark assert
option to pre-run test that a function returns an expected result.
This can be useful to ensure two snippets being tested at least return the same result.
This test works correctly:
var Benchmark = require('benchmark'),
schedule = require('../index'),
suite = new Benchmark.Suite('dependency-graph');
suite
.add('create graph', function() {
var tasks = ['A', 'B', 'C', 'D', 'E'];
schedule.dependencyGraph(tasks);
})
.on('error', function(event) {
console.log(event);
})
.on('cycle', function(event) {
console.log(String(event.target));
})
.run({async: false});
However, when I try and move the tasks
setup to setup
I get a reference error on schedule
.
var Benchmark = require('benchmark'),
schedule = require('../index'),
suite = new Benchmark.Suite('dependency-graph');
suite
.add('create graph', function() {
schedule.dependencyGraph(tasks);
},
{
'setup': function() {
var tasks = ['A', 'B', 'C', 'D', 'E'];
}
})
.on('error', function(event) {
console.log(event);
})
.on('cycle', function(event) {
console.log(String(event.target));
})
.run({async: false});
The error event reports:
error: [ReferenceError: schedule is not defined],
Looks like there is a scope issue that I've introduced by specifying setup
but I'm not sure how to resolve. What's the correct way to run setup code before every test when using Nodejs
?
Add benchmark and suite score
API.
I have some things I need to do asynchronously in a setup function. As far as I can tell, there's no way to do this -- if there is, I apologize!
Here's an example (for node) of the kind of thing I want to be able to do:
var fs = require("fs");
var Benchmark = require("benchmark");
new Benchmark.Suite().
add("Test with asynchronous setup", {
'defer': true,
'setup': function (deferred) {
var _this = this;
fs.readFile("data.txt", function (_, data) {
_this.data = data;
deferred.resolve();
});
},
'fn': function (deferred) {
console.log(this.data);
deferred.resolve();
}
}).
run();
Could you add a directory that contains local copys of the dependencies.
Mainly because if I download the benchmarkjs repository as a zip or tar.gz, the vendor folder contains emtpy folders.
This causes for test page (test\index.html) to fail since require.js doesn't exist.
I'm unable to use GIT due to firewall restrictions, so I can't fetch the dependencies automatically.
is there a setup/teardown mechanism at the suite level ?
i looked and didnt see it
Benchmark looks great.
Unfortunately, the example on the home page doesn't seem to work out of the box on node v0.8.
I tried making a new file and running it in node
var Benchmark = require('./benchmark.js');
var suite = new Benchmark.Suite;
// add tests
suite.add('RegExp#test', function() {
/o/.test('Hello World!');
})
.add('String#indexOf', function() {
'Hello World!'.indexOf('o') > -1;
})
.add('String#match', function() {
!!'Hello World!'.match(/o/);
})
// add listeners
.on('cycle', function(event) {
console.log(String(event.target));
})
.on('complete', function() {
console.log('Fastest is ' + this.filter('fastest').pluck('name'));
})
// run async
.run({ 'async': true });
which will output this:
undefined
undefined
undefined
Fastest is String#indexOf
It looks like the process is working correctly, but the Event objects are not being formed correctly. The cycle event is missing the target property and looks like this:
{ type: 'cycle' }
I can see in the source that event.type is set explicitly (e.g., line 1659 for the cycle event) while event.target is coming from elsewhere (last, bench). Nothing jumps out to me about any of this though, so I was hoping someone else might see something.
Also, I'm running on the brand new node v0.8 (released yesterday), in case that could be related.
Hi,
I want to test some async functions (database inserts).
There are two ways of testing:
I try with both 'async' options (true/false), but I notice than the benchmark doesn't wait the callbacks to give the result.
Is there any function like "done()" to call as callback?
suite.add('async fct 1', function() {
setTimeout(function() {
return true;
}, 10)
})
.add('async fct 2', function() {
setTimeout(function() {
return true;
}, 20)
})
.on('complete', function() {
console.log('1 : Fastest is ' + this.filter('fastest').pluck('name'));
})
.run({
'async' : true/false
});
Thank you,
Philmod
Why is benchmark.js not getting accurate performance data? I rewrote the test to use the Date object and it's able to determine which method is faster.
May be I did not enough RTFM, but this seems strange:
async x 119,606 ops/sec ±2.39% (76 runs sampled)
sync x 69,544,898 ops/sec ±3.24% (66 runs sampled)
Code to reproduce:
var Benchmark = require('benchmark'),
suite = new Benchmark.Suite;
suite
.add('async', {
defer: true,
fn: function(deferred) {
deferred.resolve();
}
})
.add('sync', function() {
return true;
})
.on('cycle', function(event) {
console.log(String(event.target));
})
.run();
Add unit tests for Suite
array methods like shift()
, splice()
, slice()
, and others.
And test that we cover IE issues with array-like-objects and splice()
/ shift()
.
Test case:
var Benchmark = require('benchmark');
var suite = new Benchmark.Suite('suite');
suite.add('test', {
fn: function() {
if (!this.wasSetup) {
console.error('fail');
process.exit(1);
}
console.log('fn');
this.wasSetup = false;
},
setup: function() {
this.wasSetup = true;
console.log('setup');
}
});
suite.run();
And here's the output:
% node test.js
setup
fn
setup
fn
setup
fn
setup
fn
fail
Tested with v1.0.0 of benchmark from npm.
In general, I found that setup
still does get called on occasion, but only after long runs of fn
first.
Benchmark.options.onComplete is called by the internal cloned benchmarks.
The internal cloned benchmarks should not execute the onComplete
event listeners of the host benchmark.
Add more forOwn unit tests to check enumeration issues.
hey
Im developping a lib for webgl https://github.com/jeromeetienne/tquery and im using benchmark.js to bench it. The library is real nice but i failed to find an easy way to display the results. So i did a runner. You can find it here https://github.com/jeromeetienne/benchrunner
If you like it, i will be more than happy to modify it to match the need of benchmark.js project. and do a pull request
Jerome
Cloning into vendor/platform.js...
remote: Counting objects: 691, done.
remote: Compressing objects: 100% (308/308), done.
remote: Total 691 (delta 431), reused 633 (delta 373)
Receiving objects: 100% (691/691), 2.02 MiB | 443 KiB/s, done.
Resolving deltas: 100% (431/431), done.
fatal: reference is not a tree: 61dad43825ebb1f391dc79900744fc884029aa7f
Unable to checkout '61dad43825ebb1f391dc79900744fc884029aa7f' in submodule path 'vendor/platform.js'
currently in chrome canary as window.performance.webkitNow()
It's a floating point millisecond measurement since the page was created.
spec here: http://www.w3.org/TR/hr-time/ (out of date)
Update by @mathiasbynens: The above spec link is the out of date version. Check out the latest editor’s draft here: http://dvcs.w3.org/hg/webperf/raw-file/tip/specs/HighResolutionTime/Overview.html
IE10 will probably have it as well.
var Suite = require('benchmark').Suite
var suite = new Suite
suite.add('async test', function(deferred) {
deferred.resolve()
}, {
defer: true
})
suite
.on('cycle', function(event) {
console.log(String(event.target));
})
.on('complete', function() {
console.log('Fastest is ' + this.filter('fastest').pluck('name'));
})
.run({async: true})
Outputs the following:
async test x 781 ops/sec ±0.49% (11 runs sampled)
Which is obviously no sense. What's going on here?
The number of times onCycle
is called doesn't match up with Benchmark#cycles
because onCycle
is also called per sample run as well.
Now we have public:
//cdnjs.cloudflare.com/ajax/libs/benchmark/0.3.0/benchmark.js
but better to have any new version of library on CDN.
to get started would be useful. Can you pass a callback to Suite#add()
's function or how does the async support work? The readme example has async: true
but the code isn't async
Why is it that it's always returning the first function added as the fastest?
Hey Guys,
I've been having a crack at using benchmark.js and I really like the API and the general operation of the library. One thing I'm noticing though is that when I'm trying to test things like canvas operations the browser essentially freezes up.
The same problem occurs on jsperf also. To replicate try something like:
context.clearRect(0, 0, canvas.width, canvas.height);
I'm going to keep looking to see if I can work out going on, and if I find something I'll let you know...
Cheers,
Damon.
Benchmark.version
still says 0.2.2
.
Consider the following code:
var optionsStartCount = 0,
benchStartCount = 0,
optionsCompleteCount = 0,
benchCompleteCount = 0;
Benchmark.options.onStart = function () {
optionsStartCount++;
};
Benchmark.options.onComplete = function () {
optionsCompleteCount++;
};
Benchmark("start tracker", function () {
Math.pow(20,500);
})
.on("complete", function (e) {
benchCompleteCount++;
console.log("options start count " + optionsStartCount);
console.log("bench start count " + benchStartCount);
console.log("options complete count " + optionsCompleteCount);
console.log("bench complete count " + benchCompleteCount);
})
.on("start", function () {
benchStartCount++;
}).run({async: true});
Why is the Benchmark.options.onStart
and onComplete
functions called several times? Shouldn't they be called just as many times as the benchmark's own start
and complete
event listeners?
3.Benchmark: supports loading Platform.js as a module (1, 0, 1)Rerun
1.failed Expected: "string"
Result: "object"
Diff: "string" "object"
Source: at Anonymous function (http://benchmarkjs.com/_js/test.js?t=1388900753472:113:9)
Introduced in Node 0.8; appeared in several talks at NodeConf. Will tackle when I have some time...
Add a Benchmark assert
option to pre-run and test that it returns an expected result.
This can be useful to ensure snippets being tested at least return the same result.
Hi,
Can i do something like this :
benchmark.add('test db call', function(done) {
//make the io (db call)
//and once finished, call done()
});
i would like to do a benchmark of db operations which we do, wanted to know if async support like above is possible.
Thanks.
Here is a test case: http://jsfiddle.net/9bcVB/18/
I run two deferred tests in an async suite and trigger suite.abort()
before the suite can finish, which correctly emits an abort
event.
But the benchmarks still run until the suite is complete, firing a complete
event.
I first thought this was a requestAnimFrame
error, but it also occurs for plain setTimeout
-based async benchmarks.
Is this behaviour expected, or did I find i bug? I crawled through the docs, but info on aborting and deferred runs is sparse.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.