shelljs / benchmarks Goto Github PK

With new versions available (up to date bash, nodejs), i think benchmark results are even more flattering.

Especially that line looks cool:

ShellJS was 219.411 times faster than Bash

System Information:

• Darwin
• 17.7.0
• x64
• Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz × 8

Node information

v12.5.0

• Node.js: v12.5.0
• V8: 7.5.288.22-node.14

Shell Information:

• name: /bin/bash
• version: GNU bash, version 5.0.7(1)-release (x86_64-apple-darwin17.7.0)
  Copyright (C) 2019 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
  This is free software; you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.

[echoIntoFile]

• [Bash] took 2280 milliseconds
• [ShellJS 0.5.3] took 414 milliseconds
• [ShellJS 0.6.0] took 426 milliseconds
• [ShellJS latest] took 106 milliseconds
  ShellJS was 21.509 times faster than Bash
  =======================

[envVariable]

• [Bash] took 228 milliseconds
• [ShellJS 0.5.3] took 259 milliseconds
• [ShellJS 0.6.0] took 257 milliseconds
• [ShellJS latest] took 109 milliseconds
  Output differs
  ShellJS was 2.092 times faster than Bash
  =======================

[forLoopAlternateSyntax]

• [Bash] took 79 milliseconds
• [ShellJS 0.5.3] took 140 milliseconds
• [ShellJS 0.6.0] took 134 milliseconds
• [ShellJS latest] took 110 milliseconds
  Output differs
  Bash was 1.392 times faster than ShellJS
  =======================

[helloworld]

• [Bash] took 67 milliseconds
• [ShellJS 0.5.3] took 119 milliseconds
• [ShellJS 0.6.0] took 119 milliseconds
• [ShellJS latest] took 114 milliseconds
  Output differs
  Bash was 1.701 times faster than ShellJS
  =======================

[helloworld10k]

• [Bash] took 152 milliseconds
• [ShellJS 0.5.3] took 228 milliseconds
• [ShellJS 0.6.0] took 235 milliseconds
• [ShellJS latest] took 108 milliseconds
  Output differs
  ShellJS was 1.407 times faster than Bash
  =======================

[ls10k]

• [Bash] took 23477 milliseconds
• [ShellJS 0.5.3] took 667 milliseconds
• [ShellJS 0.6.0] took 697 milliseconds
• [ShellJS latest] took 107 milliseconds
  Output differs
  ShellJS was 219.411 times faster than Bash
  =======================

[pwd10k]

• [Bash] took 631 milliseconds
• [ShellJS 0.5.3] took 2183 milliseconds
• [ShellJS 0.6.0] took 1911 milliseconds
• [ShellJS latest] took 111 milliseconds
  Output differs
  ShellJS was 5.685 times faster than Bash
  =======================

[touchSyntax0.6]

• [Bash] took 3971 milliseconds
• Skipping test for ShellJS 0.5.3
• [ShellJS 0.6.0] took 348 milliseconds
• [ShellJS latest] took 113 milliseconds
  ShellJS was 35.142 times faster than Bash
  =======================

Due to inactivity, I revoked the github token (wanted to limit security issues). If we revisit this project, we'll have to update the token again.

To get the most consistent results, we might do well to run each unit test multiple times, discard any outliers (maybe the system was in a weird state for one run), and average the remaining runs.

Doing a simple average is easy enough, and we'll see a huge amount of improvement that way, so it might be worth it to just write it in to do an average of 3-5 runs, and worry about outliers once they crop up.

Generating markdown and converting it to html works, but is very limiting. I think we can achieve much nicer results if we programmatically build a DOM and then write it out to a file. This should be easy to achieve using something like xmlbuilder. If we add in css files, then we can have complete control over styling, which is good.

Some shelljs commands that are currently untested include:

• exec
• mkdir
• cat
• find
• cp
• set
• ln

It'd be good to see some benchmarks for these as well, to see if we can get any performance wins.

Hi @IvantheDugtrio!

I was wondering if you'd consider allowing us to move this project in to the shelljs org? I've been trying to put more of an emphasis on performance in shelljs, and this tool looks really useful.

We would of course love for you to continue working on the project, and you would have full control of it.

Thank you for your consideration

cc @nfischer.

Once we start building an index.html from benchmark run, Github pages should start displaying results. This should have a similar layout to the current index.md file.

Requiring multiple versions of the same package is somewhat frowned upon in the node community, but here's a suggested workaround from isaacs:

npm/npm#5499 (comment)

We might consider investigating this in order to test against versions 0.5.3 and up. Or we can switch to requiring the latest version of shelljs and testing against that.

It might be worth it to investigate some sort of API for generating online graphs, so we can check performance changes over time, on average.

The website could really use some styling.

This is not the behavior I'm seeing on my machine. @IvantheDugtrio did you make sure you're running on the latest version of ShellJS? touch() is a new feature in v0.6

These stats aren't very helpful unless we have information about your machine's specs.