angular / benchpress-tree Goto Github PK

A reference implementation of a benchpress deep-tree benchmark as seen in Angular

JavaScript 90.84% HTML 9.16%

benchpress-tree's Introduction

Benchpress Tree Benchmark

Reference implementation of a deep tree baseline benchmark with benchpress, adapted from the tree benchmark baseline in the Angular repository.

Full documentation found at https://github.com/angular/angular/blob/master/modules/benchpress/README.js.md

Run It

$ npm install .
$ ./node_modules/.bin/webdriver-manager update #Installs chromedriver and selenium standalone
$ ./node_modules/.bin/protractor protractor.conf.js #runs benchmark spec and logs output

See The Code

Benchmark code: public/tree.html, public/tree.js
Benchmark spec: tree_benchmark.spec.js
Protractor config: protractor.conf.js

benchpress-tree's People

Contributors

Stargazers

Watchers

Forkers

tbosch agileurbanite uber5001 willbuck ochafik elmariofredo webdriverio-boneyard dsingh45 ca136 alan-agius4 hamletrp mboughaba jeffbcross biophoton isabella232

benchpress-tree's Issues

Testing error

After following the instructions and trying to run the provided protractor test, i get the following error:

Using ChromeDriver directly...
[launcher] Running 1 instances of WebDriver
[launcher] Error: SyntaxError: Unexpected token )
at goog.loadModuleFromSource_ (/Users/oplummer/Documents/projects/benchpress-guild2/benchpress-tree/node_modules/benchpress/node_modules/selenium-webdriver/lib/goog/base.js:1123:19)
at Object.goog.loadModule (/Users/oplummer/Documents/projects/benchpress-guild2/benchpress-tree/node_modules/benchpress/node_modules/selenium-webdriver/lib/goog/base.js:1085:46)
at /Users/oplummer/Documents/projects/benchpress-guild2/benchpress-tree/node_modules/benchpress/node_modules/selenium-webdriver/lib/webdriver/promise.js:1:6
at Object.exports.runInContext (vm.js:44:17)
at Object.Context.closure.goog.retrieveAndExecModule_ (/Users/oplummer/Documents/projects/benchpress-guild2/benchpress-tree/node_modules/benchpress/node_modules/selenium-webdriver/base.js:135:8)
at :1:6
at Object.exports.runInContext (vm.js:44:17)
at Context.closure.closure.vm.createContext.CLOSURE_IMPORT_SCRIPT (/Users/oplummer/Documents/projects/benchpress-guild2/benchpress-tree/node_modules/benchpress/node_modules/selenium-webdriver/base.js:104:12)
at Object.goog.importScript (/Users/oplummer/Documents/projects/benchpress-guild2/benchpress-tree/node_modules/benchpress/node_modules/selenium-webdriver/lib/goog/base.js:879:9)
at Object.goog.importModule (/Users/oplummer/Documents/projects/benchpress-guild2/benchpress-tree/node_modules/benchpress/node_modules/selenium-webdriver/lib/goog/base.js:900:14)
[launcher] Process exited with error code 100

Test Failing

Hi. Thank you for your contributions.

I am running into this error when I try to run the program:

Failures:

deep tree baseline should be fast!
Message:
Failed: WebDriverError: unknown error: call function result missing 'value'
(Session info: chrome=67.0.3396.62)
(Driver info: chromedriver=2.26.436362 (5476ec6bf7ccbada1734a0cdec7d570bb042aa30),platform=Windows NT 10.0.17134 x86_64)
Stack:
Error: WebDriverError: unknown error: call function result missing 'value'
(Session info: chrome=67.0.3396.62)
(Driver info: chromedriver=2.26.436362 (5476ec6bf7ccbada1734a0cdec7d570bb042aa30),platform=Windows NT 10.0.17134 x86_64)
at proxyDone.fail (C:\Users\tehre\Documents\benchpress-tree\node_modules\jasminewd2\index.js:87:34)
at
at process.tickCallback (internal/process/next_tick.js:188:7)
From: Task: Run it("should be fast!") in control flow
at Object. (C:\Users\tehre\Documents\benchpress-tree\node_modules\jasminewd2\index.js:79:14)
at C:\Users\tehre\Documents\benchpress-tree\node_modules\jasminewd2\index.js:16:5
at ManagedPromise.invokeCallback (C:\Users\tehre\Documents\benchpress-tree\node_modules\selenium-webdriver\lib\promise.js:1379:14)
at TaskQueue.execute_ (C:\Users\tehre\Documents\benchpress-tree\node_modules\selenium-webdriver\lib\promise.js:2913:14)
at TaskQueue.executeNext_ (C:\Users\tehre\Documents\benchpress-tree\node_modules\selenium-webdriver\lib\promise.js:2896:21)
at asyncRun (C:\Users\tehre\Documents\benchpress-tree\node_modules\selenium-webdriver\lib\promise.js:2775:27)
From asynchronous test:
Error
at Suite. (C:\Users\tehre\Documents\benchpress-tree\tree_benchmark.spec.js:15:3)
at Object. (C:\Users\tehre\Documents\benchpress-tree\tree_benchmark.spec.js:14:1)
at Module._compile (module.js:624:30)
at Object.Module._extensions..js (module.js:635:10)
at Module.load (module.js:545:32)
at tryModuleLoad (module.js:508:12)

runner.sample promise not resolving

Getting error

$ ./node_modules/.bin/protractor protractor.conf.js 
Using ChromeDriver directly...
[launcher] Running 1 instances of WebDriver
Started
A Jasmine spec timed out. Resetting the WebDriver Control Flow.
F

Failures:
1) deep tree baseline should be fast!
  Message:
    Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
  Stack:
    Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
        at [object Object]._onTimeout (/Users/x/benchpress-tree/node_modules/jasmine-core/lib/jasmine-core/jasmine.js:1812:23)

1 spec, 1 failure
Finished in 30.033 seconds

Collected data seems suspicious; Canary mode doesn't report metrics

I checked out the demo and ran it with the following results:


    forcedGcAmount |       forcedGcTime |           gcAmount |             gcTime |        majorGcTime |     pureScriptTime |         renderTime |         scriptTime
------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------
           1299.77 |              13.06 |             491.90 |              10.73 |               0.00 |              51.52 |             134.63 |              62.25
           2367.62 |              29.73 |               0.00 |               0.00 |               0.00 |              73.11 |             138.69 |              73.11
           2146.84 |              28.58 |            1400.20 |               6.02 |               0.00 |              64.64 |             138.59 |              70.66
           3729.60 |              33.67 |               0.00 |               0.00 |               0.00 |              64.24 |             141.94 |              64.24
           3765.45 |              34.57 |               0.00 |               0.00 |               0.00 |              65.50 |             137.77 |              65.50
           3775.76 |              39.67 |               0.00 |               0.00 |               0.00 |              67.22 |             135.41 |              67.22
           3769.90 |              37.77 |               0.00 |               0.00 |               0.00 |              66.30 |             142.54 |              66.30
           3776.86 |              35.73 |               0.00 |               0.00 |               0.00 |              70.08 |             146.30 |              70.08
           3761.22 |              37.83 |               0.00 |               0.00 |               0.00 |              67.39 |             139.36 |              67.39
           3762.50 |              37.86 |               0.00 |               0.00 |               0.00 |              64.04 |             148.04 |              64.04
           3794.86 |              38.85 |               0.00 |               0.00 |               0.00 |              63.96 |             135.96 |              63.96
           3781.95 |              38.89 |               0.00 |               0.00 |               0.00 |              65.23 |             143.53 |              65.23
           3764.52 |              38.47 |               0.00 |               0.00 |               0.00 |              67.48 |             144.74 |              67.48
           3759.41 |              36.89 |               0.00 |               0.00 |               0.00 |              66.44 |             137.67 |              66.44
           3762.62 |              36.82 |               0.00 |               0.00 |               0.00 |              67.38 |             142.58 |              67.38
           3774.42 |              41.91 |               0.00 |               0.00 |               0.00 |              65.47 |             138.46 |              65.47
           3764.98 |              40.21 |               0.00 |               0.00 |               0.00 |              66.89 |             140.60 |              66.89
           3759.39 |              41.70 |               0.00 |               0.00 |               0.00 |              66.40 |             144.26 |              66.40
           3755.74 |              36.05 |               0.00 |               0.00 |               0.00 |              71.11 |             137.41 |              71.11
           3764.54 |              37.54 |               0.00 |               0.00 |               0.00 |              69.03 |             143.31 |              69.03
================== | ================== | ================== | ================== | ================== | ================== | ================== | ==================
      3491.90+-19% |         35.79+-17% |        94.60+-336% |         0.84+-312% |               0.00 |          66.17+-6% |         140.59+-2% |          67.01+-3%
.

The GC times looked awfully suspicious to me (but I wasn't surprised, since the runner appeared to be running the default version of chrome). I modified chromeOptions to include 'binary': '/Applications/Google\ Chrome\ Canary.app/Contents/MacOS/Google\ Chrome\ Canary',, which did successfully cause Canary to be launched, but then the results were empty:

    forcedGcAmount |       forcedGcTime |           gcAmount |             gcTime |        majorGcTime |     pureScriptTime |         renderTime |         scriptTime
------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.26 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.27 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.61 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.46 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.22 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.44 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.57 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.36 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.31 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.28 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.52 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.37 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.61 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.43 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.53 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.37 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.28 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.62 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.41 |               0.00
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.41 |               0.00
================== | ================== | ================== | ================== | ================== | ================== | ================== | ==================
              0.00 |               0.00 |               0.00 |               0.00 |               0.00 |               0.00 |          0.42+-29% |               0.00

I'm new to angular / protractor / benchpress, so maybe I'm missing something obvious. Any ideas?

Error: Tried too often to get the ending mark: 21

I get this error when I try to run ./node_modules/.bin/protractor protractor.conf.js after a successful npm install . and ./node_modules/.bin/webdriver-manager update

[11:54:36] E/launcher - Tried too often to get the ending mark: 21 [11:54:36] E/launcher - Error: Tried too often to get the ending mark: 21 at PerflogMetric._readUntilEndMark (C:\Users\tbosch\projects\angular\modules\@angular\benchpress\src\metric\perflog_metric.ts:150:13) at Timeout._onTimeout (C:\Users\tbosch\projects\angular\modules\@angular\benchpress\src\metric\perflog_metric.ts:161:43) at ontimeout (timers.js:386:14) at tryOnTimeout (timers.js:250:5) at Timer.listOnTimeout (timers.js:214:5) [11:54:36] E/launcher - Process exited with error code 199
full trace:

$ ./node_modules/.bin/protractor protractor.conf.js
[11:54:27] I/direct - Using ChromeDriver directly...
[11:54:27] I/launcher - Running 1 instances of WebDriver
Started
*BENCHMARK deep-tree
Description:

depth: 11
forceGc: true
regressionSlopeMetric: scriptTime
sampleSize: 10
userAgent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
Metrics:
forcedGcAmount: forced gc amount in kbytes
forcedGcTime: forced gc time in ms
gcAmount: gc amount in kbytes
gcTime: gc time in ms
majorGcTime: time of major gcs in ms
pureScriptTime: script execution time in ms, without gc nor render
renderTime: render time in ms
scriptTime: script execution time in ms, including gc and render

forcedGcAmount | forcedGcTime | gcAmount | gcTime | majorGcTime | pureScriptTime | renderTime | scriptTime
------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------
[11:54:36] E/launcher - Tried too often to get the ending mark: 21
[11:54:36] E/launcher - Error: Tried too often to get the ending mark: 21
at PerflogMetric._readUntilEndMark (C:\Users\tbosch\projects\angular\modules@angular\benchpress\src\metric\perflog_metric.ts:150:13)
at Timeout._onTimeout (C:\Users\tbosch\projects\angular\modules@angular\benchpress\src\metric\perflog_metric.ts:161:43)
at ontimeout (timers.js:386:14)
at tryOnTimeout (timers.js:250:5)
at Timer.listOnTimeout (timers.js:214:5)
[11:54:36] E/launcher - Process exited with error code 199

TypeError: Object [object Object] has no method 'serialize' when running protractor

Following the instructions to install and run the project, I get:

Message:
Failed: TypeError: Object [object Object] has no method 'serialize'
Stack:
Error: Failed: TypeError: Object [object Object] has no method 'serialize'
at /Java/edorasware/test/benchpress-tree/node_modules/protractor/node_modules/jasminewd2/index.js:104:16
at /Java/edorasware/test/benchpress-tree/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/base.js:1582:15
at [object Object].webdriver.promise.ControlFlow.runInNewFrame_ (/Java/edorasware/test/benchpress-tree/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1654:20)
at notify (/Java/edorasware/test/benchpress-tree/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:465:12)

Any idea where could the issue?
Thanks!!

Error: reflect-metadata shim is required when using class decorators

Hey Jeff!

I'm trying to get this code running to start playing around with Benchpress, but following your instructions to:

node install .
./node_modules/.bin/webdriver-manager update
/node_modules/.bin/protractor protractor.conf.js

I am getting the error in the issue title.

It would appear as if this is a common problem getting started with angular2 (which I know this project isn't), and I've so far been able to deduce that I probably need to

npm install --save-dev reflect-metadata

But I'm unsure where this needs to go in order for the spec to run (I've so far tried putting it in the public/tree.html file as a src tag, but that doesn't seem to be doing the trick)

I would appreciate any help you can spare in fixing this up and will gladly submit a pull request to solve the problem if one is so warranted (it's entirely possible I've made a mistake and no PR is necessary for others to run the code).

TypeError: "capsObject.toJSON" is not a function with Protractor >=2.4.0

Documented in related issue angular/protractor#2592:

I'm playing with Benchpress demo that works just fine for Protractor < 2.4.0 but with higher versions (I've tested with 2.4.0 & 2.5.0) I got this error:

 1) deep tree baseline should be fast!
  Message:
    Failed: TypeError: capsObject.toJSON is not a function
  Stack:
    Error: TypeError: capsObject.toJSON is not a function
        at proxyDone.fail (X:\perf-tests\benchpress-tree\node_modules\protractor\node_modules\jasminewd2\index.js:89:34)
        at promiseReactionJob (X:\perf-tests\benchpress-tree\node_modules\benchpress\index.js:8379:27)
        at Immediate._onImmediate (X:\perf-tests\benchpress-tree\node_modules\benchpress\index.js:8392:11)
    From: Task: Run it("should be fast!") in control flow
        at Object. (X:\perf-tests\benchpress-tree\node_modules\protractor\node_modules\jasminewd2\index.js:81:14)
        at attemptAsync (X:\perf-tests\benchpress-tree\node_modules\jasmine-core\lib\jasmine-core\jasmine.js:1819:24)
        at QueueRunner.run (X:\perf-tests\benchpress-tree\node_modules\jasmine-core\lib\jasmine-core\jasmine.js:1774:9)
        at X:\perf-tests\benchpress-tree\node_modules\jasmine-core\lib\jasmine-core\jasmine.js:1801:16
        at X:\perf-tests\benchpress-tree\node_modules\jasmine-core\lib\jasmine-core\jasmine.js:1745:9
        at X:\perf-tests\benchpress-tree\node_modules\protractor\node_modules\jasminewd2\index.js:18:5
        at Array.forEach (native)
    From asynchronous test:
    Error
        at Suite. (X:\perf-tests\benchpress-tree\tree_benchmark.spec.js:16:3)
        at addSpecsToSuite (X:\perf-tests\benchpress-tree\node_modules\jasmine-core\lib\jasmine-core\jasmine.js:743:25)
        at Env.describe (X:\perf-tests\benchpress-tree\node_modules\jasmine-core\lib\jasmine-core\jasmine.js:713:7)
        at jasmineInterface.describe (X:\perf-tests\benchpress-tree\node_modules\jasmine-core\lib\jasmine-core\jasmine.js:3219:18)
        at Object. (X:\perf-tests\benchpress-tree\tree_benchmark.spec.js:15:1)

Failed: TypeError: Cannot read property 'GET_LOG' of undefined

On Windows encounter following error when executing ".\node_modules.bin\protractor protractor.conf.js"