Automatically generate performance report about aurelia HOT 8 CLOSED

Additional benchmarks to look at:
https://localvoid.github.io/uibench/

from aurelia.

@fkleuver I think I've spent enough time with krausest benchmark to know what and how it does it..
I can try to set it up..
We'll have to talk details over

from aurelia.

Thanks, that would be awesome!

My initial idea, for starters, was to simply duplicate the effective benchmarks but simplifying/cleaning up the infrastructure around it. I think the benchmarks as-is are pretty decent but we need some additional benchmarks as well, so we'll eventually need to break it open a bit and add some more interesting tests in there. But that's for later.
At the moment I'm primarily interested in:

1. How vNext compares to vCurrent (in startup time, repaint rate, memory use, the usual stuff)
2. How different versions of vNext compare to each other

I don't know if you had any particular approach in mind but I'd suggest to start simple and just add the krausest repo to a PR as-is, stripping out every framework except vNext and vCurrent, and just work / clean it up from there.
As long as the npm scripts are present in the package.json to bootstrap and run the tests, and some html report is produced, I can make sure it's ran automatically in CI

from aurelia.

I think there should be a workflow.. to "add" vnext by tag..
so a task that'll "add" a new framework getting specific version..
and for CI it could be "dev"
but if we want to compare across timeline.. we want to leave artifacts..

from aurelia.

I think there should be a workflow.. to "add" vnext by tag..

Yep, if you could get a working setup in for starters then I can probably easily take care of that piece of automation.

but if we want to compare across timeline.. we want to leave artifacts..

We do probably want to leave artifacts anyway. I think they can live in a performance branch or something just fine. But ultimately that's a really small implementation detail I can take care of. It would be similar to the existing publish scripts that push the artifacts to the dev branch before publishing.

But if you feel this complicates things a bit too much I've no issues leaving that bit out for now. It's much more important to have a benchmark running regularly in the first place. We can just start with plain hard coded vnext@dev version vs vcurrent@latest comparison.

from aurelia.

This is done now, right @fkleuver ?

from aurelia.

Not quite yet. We still need the performance report to include latest and dev channels so we got a direct comparison of the impact of the current PR's changes

from aurelia.

@fkleuver are we waiting on the azure/cosmos setup or can we close?

from aurelia.