kzykhys / markbench Goto Github PK

Markbench requires all the parsers to be installed and always runs the benchmark towards all of them.

Benchmarking could be useful even when ran for a single parser, but across multiple versions. For example in a CI for each push. This way finding performance hits from bug fixes and improvements to the speed would be easier.

It seems parallel execution of benchmark makes it unfair. It depends on execution order very much and one test affects another.

In order to make it fair all parsers should be tested sequentionally. PHP process should be killed after each test.