softlab-ntua / bencherl Goto Github PK

I try to make ui and it show:
Compile Error, "src/lib/rfc4627.erl" -> [{"/home/haimh/git/softlab-ntua/bencherl/ui/bencherlui/src/lib/rfc4627.erl",[{382,erl_parse,["syntax error before: ","unsigned"]}]}]
11:28:46.427 [error] Load Module Error lib_modules : [[{"/home/haimh/git/softlab-ntua/bencherl/ui/bencherlui/src/lib/rfc4627.erl",[{382,erl_parse,["syntax error before: ","unsigned"]}]}]]
11:28:46.723 [error] Compile Error, "src/controller/bencherlui_results_controller.erl" -> [{"/home/haimh/git/softlab-ntua/bencherl/ui/bencherlui/src/controller/bencherlui_results_controller.erl",[{137,erl_parse,["syntax error before: ","length"]}]}]
11:28:46.723 [error] Load Module Error controller_modules : [[{"/home/haimh/git/softlab-ntua/bencherl/ui/bencherlui/src/controller/bencherlui_results_controller.erl",[{137,erl_parse,["syntax error before: ","length"]}]}]]

Kjell's nice UI currently needs to be compiled and started with an older version of Erlang.

For now, a workaround is to change the otp version used for bencherl after starting the UI.

When using mawk 1.3.3 bencherl fails with the following error message after running the ets_bench benchmark:

awk: run time error: not enough arguments passed to printf("(delwait,"mixed=l:0.00%,=u:100.00%=131072"-[ordered_set,14,15,17,1.0,rw,1,{0,0,0}])")
FILENAME="-" FNR=1 NR=1

The problem seems to be the percentage sign. The result is that at least one data file is not produced correctly. It could be fixed by replacing all % signs with %% in the strings that are printed with awk's printf function. Everything works correctly when using gnu awk 4.1.0 instead of mawk.

See bencherl/bench/ets_test/src/ets_test.erl for the referenced code.

main bug in ets_test

The ets_test microbenchmark has a severe bug that is not catched by bencherl:
When running concurrently, the w-workers will inserts numbers from N down to 1, while the r-workers will lookup these same values from N down to 1.
If one r-worker does this faster, the match in line 75 will fail, and the worker process wil crash (badmatch). This will then bring the main Process down, as they are linked, and subsequently bring the entire benchmark to a hold.
bencherl does not catch that the benchmark application crashed, which probably could be changed to find such errors more easily on future benchmark applications.
Thus this benchmark currently measures how long it takes for a r-worker to overtake all w-workers.

The easiest way to solve this problem would be to just remove the match in line 75, but reversing lines 65 and 66 would also be an option. However, this would change the semantics of the benchmark even more than the easier change.

other problems

Another problem with the ets_test benchmark is that it uses
T = ets:new(x, [public]),
in line 43, which creates an ETS table of type set with read_concurrency and write_concurrency both set to false.
This somewhat counteracts any kind of scaling behaviour expected from this benchmark.

As there are more issues to the usefulness of this benchmark, like all workers having the exactly same access pattern to the table, the long term solution maybe is to replace it with some more expressive ETS benchmarks. I will create a pull request for mine once it is stable.