Linear regression in benchmarks does not work well about store HOT 3 CLOSED

Sorry, my mistake.

Getting consistent results seems to be a matter of passing the right rts options. In particular, parallel gc seems to spoil the results.

If I turn off parallel gc, and also only use one core, I get perfectly straight lines.

stack bench store --benchmark-arguments '--output=bench.html +RTS -N1 -s -qg -RTS

Also, the time the benchmarks take goes down significantly.

from store.

If those RTS options are necessary to get reliable results, we should add them to the ghc-options filed in store.cabal.

It's also a bit worrying that parallel GC has an effect. It should be conceptually a single-threaded program.

from store.

Yep, I agree with using ghc-options: -with-rtsopts="-N1 -s -qg" in the benchmark stanza, if that makes it more consistent. Maybe this is a good thing to have for any similar benchmark?

I thought criterion had some clever stuff to trigger GC in between benchmarks? I suppose it's unavoidable in some of these that we're at least going to have a number of minor GCs of the nursery. Would it make any sense to also use a massive nursery? (e.g. -A64m, 128 times the default)

from store.