Improve the mechanism for testing Bluespec Inc bug 1490 about bsc HOT 3 CLOSED

I confirmed that this is still a problem with the recently released GHC 9.8.2. And I also see that it's an issue with 9.6 (both 9.6.2 and 9.6.4 the latest) -- I would guess that we just jumped to 9.8.1 and hadn't run the testsuite with 9.6. I opened an issue with GHC for it (issue 24487). If I use +RTS -s to dump statistics, I don't observe much of a difference between 9.4.8 and 9.8.2, nor is the runtime much different, so I'm actually not sure if this is a behavior difference in BSC and maybe it really is more directly GHC's doing. (That could maybe be confirmed by dumping a trace from BSC of its evaluation steps, and confirming whether BSC is doing the same evaluation in both cases.)

from bsc.

It appears that GHC 9.4 and 9.8 are using the amounts of memory, it's just that GHC 9.4 was not properly handling the RTS -M flag. (As indicated on the GHC issue, there was a fix to the accounting of heap space that was introduced in 9.6). On the GHC issue, the are graphs generated by using the RTS -S flag to generate trace of heap usage and actual live bytes (which may be less than the heap size due to minimum allocatable block sizes and fragmentation) etc. Those graphs are essentially the same for 9.4 and 9.8.

So GHC 9.8 is doing the right thing and the two failing tests just need a little more heap space to compile. (This is in contract to the VsortOriginal example that exhausts a 400M heap, and when I gave it 500M it churned for longer than I was willing to wait.) So I've submitted PR #681 to remove the check of the GHC version and just slightly increase the heap given to the two newly failing tests.

I wonder if these tests need to be restructured, to somehow directly measure what is actually being tested. I don't immediately understand what the tests are testing, or I would propose something. The tests appear to have for-loops which BSC's evaluator unrolls, and if BSC is not careful in how it optimizes (and prunes branching paths), then the unrolling could expand uncontrollably. If these a testing BSC's ability to keep the unrolling out of control (by proper optimizations in ITransform), then maybe a direct way to measure that is by looking at a trace from the evaluator. I guess that heap size was deemed to be a good enough proxy for the size of the unrolling. And maybe that's OK, since the one "expected fail" uses significantly more heap. Although we might consider having GHC report the actual live bytes, as a better measure. (Interestingly, for Bug1490MyUnion.bsv that seemed to be only 10M out of 256M!?) We could keep this GitHub issue open for that, but I'm coming around to thinking that the heap is a good enough proxy and that it may not be worth our time to keep looking at it. So I'm going to close it, but feel free to comment.

from bsc.

BSC is compiled with -with-rtsopts=-H256m -K10m -i1 which means that a minimum heap size of 256M is being set. It seems that GHC is ignoring the max heap -M value if the minimum is larger. So any -M value less than 256M has no effect, which is why BSC compiled with GHC 9.4 doesn't have a heap overflow with lower values. And the change in how the current heap size is calculated (starting in GHC 4.6) explains why the example started failing (because the old accounting was less than 256M while the new accounting is greater).

from bsc.