Time To Frist Response regression on NativeAOT about benchmarks HOT 5 CLOSED

@MichalStrehovsky @eerhardt

from benchmarks.

The runtime diff: dotnet/runtime@49bf70a...b1dbbb9

There were 2 other runtime builds in between those commits - "8.0.0-rc.2.23471.3" and "8.0.0-rc.2.23471.9". Trying to isolate this, it appears the regression came in between 8.0.0-rc.2.23470.7 and 8.0.0-rc.2.23471.3. Here is the commit sha of each of the 4 builds:

8.0.0-rc.2.23470.7- 49bf70a429a6122217f2c88e778ce8115ceac3bd
8.0.0-rc.2.23471.3- 1757497f37245d1756870fb1c0b3221c0d235d54
8.0.0-rc.2.23471.9- 12fc9044307e32e143eb7cbaf1d68badf61b192c
8.0.0-rc.2.23471.30 - b1dbbb964a4daba07fe13c4150133f1d5d41ceb0

The diff between the builds that seem to cause the regression is: dotnet/runtime@49bf70a...1757497

from benchmarks.

Given the full chart with the workstation GC mode looks like this (i.e. startup with workstation GC unaffected):

My bet is on the DATAS change in that commit range. Cc @Maoni0

Btw @sebastienros any plans to start measuring main? Looks like we also have a size regression in main already and being able to pinpoint a commit range would be helpful.

from benchmarks.

@MichalStrehovsky We are already measuring main but display these in KPIs only. So we have the whole data for the past month on about 10 benchmarks. You could try a dichotomy to find the commit that regressed it if you know the benchmark that is affected (I can explain how if you have never done that with crank). But it's limited to the most common ones because the SDK doesn't support anything that has a package reference right now when targeting .net90.

from benchmarks.

Was due to switch to anoother server.

from benchmarks.