VR crashes when computing black hole subgrid masses about velociraptor-stf HOT 11 CLOSED

Hi,

This is just to confirm that we can also reproduce the error. We were on our way to double-check that the changes in #14 were correct by running VR offline against SWIFT and hit the exact same issue. This seems to be completely unrelated to #13 though.

from velociraptor-stf.

The problem seems to be that the dynamic list of open datasets is not correctly used when reading. Specifically, I think iextraoffset is not set correctly the in the loop starting on line 2516 of hdfio.cxx. The values are not set in a similar way in the first loop (line 2342) and the code does not read the correct dataset.
It "magically" works for gas extra properties as they are all at the start of the array.

A temporary very very dirty fix for the case of EAGLE/COLIBRE in SWIFT is to do this:

patch.txt

(don't do this at home!)

It forces the code to read the correct dataset. It will of course only work for the specific configuration file used by XL and friends.

from velociraptor-stf.

@MatthieuSchaller just before leaving for our break we had also been looking around these lines with @cdplagos, trying to understand where the indexing goes wrong. Of course the patch above is fragile, so a more robust solution (which we'll try to get done this week) is needed.

from velociraptor-stf.

Glad we all think the issue is around these lines. The patch is definitely not for public consumption but I listed it here for our colleagues in EAGLE using exactly the same configuration file.

If it helps with the debugging, I noticed that you also run into trouble if you add extra star properties. Only the gas seems to be OK.

from velociraptor-stf.

An update on this: I have identified the problem that is causing this crash. As we had guessed it was an indexing issue, although in a different point of the code. In #15 (comment) @MatthieuSchaller correctly indicates that the indexing into the arrays is different between the first loop (which opens datasets and dataspaces) and the second one (where they are actually read). However, the first loop indexes past the size of the arrays (writing outside the vector's allocated memory), while the second indexes within the array boundaries. This makes it seem that the correct indexing scheme is the second one, so I fixed the first rather than adjusting the second.

After this VR runs almost to completion, until a new access into these arrays in a different part of the code produces again a crash (when using -O2 or higher, maybe also with -O1). I'm currently looking at this and I hope I'll have something working soon.

from velociraptor-stf.

Thanks for the update.

Maybe adding extra star properties to the config file can also help identify problems.

from velociraptor-stf.

This problem has been fixed now, and the fix is on the latest master.

However note that, as outlined in #15 (comment), a further problem appears down the road now that prevents VR from successfully finishing. I could at least confirm that this newly found problem is not caused by the fixes introduced for this issue, and as such I'll be opening a new issue to keep track of it.

from velociraptor-stf.

The new issue described in #18 (and first mentioned in #15 (comment)) is now also fixed (via #19).

from velociraptor-stf.

Oh excellent news! Thanks for tackling these.

from velociraptor-stf.

I can confirm that things work correctly.

from velociraptor-stf.

Great, I'm closing this issue then.

from velociraptor-stf.