OpenFAST chkp corrupted on frontier about amr-wind HOT 10 CLOSED

I'm not in a position to test this at the moment. I can put it on my backlog though. This would be more of an openfast core issue than a nalu-wind issue.

from amr-wind.

Confirmed that this is fixed with OpenFAST/openfast#2097

from amr-wind.

In nalu-wind we rely on the openfast-cpp interface which also loops over indices starting at 0:
https://github.com/OpenFAST/openfast/blob/4b6337fcffe859c5eeb5445deeef2046439e5152/glue-codes/openfast-cpp/src/OpenFAST.cpp#L58-L78

We'll have to dig deeper into openfast to see if this offset is handled inside the openfoam data structures.
@gantech do you know off hand?

from amr-wind.

Yeah that's interesting. Can you throw a print statement in that if condition I mentioned and see if it ever hits that close call? I am worried this isn't being handled right with the nalu-wind code path as well.

from amr-wind.

I tried a naive approach to get rid of the off-by-1 error. Basically I replaced this line

        auto my_tid_local = fi.tid_local+1;
        fast_func(FAST_CreateCheckpoint, &my_tid_local, rst_file);

However that also resulted in segfaults when writing out the chkp files. So in the end I just commented out the if else check in
https://github.com/OpenFAST/openfast/blob/4b6337fcffe859c5eeb5445deeef2046439e5152/modules/openfast-library/src/FAST_Subs.f90#L7090 that @marchdf mentioned above.

Lawrence

from amr-wind.

Yeah that's not the right fix. Turns out this is a bit messed up. You need to increment global_id by one to pass in. That sets the TurbId, which would then fix the chkp close check. The problem is that, on initialize, that argument doesn't get passed around correctly: the call site of FAST_InitializeAll_T and the argument list don't match:

The definition of the function: https://github.com/OpenFAST/openfast/blob/main/modules/openfast-library/src/FAST_Subs.f90#L37 has TurbId as it's second parameter. So that's fine. Except that

The call site: https://github.com/OpenFAST/openfast/blob/main/modules/openfast-library/src/FAST_Library.f90#L144: passes in iTurb to that argument and not TurbId.

I am in conversation with @andrew-platt for other fixes.

from amr-wind.

tracking this here: OpenFAST/openfast#2064

from amr-wind.

I did a little digging into this. There is a discrepancy between how we handle the turbine id with FAST.Farm and with the cpp interface. With FAST.Farm, we index using the Fortran index start of 1 for the turbine array between 1:NumTurbines. However, in the FAST_AllocateTurbines routine in FAST_Library.f90, a start index of 0 is expected.

As noted above, since the closing of the checkpoint file assumes the start index of 1, it is never closed with the cpp interface. So to fix this, I think we have three options.

1. Change FAST.Farm to index turbines starting at 0,
2. Change FAST_AllocateTurbines to start with a Fortran index of 1, and change amr-wind and other codes to match
3. If FAST_AllocateTurbines is called, we could set an internal flag to correctly handle this offset of turbine number.

I'm inclined to pursue option 3 as this will preserve the existing numbering systems for cpp and FAST.Farm . I don't think it will be all that difficult to do in OpenFAST. I'll post here with when I have a proposed solution in place.

from amr-wind.

Hi @andrew-platt , thank you for looking into this! I appreciate you digging through this. Option 3 sounds good. I do still think (as I noted in the openfast issue) that there seems to be an inconsistency in the way iTurb is being passed to FAST_InitializeAll_T and not the expected (from the argument list in the function definition) TurbId. But I probably don't understand the reasoning behind that and the intricacies of the coupling to all the fortran and cpp codes. Anyway, happy to help and test potential solutions. Thanks!

from amr-wind.

Proposed solution: OpenFAST/openfast#2097

from amr-wind.