I recently compiled PeleLM on two different supercomputers, with both gnu and intel on each, and consistently receive Erroneous arithmetic operation immediately upon running the simulation. I am just running the tutorial case out of the box: 2D FlameSheet, following instructions exactly from https://pelelm.readthedocs.io/en/latest/GettingStarted.html .

I have tried editing a number of different inputs, but it seems like the error is pretty robust. I have also seen it occur in other tutorial cases. Has anyone else noticed this issue?

Hopefully there is an easy solution to the problem, but I am happy to provide more details (compilers, etc.) if they are needed. Thanks!

Hi, I was trying to compile FlameSheet with USE_OMP = True but got the following error while compling AMReX_FABUTIL_2D.F

Compiling AMReX_FABUTIL_2D.F ...
In file included from /home/amrex/Src/Base/AMReX_REAL.H:4,
                 from /home/amrex/Src/Extern/amrdata/AMReX_FABUTIL_2D.F:9:
tmp_build_dir/s/2d.gnu.OMP.EXE/AMReX_Config.H:57:2: error: #error libamrex was built with OpenMP
   57 | #error libamrex was built with OpenMP
      |  ^~~~~

Any leads on what might be causing this? I was able to the compile and run the case using with mpi.
Thanks!

I use “python ${PELE_PHYSICS_HOME}/Support/Fuego/Mechanism/Models/useful_script/script_trans.py mechanism.inp tran.dat” to obtain a new mechanism, but always get an error: File "~/useful_script/script_trans.py", line 53, in
tran_list.append(outlines[spec]), KeyError: '!'
I hope someone can help me, thanks a lot.

If this file doesn't exist in the CWD as specified by the example input files, it will just segfault rather than tell the user it doesn't exist. I can add this in the future. Creating this issue as a reminder.

AMReX-Hydro commit 68d9c764738d removes the (EB)Godunov::ComputeAofs functions called directly in PeleLM. PeleLM can either lift those functions from the prior commit (df452661a000) or make use of NSB::ComputeAofs (updated in IAMR commit 2c6603a8f55d5f)

Implement/extend unit test feature from PeleLM_NS allowing "PeleLMEvaluate"

I am currently working on bruno/HIT_forcing project. However, I am running into issues at runtime and compile time.

I manage to compile the code for the ForcedHIT_air case, however the input file is missing, I created a new one. When executing the code, I face the missing forcedata.dat file which obviously would be a file with values for the forces?

The file "forcedata.dat" is read in TurbulentForcing_def.H from line 46, there is mention of // Read magic forcing data from file I wish I could find that magic file in order to run the simulation. Can you help me build/find this file?

How to implement isotherm boundary conditions on embedded boundaries in PeleLM?
Can anyone give me some advice？？thanks in advance

Hello everyone,
I am trying to run the EB_FlamePastCylinder case with higher refinement levels. The case seems to crash when I try to increase the amr.max_level to 1. I first ran the case with amr.max_level set to 0 and getting checkpoints after every 5 steps. I set my max_step to 300000 and stop_time to 1 millisecond. This first step runs smooth. When I wish to advance the solution from 1millisecond to 2 millisecond (by setting the amr.restart=chk.... and amr.regrid_on_restart=1) using a higher-refinement level (i.e. setting amr.max_level=1it throws the following error:
amrex::Abort::0::deltaT_iters not converged !!! SIGABRT
Could I please know where am I making a mistake?
Also, is there a way of setting the surface temperature of the cylinder to a fixed value? i.e. having an isothermal cylinder surface?
Thanks

Hi,

I have a compilation error for the test case FlameSheet.

It seems some variables are removed from the IAMR but not in PeleLM source files.
Using OpenMPI and Intel compiler.

Thank you.
The error is as follows:

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(687): error: identifier "variable_vel_visc" is undefined
if (variable_vel_visc != 1)
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(692): error: identifier "variable_scal_diff" is undefined
if (variable_scal_diff != 1)
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3281): error: no suitable constructor exists to convert from "amrex::Real={amrex_real={double}}" to "amrex::Vector<amrex::MultiFab *, std::allocator<amrex::MultiFab *>>"
diffusion->diffuse_scalar(dt,state_ind,1.0,rhonph,rho_flag,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3281): error: no suitable constructor exists to convert from "const int" to "amrex::Vector<amrex::MultiFab *, std::allocator<amrex::MultiFab *>>"
diffusion->diffuse_scalar(dt,state_ind,1.0,rhonph,rho_flag,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3281): error: initial value of reference to non-const must be an lvalue
diffusion->diffuse_scalar(dt,state_ind,1.0,rhonph,rho_flag,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3281): error: no suitable user-defined conversion from "amrex::MultiFab" to "const amrex::Vector<amrex::MultiFab *, std::allocator<amrex::MultiFab *>>" exists
diffusion->diffuse_scalar(dt,state_ind,1.0,rhonph,rho_flag,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3282): error: argument of type "amrex::MultiFab **" is incompatible with parameter of type "int"
SpecDiffusionFluxn,SpecDiffusionFluxnp1,sigma,&Force,sigma,alpha,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3282): error: argument of type "amrex::MultiFab **" is incompatible with parameter of type "int"
SpecDiffusionFluxn,SpecDiffusionFluxnp1,sigma,&Force,sigma,alpha,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3282): error: argument of type "amrex::MultiFab *" is incompatible with parameter of type "amrex::Real={amrex_real={double}}"
SpecDiffusionFluxn,SpecDiffusionFluxnp1,sigma,&Force,sigma,alpha,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3282): error: no suitable constructor exists to convert from "amrex::MultiFab *" to "amrex::MultiFab"
SpecDiffusionFluxn,SpecDiffusionFluxnp1,sigma,&Force,sigma,alpha,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3283): error: argument of type "Diffusion::SolveMode" is incompatible with parameter of type "amrex::MultiFab *"
alphaComp,betan,betanp1,betaComp,solve_mode,add_old_time_divFlux);
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(3283): error #165: too few arguments in function call
alphaComp,betan,betanp1,betaComp,solve_mode,add_old_time_divFlux);
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5736): error: no suitable conversion function from "amrex::FArrayBox" to "int" exists
D_DECL(cflux[0],cflux[1],cflux[2]),
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5737): error: no suitable conversion function from "amrex::FArrayBox" to "int" exists
D_DECL(edgstate[0],edgstate[1],edgstate[2]),
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5738): error: initial value of reference to non-const must be an lvalue
Smf[S_mfi], 0, nspecies+1 , force, 0, divu, 0,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5738): error: no suitable conversion function from "const amrex::FArrayBox" to "int" exists
Smf[S_mfi], 0, nspecies+1 , force, 0, divu, 0,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5738): error: no suitable constructor exists to convert from "int" to "amrex::FArrayBox"
Smf[S_mfi], 0, nspecies+1 , force, 0, divu, 0,
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5739): error: no suitable conversion function from "amrex::FArrayBox" to "int" exists
(*aofs)[S_mfi], first_spec, advectionType, state_bc, FPU, volume[S_mfi]);
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5739): error: no suitable constructor exists to convert from "int" to "amrex::FArrayBox"
(*aofs)[S_mfi], first_spec, advectionType, state_bc, FPU, volume[S_mfi]);
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5739): error: no suitable conversion function from "amrex::Vector<AdvectionForm, std::allocator>" to "int" exists
(*aofs)[S_mfi], first_spec, advectionType, state_bc, FPU, volume[S_mfi]);
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5739): error: a reference of type "amrex::FArrayBox &" (not const-qualified) cannot be initialized with a value of type "amrex::Vector<int, std::allocator>"
(*aofs)[S_mfi], first_spec, advectionType, state_bc, FPU, volume[S_mfi]);
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5739): error: no suitable user-defined conversion from "amrex::FArrayBox" to "const amrex::Vector<AdvectionForm, std::allocator>" exists
(*aofs)[S_mfi], first_spec, advectionType, state_bc, FPU, volume[S_mfi]);
^

/home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp(5739): error #165: too few arguments in function call
(*aofs)[S_mfi], first_spec, advectionType, state_bc, FPU, volume[S_mfi]);
^

compilation aborted for /home/565/ska565/tools/PELELM/PeleLM/Source/PeleLM.cpp (code 2)
make: *** [/home/565/ska565/tools/PELELM/amrex/Tools/GNUMake/Make.rules:184: tmp_build_dir/o/2d.intel.EXE/PeleLM.o] Error 2

How to interpolate from coarse to finer resolution grid when amr.max_level = 0 in PeleLM?

Hello everyone, I am new to PeleLM and I would like to set up Moser's channel flow with periodic boundaries. For that Im struggling with the driving pressure gradient for the momentum. How can I implement this if I know the gradient?
Best regards and thanks in advance :)

The behavior of that feature default to an error if fuelname is not specified.

the make -j4 did not work on Windows. I would like to know, why this command don't work on git with Windows ?

When I compile the PeriodicCases or FlameSheet cases with gnu compilers (I tried versions 8.4 and 10.1) in DEBUG mode on Eagle, the cases run successfully to completion but when the simulation is finalizing I get the following error:

AMReX (21.11-14-ga2245756e64c) finalized
pure virtual method called
terminate called without an active exception
Aborted (core dumped)

This does not happen when compiling with DEBUG = FALSE, or with COMP = llvm and DEBUG = TRUE.

when I tried to run examples included in PeleC/PeleLM, some went wrong with the same error notes---

../../../Submodules/amrex/Src/Base/AMReX_ccse-mpi.H:14:10: fatal error: mpi.h: No such file or directory
   14 | #include <mpi.h>
      |          ^~~~~~~
compilation terminated.
make: *** [../../../Submodules/amrex/Tools/GNUMake/Make.rules:257: tmp_build_dir/o/2d.gnu.MPI.EXE/AMReX_SUNMemory.o] Error 1

Is there any way that could figure it out? Besides, just some of modules like PeleLM/Exec/RegTests/EB_FlowPastCylinder and PeleC/Exec/Production/ChallengeProblem could not work properly, the other one, PeleC/Exec/RegTests/EB-C13, I tested worked well.

Add the machinery that will allow the user to specify EB boundary Dirichlet values.

Whenever I try to run premixed flames on high resolution, I get "MLMG failed".
Particularly, FlameSheet problem on 128x128 grid fails at the initialization step, while on 32x32 grid it runs okay.

When EB and 1 AMR level are used in 3D, the code crashes during first time-step.

With DEBUG=FALSE, the code crashes on NaNs found in the routine calc_dpdt, called at line 6966 in PeleLM.cpp. A close look at the S multifab reveals that cells at coarse/fine interface contain junk.

With DEBUG=TRUE, the code crashes with amrex::Abort::0:: (8,12,0,8) is out of bound (8:15,12:19,0:15,0:0) !!!, which correspond to the call crse_level.average_down(S_fine_loc, S_crse_loc, RhoRT, 1); at line 6958, just above.
It is not clear if these two crash are linked.

This problem is relatively new, EB and AMR were working about 1 month ago, so a bug has certainly be introduced during the recent port to gpu.

The problem can be reproduced with
Exec/RegTests/EB_FlowPastCylinder/inputs.3d-regt

Is there anyone have trouble like this---Fatal Error: Cannot open module file ‘amrex_fort_module.mod’ for reading at (1): No such file or directory
compilation terminated.

The USE_WBAR = TRUE option is currently not working in the development branch. The core functions need to be rewritten when ported to GPU.

peleLM.lo_bc = Inflow NoSlipWallIsotherm
peleLM.hi_bc = Outflow NoSlipWallIsotherm
the above is my boundary settings, when starting the computation, I always get the error：

With fresh development branches from IAMR, PelePhysics and AMReX, PeleLM does not compile for the FlameSheet problem:

Compiling pmf.cpp ...
In file included from [...]/PelePhysics_GitHub/Reactions/Fuego/CPU/cvode/reactor.cpp:1:0:
[...]/PelePhysics_GitHub/Reactions/Fuego/CPU/cvode/reactor.h:6:82: fatal error: cvode/cvode.h: No such file or directory
compilation terminated.

I have done a make TPL before.

I want to cite pele since I used your code to generate data using AMR. Which reference should I go for?

Hi, everyone, I would like to ask you a question.
I follow the PeleLM Quickstart to run FlameSheet and it turns all right.
But when I only change the max_step from 20 to 200, it will abnormally stop at 115 step.
So what wrong with it and how can I make it run correctly?

Consider modifying the AMR ref. indicator API slightly in order to avoid clashing ParmParse keywords in amr.
Perhaps something like this:
amr.refinement_indicator_list = RefineBox
amr.refinement_indicator.RefineBox.in_box_lo = 0 0.1 0.2
amr.refinement_indicator.Refinebox.in_box_hi = 0.1 2.1 4.3

Hello,

I noticed that the thermal conductivity coefficient is computed incorrectly:
The code (PeleLM_K.H, line 477) states:
lambda(i,j,k) = mu_cgs * 1.0e-1_rt * PrInv / cpmix; // Constant Prandtl number

However, it should be:
lambda(i,j,k) = mu_cgs * 1.0e-1_rt * PrInv * cpmix; // Constant Prandtl number

the make -j4 did not work on MacOS because the c++ standard was not specified in the Makefile and an older standard was used by default

Adding CXXFLAG = -std=gnu++17 to the Makefile allowed the PeleLM Executable to be built

I do not know if the flag could be specified by default ?

I‘m new to this production. I'm still trying to get my environment work. But when I test it using /RegTests, there are two different kind of issues. One like ControlledInflow it didn't work with information "../../PeleLM/Tools/Make/Make.PeleLM:54: *** EOS Soave-Redlich-Kwong not availabel in PeleLM. Stop." Another like HIT something goes wrong with notes ---../../PelePhysics/Reactions/ReactorCvodePreconditioner.cpp:310:undefined reference to 'SUNDlsMat_denseGETRS. collect2:error: ld returned 1 exit status'. Unfortunately, I got no idea to fix these. I will be appreciated if you could give me some advice.

Hello, I am trying to simulate the external flow around four cylinders using PeleLM. I decided to create a new case by copying the FlowPastCylinder tutorial and tweak it for my case. I am confused about how I can create a union of the embedded boundaries of the 4 spheres in the my case. I have done this in PeleC by adding a new if case in the PeleC_init_eb.cpp file and copying it to my case directory.This worked fine. I am not able to locate a similar file where I can make changes to the EB or add a new case to simulate more complex geometries in PeleLM. Any help on this is much appreciated.
For the sake of clarity, my geometry would look something like this: (with a smaller number of cylinders):

Thanks
Surya

I have compiled a PeleLM simulation code using the icc, icpc, and ifort compilers found in the oneAPI 2023.0 release, and I tried to run the code using the mpi library found in the same release which gave me the following error. The same setup with GNU compilers did run successfully without any errors.

amrex::amrex::Abort::3::Abort::amrex::2::FORTRAN:RD_SCL_FLCTHD: infl_type is not set !!!
SIGABRT
Abort::0::FORTRAN:RD_SCL_FLCTHD: infl_type is not set !!!
SIGABRT
FORTRAN:RD_SCL_FLCTHD: infl_type is not set !!!
SIGABRT
amrex::Abort::1::FORTRAN:RD_SCL_FLCTHD: infl_type is not set !!!
SIGABRT
See Backtrace.2 file for details
See Backtrace.0 file for details
See Backtrace.1 file for details
See Backtrace.3 file for details
Abort(6) on node 2 (rank 2 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 6) - process 2

All the back trace files are the same so I have attached one of them as a txt file below.

Backtrace0.txt

Hi I am running a case with the SRK EOS of PelePhysics by setting

Eos_Model  = Soave-Redlich-Kwong

in the GNUMakefile and though having not called the function T2Hi explicitly anywhere in my case files I get the following error:

amrex::Error::0::T2Hi not physically possible for SRK EoS !!!
SIGABRT
See Backtrace.0 file for details

I guess it's called somewhere in the PeleLM.cpp file. Is there anyway I can fix this?

Thanks

Not done yet for some reason.

Hi! When I run examples in Exec, some of them went wrong with notion like:

../../../Submodules/PelePhysics/Reactions/ReactorUtils.H:197:29: error: ‘pele::physics::PhysicsType’ has not been declared
  197 |   auto eos = pele::physics::PhysicsType::eos();
      |                             ^~~~~~~~~~~
../../../Submodules/PelePhysics/Reactions/ReactorUtils.H: In function ‘void pele::physics::reactions::utils::fKernelSpec(int, int, amrex::Real, int, const Real*, amrex::Real*, const Real*, const Real*, const Real*)’:
../../../Submodules/PelePhysics/Reactions/ReactorUtils.H:307:29: error: ‘pele::physics::PhysicsType’ has not been declared
  307 |   auto eos = pele::physics::PhysicsType::eos();

Is that caused by eos unavailable in PeleLM or PelePhysics environment needed additional configuration? If you guys have any methods to figure it out, please let me know.
Thanks!

When I tried to make PeleLM/Exec/RegTests/FlameSheet, something went wrong with note---make: *** No rule to make target 'Transport.cpp', needed by 'tmp_build_dir/o/2d.gnu.DEBUG.MPI.EXE/Transport.o'. Stop.
How could I fix this?

Hi, everyone, I would like to ask you a question.
I want to export the reaction source term and diffusion term during the calculation, so how do I set it in the input file? ,

The routine for calculating element atomic weights is implemented as

Vector
ChemDriver::elementAtomicWt() const
{
Vector awt(numElements());
CD_MWT(awt.dataPtr());
return awt;
}

This returns the species molecular weights rather than the element atomic weights. The correct implementation should be:

Vector
ChemDriver::elementAtomicWt() const
{
Vector awt(numElements());
FORT_GETCKAWT(awt.dataPtr());
return awt;
}

Will need to be done in coordination with IAMR/PeleC

Hi,there. When I running Exec/RegTests/EB_FlamePastCylinder, the make process went well but something work not properly during ./PeleLM3d.gnu.MPI.ex inputs.3d-regt . The error occured with "amrex::Abort::0::MLMG failed !!!
SIGABRT
See Backtrace.0 file for details
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 6.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them."

The FlameInABox problem encounters a compilation error when using the Intel v19.0.3.199 compiler:

../../../IAMR/Source/Src_2d/PROJECTION_2D.F90(268): error #6415: This name cannot be assigned this data type because it conflicts with prior uses of the name.   [ANEL_LO]
      integer    mult,anel_lo,anel_hi
----------------------^
../../../IAMR/Source/Src_2d/PROJECTION_2D.F90(268): error #6415: This name cannot be assigned this data type because it conflicts with prior uses of the name.   [ANEL_HI]
      integer    mult,anel_lo,anel_hi
------------------------------^

To reproduce on Cori, use these commits:

amrex: ab0d7a90a
PelePhysics: 2f533e5
IAMR: f1935d5
PeleLM: d62b4f6

And execute the following:

module load cdt/19.03
module swap intel{,/19.0.3.199}
make DEBUG=FALSE USE_ACC=FALSE USE_MPI=TRUE USE_OMP=TRUE DIM=2 COMP=intel TEST=TRUE USE_ASSERTION=TRUE