PowerLLEL_Channel is a Powerful paraLLEL solver for incompressible turbulent Channel flow. It is developed and optimized for massively-parallel Direct Numerical Simulation (DNS) of turbulent channel flow at high Reynolds number, e.g.,
PowerLLEL_Channel is mainly written in Fortran 90/2003 with reasonable modularity so that it can be extended in a sustainable manner. For performance reason, a C version of the PPE solver is also provided. Using a 2D pencil-like domain decomposition and the MPI+OpenMP hybrid parallel programming model, PowerLLEL_Channel shows excellent parallel performance with up to
Reference
Jiabin Xie, Jianchao He, Yun Bao & Xi Chen (2021) A Low-Communication-Overhead Parallel DNS Method for the 3D Incompressible Wall Turbulence, International Journal of Computational Fluid Dynamics, 35:6, 413-432, DOI: 10.1080/10618562.2021.1971202 [arXiv preprint]
The prerequisites are as follows:
- CMake (3.16.0 or newer)
- MPI (MPICH, Open MPI or Intel MPI)
- HDF5 (1.10.4 or newer)
- FFTW (3.3.4 or newer) / Intel MKL (2018 or newer)
- GPTL (optional, 8.1.1 or newer)
Assume that the PowerLLEL_Channel source package is unzipped to the directory $POWERLLEL_DIR
.
First, copy the template build script $POWERLLEL_DIR/examples/template/build_th2a.sh
to $POWERLLEL_DIR
.
Then, modify the build script according to the prompts in it.
Finally, execute the script.
The script will firstly build the executable PowerLLEL
in the newly created directory $POWERLLEL_DIR/build/install/bin
,
and then launch a simple test. Without any errors, the test will end successfully.
In general, the main PowerLLEL
executable reads the parameter file param.in
and outputs results *.out/*.h5
at working directory $WORK_DIR
. The best practice is:
- After a successful build, copy the
PowerLLEL
executable to the working directory$WORK_DIR
. - Copy the template parameter file
$POWERLLEL_DIR/examples/template/param.in
to$WORK_DIR
. - Modify simulation parameters in
param.in
as required. - Launch a parallel simulation with the following command:
mpirun -n $nprocs ./PowerLLEL
Note that the total number of MPI processes nprocs
should be equal to the product of parameters p_row
and p_col
in param.in
.
PowerLLEL_Channel stores 3D field data in HDF5 format (*.h5), and the file name format is inst_xxxxxxxx_y.h5
(the string of eight 'x's indicates the timestep, character 'y' indicates a flow variable such as u, v, w or p). In order to facilitate users to visualize 3D field data directly by visualization software such as ParaView and VisIt, we provide a shell script $POWERLLEL_DIR/utils/gen_xdmf.sh
to generate a XDMF descriptive file (*.xdmf) for the HDF5 file. The generated XDMF file can be read directly by the visualization software mentioned above. Please refer to gen_xdmf.sh
for detailed instructions.
PowerLLEL_Channel was originally developed by Jiabin Xie and Jianchao He, under the guidance of Prof. Yun Bao. Later, other contributors participated in the subsequent development and optimization. Their names are listed below in alphabetical order.
- Yun Bao
- Guangnan Feng
- Junxuan Feng
- Jianchao He
- Kan Wu
- Jiabin Xie
We appreciate any contributions and feedback that can improve PowerLLEL_Channel. If you wish to contribute to the code, please get in touch with the maintainers or open an Issue in the repository. Pull Requests are also welcome.
PowerLLEL_Channel is distributed under the MIT license.