triqs / mpi Goto Github PK

Prerequisites

Having a OpenMPI version that was compiled with cxx bindings.

Description

Certain versions of OpenMPI (including the most recent stable 4.0.x) still include cxx bindings, which can be optionally compiled to a separate library libmpi_cxx. Depending on the linux distribution these may or may not be built by default.

If they were build, the mpi.h header (superficially "c only") will by default also include all the cxx bindings.

From ompi/mpi.h (https://github.com/open-mpi/ompi/blob/v4.0.x/ompi/include/mpi.h.in):

#if !defined(OMPI_SKIP_MPICXX) && OMPI_BUILD_CXX_BINDINGS && !OMPI_BUILDING
#if defined(c_plusplus) || defined(__cplusplus)
#include "openmpi/ompi/mpi/cxx/mpicxx.h"
#endif
#endif

Then when we try to compile a program against our own mpi_c target, it complains of the unresolved symbols (since we are not linking against libmpi_cxx).

The solution would be to include

#define OMPI_SKIP_MPICXX

Somewhere before we import the mpi.h header.

The only question is: do we want to do this simply as a global definition in a header. Or do this in cmake, so that that recognizes if this will be a problem and adds the appropriate flag.

Finally, MPICH2 also cxx bindings. I did not check if they could cause problem there, but a similar issue could arise.

Versions

Current unstable branch: b6e6dd3

Description

Compiling the newest version (unstable branches triqs and mpi) of triqs with MPICH does not work for me anymore. I'm using the same dockerfile as last December, when it still worked.

Steps to Reproduce

1. Following the procedure from triqs jenkins, I pasted together the Dockerfile.ubuntu-gcc and Dockerfile.build and then added a git clone command to the dockerfile to get triqs into the docker container. This compiles without problems (as expected by the successful checks by jenkins).
2. I then change from openMPI to MPICH and installing mpi4py through pip to link it to mpich, which leads to this dockerfile. Now the compilation stops with the error

[  7%] Linking CXX shared module copy_move_stat.cpython-38-x86_64-linux-gnu.so
In file included from /src/triqs/c++/triqs/utility/exceptions.hpp:23,
                 from /src/triqs/c++/triqs/atom_diag/impl/../atom_diag.hpp:26,
                 from /src/triqs/c++/triqs/atom_diag/impl/atom_diag.cpp:21:
/home/build/triqs/deps/mpi_src/c++/mpi/mpi.hpp: In member function 'void mpi::communicator::barrier(long int)':
/home/build/triqs/deps/mpi_src/c++/mpi/mpi.hpp:127:29: error: cannot convert 'std::nullptr_t' to 'MPI_Request' {aka 'int'} in initialization
  127 |           MPI_Request req = nullptr;
      |                             ^~~~~~~

Based on the error message, I'd guess that commit 25e6fe9 from Jan 25 causes the problem.

Thank you already for your help!
Cheers, Max

There is currently no all_reduce in-place operation implemented.

This would be very useful to add to avoid large copies.

CMake Error at /home/build/triqs/deps/mpi_src/c++/mpi/CMakeLists.txt:51 (target_compile_definition):
  Unknown CMake command "target_compile_definition".

I'm guessing this should be target_compile_definitions but with that I get:

CMake Error at deps/mpi/c++/mpi/CMakeLists.txt:51 (target_compile_definitions):
  target_compile_definitions may only be set INTERFACE properties on
  INTERFACE targets

FWIW, cmake 3.10.2's FindMPI seems to include MPI_CXX_SKIP_MPICXX.

Description

libmpicxx should not be required to compile the TRIQS/mpi headers, but is listed in the INTERFACE_LINK_LIBRARIES
of the mpi::mpi_c cmake target, only when building against MPICH.

See e.g. PREFIX/lib/cmake/mpi/mpi-targets.cmake file for the py38h82d08ca_3 build of the https://github.com/conda-forge/triqs-feedstock

 67 set_target_properties(mpi::mpi PROPERTIES
 68   INTERFACE_COMPILE_DEFINITIONS "MPICH_SKIP_MPICXX;OMPI_SKIP_MPICXX;_MPICC_H"
 69   INTERFACE_INCLUDE_DIRECTORIES "PREFIX/include"
 70   INTERFACE_LINK_LIBRARIES "PREFIX/lib/libmpicxx.dylib;PREFIX/lib/libmpi.dylib;PREFIX/lib/libpmpi.dylib"
 71   INTERFACE_SYSTEM_INCLUDE_DIRECTORIES "PREFIX/include"
 72 )

Versions

Confirmed for Version 1.0.0