Git Product home page Git Product logo

Comments (8)

marcoheisig avatar marcoheisig commented on September 13, 2024

Greetings!

The short answer:
$ sudo update-alternatives --set mpi /usr/lib/openmpi/include
$ sudo update-alternatives --set mpirun /usr/bin/mpirun.openmpi
The former command places a C compiler wrapper script called mpicc in the $PATH which cl-mpi uses instead of $CC to grovel all MPI constants and load all MPI libraries. The latter command makes the mpirun program accessible which is used to launch cl-mpi images in parallel.

The long answer:
There is no sane way of automatically configuring and detecting all include paths, libraries and flags necessary to use the active MPI installation of a computer - I tried and failed. What the C guys do instead is to use a wrapper script called mpicc that calls the host compiler with all necessary parameters, e.g. where mpi.h is located. Each MPI library like OpenMPI and MPICH provides their own suitable mpicc command. I use mpicc for CFFI grovelling and to build a small shared library called cl-mpi-stub.so which I then load from CFFI to pull in all library dependencies. This approach seems to work well on all systems I tested so far.

I hope this solves the problem. My development platform is also Debian 8, so I am optimistic. If not I would like to see the output of
$ mpicc -show
and the error message of (asdf:load-system 'cl-mpi).

Best regards,
Marco

from cl-mpi.

quicklisp avatar quicklisp commented on September 13, 2024

Thanks.

I get this:

; gcc -m64 -I/home/quicklisp/quicklisp-controller/dist/build-cache/cffi/e7921b040f0a965a2604c95da10e039387059c22/cffi_0.16.1/ -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/a8c717e7b6f8da88a38d108da10fb6c2ab710231/cl-mpi-20160110-git/mpi/grovel /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/a8c717e7b6f8da88a38d108da10fb6c2ab710231/cl-mpi-20160110-git/mpi/grovel.c
A nested error within --disable-debugger error handling prevents displaying the original error. Attempting to print a backtrace.

When I run that from the command line, I get this:

$ gcc -m64 -I/home/quicklisp/quicklisp-controller/dist/build-cache/cffi/e7921b040f0a965a2604c95da10e039387059c22/cffi_0.16.1/ -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/a8c717e7b6f8da88a38d108da10fb6c2ab710231/cl-mpi-20160110-git/mpi/grovel /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/a8c717e7b6f8da88a38d108da10fb6c2ab710231/cl-mpi-20160110-git/mpi/grovel.c
/home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/a8c717e7b6f8da88a38d108da10fb6c2ab710231/cl-mpi-20160110-git/mpi/grovel.c:6:17: fatal error: mpi.h: No such file or directory
 #include <mpi.h>
         ^
compilation terminated.

And the mpicc output:

$ mpicc -show
gcc -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -pthread -L/usr//lib -L/usr/lib/openmpi/lib -lmpi -ldl -lhwloc

from cl-mpi.

marcoheisig avatar marcoheisig commented on September 13, 2024

Thank you for the helpful output!

The output on the command line should begin with $ mpicc -m64 ... and I think I know why it does not. I used the following code via ASDFs :defsystem-depends-on to force CFFI to use mpicc instead of gcc for grovelling.

(defmethod perform :around ((op cffi-grovel::process-op)
                            (c cffi-grovel:grovel-file))
  (let ((cffi-grovel::*cc* "mpicc"))
    (call-next-method)))

It appears another :around method has replaced this one in your case, restoring the original behavior. This is of course my fault. I already uploaded a fix that subclasses cffi-grovel:grovel-file for MPI related operations. I hope your problem is now gone.

from cl-mpi.

quicklisp avatar quicklisp commented on September 13, 2024

Got the new commit, seeing the same issue. Here's more output:

; mpicc -shared -fPIC -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/cl-mpi-stub.so /home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/cl-mpi-stub.c
; gcc -m64 -I/home/quicklisp/quicklisp-controller/dist/build-cache/cffi/e7921b040f0a965a2604c95da10e039387059c22/cffi_0.16.1/ -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/grovel /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/grovel.c
A nested error within --disable-debugger error handling prevents displaying the original error. Attempting to print a backtrace.
cl-mpi-extensions cl-mpi-extensions
; mpicc -shared -fPIC -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/cl-mpi-stub.so /home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/cl-mpi-stub.c
; gcc -m64 -I/home/quicklisp/quicklisp-controller/dist/build-cache/cffi/e7921b040f0a965a2604c95da10e039387059c22/cffi_0.16.1/ -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/grovel /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/grovel.c
A nested error within --disable-debugger error handling prevents displaying the original error. Attempting to print a backtrace.
cl-mpi-testsuite cl-mpi-testsuite
; mpicc -shared -fPIC -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/cl-mpi-stub.so /home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/cl-mpi-stub.c
; gcc -m64 -I/home/quicklisp/quicklisp-controller/dist/build-cache/cffi/e7921b040f0a965a2604c95da10e039387059c22/cffi_0.16.1/ -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/grovel /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/grovel.c
A nested error within --disable-debugger error handling prevents displaying the original error. Attempting to print a backtrace.
cl-mpi cl-mpi
; mpicc -shared -fPIC -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/cl-mpi-stub.so /home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/cl-mpi-stub.c
; gcc -m64 -I/home/quicklisp/quicklisp-controller/dist/build-cache/cffi/e7921b040f0a965a2604c95da10e039387059c22/cffi_0.16.1/ -o /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/grovel /home/quicklisp/.cache/common-lisp/sbcl-1.3.1.188-f7d76e5-linux-x64/home/quicklisp/quicklisp-controller/dist/build-cache/cl-mpi/9a2e10da9fbf0fd92742014efa5ce6e5c170e9c4/cl-mpi-20160111-git/mpi/grovel.c
A nested error within --disable-debugger error handling prevents displaying the original error. Attempting to print a backtrace.

from cl-mpi.

marcoheisig avatar marcoheisig commented on September 13, 2024

If this does not fix it, i have just discovered that CFFI determines the C compiler with

(or (getenv "CC") *cc*)

so unsetting the environment variable $CC or setting it to mpicc might help.

from cl-mpi.

quicklisp avatar quicklisp commented on September 13, 2024

Progress! Unsetting CC worked for cl-mpi. Unfortunately, I think unsetting it breaks some other projects. I'll run through building the universe and see what happens with no CC.

from cl-mpi.

marcoheisig avatar marcoheisig commented on September 13, 2024

If it helps I can detect if $CC is set and if so, bind it to mpicc for the dynamic extent of the CFFI grovelling and restore it afterwards. Then other projects can use $CC as they want. I only fear changing environment variables (even for such a short duration) could be a horrible source of nondeterminism for automated build systems. What do you think?

from cl-mpi.

quicklisp avatar quicklisp commented on September 13, 2024

I unset CC in my build process. cl-mpi started working, and nothing broke. I'm going to leave it unset and consider things fixed. Thanks for your help!

from cl-mpi.

Related Issues (17)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.