llnl / cardioid Goto Github PK

Cardiac simulation toolkit.

License: MIT License

CMake 2.80% Makefile 1.08% Python 10.80% C++ 64.19% C 14.45% Fortran 0.91% Shell 1.76% M4 0.29% Cuda 0.38% TeX 0.33% MATLAB 0.08% Perl 2.60% Gnuplot 0.11% SystemVerilog 0.02% Starlark 0.19%

application simulation cpp

cardioid's Introduction

Cardioid

Cardioid is a cardiac multiscale simulation suite spanning from subcellular mechanisms up to simulations of organ-level clinical phenomena. The suite contains tools for simulating cardiac electrophysiology, cardiac mechanics, torso-ECGs, cardiac meshing and fiber generation tools.

Requirements

Cardioid can be broken down into many separate simulators, each with their own dependencies.

Cardioid electrics: C99, C++, OpenMP, MPI, and Lapack. It requires a valid perl installation to build. It van also take advantage of CUDA, in which case it also depends on NVTX and NVRTC.
Cardioid fibers: Depends on C++ and MFEM.
Cardioid mechanics: Depends on C++ and MFEM.
Cardioid ecg: Depends on C++ and MFEM.

Some requirements are bundled into the build system, like libkdtree and simUtil.

Getting Started

We recommend compiling Cardioid using Spack.

To build Cardioid with spack, follow the following steps:

Clone Spack and set it up:

git clone https://github.com/spack/spack.git
. spack/share/spack/setup-env.sh

Make a spack environment in order to build cardioid.

mkdir myenv
cd myenv
spack env create -d .
spacktivate .

Load any necessary modules in order to build cardioid (cmake, perl, some sort of mpi, optional cuda)

Add the necessary dependencies in order to build cardioid.

spack compiler find
spack external find cmake perl
spack external find --not-buildable openmpi mvapich2 mpich spectrum-mpi cuda

Add cardioid to the environment

spack add cardioid
# or
spack add cardioid+cuda

Add cardioid as a develop package
```
spack develop cardioid@develop
```
This will check out the cardioid source code into this directory
Concretize everything
```
spack concretize -f
```
Build!
```
spack install
```
The binary will be in .spack-env/view/bin/cardioid
Once you've done this, you can change directory to cardioid/spack-build-* and type make to build the source code manually. All the configure options will be set up for you.
To remove your changes and build from a fresh source tree again, just use undevelop and reinstall with spack
```
spack undevelop cardioid
spack install
```

Building without spack

Cardioid is built with CMake, using the BLT make system. A separate .cmake file is supplied for a variety of architectures. Please feel free to make your own .cmake architecture for your particular cluster if needed. Example .cmake architectures can be found in "arch/*.cmake" Each architecture can be given a separate name, allowing multiple different versions of the code to be built on the same system.

If no architecture file is supplied, BLT will try to pick sane defaults. To build everything, you will need to install MFEM manually to a directory of your choice and tell Cardioid where that installation lives (through the MFEM_DIR variable)

Builds are performed in build/<arch> . Executables are installed in build/<arch>/bin.

Contributing

Please submit any bugfixes or feature improvements as pull requests.

Authors

Many thanks go to Cardioid's contributors.

James P Glosli
Tomas Oppelstrup
Xiaohua Zhang
David F Richards
Robert Blake
Erik Draeger
Jean-Luc Fattebert
Jamie Bramwell
Arthur Mirin
Sebastian Laudenschlager
Viacheslav Gurev
Jeremy Rice
Changhoan Kim
and many more...

Learn more

Cardioid's history goes back a few years -- it was a finalist for the 2012 Gordon Bell Prize -- but only now is the code available as open source. Initially developed by a team of LLNL and IBM scientists, Cardioid divides the heart into a large number of manageable subdomains. This replicates the electrophysiology of the human heart, accurately simulating the activation of each heart muscle cell and cell-to-cell electric coupling.

Video: The Cardioid Project: Simulating the Human Heart
Science & Technology Review article: Venturing into the Heart of High-Performance Computing Simulations
Science & Technology Review article: Reaching for New Computational Heights with Sequoia - Cardioid helped set speed records for the Sequoia supercomputer by clocking in at nearly 12 petaflops while scaling with better than 90% parallel efficiency across all 1,572,864 cores.

License

Cardioid is distributed under the terms of the MIT license. All new contributions must be made under this license.

See LICENSE and NOTICE for details.

SPDX-License-Identifier: MIT

LLNL-CODE-764041

cardioid's People

Contributors

Stargazers

Watchers

Forkers

landajuela hauten eminsight davidpoliakoff seblaud btklein johnlee rblake-llnl sergiolaranjo liuhahayue joonvan paulkefer kiruyamomochi phoenixdong

cardioid's Issues

Build fails when OpenMP is disabled

configure says:

OpenMP Support is

but build then fails

ld: error: unable to find library -lopenmp

rev. 2f66d9f
clang-14
FreeBSD 13.1

Examples or tests with python 3?

Hi!

I have cardioid built into a container (I'm anticipating trying to run with the Flux Operator on Kubernetes) and I was trying to run tests for a demo run, and it looks like that uses a runner.py that might be suited for python 2 (I'm looking at the print statement) https://github.com/LLNL/cardioid/blob/master/test/runner.py.

Is there anyothe way to run this or something I'm missing, or would it be possible to update the script for python 3? Thank you!

Spack install cardioid errors, please help!

130 /usr/bin/ld: ../lib/libheart_gpu_aware.a(reactionFactory.cc.o): undefined reference to symbol 'dlsym@@@@GLIBC_2.2.5'
131 //lib/x86_64-linux-gnu/libdl.so.2: error adding symbols: DSO missing from command line
132 collect2: error: ld returned 1 exit status
133 elec/CMakeFiles/singleCell.dir/build.make:107: recipe for target 'bin/singleCell' failed
134 make[2]: *** [bin/singleCell] Error 1
135 make[2]: Leaving directory '/root/cardioid/build/cardioid'
136 CMakeFiles/Makefile2:407: recipe for target 'elec/CMakeFiles/singleCell.dir/all' failed
137 make[1]: *** [elec/CMakeFiles/singleCell.dir/all] Error 2
138 make[1]: *** Waiting for unfinished jobs....
139 /usr/bin/ld: ../lib/libheart_gpu_aware.a(reactionFactory.cc.o): undefined reference to symbol 'dlsym@@@@GLIBC_2.2.5'
140 //lib/x86_64-linux-gnu/libdl.so.2: error adding symbols: DSO missing from command line
141 collect2: error: ld returned 1 exit status
142 elec/CMakeFiles/cardioid.dir/build.make:96: recipe for target 'bin/cardioid' failed
143 make[2]: *** [bin/cardioid] Error 1
144 make[2]: Leaving directory '/root/cardioid/build/cardioid'
145 CMakeFiles/Makefile2:293: recipe for target 'elec/CMakeFiles/cardioid.dir/all' failed
146 make[1]: *** [elec/CMakeFiles/cardioid.dir/all] Error 2
147 make[1]: Leaving directory '/root/cardioid/build/cardioid'
148 Makefile:132: recipe for target 'all' failed
149 make: *** [all] Error 2

Build is broken with MFEM

When MFEM_DIR is set it breaks:

ld: error: undefined symbol: mfem::DenseMatrix::DenseMatrix(int)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(quat2rot(mfem::Vector const&))
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(MatrixElementPiecewiseCoefficient::Eval(mfem::DenseMatrix&, mfem::ElementTransformation&, mfem::IntegrationPoint const&))

ld: error: undefined symbol: mfem::GridFunction::GridFunction(mfem::Mesh*, std::__1::basic_istream<char, std::__1::char_traits<char> >&)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(ecg_readGF(object_st*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, mfem::Mesh*, std::__1::shared_ptr<mfem::GridFunction>&))

ld: error: undefined symbol: mfem::Mesh::Mesh(char const*, int, int, bool)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(ecg_readMeshptr(object_st*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >))

ld: error: undefined symbol: mfem::Mesh::GeneratePartitioning(int, int)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::ParMesh::ParMesh(int, mfem::Mesh&, int*, int)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::H1_FECollection::H1_FECollection(int, int, int)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::ParFiniteElementSpace::ParFiniteElementSpace(mfem::ParMesh*, mfem::FiniteElementCollection const*, int, int)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::MemoryManager::host_mem_type
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced 10 more times

ld: error: undefined symbol: mfem::Array<int>::Max() const
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::MemoryManager::New_(void*, unsigned long, mfem::MemoryType, unsigned int&)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(mfem::GridFunction::GridFunction(mfem::FiniteElementSpace*))
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(MatrixElementPiecewiseCoefficient::Eval(mfem::DenseMatrix&, mfem::ElementTransformation&, mfem::IntegrationPoint const&))
>>> referenced 1 more times

ld: error: undefined symbol: mfem::MemoryManager::Delete_(void*, mfem::MemoryType, unsigned int)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced 20 more times

ld: error: undefined symbol: vtable for mfem::ParGridFunction
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(mfem::ParBilinearForm::ParBilinearForm(mfem::ParFiniteElementSpace*))
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(mfem::ParGridFunction::~ParGridFunction())
>>> the vtable symbol may be undefined because the class is missing its key function (see https://lld.llvm.org/missingkeyfunction)

ld: error: undefined symbol: mfem::GridFunction::operator=(double)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::ParGridFunction::ParGridFunction(mfem::ParMesh*, mfem::GridFunction const*, int const*)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::Vector::Vector(mfem::Vector const&)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::Vector::operator+=(mfem::Vector const&)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::Vector::operator=(mfem::Vector const&)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::BilinearForm::AddDomainIntegrator(mfem::BilinearFormIntegrator*)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::ParBilinearForm::Assemble(int)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: undefined symbol: mfem::HypreParMatrix::HypreParMatrix()
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)
>>> referenced by ecg.cpp
>>>               ecg/CMakeFiles/ecg.dir/ecg.cpp.o:(main)

ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)

It is also unclear why does it need MFEM sources instead of MFEM being installed from the package.

update build commands

To build regular cardioid:

spack diy cardioid

gives error from spack:

==> Error: diy is not a recognized Spack command or extension command; check with `spack commands`.

Upon digging spack issues on its GitHub, I found diy has been updated to dev-build (github link)

So, i used spack dev-build cardioid but still got error message for using it without a version number.

==> Error: spack dev-build spec must have a single, concrete version. Did you forget a package version number?

What am i doing wrong? Please help. Thanks.

`spack install` takes an infinite time when installing cardioid

environment: CentOS7+gcc 8.3.0+cmake 3.21.1+cuda 11.1.105+perl 5.16.3

I was trying to install Cardioid following the instruction in README.
After running spack add cardioid+cuda and spack install, the process after ==> cardioid: Executing phase: 'cmake' is printed on my screen seems to take an infinite time to complete (I waited for at least 20 min yet nothing proceeded).

The details is as follows:

$ spack install
==> Installing environment /home/myenv
[+] /home/cmake-3.21.1 (external cmake-3.21.1-kgvtacydi7vchye4nll5bq6xatv3kzal)
[+] /usr/local/cuda (external cuda-11.1.105-szapcmcc4oxqlsjoqlmbq7b4tr6vhhs7)
[+] /usr (external perl-5.16.3-lcr6qhivgtjs74edphkzngm7ofvmpvv2)
[+] /home/openmpi (external openmpi-3.1.5-hlafh73mwzxiolbptzntznccqefxzvxt)
==> Installing openblas-0.3.17-33sulnfbl6uv34rg4htzxyou7p6enf2f
==> No binary for openblas-0.3.17-33sulnfbl6uv34rg4htzxyou7p6enf2f found: installing from source
==> Using cached archive: /home/spack/spack-develop/var/spack/cache/_source-cache/archive/df/df2934fa33d04fd84d839ca698280df55c690c86a5a1133b3f7266fce1de279f.tar.gz
==> No patches needed for openblas
==> openblas: Executing phase: 'edit'
==> openblas: Executing phase: 'build'
==> openblas: Executing phase: 'install'
==> openblas: Successfully installed openblas-0.3.17-33sulnfbl6uv34rg4htzxyou7p6enf2f
  Fetch: 0.13s.  Build: 1m 33.61s.  Total: 1m 33.74s.
[+] /home/spack/spack-develop/opt/spack/linux-centos7-skylake_avx512/gcc-8.3.0/openblas-0.3.17-33sulnfbl6uv34rg4htzxyou7p6enf2f
==> Installing cardioid-develop-lei4go23dpiitmnde5zkcmaaf5umwdgi
==> No binary for cardioid-develop-lei4go23dpiitmnde5zkcmaaf5umwdgi found: installing from source
==> No patches needed for cardioid
==> cardioid: Executing phase: 'cmake'

Same problem occurs when trying the following way to install/build:

make
modified the option in Cmakecache to enable cuda