Git Product home page Git Product logo

Comments (7)

archermarx avatar archermarx commented on August 17, 2024 2

Can you provide your installation options?

from warpx.

dpgrote avatar dpgrote commented on August 17, 2024 2

There may be something here. The error complains that it can't find a constructor for Queue with the argument list (const int, int). Looking at the blaspp source, I see that that is correct - there is no constructor with that signature, only one without the const, (int, int).

Try this change and see if it fixes the issue: edit the file Source/FieldSolver/SpectralSolver/SpectralHankelTransform/HankelTransform.cpp and change line 38 to remove the const, to be int device_id = amrex::Gpu::Device::deviceId();.

from warpx.

ax3l avatar ax3l commented on August 17, 2024 1

@jinze-liu this is interesting. Is it possible that your BLAS++ library was not built for CUDA support but you are building WarpX for CUDA? How did you build BLAS++?

I do not see an immediate breaking change for BLAS++ in their repository:
https://github.com/icl-utk-edu/blaspp/commits/master/

But you could try downgrading BLAS++ a few commits or a release as well, in case that causes the problem.

Otherwise, if you do not need RZ geometry or do not need PSATD field solvers, you can turn either off to avoid the problem as well. We only need BLAS++ and LAPACK++ if RZ+PSATD are needed.

from warpx.

jinze-liu avatar jinze-liu commented on August 17, 2024

I'm using a new cluster, so I mimicked hpc3 (ucl) for the installation and configuration. This is my environment setup

#!/bin/bash
export proj="ljz_gpu"


export MY_PROFILE=$(cd $(dirname $BASH_SOURCE) && pwd)/$(basename $BASH_SOURCE)


module load gcc/9.4.0-gcc-4.8.5
#module load gcc/11.3.0-gcc-9.4.0
module load cmake/3.25.2-oneapi-2021.4.0
module load cuda/11.8.0-gcc-4.8.5  
module load openmpi/4.1.5-gcc-9.4.0 
#module load intel-oneapi-mpi/2021.8.0-gcc-4.8.5
#module load intel-oneapi-compilers/2021.4.0-gcc-4.8.5
#module load intel-oneapi-mkl/2021.4.0-gcc-4.8.5
#module load nvhpc/22.11-gcc-4.8.5
#module load cuda/11.5.2-gcc-4.8.5
#module load cuda/11.8.0-gcc-4.8.5
module load boost/1.80.0-gcc-9.4.0 

# optional: for openPMD and PSATD+RZ support
module load openblas/0.3.21-gcc-9.4.0
#module load  hdf5/1.14.0-gcc-9.4.0  
export PATH=/ShareData1/App/abinit-dependence/hdf5-1.14.0-oneapi-2021.4.0/bin:$PATH

export CMAKE_PREFIX_PATH=${HOME}/sw/hpc3/gpu/c-blosc-1.21.1:$CMAKE_PREFIX_PATH
export CMAKE_PREFIX_PATH=${HOME}/sw/hpc3/gpu/adios2-2.8.3:$CMAKE_PREFIX_PATH
export CMAKE_PREFIX_PATH=${HOME}/sw/hpc3/gpu/blaspp-master:$CMAKE_PREFIX_PATH
export CMAKE_PREFIX_PATH=${HOME}/sw/hpc3/gpu/lapackpp-master:$CMAKE_PREFIX_PATH

export LD_LIBRARY_PATH=${HOME}/sw/hpc3/gpu/c-blosc-1.21.1/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=${HOME}/sw/hpc3/gpu/adios2-2.8.3/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=${HOME}/sw/hpc3/gpu/blaspp-master/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=${HOME}/sw/hpc3/gpu/lapackpp-master/lib64:$LD_LIBRARY_PATH

export PATH=${HOME}/sw/hpc3/gpu/adios2-2.8.3/bin:${PATH}


module load python/3.10.6-gcc-4.8.5  


if [ -d "${HOME}/sw/hpc3/gpu/venvs/warpx-gpu" ]
then
  source ${HOME}/sw/hpc3/gpu/venvs/warpx-gpu/bin/activate
fi

# an alias to request an interactive batch node for one hour
#   for parallel execution, start on the batch node: srun <command>
alias getNode="salloc -N 1 -t 0:30:00 --gres=gpu:A100:1 -p free-gpu"
# an alias to run a command on a batch node for up to 30min
#   usage: runNode <command>
alias runNode="srun -N 1 -t 0:30:00 --gres=gpu:A100:1 -p free-gpu"


export AMREX_CUDA_ARCH=8.0

# compiler environment hints
export CXX=$(which g++)
export CC=$(which gcc)
export FC=$(which gfortran)
export CUDACXX=$(which nvcc)
export CUDAHOSTCXX=${CXX}

The script for installing dependencies uses the hpc3 example script exclusively. The compile command is:

cd $HOME/src/warpx
rm -rf build

cmake -S . -B build -DWarpX_COMPUTE=CUDA -DWarpX_PSATD=ON -DWarpX_QED_TABLE_GEN=ON -DWarpX_DIMS="1;2;RZ;3"
cmake --build build -j 12

from warpx.

jinze-liu avatar jinze-liu commented on August 17, 2024

Sure, thank you very much for your response. I will try the two solutions you mentioned. In fact, the compilation was successful when I removed rz.

from warpx.

ax3l avatar ax3l commented on August 17, 2024

@dpgrote shall we post a PR to WarpX to reflect this in mainline?
Is this a recent change in BLAS++ APIs?

from warpx.

ax3l avatar ax3l commented on August 17, 2024

Fix coming via #5012

from warpx.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.