Comments (5)
jmsull
commented on August 25, 2024
1
For posterity, here is what I did:
module load PrgEnv-gnu
module load gsl
conda create --name nbodykit_env python=3.7 pip
env MPICC=cc python -m pip install --no-cache-dir mpi4py
srun -n 5 python -m mpi4py.bench helloworld # test mpi4py
pip install numpy cython
pip install nbodykit[extras]
which seems to work on a perlmutter compute node
from nbodykit.
rainwoodman
commented on August 25, 2024
One difficulty I see is that nbodykit has to be built with the PrgEnv on
NERSC, thus getting the bccp conda channel version of nbodykit won't likely
work with the PrgEnv version of mpi4py anyway.
You will likely need to rebuild all of the nbodykit dependency packages
with PrgEnvGnu. It is possible that using pip install after PrgEnvGnu can
get you quite far. Did you try that?
The scripts in the m3035 project roughly does that, but via conda-build.
The scripts there also create a conda-channel with these PrgEnv built
packages for cori (at the m3035 project folder).
The bcast-bccp-3.8 environment was using that channel. The more 'proper'
way of fixing this might be upgrading those channel building scripts for
Permutter, especially if you plan to run things at scale with the 'bcast'
style environments.
- Yu
…On Sat, Feb 4, 2023 at 10:19 PM Biwei Dai ***@***.***> wrote:
Hi,
The nbodykit built for NERSC cori (
https://nbodykit.readthedocs.io/en/latest/getting-started/install.html#nbodykit-on-nersc
) does not seem to work on Perlmutter. When trying to load nbodykit with
source /global/common/software/m3035/conda-activate.sh 3.8
I got the following error:
/global/common/software/m3035/conda/envs/bcast-bccp-3.8/etc/conda/activate.d/activate-nersc-prgenv-gnu.sh:
line 6: /etc/profile.d/nerschost.sh: No such file or directory
/global/common/software/m3035/conda/envs/bcast-bccp-3.8/etc/conda/activate.d/activate-nersc-prgenv-gnu.sh:
line 7: /etc/profile.d/modules.sh: No such file or directory
/global/common/software/m3035/conda/envs/bcast-bccp-3.8/etc/conda/activate.d/activate-nersc-prgenv-gnu.sh:
line 8: /etc/profile.d/mpi-selector.sh: No such file or directory
/global/common/software/m3035/conda/envs/bcast-bccp-3.8/etc/conda/activate.d/activate-nersc-prgenv-gnu.sh:
line 9: /etc/bash.bashrc.local: No such file or directory
So I tried to create my conda environment with nbodykit. I install mpi4py
with
module swap PrgEnv-${PE_ENV,,} PrgEnv-gnu
MPICC="cc -shared" pip install --force-reinstall --no-cache-dir
--no-binary=mpi4py mpi4py
following
https://docs.nersc.gov/development/languages/python/parallel-python/#mpi4py-in-your-custom-conda-environment
I tested it on Perlmutter computing node and it works fine.
But when I try to install nbodykit with
conda install -c bccp nbodykit
It doesn't use the mpi4py I built and reinstalls mpi4py with conda, which
no longer works on Perlmutter computing nodes. Can I force it to use the
mpi4py I built?
I also tried reinstalling mpi4py again to overwrite the mpi4py conda
installed, and I got the following error when running the code:
Attempting to use an MPI routine before initializing MPICH
—
Reply to this email directly, view it on GitHub
<#675>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABBWTDQQYK7K7RAGFAW5JTWV5A7XANCNFSM6AAAAAAURTAD34>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
from nbodykit.
jmsull
commented on August 25, 2024
Lagging behind Biwei I am just now running into this issue now that cori is gone forever (and have not yet found a way for this to work).
from nbodykit.
biweidai
commented on August 25, 2024
Yu's suggestion works for me! I manually install the nbodykit dependencies and nbodykit with PrgEnv-gnu and pip install. Have you tried this?
By the way, if you are going to run nbodykit on jupyter lab, I think the conda install should work.
from nbodykit.
jmsull
commented on August 25, 2024
@rainwoodman As you alluded to toward the top of this issue, recreating the bcast-pip scripts would be nice to have - I am running some <4 node jobs and am seeing 5-10 mins of startup time. IIRC bcast-pip improves this.
Biwei and I were discussing this today and may have some bandwidth to do this if you can get us started?
from nbodykit.
Related Issues (20)
- support for indexing in newer `dask` release HOT 2
- Bug in nbodykit.transform.CartesianToSky
- nersc installation page doesn't exist HOT 1
- `CatalogMesh.to_real_field()` output attributes HOT 1
- Help with parallelising nbodykit code HOT 5
- TPCF calculation error: "Corrfunc.mocks.DDsmu_mocks.DDsmu_mock" fails.
- Problem with painting to mesh if value has zero mean HOT 5
- ```HDFCatalog.to_mesh``` generate negative values for the density field 'delta+1'. HOT 6
- FKPCatalog failing in absence of N(z)
- Trouble understanding the matter power spectrum of lognormal mocks
- Unequal shape partition during MPI parallel mesh.
- not read in correct file in FITs format HOT 4
- FFTPower in redshift-space produces nan in the output (related to #663)
- MPI Issue on Perlmutter HOT 1
- Is there any way to parallelize the computation of multiple convolutions? HOT 8
- Does this work on Python 3.7 and later? HOT 2
- Tutorial on readthedocs: FFTPower requires `mode` argument HOT 1
- Kdcount building fail during cython build on NERSC HOT 2
- Creating a linear mesh from a CLASS power spectrum.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nbodykit.