Comments (5)
Ugh, so that's why that error looked familiar! I'm using an M1 mac, but somehow yt is working. I'm not specifically setting anything to use OpenMP there though, since I'm only running tiny test cases.
I can confirm that Mac's default compiler, clang
, does not support OpenMP. Worse, gcc
on the command line is aliased to clang
, at least by default. (I have not tried turning that off, so I don't know if it's possible.) I have previously been able to compile C code with OpenMP on my mac, but that was by (1) installing gcc e.g., from homebrew, and (2) compiling C code by calling the specific gcc version, e.g., gcc-13
instead of gcc
. This was, however, pure C code that I then called from python with ctypes
, and I think I might actually have had to give up getting that to work on the M1 mac. (The 'clang
not supporting OpenMP' issue is not new.)
I'm not sure if there's a way to get python to look up if there's a real gcc
version on a system and to use that instead. Overall, I suppose apple had some reason for aliasing gcc
to clang
, but it's a real pain when it doesn't actually support some of gcc
's features.
from yt.
... so I checked, and I can confirm that although the C code compiles (and I can run a test C program from the command line), I get a similar issue to the one Jack reported if I try to call the .so
file from python:
OSError: dlopen(/Users/nastasha/code/proj-an-c/interp2d/interp.so, 0x0006): tried: '/Users/nastasha/code/proj-an-c/interp2d/interp.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/nastasha/code/proj-an-c/interp2d/interp.so' (no such file), '/Users/nastasha/code/proj-an-c/interp2d/interp.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))
The .so
file was compiled with gcc-13
, specifically Homebrew GCC 13.2.0
. By the way, I had to deactivate conda
before it would even compile due to some linking library issue, but I did activate conda
again to run from python.
Overall, this issue seems to be a general headache on Macs. This stackexchange thread seems to have some ideas: https://stackoverflow.com/questions/28010801/compiling-parallelized-cython-with-clang , but it requires installing libraries on your own computer first, and adding a bunch of stuff on the command line. I don't get the impression that there's a straightforward way to get this to work 'off the shelf'.
Honestly, my own 'solution' has been to do small bits of C-from-python development on the login nodes of the university linux cluster. I run all my production analysis on linux-managed clusters anyway.
from yt.
I'm using an M1 mac, but somehow yt is working. I'm not specifically setting anything to use OpenMP there though, since I'm only running tiny test cases.
Building for mac arm64 without OpenMP is definitely something we've been exercising (and we've been publishing wheels for it since yt 4.0.4), so as long as you're not trying to enable it, yt is expected to build and run correctly on this arch (albeit at sub-optimal performance).
from yt.
Ugh, yeah this mostly makes me wish I trusted myself to maintain a linux system
from yt.
I dug a little bit (with @cphyc's help) and found a reasonably painless way to build yt with OpenMP on this platform
# test_omp.sh
set -euxo pipefail
brew install gcc
export CXX=g++-13
export CC=gcc-13
rm -fr .venv | true
python -m venv .venv
source .venv/bin/activate
python -m pip install build
git clone https://github.com/yt-project/ewah_bool_utils.git _ewah_bool_utils
pushd _ewah_bool_utils
rm -fr dist | true
python -m build --wheel
python -m pip install dist/*.whl
popd
git clone https://github.com/yt-project/yt.git _yt
pushd _yt
rm -fr dist | true
python -m build --wheel
python -m pip install dist/*.whl
popd
python -m pip install pandas h5py pooch
OMP_NUM_THREADS=4 python t.py
OMP_NUM_THREADS=2 python t.py
# t.py
import yt
from time import monotonic_ns
import os
from tqdm import tqdm
ds = yt.load_sample("output_00080")
NREP=10
tstart = monotonic_ns()
for i in tqdm(range(NREP)):
p = yt.ProjectionPlot(ds, [1, 1, 1], ("gas", "density"))
p.render()
tstop = monotonic_ns()
dt = (tstop-tstart) / 1e9 # in s
print(f"Took {dt:.1e} s ({dt/NREP:.1e} s/it)", end="")
if (omp_num_threads:=os.environ.get("OMP_NUM_THREADS")) is not None:
print(f" using {omp_num_threads} OpenMP threads")
else:
print()
However, because this technique involves dynamically linking libgomp
I got from homebrew (/opt/homebrew/opt/gcc/lib/gcc/current/libgomp.1.dylib
), the resulting wheel isn't portable, so we cannot apply this to the publishing process.
As noted by @cphyc, portability may be addressable on the conda-forge side (if we're not doing it already).
Meanwhile, we could document this technique, but we need to know whether this is also an issue with conda-forge binaries first.
from yt.
Related Issues (20)
- BUG: ResourceWarnings for unclosed files in boxlib frontend HOT 2
- BUG: convert_to_cartesian from _sanitize_center fails for Geographic geometry when bbox is subset of globe
- BLD: non-isolated builds are broken
- TST: Upcoming dependency test failures HOT 1
- How to change the particle size in function "ParticlePhasePlot" HOT 2
- BUG: Multiple fields break sanitization HOT 2
- BUG: segault on manylinux2014 image HOT 5
- Deprecation warning in GDF (and maybe more?) HOT 3
- Try to project gas particles (SPH) to a mesh with octree structure HOT 2
- ImportError when compiling with gcc 14.1.1 and conda HOT 6
- DOC: docs builds are failing HOT 7
- Incorrect parameter sanitation to np.logspace HOT 2
- CPython 3.13 support (tracker issue)
- Question: How the weight field operates to an yt.create_profile average? HOT 6
- BUG: segfault on macOS (amr64) HOT 11
- Nose testing image comparison: label which image is which? HOT 3
- Editable Installations may be broken in conda environment HOT 7
- BUG: Segfault in smoothing length calculation on Mac HOT 9
- Tipsy Frontend BUG?: oddities with smoothing lengths/positions and bounding box HOT 8
- failing enzo answer tests HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yt.