Comments (3)
Thanks for the note on this, there was some discussion of this previously in issue #36 . The conclusion at the time was to leave things as they were and rely on multiprocessing for parallelism. I still tend toward this as shared memory parallelism for computational bottlenecks is tricky, c-Python is not really designed for it, and there could be unforeseen consequences.
Do you have a use case where multiprocessing would not work? If there is a big demand for this use case we could revisit this decision. As discussed in #36 we could add an argument that would make copies of all the data and release the GIL during the computational intensive part.
from scikit-fmm.
Thanks for your reply! The use case is large 3d calculations where multiprocessing is too slow and/or uses too much memory. Multiprocessing also does not work well in combination with some other big threaded libraries we use. On Windows (or other platforms using 'spawn' instead of 'fork') these issues are more problematic. In #36 you discuss the option "Release the GIL without copying the parameters, and leave on the programmer the responsibility for thread safeness. It looks like a bad idea, and is what my previous commit does." This is exactly what I would like, maybe controlled by an allow_threading argument? I would argue that this is much easier to understand and deal with than multiprocessing.
Perhaps it is possible to guarantee that the solver still terminates even if the array content is modified during the run? Copying the input would not work for me, I would run out of memory.
from scikit-fmm.
I have thought more about this and I am just not comfortable releasing a version of the library that is not thread safe even if it is an option and is explained in the documentation. It has been my experience that threading and reference counting are really difficult to get correct and that just releasing the GIL would lead to Heisenbugs and unexpected consequences. Other than forcing a copy of the data scikit-fmm uses, I am not sure what the options are, do you have examples of how other Python libraries handle this?
from scikit-fmm.
Related Issues (20)
- Pip install broken on CIs HOT 4
- about cfmm HOT 1
- ValueError: the array phi contains no zero contour (no zero level set) HOT 1
- cannot import skfmm due to arm64 incompatibility? HOT 1
- Support for python 3.11 HOT 2
- Installation went well, but unable to import HOT 2
- Marching issue in a skeleton HOT 2
- numpy.distutils is no longer available in Python 3.12 HOT 26
- Inaccurate travel paths HOT 2
- Add a less formal introduction of FMM and what it can be used for in the README HOT 1
- error: Multiple top-level packages discovered in a flat-layout: ['skfmm', 'profile']. HOT 5
- Negative curvature of signed distance HOT 9
- a relevant way to cite scikit-fmm
- [Enhancement] Release GIL HOT 3
- Exact representation of contour HOT 13
- Incompatible to numpy 2.0 - numpy removed numpy/noprefix.h HOT 6
- pip instal scikit-fmm error HOT 1
- DEPENDENCY - Update code to use numpy 2.0 HOT 7
- Provide Wheels for scikit-fmm HOT 24
- WARNING: Failed to activate VS environment: Could not find C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scikit-fmm.