Comments (5)
paulgavrikov
commented on May 26, 2024
hmm, yeah sounds like it could be related to multiprocessing, as it allocates workers differently on windows. Could you try setting mp_context with "fork", "forksever", or "spawn" and see if that fixes your issue?
from parallel-matplotlib-grid.
Laser-Cho
commented on May 26, 2024
sure.
fork
>>> parallel_plot(plot_fn=scatter, data=X, grid_shape=(1, 5), show_progress=False , mp_context="fork")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "\WIN_VENV\lib\site-packages\parallelplot-0.0.4-py3.10.egg\parallelplot\plot.py", line 139, in parallel_plot
File "\WIN_VENV\lib\site-packages\parallelplot-0.0.4-py3.10.egg\parallelplot\plot.py", line 60, in _make_subplots
File "\Python310\lib\multiprocessing\context.py", line 239, in get_context
return super().get_context(method)
File "\Python310\lib\multiprocessing\context.py", line 193, in get_context
raise ValueError('cannot find context for %r' % method) from None
ValueError: cannot find context for 'fork'
forksever
>>> parallel_plot(plot_fn=scatter, data=X, grid_shape=(1, 5), show_progress=False, mp_context="forkserver")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "WIN_VENV\lib\site-packages\parallelplot-0.0.4-py3.10.egg\parallelplot\plot.py", line 139, in parallel_plot
File "WIN_VENV\lib\site-packages\parallelplot-0.0.4-py3.10.egg\parallelplot\plot.py", line 60, in _make_subplots
File "\Python310\lib\multiprocessing\context.py", line 239, in get_context
return super().get_context(method)
File "\Python310\lib\multiprocessing\context.py", line 193, in get_context
raise ValueError('cannot find context for %r' % method) from None
ValueError: cannot find context for 'forkserver'
>>>
spawn
>>> parallel_plot(plot_fn=scatter, data=X, grid_shape=(1, 5), show_progress=False, mp_context="spawn")
Process SpawnPoolWorker-1:
Traceback (most recent call last):
File "Python310\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "Python310\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "Python310\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "Python310\lib\multiprocessing\queues.py", line 367, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'scatter' on <module '__main__' (built-in)>
Process SpawnPoolWorker-2:
Process SpawnPoolWorker-4:
Traceback (most recent call last):
File "Python310\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "Python310\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "Python310\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "Python310\lib\multiprocessing\queues.py", line 367, in get
return _ForkingPickler.loads(res)
Traceback (most recent call last):
AttributeError: Can't get attribute 'scatter' on <module '__main__' (built-in)>
File "Python310\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "Python310\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "Python310\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "Python310\lib\multiprocessing\queues.py", line 367, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'scatter' on <module '__main__' (built-in)>
Process SpawnPoolWorker-3:
Process SpawnPoolWorker-5:
Traceback (most recent call last):
Traceback (most recent call last):
File "Python310\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "Python310\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "Python310\lib\multiprocessing\process.py", line 315, in _bootstrap
self.run()
File "Python310\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "Python310\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "Python310\lib\multiprocessing\pool.py", line 114, in worker
task = get()
File "Python310\lib\multiprocessing\queues.py", line 367, in get
return _ForkingPickler.loads(res)
File "Python310\lib\multiprocessing\queues.py", line 367, in get
return _ForkingPickler.loads(res)
AttributeError: Can't get attribute 'scatter' on <module '__main__' (built-in)>
AttributeError: Can't get attribute 'scatter' on <module '__main__' (built-in)>
from parallel-matplotlib-grid.
Laser-Cho
commented on May 26, 2024
I solved this problem. in Windows, multiprocessing needs "main" . So beloe code wroks.
import numpy as np
from parallelplot import parallel_plot
def scatter(data, fig, axes):
axes.scatter(data[:, 0], data[:, 1])
X = np.random.uniform(low=-1, high=1, size=(5, 100, 100))
if __name__=='__main__':
fig,axes = parallel_plot(plot_fn=scatter, data=X, grid_shape=(1, 5), show_progress=False, mp_context="spawn" )
fig.savefig("result.jpeg")
from parallel-matplotlib-grid.
Laser-Cho
commented on May 26, 2024
even I solved it, but.... I'm sorry to say like this, But it is just faster very slightly than normal ploting, even it use all cores. probably coping data makes new bottleneck.
from parallel-matplotlib-grid.
paulgavrikov
commented on May 26, 2024
There is of course some overhead associated with the allocation of workers and IO. The context itself also impacts performance. On Linux, I definitely saw a massive speedup.
from parallel-matplotlib-grid.
Related Issues (4)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from parallel-matplotlib-grid.