Comments (5)
That's a typo in the docs, it should be TimeoutError
.
from pebble.
Thank you, sorry for that, I should have found it...
I had to add from pebble.common import ProcessExpired
at the beginning.
The code is :
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# From http://pythonhosted.org/Pebble/#examples
from pebble import ProcessPool
from pebble.common import ProcessExpired
from concurrent.futures import TimeoutError
def function(foo, bar=0):
return foo + bar
elements = list(range(1000))
with ProcessPool() as pool:
iterator = pool.map(function, elements, timeout=10)
while True:
try:
results = next(iterator)
print(results)
except TimeoutError as error:
print("Function took longer than %d seconds" % error.args[1])
except ProcessExpired as error:
print("%s. Exit code: %d" % (error, error.exitcode))
except Exception as error:
print("function raised %s" % error)
print(error.traceback) # Python's traceback of remote process
except StopIteration:
break
Now the execution starts but at the end there are the two errors below :
function raised
Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/local/lib/python2.7/site-packages/pebble/pool/process.py", line 155, in message_manager_loop
pool_manager.process_next_message(SLEEP_UNIT)
File "/usr/local/lib/python2.7/site-packages/pebble/pool/process.py", line 184, in process_next_message
message = self.worker_manager.receive(timeout)
File "/usr/local/lib/python2.7/site-packages/pebble/pool/process.py", line 304, in receive
if self.pool_channel.poll(timeout):
File "/usr/local/lib/python2.7/site-packages/pebble/pool/channel.py", line 44, in unix_poll
return bool(select([self.reader], [], [], timeout)[0])
error: (9, 'Bad file descriptor')
and
Traceback (most recent call last):
File "./travaux/multipro/test_pebble_2.py", line 29, in <module>
print(error.traceback) # Python's traceback of remote process
AttributeError: 'exceptions.StopIteration' object has no attribute 'traceback'
This leads to the second issue error: (9, 'Bad file descriptor')
I wrote you yesterday about the ProcessPool
class.
from pebble.
I will update the documentation making sure all examples are correct.
Keep in mind that the order of exceptions is important. Exception
is the most generic one, therefore it needs to be last.
from pebble import ProcessPool
from pebble.common import ProcessExpired
from concurrent.futures import TimeoutError
def function(foo, bar=0):
return foo + bar
elements = list(range(1000))
with ProcessPool() as pool:
iterator = pool.map(function, elements, timeout=10)
while True:
try:
results = next(iterator)
print(results)
except TimeoutError as error:
print("Function took longer than %d seconds" % error.args[1])
except ProcessExpired as error:
print("%s. Exit code: %d" % (error, error.exitcode))
except StopIteration:
break
except Exception as error:
print("function raised %s" % error)
print(error.traceback) # Python's traceback of remote process
This logic won't raise the AttributeError
anymore. For the other issue let's use the #10 to track it.
from pebble.
Ok I understand !
Thank-you again.
from pebble.
Fixed in release 4.3.1.
from pebble.
Related Issues (20)
- Bug: new `submit` function makes it impossible to call a function that has a `timeout` argument HOT 6
- Logging process name inside concurrent.process HOT 2
- How to handle errors when using pool.schedule HOT 1
- shutdown of main program HOT 2
- @concurrent.process returned future blocks/hangs on running(), cancelled(), done() calls HOT 2
- Get information about broken process HOT 4
- Documentation for Pebble indicates threads created with a ThreadPool are cancellable HOT 1
- Channel mutex timeout HOT 6
- How can I use a multiprocessing.manager alongside with pebble to avoid re-importing the function everytime? HOT 1
- Type hint error of wrapped function HOT 7
- ISSUE with with ProcessPool when scheduled function return exception (not raise it) HOT 1
- BUG with ProcessPool when scheduled function raise BaseException HOT 1
- issue with handling frozen exceptions in worker thread or process HOT 1
- Cannot create pebble.ProcessPool() multiple times within a single run, all task got stuck and time out HOT 3
- error while running @concurrent.process repetitively HOT 4
- using multiprocess context (for dill support) no longer working HOT 2
- Workers using 100% CPU not getting killed after timeout HOT 11
- Behavior of Process.map with chunksize > 1 and a timeout HOT 5
- map function is extremely slow, seems to be executing sequentially HOT 1
- Cancel future returned from run_in_executor caused SIGTERM signal to main process HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pebble.