thanks for making subprocess32 available, it really helped us fighting random lock-ups in our application. I would not have dared to backport subprocess from Python 3 and resorted to workarounds.
After issues with mixups between subprocess32 and subprocess (some libraries still use subprocess and know nothing about subprocess32) we decided to heed this advice:
So far this looked good. Out of curiousity I ran the Python 2.7 stdlib test suite against subprocess32 and was surprised that it failed.
$ cat Dockerfile
FROM debian:9
RUN apt-get update
RUN apt-get install -y python2.7 libpython2.7-testsuite python-pip
# Install subprocess32, replace subprocess
RUN pip install subprocess32
RUN cp /usr/local/lib/python2.7/dist-packages/subprocess32.py /usr/lib/python2.7/subprocess.py
RUN cp /usr/local/lib/python2.7/dist-packages/_posixsubprocess32.so /usr/lib/python2.7/lib-dynload/_posixsubprocess32.so
# Run regrtest
RUN python -m test.regrtest --verbose test_subprocess
$ sudo docker build .
Sending build context to Docker daemon 2.048kB
Step 1/7 : FROM debian:9
---> de8b49d4b0b3
Step 2/7 : RUN apt-get update
---> Using cache
---> 21f3d4930519
[...]
Step 7/7 : RUN python -m test.regrtest --verbose test_subprocess
---> Running in 58064776ae73
== CPython 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516]
== Linux-4.15.0-47-generic-x86_64-with-debian-9.6 little-endian
== /tmp/test_python_7
Testing with flags: sys.flags(debug=0, py3k_warning=0, division_warning=0, division_new=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, tabcheck=0, verbose=0, unicode=0, bytes_warning=0, hash_randomization=0)
[1/1] test_subprocess
test_call_kwargs (test.test_subprocess.ProcessTestCase) ... ok
[...]
test test_subprocess failed -- multiple errors occurred
skipped 'mswindows only'
======================================================================
ERROR: test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 714, in test_communicate_epipe_only_stdin
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 712, in communicate
self.stdin.write(input)
IOError: [Errno 32] Broken pipe
======================================================================
ERROR: test_exceptions (test.test_subprocess.POSIXProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 803, in test_exceptions
self.assertIn("os.chdir", c.exception.child_traceback)
AttributeError: 'exceptions.OSError' object has no attribute 'child_traceback'
======================================================================
ERROR: test_preexec_errpipe_does_not_double_close_pipes (test.test_subprocess.POSIXProcessTestCase)
Issue16140: Don't double close pipes on preexec error.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 868, in test_preexec_errpipe_does_not_double_close_pipes
stderr=subprocess.PIPE, preexec_fn=raise_it)
File "/usr/lib/python2.7/test/test_subprocess.py", line 827, in __init__
subprocess.Popen.__init__(self, *args, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 614, in __init__
restore_signals, start_new_session)
TypeError: _execute_child() takes exactly 18 arguments (20 given)
======================================================================
ERROR: test_communicate_epipe (test.test_subprocess.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 706, in test_communicate_epipe
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 724, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python2.7/subprocess.py", line 1538, in _communicate
orig_timeout)
File "/usr/lib/python2.7/subprocess.py", line 1684, in _communicate_with_select
bytes_written = os.write(self.stdin.fileno(), chunk)
OSError: [Errno 32] Broken pipe
======================================================================
ERROR: test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 714, in test_communicate_epipe_only_stdin
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 712, in communicate
self.stdin.write(input)
IOError: [Errno 32] Broken pipe
----------------------------------------------------------------------
Ran 146 tests in 16.424s
Some tests are bound to fail given that they access the internals of the Python 2.7 implementation. But these look relevant:
======================================================================
ERROR: test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 714, in test_communicate_epipe_only_stdin
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 712, in communicate
self.stdin.write(input)
IOError: [Errno 32] Broken pipe
======================================================================
ERROR: test_communicate_epipe (test.test_subprocess.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 706, in test_communicate_epipe
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 724, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python2.7/subprocess.py", line 1538, in _communicate
orig_timeout)
File "/usr/lib/python2.7/subprocess.py", line 1684, in _communicate_with_select
bytes_written = os.write(self.stdin.fileno(), chunk)
OSError: [Errno 32] Broken pipe
======================================================================
ERROR: test_communicate_epipe_only_stdin (test.test_subprocess.ProcessTestCaseNoPoll)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/test/test_subprocess.py", line 714, in test_communicate_epipe_only_stdin
p.communicate("x" * 2**20)
File "/usr/lib/python2.7/subprocess.py", line 712, in communicate
self.stdin.write(input)
IOError: [Errno 32] Broken pipe
I intent to find out the cause and fix this, but for reference I'd like to have this issue public in the mean time.
~/workspace/python-subprocess32$ grep -i epipe test_subprocess32.py