radiasoft / pykern Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
This is tricky, because we are using str.format
, but we should avoid massive output due to large (and possibly recursive) data structures.
When I run pip install -U pyyaml
in my environment, it picks 3.13, which seems to be the current released version. However, travis says Best match: PyYAML 5.1b3
, which is broken (big time).
Also, pkdeploy test fails with:
SyntaxError: Non-ASCII character '\xc3' in file /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/Sphinx-2.0.0b1-py2.7.egg/sphinx/ext/apidoc.py on line 11
This also is a "b" (beta) release.
In order to serialize all log output, writes should go through a threading lock
@njsmith I was wondering if you know anything about this. I'm trying to get pykern py3 compatible.
In py.test, this form works in py2:
with pytest.raises(argh.CommandError) as e:
pkcli.command_error('{abc}', abc='abcdef')
assert 'abcdef' in e.value, \
'When passed a format, command_error should output formatted result'
But it fails in py3 with: TypeError: argument of type 'CommandError' is not iterable
.
I wrote this test, which gets the same error:
from __future__ import print_function
import argh
try:
raise argh.CommandError('foo')
except Exception as e:
assert 'foo' in e
CommandError is a simple subclass of Exception. This works in py2, but something changed in py3, which I can't find quickly. The work around in this case is to str(e)
, but it might be something that I don't understand about in
in py3 vs py2.
Should copy and merge rsmanifest.json into directory. Directory named by date and description
in meta data.
As a result, the instructions here
https://github.com/radiasoft/pykern/wiki/CreatePythonProject
are broken
It seems the 'init-rs-tree' option is gone, so one must use 'init-tree'
However, it is (seemingly) impossible to satisfy the demands of 'init-tree'
As a result, I don't know how to correct the instructions in the wiki
We need to be able to copy the config to a new process, e.g. this test.
It hasn't been used, and the way we are configuring doesn't seem to require it so allow it, but don't force it to exist.
This fixes build: https://travis-ci.org/radiasoft/rssynergia/builds/163203076
Logging during a test should go to capsys. This probably means fixing #22. See discussion in Sirepo
There needs to be an exec which doesn't loop (via env var?) when pip updates. Otherwise, you get package issues.
sirepo server.py has the config:
oauth_login=(False, bool, 'OAUTH: enable login'),
From the command line, both
export SIREPO_SERVER_OAUTH_LOGIN=0
and
export SIREPO_SERVER_OAUTH_LOGIN=False
result in True values.
Unsetting the value works because False is the default value:
unset SIREPO_SERVER_OAUTH_LOGIN
There would be no way to turn off a value if it has True as the default, for example server.py's enable_source_cache_key.
Possibly related, it isn't possible to turn off a True value from a test, for example, if SIREPO_SERVER_OAUTH_LOGIN=1 in the environment, adding the code below to pkcli/admin_test.py has no effect:
pkconfig.reset_state_for_testing({
'SIREPO_SERVER_OAUTH_LOGIN': False,
})
Instead, the value needed to get set directly:
server.cfg.oauth_login = False
pytest-forked (invoked with py.test --forked
) runs each test function (case) in its own process. This makes it impossible to have module level fixtures, everything fixture is per function. We need the ability to start/stop a process at the module (radiasoft/sirepo#2061).
https://travis-ci.org/radiasoft/pykern/builds/538817044
Since we are using a shared IP, I suspect we'll git hit with this.
There is currently
(a bug in setuptools)[http://stackoverflow.com/questions/35625488/invalidrequirement-invalid-requirement-parse-error-error-after-updating-a-py]
on PyPI. The fix to travis.yml is:
install:
- pip install -U pip
- pip uninstall setuptools
- pip install 'setuptools<20.2'
This would have to be changed in every travis.yml. Rather simplify with:
script:
- python setup.py pkdeploy
The workaround could be embedded in pkdeploy. In general, we want to install requirements.txt first so restarting setup.py would fix that problem, too. Might be better to encapsulate in anything that requires a build. Would need some type of sentinel (could be an environment variable with a nested restart).
Currently mapping_merge() copies over key values. Instead, it should merge values with duplicate keys into a union. Merging should recurse so that a structure with values a.b.c.d merged with with a.b.c.z would contain both values:
{
a: {
b: {
c: {
d: v1,
z: v2,
}
}
}
}
Perhaps this exists somewhere in Python libraries, but documenting here. I want this for a dictionary:
Seeing this on parsing job_driver.docker.hosts:
AssertionError: tls file does not exist for host=v file=/srv/sirepo_job_supervisor/docker_tls/v/cacert.pem
If run py.test tests
, pkconfig_test passes. The problem is that pkcli_test imports pkconfig and "collection" runs in the same process, not forked. The fork only takes over when running the tests themselves.
Traceback (most recent call last):
File "/home/vagrant/src/radiasoft/pykern/tests/pkconfig_test.py", line 36, in test_channel_in
assert pkconfig.channel_in(_CHANNEL), \
File "/home/vagrant/src/radiasoft/pykern/pykern/pkconfig.py", line 344, in channel_in
to_test = cfg.channel
AttributeError: 'NoneType' object has no attribute 'channel'
https://travis-ci.org/radiasoft/pykern/builds/656481082
py27 create: /home/travis/build/radiasoft/pykern/tests/pkcli/projex_work/proj1/.tox/py27
py27 installdeps: -e/home/travis/build/radiasoft/pykern
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
___________________________________ summary ____________________________________
py27: commands succeeded
congratulations :)
Traceback (most recent call last):
File "/home/travis/build/radiasoft/pykern/.tox/py27/bin/tox", line 11, in <module>
sys.exit(cmdline())
File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/__init__.py", line 44, in cmdline
main(args)
File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/__init__.py", line 68, in main
exit_code = session.runcommand()
File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/__init__.py", line 194, in runcommand
return self.subcommand_test()
File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/__init__.py", line 222, in subcommand_test
run_sequential(self.config, self.venv_dict)
File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/commands/run/sequential.py", line 9, in run_sequential
if venv.setupenv():
File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/venv.py", line 621, in setupenv
envlog.set_python_info(command_path)
File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/logs/env.py", line 18, in set_python_info
answer["executable"] = python_executable
TypeError: 'NoneType' object does not support item assignment
When calling subprocesses, it's important to cascade config, possibly all. Need a routine for this.
Travis 147 failed with:
Submitting .tox/dist/pykern-20170703.223032.zip to https://pypi.python.org/pypi
Upload failed (410): Gone (This API has been deprecated and removed from legacy PyPI in favor of
using the APIs available in the new PyPI.org implementation of PyPI (located at https://pypi.org/).
For more information about migrating your use of this API to PyPI.org, please see
https://packaging.python.org/guides/migrating-to-pypi-org/#uploading. For more information about
the sunsetting of this API, please see
https://mail.python.org/pipermail/distutils-sig/2017-June/030766.html)
When issuing pip install
for a module packaged with pksetup
, we get the following error: TypeError: Can't mix strings and bytes in path components
.
Full traceback:
Running setup.py install for rslinac ... error
Complete output from command /tmp/test/35/venv/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-h4bkoqui-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-w1xvs2r3-record/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/test/35/venv/bin/../include/site/python3.5/rslinac:
running install
running build
running build_py
creating build/lib.macosx-10.11-x86_64-3.5/rslinac
copying rslinac/__init__.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac
copying rslinac/base_pkconfig.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac
copying rslinac/beam_solver.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac
copying rslinac/rslinac_console.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac
creating build/lib.macosx-10.11-x86_64-3.5/rslinac/pkcli
copying rslinac/pkcli/__init__.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac/pkcli
copying rslinac/pkcli/beam_solver.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac/pkcli
running egg_info
writing requirements to rslinac.egg-info/requires.txt
writing top-level names to rslinac.egg-info/top_level.txt
writing rslinac.egg-info/PKG-INFO
writing dependency_links to rslinac.egg-info/dependency_links.txt
writing entry points to rslinac.egg-info/entry_points.txt
warning: manifest_maker: standard file '-c' not found
reading manifest file 'rslinac.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'rslinac.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-h4bkoqui-build/setup.py", line 63, in <module>
extra_compile_args=get_compile_args(),
File "/tmp/test/35/venv/lib/python3.5/site-packages/pykern/pksetup.py", line 377, in setup
op(**base)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/install.py", line 539, in run
self.run_command('build')
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 53, in run
self.build_package_data()
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 118, in build_package_data
for package, src_dir, build_dir, filenames in self.data_files:
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 66, in __getattr__
self.data_files = self._get_data_files()
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 83, in _get_data_files
return list(map(self._get_pkg_data_files, self.packages or ()))
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 95, in _get_pkg_data_files
for file in self.find_data_files(package, src_dir)
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 114, in find_data_files
return self.exclude_data_files(package, src_dir, files)
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 198, in exclude_data_files
files = list(files)
File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 234, in <genexpr>
for pattern in raw_patterns
File "/tmp/test/35/venv/bin/../lib/python3.5/posixpath.py", line 89, in join
genericpath._check_arg_types('join', a, *p)
File "/tmp/test/35/venv/bin/../lib/python3.5/genericpath.py", line 145, in _check_arg_types
raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components
----------------------------------------
Command "/tmp/test/35/venv/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-h4bkoqui-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-w1xvs2r3-record/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/test/35/venv/bin/../include/site/python3.5/rslinac" failed with error code 1 in /var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-h4bkoqui-build/
It would be nice to have a pkio.io text reader methd which went line by line, rather than reading the whole file into memory with read_text(). Maybe with a callback function by line. Something like:
with io.open(str(path), encoding=locale.getpreferredencoding()) as f:
for num, line in enumerate(f):
callback(line, num)
I'm not sure this is in the pytest_plugin or not, because the loop seems like it would terminate, but the strace below is happening in a container with only /home/vagrant/src/radiasoft/sirepo/tests
mounted, i.e. there is no setup.py so the loop would exit with a return of None.
getcwd("/home/vagrant/src/radiasoft/sirepo/tests", 1024) = 41
stat("/home/vagrant/src/radiasoft/sirepo/tests/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/radiasoft/sirepo/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/radiasoft/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
getcwd("/home/vagrant/src/radiasoft/sirepo/tests", 1024) = 41
stat("/home/vagrant/src/radiasoft/sirepo/tests/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/radiasoft/sirepo/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/radiasoft/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
getcwd("/home/vagrant/src/radiasoft/sirepo/tests", 1024) = 41
@moellep @mkeilman @e-carlin I think pkconfig should use PKDict, not OrderedMapping. PKDict is a full dict
which has benefits, e.g. being able to call get. OrderedMapping is not a dict.
I tried it, and it works fine. No code has to change anywhere. The only problem would be if existing cfg values would collide with dict attributes (get, copy, etc.).
This is in the logging
test of pkdebug_test. Forcing pytest to <=3.2.3 works, but we'll need to figure this out eventually:
File "/home/travis/build/radiasoft/pykern/tests/pkdebug_test.py", line 115, in test_logging
assert 'WARNING:root:warn_xyzzy\n' == err, \
AssertionError: When logging is first initialized, warn should output
assert 'WARNING:root:warn_xyzzy\n' == ''
- WARNING:root:warn_xyzzy
Steps to reproduce:
sirepo test_http
pkdfromat cannot handle input like 'Hello {user}'
. We should validate that there are no words between {}
and/or validate that the number of {} == len(kwargs)
Sirepo uses exec this way, but for it to work in py3 (see shadow), we need to make a copy of locals()
. pkrunpy should encapsulate this.
py.path.local('foo/bar')
is an absolute path relative to the current directory. There does not seem to be a way to identify the original portion of the path. There's also no query like os.path.isabs
.
I'm surprised I didn't notice this before, but it's a real problem. If you have something like json_filename('foo', run_dir)
which should return a relative file name from other a py.path.local or a string, it can't in the former case, that is, json_filename(py.path.local('foo'), run_dir)
is always relative to the current directory, not the run_dir
.
I think we have to drop py.path in favor of another path object or just using os.path explicitly with higher level functions in pkio.
configuring from files is not used. load_path prevents modules from importing (below) and complicates coalescing values for export. Inputs only from environ.
Traceback (most recent call last):
File "server.py", line 16, in <module>
from sirepo import simulation_db
File "/home/vagrant/src/radiasoft/sirepo/sirepo/simulation_db.py", line 31, in <module>
import sirepo.auth
File "/home/vagrant/src/radiasoft/sirepo/sirepo/auth/__init__.py", line 14, in <module>
from sirepo import cookie
File "/home/vagrant/src/radiasoft/sirepo/sirepo/cookie.py", line 226, in <module>
'Add secure attriute to Set-Cookie',
File "/home/vagrant/src/radiasoft/pykern/pykern/pkconfig.py", line 384, in init
'{}: module root not in load_path ({})'.format(m.__name__, _load_path)
AssertionError: sirepo.cookie: module root not in load_path (['pykern'])
If you look at the previous build, it's got the same version of pip
, but it doesn't get this error:
$ python --version
Python 2.7.14
$ pip --version
pip 9.0.1 from /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages (python 2.7)
install.1
$ pip install setuptools==32.1.3
$ python setup.py pkdeploy
Traceback (most recent call last):
File "setup.py", line 7, in <module>
from pykern.pksetup import setup
File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 51, in <module>
import pip.download
ImportError: No module named download
See radiasoft/rsconf#96 for a cascaded error that would have been fixed if sirepo.auth.cfg.methods was a frozenset not a tuple.
sets are more logical for lists of things to be configured unless order matters, which it shouldn't (in general).
The failure doesn't happen on my machine. It only seems to fail Travis when initiated by GitHub's webhook. If you restart the build, it works --
complete failure log.
�[31m�[1m_________________________ test_check_call_with_signals _________________________�[0m
def test_check_call_with_signals():
from pykern import pksubprocess
from pykern import pkunit
import os
import signal
messages = []
def msg(*args):
s = args[0]
messages.append(s.format(*args[1:]))
signals = []
def signal_handler(sig, frame):
signals.append(sig)
with pkunit.save_chdir_work():
with open('true.out', 'w+') as o:
pksubprocess.check_call_with_signals(['true'], output=o)
o.seek(0)
actual = o.read()
assert '' == actual, \
'Expecting empty output "{}"'.format(actual)
with open('echo.out', 'w+') as o:
messages = []
tag = 'xyzzy'
pksubprocess.check_call_with_signals(['echo', tag], output=o, msg=msg)
o.seek(0)
actual = o.read()
assert tag in actual, \
'"{}" not in output "{}"'.format(tag, actual)
assert 'started' in messages[0], \
'"started" not in messages[0] "{}"'.format(messages[0])
assert 'normal exit' in messages[1], \
'"normal exit" not in messages[1] "{}"'.format(messages[1])
with open('kill.out', 'w+') as o:
messages = []
signals = []
signal.signal(signal.SIGTERM, signal_handler)
with open('kill.sh', 'w') as f:
f.write('kill -TERM {}\nsleep 10'.format(os.getpid()))
cmd = ['sh', 'kill.sh']
with pytest.raises(RuntimeError):
> pksubprocess.check_call_with_signals(cmd, output=o, msg=msg)
E Failed: DID NOT RAISE <type 'exceptions.RuntimeError'>
tests/pksubprocess_test.py:54: Failed
�[31m�[1m==================== 1 failed, 62 passed in 107.01 seconds =====================�[0m
�[31mERROR: InvocationError: '/home/travis/build/radiasoft/pykern/.tox/py27/bin/python setup.py build test'�[0m
___________________________________ summary ____________________________________
�[31mERROR: py27: commands failed�[0m
Traceback (most recent call last):
File "setup.py", line 37, in <module>
'Topic :: Utilities',
File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 378, in setup
op(**base)
File "/opt/python/2.7.9/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 128, in run
self.__run_cmd('tox')
File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 162, in __run_cmd
cmd.run()
File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 273, in run
subprocess.check_call(['tox'])
File "/opt/python/2.7.9/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['tox']' returned non-zero exit status 1
travis_time:end:10019841:start=1473455463574674540,finish=1473455597563051870,duration=133988377330
�[0K
�[31;1mThe command "python setup.py pkdeploy" exited with 1.�[0m
When an error occurs about 100KB of nulls are output.
This has been ongoing for years. I narrowed it down a bit. It's definitely related to the xdist plugin which forks processes. It also is caused by outputting pkdebug to /dev/stderr. Setting it to /dev/tty doesn't cause nulls to be output.
Turning off xdist is not an option for all tests, because they modify global state. I could maybe get the tests to run this way, but it would be tricky, e.g. the test to capture IPython output.
I'm going to keep exploring, because it is really annoying.
The pkdebug doc is out of date
Needed by Salt on Fedora 23
pykern.pytest_plugin now sets $PYKERN_PKDEBUG_OUTPUT to /dev/stderr
so output comes out during pytest runs if you use pkdp, pkdc, etc.
This works around pytest-dev/pytest#935
This is added by projex automatically, but not on existing projects.
It's necessary that we have our own conftest.py that loads pykern.pytest_plugin.
We want to collate all logging output in a single output file. The user doesn't want to have to configure flask and pkdebug logging. By default, pkdebug.cfg.redirect_logging should be True so that all output goes to a single place for our purposes.
/home/vagrant/src/radiasoft/pykern/pykern/pkyaml.py:51: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
In Python 3, exceptions carry more contextual information than just their traceback. They also have .__context__
and .__cause__
attributes that carry information about other exceptions that may have triggered this one. And then those exceptions might have their own .__context__
and .__cause__
. (See PEP 3134 for details.)
This gets even more complex with trio, because you have to handle "exception groups" (when multiple unhandled exceptions are raised simultaneously in different branches of the call tree, and then end up getting grouped together).
Unfortunately, pkdexc doesn't know about any of this stuff: it just formats the latest exception's traceback, and throws away all the contextual information:
Lines 159 to 165 in 784dded
I think the simplest thing to do on python 3 is to use traceback.format_exception
which does know how to handle all this stuff. For example:
In [2]: try:
...: try:
...: raise ValueError
...: except:
...: raise RuntimeError
...: except Exception as e:
...: exc = e
In [3]: traceback.format_exception(type(exc), exc, exc.__traceback__)
Out[3]:
[
'Traceback (most recent call last):\n',
' File "<ipython-input-2-8807d67037fa>", line 3, in <module>\n '
'raise ValueError\n',
'ValueError\n',
'\nDuring handling of the above exception, another exception '
'occurred:\n\n',
'Traceback (most recent call last):\n',
' File "<ipython-input-2-8807d67037fa>", line 5, in <module>\n '
'raise RuntimeError\n',
'RuntimeError\n'
]
This does mean we can't as easily graft the current stack onto the exception's stack, like pkdexc
does now. But we could print the current stack with a little header like Exception encountered at:\n
+ current stack + format_exception
output.
Have tried pip install pykern
and pip install pykern --upgrade
and get the same error w/ traceback:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-build-q2q8jb/pykern/setup.py", line 40, in <module>
'Topic :: Utilities',
File "pykern/pksetup.py", line 328, in setup
op(**base)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools/dist.py", line 315, in __init__
self.fetch_build_eggs(attrs['setup_requires'])
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
replace_conflicting=True,
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 850, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1115, in best_match
dist = working_set.find(req)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 719, in find
raise VersionConflict(dist, req)
pkg_resources.VersionConflict: (Sphinx 1.4.8 (/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages), Requirement.parse('sphinx==1.3.5'))
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-build-q2q8jb/pykern/
when there are problems with .travis.yml, all projects need to be updated. .travis.yml should be dynamic as possible with a curl installer.
Add something like pkconfig.Alias, which allows absolute names. Then alias PYKERN_PKDEBUG_CONTROL to PKDEBUG to make easier to set output controls.
This case should be caught:
pykern github labels radiasoft/foobar
github.py:191:_cfg ~/.netrc access too permissive: access permissions must rest\
rict access to only the owner (/home/vagrant/.netrc, line 3)
pykern.pkcli.github.password: config value missing and is required
pypa/setuptools#1684 talks about deprecating python setup.py test
and all other arguments. The plan is to use tox, which we use for testing, but it's unwieldy for other things, like deployment.
Migrate the code out of pysetup into another tool that supports deployment, documentation, etc.
Read the docs build process should be looked, because it calls python setup.py. Perhaps there's a way to just call an arbitrary command or have setup.py do that instead. see https://github.com/rtfd/readthedocs.org/tree/master/readthedocs/doc_builder
Jan 31 23:17:12 1463 0 ../../../../../../../sirepo/pkcli/job_agent.py:614:start invalid format format=cmd={} stdin={} args=<generator object pkdformat.<locals>.<genexpr> at 0x7f70114\
e24f8> kwargs={} stack=Traceback (most recent call last):
File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 638, in _format_arg
return _string(s)
File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 612, in _string
string = string.decode('string_escape')
AttributeError: 'str' object has no attribute 'decode'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 220, in pkdformat
return fmt.format(*args, **kwargs)
File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 218, in <genexpr>
args = (_format_arg(a) for a in args)
File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 641, in _format_arg
'unable to format obj type={} exception={}'.format(type(obj), e),
NameError: name 'e' is not defined
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.