Git Product home page Git Product logo

pykern's People

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

pykern's Issues

travis seems to pick wrong versions

When I run pip install -U pyyaml in my environment, it picks 3.13, which seems to be the current released version. However, travis says Best match: PyYAML 5.1b3, which is broken (big time).

Also, pkdeploy test fails with:

SyntaxError: Non-ASCII character '\xc3' in file /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/Sphinx-2.0.0b1-py2.7.egg/sphinx/ext/apidoc.py on line 11

This also is a "b" (beta) release.

py3: Exception doesn't stringify "in"?

@njsmith I was wondering if you know anything about this. I'm trying to get pykern py3 compatible.

In py.test, this form works in py2:

    with pytest.raises(argh.CommandError) as e:
        pkcli.command_error('{abc}', abc='abcdef')
    assert 'abcdef' in e.value, \
        'When passed a format, command_error should output formatted result'

But it fails in py3 with: TypeError: argument of type 'CommandError' is not iterable.

I wrote this test, which gets the same error:

from __future__ import print_function
import argh
try:
    raise argh.CommandError('foo')
except Exception as e:
    assert 'foo' in e

CommandError is a simple subclass of Exception. This works in py2, but something changed in py3, which I can't find quickly. The work around in this case is to str(e), but it might be something that I don't understand about in in py3 vs py2.

Can't unset a boolean from command line or test

sirepo server.py has the config:

oauth_login=(False, bool, 'OAUTH: enable login'),

From the command line, both
export SIREPO_SERVER_OAUTH_LOGIN=0
and
export SIREPO_SERVER_OAUTH_LOGIN=False
result in True values.

Unsetting the value works because False is the default value:
unset SIREPO_SERVER_OAUTH_LOGIN

There would be no way to turn off a value if it has True as the default, for example server.py's enable_source_cache_key.

Possibly related, it isn't possible to turn off a True value from a test, for example, if SIREPO_SERVER_OAUTH_LOGIN=1 in the environment, adding the code below to pkcli/admin_test.py has no effect:

pkconfig.reset_state_for_testing({
'SIREPO_SERVER_OAUTH_LOGIN': False,
})

Instead, the value needed to get set directly:

server.cfg.oauth_login = False

Enhance pkdeploy to install requirements.txt and restart itself

There is currently
(a bug in setuptools)[http://stackoverflow.com/questions/35625488/invalidrequirement-invalid-requirement-parse-error-error-after-updating-a-py]
on PyPI. The fix to travis.yml is:

install:
  - pip install -U pip
  - pip uninstall setuptools
  - pip install 'setuptools<20.2'

This would have to be changed in every travis.yml. Rather simplify with:

script:
  - python setup.py pkdeploy

The workaround could be embedded in pkdeploy. In general, we want to install requirements.txt first so restarting setup.py would fix that problem, too. Might be better to encapsulate in anything that requires a build. Would need some type of sentinel (could be an environment variable with a nested restart).

pkcollections.mapping_merge() should do a deep merge

Currently mapping_merge() copies over key values. Instead, it should merge values with duplicate keys into a union. Merging should recurse so that a structure with values a.b.c.d merged with with a.b.c.z would contain both values:

{
  a: {
    b: {
      c: {
        d: v1,
        z: v2,
      }
    }
  }
}

More flexible dict()

Perhaps this exists somewhere in Python libraries, but documenting here. I want this for a dictionary:

  • d.pkdel('x') - does not raise a key error if 'x does not exist
  • d.pksetdefault('x', some_func) - calls some_func to init if executable else init with value
  • d.some_key = val - refer to keys as attributes if have underscore in the middle or camelcase
  • this.pkmerge(that) - recursively merge d and
  • this.pkimport(json) - initialize from json/yaml/etc.
  • this.pkexport(json) - convert back to json/yaml/etc.

Required not passing args properly

Seeing this on parsing job_driver.docker.hosts:

AssertionError: tls file does not exist for host=v file=/srv/sirepo_job_supervisor/docker_tls/v/cacert.pem

py.test tests/pkconfig_test.py fails

If run py.test tests, pkconfig_test passes. The problem is that pkcli_test imports pkconfig and "collection" runs in the same process, not forked. The fork only takes over when running the tests themselves.

Traceback (most recent call last):
  File "/home/vagrant/src/radiasoft/pykern/tests/pkconfig_test.py", line 36, in test_channel_in
    assert pkconfig.channel_in(_CHANNEL), \
  File "/home/vagrant/src/radiasoft/pykern/pykern/pkconfig.py", line 344, in channel_in
    to_test = cfg.channel
AttributeError: 'NoneType' object has no attribute 'channel'

projex_test failing in travis/tox

https://travis-ci.org/radiasoft/pykern/builds/656481082

py27 create: /home/travis/build/radiasoft/pykern/tests/pkcli/projex_work/proj1/.tox/py27
py27 installdeps: -e/home/travis/build/radiasoft/pykern
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
___________________________________ summary ____________________________________
  py27: commands succeeded
  congratulations :)
Traceback (most recent call last):
  File "/home/travis/build/radiasoft/pykern/.tox/py27/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/__init__.py", line 44, in cmdline
    main(args)
  File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/__init__.py", line 68, in main
    exit_code = session.runcommand()
  File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/__init__.py", line 194, in runcommand
    return self.subcommand_test()
  File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/__init__.py", line 222, in subcommand_test
    run_sequential(self.config, self.venv_dict)
  File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/session/commands/run/sequential.py", line 9, in run_sequential
    if venv.setupenv():
  File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/venv.py", line 621, in setupenv
    envlog.set_python_info(command_path)
  File "/home/travis/build/radiasoft/pykern/.tox/py27/lib/python2.7/site-packages/tox/logs/env.py", line 18, in set_python_info
    answer["executable"] = python_executable
TypeError: 'NoneType' object does not support item assignment

pkdeploy broken: PyPI changed interface

Travis 147 failed with:

Submitting .tox/dist/pykern-20170703.223032.zip to https://pypi.python.org/pypi
Upload failed (410): Gone (This API has been deprecated and removed from legacy PyPI in favor of 
using the APIs available in the new PyPI.org implementation of PyPI (located at https://pypi.org/). 
For more information about migrating your use of this API to PyPI.org, please see 
https://packaging.python.org/guides/migrating-to-pypi-org/#uploading. For more information about
 the sunsetting of this API, please see 
https://mail.python.org/pipermail/distutils-sig/2017-June/030766.html)

pksetup pip installation error with Python 3

When issuing pip install for a module packaged with pksetup, we get the following error: TypeError: Can't mix strings and bytes in path components.

Full traceback:

  Running setup.py install for rslinac ... error
    Complete output from command /tmp/test/35/venv/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-h4bkoqui-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-w1xvs2r3-record/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/test/35/venv/bin/../include/site/python3.5/rslinac:
    running install
    running build
    running build_py
    creating build/lib.macosx-10.11-x86_64-3.5/rslinac
    copying rslinac/__init__.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac
    copying rslinac/base_pkconfig.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac
    copying rslinac/beam_solver.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac
    copying rslinac/rslinac_console.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac
    creating build/lib.macosx-10.11-x86_64-3.5/rslinac/pkcli
    copying rslinac/pkcli/__init__.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac/pkcli
    copying rslinac/pkcli/beam_solver.py -> build/lib.macosx-10.11-x86_64-3.5/rslinac/pkcli
    running egg_info
    writing requirements to rslinac.egg-info/requires.txt
    writing top-level names to rslinac.egg-info/top_level.txt
    writing rslinac.egg-info/PKG-INFO
    writing dependency_links to rslinac.egg-info/dependency_links.txt
    writing entry points to rslinac.egg-info/entry_points.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'rslinac.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'rslinac.egg-info/SOURCES.txt'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-h4bkoqui-build/setup.py", line 63, in <module>
        extra_compile_args=get_compile_args(),
      File "/tmp/test/35/venv/lib/python3.5/site-packages/pykern/pksetup.py", line 377, in setup
        op(**base)
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/install.py", line 539, in run
        self.run_command('build')
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 53, in run
        self.build_package_data()
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 118, in build_package_data
        for package, src_dir, build_dir, filenames in self.data_files:
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 66, in __getattr__
        self.data_files = self._get_data_files()
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 83, in _get_data_files
        return list(map(self._get_pkg_data_files, self.packages or ()))
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 95, in _get_pkg_data_files
        for file in self.find_data_files(package, src_dir)
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 114, in find_data_files
        return self.exclude_data_files(package, src_dir, files)
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 198, in exclude_data_files
        files = list(files)
      File "/tmp/test/35/venv/lib/python3.5/site-packages/setuptools/command/build_py.py", line 234, in <genexpr>
        for pattern in raw_patterns
      File "/tmp/test/35/venv/bin/../lib/python3.5/posixpath.py", line 89, in join
        genericpath._check_arg_types('join', a, *p)
      File "/tmp/test/35/venv/bin/../lib/python3.5/genericpath.py", line 145, in _check_arg_types
        raise TypeError("Can't mix strings and bytes in path components") from None
    TypeError: Can't mix strings and bytes in path components

    ----------------------------------------
Command "/tmp/test/35/venv/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-h4bkoqui-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-w1xvs2r3-record/install-record.txt --single-version-externally-managed --compile --install-headers /tmp/test/35/venv/bin/../include/site/python3.5/rslinac" failed with error code 1 in /var/folders/63/3jkz49w174x2r847ttp6whgr0000gn/T/pip-h4bkoqui-build/

text line reader

It would be nice to have a pkio.io text reader methd which went line by line, rather than reading the whole file into memory with read_text(). Maybe with a callback function by line. Something like:

    with io.open(str(path), encoding=locale.getpreferredencoding()) as f:
        for num, line in enumerate(f):
            callback(line, num)

infinite loop in pykern.pytest_plugin

I'm not sure this is in the pytest_plugin or not, because the loop seems like it would terminate, but the strace below is happening in a container with only /home/vagrant/src/radiasoft/sirepo/tests mounted, i.e. there is no setup.py so the loop would exit with a return of None.

getcwd("/home/vagrant/src/radiasoft/sirepo/tests", 1024) = 41
stat("/home/vagrant/src/radiasoft/sirepo/tests/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/radiasoft/sirepo/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/radiasoft/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/setup.py", 0x7ffcf8df0260)  = -1 ENOENT (No such file or directory)
getcwd("/home/vagrant/src/radiasoft/sirepo/tests", 1024) = 41
stat("/home/vagrant/src/radiasoft/sirepo/tests/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/radiasoft/sirepo/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/radiasoft/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/src/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/vagrant/setup.py", 0x7ffcf8df0260) = -1 ENOENT (No such file or directory)
stat("/home/setup.py", 0x7ffcf8df0260)  = -1 ENOENT (No such file or directory)
getcwd("/home/vagrant/src/radiasoft/sirepo/tests", 1024) = 41

pkconfig should use PKDict instead of OrderedMapping

@moellep @mkeilman @e-carlin I think pkconfig should use PKDict, not OrderedMapping. PKDict is a full dict which has benefits, e.g. being able to call get. OrderedMapping is not a dict.

I tried it, and it works fine. No code has to change anywhere. The only problem would be if existing cfg values would collide with dict attributes (get, copy, etc.).

pytest 3.3 breaks capsys

This is in the logging test of pkdebug_test. Forcing pytest to <=3.2.3 works, but we'll need to figure this out eventually:

  File "/home/travis/build/radiasoft/pykern/tests/pkdebug_test.py", line 115, in test_logging
    assert 'WARNING:root:warn_xyzzy\n' == err, \
AssertionError: When logging is first initialized, warn should output
assert 'WARNING:root:warn_xyzzy\n' == ''
  - WARNING:root:warn_xyzzy

pkdformat should validate input

pkdfromat cannot handle input like 'Hello {user}'. We should validate that there are no words between {} and/or validate that the number of {} == len(kwargs)

Drop py.path.local, because cannot hold a relative path

py.path.local('foo/bar') is an absolute path relative to the current directory. There does not seem to be a way to identify the original portion of the path. There's also no query like os.path.isabs.

I'm surprised I didn't notice this before, but it's a real problem. If you have something like json_filename('foo', run_dir) which should return a relative file name from other a py.path.local or a string, it can't in the former case, that is, json_filename(py.path.local('foo'), run_dir) is always relative to the current directory, not the run_dir.

I think we have to drop py.path in favor of another path object or just using os.path explicitly with higher level functions in pkio.

Remove load_path from pkconfig

configuring from files is not used. load_path prevents modules from importing (below) and complicates coalescing values for export. Inputs only from environ.

Traceback (most recent call last):
  File "server.py", line 16, in <module>
    from sirepo import simulation_db
  File "/home/vagrant/src/radiasoft/sirepo/sirepo/simulation_db.py", line 31, in <module>
    import sirepo.auth
  File "/home/vagrant/src/radiasoft/sirepo/sirepo/auth/__init__.py", line 14, in <module>
    from sirepo import cookie
  File "/home/vagrant/src/radiasoft/sirepo/sirepo/cookie.py", line 226, in <module>
    'Add secure attriute to Set-Cookie',
  File "/home/vagrant/src/radiasoft/pykern/pykern/pkconfig.py", line 384, in init
    '{}: module root not in load_path ({})'.format(m.__name__, _load_path)
AssertionError: sirepo.cookie: module root not in load_path (['pykern'])

On travis: ImportError: No module named download

If you look at the previous build, it's got the same version of pip, but it doesn't get this error:

$ python --version
Python 2.7.14
$ pip --version
pip 9.0.1 from /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages (python 2.7)
install.1
$ pip install setuptools==32.1.3
$ python setup.py pkdeploy
Traceback (most recent call last):
  File "setup.py", line 7, in <module>
    from pykern.pksetup import setup
  File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 51, in <module>
    import pip.download
ImportError: No module named download

test_check_call_with_signals sometimes fails with: DID NOT RAISE <type 'exceptions.RuntimeError'>

The failure doesn't happen on my machine. It only seems to fail Travis when initiated by GitHub's webhook. If you restart the build, it works --
complete failure log.

�[31m�[1m_________________________ test_check_call_with_signals _________________________�[0m
def test_check_call_with_signals():
        from pykern import pksubprocess
        from pykern import pkunit
        import os
        import signal

        messages = []
        def msg(*args):
            s = args[0]
            messages.append(s.format(*args[1:]))

        signals = []
        def signal_handler(sig, frame):
            signals.append(sig)

        with pkunit.save_chdir_work():
            with open('true.out', 'w+') as o:
                pksubprocess.check_call_with_signals(['true'], output=o)
                o.seek(0)
                actual = o.read()
                assert '' == actual, \
                    'Expecting empty output "{}"'.format(actual)

            with open('echo.out', 'w+') as o:
                messages = []
                tag = 'xyzzy'
                pksubprocess.check_call_with_signals(['echo', tag], output=o, msg=msg)
                o.seek(0)
                actual = o.read()
                assert tag in actual, \
                    '"{}" not in output "{}"'.format(tag, actual)
                assert 'started' in messages[0], \
                    '"started" not in messages[0] "{}"'.format(messages[0])
                assert 'normal exit' in messages[1], \
                    '"normal exit" not in messages[1] "{}"'.format(messages[1])

            with open('kill.out', 'w+') as o:
                messages = []
                signals = []
                signal.signal(signal.SIGTERM, signal_handler)
                with open('kill.sh', 'w') as f:
                    f.write('kill -TERM {}\nsleep 10'.format(os.getpid()))
                cmd = ['sh', 'kill.sh']
                with pytest.raises(RuntimeError):
>                   pksubprocess.check_call_with_signals(cmd, output=o, msg=msg)
E                   Failed: DID NOT RAISE <type 'exceptions.RuntimeError'>

tests/pksubprocess_test.py:54: Failed
�[31m�[1m==================== 1 failed, 62 passed in 107.01 seconds =====================�[0m
�[31mERROR: InvocationError: '/home/travis/build/radiasoft/pykern/.tox/py27/bin/python setup.py build test'�[0m
___________________________________ summary ____________________________________
�[31mERROR:   py27: commands failed�[0m
Traceback (most recent call last):
  File "setup.py", line 37, in <module>
    'Topic :: Utilities',
  File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 378, in setup
    op(**base)
  File "/opt/python/2.7.9/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 128, in run
    self.__run_cmd('tox')
  File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 162, in __run_cmd
    cmd.run()
  File "/home/travis/build/radiasoft/pykern/pykern/pksetup.py", line 273, in run
    subprocess.check_call(['tox'])
  File "/opt/python/2.7.9/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['tox']' returned non-zero exit status 1

travis_time:end:10019841:start=1473455463574674540,finish=1473455597563051870,duration=133988377330
�[0K
�[31;1mThe command "python setup.py pkdeploy" exited with 1.�[0m

pytest outputs nulls sometimes on failure

When an error occurs about 100KB of nulls are output.

This has been ongoing for years. I narrowed it down a bit. It's definitely related to the xdist plugin which forks processes. It also is caused by outputting pkdebug to /dev/stderr. Setting it to /dev/tty doesn't cause nulls to be output.

Turning off xdist is not an option for all tests, because they modify global state. I could maybe get the tests to run this way, but it would be tricky, e.g. the test to capture IPython output.

I'm going to keep exploring, because it is really annoying.

pkdebug needs to redirect Python logging on request

We want to collate all logging output in a single output file. The user doesn't want to have to configure flask and pkdebug logging. By default, pkdebug.cfg.redirect_logging should be True so that all output goes to a single place for our purposes.

Use SafeLoader for pkyaml

/home/vagrant/src/radiasoft/pykern/pykern/pkyaml.py:51: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

pkdexc leaves out important information on python 3

In Python 3, exceptions carry more contextual information than just their traceback. They also have .__context__ and .__cause__ attributes that carry information about other exceptions that may have triggered this one. And then those exceptions might have their own .__context__ and .__cause__. (See PEP 3134 for details.)

This gets even more complex with trio, because you have to handle "exception groups" (when multiple unhandled exceptions are raised simultaneously in different branches of the call tree, and then end up getting grouped together).

Unfortunately, pkdexc doesn't know about any of this stuff: it just formats the latest exception's traceback, and throws away all the contextual information:

pykern/pykern/pkdebug.py

Lines 159 to 165 in 784dded

try:
stack = traceback.format_stack()[:-2]
e = sys.exc_info()
stack += traceback.format_tb(e[2])
return ''.join(traceback.format_exception_only(e[0], e[1]) + stack)
except Exception as e:
return 'pykern.pkdebug.pkdexc: unable to retrieve exception info'

I think the simplest thing to do on python 3 is to use traceback.format_exceptionwhich does know how to handle all this stuff. For example:

In [2]: try: 
   ...:     try: 
   ...:         raise ValueError 
   ...:     except: 
   ...:         raise RuntimeError 
   ...: except Exception as e: 
   ...:     exc = e                                                             

In [3]: traceback.format_exception(type(exc), exc, exc.__traceback__)           
Out[3]: 
[
    'Traceback (most recent call last):\n',
    '  File "<ipython-input-2-8807d67037fa>", line 3, in <module>\n    '
        'raise ValueError\n',
    'ValueError\n',
    '\nDuring handling of the above exception, another exception '
        'occurred:\n\n',
    'Traceback (most recent call last):\n',
    '  File "<ipython-input-2-8807d67037fa>", line 5, in <module>\n    '
        'raise RuntimeError\n',
    'RuntimeError\n'
]

This does mean we can't as easily graft the current stack onto the exception's stack, like pkdexc does now. But we could print the current stack with a little header like Exception encountered at:\n + current stack + format_exception output.

install bug with pykern

Have tried pip install pykern and pip install pykern --upgrade and get the same error w/ traceback:

 Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/private/tmp/pip-build-q2q8jb/pykern/setup.py", line 40, in <module>
    'Topic :: Utilities',
  File "pykern/pksetup.py", line 328, in setup
    op(**base)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
    _setup_distribution = dist = klass(attrs)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools/dist.py", line 315, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
    replace_conflicting=True,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 850, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1115, in best_match
    dist = working_set.find(req)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 719, in find
    raise VersionConflict(dist, req)
pkg_resources.VersionConflict: (Sphinx 1.4.8 (/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages), Requirement.parse('sphinx==1.3.5'))

----------------------------------------
 Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-build-q2q8jb/pykern/

pkcli.github.labels needs to analyze netrc exception

This case should be caught:

pykern github labels radiasoft/foobar
github.py:191:_cfg ~/.netrc access too permissive: access permissions must rest\
rict access to only the owner (/home/vagrant/.netrc, line 3)
pykern.pkcli.github.password: config value missing and is required

python setup.py <anything> is deprecated

pypa/setuptools#1684 talks about deprecating python setup.py test and all other arguments. The plan is to use tox, which we use for testing, but it's unwieldy for other things, like deployment.

Migrate the code out of pysetup into another tool that supports deployment, documentation, etc.

Read the docs build process should be looked, because it calls python setup.py. Perhaps there's a way to just call an arbitrary command or have setup.py do that instead. see https://github.com/rtfd/readthedocs.org/tree/master/readthedocs/doc_builder

See https://github.com/jaraco/skeleton

'str' object has no attribute 'decode'

Jan 31 23:17:12  1463     0 ../../../../../../../sirepo/pkcli/job_agent.py:614:start invalid format format=cmd={} stdin={} args=<generator object pkdformat.<locals>.<genexpr> at 0x7f70114\
e24f8> kwargs={} stack=Traceback (most recent call last):
  File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 638, in _format_arg
    return _string(s)
  File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 612, in _string
    string = string.decode('string_escape')
AttributeError: 'str' object has no attribute 'decode'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 220, in pkdformat
    return fmt.format(*args, **kwargs)
  File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 218, in <genexpr>
    args = (_format_arg(a) for a in args)
  File "/home/vagrant/src/radiasoft/pykern/pykern/pkdebug.py", line 641, in _format_arg
    'unable to format obj type={} exception={}'.format(type(obj), e),
NameError: name 'e' is not defined

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.