aurzenligl / pytest-logger Goto Github PK

View Code? Open in Web Editor NEW

33.0 3.0 12.0 209 KB

Pytest plugin configuring handlers for loggers from Python logging module.

License: MIT License

Python 99.36% Shell 0.64%

pytest python

pytest-logger's Introduction

Pytest-logger is a pytest plugin configuring handlers for loggers from Python logging module.

You can install pytest-logger via pip from PyPI:

$ [sudo] pip install pytest-logger

Plugin puts logs on per-logger basis to:

standard output,
files within log-specific directory under pytest's tmpdir_factory session directory.

You can setup plugin using hook:

#conftest.py
import os

def pytest_logger_config(logger_config):
    logger_config.add_loggers(['foo', 'bar', 'baz'], stdout_level='info')
    logger_config.set_log_option_default('foo,bar')

def pytest_logger_logdirlink(config):
    return os.path.join(os.path.dirname(__file__), 'mylogs')

have logging tests or libraries (including fixtures):

#test_something.py
import pytest
import logging

foo = logging.getLogger('foo')
bar = logging.getLogger('bar')
baz = logging.getLogger('baz')

@pytest.yield_fixture(scope='session')
def session_thing():
    foo.debug('constructing session thing')
    yield
    foo.debug('destroying session thing')

@pytest.yield_fixture
def testcase_thing():
    foo.debug('constructing testcase thing')
    yield
    foo.debug('destroying testcase thing')

def test_one(session_thing, testcase_thing):
    foo.info('one executes')
    bar.warning('this test does nothing aside from logging')
    baz.info('extra log, rarely read')

def test_two(session_thing, testcase_thing):
    foo.info('two executes')
    bar.warning('neither does this')
    baz.info('extra log, not enabled by default')

and expect output in terminal (if not captured):

$ py.test -s -v
(...)
test_something.py::test_one
00:00.002 inf foo: one executes
00:00.002 wrn bar: this test does nothing aside from logging
PASSED

test_something.py::test_two
00:00.000 inf foo: two executes
00:00.000 wrn bar: neither does this
PASSED

being able to change this output by cmdline option:

$ pytest -s -v --log foo.debug,baz
(...)
test_something.py::test_one
00:00.002 dbg foo: constructing session thing
00:00.002 dbg foo: constructing testcase thing
00:00.002 inf foo: one executes
00:00.003 inf baz: extra log, rarely read
PASSED

test_something.py::test_two
00:00.000 dbg foo: constructing testcase thing
00:00.000 inf foo: two executes
00:00.001 inf baz: extra log, not enabled by default
PASSED

and - the same - in filesystem:

$ file mylogs
mylogs: symbolic link to `/tmp/pytest-of-aurzenligl/pytest-48/logs'

$ tree mylogs
mylogs
`-- test_something.py
    |-- test_one
    |   |-- bar
    |   |-- baz
    |   `-- foo
    `-- test_two
        |-- bar
        |-- baz
        `-- foo

Distributed under the terms of the MIT license, pytest-logger is free and open source software.

pytest-logger's People

Contributors

Stargazers

Watchers

Forkers

skrchnavy lorengordon farquet ethantang95 florczakraf mulawamichal zmc andreabisello tbishop3 remylau nikaswoon lcosmin

pytest-logger's Issues

Possible to set level depending on if -v/-vv/-vvv is set

I want to set different level on different loggers, depending on if -v or -vv or -vvv is used. Can that be done?

Allow relative path in pytest_logger_logdirlink

It's enough to use request.config.rootdir to be able to write something like this:

def pytest_logger_logdirlink(config):
    return 'logs'

How to not write to log but append to the log file.

Hi, i am new to pytest and also python. I am trying out the example you gave.
Example which i following, here

During run my test, logs are always write to my generated log files but not append to end of file. Any how to change to append instead of writing?

Remove dependency on "py" and "future"

Plugin doesn't need them currently:

"py" is in maintenance mode, there are alternatives for all its features
"future" was important for python 2 vs 3 compatibility, now it's useless

Support for logbook

This extension works with python logging module.
Is there a plan to support also logbook (https://logbook.readthedocs.io/en/stable/index.html) logging library?

Option --log error in pytest 3.4

In pytest 3.4 --log option gets confused with pytest-logging options:

$ pytest -s --log 123
usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: ambiguous option: --log could match --log-cli-format, --log-file-date-format, --log-file-level, --log-format, --log-file, --log-cli-level, --log-file-format, --log-level, --log-date-format, --log-cli-date-format

Im using the plugin its really useful. However right now I see the log formats are in the following format which I want to to be formatted as per the dates e.g:
00:00.024 inf testcase: one executes : 1
00:00.024 wrn testcase: this test does nothing aside from logging
00:00.025 inf testcase: extra log, rarely read : 1
PASSED

tests/examples/tests2.py::test_one[2]
00:00.001 inf testcase: one executes : 2
00:00.001 wrn testcase: this test does nothing aside from logging
00:00.001 inf testcase: extra log, rarely read : 2
PASSED

tests/examples/tests2.py::test_one[3]
00:00.001 inf testcase: one executes : 3
00:00.001 wrn testcase: this test does nothing aside from logging
00:00.001 inf testcase: extra log, rarely read : 3
PASSED

tests/examples/tests2.py::test_two
00:00.000 inf testcase: two executes
00:00.000 wrn testcase: neither does this
00:00.000 inf testcase: extra log, not enabled by default
PASSED

Please let me know how I can go about changing the format for the logs.

Thanks.

Missing logdir fixture documentation

There is no docstring:

------------------------------- fixtures defined from pytest_logger.plugin -------------------------------
logdir
    py/local/lib/python2.7/site-packages/pytest_logger/plugin.py:268: no docstring available

Option for root logs dir

currently log files are created in directory based on tmpdir fixture.
It would be nice to be able to define root dir, where it is created in command line / configuration file.

workaround is to use --basetemp, but it could influence other features.

Open logs in append mode

Is it possible to open a logger in append.
I have process which write to the process multiple times but what i observe it that the writes are overridden and only the last write is saved?

Update setup.py

There is some outdated information in the setup.py file:

author's mail
required pytest (I believe it should be at least 3.0 or the oldest one that's used in testing at the time of resolving this issue)
python versions (2.6 is dead, 3.6 and 3.7 are out)

Format from pytest.ini is not used

When I have added a log_format to pytest.ini I would have expected that format to be used instead of pytest-loggers format.
I can't see any info on that in the documentation.

Option --logger-logsdir doesn't work well with pytest-xdist

When test are run in parallel all workers race to recreate given directory (check/remove/mkdir). When second worker does this, it may remove files which first worker holds opened.

This function and it's default argument (which plugin user cannot influence) causes problem:

def _make_logsdir_dir(dstname, cleandir=True):
    logsdir = py.path.local(dstname)
    if cleandir:
        if logsdir.check():
            logsdir.remove()
        logsdir.mkdir()
    return logsdir

symlink on windows not supported

Small bug in implementation, os.symlink is not available on windows.
Unfortunately common solution for all versions deesn't exist. but from vista, it is possible to implement if using ctypes.windll.kernel32.CreateSymbolicLinkW, example here:
http://stackoverflow.com/questions/6260149/os-symlink-support-in-windows

Log redirect

Hello,

First I would like to appreciate your work on creating logger plugin that simplifies the logging job when running in a large test environment.

I'm trying to use your plugin for my test cases logging. As per its design, it works perfectly so this is not a bug report. However, I have a use case that I would like to see if this plugin can support as a new feature.

My use case is for some of the testcase I want to redirect logs to the file other than what default, that is being created by this plugin. Let's take an example.

I'm adding the file logging for "app/testcases/test_app.py", which has two test cases under a class MYTESTS, test_foo and test_bar. Now as per existing plugin features the log files will get created like this.

/tmp/pytest-of-rajiv/logs/app/testcases/test_app/MYTEST.test_foo/pytest_logger
and
/tmp/pytest-of-rajiv/logs/app/testcases/test_app/MYTEST.test_bar/pytest_logger

However, for MYTEST.test_bar I want to redirect the log file to some other location with the different name like this.

/tharnlog/custer1/20180106/test_app/MYTEST.test_bar.log

This something I should be able to control using pytest_test_setup hook based on certain markers.

Currently, the plugin uses nodeid to create the filehandler which will be the by default getting created using full file path from the root directory. There is no option to modify the file path except modifying the --logger-logsdir. Basically, redirect the log file to some other location other than default filehandler.

After going through your code I feel that it is possible to do it by modifying the LoggerState object and pass it to make handler.


class LoggerPlugin(object):
    def __init__(self, config, logcfg):
        self._config = config
        self._logdirlinks = config.hook.pytest_logger_logdirlink(config=config)
        self._loggers = _loggers_from_logcfg(logcfg, config.getoption('log')) if logcfg._enabled else None
        self._formatter_class = logcfg._formatter_class or DefaultFormatter
        self._logsdir = None
        self._redirectpath = None

Modify the state object by influencing this hook. I did the hookwrapper in my conftest to make sure my hook gets called first and I can influence the loggerstate.

    def pytest_runtest_setup(self, item):
        loggers = _choose_loggers(self._loggers, _loggers_from_hooks(item))
        formatter = self._formatter_class()
        item._logger = state = LoggerState(item=item,
                                           stdoutloggers=loggers.stdout,
                                           fileloggers=loggers.file,
                                           formatter=formatter,
                                           redirectpath=self._redirectpath)
        state.on_setup()

class LoggerState(object):
    def __init__(self, item, stdoutloggers, fileloggers, formatter, redirectpath=None):
        self._put_newlines = bool(item.config.option.capture == 'no' and stdoutloggers)
        self.handlers = _make_handlers(stdoutloggers, fileloggers, item, formatter, redirectpath)
        self.root_enabler = RootEnabler(bool(stdoutloggers and fileloggers))

def _make_logdir(item, redirectpath=None):
    plugin = item.config.pluginmanager.getplugin('_logger')
    if (redirectpath):
        return plugin.logsdir().join(redirectpath).ensure(dir=1)
    else:
        return plugin.logsdir().join(_sanitize_nodeid(item.nodeid)).ensure(dir=1)

def _make_handlers(stdoutloggers, fileloggers, item, formatter, redirectpath=None):
    handlers = []
    if stdoutloggers:
        handlers += _make_stdout_handlers(stdoutloggers, formatter)
    if fileloggers:
        logdir = _make_logdir(item, redirectpath)
        handlers += _make_file_handlers(fileloggers, formatter, logdir)
    return handlers

And this is how we can influence this hook from conftest

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_setup(item):

    # Let's get the logger plugin
    logger = item.config.pluginmanager.getplugin('_logger')

    #add a File logger
    testname, filename = get_test_file_name(item.nodeid)

    filename = os.path.splitext(filename)[0]
    testnames = (testname, logging.DEBUG)
    logger._loggers.file = [(testnames)]
    newpath = filename
    logger._redirectpath = newpath

    outcome = yield

I find this use case is very useful and generic because many of the times we want to record the logging to commonplace for specific reasons. For me, the log files are generated from various test systems in a huge test infrastructure setup. That setup is already generating log files with that naming convention. I can write something as post-test teardown that will copy the log file into that location but that will not be real time and slow down my testing.

I would greatly appreciate your input. I'm also ready to contribute under your guidance.

Thanks,
Rajiv Gupta

Encoding for FileHandler

logging.FileHandler has an encoding keyword.

It would be very nice to have control over this. My tests logs are written un iso encoding, while I would like to have them in UTF-8.

I guess this could be done through configuration file.

storing output of failures

I'm using pytest-logger and I have example test:

def test_random():
    import random
    assert random.choice([1,2,3,4]) == 2

Of course this test fails sometimes.
The output of assertion is only printed to stdout/err.

test_random.py F                                                                                                                                                        [100%]

==================================================================================================== FAILURES =====================================================================================================
___________________________________________________________________________________________________ test_random ___________________________________________________________________________________________________
test_random.py:49: in test_random
    assert random.choice([1,2,3,4]) == 2
E   AssertionError: assert 1 == 2
E    +  where 1 = <bound method Random.choice of <random.Random object at 0x28b5a78>>([1, 2, 3, 4])
E    +    where <bound method Random.choice of <random.Random object at 0x28b5a78>> = <module 'random' from '/usr/lib/python3.6/random.py'>.choice
============================================================================================= short test summary info =============================================================================================
FAILED test_random::test_random - AssertionError: assert 1 == 2
======================================================================================== 1 failed, 544 deselected in 1.51s ========================================================================================
ERROR: InvocationError for command .tox/super_test/bin/py.test -n auto -raR --tb=short --sut-name=auto -k test_random -n0 (exited with code 1)
_____________________________________________________________________________________________________ summary _____________________________________________________________________________________________________
ERROR:   super_test: commands failed

Is it possible to store such output in logs for each failed test? How to do that?
This would be very helpful to analyze issues. Currently there are only files created by pytest-logger itself and they do not contain such exceptions.

Disabling root logger prematurely

self.root_enabler.disable should be called on makereport rather than teardown.

log into separate log files bases on log levels when logged into a single logger

Hi, I would like to use a single logger which when used in the scripts should create 2 log files under the test case. For example : 1 log file with all log lines above debug level and one file with all log lines above info level. But, inside the scripts I should be logging only into one logger. Is that possible with the current code ? Or is it a future enhancement ? Thanks for your help

No way to modify formatter

Hi,
Would that be possible to add a way to modify the logging handler/formatter from the LoggerConfig object ?
From the code, I see no way of modifying the formatter without hacking the private self._loggers in LoggerConfig.
Cool plugin by the way !

AttributeError: 'NoneType' object has no attribute '_logcfg'

Possibly due to new pytest versions, or due to interaction with other pytest plugins:

> python3 -m pytest --help
/usr/lib/python3.8/site-packages/pytest_spec/patch.py:168: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if test_name[:1] is ' ':
/usr/lib/python3.8/site-packages/nose/importer.py:12: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  from imp import find_module, load_module, acquire_lock, release_lock
/usr/lib/python3.8/site-packages/pytest_profiling.py:20: DeprecationWarning: invalid escape sequence \:
  forbidden_chars = set('/?<>\:*|"')
/usr/lib/python3.8/site-packages/nbformat/notebooknode.py:4: DeprecationWarning:

Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working

/usr/lib/python3.8/site-packages/pytest_clarity/hints.py:1: DeprecationWarning:

Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working

/usr/lib/python3.8/site-packages/betamax/decorator.py:11: DeprecationWarning:

invalid escape sequence \*

/usr/lib/python3.8/site-packages/html5lib/_trie/datrie.py:3: DeprecationWarning:

Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working

/usr/lib/python3.8/site-packages/django_assets/glob.py:124: DeprecationWarning:

invalid escape sequence \Z

/usr/lib/python3.8/site-packages/pluggy/callers.py:187: DeprecationWarning:

`type` argument to addoption() is the string 'string',  but when supplied should be a type (for example `str` or `int`). (options: ('--template',))

Traceback (most recent call last):
  File "/usr/lib64/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.8/site-packages/pytest/__main__.py", line 7, in <module>
    raise SystemExit(pytest.main())
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 92, in main
    ret = config.hook.pytest_cmdline_main(
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/_pytest/helpconfig.py", line 134, in pytest_cmdline_main
    config._do_configure()
  File "/usr/lib/python3.8/site-packages/_pytest/config/__init__.py", line 778, in _do_configure
    self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
  File "/usr/lib/python3.8/site-packages/pluggy/hooks.py", line 308, in call_historic
    res = self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/usr/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3.8/site-packages/pytest_logger/plugin.py", line 51, in pytest_configure
    config.pluginmanager.register(LoggerPlugin(config, early_logger._logcfg), '_logger')
AttributeError: 'NoneType' object has no attribute '_logcfg'

[help] - log folder and logs file not created

issue : folder log with log file not created

how to reproduce :

windows 10 python 3.8.4 64bit
pytest 5.4.3
pytest-logger 0.5.1
pytest-xdist 1.34
clone this fork https://github.com/andreabisello/pytest-logger.git
take a look at example folder
run pytest -s in example folder

you got

but mylogs folder with logs file is not created.

using pytest-logger with pytest-rerunfailures plugin

Let's have a test:

def test_random():
    import random
    assert random.choice([1,2,3,4]) == 2

It fails randomly.
I'd like to use pytest-rerunfailures to run it copule of times and store log for every rerun if failed.
Now pytest-logger stores only last run, so if finally test passes I do not have logs from reruns.

Do you have any idea how to connect those two plugins so I'll have logs from reruns? Then I could analyze what was wrong in random test