allure-framework / allure-nose Goto Github PK

for the given test_me.py

#!/usr/bin/env python3
 
def yield_int(i):
     assert i < 3
 
def test_me():
     for i in range(0,3):
         yield yield_int, i

running Nose with Allure performs three test cases

# nosetests --with-allure --logdir=allure-results
...
----------------------------------------------------------------------
Ran 3 tests in 0.004s

OK

and if we browse result allure xml for that run we discover three test cases, that's OK

<ns0:test-suite xmlns:ns0="urn:model.allure.qatools.yandex.ru" start="1511096337473" stop="1511096337474">
  <name>test_me</name>
  <labels/>
  <test-cases>
    <test-case start="1511096337473" status="passed" stop="1511096337473">
      <name>test_me.test_me</name>
      <attachments/>
      <labels/>
      <steps/>
    </test-case>
    <test-case start="1511096337473" status="passed" stop="1511096337473">
      <name>test_me.test_me</name>
      <attachments/>
      <labels/>
      <steps/>
    </test-case>
    <test-case start="1511096337473" status="passed" stop="1511096337473">
      <name>test_me.test_me</name>
      <attachments/>
      <labels/>
      <steps/>
    </test-case>
  </test-cases>
</ns0:test-suite>

but as you can see, all <name> tags in each <test-case> of the report are the same for all yielded tests and if thus if we run

# allure serve allure-results

we would see only one test case, that's NOT OK.

Probably, that's a problem of Nose, is there any workaround for that issue?

Hi EveryOne,

I have suspected the bug in the pytest-allure-adaptor-1.7.10

The problem is: 1. when I install: pip install pytest-allure-adaptor it is Generate a report without an image 2. If I uninstall: pip uninstall pytest-allure-adaptor the version gives me this message TypeError: Object of type Step is not JSON serializable

Note I implements the code on Behave framework def before_step(context,step="Step Name"): attach_png((context.web._web_driver).get_screenshot_as_png(),step)

please help what to do ?

Pre-conditions:
Nose + Nose Allure Adaptor installed. Use Allure annotations/decorators like this:

with nose.allure.step('Clearing login field'): self.login_page.get_login_field().clear()

@nose.allure.step('Clicking Forgot Password link') def click_forgot_password_link(self): self.get_forgot_password_link().click()

Steps to reproduce:

1. Install multiprocessing: pip install multiprocessing
2. Run tests with key --processes=N

Expected result

Tests should run in N threads

Actual result

Failure:

File "/usr/local/lib/python2.7/site-packages/allure/common.py", line 56, in impl
with StepContext(self.allure, self.title.format(_a, *_kw)):
File "/usr/local/lib/python2.7/site-packages/allure/common.py", line 34, in enter
self.step = self.allure.start_step(self.title)
File "/usr/local/lib/python2.7/site-packages/allure/common.py", line 128, in start_step
self.stack[-1].steps.append(step)
IndexError: list index out of range

After i launch my tests via nose and get first failure (in my case assertion error 500 status code instead of 200), i've got attribute error. Here's the log:

[Step 2/2] Status code - 500
[18:29:25][Step 2/2] .Traceback (most recent call last):
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/case.py", line 134, in run
[18:29:25][Step 2/2] self.runTest(result)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/case.py", line 152, in runTest
[18:29:25][Step 2/2] test(result)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 625, in call
[18:29:25][Step 2/2] return self.run(_args, *_kwds)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 585, in run
[18:29:25][Step 2/2] self._feedErrorsToResult(result, outcome.errors)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 516, in _feedErrorsToResult
[18:29:25][Step 2/2] result.addFailure(test, exc_info)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/proxy.py", line 146, in addFailure
[18:29:25][Step 2/2] plugins.addFailure(self.test, err)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/plugins/manager.py", line 99, in call
[18:29:25][Step 2/2] return self.call(_arg, *_kw)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/plugins/manager.py", line 167, in simple
[18:29:25][Step 2/2] result = meth(_arg, *_kw)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose_allure/init.py", line 26, in wrapper
[18:29:25][Step 2/2] return func(self, _args, *_kwargs)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose_allure/init.py", line 134, in addFailure
[18:29:25][Step 2/2] message, trace = self._parse_tb(err)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose_allure/init.py", line 152, in _parse_tb
[18:29:25][Step 2/2] trace = ''.join(traceback.format_exception(*trace)).strip()
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 181, in format_exception
[18:29:25][Step 2/2] return list(_format_exception_iter(etype, value, tb, limit, chain))
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 146, in _format_exception_iter
[18:29:25][Step 2/2] for value, tb in values:
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 125, in _iter_chain
[18:29:25][Step 2/2] context = exc.context
[18:29:25][Step 2/2] AttributeError: 'str' object has no attribute 'context'
[18:29:25][Step 2/2]
[18:29:25][Step 2/2] During handling of the above exception, another exception occurred:
[18:29:25][Step 2/2]
[18:29:25][Step 2/2] Traceback (most recent call last):
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/bin/nosetests", line 11, in
[18:29:25][Step 2/2] sys.exit(run_exit())
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/core.py", line 121, in init
[18:29:25][Step 2/2] *_extra_args)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/main.py", line 93, in init
[18:29:25][Step 2/2] self.runTests()
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/core.py", line 207, in runTests
[18:29:25][Step 2/2] result = self.testRunner.run(self.test)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/core.py", line 62, in run
[18:29:25][Step 2/2] test(result)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/suite.py", line 178, in call
[18:29:25][Step 2/2] return self.run(_arg, *_kw)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/suite.py", line 225, in run
[18:29:25][Step 2/2] test(orig)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/suite.py", line 178, in call
[18:29:25][Step 2/2] return self.run(_arg, *_kw)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/suite.py", line 225, in run
[18:29:25][Step 2/2] test(orig)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/suite.py", line 178, in call
[18:29:25][Step 2/2] return self.run(_arg, *_kw)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/suite.py", line 225, in run
[18:29:25][Step 2/2] test(orig)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/suite.py", line 178, in call
[18:29:25][Step 2/2] return self.run(_arg, *_kw)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/suite.py", line 225, in run
[18:29:25][Step 2/2] test(orig)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/case.py", line 46, in call
[18:29:25][Step 2/2] return self.run(_arg, *_kwarg)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/case.py", line 139, in run
[18:29:25][Step 2/2] result.addError(self, err)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/proxy.py", line 131, in addError
[18:29:25][Step 2/2] plugins.addError(self.test, err)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/plugins/manager.py", line 99, in call
[18:29:25][Step 2/2] return self.call(_arg, *_kw)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose/plugins/manager.py", line 167, in simple
[18:29:25][Step 2/2] result = meth(_arg, *_kw)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose_allure/init.py", line 26, in wrapper
[18:29:25][Step 2/2] return func(self, *args, *_kwargs)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose_allure/init.py", line 129, in addError
[18:29:25][Step 2/2] message, trace = self._parse_tb(err)
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/nose_allure/init.py", line 152, in _parse_tb
[18:29:25][Step 2/2] trace = ''.join(traceback.format_exception(*trace)).strip()
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 181, in format_exception
[18:29:25][Step 2/2] return list(_format_exception_iter(etype, value, tb, limit, chain))
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 146, in _format_exception_iter
[18:29:25][Step 2/2] for value, tb in values:
[18:29:25][Step 2/2] File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/traceback.py", line 125, in _iter_chain
[18:29:25][Step 2/2] context = exc.context
[18:29:25][Step 2/2] AttributeError: 'str' object has no attribute 'context'
[18:29:25][Step 2/2] Process exited with code 1

Seems to be possible in the allure-core but not in nose adaptator

have installed the "nose-allure-plugin" adapter & able to generate the xml using " --with-allure" flag but what plugin dependency is required to get the allure html report?
do I need write my own xml->html parser ?
with TestNg, it is direct in this example but does same exists for nose ?

It would be great to have a configuration allowing to change allure-suite definition strategy from 'by module' into 'by package'.

Often there is only one test inside .py file and in that case allure-suite = py module = allure-test -> no structure.

What about django? how can I run my tests from django and jenkins in order to have all the reports in allure?. Could you provide an example or link please?

for given test_steps.py

#!/usr/bin/env python3

import allure

def test_me():
    with allure.step('step that passed'):
        assert 1 == 1
    with allure.step('step that failed'):
        assert 1 == 0

if we run it with py.test and allure

# py.test --alluredir=pytest-allure test_steps.py

then, correct .xml with two steps will be generated, and they properly show up in the report

however, if we run it with nose and allure

# nosetests --with-allure --logdir=nose-allure test_steps.py

then no steps will be captured.

My tests are organized to extend unittest.TestCase, so my tests classes are like this:

class TestLogin(MySetup):
    def test_LogIn_1(self):

I can run such tests both from debug (for example right click and select debug 'test_LogIn_1') and from command line with nose

i added allure decorators

import nose

class TestLogin(MySetup):
    @nose.allure.feature('Logging in')
    @nose.allure.story('As admin I can login in with valid data')
    def test_LogIn_1(self):

i can run such tests only from command line with nose and --with-allure, but when i try to debug, im getting error:

AttributeError: 'module' object has no attribute 'allure'

i checked it, and indeed there is no nose.allure, only \site-packages\nose_allure, \site-packages\allure, but no allure in site-packages\nose

now, to be able to debug, i need to comment out any allure decorator, which is unacceptable

edit

I'm able to workaround with some ugly hack. In file were class MySetup is, i added:

try:
    feature = nose.allure.feature
    story = nose.allure.story
    issue = nose.allure.issue
except (AttributeError, ImportError):
    def options(value):
        def decorator(target):
            log.debug(value)
            return target
        return decorator
    feature = options
    story = options
    issue = options

then in my tests files i import

from my_setup import MySetup, feature, story

and i decorate tests with @feature and @story, instead of @nose.allure.feature and @nose.allure.story

nosetests -d -v --with-allure --logdir=allure-results
/Library/Python/2.7/site-packages/nose/plugins/manager.py:395: RuntimeWarning: Unable to load plugin allure = nose_allure:Allure: cannot import name python_2_unicode_compatible
  RuntimeWarning)
Usage: nosetests [options]

nosetests: error: no such option: --with-allure

I checked allure-framework/allure-python#57 and dependencies seems correct

pip show nose nose-allure-plugin pytest-allure-adaptor lxml pytest namedlist six enum34

---
Name: nose
Version: 1.3.4
Location: /Library/Python/2.7/site-packages
Requires:

---
Name: nose-allure-plugin
Version: 1.0
Location: /Library/Python/2.7/site-packages
Requires: nose, pytest-allure-adaptor

---
Name: pytest-allure-adaptor
Version: 1.5.4
Location: /Library/Python/2.7/site-packages
Requires: lxml, pytest, namedlist, six, enum34

---
Name: lxml
Version: 3.4.2
Location: /Library/Python/2.7/site-packages
Requires:

---
Name: pytest
Version: 2.6.4
Location: /Library/Python/2.7/site-packages
Requires: py

---
Name: namedlist
Version: 1.6
Location: /Library/Python/2.7/site-packages
Requires:

---
Name: six
Version: 1.9.0
Location: /Library/Python/2.7/site-packages
Requires:

---
Name: enum34
Version: 1.0.4
Location: /Library/Python/2.7/site-packages
Requires:

I have written below sample code for setup_module with allure.step, But it did not worked.

import logging
import nose

logger = logging.getLogger("TC")


@nose.allure.step("setup module")
def setup_module():
    a = 1 + 10;
    globals().update(locals())
    logger.debug("a=" + str(a))


def teardown_module():
    with nose.allure.step("multiply"):
        b = globals().get('a') * 1000;
        logger.debug("b=" + str(b))


def test_multiply():
    with nose.allure.step("caller"):
        logger.debug(globals().get('a'))

Expected output

allure step should be able to add into setup_module function.

Inherits allure-framework/allure-python#8
Tests were executed in single process.

$nosetests --verbose -d --with-allure --logdir=allure-results --with-xunit --xunit-file=report.xml stories
Traceback (most recent call last):
  File "/usr/local/bin/nosetests", line 9, in <module>
    load_entry_point('nose==1.3.4', 'console_scripts', 'nosetests')()
  File "/usr/local/lib/python2.7/dist-packages/nose/core.py", line 121, in __init__
    **extra_args)
  File "/usr/lib/python2.7/unittest/main.py", line 95, in __init__
    self.runTests()
  File "/usr/local/lib/python2.7/dist-packages/nose/core.py", line 207, in runTests
    result = self.testRunner.run(self.test)
  File "/usr/local/lib/python2.7/dist-packages/nose/core.py", line 62, in run
    test(result)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 177, in __call__
    return self.run(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 224, in run
    test(orig)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 177, in __call__
    return self.run(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 224, in run
    test(orig)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 177, in __call__
    return self.run(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 224, in run
    test(orig)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 177, in __call__
    return self.run(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 224, in run
    test(orig)
  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 45, in __call__
    return self.run(*arg, **kwarg)
  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 138, in run
    result.addError(self, err)
  File "/usr/local/lib/python2.7/dist-packages/nose/proxy.py", line 131, in addError
    plugins.addError(self.test, err)
  File "/usr/local/lib/python2.7/dist-packages/nose/plugins/manager.py", line 99, in __call__
    return self.call(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose/plugins/manager.py", line 167, in simple
    result = meth(*arg, **kw)
  File "/usr/local/lib/python2.7/dist-packages/nose_allure/__init__.py", line 70, in addError
    self.allure.impl.stop_case(Status.BROKEN, message=message, trace=trace)
  File "/usr/local/lib/python2.7/dist-packages/allure/common.py", line 161, in stop_case
    test = self.stack[-1]
IndexError: list index out of range

Hi,

Is there a way to specify a default severity for all tests and/or specify the default severity in the class tests ? I tried to put the decorator at the class level but it does not seem to have impact when I generate the allure report.

Thanks

steps

create python packages structure like

root
    __init__.py
    tests
           __init__.py
           test.py

wiht test.py like this

import nose

def test():
    with nose.allure.step("Step"):
        assert True

after test run, two xml with similar content are created.

suggestion

It looks like a problem is in this place of nose plugin

    def stopContext(self, context):
        if isinstance(context, ModuleType):
            self.allure.impl.stop_suite()

stop_suite is called for each module in hierarchy.

I think the following code should manage this problem

    def stopContext(self, context):
        if isinstance(context, ModuleType) and self.allure.impl.testsuite is not None:
            self.allure.impl.stop_suite()
            self.allure.impl.testsuite = None

When running nose programatically like this:
nose.run(suite=suite, argv=[sys.argv[0], '--with-allure'], plugins=[Allure()] )

Allure plugin fails and crashes test run with this exception

File "/Users/ip/Library/Python/2.7/lib/python/site-packages/nose_allure/__init__.py", line 81, in configure if options.attr: AttributeError: Values instance has no attribute 'attr'