Git Product home page Git Product logo

behave-xray's People

Contributors

fundakol avatar kconkas avatar

Stargazers

avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

haseebkhanniazi goldexapp kconkas

behave-xray's Issues

unable to update Scenario Outlines on Jira

I have a test which is a Scenario outline:

e.g.

  Given there are <start> cucumbers
  When I eat <eat> cucumbers
  Then I should have <left> cucumbers

  Examples:
    | start | eat | left |
    |    12 |   5 |    7 |
    |    20 |   5 |   15  |

When I run that as part of a test suite I get the following error:

Uploaded results to JIRA XRAY Test Execution: xx-1234
Exception IndexError: list index out of range
Traceback (most recent call last):
  File "C:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "D:\git\testing\windows-application-framework\venv\Scripts\behave.exe\__main__.py", line 7, in <module>
  File "D:\git\testing\windows-application-framework\venv\lib\site-packages\behave\__main__.py", line 183, in main
    return run_behave(config)
  File "D:\git\testing\windows-application-framework\venv\lib\site-packages\behave\__main__.py", line 127, in run_behave
    failed = runner.run()
  File "D:\git\testing\windows-application-framework\venv\lib\site-packages\behave\runner.py", line 804, in run
    return self.run_with_paths()
  File "D:\git\testing\windows-application-framework\venv\lib\site-packages\behave\runner.py", line 824, in run_with_paths
    return self.run_model()
  File "D:\git\testing\windows-application-framework\venv\lib\site-packages\behave\runner.py", line 626, in run_model
    failed = feature.run(self)
  File "D:\git\testing\windows-application-framework\venv\lib\site-packages\behave\model.py", line 350, in run
    formatter.eof()
  File "D:\git\testing\windows-application-framework\venv\lib\site-packages\behave_xray\formatter.py", line 221, in eof
    self.collect_tests()
  File "D:\git\testing\windows-application-framework\venv\lib\site-packages\behave_xray\formatter.py", line 235, in collect_tests
    testcase.status = self._get_xray_status(tc_status.statuses[0].name)
IndexError: list index out of range

Cannot publish results to xray jira

I have set my system environments (Bearer authentication) for Xray Cloud:

(Yes, i have set them on the pipeline environmental variables)
export XRAY_API_BASE_URL= "https://xray.cloud.getxray.app"
export XRAY_CLIENT_ID=
export XRAY_CLIENT_SECRET=

Using the command "- behave -f allure_behave.formatter:AllureFormatter -o report/allure/results -f pretty --f xray ./features/" I am running the test

And on the Feature file I have added the Test execution at the top followed by test case if for the scenario

Feature file:
@jira.test_execution('TEST-xxx')
Feature: feature name
Background: Common Steps
Given xxx
@jira.testcase('TEST-xxxx')
Scenario: Scenario name

On the pipeline I get below error -
HTTPError: Could not post to JIRA service https://xray.cloud.getxray.app/. Response status code: 404
Exception JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/local/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/behave", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/site-packages/behave/main.py", line 183, in main
return run_behave(config)
File "/usr/local/lib/python3.10/site-packages/behave/main.py", line 127, in run_behave
failed = runner.run()
File "/usr/local/lib/python3.10/site-packages/behave/runner.py", line 804, in run
return self.run_with_paths()
File "/usr/local/lib/python3.10/site-packages/behave/runner.py", line 824, in run_with_paths
return self.run_model()
File "/usr/local/lib/python3.10/site-packages/behave/runner.py", line 626, in run_model
failed = feature.run(self)
File "/usr/local/lib/python3.10/site-packages/behave/model.py", line 350, in run
formatter.eof()
File "/usr/local/lib/python3.10/site-packages/behave_xray/formatter.py", line 127, in eof
self.xray_publisher.publish(self.test_execution.as_dict())
File "/usr/local/lib/python3.10/site-packages/behave_xray/xray_publisher.py", line 51, in publish
result = self.publish_xray_results(self.endpoint_url, self.auth, test_execution)
File "/usr/local/lib/python3.10/site-packages/behave_xray/xray_publisher.py", line 45, in publish_xray_results
_logger.error('Response error: %s', response.json())
File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

XrayCloudFormatter calls Xray Server test execution endpoint

While trying to run an import using the XrayCloudFormatter, I am getting the following exception:

% behave -f behave_xray:XrayCloudFormatter features/1_MYPROJ-1234.feature
HTTPError: Could not post to JIRA service at https://xray.cloud.getxray.app/rest/raven/2.0/import/execution. Response status code: 404
Traceback (most recent call last):
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave_xray-0.0.0-py3.7.egg/behave_xray/xray_publisher.py", line 41, in publish_xray_results
    response.raise_for_status()
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://xray.cloud.getxray.app/rest/raven/2.0/import/execution
Exception JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/bin/behave", line 33, in <module>
    sys.exit(load_entry_point('behave==1.2.6', 'console_scripts', 'behave')())
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave/__main__.py", line 183, in main
    return run_behave(config)
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave/__main__.py", line 127, in run_behave
    failed = runner.run()
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave/runner.py", line 804, in run
    return self.run_with_paths()
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave/runner.py", line 824, in run_with_paths
    return self.run_model()
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave/runner.py", line 626, in run_model
    failed = feature.run(self)
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave/model.py", line 350, in run
    formatter.eof()
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave_xray-0.0.0-py3.7.egg/behave_xray/formatter.py", line 160, in eof
    # This will die of IndexError when counter is too big
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave_xray-0.0.0-py3.7.egg/behave_xray/xray_publisher.py", line 56, in publish
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave_xray-0.0.0-py3.7.egg/behave_xray/xray_publisher.py", line 46, in publish_xray_results
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Support for Xray-managed cucumber tests?

Xray can manage tests in Cucumber format (see the Jira based Cucumber workflow chapter in the documentation and, when exporting them as feature files, adds tags that correspond to Xray scenarios and preconditions.

For example, a test MYPROJ-1604 with precondition PRECOND_MYPROJ-1703 after export becomes:

Feature: Calculator can sum positive integers

	Background:
		#@PRECOND_MYPROJ-1703
		Given I have a calculator

	#My issue comments
	@TEST_MYPROJ-1604 @positive
	Scenario Outline: Calculator can sum positive integers
		When I use my calculator to calculate the sum of <first_integer> and <second_integer>
		Then I see the result is <result>
		Examples:
		    | first_integer | second_integer | result |
		    | 1             | 1              | 2      |
		    | 100           | 5              | 103    |

It would be really neat to be able to run behave on these files and have them seamlessly import into Xray as that'd allow out of the box Xray integration using the recommended workflow.

Feature Request: Evidence Attachment

I've found this library works really well, the only thing it is missing for me is the 'evidence' attachment for a test scenario execution.

image

I think the simplest implementation of this would be an environment variable which points towards a test evidence directory, e.g. XRAY_EVIDENCE_DIR=/test_evidence and all files in this directory would get uploaded.

This way, it can be left to me (the user) to implement a cleanup of the evidence directory between scenarios, this could be quite straightforward with Behave's after_scenario hook.

I've had a quick look at the XRay 2.0 API reference, and there looks like a POST method for adding attachments :-)

Let me know your thoughts, and thanks for making this package - really useful and worked for us straight out the box.

Xray Cloud: unable to create test execution due to missing summary

I have installed behave-xray from source, commit 4164777 and configured my feature files as described in README.md (namely added @jira.test_plan and @jira.testcase). When I try to run behave test with behave_xray:XrayCloudFormatter formatter, I get the following error:

EBUG:urllib3.connectionpool:https://xray.cloud.getxray.app:443 "POST /api/v2/import/execution HTTP/1.1" 400 155
ERROR:behave_xray.xray_publisher:HTTPError: Could not post to JIRA service at https://xray.cloud.getxray.app/api/v2/import/execution. Response status code: 400
Traceback (most recent call last):
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/behave_xray-0.0.0-py3.7.egg/behave_xray/xray_publisher.py", line 41, in publish_xray_results
    response.raise_for_status()
  File "/Users/kristijan.conkas/.pyenv/versions/3.7.13/lib/python3.7/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://xray.cloud.getxray.app/api/v2/import/execution
ERROR:behave_xray.xray_publisher:Error message from server: Error creating Test Execution - Issue create failed! - summary: You must specify a summary of the issue., description: Description is required.
ERROR:behave_xray.xray_publisher:Could not publish results to Jira XRAY
ERROR:behave_xray.xray_publisher:HTTPError: Could not post to JIRA service at https://xray.cloud.getxray.app/api/v2/import/execution. Response status code: 400
Error message from server: Error creating Test Execution - Issue create failed! - summary: You must specify a summary of the issue., description: Description is required.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.