lemoncheesecake / lemoncheesecake Goto Github PK
View Code? Open in Web Editor NEWPython framework for end-to-end / QA testing
Home Page: http://lemoncheesecake.io
License: Other
Python framework for end-to-end / QA testing
Home Page: http://lemoncheesecake.io
License: Other
Good time of the day!
I need to test python library with async/await functional api.
How i can write test in which async methods are use?
e.g. if I make 'setup' or 'teardown' functions asynchronous, test not run, because 'setup' function return coroutine object to lcc; I think with test functions it will be the same.
Maybe you can explain for me, how can I use awaitable objects in lcc tests.
Thanks in advance
Hi,
When I am trying to run my lcc test suites through command "lcc run", it is showing me Attribute Error " AttributeError: 'InjectedFixture' object has no attribute 'get".
Fixture.py
@lcc.fixture(names=("driver", "driver_object"), scope="test")
def setup():
lcc.log_info("Initialising the webdriver object, opening the browser...")
if headless == "yes":
options = Options()
options.add_argument('--headless')
options.add_argument("window-size=1920,1080 ")
options.add_argument('--no-sandbox')
driver = webdriver.Chrome(
ChromeDriverManager().install(),
chrome_options=options)
logging.info(
"Chrome driver has been initialised successfully in headless mode")
else:
driver = webdriver.Chrome(ChromeDriverManager().install())
logging.info("Chrome driver has been initialised successfully")
driver.implicitly_wait(15)
driver.maximize_window()
driver.implicitly_wait(10)
driver.get(url)
print(url)
yield driver
driver.close()
driver.quit()
test.py
@lcc.suite("Sample test", rank=1)
class sample_test(Screenshot):
driver = lcc.inject_fixture("driver_object")
@lcc.test("Create sample test")
def create_sample_test(self):
url = fixture.url
self.driver.get(url)
I am trying to run a test with junit reporter but getting an error mentioning "unknown reporting backend junit"
I have tried command mentioned on the documentation page
http://docs.lemoncheesecake.io/en/latest/cli.html#cmdoption-reporting
System information:
Python: 3.10.2
LCC: 1.12.0
What is going wrong?
ENV: Win 10 + Python 3.5
just a simple code
import lemoncheesecake.api as lcc
from lemoncheesecake.matching import *
SUITE ={u"some page"
@lcc.test(u"page")
def my_test():
check_that(u"page","foo", equal_to("zoo"))
But got the error info after running "lcc run" in project folder
Traceback (most recent call last): File "c:\users\test\appdata\local\programs\python\python35\lib\runpy.py", line 184, in _run_module_as_main "__main__", mod_spec) File "c:\users\test\appdata\local\programs\python\python35\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "C:\Users\test\AppData\Local\Programs\Python\Python35\Scripts\lcc.exe\__main__.py", line 9, in <module> File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\cli\main.py", line 37, in main return command.run_cmd(cli_args) File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\cli\commands\run.py", line 181, in run_cmd return run_project(load_project(), cli_args) File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\cli\commands\run.py", line 50, in run_project suites = get_suites_from_project(project, cli_args) File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\cli\utils.py", line 34, in get_suites_from_project suites = project.get_suites() File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\project.py", line 145, in get_suites suites = self._config.get_suites() File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\project.py", line 103, in get_suites return load_suites_from_directory(self._suites_dir) File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\suite\loader.py", line 253, in load_suites_from_directory suite = load_suite_from_file(filename) File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\suite\loader.py", line 205, in load_suite_from_file suite = load_suite_from_module(mod) File "c:\users\test\appdata\local\programs\python\python35\lib\site-packages\lemoncheesecake\suite\loader.py", line 155, in load_suite_from_module suite_condition = suite_info.get("conditional") AttributeError: 'set' object has no attribute 'get'
Hi Everyone,
I have noticed that teardown_suite hooks method doesn't take any parameters whereas the setup_suite method does allow it for example a selenium webdriver instance.
if we try to pass the parameter we are presented with below error : TypeError: teardown_suite() missing 1 required positional argument: 'driver'
fixture.py -->
import os
import logging
from selenium import webdriver
import lemoncheesecake.api as lcc
from webdriver_manager.chrome import ChromeDriverManager
@lcc.fixture(names=("driver", "driver_obj"), scope="session")
def setup():
driver = webdriver.Chrome(ChromeDriverManager(path=os.environ['PYTHONPATH']).install())
logging.info("Webdriver has been initialised successfully")
driver.maximize_window()
yield driver
lcc.log_info("Closing the browser window...")
driver.close()
driver.quit()
Testsuite.py -->
import lemoncheesecake.api as lcc
from webdriver_manager import driver
def setup_suite(driver):
driver.get('https://www.google.com')
@lcc.test("Verify test")
def test_user_suite(driver):
lcc.log_info('user name test')
def teardown_suite(driver):
lcc.log_info(driver)
user_logout(driver)
For instance the above snippet requires the driver for a user defined function inside the teardown method
I'm not sure what the intended behavior for this but it would be nice to be able to define setup_test and teardown_test methods that run for all tests in a suite including tests defined in inner classes.
MacOS 10.15.3
Python 3.7.7
lemoncheesecake 1.5.0
lcc bootstrap somedir
create file suites/demo.py:
import lemoncheesecake.api as lcc
@lcc.suite('Demo')
class demo:
def setup_test(self, test):
lcc.log_info('outer setup_test')
def teardown_test(self, test, status):
lcc.log_info('outer teardown_test')
@lcc.suite('inner')
class inner:
def setup_test(self, test):
lcc.log_info('inner setup_test')
def teardown_test(self, test, status):
lcc.log_info('inner teardown_test')
@lcc.test('something')
def something(self):
pass
run with
lcc run
Both the inner and outer setup_test and teardown_test methods should run.
Only the inner setup_test and teardown_test methods run.
Running lcc run
with the --reporting [REPORTING]
argument does not run the test anymore, after v1.0.0
For example, lcc run --reporting junit
just creates a junit report of the last test run under report directory, as opposed to running a test and saving the html, js, and xml reports under the same, like it did earlier with --enable-reporting junit
.
Am I missing something?
Hi everyone,
Is there a way to pass the dependency test by name without the id ?
We have a use case where the list of parametrized can be dynamic:
PARAMETERS = (
{"profile": "profile1"},
{"profile": "profile2"},
{"profile": "profile3"},
....
{"profile": "profileN"}
)
Then we declare a test:
@lcc.test("First test")
@lcc.parametrized(PARAMETERS)
def test_a_lot_of_stuff(profile):
# Do Test
And when we need a test who depends on this test, we have to declare all the dependencies but it is dynamic, so we fixed that with an ugly trick:
@lcc.test("Second test")
@lcc.depends_on(*["test_a_lot_of_stuff_" + str(index + 1) for (index, _) in enumerate(PARAMETERS)])
@lcc.parametrized(PARAMETERS)
def test_other_stuff(profile):
# Do Test
We did not find a way to declare dependencies like:
@lcc.depends_on("test_a_lot_of_stuff_*")
It would be great if there could be an option to tag links for one or more bugs in the test report so that the person checking out the report learns about the bugs associated with each and every test failure in a particular suite.
Hi,
I am trying to run my lcc test cases in parallel so as to minimize execution time of test suites. When I am hitting command "lcc run --thread 5" then it is showing me "Connection pool is full, discarding connection: 127.0.0.1" after 3-4 test case.
I thought I should decrease no of thread then I put it as "lcc run --thread 2" but still I am getting same connection pool error.
My test cases are running on same web browser instance and on single web browser which is causing one more issue when I am using threading on my test suites. Issue arise here is one thread is trying to execute one test case and before completion of this running test case another thread starting execution of another test case which leads to failure of 1st running test case.
Is there any other way to minimize test execute time? Is @lcc.Thread will help in this as I have not used @lcc.thread in my test suite yet?
fixture.py file
@lcc.fixture(names=("driver", "driver_obj"), scope="session")
def setup():
lcc.log_info("Initialising the webdriver object, opening the browser...")
# Initialise the global webdriver, open the browser and maximise the
# browser window
if headless == "yes":
options = Options()
options.add_argument('--headless')
options.add_argument("window-size=1920,1080 ")
options.add_argument('--no-sandbox')
driver = webdriver.Chrome(
ChromeDriverManager().install(),
chrome_options=options)
logging.info(
"Chrome driver has been initialised successfully in headless mode")
else:
driver = webdriver.Chrome(ChromeDriverManager().install())
#caps = DesiredCapabilities.FIREFOX.copy()
#driver = webdriver.Firefox(executable_path=GeckoDriverManager().install(), capabilities=caps)
logging.info("Chrome driver has been initialised successfully")
driver.implicitly_wait(15)
driver.maximize_window()
driver.implicitly_wait(10)
driver.get(url)
print(url)
# the global driver object can be used globally in the tests.
yield driver
lcc.log_info("Closing the browser window...")
driver.close()
driver.quit()
Hi,
I archived the build artifacts as 'report.html' and 'report.js' after running the tests on Jenkins. So, I am trying to run the tests on a docker container and then I archive the two files in Jenkins workspace. When I try to click on the link which reads report.html, it does not pick up the content from report.js and display the required results. This is what it looks like:
report.html -> shows a blank page
report.js -> shows the expected json output.
But, on local machine when we open the report.html in a browser it picks up the values from json and shows the relevant report.
Can you share if there is a way that html report needs to be generated so that jenkins displays them normally and also is able to publish it using the "Publish HTML report" plugin.
x-special/nautilus-clipboard
copy
file:///home/anarang/Pictures/Screenshot%20from%202020-03-24%2000-44-55.png
x-special/nautilus-clipboard
copy
file:///home/anarang/Pictures/Screenshot%20from%202020-03-24%2000-51-50.png
Hi,
I am trying to run the following setup before any of the tests start to run but getting an error:
`
import lemoncheesecake as lcc
import os
from git import Repo
@lcc.fixture(scope="pre_run")
def setup_test_repo():
repo = Repo()
repo.clone_from("", "test-repo/")
os.chdir("test-repo/")
os.system('<perform steps')
When trying to run 'lcc run
. I am getting the error:
File "<path>/fixtures/fixture.py", line 6, in <module> @lcc.fixture(scope="pre_run") TypeError: 'module' object is not callable
Can you please help me fix this or guide me how to add a setup step which runs only at the start and only once.
I have defined a few tests in my suite with the suite being defined as a module. The tests are executed in a proper order, but they appear in a different order in the test report. It seems like the skipped tests are appearing first right after setup_suite()
.
lemoncheesecake version: 0.19.3
Hi,
I noticed that the rank numbers aren't working for me, the tests execute randomly and are not following the rank number as in the suite description.
@lcc.suite(description="Suite: Verify search functionality", rank=2)
class test_search:
I also tried with rank="1" and that throws an exception which I think is valid:
File "/home/<username>/.virtualenvs/api-test-env/lib/python3.8/site-packages/lemoncheesecake/suite/loader.py", line 325, in load_suites_from_directory
return sorted(sorted(filter(lambda s: not s.is_empty(), suites.values()), key=lambda s: s.name), key=lambda s: s.rank)
TypeError: '<' not supported between instances of 'str' and 'int'
however in one of the other projects I have been using like this:
@lcc.suite("Suite: Create a new product and versions", rank="2")
and that works.
$ lcc --version
lemoncheesecake version 1.5.2 (using Python 3.8.5 - /home/anarang/.virtualenvs/api-test-env/bin/python)
@ndelon Can you share what I might be missing here?
Hi,
I think I screwed up my python installation.
I updated all my pip packages and now I'm getting this error.
Caught unexpected exception while running test: Traceback (most recent call last): File "/home/josepita/.local/lib/python2.7/site-packages/lemoncheesecake/runner.py", line 223, in run_test test.callback(**test_func_params) File "/home/josepita/lccTests/myproject/suites/auth.py", line 21, in login lcc.log_info(resp.status_code) File "/home/josepita/.local/lib/python2.7/site-packages/lemoncheesecake/runtime.py", line 124, in log_info get_runtime().log(LOG_LEVEL_INFO, content) File "/home/josepita/.local/lib/python2.7/site-packages/lemoncheesecake/runtime.py", line 65, in log events.fire("on_log", level, content) File "/home/josepita/.local/lib/python2.7/site-packages/lemoncheesecake/events.py", line 107, in fire self._call_event_type(event_type_name, lambda et: et.fire(*args, **kwargs)) File "/home/josepita/.local/lib/python2.7/site-packages/lemoncheesecake/events.py", line 94, in _call_event_type callback(event_type) File "/home/josepita/.local/lib/python2.7/site-packages/lemoncheesecake/events.py", line 107, in <lambda> self._call_event_type(event_type_name, lambda et: et.fire(*args, **kwargs)) File "/home/josepita/.local/lib/python2.7/site-packages/lemoncheesecake/events.py", line 62, in fire self._event_type_name, event_arg, i+1, type(handler_arg) MismatchingEventArguments: For event type 'on_log', expecting type '<type 'basestring'>' for argument #2, got '<type 'int'>' 14:21:51.304
this is the code I'm trying to run
`import requests
import lemoncheesecake.api as lcc
email = "jpita+"
password = ""
base_URL = ""
expose_url = ""
@lcc.suite("Auth tests")
class Auth:
@lcc.test("Test login")
def login(self):
lcc.set_step("Get token")
lcc.log_info("GET %s" % base_URL)
resp = requests.get(base_URL + "/auth/login", data={"email": email, "password": password})
lcc.log_info(resp.status_code)
print(resp.status_code)`
according to the logs I'm using python2 but while writing code I have it set for python3 on pycharm, maybe related?
how do I run lcc with python 3?
Hi guys ๐,
Is there a way (or I'm doing something wrong) to have a common package with methods that different suites or tests can use. For example create a randomized payload.
My current set up is as follows:
.
โโโ common
โย ย โโโ utils.py
โย ย โโโ__init__.py
โโโ fixtures
โโโ input_data
โโโ project.py
โโโ suites
but a ModuleNotFoundError occurs while importing the package. I also tried moving common to suites directory and making the import in Project pre_run() but both fails. My current workaround is a symlink to the venv.
Is there a way to achieve this or is it impossible?
I am using LCC framework, I have created different folder inside test suite.. but for some reason , the test file present inside the folder is not running by lcc run
command. any one know, where I can provide the path for same ?
I tried to add path project.py.
project_dir = os.path.dirname(__file__)
project = Project(project_dir)
suites_dir=os.path.join(project_dir, "suites/**")
Hi,
I am looking for an env variable that can print the name of the current test, like this:
test_create_product.create_product_blank_name
I tried lcc.test.name but that doesn't seem to be working.
I am looking at using that while creating the filename for screenshots for a particular test as below:
name = $current_test_name
now = datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
filename_ss = name + now
Looking forward to your help.
Thanks :)
MacOS 10.15.3
Python 3.6.4
lemoncheesecake 1.4.5
lcc bootstrap somedir
create file suites/demo.py:
import lemoncheesecake.api as lcc
@lcc.suite('Demo')
class demo:
def teardown_suite(self):
raise RuntimeError('๐คฎ')
@lcc.test('something')
def something(self):
pass
run with
lcc run --exit-error-on-failure; echo $?
nonzero exit code and some indication that the teardown failed
==================================== demo =====================================
OK 1 # demo.something
Statistics :
* Duration: 0.004s
* Tests: 1
* Successes: 1 (100%)
* Failures: 0
0
The failure only shows up in the html report:
Caught unexpected exception while running test: Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/lemoncheesecake/runner.py", line 87, in run_teardown_funcs
teardown_func()
File "/Users/chris/projects/other/lcc-bug/suites/demo.py", line 6, in teardown_suite
raise RuntimeError('๐คฎ')
RuntimeError: ๐คฎ
Cool test framework. I really like the html report.
Minor problem: the setup and teardown still run when a suite is disabled. The scenario is I have a test suite that is working locally but the setup part doesn't run in our CI environment yet so I need to disable the suite.
MacOS 10.15.3
Python 3.7.7
lemoncheesecake 1.4.7
lcc bootstrap somedir
create file suites/demo.py:
import lemoncheesecake.api as lcc
@lcc.suite('Demo')
@lcc.disabled()
class demo:
def setup_suite(self):
raise RuntimeError('๐คฎ')
def teardown_suite(self):
raise RuntimeError('๐ฅ')
@lcc.test('something')
def something(self):
pass
run with
lcc run --exit-error-on-failure; echo $?
Setup and teardown should not run if the entire suite is disabled.
==================================== demo =====================================
-- 1 # demo.something
Statistics :
* Duration: 0.004s
* Tests: 1
* Successes: 0 (0%)
* Failures: 0
* Disabled: 1
1
The failure shows up in the html report.
For comparison, if I disable a test suite in mocha (the describe block) the before hook doesn't run. It does run if I have the describe block enabled but every test case individually disabled.
describe.skip('', () => {
before(() => {
console.log('in before');
});
it('', () => {
console.log('in test case');
});
});
Hi,
I wanted to know if there a known way to implement 'capture a screenshot when a check fails or error occurs' using lemoncheesecake and selenium webdriver.
Thanks.
Please refer the below error
lemoncheesecake[junit,xml]==1.2.0
18:06:45 return wrap_text(description, int((self.max_width - self._table_overhead) * 0.75))
18:06:45 File "/var/lib/jenkins/workspace/cms-qa/cms-translation-automation/venv/local/lib/python2.7/site-packages/lemoncheesecake/helpers/text.py", line 21, in wrap_text
18:06:45 textwrap.wrap(line, width) for line in text.split("\n")
18:06:45 AttributeError: 'bool' object has no attribute 'split'
Hi,
just moved to a brand new mac and, after installing with pip, it says it doesn't know the command.
any ideas?
Hi,
I wanted to know is there any way that I can capture screenshot of fail test cases if I am using "per_thread" mechanism and also fixture is having scope as "session"?
Hi,
I have some testes written in lcc and I want the xml/junit result format so that I can push them further.
I follow this doc to run the tests :
http://docs.lemoncheesecake.io/en/latest/cli.html
But I am having some issues it's giving error:
Unknown reporting backend
Here are the commands I tried:
(project_env) [vipikuma@kvy ]$ lcc run --reporting +xml
Unknown reporting backend 'xml'
(project_env) [vipikuma@kvy ]$ lcc run --reporting +junit
Unknown reporting backend 'junit'
(project_env) [vipikuma@kvy ]$ lcc --version
lemoncheesecake version 1.11.2 (using Python 3.9.7 - ....../project_env/bin/python3)
So can you help me with the same?
I understand the difference which is mentioned in the docs but I still had a few questions:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.