openedx-unsupported / bok-choy Goto Github PK

The tutorial code for testing the search results page has bug in it that prevents it from passing. I'm working on a fix

I'm currently using bok-choy for some tests. I'd like the following:

1. Run tests on different operating systems, versions, etc through a testing service, like SauceLabs or BrowserStack.
2. Pass additional metadata about my test runs that those services use to tag/filter runs in reports.

Normally these are set through desired_capabilities passed to the remote WebDriver, not just the tags field that Sauce Labs uses.

Currently, desired_capabilities is derived from _capabilities_dict. It manually maps environment variables to specific, hardcoded capability keys.

Instead of adding more env vars that must be manually mapped and maintained, how about exposing an other_caps parameter to the browser call that will get merged with the derived capabilities?

To maintain existing env-var-mapping behavior, the derived capabilities would override any passed in capabilities from the user.

Thoughts?

Looking at the code, I believe the change is fairly simple and I can do it myself. Would like to brainstorm it with you all first.

The __init.py__ method of the bok-choy Promise class has a default value for timeout. This means that when you pass a value for try_limit, it will stop when the first of the limits (tries or time) is reached, which could have been at 30 seconds (the default value) rather than after the number of tries that you had intended. :(

If you only specify try_limit I would assume that your intention is to actually use the value you gave it, not a mysterious default timeout.

We should be careful with a fix tho, because if we change the behavior now it could cause existing tests (not just ours, as edX is not the only org using bok-choy) to fail.

@benpatterson @nedbat @clytwynec Thoughts?

tox runs pylint against the bok_choy directory, but not against the tests directory; it should be run against both. It currently reports some issues there, so those should be fixed.

Also, tox should run pylint --py3k on all the code to catch any problems with Python 3 compatibility. This looks like it would also require fixing a few reported issues. Note that enabling the --py3k option disables all the non-porting checks, so this has to be a separate run of pylint.

Right now we are adding the screenshot of bokchoy failed tests in reports. I guess it would be beneficial to the developer if we add the html source to a file of failed test. We should take the html source of page when we are taking the screenshot.

After retrieving a parent element I would like to be able to make queries on the child elements. jQuery and Selenium offer real-world implementation examples.

<div class="parent">
  <div class="child"></div>
  <div class="child"></div>
</div>

page = ...

# Get the parent element
parent = page.q(css='div.parent')

# Get all of the children nested in the parent element
children = parent.q(css='div.child')

# Get the first child element
child = children[0]

# Infinite chaining (yes, the call below should fail since there is no span)
child.q(css='span')

The GitHub search page has changed, and now the tutorial part 2 (and 3 most likely) fails.

The is_browser_on_page method for the search page is returning too quickly, so that the next steps of filling in the search terms and hitting the Search button are not performed successfully.

Perhaps there is too much client-side stuff going on that needs to be waited for and that will be hard to explain in step 2 of the tutorial. Or maybe it's a good representative use case?

Reproducible when running tests with run_tests.sh

From this travis build.

I'm guessing that something changed in the GitHub web app so that one of the css matchers is no longer matching.

====== Run tutorial examples
.
----------------------------------------------------------------------
Ran 1 test in 2.577s
OK
.E
======================================================================
ERROR: test_search (__main__.TestGitHub)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "docs/code/round_2/test_search.py", line 27, in test_search
    self.github_search_page.visit().search_for_terms('user:edx repo:edx-platform')
  File "build/bdist.linux-x86_64/egg/bok_choy/page_object.py", line 64, in wrapper
    return method(self, *args, **kwargs)
  File "/home/travis/build/edx/bok-choy/docs/code/round_2/pages.py", line 51, in search_for_terms
    self.search()
  File "build/bdist.linux-x86_64/egg/bok_choy/page_object.py", line 64, in wrapper
    return method(self, *args, **kwargs)
  File "/home/travis/build/edx/bok-choy/docs/code/round_2/pages.py", line 43, in search
    GitHubSearchResultsPage(self.browser).wait_for_page()
  File "build/bdist.linux-x86_64/egg/bok_choy/page_object.py", line 326, in wait_for_page
    timeout=timeout
  File "build/bdist.linux-x86_64/egg/bok_choy/promise.py", line 96, in fulfill
    raise BrokenPromise(self)
BrokenPromise: Promise not satisfied: loaded page <pages.GitHubSearchResultsPage object at 0x7f40caee9690>
----------------------------------------------------------------------
Ran 2 tests in 37.259s

In the docs, your example classes don't need to use @property for name or page_object_classes. Both of those are static in the example classes, so they can be class properties. If they're dynamic, it's likely that they'll just depend on arguments to __init__, in which case they can be set as instance properties there.

Selenium 2.42.1 doesn't seem to work with the latest version of Firefox (32). Selenium 2.43.0 does.

Are there any plans for updating bok-choy to be Python 3 compatible?

Python 3 is finally seeing major adoption... and I think the following Python versions should be targeted: 2.7, 3.3, 3.4, 3.5

Rather than creating a Python3 fork and maintaining 2 code bases, this would be best done by using a single code base that runs under Python 2 or Python 3. (perhaps using six as a compatibility helper).

If there are no plans for this on the horizon, would a PR contributing Python3 support be considered for merging? (I might have spare cycles to take this on)

URLs that include validation (e.g. https://user:[email protected]/) fail. PageObject.validate_url doesn't account for authentication.

It would be great if we distribute wheels from pypi rather than forcing everybody to recompile the software during every install.

See http://pythonwheels.com/

This is a feature that we had put into bok-choy, but never saw any real adoption at edX because we are now using other tools (mostly. sitespeed.io and devs using browser developer tools directly).

Couple options:

• make a fancy setup.py with an option to specify (e.g. [har]) that you will be enabling that feature and only install those requirements (e.g. browsermobproxy) in that case
• rip out the code entirely. This seems preferable, especially if we want to distribute wheels from pypi.

See element_to_be_clickable expected condition.

Add a promise for this to the click method so that we do not try clicking on the element too early.

‼️ just a friendly PSA:

The existing FirefoxDriver in selenium is deprecated and will cease to work with new versions of Firefox (starting with Firefox 47). This breaks bok-choy's ability to run tests with Firefox. To fix it, an update to bok-choy will be necessary (which may be a bit tricky since the actual use of FirefoxDriver is done through the needle package)

The fix for this is to use GeckoDriver/Marionette. Unlike FirefoxDriver, this component is not bundled inside the selenium package. It's more like Chrome/ChromeDriver, where you install a separate package to use the driver. (note: GeckoDriver used to be called Wires, so you might not find much info yet if you search the interwebz for "GeckoDriver")

btw, Mozilla and the Selenium maintainers have done a really poor job of publicizing and easing this transition... so expect some pain 👊

https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette
https://github.com/mozilla/geckodriver

According to the docs, when trying to do visual diff testing and running bok-choy with pytest, all that is needed to create initial baseline shots is to set the environment variable NEEDLE_SAVE_BASELINE. However this does not work because the code that checks for this environment variable is in the setUpClass method of NeedleTestCase which is skipped by BokChoy's WebAppTest setUpClass method so setting the environment variable has no effect when running test cases and instead throws an error.