angular-seattle / blue-harvest Goto Github PK

License: MIT License

TypeScript 46.58% JavaScript 53.08% Shell 0.34%

blue-harvest's Introduction

blue-harvest

Install

You can npm to install the package: npm install blue-harvest --save-dev, or if you are using Yarn, yarn add blue-harvest -D

Writing Tests

Configurations

In order for you to use async await in the tests, the following line

SELENIUM_PROMISE_MANAGER: false;

should be added to the protractor.conf.js file.

To regulate the resolution of the screenshots, add

onPrepare() {
    browser.driver.manage().window().setSize(1366, 1024);
    jasmine.getEnv().addReporter(new SpecReporter({ spec: { displayStacktrace: true } }));
  }

in protractor.conf.js, inside the config block.

Comparing Screenshots

To test if a page has changed from a previous screenshot:

let result =  await blueharvest.compareScreenshot(
                   await browser.takeScreenShot(), pathToGolden);

The difference between the two screenshots will be highlighted in pink.

Updating Reference Screenshots

When a change in a page is expected and not a regression, you can update the reference screenshots by setting UPDATE_GOLDENS=1 when you run the tests. In this case, if any test fails, blue-harvest will update the reference image with the new one. Only tests involving the reference screenshots (goldens) are required. You can edit the specs in protractor.conf.js so only necessary tests are run in an update.

In a Linux system, you can simply run UPDATE_GOLDENS=1 npm run e2e to update the reference screenshots. In a windows environment where the command cannot be directly run, you can put it in a shell file and add a npm script that runs bash ./filename.sh to update the reference screenshots.

Adding Masks

If there is a part on a page that should be ignored in the screenshot tests, you can add a coloured mask to a certain element before you take the screenshot so it will not be compared each time the tests run. addMask() returns the mask element, which can be passed to removeMask() to be removed.

const e = element(by.css('element_to_ignore'));
const mask = await blueharvest.addMask(e, 'gray', 99999, 10, 20, 1.1);
const screenshot = await browser.takeScreenShot();
blueharvest.removeMask(mask);

The default z-index for the mask element is 10000. An optional zIndex argument can be passed to addMask() if you would like to increase the value of a mask's z-index. If the mask appears to be off the desired place, you can pass optional xOffset, yOffset and sizeMultiplier arguments to addMask() to move/shape the mask manually.

blue-harvest's People

Contributors

Stargazers

Watchers

Forkers

cnishina heathkit jacobjojan rpd10 pacobz copperleaftech vishalc99 jlewiscode rupamroy awang-copperleaf hunglethanh9 rsheasby evilweed mexicanace henry-alakazhang iso13 deshorsley

blue-harvest's Issues

Error executing JavaScript error running click() on internet explorer

I used click('buttonName') in one of my e2e tests, and it works fine on other browsers but not for the internet explorer. The error message is Failed: Failed to find click("Open Warning"): Error executing JavaScript. I have identified that it is thrown at find.js in retryingFind(), where browserSideScripts.browserSideFind is executed. I tried to debug more but couldn't identify the bug further in browserSideScripts.js.

Allow multiple elements to be found by browserSideFind & find(...)

Problem

When attempting to use the find(...) (or see(...)) action helper, I'm unable to select a specific element when multiple are found on the page. Instead, I would be presented with the following error:

> see('Aug 26, 2019');

> "Looking for Aug 26, 2019 failed. More than one element is displayed, number of displayed elements: 8
Elements:
<td _ngcontent-wup-c43="" class="mat-cell cdk-column-created mat-column-created ng-star-inserted" mat-cell="" role="gridcell">Aug 26, 2019</td>
<td _ngcontent-wup-c43="" class="mat-cell cdk-column-created mat-column-created ng-star-inserted" mat-cell="" role="gridcell">Aug 26, 2019</td>
<td _ngcontent-wup-c43="" class="mat-cell cdk-column-created mat-column-created ng-star-inserted" mat-cell="" role="gridcell">Aug 26, 2019</td>
...
"

Solution

Update the findGlobal(locator) function to look for a new property on the locator, e.g. locator.index and/or locator.wantMultiple
Update the opt_options of browserSideFind(locators, opt_options) to include the new property
Update retrying_find to return the new possible type of WebElement[]
Update BrowserSideOptions with the new property (e.g. locator.wantMultiple)
Update getElement(locator: FlexibleLocator, description: string): Promise<WebElement> to also accept and use options: BrowserSideOptions and return Promise<WebElement|WebElement[]>

Ultimately, this would result in the end user having the following experience:

find('Aug 26, 2019', {wantMultiple: true})[2].click();
// or
const dates = below('Created').find('Aug 26, 2019', {wantMultiple: true});
expect(dates.length).toBe(8);

Golden screenshots created on Retina differ in size from those on Linux in CI/CD

Golden screenshots created on Retina (MacOS) differ in size from those on Linux in CI/CD (for example Ubuntu Docker on CircleCI).

Dropdowns/Selects unsupported

Seems that Dropdowns/Selects are unsupported.

Where is the documentation

Hello,
First of all, the helpers are really great.
But, do you plan to write a documentation on how to use these helpers in end-to-end tests?

compareScreenshot() does not support saving diff.png to a defined folder

diff.png is saved in the root folder by default, which may be inconvenient for some users.
The promise returned by compareScreenshot() rejects when there is an error in looksSame.createDiff() instead of rejecting in the case of !equal.
compareScreenshot() returns either a boolean or a promise of a string (if rejected) or a boolean (if resolved), which works but can be confusing.

Allow overriding of looks-same options

The looks-same options sometimes need to be tweaked, e.g. to bump the tolerance of the antialiasingTolerance levels, is order for a specific test of test suite to pass (e.g. difference of font rendering between OSs can sometimes be tricky)

Having compareScreenshot take an optional LooksSameOptions 4th param would allow more flexibility, without having to fork the lib.

Save current screenshot alongside the diff image on failure

Sometimes it's hard to understand why an image doesn't match only based on the original & diff image (e.g. in my case it was due to the difference of color profiles). Having the current screenshot saved in the output folder alongside the diff image would allow to more easily debug un-matching images.

Diffs aren't always saved

I've noticed that blue-harvest doesn't always save the diff images even though it outputs a message saying they were, looking into it I noticed that looks-same does some funky stuff with setImmediate which explains the behaviour I was seeing, e.g. nothing saved if the e2e run finishes quickly, or only the first image being saved.

Where is diff.png stored?

I'm trying to implement blue-harvest screen comparison in a E2E test setup, running in Azure (windows machines).

When using the screen comparison on localhost, the diff.png is written within the root of the angular project that I run e2e test on, and I can copy the diff image to our test reports.. But when the test is run on the azure build agents, then the diff.png is nowhere to be found.

The question is, is there a defined place where the png could be written to?

if the element we want to mask has a z-index greater than 0, the mask is not visible since it's under the element we want to hide
if the element is "dynamically" added, it seems that the element is not found and the mask is not added
to remove the mask, one needs to execute the following script manually document.body.removeChild(document.querySelector('#BP_ELEMENT_HIGHLIGHT__')). Maybe it would be a good idea to add a removeMask function
for the moment, if you add multiple mask, all the mask elements will have the same id which is not great and if you want to remove one of the mask it is not possible. Maybe the mask should not have an id but a class and the addMask function should allow one to specify a suffix for the class name (same for the removeMask)