Git Product home page Git Product logo

codeceptjs-resemblehelper's Introduction

Stand With Ukraine

NPM version AI features StandWithUkraine

Build Status:

Appium Helper: Appium V2 Tests - Android

Web Helper: Playwright Tests Puppeteer Tests WebDriver Tests TestCafe Tests

CodeceptJS Made in Ukraine

Reference: Helpers API

Supercharged E2E Testing

CodeceptJS is a new testing framework for end-to-end testing with WebDriver (or others). It abstracts browser interaction to simple steps that are written from a user's perspective. A simple test that verifies the "Welcome" text is present on a main page of a site will look like:

Feature('CodeceptJS demo');

Scenario('check Welcome page on site', ({ I }) => {
  I.amOnPage('/');
  I.see('Welcome');
});

CodeceptJS tests are:

  • Synchronous. You don't need to care about callbacks or promises or test scenarios which are linear. But, your tests should be linear.
  • Written from user's perspective. Every action is a method of I. That makes test easy to read, write and maintain even for non-tech persons.
  • Backend API agnostic. We don't know which WebDriver implementation is running this test.

CodeceptJS uses Helper modules to provide actions to I object. Currently, CodeceptJS has these helpers:

  • Playwright - is a Node library to automate the Chromium, WebKit and Firefox browsers with a single API.
  • Puppeteer - uses Google Chrome's Puppeteer for fast headless testing.
  • WebDriver - uses webdriverio to run tests via WebDriver or Devtools protocol.
  • TestCafe - cheap and fast cross-browser test automation.
  • Appium - for mobile testing with Appium
  • Detox - This is a wrapper on top of Detox library, aimed to unify testing experience for CodeceptJS framework. Detox provides a grey box testing for mobile applications, playing especially well for React Native apps.

And more to come...

Why CodeceptJS?

CodeceptJS is a successor of Codeception, a popular full-stack testing framework for PHP. With CodeceptJS your scenario-driven functional and acceptance tests will be as simple and clean as they can be. You don't need to worry about asynchronous nature of NodeJS or about various APIs of Playwright, Selenium, Puppeteer, TestCafe, etc. as CodeceptJS unifies them and makes them work as they are synchronous.

Features

  • 🪄 AI-powered with GPT features to assist and heal failing tests.
  • ☕ Based on Mocha testing framework.
  • 💼 Designed for scenario driven acceptance testing in BDD-style.
  • 💻 Uses ES6 natively without transpiler.
  • Also plays nice with TypeScript.
  • </> Smart locators: use names, labels, matching text, CSS or XPath to locate elements.
  • 🌐 Interactive debugging shell: pause test at any point and try different commands in a browser.
  • Easily create tests, pageobjects, stepobjects with CLI generators.

Installation

npm i codeceptjs --save

Move to directory where you'd like to have your tests (and CodeceptJS config) stored, and execute:

npx codeceptjs init

to create and configure test environment. It is recommended to select WebDriver from the list of helpers, if you need to write Selenium WebDriver tests.

After that create your first test by executing:

npx codeceptjs generate:test

Now test is created and can be executed with

npx codeceptjs run

If you want to write your tests using TypeScript just generate standard Type Definitions by executing:

npx codeceptjs def .

Later you can even automagically update Type Definitions to include your own custom helpers methods.

Note:

  • CodeceptJS requires Node.js version 12+ or later.

Usage

Learn CodeceptJS by examples. Let's assume we have CodeceptJS installed and WebDriver helper enabled.

Basics

Let's see how we can handle basic form testing:

Feature('CodeceptJS Demonstration');

Scenario('test some forms', ({ I }) => {
  I.amOnPage('http://simple-form-bootstrap.plataformatec.com.br/documentation');
  I.fillField('Email', '[email protected]');
  I.fillField('Password', secret('123456'));
  I.checkOption('Active');
  I.checkOption('Male');
  I.click('Create User');
  I.see('User is valid');
  I.dontSeeInCurrentUrl('/documentation');
});

All actions are performed by I object; assertions functions start with see function. In these examples all methods of I are taken from WebDriver helper, see reference to learn how to use them.

Let's execute this test with run command. Additional option --steps will show us the running process. We recommend use --steps or --debug during development.

npx codeceptjs run --steps

This will produce an output:

CodeceptJS Demonstration --
 test some forms
 • I am on page "http://simple-form-bootstrap.plataformatec.com.br/documentation"
 • I fill field "Email", "[email protected]"
 • I fill field "Password", "****"
 • I check option "Active"
 • I check option "Male"
 • I click "Create User"
 • I see "User is valid"
 • I dont see in current url "/documentation"
 ✓ OK in 17752ms

CodeceptJS has an ultimate feature to help you develop and debug your test. You can pause execution of test in any place and use interactive shell to try different actions and locators. Just add pause() call at any place in a test and run it.

Interactive shell can be started outside test context by running:

npx codeceptjs shell

Actions

We filled form with fillField methods, which located form elements by their label. The same way you can locate element by name, CSS or XPath locators in tests:

// by name
I.fillField('user_basic[email]', '[email protected]');
// by CSS
I.fillField('#user_basic_email', '[email protected]');
// don't make us guess locator type, specify it
I.fillField({css: '#user_basic_email'}, '[email protected]');

Other methods like checkOption, and click work in a similar manner. They can take labels or CSS or XPath locators to find elements to interact.

Assertions

Assertions start with see or dontSee prefix. In our case we are asserting that string 'User is valid' is somewhere in a webpage. However, we can narrow the search to particular element by providing a second parameter:

I.see('User is valid');
// better to specify context:
I.see('User is valid', '.alert-success');

In this case 'User is valid' string will be searched only inside elements located by CSS .alert-success.

Grabbers

In case you need to return a value from a webpage and use it directly in test, you should use methods with grab prefix. They are expected to be used inside async/await functions, and their results will be available in test:

Feature('CodeceptJS Demonstration');

Scenario('test page title', async ({ I }) => {
  I.amOnPage('http://simple-form-bootstrap.plataformatec.com.br/documentation');
  const title = await I.grabTitle();
  I.expectEqual(title, 'Example application with SimpleForm and Twitter Bootstrap'); // Avaiable with Expect helper. -> https://codecept.io/helpers/Expect/
});

The same way you can grab text, attributes, or form values and use them in next test steps.

Before/After

Common preparation steps like opening a web page, logging in a user, can be placed in Before or Background:

const { I } = inject();

Feature('CodeceptJS Demonstration');

Before(() => { // or Background
  I.amOnPage('http://simple-form-bootstrap.plataformatec.com.br/documentation');
});

Scenario('test some forms', () => {
  I.click('Create User');
  I.see('User is valid');
  I.dontSeeInCurrentUrl('/documentation');
});

Scenario('test title', () => {
  I.seeInTitle('Example application');
});

PageObjects

CodeceptJS provides the most simple way to create and use page objects in your test. You can create one by running

npx codeceptjs generate pageobject

It will create a page object file for you and add it to the config. Let's assume we created one named docsPage:

const { I } = inject();

module.exports = {
  fields: {
    email: '#user_basic_email',
    password: '#user_basic_password'
  },
  submitButton: {css: '#new_user_basic input[type=submit]'},

  sendForm(email, password) {
    I.fillField(this.fields.email, email);
    I.fillField(this.fields.password, password);
    I.click(this.submitButton);
  }
}

You can easily inject it to test by providing its name in test arguments:

Feature('CodeceptJS Demonstration');

Before(({ I }) => { // or Background
  I.amOnPage('http://simple-form-bootstrap.plataformatec.com.br/documentation');
});

Scenario('test some forms', ({ I, docsPage }) => {
  docsPage.sendForm('[email protected]','123456');
  I.see('User is valid');
  I.dontSeeInCurrentUrl('/documentation');
});

When using Typescript, replace module.exports with export for autocompletion.

Contributing

Contributors

Thanks all to those who are and will have contributing to this awesome project!

License

MIT © CodeceptJS Team

codeceptjs-resemblehelper's People

Contributors

arhell avatar davertmik avatar dependabot[bot] avatar fsblemos avatar kartikeya99 avatar kobenguyent avatar komarov-v avatar mirao avatar pablopaul avatar pgoellnitz avatar puneet0191 avatar rnovikovp avatar saadichouaib avatar sabrinovsky avatar sharath2106 avatar tairun avatar tfr-ds avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

codeceptjs-resemblehelper's Issues

Add custom assert message

Hi, it would be nice to have the ability to provide a custom assert error message on the visualDiff functions, so that it looks better in the reports.

Current assert error message:
MissMatch Percentage 1.86 + expected - actual -false +true

Desired assert error message example:
Images A and B do not match. (MissMatch Percentage 1.86) + expected - actual -false +true

Canvas triggering error during build on Node >10

When we include:

"codeceptjs-resemblehelper": "^1.9.0"

in our package.json and run "codeceptjs": "^2.6.3" test:

Feature("Home page");

Scenario("Search for something", (I) => {
  I.amOnPage("http://localhost:3000");
  I.fillField("Search", "Miles");
  I.click("Search");
  I.seeInCurrentUrl("/");
});

Scenario("Click on production link", (I) => {
  I.amOnPage("http://localhost:3000");
  I.click("The Link");
  I.seeInCurrentUrl("/link");
});

Scenario("Scroll to footer and click on privacy policy", (I) => {
  I.amOnPage("http://localhost:3000");
  I.scrollTo("footer");
  I.click("Privacy policy");
  I.seeInCurrentUrl("/privacy");
});

we get the following error:

error /home/node/node_modules/canvas: Command failed.
Exit code: 1
Command: node-pre-gyp install --fallback-to-build
Arguments: 
Directory: /home/node/node_modules/canvas
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using [email protected]
node-pre-gyp info using [email protected] | linux | x64
node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp info check checked for "/home/node/node_modules/canvas/build/Release/canvas.node" (not found)
node-pre-gyp http GET https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v83-linux-musl-x64.tar.gz
node-pre-gyp http 404 https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v83-linux-musl-x64.tar.gz
node-pre-gyp WARN Tried to download(404): https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v83-linux-musl-x64.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v83 ABI, musl) (falling back to source compile with node-gyp) 
node-pre-gyp http 404 status code downloading tarball https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v83-linux-musl-x64.tar.gz 
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp info ok 
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp info find Python using Python version 3.8.2 found at "/usr/bin/python3"
gyp http GET https://unofficial-builds.nodejs.org/download/release/v14.2.0/node-v14.2.0-headers.tar.gz
gyp http 200 https://unofficial-builds.nodejs.org/download/release/v14.2.0/node-v14.2.0-headers.tar.gz
gyp http GET https://unofficial-builds.nodejs.org/download/release/v14.2.0/SHASUMS256.txt
gyp http 200 https://unofficial-builds.nodejs.org/download/release/v14.2.0/SHASUMS256.txt
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/node/node_modules/canvas/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/usr/local/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/root/.cache/node-gyp/14.2.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/14.2.0',
gyp info spawn args   '-Dnode_gyp_dir=/usr/local/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/14.2.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/node/node_modules/canvas',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
/bin/sh: pkg-config: not found
gyp: Call to 'pkg-config pixman-1 --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:276:12)
gyp ERR! System Linux 4.19.76-linuxkit
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/home/node/node_modules/canvas/build/Release/canvas.node" "--module_name=canvas" "--module_path=/home/node/node_modules/canvas/build/Release" "--napi_version=6" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v83"
gyp ERR! cwd /home/node/node_modules/canvas
gyp ERR! node -v v14.2.0
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/home/node/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/home/node/node_modules/canvas/build/Release --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/home/node/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1051:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
node-pre-gyp ERR! System Linux 4.19.76-linuxkit
node-pre-gyp ERR! command "/usr/local/bin/node" "/home/node/node_modules/canvas/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /home/node/node_modules/canvas
node-pre-gyp ERR! node -v v14.2.0
node-pre-gyp ERR! node-pre-gyp -v v0.11.0
node-pre-gyp ERR! not ok 
Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/home/node/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/home/node/node_modules/canvas/build/Release --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)

If we leave this helper package out, then codeceptjs reports the error:

$ npx codeceptjs run --reporter mocha-junit-reporter --steps --verbose
 
Could not load helper ResembleHelper from module 'codeceptjs-resemblehelper':
Cannot find module 'codeceptjs-resemblehelper'

We're not using the ResembleHelper in our tests, however it appears that without it, our codeceptjs tests will not work.

Any idea what we can do to remedy this?

Node crash if dimension mismatch in `I.seeVisualDiff`

  • codeceptjs-resemblehelper: 1.6.0
  • codeceptjs: 2.1.5
  • node: 10.14.0

When using release 1.6.0 of this helper, node crashes with the following error if the dimension of the screenshots is not the same:

error

This is most likely due to the version of resemblejs being used. In the package.json the version specified is: ^3.0.0 which installs 3.2.0. A simple fix would be to always use 3.0.0, which worked for me.

Test passes with different images when using 'seeVisualDiffForElement' for comparison.

Hello!

I've got a test looking like that:

1 Scenario.only('Cross can be added', async (I, editorPage) => {
2    I.click(editorPage.tool.cross);
3    await I.addTool(50, 50);
4    I.saveElementScreenshot(editorPage.lastAddedTool, 'CROSS_ELEM.png');
5    I.seeVisualDiffForElement(editorPage.lastAddedTool, 'CROSS_ELEM.png',  {prepareBaseImage: false, tolerance: 0});
6 }).tag('editor').tag('tools').tag('cross');

If I change added tool (2nd line) test keep passing instead of fail.
Both images have the same dimensions.
Also, for some reason, test produces diff image, without highlighted difference.

Test run results

Test Output image: OUTPUT_CROSS_ELEM

Base image: BASE_CROSS_ELEM

Generated diff image (generates copy of base image): Diff_CROSS_ELEM

Console output:
Screenshot

Additional info

  • OS: Windows 10
  • CodeceptJS version: 2.6.10
  • codeceptjs-resemblehelper version: 1.9.2

Checked with different 'tolerance' level but got almost the same results. Diff image generated only with {tolerance: 0}
Also checked larger images - same results.

OS specific resemblejs tolerance issues

What are you trying to achieve?

Hello, first thanks a million for this great helper! It makes my life easier :)

I am experimenting Visual Testing based on resemblejs, and I face a strange issue:

The tests pass when I run them locally on the computer that originally created the screenshot (Mac OS), weather HEADLESS is on or off.
However, some tests fail from a CI pipeline (bitbucket pipelines) based on Linux (HEADLESS).

I have 2 main questions:

  1. is this expected? Where can I find information about the visual comparison "tolerance" and overall options? I do not find a reference to it in their documentation: https://github.com/rsmbl/Resemble.js
  2. what is the intended approach when it comes to update screenshots? Is there an "update" option somehow?

What do you get instead?

Provide console output if related. Use --verbose mode for more details.

MissMatch Percentage 3.82

Scenario Steps:
- I.seeVisualDiff("homepage-mobile.png", {"tolerance":2,"prepareBaseImage":false}) at Test.<anonymous> (tests/visual-regression/homepage.test.js:15:5)
- I.saveScreenshot("homepage-mobile.png") at Test.<anonymous> (tests/visual-regression/homepage.test.js:14:5)
- I.amOnPage("/") at Test.<anonymous> (tests/visual-regression/homepage.test.js:12:5)
- I.resizeWindow(320, 480) at Test.<anonymous> (tests/visual-regression/homepage.test.js:11:5)

Provide test source code if related

Feature('Screen comparison of the Home page');

Scenario('Compare desktop Home Page @visual-test', I => {
  I.amOnPage('/');

  I.saveScreenshot('homepage.png');
  I.seeVisualDiff('homepage.png', { tolerance: 2, prepareBaseImage: false });
});

Scenario('Compare mobile Home Page @visual-test', I => {
  I.resizeWindow(320, 480);
  I.amOnPage('/');

  I.saveScreenshot('homepage-mobile.png');
  I.seeVisualDiff('homepage-mobile.png', { tolerance: 2, prepareBaseImage: false });
});

Details

require('dotenv').config();
import { setHeadlessWhen, setWindowSize } from '@codeceptjs/configure';

const headless = (process.env.HEADLESS || process.env.CI) && process.env.BROWSER !== 'firefox';
const profile = ['e2e', 'vr'].includes(process.profile) ? process.profile : 'e2e';

// turn on headless mode (not opening browser)
setHeadlessWhen(headless);
setWindowSize(1600, 1200);

export const config = {
  name: 'my-project',
  tests: {
    e2e: 'tests/end-to-end/scenarios/**/*.test.js',
    vr: 'tests/visual-regression/**/*.test.js'
  }[profile],
  include: {
    I: './tests/end-to-end/steps_file.js'
  },
  output: 'test-reports/end-to-end',
  bootstrap: null,
  mocha: {
    reporterOptions: {
      mochaFile: {
        e2e: 'test-reports/end-to-end.xml',
        vr: 'test-reports/visual-regression.xml'
      }[profile]
    }
  },
  helpers: {
    Puppeteer: {
      browser: process.env.BROWSER || 'chrome',
      url: process.env.TEST_BASE_URL,
      show: true,
      waitForNavigation: 'networkidle0',
      waitForAction: 200,
      pressKeyDelay: 50,
      chrome: {
        args: [
          '--no-sandbox',
          '--ignore-certificate-errors',
          '--disable-dev-shm-usage',
          '--disable-gpu',
          '--disable-setuid-sandbox'
        ]
      }
    },
    ResembleHelper: {
      require: 'codeceptjs-resemblehelper',
      screenshotFolder: './test-reports/screenshots/',
      baseFolder: './test-reports/screenshots/base/',
      diffFolder: './test-reports/screenshots/diff/'
    }
  },
  plugins: {
    allure: {},
    autoDelay: {
      enabled: true
    },
    pauseOnFailure: {},
    retryFailedStep: {
      enabled: true
    },
    screenshotOnFail: {
      enabled: !process.env.CI
    }
  }
};

Mochawesome context gets absolute paths instead of relative to Images

In order for the mocha awesome HTML report to be more helpful it would be nice if _addMochaContext https://github.com/codeceptjs/codeceptjs-resemblehelper/blob/master/index.js#L168 would supply relative paths to the images.As we (like most) probably run CI/CD the build artefacts are not viewed on the same system they are created, thus they won't be shown:
Bildschirmfoto 2021-09-10 um 01 05 55

Check the following Mochawesome report.json snippet (the report.html code is a mess, but looks similar):

   "context": "[
   \n  \"Base Image\",\n  \"/builds/project-management/myproject/tests/output/base/RCA-desktop.png\",
   \n  \"ScreenShot Image\",\n  \"/builds/project-management/myproject/tests/output/RCA-desktop.png\",
   \n  \"Diff Image\",\n  \"/builds/project-management/myproject/tests/output/diff/Diff_RCA-desktop.png\",
   \n  \"Screenshot_e693b747-f9a3-481a-8060-8ef6b6df1005.failed.png\"\n]",

The first 3 are generated by codecept-resemblehelper the last by ScreenShotOnFailure plugin. Thus it seems plausible that relative paths work.

(Don't have a fix, nor a MWE, nor a test yet, just found that as GitLab artefact view was broken)

browser.element is not a function

when I updated to last WebdriverIO 5 my tests are failing

` browser.element is not a function

Scenario Steps:

  • I.seeVisualDiffForElement(".infaOnglet", "Home_Page_Image.png", {"prepareBaseImage":false,"tolerance":1}) at Test.I (tests\visual-testing\homePage.js:27:7)
  • I.saveScreenshot("Home_Page_Image.png") at Test.I (tests\visual-testing\homePage.js:23:7)

`

Screenshot comparison is not working

We are using codeceptjs-resemblehelper to perform visual regression testing for our application that supports multiple locales. We are logging in with users mapped to these locales to achieve this.

We were trying a negative scenario where we compared two search boxes of different locales.

searchField-en-GB

searchField-ja-JP

The tests are passing even though the locales are not the same. I used another screenshot for comparison to see if the screen comparison is actually working or not. It looks like only the dimensions are considered and not the contents in the screenshot which is weird.

Could you please take a look at this?

Percentage Calculated is 0.00 for large images

It looks like when an image is large enough the calculated percentage is 0.00.

Test images:

white background with black line
base

just white background
base-blank

The diff image generated show the difference correctly.

Diff_base

Is there a way to get more accurate percentage or perhaps some other value to indicate that there differences between the two images.

Allow user to choose screenshotFolder

Currently the screenshotFolder is defaulted to the codeceptjs output folder no matter what you set in the ResembleHelper config.

I would like to set the screenshot folder because:

I currently run both Webkit and Chromium tests and taking screenshots differ between them by a little. I want to have separate base images for webkit and chromium.

I want to have the base images for the same test in two different folders - webkit and chromium. But the inability to change the screenshotFolder means that I cannot have two images of the same file name. The workaround is to just name the base images chromium-filename and webkit-filename but I feel like this is messier.

node crashes when image comparison difference is larger than the tolerance threshold.

Hi There,

I ran below error when I do visual testing via resemblehelper. Can anyone help to take a look? Thanks a lot.
The helper works well when the difference is under the tolerance indicator, only crashes when the image comparison difference is bigger than the tolerance.
image

The project dependencies as below :

"devDependencies": {
"mocha": "^5.2.0",
"mocha-multi": "^1.0.1",
"puppeteer": "^1.15.0"
},
"dependencies": {
"codeceptjs": "^2.6.5",
"codeceptjs-resemblehelper": "1.9.0",
"mocha-junit-reporter": "^1.18.0",
"mochawesome": "^3.1.1"
}

And node version is : 8.11.3
Also tried with node: 10.20.1, the issue still exists.

image

Add support for Cloud Storage

Need a function that can download base images from a cloud platform like Google -Drive, it should be parameterized, with the user, key and folder name as three input parameters.

Base images can be downloaded on the fly from the cloud platform, preventing the need to store images on Github Repo.

Similarly, Diff images generated can be uploaded on the cloud platform and link returned to the users.

Comparison different image didn't added in html test report.

Hi There,

Currently, i can run visual comparison with codeceptjs-resemblehelper. But found the diff image can't be added in allure test report. Only latest screenshot could be added.

image

Have configured allure in my test as below:
image

Running command as :

codeceptjs run --reporter mocha-multi --config=config-files/codecept_demo_visual_regression_config.js --verbose

Is there a way to add the diff result image in the test report? That will make the report checking easy when run in cloud with Jenkins.

Download base images doesn't work only with prepareBaseImage global

When using method _assertVisualDiff , we are only downloading the base image if the options param explicitly includes the option {prepareBaseImage: false}. Otherwise, it will omit downloading the base file, even if the helper configuration includes the prepareBaseImage option.

My suggestion is to check the config option for prepareBaseImage when this option is missing in the _assertVisualDiff method:

const prepareBaseImage = options.prepareBaseImage !== undefined
      ? options.prepareBaseImage
      : (this.prepareBaseImage === true)
if (awsC !== undefined && prepareBaseImage === false) {
      await this._download(awsC.accessKeyId, awsC.secretAccessKey, awsC.region, awsC.bucketName, baseImage);
}

Check PR #79 here

Config: Why do we need screenshotFolder?

https://codecept.io/visual says

screenshotFolder : This will always have the same value as output in Codecept configuration

Why do we need it then as config here? Instead the plugin could pull it directly from the CodeceptJS config, what do you think?

Smaller config surface, better plugin I would say :)

resemble.js Output Settings working?

Hi, is this option resemble.js Output Settings working?
Tried to follow instructions, tried ignoredBox. No effect. I want to use ignoredBoxes.
const outputSettings = {
ignoredBox: { left: 184, top: 231, right: 284, bottom: 254}
};
I.seeVisualDiff("Reporing_TimeCorrection-CorrectionOfDetails.png", {prepareBaseImage: baseImage.desktopPages.reporting.timeCorrection, tolerance: 0.00, outputSettings:outputSettings})

Unable to pass boundingBoxes prop

Hi Team,

Could you please add support for boundingBoxes similar to boundingBox/ignoredBox (#12)

await visual.seeVisualDiffForElement(selector, `test.png`, {
            prepareBaseImage: false,
            tolerance: 1,
            needsSameDimension: false,
            boundingBoxes: null
        });

The tests passes even when it is over threshold.

Hi there,

if (options.skipFailure === false) {
      assert(misMatch <= options.tolerance, "Screenshot does not match with the baseline " + baseImage + " when MissMatch Percentage is " + misMatch);
    }

As the previous code snippet shows, since the options.skipFailure is undefined, the comparison would always return false. Because of this, the assert would never run, and the pipeline passes even when the tolerance is clearly over the threshold.

Please define false as default, because previously the assert run automatically, did not depend on the value of skipFailure.

Global prepareBaseImage not working

Hi!

After this change, the global prepareBaseImage is not generating the base images. It's only working when passing to seeVisualDiff.

I'm opening a PR to fix it, can you take a look?

Thanks for you work

Suggestion to add width and height into "The images are not of same dimensions" error

Hello ✋
I have a proposal. I think error message that throws when images has different dimensions can be more informative.
Right now message is:
"The images are not of same dimensions. Please use images of same dimensions so as to avoid any unexpected results."
You can find it there https://github.com/Percona-Lab/codeceptjs-resemblehelper/blob/c043b88b2281a280450983f010a2148ab74a460d/index.js#L43
I am suggesting to add height and width there. Like:
"The images are not of same dimensions. Image 1 is 1920x1080 and image 2 is 1920x1017. Please use images of same dimensions so as to avoid any unexpected results."

failing to run in codeceptjs/codeceptjs docker image

i would like to use this in codeceptjs/codeceptjs docker container, but i'm not sure if i'm doing it right.

this dockerfile builds fine:

FROM codeceptjs/codeceptjs
RUN npm install codeceptjs-resemblehelper --prefix /codecept

but running the docker container gives these errors:

Could not load helper ResembleHelper from module 'codeceptjs-resemblehelper':
Cannot find module 'codeceptjs/lib/container'
Require stack:
- /codecept/node_modules/codeceptjs-resemblehelper/index.js
- /codecept/lib/container.js
- /codecept/lib/codecept.js
- /codecept/bin/codecept.js
Error: Cannot find module 'codeceptjs/lib/container'
Require stack:
- /codecept/node_modules/codeceptjs-resemblehelper/index.js
- /codecept/lib/container.js
- /codecept/lib/codecept.js
- /codecept/bin/codecept.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Module.require (internal/modules/cjs/loader.js:974:19)
    at require (internal/modules/cjs/helpers.js:92:18)
    at Object.<anonymous> (/codecept/node_modules/codeceptjs-resemblehelper/index.js:9:19)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:14)
    at Module.require (internal/modules/cjs/loader.js:974:19)

Error:
    at createHelpers (/codecept/lib/container.js:182:13)
    at Function.create (/codecept/lib/container.js:43:25)
    at Codecept.init (/codecept/lib/codecept.js:56:15)
    at Command.module.exports (/codecept/lib/command/run.js:24:14)
    at Command.listener (/codecept/node_modules/commander/index.js:315:8)
    at Command.emit (events.js:375:28)
    at Command.parseArgs (/codecept/node_modules/commander/index.js:651:12)
    at Command.parse (/codecept/node_modules/commander/index.js:474:21)
    at Object.<anonymous> (/codecept/bin/codecept.js:192:9)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
C:\docker\as\as-jtlshop5-dev\plugins\UpwaveEnergielabel\tests

not working with debug mode with codeceptjs

cant work with debug mode getting errors:

the conf codeceptjs :

ResembleHelper: { require: "codeceptjs-resemblehelper", screenshotFolder: "./output/", baseFolder: "./tests/visual-testing/screenshots/base/", diffFolder: "./output/diff/" }
the error:

Could not load helper ResembleHelper from module 'codeceptjs-resemblehelper': Cannot find module 'codeceptjs-resemblehelper'

test-assets\codeceptjs\node_modules\codeceptjs\lib\container.js:153:13)

codeceptjs version 2.1.3

Add support different endpoint for S3

Now ResembleHelper can use only AWS S3 as a storage provider, but s3-sdk can be configure for different providers just by passing a "endpoint" param to the config.

Setting needsSameDimension flag to false isn't validating the difference of text between two locales.

We are using codeceptjs-resemblehelper to perform visual regression testing for our application that supports multiple locales. We are logging in with users mapped to these locales to achieve this.

Since our page is dynamic and it contains data. We are using I.seeVisualDiffForElement method and find the diff only for the elements that are static in the application. Some translations are rendering elements tad bigger than others.

On executing a negative case where the user belongs to the locale en-GB and compares the image for fr-FR, we see the below error -
The base image is of 65 X 156 and actual image is of 62 X 156. Please use images of same dimensions so as to avoid any unexpected results.

searchBar-en-GB

searchBar-fr-FR

In order to ignore image size, we used the attribute needSameDimensions: false. But unfortunately, after using this attribute in our tests, the screen comparison is not working. The diff is not considered, and the tests are passing even though the locale is completely different.

Can you take a look at this issue?

update documentation

update documentation, have real examples of each function, and then move the main code to a new section, doc-blocks are not helping on readme, rather have calls

FEATURE: allow resemble.compare even if dimensions are not the same

The documentation says:

Note: seeVisualDiff and seeVisualDiffElement work only when the dimensions of the 
screenshot as well as the base image are same so as to avoid unexpected results.

which is found in the code

if (!data.isSameDimensions) {
let dimensions1 = sizeOf(baseImage);
let dimensions2 = sizeOf(actualImage);
reject(new Error(`The base image is of ${dimensions1.height} X ${dimensions1.width} and actual image is of ${dimensions2.height} X ${dimensions2.width}. Please use images of same dimensions so as to avoid any unexpected results.`));
}

However the underlying resemble.js works just fine in calculating the diff percentage and tolerance and taking that into account.

I propose to make this check optional and thus introduce an option/parameter to skip that check

Background:
We use visual regression testing for webpages and as we allow some tolerance for minor layout changes. But this could also make a full page screenshot some few pixel longer.
This should be able to fail the test (as is the state now), but in fact only with 0 tolerance. However if you have a tolerance defined, tests should be able to run fine nevertheless if the change is within the tolerance.

Debug Mode Imporvements

  1. Add support for debugMode
  2. DebugMode improvements, ask for saving the base image, ask for updating the base image in case comparison doesn't work as expected.

Minor Improvements

  • add support for IgnoredBox, with updated documentation

  • add a debug to print the path to file, for printing the URL of the Diff Image in the console.

Helper attempts to confirm baseline and comparison images are accessible in write mode

I'm trying to have Codecept's resemble-helper compare a read-only baseline file to a writable current file. The reason baseline files are read-only are because they are versioned in our version control system, and due to the file locking approach of our version control system (perforce) the files are always read-only until checked out.

The resemble-helper fails when attempting to open the read-only baseline file with a 'file is read-only' error

Console output of error occurring:

    I screenshot element ".fld-prop-lbl-length>i", "be-field-property-lbl-icon"
Error: C:\perforce-MDM.Next\MDM.Next\splourde_mdmui-config-app-main\mdmui-config-app-ui\e2e\output\resemble-screenshots\base/be-field-property-error-icon.png is read-only
Error: C:\perforce-MDM.Next\MDM.Next\splourde_mdmui-config-app-main\mdmui-config-app-ui\e2e\output\resemble-screenshots\base/be-field-property-error-icon.png is read-only
    at fs.access (C:\perforce-MDM.Next\MDM.Next\splourde_mdmui-config-app-main\mdmui-config-app-ui\e2e\node_modules\codeceptjs-resemblehelper\index.js:47:15)
    at FSReqWrap.args [as oncomplete] (fs.js:140:20)
Uncaught undefined exception

It looks like the issue lies in codeceptjs-resemblehelper/index.js, lines 44 - 57:

    // check whether the base and the screenshot images are present.
    fs.access(baseImage, fs.constants.F_OK | fs.constants.W_OK, (err) => {
      if (err) {
        throw new Error(
          `${baseImage} ${err.code === 'ENOENT' ? 'base image does not exist' : 'is read-only'}`);
      }
    });

    fs.access(actualImage, fs.constants.F_OK | fs.constants.W_OK, (err) => {
      if (err) {
        throw new Error(
          `${actualImage} ${err.code === 'ENOENT' ? 'screenshot image does not exist' : 'is read-only'}`);
      }
    });

I think that we just need to update this check to be F_OK | R_OK (write is not necessary for either file, I have confirmed this in a test run)

Details

  • CodeceptJS version: 2.6.1
  • codeceptjs-resemblehelper version: 1.9.1
  • Operating System: windows

Update [email protected] to [email protected] because of dependency vulnerability

Need to update mkdirp dependency, because of 0.5.1 version's dependency affected prototype pollution.

snyk detected the vulnerability.

Issues with no direct upgrade or patch:
  ✗ Prototype Pollution [Medium Severity][https://snyk.io/vuln/SNYK-JS-MINIMIST-559764] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] and 6 other path(s)
  This issue was fixed in versions: 1.2.2

https://snyk.io/vuln/SNYK-JS-MINIMIST-559764

Please upgrade mkdirp to 1.0.3, what no use minimists.

Not installed on MAC ERR! [email protected]

try to install package on MAC

npm install codeceptjs-resemblehelper
npm install --build-from-source codeceptjs-resemblehelper

and see this error

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node-pre-gyp install --fallback-to-build

what's wrong?

Allure report does not show diff screenshot

Allure report shows the diff screenshot as corrupted; since the fs.writeFile() call is not finished when allure.addAttachment is called.

I replaced the fs.writeFile() call with fs.writeFileSync() in #51

Also covered some warnings the IDE highlighted.

Again, looking forward to your review :)

Installation fails with node16+

I can't install the package codeceptjs-resemblehelper anymore because of gyp

npm install codeceptjs-resemblehelper
fails with HTTP 404 in:
error response status 404 Not Found on https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v93-linux-glibc-x64.tar.gz

Executed in docker image: mcr.microsoft.com/playwright:focal
It worked some weeks ago in exactly the same setup. I am afraid the dependency gyp is now completely broken.

Detailed output:

npm WARN deprecated [email protected]: The
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: Please upgrade to @mapbox/node-pre-gyp: the non-scoped node-pre-gyp package is deprecated and only the @mapbox scoped package will recieve updates in the future
npm ERR! code 1
npm ERR! path /test/node_modules/resemblejs/node_modules/canvas
npm ERR! command failed
npm ERR! command sh -c node-pre-gyp install --fallback-to-build
npm ERR! Failed to execute '/usr/bin/node /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/test/node_modules/resemblejs/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/test/node_modules/resemblejs/node_modules/canvas/build/Release --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v93' (1)
npm ERR! node-pre-gyp info it worked if it ends with ok
npm ERR! node-pre-gyp info using [email protected]
npm ERR! node-pre-gyp info using [email protected] | linux | x64
npm ERR! node-pre-gyp info check checked for "/test/node_modules/resemblejs/node_modules/canvas/build/Release/canvas.node" (not found)
npm ERR! node-pre-gyp http GET https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v93-linux-glibc-x64.tar.gz
npm ERR! node-pre-gyp ERR! install response status 404 Not Found on https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v93-linux-glibc-x64.tar.gz
npm ERR! node-pre-gyp WARN Pre-built binaries not installable for [email protected] and [email protected] (node-v93 ABI, glibc) (falling back to source compile with node-gyp)
npm ERR! node-pre-gyp WARN Hit error response status 404 Not Found on https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v93-linux-glibc-x64.tar.gz
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | x64
npm ERR! gyp info ok
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | linux | x64
npm ERR! gyp info find Python using Python version 3.8.5 found at "/usr/bin/python3"
npm ERR! gyp http GET https://nodejs.org/download/release/v16.4.0/node-v16.4.0-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.4.0/node-v16.4.0-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v16.4.0/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v16.4.0/SHASUMS256.txt
npm ERR! (node:69) [DEP0150] DeprecationWarning: Setting process.config is deprecated. In the future the property will be read-only.
npm ERR! (Use `node --trace-deprecation ...` to show where the warning was created)
npm ERR! gyp info spawn /usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/test/node_modules/resemblejs/node_modules/canvas/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/root/.cache/node-gyp/16.4.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/16.4.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/16.4.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/test/node_modules/resemblejs/node_modules/canvas',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! /bin/sh: 1: pkg-config: not found
npm ERR! gyp: Call to 'pkg-config pixman-1 --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
npm ERR! gyp ERR! configure error
npm ERR! gyp ERR! stack Error: `gyp` failed with exit code: 1
npm ERR! gyp ERR! stack     at ChildProcess.onCpExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:394:28)
npm ERR! gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
npm ERR! gyp ERR! System Linux 5.10.25-linuxkit
npm ERR! gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/test/node_modules/resemblejs/node_modules/canvas/build/Release/canvas.node" "--module_name=canvas" "--module_path=/test/node_modules/resemblejs/node_modules/canvas/build/Release" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v93"
npm ERR! gyp ERR! cwd /test/node_modules/resemblejs/node_modules/canvas
npm ERR! gyp ERR! node -v v16.4.0
npm ERR! gyp ERR! node-gyp -v v7.1.2
npm ERR! gyp ERR! not ok
npm ERR! node-pre-gyp ERR! build error
npm ERR! node-pre-gyp ERR! stack Error: Failed to execute '/usr/bin/node /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/test/node_modules/resemblejs/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/test/node_modules/resemblejs/node_modules/canvas/build/Release --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v93' (1)
npm ERR! node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/test/node_modules/@mapbox/node-pre-gyp/lib/util/compile.js:89:23)
npm ERR! node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:394:28)
npm ERR! node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1067:16)
npm ERR! node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)
npm ERR! node-pre-gyp ERR! System Linux 5.10.25-linuxkit
npm ERR! node-pre-gyp ERR! command "/usr/bin/node" "/test/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
npm ERR! node-pre-gyp ERR! cwd /test/node_modules/resemblejs/node_modules/canvas
npm ERR! node-pre-gyp ERR! node -v v16.4.0
npm ERR! node-pre-gyp ERR! node-pre-gyp -v v1.0.5
npm ERR! node-pre-gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2021-07-13T10_49_05_699Z-debug.log

Images cannot be found if the user's directory named using Russian language

OS: Windows 10

If you have your user's directory named in Russian (for example "C:\Users\Васян") the whole ResembleHelper wont work on your system. That's some kind of path-related issues.

That's the error what you get in that case:
Uncaught C:\\Users\\Васян\\work\\project\\tests\\end-to-end\\output/firstPage.png does not exist

I've changed my user's directory name to another one using only English letters and now it works well. Might be helpful for somebody.

Seems like you shouldn't have your main directories named in any language rather than English at all

Docker Could not load helper ResembleHelper

Hi,
I am struggling to run my visual tests in docker container.
I am able to run functional tests without codeceptjs-resemblehelper helper in docker.
I am able to run visual tests without docker container.
I have codeceptjs-resemblehelper module added.

Tried to give the absolute path. REquired module manually. Nothing helped.
SImple docker command
docker run --net=host -v “$(pwd):/tests” codeception/codeceptjs codeceptjs run

also tried to give a direct path for mount

All I got is this

Could not load helper ResembleHelper from module ‘codeceptjs-resemblehelper’:
Cannot find module ‘codeceptjs-resemblehelper’
Require stack:
- /codecept/lib/container.js
- /codecept/lib/codecept.js
- /codecept/bin/codecept.js

Error:
    at createHelpers (/codecept/lib/container.js:166:13)
    at Function.create (/codecept/lib/container.js:42:25)
    at Codecept.init (/codecept/lib/codecept.js:57:15)
    at Command.module.exports (/codecept/lib/command/run.js:28:14)
    at Command.listener (/codecept/node_modules/commander/index.js:315:8)
    at Command.emit (events.js:209:13)
    at Command.parseArgs (/codecept/node_modules/commander/index.js:651:12)
at Command.parse (/codecept/node_modules/commander/index.js:474:21)
at Object.<anonymous> (/codecept/bin/codecept.js:228:9)
at Module._compile (internal/modules/cjs/loader.js:936:30)

Isolated issue by creating a basic project.

My config

const { setHeadlessWhen } = require('@codeceptjs/configure');

// turn on headless mode when running with HEADLESS=true environment variable
// HEADLESS=true npx codecept run
setHeadlessWhen(process.env.HEADLESS);

exports.config = {
  tests: './*_test.js',
  output: './output',
  helpers: {
    Puppeteer: {
      url: 'https://www.ss.lv/',
      show: true,
      windowSize: '1200x900',
	  chrome: {args: ['--no-sandbox']}
    },
     "ResembleHelper" : {
       "require": "codeceptjs-resemblehelper",
       "screenshotFolder" : "./tests/output/",
       "baseFolder": "./tests/screenshots/base/",
       "diffFolder": "./tests/screenshots/diff/"
     }
  },
  include: {
    I: './steps_file.js'
  },
  bootstrap: null,
  mocha: {},
  name: 'Tests2',
  plugins: {
    retryFailedStep: {
      enabled: true
    },
    screenshotOnFail: {
      enabled: true
    }
  }
}

Zero tolerance causes: UnhandledPromiseRejectionWarning: TypeError: this.resolveImagePathRelativeFromReport is not a function

It seems to be a regression in 1.9.5
It works well for me with 1.9.4

Run test with this step

I.seeVisualDiff(image, { prepareBaseImage: false, tolerance: 0 });

Expected result:

  • No issue

Actual result:

  • Test passes, but an exception is logged:
I see visual diff "DND_Editor_Default_Position_chrome.png", {"prepareBaseImage":false,"tolerance":0}
      › Tolerance Level Provided 0
      › Diff Image File Saved to: /home/mirao/workspace/codeceptjs/tests/spork-publisher/home/mirao/workspace/codeceptjs/tests/spork-publisher/screenshots/diff/Diff_DND_Editor_Default_Position_chrome.png
      › MisMatch Percentage Calculated is 0.00 for baseline DND_Editor_Default_Position_chrome.png
(node:806286) UnhandledPromiseRejectionWarning: TypeError: this.resolveImagePathRelativeFromReport is not a function
    at ResembleHelper._addMochaContext (/home/mirao/workspace/codeceptjs/node_modules/codeceptjs-resemblehelper/index.js:194:46)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:806286) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:806286) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Workaround:

  • Use a non zero tolerance, e.g. { tolerance: 0.01 }

Used SW:

  • Node v14.18.1
  • CodeceptJS 3.0.7
  • codeceptjs-resemblehelper 1.9.5

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.