Git Product home page Git Product logo

testcafe-live's Introduction

TestCafe Live

This module is now deprecated.

In TestCafe v1.0.0, we have integrated features from the testcafe-live module into the main TestCafe code. We have introduced the live mode, which provides experience similar to testcafe-live. See Live Mode for more information.

See instant feedback when working on tests.

What is it?

TestCafe Live provides a service that keeps the TestCafe process and browsers opened the whole time you are working on tests. Changes you make in code immediately restart the tests. That is, TestCafe Live allows you to see test results instantly.

TestCafe Live Demo

Watch the full review on YouTube.

Install

TestCafe Live is a CLI tool. To start using it, you need to install both testcafe and testcafe-live:

npm install testcafe testcafe-live -g

If you have already installed the testcafe module (version 0.18.0 or higher) you can install only testcafe-live:

npm install testcafe-live -g

This installs modules on your machine globally.

If necessary, you can add these modules locally to your project:

cd <your-project-dir>
npm install testcafe testcafe-live --save-dev

If you have installed testcafe-live locally to your project, add an npm script to package.json to run tests:

"scripts": {
  "testcafe-live": "testcafe-live chrome tests/"
},
npm run testcafe-live

How to use

Run tests with testcafe-live in the same way as you do with testcafe:

testcafe-live chrome tests/

Use standard testcafe arguments to run tests with testcafe-live. It opens the required browsers, run tests there, shows the reports and waits for your further actions.

TestCafe Live watches files that you pass as the src argument and files that are required in them. Once you make changes in files and save them, TestCafe Live immediately reruns your tests.

When the tests are done, browsers stay on the last opened page so you can work with it and explore it by using the browser's developer tools.

Commands

  • ctrl+s - stop current test run;
  • ctrl+r - restart current test run;
  • ctrl+w - turn off/on files watching;
  • ctrl+c - close opened browsers and terminate the process.

Features

  • TestCafe Live watches files with tests and helper modules rerunning the tests once changes are saved;
  • You can explore the tested page in the same browser when tests are finished;
  • If tests authenticate into your web application using User Roles, you do not need to execute login actions every test run, it saves your working time;
  • Use the same API as TestCafe;
  • CLI interface allows you to stop test runs, restart them and pause file watching;
  • You can use TestCafe Live with any browsers (local, remote, mobile or headless).

Why TestCafe Live is a separate repository

TestCafe Live is developed by the TestCafe Team as an addition to the main TestCafe module. Keeping it a separate project provides many benefits:

  • We can deliver new functionality once it's ready regardless of the TestCafe release cycle;
  • We can get feedback early and make new releases as fast as necessary to provide the best experience for developers;
  • We can try experimental features that may be added to TestCafe later and get early feedback about them.

Will this functionality be released in the main TestCafe tool

We will decide when we have more feedback and when we consider TestCafe Live finished and stable. Since TestCafe is a test runner it is possible that live mode will exist as an additional tool.

Tips and Tricks

Which path should I pass as the src argument

You can pass either a path to a file with tests or a path to a directory.

If you specify a single file, testcafe-live will watch changes in it. Additionally, it will watch changes in files that are required from this file. Once you save changes in this file or in one of the required files, tests are rerun.

testcafe-live chrome tests/test.js

You can also pass a path to a directory where your files with tests are stored.

testcafe-live chrome tests/

TestCafe will watch all files in this directory and all files that are required from there and restart all tests once one of them is changed.

I have lots of tests but would like to restart only one

When you work on a particular test, just add the .only call for it:

test.only('Current test', async t => {});

Once you are done with it and ready to run the whole suite, just remove the .only directive and save the file.

Should I use TestCafe Live or TestCafe for CI

TestCafe Live is designed to work with tests locally. So use the main testcafe module for CI.

How avoid performing authentication actions every run

If you test a page with a login form, you need to enter credentials at the beginning of each test. TestCafe provides User Roles to impove your experience here.

At first, create a user role with authentication steps in a separate file and export it:

// roles.js

import { Role, t } from 'testcafe';

export loggedUser = Role('http://mysite/login-page', async t => {
  await t
    .typeText('#login-input', 'my-login')
    .typeText('#password-input', 'my-input')
    .click('#submit-btn');
}, { preserveUrl: true });

Import the role into a file with tests and use it in the beforeEach function:

// test.js
import { loggedUser } from './roles.js';

fixture `Check logged user`
    .page `http://mysite/profile`
    .beforeEach(async t => {
      await t.useRole(loggedUser);
    });
    
test('My test with logged user', async t => {
    // perform any action here as a logged user
});

When you run tests with TestCafe Live for the first time, role initialization steps will be executed and your tests will run with an authorized user profile. If you change the test file, the next run will skip role initialization and just load the page with saved credentials. If you change code in a role, it will be 'refreshed' and role initialization steps will be executed at the next run.

I'd like to make changes in several files before running tests

Just focus your terminal and press ctrl+p. TestCafe Live will not run tests until your press ctrl+r.

Feedback

Report issues and leave proposals regarding this 'live' mode in this repository. Please address all issues about TestCafe to the main TestCafe repository. If you like this mode please let us know. We will be glad to hear your proposals on how to make it more convinient. Feel free to share your experience with other developers.

Author

Developer Express Inc. (https://devexpress.com)

testcafe-live's People

Contributors

alexandermoskovkin avatar andreybelym avatar danielgek avatar hdorgeval avatar tomardern avatar vasilystrelyaev avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

testcafe-live's Issues

Feature: Allow Use of TestCafe Live

Summary
Allow users to use TestCafe Live when executing tests.

Expected Outcome
A user should be able to specify which runner to use.

I'm sending up a PR right now.

TypeError: process.stdin.setRawMode is not a function

npm run testcafe-live

> [email protected] testcafe-live D:\...
> testcafe-live firefox tests/


Stopping TestCafe Live...
D:\...\node_modules\testcafe-live\lib\index.js:38
process.stdin.setRawMode(true);
              ^

TypeError: process.stdin.setRawMode is not a function
    at Object.<anonymous> (D:\...\node_modules\testcafe-live\lib\index.js:38:15)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (D:\...\node_modules\testcafe-live\bin\testcafe-live.js:5:1)
    at Module._compile (module.js:569:30)

fix is in #17

Testcafe-live can run interface file?

I try to run script:

> testcafe-live index.interface.js

I receive output

Test run is starting...

Make changes in the source files or press ctrl+r to restart test run.
ERROR: Error: Unable to find the browser. "index.interface.js" is not a browser alias or path to an executable file.

Test code debugging when using testcafe-live

Hey! thanks for contributing this lib! I'm evaluating TestCafe / Cypress to adopt it at our company and the live runner makes a big difference (to us)!

Was wondering about debugging 'test code' when running tests through testcafe-live.
When using testcafe binary we do so by setting the flag --inspect-brk. Any way of achieving this with testcafe-live?

Thanks!

Testcafe-live does not reflect changes

cloned from DevExpress/testcafe#2115 (created by @hdorgeval)

Are you requesting a feature or reporting a bug?

bug

What is the current behavior?

When changing code outside the test file, testcafe-live restarts the test but saved modifications seems to be ignored.

What is the expected behavior?

testcafe-live should reflect the most recent save

How would you reproduce the current behavior (if this is a bug)?

Clone the github repo testcafe-starter
rename the file "testcafe-sample-page.spec.ts" to "testcafe-sample-page.live.ts"
Set .only to the test "Scenario: can send feedback with my name only"
run the command npm run test:live
Open the file "i-enter-my-name.ts"
Comment the line ".typeText(...."
Save the file

=> testcafe-live restart the test, but text is still typed in the field

Provide the test code and the tested page URL (if applicable)

https://github.com/hdorgeval/testcafe-starter

Tested page URL:

Test code

Specify your

  • operating system: windows 10 & MacOSX
  • testcafe version: 0.18.6
  • node.js version: 9.2.0

Error when using testcafe-angular-selectors

Hi, all.

My test case failed at waitForAngular() call when I use testcafe-live, and it's okay if I use testcafe.

My test case is very simple:

import {waitForAngular} from "testcafe-angular-selectors";

fixture `Getting Started`
  .page `http://www.google.com/`
  .beforeEach(async () => {
    await waitForAngular();
  });

test ('My first test', async t => {
  await t;
});

When I run it with testcafe-live, the error occurs:

1) - Error in fixture.beforeEach hook -
  Error: Cannot prepare tests due to an error.

  ClientFunction cannot implicitly resolve the test run in context of which it should be executed. If you need to call ClientFunction from the Node.js API callback, pass the test controller manually via ClientFunction's `.with({ boundTestRun: t })` method first. Note that you cannot execute ClientFunction outside the test code.

Feature: export API, or add watch option to the testcafe API

Something like

    const failedCount = await [isCI ? testcafe : testcafeLive] // same api
        .createRunner()
        .screenshots(path.join(logPath, 'testcafe'))
        .src(files)
        .browsers(`chrome${headless || isCI ? ':headless --no-sandbox' : ''}`)
        .reporter('json', stream)
        .reporter('spec')
        .run();

or, preferably, integrated with testcafe:

    const failedCount = await testcafe
        .createRunner({ watch: !isCI }) // new watch option
        .screenshots(path.join(logPath, 'testcafe'))
        .src(files)
        .browsers(`chrome${headless || isCI ? ':headless --no-sandbox' : ''}`)
        .reporter('json', stream)
        .reporter('spec')
        .run();

unhandled promise rejection during start with newest [email protected]

[email protected]

$ npm --version
5.7.1

$ node --version
v8.12.0

Windows 7

$~\tmp\systemtest-testcafe-example>npm run test-chrome:live --verbose
npm info it worked if it ends with ok
npm verb cli [ 'C:\\Program Files\\nodejs\\node.exe',
npm verb cli   'C:\\Users\\example\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js',
npm verb cli   'run',
npm verb cli   'test-chrome:live',
npm verb cli   '--verbose' ]
npm info using [email protected]
npm info using [email protected]
npm verb run-script [ 'pretest-chrome:live',
npm verb run-script   'test-chrome:live',
npm verb run-script   'posttest-chrome:live' ]
npm info lifecycle [email protected]~pretest-chrome:live: [email protected]
npm info lifecycle [email protected]~test-chrome:live: [email protected]

> [email protected] test-chrome:live C:\Users\example\test-puppeteer\tmp\systemtest-testcafe-example
> testcafe-live --speed 1 --assertion-timeout 1000 --selector-timeout 10000 --page-load-timeout 10000 chrome tests/*.js

(node:5312) UnhandledPromiseRejectionWarning: Error: watch tests/*.js ENOENT
    at FSWatcher.start (fs.js:1382:19)
    at Object.fs.watch (fs.js:1408:11)
    at FileWatcher.addFile (C:\Users\example\test-puppeteer\tmp\systemtest-testcafe-example\node_modules\testcafe-live\lib\file-watcher\index.js:53:38)
    at FileWatcher.files.forEach.f (C:\Users\example\test-puppeteer\tmp\systemtest-testcafe-example\node_modules\testcafe-live\lib\file-watcher\index.js:18:33)
    at Array.forEach (<anonymous>)
    at new FileWatcher (C:\Users\example\test-puppeteer\tmp\systemtest-testcafe-example\node_modules\testcafe-live\lib\file-watcher\index.js:18:15)
    at Controller._initFileWatching (C:\Users\example\test-puppeteer\tmp\systemtest-testcafe-example\node_modules\testcafe-live\lib\controller.js:52:29)
    at Controller.init (C:\Users\example\test-puppeteer\tmp\systemtest-testcafe-example\node_modules\testcafe-live\lib\controller.js:24:14)
    at tcArguments.parse.then (C:\Users\example\test-puppeteer\tmp\systemtest-testcafe-example\node_modules\testcafe-live\lib\index.js:16:28)
    at <anonymous>
(node:5312) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:5312) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

publish on npm

The latest package on npm is 2 commits behind on this repo. Thanks

Ability to specify TestCafe Live in the runner class

Are you requesting a feature or reporting a bug?

Feature

What is the current behavior?

For TestCafe users that are using CucumberJS to drive tests it's difficult to use TestCafe Live. I've already tried using concurrently to run Cucumber and TestCafe-Live in parallel, but I get the following error.

Stopping TestCafe Live...
[0] C:\Users\rquellhorst\Desktop\testcafe-cucumber\node_modules\testcafe-live\lib\index.js:38
[0] process.stdin.setRawMode(true);
[0]               ^
[0]
[0] TypeError: process.stdin.setRawMode is not a function
[0]     at Object.<anonymous> (C:\Users\rquellhorst\Desktop\testcafe-cucumber\node_modules\testcafe-live\lib\index.js:38:15)
[0]     at Module._compile (module.js:635:30)
[0]     at Object.Module._extensions..js (module.js:646:10)
[0]     at Module.load (module.js:554:32)
[0]     at tryModuleLoad (module.js:497:12)
[0]     at Function.Module._load (module.js:489:3)
[0]     at Module.require (module.js:579:17)
[0]     at require (internal/module.js:11:18)
[0]     at Object.<anonymous> (C:\Users\rquellhorst\Desktop\testcafe-cucumber\node_modules\testcafe-live\bin\testcafe-live.js:5:1)
[0]     at Module._compile (module.js:635:30)
[0] npm ERR! code ELIFECYCLE
[0] npm ERR! errno 1
[0] npm ERR! [email protected] testcafe-live: `testcafe-live.cmd chrome test.js`
[0] npm ERR! Exit status 1
[0] npm ERR!
[0] npm ERR! Failed at the [email protected] testcafe-live script.

What is the expected behavior?

It would be nice to specify that you want to use TestCafe-Live in the runner class.

How would you reproduce the current behavior (if this is a bug)?

You could pull my repo to see my code.

Provide the test code and the tested page URL (if applicable)

NA

Specify your

  • operating system: Windows 10.0.0
  • testcafe version: 0.19.0
  • testcafe-live version: 0.1.3
  • node.js version: 0.8.3

Possible to run testcafe-live via Node?

Hi,
Is it possible to run testcafe-live version via node?
I would like to start my runner class from CMD ">node tRunner.js chrome" but I would actually like to start the live version. I.e. I would like to monitor the test via devtools.

What should I write in my tRunner.js file in order to starta the testcafe-live version?
return runner
.src('./tTest.js')
.browsers([args])

TestCafe documentation

Hi,

I am a beginner in test automation with testcafe. Sometimes I run into some problems and I try to find some documentation to solve my issues but I find it very hard. Maybe I am just missing some point here but I feel that it is better to ask than not to know :-).

Example: I checked this testcafe page and there was this example on how to run some browser with some extra arguments, like writing "testcafe chrome test.js --new-window"
http://devexpress.github.io/testcafe/documentation/using-testcafe/programming-interface/runner.html#using-browser-aliases

runner.browsers({
path: 'C:\Program Files\Google\Chrome\Application\chrome.exe',
cmd: '--new-window'
});

Now my question is, assume this example was not there, how would I know this is possible to write in this way? I mean I will run into other issues and where can I read that this is possible to write? What other option can I write there?

Usually I run my tests both with "testcafe chrome" option from CMD but also via Node "node runner.js chrome". But also here knowing how to write some command in the runner.js file which I know who to execute via testcafe option is, for me, quite difficult. I spend a lot of time trying to figure out these question.

Would you please explain where I find the neccessary documentation in order to find out that info myself?

Error while navigating through sign-in

Hi All,

I have a authentication form which asks for username and password, then the url in the window is sent to another api to validate and thats when the error occurs

The expected url by the API from the browser window,
http://www.x.com

But the url received by the API when I use test-cafe,
http://192.xxx.xxx.xxx/qsqsqwwqwd/http://www.x.com

Give me a solution for this. I cant surely change the API's behavious, should happen from the testcafe end.

I am getting the below error as response -
"status":403,"error":"Forbidden","message":"Invalid CSRF Token '' was found on the request parameter '_csrf' or header 'X-C'.","path":"/api/login"

Watching stops after first change

I have a strange behavior here. I run testcafe-live --page-load-timeout 60 -Ss 'tmp/testcafe/screenshots' -a 'bin/webpack' chrome e2e/

This works very well. The tests run through. After that I change my test file and the tests automatically run again. But when I change the file again, nothing happens until I restart testcafe-live.

Do you have any idea how this could happen?

I'm thankful for any advice :)

TypeError when testcafe is stopping

Stopping TestCafe Live...
/home/taupan/nobackup/git/myproject/node_modules/testcafe-live/lib/test-run-controller.js:175
                    testWrapper.testRunWrappers.forEach(w => w.finish());
                                                               ^

TypeError: w.finish is not a function
    at testWrapper.testRunWrappers.forEach.w (/home/taupan/nobackup/git/myproject/node_modules/testcafe-live/lib/test-run-controller.js:175:64)
    at Array.forEach (<anonymous>)
    at Timeout.setTimeout (/home/taupan/nobackup/git/myproject/node_modules/testcafe-live/lib/test-run-controller.js:175:49)
    at ontimeout (timers.js:475:11)
    at tryOnTimeout (timers.js:310:5)
    at Timer.listOnTimeout (timers.js:270:5)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

(This is an actual backtrace, but I redacted the directory name to be "myproject" .)

I see these kinds of errors sometimes when testcafe-live sees a test failure, in which case they occur persistently with every run.

Command line is

cross-env DJANGO_SETTINGS_MODULE=myproject.settings_testcafe testcafe-live chromium myproject/js/testcafe/"test_*.js" --hostname localhost --screenshots myproject/js/testcafe/screenshots --quarantine-mode --screenshots-on-fails --app-init-delay 3000 --app "yarn testcafe-db; yarn testcafe-server --noreload || sleep 2; netcat -z -w 2 localhost 8003" --concurrency 8

(Edit: Inserted command and browser.)

Maximum call stack size exceeded

When I run TestCafe Live, the test runs fine but then as soon as I make a change in a file it crashes:

Make changes in the source files or press ctrl+r to restart test run.

Stopping TestCafe Live...
.../node_modules/lodash/lodash.js:12148
    function isSymbol(value) {
                     ^

RangeError: Maximum call stack size exceeded
    at isSymbol (.../node_modules/lodash/lodash.js:12148:22)
    at isKey (.../node_modules/lodash/lodash.js:6320:28)
    at castPath (.../node_modules/lodash/lodash.js:4471:14)
    at hasPath (.../node_modules/lodash/lodash.js:6132:14)
    at Function.has (.../node_modules/lodash/lodash.js:13150:32)
    at Graph.hasNode (.../node_modules/graphlib/lib/graph.js:169:12)
    at ModulesGraph.addNode (.../node_modules/testcafe-live/lib/file-watcher/modules-graph.js:32:24)
    at children.forEach.child (.../node_modules/testcafe-live/lib/file-watcher/modules-graph.js:25:22)
    at Array.forEach (<anonymous>)
    at ModulesGraph._updateChildren (.../node_modules/testcafe-live/lib/file-watcher/modules-graph.js:21:22)

I'm using Node 8.9 and TestCafe 0.18.6 on OSX 10.11.

Test run never re-starts despite saying it will

Hey there,
testcafe-live says:

Sources have been changed. Test run is starting...

but the tests never actually re-run.

image

It is stuck like that forever. Any idea of what is going wrong there?

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.