I'm setting up the loki + storybook, I installed loki as follows:

  "devDependencies": {
    "loki": "oblador/loki"
  },
  "loki": {
    "diffingEngine": "looks-same",
    "configurations": {
      "chrome.palm": {
        "target": "chrome.docker",
        "width": 640,
        "height": 480,
        "deviceScaleFactor": 1,
        "mobile": false,
        "fitWindow": false
      }
    }
  }

On CI is throws

     Fetch list of stories
     Test chrome.palm
     Stop
Timeout
Error: Timeout
    at Timeout._onTimeout (/home/circleci/repo/webapp/node_modules/wait-on/lib/wait-on.js:91:10)
    at ontimeout (timers.js:475:11)
    at tryOnTimeout (timers.js:310:5)
    at Timer.listOnTimeout (timers.js:270:5)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Exited with code 1

Here is the log, this time it also does't work on my machine.

My stories were being skipped:

So, after trying several things, I found my story was using an internal utility I built that uses react-jsonschema-form and that import was causing the story to be skipped. If I remove import Form from 'react-jsonschema-form'; and update the code it works:

Also, is there a way to debug to know why stories are being skipped?

Kudos for building a cool visual regression suite. I love it. It's fast with minimal setup and just works.

While testing the tool using the existing react example, I discovered whenever the visual test failed the chrome app did not close.

Machine Specs:
Macbook - OSX - Yosemite.
Chrome Browser - Version 60.0.3112.78 (Official Build) (64-bit).

All our developers speak russian language and we name all our stories with cyrillic symbols

What about using this package for naming files https://www.npmjs.com/package/transliteration ?

please advice what is wrong? thanks

Hi, I have this error on CircleCI (locally everything is fine):

loki test v0.13.0
[14:48:46] Chrome (docker) [started]
[14:48:46] Prepare environment [started]
[14:48:52] Prepare environment [completed]
[14:48:52] Start [started]
[14:49:25] Start [failed]
[14:49:25] → Timeout
[14:49:25] Chrome (docker) [failed]
[14:49:25] → Timeout
Timeout
Error: Timeout

I increased the timeout in docker.js to 30000 as suggested in #38 but this did not fix the issue.

CircleCI is using circleci/ruby:2.5-node image.

Any tips on debugging this issue?

Posting this for discussion would love any type of input here, either do/don't/doesn't matter, do this instead, other considerations, what's most important to you, what you'd be able to help out with etc.

Move to jest

It's hard to write a great test runner with all types of different outputs (like JUnit, coverage etc). Currently we're using Listr to get a slightly nicer output of the progress, but it's flawed because it will swallow console.log output, is limited in how it handles concurrency also sometimes it will just output every frame instead of updating it. So my thought was to not try to innovate/solve all of these problems but use something existing that works well and my thought was to use jest.

Native concurrency

Running native tests are slow due to the screenshot process in the simulator itself being slow. That can be improved in multiple ways, for one we could take the screenshot from within the app itself and feed it over websockets (I've done this and it's fast) but it has the problem of not being accurate in some edge cases and requires integration of native libraries which makes it harder to get started. It should now be possible to run multiple simulators simultaneously but there's currently no way of sending commands to a specific connected device without doing alterations to the code itself. I've described it here: #24 I guess my consideration is if I should give up on being a very light integration or trying to solve it in making storybook more flexible.

Visualisation/management

In my research I didn't find a good OSS library that just handled the diff management well (showing you what the differences were in the browser and enabling you to accept changes). There are some projects, but they either try to solve a bigger problem that are similar but not really the same or they just aren't solving it well enough. There's a few hosted services that one could use, which is probably the best solution for most cases. However there are OSS projects w/o money and sensitive projects where using hosted solutions is difficult for multiple reasons where a self hosted thing would be much easier. At some point I wanted to make a stand alone project (I named it Sigyn) to cater to these needs, but I'm not sure if it's better to focus on solving loki problems first. I'm afraid of that both will hinder loki adoption, either loki itself doesn't get enough attention to be good or the overall experience is not good to be used by devs.

Ping @nowells & @nuc

I'm using gitlabCI as CI server.

Situation is next: locally all tests pass, but on gitlabci - they fail. Diff in attachment. Everything works in Docker so I'm confused. Any ideas?

  "loki": {
    "chromeSelector": ".uil-story-wrapper > *",
    "preset": "Retina Macbook Pro 15",
    "skipStories": "(Row|Col|Map|Calendar|Pagination|ContactCard|TextArea)$",
    "configurations": {
      "chrome.laptop": {
        "target": "chrome.docker",
        "width": 1366,
        "height": 768,
        "deviceScaleFactor": 1,
        "mobile": false,
        "fitWindow": false
      }
    }
  }

Hey.

I installed loki with npm and have found that it's not starting. The error is about that Lighthouse/chrome-launcher/random-port is unavailable.

I could try to migrate to 2.5.0 if it's needed.

Hi,
I'm running the command as per docs to run loki in CI (gitlabs) - build-storybook && loki --requireReference --reactUri file:./storybook-static. It builds ok but when it trys to fetch stories it fails because the server is down - I didn't think we had to run storybook if we used static, is that correct or have I misunderstood?

I'm using react-motion and it always generates new reference images

When a story has blank content, Page.captureScreenshot() waits indefinitely causing a timeout. Instead, an error should be thrown immediately.

Hi,

I'm unable to run loki in my web storybook. I'm able to run it react-native storybook though.

I can run the react example fine. I tried downgrading to storybook 3.1.6 - 3.1.7, but that didn't help either.
I've tried 3.2.16 & 3.3.5

loki update v0.8.0
 ❯ Chrome (docker)
   ✔ Start
   ✖ Fetch list of stories
     → Cannot read property 'prototype' of undefined
     Test chrome.laptop
     Test chrome.iphone7
     Stop
Cannot read property 'prototype' of undefined
Error: Cannot read property 'prototype' of undefined
    at createError (/Users/julius/workspace/shared-components/storybooks/web/node_modules/loki/src/targets/chrome/fetch-storybook.js:10:14)
    at Worker.worker.on.message (/Users/julius/workspace/shared-components/storybooks/web/node_modules/loki/src/targets/chrome/fetch-storybook.js:33:16)
    at Worker.emit (events.js:164:20)
    at ChildProcess.Worker.process.on (internal/cluster/worker.js:28:12)
    at ChildProcess.emit (events.js:159:13)
    at emit (internal/child_process.js:790:12)
    at process._tickCallback (internal/process/next_tick.js:152:19)
error Command failed with exit code 1.

I tried with and without docker, but I always get the same error.
Any idea what I should try?

Is there some known addon that might cause problems with loki?

I'm trying to make screenshot more specific. However if chromeSelector differs from #root > * -
it doesn't work

→ Failed with "Error: Unable to get position of selector "#root .SplitPane > *". Review the `chromeSelector` option and make sure your story doesn't crash." after 4 tries

    "chromeSelector": "#root .SplitPane > *",

Could someone share an example of specifying it right to component? (to iframe in an ideal case)

I'm receiving a 'Cannot read property 'react-native' of undefined' when trying to initialise. My project is in React.

Would be awesome to have a .prettierrc to use it with code editors or IDEs that support prettier and immediately get the code formatted as soon as you save.

Prettier / Configuration File
https://prettier.io/docs/en/configuration.html

The current way is not bad at all, but running yarn format manually gets annoying after a while =P

Steps to reproduce:

1. create a 150x150 blue square component.
2. run loki update
3. change the size to 300x300
4. run loki test
   Expected: test should fail
   Actual: test is passing

Hi,

I've been messing around with loki running locally and found it super easy to get up and running.

I'm now looking to get it working as part of a CI pipeline. I am using the command found on your docs for starters (build-storybook && loki --requireReference --reactUri file:./storybook-static) but I'm getting an error that 1 request failed to load. The error messages don't really give me that much to go on, any suggestions/help would be greatly appreciated.

Is it possible or not? (Or feature request, if not) - add class / detect by user-agent and so on.

What for it might be used - to ignore/disable particular areas on screen shot ( set opacity for example )

Recently loki started showing strange behaviour:
loki test complete without an error, also the the difference folder is empty. But if I do loki approve afterwards, it suddenly changes files in the reference folder.
This also occurs if the tests fail, and I approve afterwards: All failed images get moved to references but also some other images suddenly get changed in the reference folder.

v.0.12.0 with following options:
--chromeRetries 3 --diffingEngine gm --requireReference --chromeTolerance 1.0 --chromeConcurrency 10 --verboseRenderer true --chromeDockerImage yukinying/chrome-headless:63.0.3230.2

You can easily create a HTML report using reg-cli:

npm install -g reg-cli
cd .loki
reg-cli ./current/ ./reference/ ./reg-diff/ -R ./report.html
cp -r difference/ reg-diff # Use loki's diffs
open report.html

This will generate a report that looks something like this

everything is in the title ... i made a project to let you test it :
https://github.com/erochOnGit/reactstorybookdockerRxJS

Tell me if you have some issue with the setup
i'm using node 8.* and loki 0.6.0

Hey there,

Although locally the tests are green, when running on CircleCI we get that error for every single file.

That string doesn't exist inside loki's codebase. Any ideas on what might be causing it?

Thanks!

Hey,
When using with RN i get an accessing nonextisting addons channel issue.

Looks like the channel is not set fast enough.

loki/configure-react-native.js

increasing setTimeout from 1 to 1000 fixes the issue.

Is there an easy way to test print sylesheets with loki?

Hi,

any plans on adding an option to set a threshold for tests to change the sensitivity of diff tests?

thanks for cool lib :)

Right now when story doesn't have reference, the test simply pass. A warning could be really helpful so people wouldn't forget to add new references for stories.

There is a scenario:

I have added a new story that has no reference files. At the same time as adding maybe unintentionally regress another story, this would now fail loki test.

Before running loki test I do loki update - this then adds the new reference for my new component but also replaces the old reference for the now broken component meaning I could accidentally introduce regression and not be aware of it.

Suggestion: Add a flag that I can pass into loki update that will only add references for those stories which currently have no reference images.

0.6.2 works fine on CI, but on my machine I can only test and update reference images with v0.6.2

0.6.2 breaks with:

loki test v0.6.2
 ❯ Chrome (docker)
   ✖ Start
     → Timeout
     Fetch list of stories
     Test chrome.palm
     Stop
Timeout
Error: Timeout
    at Timeout._onTimeout (/home/felquis/pagarme/pilot/webapp/node_modules/wait-on/lib/wait-on.js:91:10)
    at ontimeout (timers.js:488:11)
    at tryOnTimeout (timers.js:323:5)
    at Timer.listOnTimeout (timers.js:283:5)
error Command failed with exit code 1.

uname -a 4.10.0-35-generic #39~16.04.1-Ubuntu SMP Wed Sep 13 09:02:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
node -v 8.1.2

Hi!

This is freaking awesome @oblador !
I'd be interested in your opinion on improving the integration with storybook. What can we do to make this better on our side?

♥️ - storybook team

lokiSkip() and asyc are pretty handy functions, but they break other storybook plugins:
e.g. the storyshots plugin, which complains, that lokiSkip isn't a function.
this happens because loki just registers this functions when it is configuring storybook.
All other plugins, don't use the loki config-storybook and will therefor break.

Please find another possibility to add this functions (just register a normal storybook plugin?) or keep the cli option to exclude stories.

First of all, thanks for your wonderful addon!
I found a problem with empty components. If a component returns null and a story renders it, loki fails with this error:

Unable to get position of selector "#root > *". Review the `chromeSelector` option and make sure your story doesn't crash.

If an error occurs while capturing screenshots (e.g. the selector can't be found), then the browser tab stays open.

When enough tabs are open, screenshot capturing stops working and timeouts start happening.

Hi, it may help you and maybe you could help me too

I'm runnign this branch shooting-stats you can see .travis.yml in there, using travis it exits 1 with:

docker: Error response from daemon: linux seccomp: seccomp profiles are not supported on this daemon, you cannot specify a custom seccomp profile.

💯 🔥 👍

Hi, we're trying to get the latest version of Loki working with React & Lerna, but when running yarn loki update -- --port 8080 I get:

somewhere:react-project ebmocces$ yarn loki update -- --port 8080
yarn loki v0.27.5
$ "/src/react-project/node_modules/.bin/loki" "update" "--port" "8080"
loki update v0.7.0
 ✔ Chrome (app)
Error: Uncaught [TypeError: source.close is not a function]
    at reportException (/src/uitk-react/node_modules/jsdom/lib/jsdom/living/helpers/runtime-script-errors.js:66:24)
    at Timeout.callback (/src/uitk-react/node_modules/jsdom/lib/jsdom/browser/Window.js:544:7)
    at ontimeout (timers.js:488:11)
    at tryOnTimeout (timers.js:323:5)
    at Timer.listOnTimeout (timers.js:283:5) TypeError: source.close is not a function
    at handleDisconnect (evalmachine.<anonymous>:42296:12)
    at Window.<anonymous> (evalmachine.<anonymous>:42271:7)
    at Timeout.callback (/src/uitk-react/node_modules/jsdom/lib/jsdom/browser/Window.js:542:19)
    at ontimeout (timers.js:488:11)
    at tryOnTimeout (timers.js:323:5)
    at Timer.listOnTimeout (timers.js:283:5)
Done in 22.00s.

Any ideas? I've tried looking into jsdom to see what's causing it to fall over, but can't seem to see anything.

Node v8.1.2
React v16.0.0
Lerna v2.4.0

Adding https://github.com/GoogleChrome/puppeteer as a target (easily installable via npm) should make this work out-of-the-box in most dev and CI platforms.

When I run loki update on my computer monitor, I get updated reference images and my tests pass. When I unplug from my monitor and run loki test on my laptop, the tests fail. My monitor is larger than my computer. Any ideas about why this might be?

I'm getting this in my own app as well as when I run the example app.

loki update v0.7.0
 ❯ Chrome (docker)
   ✔ Start
   ✖ Fetch list of stories
     → Could not parse url argument "?" to pushState against base URL "about:blank".
     Test chrome.laptop
     Stop
Could not parse url argument "?" to pushState against base URL "about:blank".
SecurityError: Could not parse url argument "?" to pushState against base URL "about:blank".
    at HistoryImpl._sharedPushAndReplaceState (/Users/athomson/sandbox/nova/luna/node_modules/jsdom/lib/jsdom/living/window/History-impl.js:73:15)
    at HistoryImpl.pushState (/Users/athomson/sandbox/nova/luna/node_modules/jsdom/lib/jsdom/living/window/History-impl.js:54:10)
    at History.pushState (/Users/athomson/sandbox/nova/luna/node_modules/jsdom/lib/jsdom/living/generated/History.js:89:21)
    at evalmachine.<anonymous>:65329:20
    at Object.dispatch (evalmachine.<anonymous>:50529:7)
    at ConfigApi._renderError (evalmachine.<anonymous>:63464:24)
    at render (evalmachine.<anonymous>:63483:19)
    at ConfigApi.configure (evalmachine.<anonymous>:63498:9)
    at Object.<anonymous> (evalmachine.<anonymous>:57457:22)
    at Object.<anonymous> (evalmachine.<anonymous>:57458:30)

Any ideas?

https://github.com/oblador/loki#documentation
the following links are dead (404)
Introduction
Getting started
Configuration
Command line arguments

The :latest tag on npm points to an older version (0.6.5).

.

Thanks!

Hi,
I am running storybook on port 6006 and i get this error when running:
yarn loki update
'loki/configure-react' is already imported after i ran loki init

I am using node.js v7.6.0.

When I run any of loki commands I'm getting following error:

setImmediate is not defined

Is this error assumes that I possibly forgot on something during setup?

[email protected]

Hi,

What is the best way to run loki in CI like CircleCI or Codeship?

Can it run across different simulators/emulators?

Thanks,
Leena

I'm trying out the new alpha version of Storybook, but I can't get Loki to work with it. I'm wondering if this error message might be a red herring, and something is actually breaking inside the addon.

Are there known compatibility issues with the latest Storybook alpha? They do seem to have changed some APIs.

I'm trying to start using loki, but get error making reference images:

Unexpected token <
Error: Unexpected token <
    at createError (/Users/varyastepanova/WebDev/SC5/design-systems/ds-workshop/ds-workshop-storybook-boilerplate/node_modules/loki/src/targets/chrome/fetch-storybook.js:10:14)
    at Worker.worker.on.message (/Users/varyastepanova/WebDev/SC5/design-systems/ds-workshop/ds-workshop-storybook-boilerplate/node_modules/loki/src/targets/chrome/fetch-storybook.js:33:16)

[email protected] (also reproduced this at [email protected])

I am getting a timeout error when trying to run loki in gitlabs CI.

Timed out waiting for Chrome Debugger to appear on 172.x.x.x:42676

Using this version of Loki: #90

.gitlab-ci.yml

visual-regression-test:
  <<: *apply_runner_tags
  <<: *skip_on_tags
  stage: test
  image: an-internal- (this image installs node 8.x and installs docker via apk add --update --no-cache docker)
  script:
    - cd packages/ui-components
    - docker info
    - npm run vr:ci-test (build-storybook && loki --requireReference --reactUri file:./storybook-static)

loki config

  "loki": {
    "storiesFilter": "^(?:(?!(Spinner)|(Carousel)|(Scenes)).)*$",
    "configurations": {
      "chrome.laptop": {
        "target": "chrome.docker",
        "width": 1200,
        "height": 768
      },
      "chrome.iphone7": {
        "target": "chrome.docker",
        "preset": "iPhone 7"
      }
    }
  }

I believe it comes from this line https://github.com/oblador/loki/pull/90/files#diff-b75c1e1e6b0fa125e76a9f03fe7716c5R171, it seems that maybe to it thinks its found docker (or maybe it has) but the ip/port is wrong.

I've a React Native app setup with Expo. I am trying to setup Loki for the same but it is failing with the following error:

Failed preparing for loki. Make sure the app is configured and running in storybook mode.

I've both the simulator and story-book running. The Storybook app works well in the simulator too. Any clue on why it is failing?

First of all, thank you for this great tool, I really like it!

However, I'm having difficulties to make it run in GitlabCI. Target chome.docker requires Docker to be installed because it's pulling it own docker image and this does not combine well with GitlabCI.
That means I need to provide to my CI Job a Docker image with Node and Docker installed.
Since my skills with docker are limited, I was not able to make it work so far.

storybook-test:
  stage: test
  tags: [docker]
  image: a-docker-image/with-node-and-docker
  script:
    - npm i
    - npx build-storybook -s ./public
    - npx loki --requireReference --reactUri file:./public

Am I missing something? Do you have any advice to make it work on GitlabCI?

nb: a-docker-image/with-node-and-docker is not a real image, this is just ti illustrate the snippet

HI. Thanks for a awesome tool!

I have a problem that character string of the created image is missing.

I expected to get

If I change chrome/create-chrome-target.js as following, this problem is not occured.

       try {
         await withTimeout(options.chromeLoadTimeout)(tab.loadUrl(url));
       } catch (err) {
         if (err instanceof TimeoutError) {
           debug(`Timed out waiting for "${url}" to load`);
         } else {
           throw err;
         }
       }
  
+     const sleep = msec => new Promise(resolve => setTimeout(resolve, msec));
+     await sleep(2000)

       const screenshot = await tab.captureScreenshot(selector);
       await fs.outputFile(outputPath, screenshot);

Could you please tell me if you know something?

loki version

• loki-0.7.0

Script command

build-storybook && loki --chromeConcurrency 1 --reactUri file:./storybook-static

loki property in package.json

  "loki": {
    "configurations": {
      "chrome.laptop": {
        "target": "chrome.docker",
        "width": 1366,
        "height": 768,
        "deviceScaleFactor": 1,
        "mobile": false,
        "fitWindow": false
      }
    }
  }

Console log

...
 ❯ Chrome (docker)
   ✔ Start
   ✔ Fetch list of stories
   ❯ Test chrome.laptop
     ❯ DailyCard
       ✔ Summary               <=== Missing character strings (100%)
       ✔ Appearance           <=== Missing character strings (100%)
       ✔ Sort                       <=== Missing character strings (100%)
       ✔ Time lamps           <=== Missing character strings (60%)
       ✔ Feed icon              <===  Fine
       ✔ Feed color             <===  Fine
       ✔ Milestone              <===  Fine
       ✔ Seal                       <===  Fine
       ✔ Warning                <===  Fine
       ✔ Life status              <===  Fine
       ✔ Unknown estimates      <===  Fine
...

hello,
I have my storybook running on localhost:9001
I run yarn loki init in my react project
then yarn loki update -- --port 9001
and loki tell me :
yarn loki v0.27.5
$ "/home/emile/projet/front-commerce/node_modules/.bin/loki" "update" "--port" "9001"
loki update v0.5.3
❯ Chrome (docker)
✔ Start
✖ Fetch list of stories
→ Cannot read property 'assign' of undefined
Test chrome.laptop
Stop
TypeError: Cannot read property 'assign' of undefined
error Command failed with exit code 1.

could you help me to find out what's going wrong ?