Possible bug introduced in beta 59?

Renovate bot is trying to update to @percy/cli to v1.0.0-beta.59 and our action that runs E2E tests and take screenshots is now falling with the following error:

(node:1969) Error Plugin: @percy/cli: can't resolve reference /snapshot#/properties/name from id /config/static
module: @oclif/[email protected]
task: runHook init
plugin: @percy/cli
root: /home/runner/work/montevideo/montevideo/node_modules/@percy/cli

The same action was running properly on beta.58

Our script just runs: percy exec -- cypress run (and some cypress env variables are used too with the --env flag)

When using the percy upload command, the file extension is kept in snapshot names. This is because you can have multiple files with the same name but different extensions, i.e. image.png, image.jpg, image.jpeg. If file extensions were automatically removed, only the first upload will work and the following uploads will return a duplicate snapshot API error.

However, having matching filenames with differing extensions probably isn't very common. More likely what is common, is the filenames without extensions are intended to be snapshot names. With intentional names, the duplicate snapshot API error is doing its job.

Rather than change the behavior and suddenly rename all existing snapshots for current users (resulting in "new" snapshots and broken comparisons), we can add an opt-in flag and/or config file option to strip extensions from the resulting snapshot name. This could be a boolean option, or maybe a more complicated config option for more custom behaviors.

I'm facing an error using percy cli command that probabilistic occurrence.

I'm currently taking dozens of snapshot using via percy command from @percy/cli 1.0.0-beta.58 and percySnapshot function from @percy/puppeteer 2.0.0.

My command is below:

percy exec -- node percy.js

percy.js javascript file contains puppeteer scripts like

• load cookies by page.setCookie
• login by HTML form
• swap various image to placeholder
• swap various text to placeholder
• wait and hide ad iframe

Example error message is below:

/app/node_modules/@percy/core/dist/page.js:367
    let error = new Error();
                ^

Error: Protocol error (Fetch.continueWithAuth): Page closed.
    at Page.send (/app/node_modules/@percy/core/dist/page.js:367:17)
    at Page.<anonymous> (/app/node_modules/@percy/core/dist/network.js:78:23)
    at Page.emit (node:events:394:28)
    at Page._handleMessage (/app/node_modules/@percy/core/dist/page.js:411:12)
    at Browser._handleMessage (/app/node_modules/@percy/core/dist/browser.js:314:56)
    at WebSocket.<anonymous> (/app/node_modules/@percy/core/dist/browser.js:116:40)
    at WebSocket.emit (node:events:394:28)
    at Receiver.receiverOnMessage (/app/node_modules/ws/lib/websocket.js:834:20)
    at Receiver.emit (node:events:394:28)
    at Receiver.dataMessage (/app/node_modules/ws/lib/receiver.js:437:14)

This error does not happen always, but happens so frequently.
I believe this problem has started to occur in the last week or so.
I'm trying to revert my dependency to @percy/cli 1.0.0-beta56 or more older version.

My question is below:

• What means this error?
• Why happens it?
• How can I fix it?
• Or could you fix it?

Weird, it seemed to work the other day in another package, did a copy paste and am now getting this error:

// package.json
"scripts": {
    "serve:insecure": "webpack serve --config webpack.slice.ts --env=devMode=true",
    "test:visual": "start-server-and-test serve:insecure http-get://localhost:8080 test:percy",
    "test:percy": "percy exec -- jest .visual.test.*$ --forceExit"
},
"devDependencies": {
    "@percy/cli": "^1.0.0-beta.49",
    "@percy/puppeteer": "^2.0.0",
    "puppeteer": "^9.1.1",
    "start-server-and-test": "^1.12.2"
}

// Session.visual.test.ts
import percySnapshot from '@percy/puppeteer';
import { launch } from 'puppeteer';

describe('Visual Regression: Session', () => {
  it('Displays the welcome page by default', async () => {
    const browser = await launch();
    const page = await browser.newPage();
    await page.goto('http://localhost:8080');
    await percySnapshot(page, 'Session: Welcome');
  });
});

CLI Output

｢wdm｣: Compiled successfully.
[percy] Unsupported Percy CLI version, disabling snapshots
[percy] stopping percy...
[percy] waiting for 0 snapshots to complete...
[percy] done.

Any ideas of what I should be doing instead?

It's fairly common for some advertising or tracking scripts to inject content into the head such as iframes or images. iframes are allowed in head elements if they themselves only contain meta content elements, but otherwise anything that is not a meta content element is not allowed in a head element.

When re-rendering, browsers insert an implicit </head> before invalid content which pushes any following elements, meta content or not, into the body. Usually resulting in a broken page.

We currently remove all iframes from the head since they do not influence the page visually. There is a very short list of allowed elements in headers. https://developer.mozilla.org/en-US/docs/Web/HTML/Element/head#See_also

Should we start by removing images and eventually expand that to include other common elements? Or should we iterate through head elements and remove any that are not explicitly allowed?

This was working find till [email protected]. Chalk people recently upgraded to version 4.1.1 and percy/cli started failing with this new version. Error - No matching version found for [email protected].
Raising the issue here as I could find the package on package published on npm.

Percy config files get automatically validated so helpful warnings can be shown about possibly non-existent options and also so unexpected options do not make their way into the config objects. We should do the same for page listings when using percy snapshot.

We recently updated our app from @percy/agent to use the CLI package. The docs for skipping downloading Chromium appear to no longer accurate (or are solely for the agent package).

I tried searching for PUPPETEER_SKIP_CHROMIUM_DOWNLOAD and PUPPETEER_EXECUTABLE_PATH in this repo and it looks like they recently disappeared in the last two weeks. The CLI fails to find our previously installed browser because (I think?) the code assumes a fixed location for the pre-existing Chrome/Chromium executable.

This is a blocker for us to update because in addition to the extra bulk of downloading Chromium, the automated download doesn't work on Alpine Linux.

I think this probably works if I can symlink our existing executable (currently installed at /usr/bin/chromium-browser) to the location that the package expects. Would that be enough? Where would we point it the symlink to work around this?

Thanks for your help.

I'm getting the following error:

const client = new PercyClient(options);
                 ^

TypeError: PercyClient is not a constructor

When I run:

const PercyClient = require('@percy/client');

function run() {
  const client = new PercyClient();
};

run();

Please advise. Thank you!

Hello Percy team!

We currently run our browser-based tests using Chrome in a separate container from our application. There's a few reasons for doing this, including making it easy for us to swap out different versions of Chrome for testing, minimizing Docker image size, etc. We currently end up installing Chromium in order to have something present for asset discovery, but it's only used by Percy.

It looks like you're now controlling the browser over a websocket now as you're moving away from being coupled to Puppeteer. Would you be open to a change where a user could supply an environment variable like PERCY_REMOTE_BROWSER_URL and when that is present, the executable-launching (and installing) steps would be bypassed and the URL would be used to connect here:

This would save us a 130MB of install in some of our CI images but also would let us get rid of a lot of complexity, as Chrome and Chromium are kind of awkward to use in Alpine Linux.

Thanks for your help.

Hello, I see Percy is using dotenv to read .env variables, in the rails way which is very useful.
However, the ruby dotenv implementation also does variable interpolation.
Example:

USER=postgres
DATABASE_TEST_URL="postgres://$USER:@localhost/db"

Could you add it, e.g. with https://github.com/motdotla/dotenv-expand ?

Thank you.

This is a followup to #329.

Cypress version 5.0 now support test retries, so a test may have multiple attempts meaning cy.percySnapshot may be called multiple times per-test if the test retries after the snapshot
This causes the following error:

[percy] snapshot taken: 'Test retry'
[percy] snapshot taken: 'Test retry'
[percy] StatusCodeError 400 - {"errors":[{"status":"bad_request"},
{"source":{"pointer":"/data/attributes/base"},
"detail":"The name of each snapshot must be unique, and this name already exists in the build: 
'Test retry' -- You can fix this by passing a 'name' param when creating the snapshot. 
See the docs for more info on identifying snapshots for your specific client: 
https://percy.io/docs"}]} 

(see cypress-io/cypress-react-unit-test#362 an example error'd run)

In order to support Cypress test retries, the plugin will need to hold off on sending the DOM snapshot until after the final attempt has run. Likely:

let testSnapshots = {}
snapshot() {
  // ...
  testSnapshots[name] = snapshotData
  // ...
}

// runs for every attempt
Cypress.on('test:after:run:async', (test) => {
  if (test.final) {
    // Cypress adds .final === true on the final attempt of a test
    sendSnapshots(testSnapshots)
	  testSnapshots = {}
  }
})

see the PR to fix cypress-plugin-snapshots as an example of adding tests for multiple Cypress versions meinaart/cypress-plugin-snapshots#141

Previously, asset discovery would outright abort remote resource requests that would never be captured. This behavior was removed in #369 so we can capture the DOM of page while simultaneously discovering resources. We don't want to abort requests in this scenario so the page can be accurately rendered before serializing the DOM.

Another reason not to outright abort remote resources, would be in cases of "local" resources originating from remote resources. For example, say a page has a stylesheet at a cross-origin domain, however that stylesheet includes images or fonts that are present at the root domain. By aborting the cross-origin stylesheet that will not be captured, the same-origin images would never be requested. Think along the lines of a CDN that hosts custom stylesheets for a website.

These remote resources though can sometimes take a while to resolve. And when the resource will not be captured anyway, it blocks asset discovery from completing.

To speed up asset discovery when encountering resources that will not be captured, we should not track them at all. Similar to event stream requests, we can untrack resources that will never be captured. This is better than aborting them completely, and should have the same speed improvements.

Hi guys & gals,

Sorry if this has already been brought up but I couldn't find it in the existing issues.

We use yarn 2 w/ workspaces for our monorepo setup.

The issue i'm having is that @percy/cli lists all it's dependencies as devDependencies instead of dependencies.

Yarn 2 won't install devDependencies so I end up with this:
(When trying to run: yarn percy storybook ./storybook-static --widths=320,768,1024)

I feel like if the package uses these dependencies they should probably be normal dependencies and not devDependencies?

I couldn't find anything in the docs to help with this issue. However, just doing this for example: build-storybook && npx -p @percy/cli@latest -p @percy/storybook@next -p @storybook/router -c 'percy storybook ./storybook-static' --widths=320,768,1024 (using npx) works fine.

Currently, the percy upload command uploads a single snapshot at a time. This command uses @percy/client directly, and so does not have access to the concurrency used in @percy/core.

We can still utilize the core queue class to create an image upload queue and make this process a bit faster for this command.

When trying to use percy exec with Yarn 2, the following is thrown:

[MODULE_NOT_FOUND] Error Plugin: @percy/cli: Qualified path resolution failed - none of the candidates can be found on the disk.

this is likely because you can't necessarily make the assumption that repos will be available, for example: node_modules/@percy/cli-exec/dist/hooks/init won't be correct because they're not under a node_modules folder. However, even with this, you could theoretically install each plugin and yarn unplug it.

I think a biggest problem comes because it can't seem to find the index.js file when pointed at a folder:

Source path: /Users/aaron/Work/web/node_modules/@percy/cli-exec/dist/commands/exec
    Rejected candidate: 
    /Users/aaron/Work/web/node_modules/@percy/cli-exec/dist/commands/exec
    Rejected candidate: 
    /Users/aaron/Work/web/node_modules/@percy/cli-exec/dist/commands/exec.js
    Rejected candidate: 
    /Users/aaron/Work/web/node_modules/@percy/cli-exec/dist/commands/exec.json
    Rejected candidate: 
    /Users/aaron/Work/web/node_modules/@percy/cli-exec/dist/commands/exec.node

Any suggestions here?

When using build:wait (and client.waitForBuild), the project slug should include the organization, i.e. Ember/Ember-Website. However, this can be confused for the project name or even the slug without an organization attached. We should check for the presence of a forward slash and error when one is not found.

Since snapshots begin processing as soon as the API receives them, a build can fail before all snapshots have finished uploading. In this case, a 422 error is returned from the API. We should catch these errors and stop further snapshot handling

Right now, there doesn't seem to be support for GitHub Actions without having to configure things ourselves.

Right now, this is my current configuration (might be helpful when creating built-in support):

env:
  NODE_ENV: test
  PERCY_PARALLEL_NONCE: ${{ github.run_id }}
  PERCY_PARALLEL_TOTAL: 10

We're using @percy/cli (1.0.0-beta.54) together with @percy/puppeteer (2.0.0) and are facing a situation where Percy errors uploading screenshots because resource URLs are identical.

Here's an example scenario where this happens:

await page.goto('http://localhost:8000');

await percySnapshot(page, 'With JS');

await page.setJavaScriptEnabled(false);

await page.goto('http://localhost:8000');

await percySnapshot(page, 'Without JS');

await page.setJavaScriptEnabled(true);

But this can of course also happen when taking multiple screenshots in an SPA where the URL never changes:

await page.goto('http://localhost:8000');

await percySnapshot(page, 'Initial state');

await page.click('#button_that_opens_modal');

await percySnapshot(page, 'Modal open');

Is this something you can fix? If not, any advice on how to prevent this on our side?

Trying to get percy running on my local machine. Things seem to be set as I can see in logs that percy has started and snapshot is taken. But after that, I am not able to see my snapshots on the percy portal. The builds are in "Receiving build" state. They are stuck there. I am following java-selenium turorial from here: https://docs.percy.io/docs/java-selenium

I see these:
[percy] Percy has started!
[percy] Snapshot taken: Home Page

But I do not see any of these:

[percy] Created build #1: https://percy.io/[your-project]

[percy] Stopping percy...
[percy] Finalized build #1: https://percy.io/[your-project]
[percy] Done!

Am I missing something?

When a same-origin frame is serialized, all of its content becomes relative to the frame parent.

In the example below, localhost:8080/bar/styles.css would be proper location of the stylesheet.

<!-- localhost:8080/foo -->
<iframe src="/bar"></iframe>

<!-- localhost:8080/bar -->
<link href="styles.css" rel="stylesheet"/>

After serializing, the resulting DOM would look like:

<!-- localhost:8080/foo -->
<iframe srcdoc="<link href=\"styles.css\" rel=\"stylesheet\"/>"></iframe>

And with that DOM, the stylesheet request will be interpreted as localhost:8080/foo/style.css. Which is incorrect. This issue might also be present if the embedded iframe is not same-origin but allows cross-origin resource sharing instead.

We can likely fix this, albeit a little heavy handed, by prepending all nested relative URLs with the iframe's src.

Open to other solutions as well

Hi again!

I'm trying to use the Percy CLI to test snapshots with two extra requirements that the current capture API doesn't seem to account for. One of those cases is pages behind logins and the other is an in-house implementation of frontend deploy previews for PRs.

The login case requires programmatically navigating to our company's login page and entering credentials before being redirected back to the page on which I actually want to take the screenshot. The deploy preview case requires navigating to a programmatically generated URL that sets a cookie on the client which then enables viewing the specific frontend assets associated with the corresponding PR.

At one point I was hoping to use the execute argument but if that execute function navigates to a separate page I get the following error:

[percy] Encountered an error for page: ...
[percy] Error: Evaluation failed: ReferenceError: PercyDOM is not defined
    at __puppeteer_evaluation_script__:5:20

Looking at the code this makes sense since the SDK injects @percy/dom before the execute function runs so navigating away from the page loses that injection.

Do you have a suggestion for handling this use case? If there is no straightforward answer would you all be open to a PR introducing a new argument something like executeBeforePercySetup to run any login/other setup logic before that injection?

I'm having a major issue with running my snapshots on production websites, so authentication cannot be an issue.

When my Cypress / Percy test starts taking screenshots the 1st image taken/uploaded is displayed and formatted correctly with all CSS styles showing. But all snapshots taken/uploaded after that do Not contain CSS styles.

I'm not seeing anything obvious in the verbose logging. What can be the cause of this?

I created a mini repo here to reproduce the issue: https://github.com/bennymeade/cypress-with-percy

Sometimes servers block requests from headless browsers, or do other shenanigans with UA detection. We should allow a userAgent option (or something similar) to be sent per-snapshot to be used with asset discovery. SDKs can then automatically grab the test browser's UA and pass that along as a default.

Problem

If there are a large number of canvas elements on a page, the DOM clone can become too large of an asset to upload to the API. This is because the serialized canvas elements are put into the DOM clone as base64 encoded PNGs.

Possible solve

In order to avoid this, we should create proper assets from the serialized canvas elements and write them to disk. Then create an asset for Percy and upload it to the API.

"@percy/cli": "^1.0.0-beta.55",

The execute() function seems to work when in the "parent" snapshot:

// snapshots.js

module.exports = [
	{
		name: 'snapshot one',
		url: 'http://localhost:8080',
		waitForSelector: '.some-element',
		execute() {
			document.querySelector('h1').innerHTML = 'Tilting on windmills...';
		}
	}
]

However, trying to run the same execute() in an additionalSnapshot (like in this pages.js example) it does not work, and the <h1> content stays the same :/

// snapshots.js

module.exports = [
	{
		name: 'snapshot one',
		url: 'http://localhost:8080',
		waitForSelector: '.some-element',
		additionalSnapshots: [
			{
				suffix: '--tilt',
				execute() {
					document.querySelector('h1').innerHTML = 'Tilting on windmills...';
				}
			}
		]
	}
]

While trying to upgrade to v1.0.0-beta.53 of @percy/cli I noticed in your snapshots that the Percy specific css we have defined in the .percyrc file no longer gets applied

We see that fonts & images are not being loaded intermittently in our CI builds. This had happened once in a while all the time, but it had started happening frequently on Wed 6/23 around 10am EST. For example, for one of our projects we a constantly seeing 1-2 screenshots out of 3 that have no fonts loaded. As a result, we have "visual changes on master builds" notifications almost hourly, and Percy PR checks are always red because of the fonts mismatch. This renders Percy pretty much useless for given project.

We on 1.0-beta version of Percy CLI. We upgraded from b52 to b54/b55, then downgraded back to b52 - with the same issues.
Any help will be greatly appreciated.

Nothing rendered in the shadow-dom, even if open, shows up in percy-cypress snapshots.

The below cypress test passes fine.

Repo:

<html>
  <head>
    <script lang="javascript" defer>
      document.addEventListener('DOMContentLoaded', () => {
        const contentEl = document.querySelector('#content');
        const shadow = contentEl.attachShadow({mode: 'open'});
        const paragraphEl = document.createElement('p');
        paragraphEl.textContent = 'Hey Percy! Can you see this?';
        shadow.appendChild(paragraphEl);
      });
    </script>
  </head>
  <body>
    <div>
      <h1>Shadow DOM repro</h1>
      <div id="content"></div>
    </div>
  </body>
</html>

it.only("shadow dom test", () => {
  cy.visit(`/`);
  cy.contains("Hey Percy!");
  cy.document().then((doc) => console.log(doc.documentElement.outerHTML));
  cy.percySnapshot();
});

The outerHTML logged out does not include the text, and apparently that's how percy gets the DOM...

Any suggestions on workarounds? Getting rid of shadow-dom is not an option for my application.

percy - v
@percy/cli/1.0.0-beta.39 darwin-x64 node-v12.16.2

The documentation does not match output of --help from the CLI. Everything that's documented returns an error Error: Unexpected arguments:

Only after 45 minutes of trial and error did I learn that the correct arguments are:

Results in snapshotting what I want for one use case

percy snapshot out --files blog --dry-run

It seems globs are not working. 🤦 ie:

percy snapshot out --files blog/* --dry-run
zsh: no matches found: blog/*

Ignore is also not working with and without glob

node_modules/.bin/percy snapshot out --ignore blog docs legacy-docs --dry-run
[percy] Error: No snapshots found

It appears removing dry run may fix this????

node_modules/.bin/percy snapshot out --ignore blog docs legacy-docs --enable_javascript
[percy] Percy has started
...

I am exploring percy for our organization's visual testing. But I am not able to use percy cli even after installing it.
I installed percy as per given command in your documentation: npm install --save-dev @percy/cli

It creates a local node_modules folder under which I can see a @percy folder. But when I try to execute percy command: percy exec -- [my command], it says the command is not recognized. I attaching a screenshot for your reference. Do I need to set any path in environment variable. Could you please help asap?

BUG

About

We encountered issue which was even worse as it was on auto-approve branch master. One cy.percySnapshot() command failed (unknown reason, and it happen only once) but we were surprised that cypress test pass without issue, we expected that there is some assertion that cy.percySnapshot() was successful but it looks like there isn't? Result was that there was missing snapshot which was auto-approved, so in next tests we got diff that we have new snapshot even we didn't add any.

I work on smaller projects that routinely don't have the budget or scale to warrant a full CI/CD workflow, but become complicated enough, eventually where Visual Regression testing becomes useful.

I'm hoping someone here can share a workflow with Percy that gives me the steps and gives me example documentation/configuration files - to do the following.

I want to percy to be able to establish baseline screen shots against a production URL, and point percy at "www.domain.com", and have it look at a list of paths on that server, to build that initial set of screenshots as a baseline. (where "www" in this example is my production environment)

And then be able to give it an arbitrary domain, like. "branch-name.domain.com" (where "branch-name" is a dev server for our production environment, and can be any subdomain as the branches change...), and have it look at the same list of paths, and build screenshots for those, so that I can then share the results via a "https://percy.io/12345/project-name" URL, and let my client validate that the work is complete and not blowing up the website.

I've tried to follow the documentaton on perci cli, using the percy snapshot snapshots.yml example on the website, but it leaves me with no follow up direction one I figure out how to write the list of urls, and how to run the command.

Does anyone have an use case like this they can share with me?

Thanks!

As long as the option name and type is correct it is thought to be valid. We should additionally validate the values themselves to help avoid confusion and footguns.

Examples:

• maximum network idle timeout
• valid allowed hostnames
• widths/height outside the bounds of the API min/max
• etc.

Using percyCSS attribute in my .percy.js config does not affect my webpage.

module.exports = {
  version: 2,
  snapshot: {
    percyCSS: 'iframe { display: none; } .footer__wrapper, .c-news-feed, .page-share { content-visibility: visible !important; }'
  },
  discovery: {
    disableCache: true,
    requestHeaders: {
      Authorization: process.env.CYPRESS_BASIC_AUTH
    }
  }
}

As a workaround I have to inject the css style with my Cypress test.
Am I using the attribute correctly in the above code example?

When setting HTTPS_PROXY to port 80 (HTTPS_PROXY="http://localhost:80"), we accidentally strip the port from the URL, which throws an error:

npx percy exec -- node snapshots.js
[percy] Skipping visual tests - Port should be >= 0 and < 65536. Received .

Since migrating to @percy/cypress 3 (the first version of @percy/cypress that uses @percy/cli), parallel runs have started to be treated as separate Percy builds.

The user-facing result of this is that almost every PR reports a Percy diff that needs approval because the set of screenshots treated as the canonical ones are whatever shard finished first on the latest master build, which is likely to not be the same set of screenshots from the shard that finished first on the current branch build.

We run our tests in CircleCI on 5 separate shards. After a single CI run of a new PR, the following appeared in the Percy dashboard:

Up until this migration, parallel tests "just worked" in the way suggested by this documentation: https://docs.percy.io/docs/parallel-test-suites. Based on this test, it certainly looks like the intention is for parallel Percy tests to continue to "just work" on CircleCI. However, it looks like they don't.

A few potential causes for what's going on:

• PERCY_PARALLEL_TOTAL isn't being set to -1
• PERCY_PARALLEL_NONCE isn't correctly being set to the CircleCI build number
• The user now needs to supply the --parallel flag to percy-cli, which they didn't before @percy/cli.
  Update: I tried supplying the --parallel flag and it had no effect.
• ???

I'm going to continue to debug this, but thought I'd file an issue so that the team is aware this is a problem.

If it's helpful for debugging, the Percy build number for one of the affected runs is 10623605.

EDIT: If it's helpful, the command that we use to run our Percy tests in parallel is:

$ circleci tests glob "cypress/integration/**/*.js" | circleci tests split --split-by=timings > ../test-results/cypress_tests.txt
$ yarn run percy exec --parallel -- cypress run --reporter junit --reporter-options "mochaFile=../test-results/cypress/results-[hash].xml" --spec $(cat ../test-results/cypress_tests.txt)

My run command npx percy exec --verbose -- cypress run --spec 'cypress/integration/site-monitoring/visual-comparison.spec.ts'

Does not log anything:

  Running:  site-monitoring/visual-comparison.spec.ts                                       (1 of 1)

  Visual Comparison
[percy] Snapshot taken: https://www.company.com/page1/
    ✓ Visual comparison of pages within URL path: https://www.company.com/page1/ (10667ms)
[percy] Snapshot taken: https://www.company.com/page2/
    ✓ Visual comparison of pages within URL path: https://www.company.com/page2/ (6469ms)


  2 passing (20s)

My devDependencies:

"@percy/cli": "1.0.0-beta.58",
"@percy/cypress": "3.1.0",

And i've deleted my node_modules and reinstalled all packages.

I also have a config file .percy.js:

module.exports = {
  version: 2,
  discovery: {
    disableCache: false,
    requestHeaders: {
      Authorization: process.env.CYPRESS_BASIC_AUTH
    }
  }
}

What could be the cause?

Hi there!

I've been using Percy in my project for a few weeks now, however I've noticed some bizarre behaviour that I'd like some clarification on:

My setup

Dependencies:

"@percy/cli": "1.0.0-beta.52",
"@percy/webdriverio": "2.0.0",

Scenario
I have the following UI component that I'm testing in Storybook using the WebDriverIO SDK. This component can be found here - https://justeat.github.io/fozzie-components/@justeat/storybook/iframe.html?id=components-organisms--footer-component&viewMode=story

You'll notice that when you resize this UI component to a mobile viewport (I'm doing 414px), the headers such as 'Customer Service' / 'Top Cuisines' etc have a clickable button next to them.

Whilst my desktop screenshots look okay, my mobile viewport tests don't look correct (only when running Percy tests), as seen below:

My question

Is this bug likely caused by the fact Percy takes a single DOM snapshot, then just loads the snapshot and resizes it?

In this case, taking a Desktop snapshot (with the H2 headings), and resizing it after the fact in mobile?

If I'm correct, it would be good if Percy could take a unique snapshot for each of the viewports :)

If you wish to run my tests for yourself, please see my branch here - justeattakeaway/fozzie-components#1030

See this CI run. I see many errors like:

[percy] Encountered an error for page: http://localhost:40611/post/146314072245/do-publicly-owned-planned-economies-work/index.html
[percy] TypeError: Cannot read property 'outerHTML' of null
    at serializeDOM (<anonymous>:206:33)
    at serializeFrames (<anonymous>:87:28)
    at serializeDOM (<anonymous>:187:7)
    at serializeFrames (<anonymous>:87:28)
    at Object.serializeDOM (<anonymous>:187:7)
    at <anonymous>:5:35
    at withPercyHelpers (<anonymous>:9:14)

I am surprised because all pages are more or less the same (blog posts) so there is not really a reason that one would fail and another one not.

Is there any way to get those errors down?

For pages such as blog/index.html, the desired URL path is /blog. Sometimes entire sites are built this way and things on the page might redirect to the pretty URL. We should add an option to automatically support pretty paths for static snapshots. This should be fairly easy as the serve handler we use accepts a few different options.

So I have a static website with not too crazy amount of pages, but one run with running npx percy snapshot ./public still takes 35 minutes. I was wondering if the CLI tool could add some support for parallel execution over a static directory, where I could specify the number of parallel instances and the ID of this instance, something like:

npx percy snapshot --parallel-nonce <unique> --parallel-index 1 --parallel-total 5 ./public

If those arguments are provided, the CLI tool would index all files in the directory, but then in a deterministic way take only 1/5 of the files to snapshot.

We use those two strings for debugging SDKs. I think we should log a warning if at any point that info is missing from an implemented SDK. Without the user agent info, it's pretty hard to know which SDK / test framework the snapshots are coming from.

There is an old Docker image for Percy Agent, but I could not find any for CLI. It would be great to have one slim Docker image to use.

This issue aims to be a place where we can track which SDKs need to be worked on and which SDKs are ready to go.

Completed & released

100% released packages (docs, example apps, & stable major release cut)

• percy-selenium-python
• @percy/puppeteer
• @percy/webdriverio
• @percy/protractor
• @percy/nightmare
• @percy/nightwatch
• @percy/selenium-webdriver
• @percy/testcafe
• @percy/cypress
• @percy/ember
• percy-selenium-java
• percy-capybara
• @percy/storybook
• gatsby-plugin-percy

Code migration completed

These package have betas released

• N/A

WIP

Documentation currently says:

ignore: Glob or comma-separated string of globs for matching the files and directories to ignore.

But I could not get multiple globs to work by using comma between them. Are you sure this works? I could also not find anything in the code to make this work. I think it should be a string or an array of strings, but not comma-separated string.

While cookies are sent as headers, it appears as though providing a Cookie with the discovery.requestHeaders option does not work as expected.

It looks like there is a CDP method that allows setting cookies. We should add support for this via discovery.cookies.

What should the syntax look like? Mirror the headers option and allow key: value cookies? Or mirror the CDP API and allow an array of cookie param objects? Both (if object, a, if array, b)?

I've ran across a few issues referencing problems with loading web fonts. But, I'm still experiencing some issues, and wondering if there's something more I should be trying.

I'm migrating to @percy/cli from PercyScript.

My .percy.json file:

{
	"version": 2,
	"snapshot": {
		"widths": [
			320,
			1280
		]
	},
	"discovery": {
		"disable-cache": true
	}
}

Adding a ridiculously long timeout didn't seem to help.

// snapshots.js

module.exports = [
	{
		name: 'snapshot',
		url: 'http://localhost:8080',
		waitForTimeout: 15000,
	},
]

I'm currently running Percy against a local dev build that is available on localhost, and with --verbose flag, here is an excerpt of the output:

...
[percy:core:discovery] Handling request: https://cdn.mycompany.net/fonts/foo/foo-medium.woff2 (1212ms)
[percy:core:discovery] Handling request: https://cdn.mycompany.net/fonts/foo/foo-bold.woff2 (0ms)
[percy:core:discovery] Handling request: https://cdn.mycompany.net/fonts/foo/foo-regular.woff2 (1ms)
[percy:core:discovery] Handling request: https://cdn.mycompany.net/fonts/foo/foo-regularitalic.woff2 (0ms)
[percy:core:discovery] Processing resource: http://localhost:8080/app.css (18ms)
[percy:core:discovery] -> sha: 70c49d43caaee49f8583a4b5992d8c0ae591071f699318ddf98e2c7a3b018d0a (1ms)
[percy:core:discovery] -> mimetype: text/css (0ms)
[percy:core:discovery] Processing resource: https://cdn.mycompany.net/fonts/foo/foo-medium.woff2 (381ms)
[percy:core:discovery] -> Skipping remote resource (0ms)
[percy:core:discovery] Processing resource: https://cdn.mycompany.net/fonts/foo/foo-bold.woff2 (9ms)
[percy:core:discovery] -> Skipping remote resource (1ms)
[percy:core:discovery] Processing resource: https://cdn.mycompany.net/fonts/foo/foo-regular.woff2 (0ms)
[percy:core:discovery] -> Skipping remote resource (0ms)
[percy:core:discovery] Processing resource: https://cdn.mycompany.net/fonts/foo/foo-regularitalic.woff2 (2ms)
[percy:core:discovery] -> Skipping remote resource (0ms)
...