cburgmer / csscritic Goto Github PK

To work with a big number of test cases, passing tests should be hidden, so that only broken tests are shown.

When comparing bigger pages, a line break will occur between rendered page and reference image. Scrolling through the reporter's output it is difficult to see quickly what's an actual page, and what's a reference.

Documentation on http://cburgmer.github.com/csscritic/ works but is old.

(continuing from twitter)

here is how i imagine it:

after initial set up (which probably needs dev support) a designer can run a generator script against a css file.

this generates tests for all selectors in the file.

a ui lets you pick the tests you want to keep (in the way you already mark failed tests as accepted).

this approach would remove the need for the generator to be intelligent in any way (eg recognize components, as you mentioned).

then again, the script maybe could also be run directly against a selector. that already would be a huge help in setting up initial tests before refactoring.

As a developer,
when I test a page,
then I want to be informed about resources that failed to load.

In some instances we might not be able to integrate testing into the application and run tests against a local code base. The same origin restriction doesn't allow us to test against a deployed website served from a different origin.

If a version valid in the past is changed to be invalid (change in what the user accepts) an old reference image might still pass the test. Reference images should be linked to a particular signed off version.

Instead they can be built on npm publish

nuff said

Now that we support loading images we currently do two requests if the url is not an image, i.e. a HTML page. We should take the content returned from the first request and directly pass this to rasterizeHTML instead of starting the second request.

• Make the test report show the screenshot even when the test passed
• Think about rendering actual page in an iframe for the "passed" case
• Add a test description field for documenting the screenshot (#12)

Find a good way where to save local references.

• configurable, or
• in ~/.csscritic/
• ...

Font's seem not to be rendered using antialiasing inside the canvas triggering different renderings. While this doesn't affect the diffing, it makes it harder to spot the differences.

We should try rendering the actual page inside a canvas element, as the reference image is calculated by going through a canvas.

Add something like --logPath= and log only if that is specified.

When defining a font-face rule including a local() directive such as

@font-face {
  font-family: "font";
  font-style: normal;
  font-weight: 700;
  src: local("font"), url(font.ttf) format("truetype");
}

Firefox seems to ignore the definition.

Removing the local() bit seems to solve that.

If a test fails or a reference has yet to be created, the rendered page can be changed in size by dragging the handle in the lower right cornet. Doing so will scroll to the top of the page if the page has so much output that it is scrollable in the browser and the test in question is shown in the lower part.

See https://bugzilla.mozilla.org/show_bug.cgi?id=986403.
This should probably be worked around when displaying the rendered page.

Tests for the console runner (phantomjs) are currently signed of via fingerprints, in the browser via a click on "accept". Does it make sense to integrate both solutions? How can this be done?

… to include update dependency info

Pages under test are rendered as bitmap in the browser through SVG. Certain browser styles are not applied when rendering HTML inside SVG which result in an incomplete/wrong rendering.

One example seems to be links (anchors "a"). Both coloring and underlining is missing.

A solution would be to provide a "reset.css"-like style by default when rendering pages.

When rendering a page, the browser loads some resources through AJAX. Some browsers seem to aggressively cache those. Reloading the RegressionRunner will then not render the newest page.

How to run.

At around ~50 large test cases Firefox' localStorage quota is reached throwing a NS_ERROR_DOM_QUOTA_REACHED.

Accepting further images as reference then silently fails.

Corner case:

The BasicHtmlReporter uses an id, which when CSS critic tests itself will clash with the "workaround SVG" being written by rasterizeHTML.js once more than one page is tested.

Putting the workaround SVG into an iframe solves the issue, but makes Firefox more flaky which is what the workaround is actually tries to circumvent.

Possible solutions:

• wait for the workaround to become obsolete
• find a different workaround that doesn't include adding the SVG to the DOM

Maybe show a hint to let people know they can resize the canvas.

http://cburgmer.github.com/csscritic/RegressionRunner.html doesn't show any visible output. Firefox is fine.

Both in the browser and on the console.

Because of the solution for #45

Signing off on a file's fingerprint is currently implemented as a reporter. This has some technical limitations and should find its way into the core instead.

The allinone bundle and ./csscritic.js are a central entry points to the application. They should be smoke tested after a successful build.

"? DUPs"

Fingerprints are calculated through downloading all referenced content, and only works for a remote server if it supports CORS and accepts those requests.

Unicode in HTML documents is garbled when loading URLs in the browser.

Background:

To render both images and HTML currently everything is loaded as binary and thus non-ASCII text is not correctly decoded. Apparently XHR needs to be told in advance whether binary data or text is expected and thus doesn't leave us with many choices.

Solutions:

• Find a workaround for loading both types through once request
• Do 2 requests worst case, a second one in case the first was the wrong binary/text setting
• Detect content tpye from path extension (.png, .gif, .jpe?g -> binary, * -> text)

csscritic.allinone.js is the one generally needed and should probably be renamed to csscritic-browser.js

Re-running a test suite from the RegressionRunner.html file seems to get slower and slower with every run in Firefox, adding around 60% time to the previous run. Restarting the browser seems to temporarily fix the issue.

Elements styled with border-radius will break the comparison such that the reference and the actual page will never be exactly the same.

Hey! I have problems with how Cyrillic websites are rendered. Here's the page: http://relfak.sfi.ru/holo/docs_critic/moleculas_header-group.html
And that's how it's rendered:

For window.devicePixelRatio > 1 a high density image should be saved to provide more detail for the reference image.

If a test fails on a difference of only few pixels in addition to a slight color difference producing a next to black color this error is not directly visible. This leads to the developer believing in false positive test errors.

A hyperbolic multiplier could put more weight on small changes.

The NiceReporter should replace the BasicHTMLReporter in the future. There's a bit of things missing though:

• Show errored page
• Show page load errors
• Feedback on 'accept'
• Resizable image
• Mouseover hints
• Total execution time
• Test parameters

Consider the following problem:

Although you have broken down your CSS + HTML into small components you still might want to only test a certain subset of the actual page.

Especially when testing against page object code (JS) and corner cases in the view implementation one might only be interested in parts of the page.

Testing against the full page will make more tests break when refactoring and accepting changed test cases will be the norm making it more likely to miss actual breaking cases.

Jasmine's default reporters collided with grunt-jasmine-task. As a result not all tests are run.

Once you surpass a certain number, grouping can help to organize the tests.

Describing a test case should probably not be limited to the document name only.

The integration test added in commit d999529 for testing image comparison does not work in Safari & Webkit.

Resizing an element at the bottom of the page can only happen in minimum steps, as the cursor quickly hits the lower edge of the screen. Also when leaving the browser window, the mouse release is not triggered, thus not repainting the inner canvas.

While css critic is not functional in both of these browsers, for demo purposes it would be nice if at least the main UI works as intended.