According to https://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html#data:-urls-and-http, only requests with method GET have the prescribed processing applied, therefore asserting that the prescribed processing is applied for requests with the POST method is incorrect.

If anything, the POST requests should assert that no special processing is applied, so presumably a status code of 0 (but I don't believe there is any prescribed status code or header set)

This has been approved by the WG's chair. Directory name for this test suite will be orientation-event.

The existing test suite can be found here.

/cc @dontcallmedom, @darobin

Would be good to test the behaviour described here:

https://www.w3.org/Bugs/Public/show_bug.cgi?id=25460#c14

...since it's currently not interoperable between browsers.

Arriving at python tools/scripts/manifest.py MANIFEST.json, I met an error:

Traceback (most recent call last):
  File "tools/scripts/manifest.py", line 466, in <module>
    update_manifest(repo_root, out_dir)
  File "tools/scripts/manifest.py", line 456, in update_manifest
    update(manifest)
  File "tools/scripts/manifest.py", line 436, in update
    import html5lib
ImportError: No module named html5lib

In urltestdata.txt any line starting with # is a comment, but there are a few test lines that start with #:

#  s:http h:example.org p:/foo/bar f:#
#/  s:http h:example.org p:/foo/bar f:#/
#\\  s:http h:example.org p:/foo/bar f:#\\
#;?  s:http h:example.org p:/foo/bar f:#;?

Comment syntax should be changed to something that is not ambiguous. For example:
/* ... */
-- ...

But not ; ..., which could lead to the same problem quite easily.

Proposed test:

http://software.hixie.ch/utilities/js/live-dom-viewer/saved/3026

Checks that can't match :invalid, get an 'invalid' event, or block submission.

I have tried to fix a few issues with some of the tests with /webmessaging, there are still some outstanding issues.

I just want to leave a quick message for those that might stumble upon them.

Most are caused by the messages being sent from testharness.js that arrive before the actual message being tested. Therefore, without proper testing of either data type or data itself the test were failing.

audio.getAudioTracks()[0].stop();
assert_true(audio.ended, "Audio stream is ended after stopping its only audio track");
assert_throws("INVALID_STATE_ERR", function () { ... }, "Adding a track from a finished stream raises an INVALID_STATE_ERR exception");

Since following Bugs, INVALID_STATE_ERR is not defined.

Bug 24928: Remove MediaStream state check from addTrack() algorithm.
Bug 24930: Remove MediaStream state check from the removeTrack() algorithm.

And the spec document have not defined "If track is ended, throw an INVALID_STATE_ERR exception."

So, 'mediastream-finished-add' looks like a test for undefined behavior.
It looks like remove this test or change the spec.

We don't have tests for this one

STR:

git clone [email protected]:w3c/web-platform-tests.git
cd web-platform-tests
git submodule init
git submodule update --recursive
python tools/scripts/manifest.py MANIFEST.json

The manifest file isn't generated and the following is printed:

INFO:Web platform tests:Working directory not clean, adding local changes

git status reveals modified: tools/pywebsocket (untracked content) which contains a bunch of untracked src/mod_pywebsocket/*.pyc files.

Reverting changes allows to generate the manifest, but then python serve.py doesn't work. That's solve by doing git submodule update --recursive again.

Maybe the issue can be solved with adding the guilty files to .gitignore somewhere.

This affects html/semantics/forms/the-input-element/radio.html

See the following
https://critic.hoppipolla.co.uk/showcomment?chain=3845
https://www.w3.org/Bugs/Public/show_bug.cgi?id=25254

See #1019. Would be nice to prettify the styling of area where the pathnames are displayed. Maybe @darobin has some idea.

XMLHttpRequest/xmlhttprequest-timeout-worker-aborted.html and XMLHttpRequest/xmlhttprequest-timeout-aborted.html 's third tests fail, but I think this is actually because delay.py is never used. serve.py is giving us a 404 for this request immediately after, so the tests are broken.

I'll see if I can figure out why, but my python-fu isn't the greatest. Broken tests aren't super helpful :3

https://github.com/w3c/web-platform-tests/blob/master/url/urltestdata.txt#L30 is intended to be a test but is a comment. (ack @xiaojunwu )

Any time somebody creates a new issue or PR, they'll see a dialog that includes a link to
the CONTRIBUTING.md file, with the link text saying:

Please review the guidelines for contributing to this repository.

Unfortunately the current CONTRIBUTING.md file only contains information about the license for the test suite—it doesn't contain any actual guidelines for contributing.

We should update it to include some (at least minimal) guidelines for contributing.

CC @xiaojunwu
set_dirty is using non-standard behaviour not supported by Gecko to make an <input> "dirty" via document.execCommand("delete"). Using execCommand on without designMode/contentEditable is not specified in any current spec AFAIK. The suites setting dirty: true do not have accurate results because of this issue e.g.:

• form-validation-validity-tooLong.html
• form-validation-validity-tooShort.html
• form-validation-validity-valid.html

https://github.com/w3c/web-platform-tests/blob/master/html/infrastructure/urls/resolving-urls/query-encoding/resources/resolve-url.js should test https://dvcs.w3.org/hg/webperf/raw-file/tip/specs/Beacon/Overview.html#sec-processing-model

@gsnedders submitted a bunch of "Loading Web Pages" tests in PR #304 but had to abandon work on them and the tests have not found any owner since then.

Why are empty directories like https://github.com/w3c/web-platform-tests/tree/master/html/syntax/named-character-references still in the repository? It looks like the test files have moved to https://github.com/w3c/web-platform-tests/tree/master/html/syntax/parsing. Just wondering.

while page loading:

var x = new XMLHttpRequest();
x.open("GET", "some-url", true);
x.onabort = function () {
  // abort should be called!
};
x.send();

window.stop();

see https://developer.mozilla.org/en-US/docs/Web/API/window.stop and
http://xhr.spec.whatwg.org/#infrastructure-for-the-send()-method

The expected result is impossible to attain due to the structure of the test.

It expects the following collection

[1, 4, "upload.abort", "upload.loadend", "abort", "loadend"];

however it performs the assertion in the event handler in upload.onloadend(), and so the assertions are evaluated before the other events are dispatched.

This would have worked, prior to 10010b1 --- perhaps the Verify function needs to be moved as well?

I'll submit a quick patch for that, because yeah I'm fairly sure this is what was intended

"One of the original requirements, and something I think is useful, would be the ability to identify tests to run by an input file where you can explicitly identify multiple directories, multiple test cases in a directory, or exclude test cases in a directory that has been selected." — http://www.w3.org/mid/[email protected]

https://github.com/w3c/web-platform-tests/blob/master/workers/semantics/structured-clone/worker-common.js#L67 is wrong for the Array/Object tests, it needs to send the original input

Recently a contributor to jsdom noticed that our implementation of attributes was a holdover from the old W3C DOM days, and did some impressive work (jsdom/jsdom@4a630a3) to bring us up to spec with the WHATWG DOM spec.

In the course of doing so, he wrote some tests that I thought might be useful to you folks:

https://github.com/tmpvar/jsdom/blob/0c92f20d2a458c016fa82cd8742d15262770d563/test/living-dom/attributes.js

I unfortunately don't have the bandwidth currently to convert these into WPT format, but I thought it'd be cool to make you aware of it in case someone does want to take this on. (And yes, I know that ideally jsdom should just be running all the web platform tests; that's jsdom/jsdom#666.)

These tests need to be cleaned up so that the common code is in an external js file rather than repeated lots of times.

From http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html

For response messages, whether or not a message-body is included with a message is dependent on both the request method and the response status code (section 6.1.1). All responses to the HEAD request method MUST NOT include a message-body, even though the presence of entity- header fields might lead one to believe they do.

So, what you really want to assert here, is that there is no request message-body, not whether there is a Content-Length header or not.

In url/urltestdata.txt, expected query and fragment always have a leading '?' and '#'. Example:

http://user:pass@foo:21/bar;par?b#c s:http u:user pass:pass h:foo port:21 p:/bar;par q:?b f:#c

The spec says the leading '?' and '#' are not part of query and fragment themselves, but these test could misguide implementors into including them "to pass the tests".

If you give ok to this change, I can provide a quick pull request.

When run http://web-platform.test:8000/eventsource/request-credentials.htm, get error message "EventSource cannot load http://www2.web-platform.test:8000/eventsource/resources/cors-cookie.py?ident=1_1390961730302. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://web-platform.test:8000' is therefore not allowed access."

See PR #851

web-platform-tests / 2dcontext / drawing-paths-to-the-canvas / canvas_focus_drawCustomFocusRing_001.html

will need to be revisited once we know what can be done with it.

WPT/README.me should include information about how to create a Bug Report for a test case or it should include a link to that information.

See these e-mail threads FMI:

Art: http://lists.w3.org/Archives/Public/public-webapps/2013OctDec/1090.html
Scott: http://lists.w3.org/Archives/Public/public-webapps/2013OctDec/1092.html
Tobie: http://lists.w3.org/Archives/Public/public-test-infra/2013OctDec/0084.html

Workers test case bug 24074:

[[
https://www.w3.org/Bugs/Public/show_bug.cgi?id=24074

http://w3c-test.org/web-platform-tests/master/workers/Worker_cross_origin_security_err.htm has the wrong pass condition.

[[
If the scheme component of worker URL is not "data", and the origin of worker URL is not the same as the origin specified by the incumbent settings object, then throw a SecurityError exception and abort these steps.
]]
http://www.whatwg.org/specs/web-apps/current-work/multipage/workers.html#dom-worker
]]

This bug was filed as Bugzilla bug 24077. Here is comment #1 but note there are additional comments in the bug:

[[
https://www.w3.org/Bugs/Public/show_bug.cgi?id=24077

The test has:

try {
postMessage(1, null);
} catch(e) {
postMessage(''+e);
}

and asserts that 1 is posted.

But the spec for postMessage is:

void postMessage(any message, optional sequence transfer);

and passing null to the second argument of that should throw, per webidl.

I recommend we do three things:

1. Add a copy of this test that replaces the first postMessage line with:

 postMessage(1);

2. Add a copy of this test that replaces the first postMessage line with:

 postMessage(1, undefined);

3. Modify this test to check that postMessage(1, null) in fact throws.
   #1 and #2 would test correct handling of the optional argument when it's not actually passed and #3 would test correct handling of null here.

]]

This bug was filed as Bugzilla bug 19878. Here is the original comment from @zcorpan but note there are additional comments in the bug:

[[
https://www.w3.org/Bugs/Public/show_bug.cgi?id=19878

http://dvcs.w3.org/hg/webapps/rev/d0ec1879951a

I don't understand why this change was made. The assert seems useful for the purpose of finding impl bugs. Is it tested somewhere else? Should we revert the change?
]]

dataset: should auto change string type to boolean for property who's data-* properties has value of "true" or "false"
dataset: should auto change string type to number for property who's data-* properties has value of "2"(int) or "1.1"(float)
It's just a recommendation though, but I hope the browser can do this for me !

Matt Wobensmith, SQA Engineer at Mozilla did an awesomesauce review, that I have not fixed (originally from #112 ):

General:

• CORS can be used for other network requests besides XHR
  • img, script, css, fonts
• Cross-protocol - access a resource that is hosted via protocol that provides headers
  • FTP, HTTP, HTTPS to HTTP, HTTPS - various combinations

Access-Control-Allow-Origin response header

• Simple request: when using "*" it cannot be used for a resource that supports credentials
• Boundary/error values: long domain names, large amount of domain tokens, IP addresses, International Domain Names
• Default request method for cross-origin request is GET
• Default request headers empty by default.
• Default entity body empty by default
• Exclude Referer header if source origin is a globally unique identifier
• if header value is "*" and omit credentials flag is false, fail

Access-Control-Allow-Methods

• if request method is not case-sensitive match for methods and is not a simple method, error
• If method is a simple method, including method value here - while not required - should not fail

Origin Request header

• Verify that it is emitted from both x-domain request and preflight request

Access-Control-Allow-Credentials

• If omit-credentials flag is not set and response includes 0 or more than one Access-Control-Allow-Credentials headers, fail.
• If omit-credentials flag is not set and Access-Control-Allow-Credentials header value is not case-sensitive match for "true", fail.

Preflight:

• Includes Access-Control-Request-Method header that includes the request method, even when this is a simple method
• Includes Access-Control-Request-Headers with comma-seperated list of headers, in lexographical order, converted to ASCII lowercase even when one or more are a simple header
• excludes author request headers
• excludes user credentials
• excludes the request entity body
• if header parsing fails, error
• in case of unable to cache due to disk space - proceed to actual request

Preflight cache:

• Case-insensitive match for Origin invalidates preflight result cache
• Access-Control-Max-Age - validate correct caching time
• Access-Control-Max-Age - maximum cache time
• Access-Control-Max-Age - error arguments (too large, too small, non-numeric)
• Access-Control-Max-Age - if UA imposes high-end limit but response's max-age is higher, let maximum be equal to the response's max-age
• If Access-Control-Max-Age is missing, duplicated or parsing failed, let max-age be at the discretion of UA (zero is allowed)
• Cache match testing
  • Based on rules in spec, determine whether a resource is properly cached or not
  • Verify that an expired resource no longer in cache invokes new preflight request
  • If removed and then reinstated into cache, new values should be respected
  • Origin is removed from cache if error occurs
  • If list of exposed headers in a response is not empty, and Access-Control-Expose-Headers does not contain values, the preflight cache can be cleared for all case-sensitive matches to this Origin and URL.

When we moved the tests, the Python script that generates them was not updated. It needs to be fixed to take the new structure into account. This will likely require data about the spec's layout.

Also check that proper warnings for the edition of generated tests are included.

Two tests inside additions-to-the-steps-to-clone-a-node assume that cloneNode() is equivalent to cloneNodes(true). However, this is no longer true per https://www.w3.org/Bugs/Public/show_bug.cgi?id=23565.

This bug was filed by @zcorpan as Bugzilla bug 24079:

[[
https://www.w3.org/Bugs/Public/show_bug.cgi?id=24079

http://w3c-test.org/web-platform-tests/master/workers/Worker_dispatchEvent_ErrorEvent.htm has the wrong pass condition, there is no initErrorEvent in the spec.
]]

This has been approved by the WG's chair. Directory name for this test suite will be geolocation-API.

This includes:

• migrating and reviewing the existing idlharness tests.
• migrating the existing test suite and converting it to testharness.js tests.

/cc @dontcallmedom, @darobin

On line 39 https://github.com/w3c/web-platform-tests/blob/master/html/semantics/grouping-content/the-blockquote-element/grouping-blockquote.sub.html#L39

The test fails.

{actual: "blehblah", resolved: document.location.protocol + "//" + document.location.host + "/blehblah"},

It seems there should be the local context of the file for the path, aka the start of document.location.pathname without the final file.

A link in http://example.com/foo/bar/blah.html with the following markup

<a href="blehblah">boo</a>

would link to http://example.com/foo/bar/blehblah

A way to fix the test is to just test with '/'

{actual: "/blehblah", resolved: document.location.protocol + "//" + document.location.host + "/blehblah"},
or to extract the file from the document.location.pathname part, but a bit more annoying to do.

https://github.com/w3c/web-platform-tests/blob/master/html/dom/elements-embedded.js#L8 should use nonCanon like video

The canvas tests that fail per canvas-tests.js logging are nevertheless logging as pass in testharness.js. E.g.,

http://w3c-test.org/2dcontext/text-styles/2d.text.draw.space.collapse.start.html

Looks like canvas-tests.js needs some integration work to log failures out to testharness.js (currently it only changes page color and renders failure text).

Removed in http://html5.org/tools/web-apps-tracker?from=7045&to=7046, seems like IE still supports it.

There is mention on the W3 website (http://www.w3.org/International/tests/html5/the-input-byte-stream/results-basics) of some files like "the-input-byte-stream-027.html" which are not present in this repository.

They are however mentioned in the htaccess file (https://github.com/w3c/web-platform-tests/blob/master/html/syntax/parsing-html-fragments/.htaccess).

I though for a moment it was because they are "Exploratory tests" but then the "the-input-byte-stream-030.html" should not be there. (I'd prefer it they were all here ;-)

Thanks in advance!

See https://critic.hoppipolla.co.uk/showcomment?chain=54. CC @velmont @annevk