ShadowRealm Web Platform integration testing and expectations about proposal-shadowrealm HOT 8 OPEN

@mgaudet Here are the ShadowRealm enabled tests in WPT as of now: https://github.com/web-platform-tests/wpt/pulls?q=is%3Apr+author%3Arwaldron+shadowrealm+is%3Aclosed

These are pending HTML spec update: https://github.com/web-platform-tests/wpt/pulls?q=is%3Aopen+is%3Apr+author%3Arwaldron+shadowrealm

The tests that are currently in WPT run with varying support across Chrome Canary, Firefox Nightly and Safari Technology Preview.

For Chrome Canary:

/Applications/Google\ Chrome\ Canary.app/Contents/MacOS/Google\ Chrome\ Canary --js-flags="--harmony-shadow-realm"

For Firefox Nightly:

Open "about:config", type:

javascript.options.experimental.shadow_realms

Toggle to true

For Safari Technology Preview:

env JSC_useShadowRealm=1 __XPC_JSC_useShadowRealm=1 /Applications/Safari\ Technology\ Preview.app/Contents/MacOS/Safari\ Technology\ Preview 

from proposal-shadowrealm.

cc @rwaldron

from proposal-shadowrealm.

(Oh, I see the global object definition is now in flux at whatwg/html#9893)

from proposal-shadowrealm.

(In case anyone wants to see current results from Firefox, with some speculative enablement, the job results from CI are here)

from proposal-shadowrealm.

We've now landed 2 of several PRs that are enabling functional testing of the [Exposed=*] APIs. More are awaiting review, and still more are in progress.

from proposal-shadowrealm.

Hey @rwaldron , apologies for the long feedback delay.

So, running the tests as ./mach wpt --headless /test/path/ --setpref javascript.options.experimental.shadow_realms=true

I am noticing a lot of test failures related to undefined references. For example in the wasm tests, lots of failures of the shape assert_Table is not defined,

There's also some tests (most of the Streams tests) that for example want setTimeout, but that's currently defined as a part of the WindowOrWorkerGlobalScope mixin, which currently the ShadowRealmGlobalScope doesn't include.

I can file more bugs in the appropriate repos, but I'm not sure where would be best.

On the good news side, we're definitely getting some UNEXPECTED-OK results, which is nice.

from proposal-shadowrealm.

@mgaudet I don't have any comment on whether the tests pass or not. My only goal was to ensure that the APIs that are likely to be exposed in shadowrealm are being tested in shadowrealm. It's up to implementers to run those tests and make sure their implementations pass.

from proposal-shadowrealm.

Sure -- my feedback was more about failures which seemed like test integration problems (e.g. the missing assert_Table reference) or depended on undetermined elements.

from proposal-shadowrealm.