Git Product home page Git Product logo

webassembly / esm-integration Goto Github PK

View Code? Open in Web Editor NEW
350.0 82.0 30.0 61.11 MB

ECMAScript module integration

Home Page: https://webassembly.github.io/esm-integration/js-api/index.html#esm-integration

License: Other

Makefile 0.20% Python 5.02% Batchfile 0.07% CSS 0.01% Shell 0.06% HTML 0.03% OCaml 3.43% Standard ML 0.01% WebAssembly 88.01% JavaScript 2.26% Perl 0.13% TeX 0.01% Bikeshed 0.77%
proposal

esm-integration's Introduction

CI for specs CI for interpreter & tests

ES Module Integration Proposal for WebAssembly

This repository is a clone of github.com/WebAssembly/spec/. It is meant for discussion, prototype specification and implementation of a proposal to add ES module integration to WebAssembly.

The specifics of the ES Module integration Proposal are found in this subfolder.

A formatted version of the spec, including this proposal, is available here: webassembly.github.io/esm-integration.

Original README from upstream repository follows...

spec

This repository holds the sources for the WebAssembly draft specification (to seed a future WebAssembly Working Group), a reference implementation, and the official testsuite.

A formatted version of the spec is available here: webassembly.github.io/spec,

Participation is welcome. Discussions about new features, significant semantic changes, or any specification change likely to generate substantial discussion should take place in the WebAssembly design repository first, so that this spec repository can remain focused. And please follow the guidelines for contributing.

citing

For citing WebAssembly in LaTeX, use this bibtex file.

esm-integration's People

Contributors

andrewscheidecker avatar backes avatar binji avatar bnjbvr avatar cellule avatar chfast avatar dschuff avatar flagxor avatar gahaas avatar gumb0 avatar guybedford avatar honry avatar ia0 avatar jfbastien avatar keithw avatar kg avatar kripken avatar linclark avatar littledan avatar lukewagner avatar ms2ger avatar ngzhian avatar nicolo-ribaudo avatar pjuftring avatar ppopth avatar rossberg avatar sunfishcode avatar takikawa avatar tlively avatar xtuc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

esm-integration's Issues

Tracking phase progress

This is a tracking issue for the phase requirements and other issues that should be resolved for the proposal.

Phase 2 requirements

  • Full proposed English spec text available in a forked repo around which a reasonably high level of consensus exists

Phase 3 requirements

Phase 4 Requirements

  • Two or more Web VMs implement the feature

  • At least one toolchain implements the feature

  • The formalized spec is updated (N/A but HTML spec should be updated)

  • Community Group has reached consensus in support of the feature

CSP Support for ESM Integration

Somewhat related to #41, it's worth thinking about what CSP policies will apply to the ESM integration when importing Wasm.

Currently the Wasm CSP spec defines wasm-unsafe-eval to be required for all JS API usages.

Would the ESM integration form an exception here when importing Wasm in that it maintains full control of the script src? Or would it simply adopt this same policy?

What will it take for `<script type="module" src="foo.wasm">` to be useful?

Hello, first I want to say that I am relatively new here. I have been interested in WebAssembly for a long time, but also like many other web developers I haven't been interested in jumping ship from TypeScript/JavaScript to another language and ecosystem in order to use WebAssembly, especially on one-man projects where the benefit will not outweigh the cost, especially when it comes to developer experience, maintainability, and ecosystem.

But then AssemblyScript happened. This is a great thing for WebAssembly because...

...many web developers like me who do not wish to move to other languages like C/C++ or Rust are discovering AssemblyScript and getting really interested in actually using WebAssembly this way. AssemblyScript is bringing WebAssembly to these people, rather than them having to leave their language ecosystem to go to WebAssembly.

This has placed AssemblyScript as one of the top three languages for WebAssembly already, despite how new it is. At this pace it may not be a surprise if it gets to the number one spot.

As I heard @linclark say at Wasm Summit in person (summarized, not quoted): get users first. Without them, the system is meaningless.

I believe AssemblyScript is a excellent tool for this purpose, but it needs some kindling and consideration.

I've been making asdom, DOM bindings for AssemblyScript. They are hand-rolled bindings, as I haven't figured how auto-generated bindings can be possible yet without sacrificing performance in so many edge cases.

The following is a recording of the first DOM-centric web app (WIP) written entirely in AssemblyScript (after importing asdom's JS glue code for DOM APIs):

aspkg-html.mp4

The video shows Custom Elements being manipulated based on and Location/History routing (all of it written in AssemblyScript).

The code looks as familiar as you would expect, with customElements.define(), connectedCallback, disconnectedCallback, etc:

https://github.com/aspkg/website/blob/master/assembly/elements/MainContent.ts

Once JSX and tagged template string functions are in place, things are going to get interesting (think React- or Solidjs-style libraries in AssemblyScript), and I would imagine this will drive even more people to WebAssembly via AssemblyScript.

Back to the main question

The idea of <script type="module" src="foo.wasm"> seems very nice if such Wasm modules will have access to browser APIs without requiring help from JavaScript. If they don't have that access, then that approach will not be very useful for anything because the Wasm modules will be black boxes that run something unobservable inside of them without access to external APIs.

I'm imagining the module loads, and there need not be auto-generated glue, or hand-written glue, just Wasm modules with access to browser APIs. People in other languages can write an entire web app without toolchains needed to generate bindings (assuming the module does not depend on anything else except built-in browser APIs, fully self-contained), and just stick the Wasm module in via script tag, fulfilling <script>'s multi-language destiny.

What other proposals exist (or are needed) for the <script> form to be useful (namely without JavaScript in the picture)?

Is there already something in place that defines how Wasm modules can handle the dynamic non-statically-analyzable nature of anything JS can export? (that's still not as nice as being able to use web APIs directly without JavaScript being anywhere in the picture).

Allow URL as ModuleSpecifier?

The ModuleSpecifier in the ECMAScript specification is a string and some environment allows to load a module from an arbitrary URL.

I don't see a reason to explicitly disallow it but it should be under the CSP and the same JavaScript Module loading checks should be applied. That's probably a case where HTTPs should be required?

While I don't really see a use case in bundlers, in browser it would make sense to me. It could also act as a RPC framework.

Here's an example:

(import "https://doh.com/module.js" "specifier" (func))

It would probably make sense to add a few restriction, disallowing this for example:

(import "https://remote-memory.com/module.js" "specifier" (memory 1))
(import "https://remote-table.com/module.js" "specifier" (table 1 anyfunc))

We will need appropriate error handling, if the imported module isn't reachable you will end up with a NULL pointer in the wasm module?

Interop with commonjs?

We need to keep in mind that currently JavaScript Modules are mostly transpiled to commonjs.

While most of the bundlers also understand ESM and handle it accordingly, some environment like Node have a strong dependency on commonjs and transpiled distributed modules.

This is not really an issue since I think that we would handle them transparently (especially if #9 is not permitted). I just wanted to make you aware of that and we also should make it clear in the documentation.

Proposal: Higher Order ESM Integration

The last few issues on this repo have brought up the security question of whether or not Web Assembly provides a more secure execution environment in comparison to JS executions.

In addition, there've been much discussion over the exact workflows for the Web Assembly start fuction and binding process in terms of ensuring that the ESM import workflows provide the major use cases achieved with the declarative WebAssembly.instantiate APIs that are currently used today in Web Assembly applications.

Originally in this repo, @alexcrichton suggested in #14 an API for the Web Assembly ESM integration to support importing a WebAssembly.Module object directly, in order to allow more easily and flexibly working with a compiled module.

Along the lines above, I'd like to propose changing the ESM semantics of importing Web Assembly in order to better achieve security and use case flexibility for Web Assembly applications.

Proposal

Web Assembly offers some highly compelling execution security properties in providing a strictly defined secure execution sandbox, down to the imported bindings provided to it.

By default the ESM integration would not naturally benefit from these security properties since it permits arbitrary JS imports from Web Assembly modules.

Instead of following the naive ESM integration, the proposal would be for all Web Assembly modules that are imported, to be imported as compiled Module objects, leaving the binding process to the JS wrapper code entirely:

import module from './app.wasm';

// true
module instanceof WebAssembly.Module

The imported value of the Web Assembly module is only a { default: Module } ES module, that can then be instantiated with a JS call to WebAssembly.instantiate:

const { exports } = await WebAssembly.instantiate(mod, { env: wasmEnv ));
wasmEnv.bind(exports.memory);

export function wasmFn (arg) {
  return exports.method(arg)
)

By only providing the uninstantiated Module, this supports a number of useful properties:

  1. Run-to-completion Web Assembly binaries can be easily reinstantiated or rebound multiple times during the lifetime of the application.
  2. Many common Web Assembly binaries today would not support being imported in the Web Assembly ESM integration. With the above, all Web Assembly binaries that exist today can be imported and used in applications
  3. The import of Web Assembly is now a secure operation in itself, and the security properties of the Web Assembly module sandboxing are fully maintained for the user.
  4. Now that Web Assembly imports are a secure operation, it then also makes sense to support an import assertion for this case

How is this better than workflows today

The first criticism of this proposal might be - what benefit exactly does the ESM integration provide at all, if this is the case?

How is this a benefit over just fetching and compiling the Wasm module directly? Eg via:

const res = await fetch(new URL('./app.wasm', import.meta.url));
const module = await WebAssembly.compileStreaming(res);

And this is true, the benefit is exactly in just reifying the above pattern into an import pattern, including:

  • Not all JS environments provide a fetch global. This means that Web Assembly instantiations are inconsistent and there exists no "universal" easy pattern.
  • As a result of the above, build tools are not able to easily analyze Web Assembly compilations, when Web Assembly compilations should clearly be first-class constructs for JS / Wasm source analysis.
  • The import.meta.url relative fetch pattern is still very new and not many tools support it. Very few JS build tools today will properly detect that app.wasm is a binary that needs to be relocated included in the build folder in the above pattern.
  • There is a common problem with JS applications importing Web Assembly due to the above, where many libraries explicitly allow defining the Wasm binary location via a global or configuration option explicitly to get around this problem of locating the Wasm binary. The encapsulated modularity of the binary is not carried through in npm library import workflows. Some packages even ship base64 Wasm encodings to get around this issue. It is clearly a pain point with JS ecosystem integrations today.

Interaction with Import Assertions

Since importing Wasm becomes a non-executing and secure operation, it then makes sense that a Wasm module import assertion can verify this property:

import module from './app.wasm' assert { type: 'wasm-module' }

This fits the definition of import assertions in being entirely validation based, and importantly not splitting the interpretation of the module depending on the assertion / mode used.

Interaction with Module Linking Proposal

The module linking proposal provides a way for Web Assembly to handle instantiation and binding setup between multiple Web Assembly modules to provide a richer end-user API directly.

These Web Assembly modules can effectively be thought of as a new type of higher-order Web Assembly module for these wiring needs.

In the ESM integration for the module linking proposal, it would be possible to define that these modules are treated differently, such that you would get the exports and imports applying as one might expect with a JS import or the naive ESM integration.

These modules would lose the sandboxing properties so the import assertion would not apply to these forms of modules making the distinction between these two different ESM integration cases clearer.

Relationship to the component model?

Linking together the module graphs of JS and Wasm is only part of what's required for fluid interoperation between JS and code compiled to Wasm (see, e.g., #53 for some open questions in this area). Regularly in Wasm CG discussions, the component model is cited as the overarching means by which cross-language interop should work. But I see no mention of it in this proposal (nor in the FAQ).

Is it plausible that in a component model world, the proper things to link together are not ES modules and Wasm modules, but ES modules and Wasm components? If that were the case, there might be risks in standardizing ESM modules before the component model is further along in its standardization.

I'm interested to hear the proposal champions' take, as well as @lukewagner & other component model folks.

Examples: why an error in the wasm<->JS cycle case for value exports?

In the examples document, there is a case for cyclic imports between Wasm and JS where the Wasm module is higher in the graph.

The value import case is shown as an error in the table below:

JS exports

export type value (not a WebAssembly.Global)* global memory table function
Error snapshot snapshot snapshot snapshot

Does this case really need to be an error? Since the JS module is evaluated first, it seems like the value should be available at Wasm import time, and it can call ToWasmValue on this normally to allocate a suitable global.

Exporting the WebAssembly.Module object

One of the really nifty things about a WebAssembly.Module object is that it can be posted to web workers in a cheap fashion, avoiding unnecessary recompilation of the internals. One of the primary use cases for this seems to be to help support the upcoming threads proposal where a module can be instantiated in multiple threads.

It seems, though, that if a world is envisioned where wasm modules are all ES modules, this may be difficult to do currently! As proposed I think that there's no way for an wasm-module-as-ES-module to export its own WebAssembly.Module instance. That means it's not possible, when using native ES module integration for wasm, to cheaply instantiate the module on multiple workers.

Would it be possible to extend this proposal to allow a module to sort of export itself? It'd be cool if as part of wat we could do something like:

(export "module" (module))

(or something like that).

By providing the ability to hook into it that should be enough for various other tools I think to use the natively compiled module as part of ESM integration to send to other threads and such.

Now there's still a lot of weird questions specifically with the threading-related proposal about things like imports, but allowing a handle to the WebAssembly.Module itself would also allow modules to introspect their own custom sections (I believe) even when integrated with native ESM integration.

ESM isn't suited for importing objects containing non-JavaScript identifiers

Example:
A JavaScript module imports a Wasm module, the Wasm module is as follows:

(func (export "+") (param f64 f64) (result f64)
    local.get 0
    local.get 1
    f64.add
)

How does JavaScript import that?

Yes, ESM allows renaming imports, but it doesn' not allow surrounding them with quotes or something that would allow any name to be imported and renamed.

Currently, this would result in a syntax error:

import { "+" as add } from "./module.wasm";

How would this be handed?

Should ESM integration provide Modules rather than Instances?

There's been some feedback provided recently that toolchains and bundlers wouldn't be able to take advantage of the current Wasm/ESM integration proposal design, because it focuses on providing instantiated Wasm modules to interoperate in the JS module graph.

Instead, it could be possible for the default mode of exporting Wasm modules into the JS module graph to be an uninstantiated module (a related idea was suggested in #14, which was about optionally exposing a special export for the module object). In that case, importing a module and instantiating a Wasm module using ESM might look like:

import mod from "foo.wasm" assert { type: "webassembly" };
WebAssembly.instantiate(mod, importObject);

This uses an import assertion requiring that the import is an uninstantiated "webassembly" module (this would be required for the same reason that it's required for JSON and CSS modules; they have different execution behavior than JS modules).

Alternatively, this could be done using the import reflection proposal (https://github.com/tc39/proposal-import-reflection), which would provide more flexibility to provide both (instantiated and uninstantiated) behaviors by using different reflection types. This would allow the current Wasm/ESM behavior to be either kept or deferred to the future.

I'd be interested in feedback from people working in toolchains and bundlers what would work better, and if there are cases where the original behavior of instantiating modules would be more useful.

Should WebAssembly be at the same "privilege level" as JS?

This proposal lets WebAssembly import JS modules just as much as it lets JS import WebAssembly modules. The idea is that they're all part of the same big module graph. Logically, they can be thought of at the same "privilege level"--in HTML, this level would be based on the origin of the document that the module graph was loaded from.

Having WebAssembly and JS modules at the same privilege level contradicts the idea, which the Wasm/JS API supports, of WebAssembly only having access to what's passed into it explicitly. Instead, Wasm modules within ESM can make a request for any JS module--even on a different origin--and that JS module could package up any Web API in a form that can be accessed by Wasm. So, the thinking is a bit different.

Should we add Wasm/JS integration in a way which equalizes their privilege levels like this? Or should we consider certain restrictions, such as allowing JS to import Wasm, but not the reverse?

(Note, if we did have a "lower privilege" way of running code, there would be a massive demand to have this for JS modules as well. But it's unclear to me how this design should look. We're not talking about hard security boundaries here, since it's all within the same process, but more like softer isolation, what is given direct access to what, as with the Realm API.)

Imported memory and shared imported memory

If multiple relocatable WASM modules import memory instead of exporting it, a post-processor / linker / loader can relocate them so that they can use different parts of the same imported memory. This means that at runtime, values placed into this common memory can be directly accessed by multiple WASM modules, whereas with each module exporting its own memory some code in the host environment (ie userland JS) has to get involved and copy values around.

I imagine that relocatable, memory-importing modules will be a good way to ship binary-only modules in the future, when relocation and linking are stabilized in the WASM spec.

So there are a few related questions (which can be split up into multiple issues if you want):

  • How would a WASM module that wants to import memory work with the ESM import statement?

  • How would a module graph that wants to use the same imported memory for multiple modules do that with the ESM import statement?

  • Would there be a way to hook into the module-import mechanism so that a linker/loader implemented in userland JS can post-process a relocatable module into a relocated one if the module is imported with the ESM import statement? (Perhaps the answer to this is "It won't; you have to fall back to instantiateStreaming" or "Let's wait for relocation to be stabilized first".)

function's result type without `return`

In the example of wasm imports <- wasm exports, the exported function increment in counter.wasm has no return value, when importing function increment in main.wasm, why setting the function's result type is i32?

;; main.wat --> main.wasm
(module
  (import "./counter.wasm" "count" (global i32))
  (import "./counter.wasm" "increment" (func $increment (result i32)))
)

;; counter.wat --> counter.wasm
(module
  (func (export "increment")
    (global.set 0
      (i32.add
        (global.get 0)
        (i32.const 1))))
  (global (export "count") (mut i32) i32.const 5)
)

Importing a Wasm program

This came up in the nodejs PR (nodejs/node#27850), there are some concerns about import a WebAssembly Module that is a program.

We might want to:

  • only import some exported symboles and avoiding calling the main function.
  • calling the main function in a safe way; seperated memory, forked from the main process.

We don't provide any solution for this, apart from transforming the module AOT because it's part of the spec semantics.

Possible inconsistency in js<->wasm cycle example for value import

In the JS<->Wasm import example from the examples page, there is a case for a value import in which the Wasm module imports a non-WebAssembly.Global value from JS as a Wasm global.

This case says it should result in "0 if a const import and not in TDZ, otherwise Error", but it seems like it should always be an error to be consistent with the JS API semantics for normal Wasm imports.

I think the 0 there comes from the fact that the JS binding has value undefined, which converts to 0 via ToInt32. But the Wasm JS API spec only does the ToWebAssemblyValue conversion on value imports when it satisfies the check "If Type(v) is Number or BigInt," (see 3.5.1 in https://webassembly.github.io/esm-integration/js-api/index.html#read-the-imports)

Example:

function module(bytes, valid = true) {
  let buffer = new ArrayBuffer(bytes.length);
  let view = new Uint8Array(buffer);
  for (let i = 0; i < bytes.length; ++i) {
    view[i] = bytes.charCodeAt(i);
  }
  return new WebAssembly.Module(buffer);
}

// throws LinkError
// The Wasm is (module (global (export "g") (import "m" "g") i32))
const inst = new WebAssembly.Instance(module("\x00\x61\x73\x6d\x01\x00\x00\x00\x02\x08\x01\x01\x6d\x01\x67\x03\x7f\x00\x07\x05\x01\x01\x67\x03\x00"),
                                      { m: { g: undefined } });

What happens when an async function is imported?

For example, if we modify the function import example and make getCount async.

;; main.wat --> main.wasm
(module
  (import "./counter.js" "getCount" (func $getCount (func (result i32))))
)
// counter.js
async function getCount() {
    const response = await fetch('//api/get-count');
    ...
    return count;
}
export {getCount};

Given existing behaviour, it would crash at runtime. But should we do better?

Should this rely on or interact with https://github.com/WebAssembly/js-promise-integration?

(X)HTML import name suggestion

Unless I am mistaken, it is currently being proposed that ECMAScript be able to use MJS syntax for imports, and (X)HTML to use a module script tag?

For the MJS part, I have no opinion, for the (X)HTML web page it is being suggested that we could use something along the lines of:

<script type="module" src="./main.wasm" />

If so, might I suggest that we look at how JavaScript was originally intended to be imported:

<script type="text/javascript" src="./main.js" />

Google's Dartium browser used the Dart mime for executing Dart:

<script type="application/dart" src="./main.dart" />

Thus, shouldn't we use the Wasm mime-type as the "type" in (X)HTML?

This would make it much clearer to the developer, be clearer to the browser, and be better compatibility-wise: older browsers would not attempt to request the file at all, only newer browsers would. Even a current browser would expect JavaScript, thus waste client resources just to error upon seeing Wasm bytecode.

Unless this proposal is purely for a more ergonomic syntax when used in modules.

Security concern when importing executable modules as non-executable data

ECMAScript proposals that directly affect ESM, thus indirectly affect this Wasm proposal for reference:

If this proposal, and both, the ECMAScript JSON and module type assertions proposal all succeed, then Wasm will deal with the same problem that ECMAScript would have with JSON/non-executable modules.

As the JSON import proposal seems very general, simply allowing parsing and importing of JSON files via URLs, coupled with the Wasm interface type proposal, Wasm could very well be capable of importing structs, arrays, and everything necessary to define the shape of an entire JSON module.

Thus, I believe, that this proposal should attempt to define a way for Wasm to do those same host import assertions, even if the ECMAScript assertion proposal were to fail, unless the ECMAScript JSON proposal doesn't become a part of the spec.

If this is pushed out to implementors and the ESM assertion proposal is accepted and implemented first, then Wasm will not be at equal footing with ECMAScript.

At a minimum, a set of bytes should be reserved or defined for this capability.

Introduce evaluation as a semantic phase?

The core spec explicitly lists the semantic phases.

These are currently:

  • Decoding
  • Validation
  • Execution
    • Instantiation
    • Invocation

Given that ES modules have separate instantiation and evaluation phases, should this section make note of this separation?

Integration with import assertions

Now that import assertions are Stage 3, we have to decide how they will integrate with Wasm/ESM integration. There are two aspects of this:

  • When importing a Wasm module from JS, does it need an explicit assert { type: "wasm" } or is this OK to be implicit (given that, currently, JS and Wasm run at the same privilege level #41)?
  • WebAssembly modules will need a custom section which somehow note the import assertions. We'll want to design this custom section format to be future-proof to possible expansions of the import assertion grammar, and the possible creation of "evaluator" attributes. (previous mention)

How ImportObject works with ESM?

I don't think this is an issue with the current spec but rather a convention in the tooling (but still wanted to discuss about it).

For context, here's the definition of a module import:
screenshot from 2018-03-30 14-06-46

We can easily map that into ESM:

  • module is a ES module
  • name the identifier used in the ExportDeclaration of the module.

If the notion of module has the same semantics as ES Module it should be possible to include modules directly from npm as well as local files (or custom rules defined by bundlers).

Here's my issue

When using emscripten (for example) the imports of the wasm modules are using the env or global module, which can't be found using the current bundlers (since they will resolve it to an npm module, note that it's even dangerous).

While there's not reason to prevent using npm modules we should refer to local files first. I can see a few solutions for that problem:

  • Bundlers could search for local files before searching on npm.
  • Emscripten could use ./env and ./global instead (impacts manual usage).
  • Extra linking phase that changes the name of the modules.

"Bundlers" means currently Webpack and Rollup in the future.

Permit imports of Numbers as const globals?

If something is not a WebAssembly.Global object, and it's imported with a numeric mutable global type, it makes sense to call this an error, since there's no way a live binding could be of just numeric type. I like the idea of making this possible with a combination of host bindings and anyref.

For const global numeric imports, though, if we take the strategy in #13, I don't see the problem with snapshotting them (after casting them to the right numeric type). Actually, if we did snapshot them, it'd be more consistent with how the JS API works, which implicitly creates const globals for Numbers which are passed into the API.

Thoughts?

Circular imports of functions as a host-bindings follow-on?

Thanks to @xtuc and @sokra, webpack supports WebAssembly modules. We're currently discussing how to align webpack and the new specification in this repository.

One interesting thing about webpack is that (if I understand correctly) it supports cyclic Wasm modules importing globals and functions, rewriting parts of the code to implement live bindings.

Webpack implements these circular imports on top of the Wasm/JS API, so it's doing it without interrupting the Wasm instantiate path. I believe that's just the invariant we need we need in order to make type imports work.

As an MVP, I think it's OK to leave out circular Wasm modules, but I'd like to go about this design thinking about enabling some limited use of circular modules as a follow-on proposal. It would still be based on, modules are instantiated one by one, separately, but if you mark an import as "circular" with host bindings, then an extra snippet is generated to create a function that throws an exception, until the other module is loaded, and then it forwards to that other function.

Thoughts?

Nit: What should we use the term "live binding" for?

I originally thought "live binding" referred to a mutable variable storage, which could be exported, and then overwriting the variable on one end would affect the live binding on the other end.

The README in this repository, and under #13, seems to use "live binding" to mean something else: when wasm exports something (e.g., a Global, Memory, or Table) which is somehow mutable, then the binding is "live" in the sense that, when that object is mutated, it's reflected in the exported version. This usage was initially surprising to me; I would've thought of this as a kind of snapshot (just a shallow one, of a mutable object)

I'd suggest that, in this proposal, we either clarify that we're using this broader sense of "live binding" (could be just a couple sentences), or avoid the term.

Disallow imported default in wasm

Currently we don't allow the user to import a default export, the syntax and the ImportObject ensures that:

(import "./module.js" "specifier" (func))

My opinion is that it's fine and otherwise would add unnecessarily complexity to the semantics/integration. We need to make it clear in the documentation because JavaScript permits it and uses it in distributed modules.

On a side note, we should disallow default exports in wasm as well. There would be no way to distinguish a named "default" export and a default export.

Extending Wasm JS API with module namespace reflection

As far as I'm aware, when the original WebAssembly JS API was developed the ES module spec namespace exotic object had not been fully defined, thus exports was created as a plain JavaScript object, and treated as frozen for some defensibility.

As a result, when the ESM integration is complete, there are two JS objects that can be used to represent a Wasm module instance - this old frozen exports object (on top of which the ESM integration is currently specified), and the module namespace exotic object,

When module linking instantiates a Wasm module, it treats it as an "instance type", which is a type it can apply to any module instance - Wasm OR JS (//cc @lukewagner if I'm getting this correct).

But the representation in JS for this instance can either be a plain JS object or a module namespace depending on how the Wasm module was instantiated (ESM integration v JS API v Wasm module linking).

This leads to some questions:

  • Should module linking expose all Wasm instances as namespace objects (I believe this is the current plan)
  • If so, would that be done via a conversion when interfacing into JS only? Or would it be done eagerly?
  • Should the WebAssembly JS API be extended to permit namespace exotic object construction for a given exports object? For example via a moduleInstance.getNamespace() or similar function?
  • Alternatively, should the WebAssembly JS API update the .exports intance property to be a JS module namespace exotic object? Since the object was defensively frozen, this should mostly be backwards compatible down to the null prototype differences - .hasOwnProperty checks etc which may cause possible ecosystem issues.

It could be beneficial to start lining the specs up now along these lines to ensure there aren't any roadblocks when we get there.

Tracking progress of .wasm fetch

The current fetch approach of loading a wasm module allows you to track the progress of the (often large) network resource, and indicate its progress to the user (you just wrap fetch with something like this).

The whole point of this proposal is to make the wasm exports available "statically", so the entire loading phase is made opaque. But I wonder if a dynamic import could somehow provide hooks to track progress?

const wasm = import('some.wasm');

wasm.onProgress(...)

const {exportedFn} = await wasm;

I understand this is not possible with dynamic imports today, so this is probably the wrong place to talk about this idea... In any case, it'd be good to mention "indicate progress" use case as a non-goal, as it's the first thing I thought of (and is a reason I wouldn't make use of this if the binary is large enough).

Incorrect behavior of function importing?

From EXAMPLES.md:

If the exported value is a WebAssembly exported function based on the same types, make use of it. Otherwise, create a host function out of the JS function which includes the casts implied by its type.

Shouldn't this throw if it is another Wasm function, but with a mismatching call signature?

Multiple imports of the same wasm file: same instance or different instances?

As per the title, the basic case is like:

//a.js
import {someFunc} from someModule.wasm
someFunc();
//b.js
import {someFunc} from someModule.wasm
someFunc();
//c.js
import a.js
import b.js

While instantiating c.js, there will be a single wasm module whose export 'someFunc' has been called twice OR two different instance of the same module, each called once?


Then depending on the answer, there might be possible problems (or at least some need of disambiguation / explanation) how to avoid some common problems.
With separate instances, the fact that memory might become an issue, with same instance the fact that different components might (without knowledge) share the same internal state.

I implemented ES6 module's support in Cheerp (C++ to JS/Wasm compiler), and we ended up with modules exporting a default initialization function that when called returns (asynchronously) a new instance of the module.

Here the explanation of the approach: https://docs.leaningtech.com/cheerp/ES6-Modules

Wasm/ESM integration and additional start/main functions

When starting up a Wasm module or application, it's often not enough to call the Wasm start function. Often, there's some kind of external code driving it, maybe generated by the toolchain or otherwise present in the host environment.

One idea from WebAssembly/design#1160 to reduce the need for this sort of "driver" code was to make Wasm/ESM integration call a secondary "_start" function, after the exports are initialized, to permit the driver module to have the Wasm exports before functions that it exports to Wasm are run. However, there are some cases which don't quite meet this model:

I suspect, if we examine more systems, we'll find more kinds of mismatches or subtle cases where more behavior is needed.

The goal for this issue is to compile patterns from various Wasm toolchains and environments, and see what would make sense to standardize. This standard initialization behavior could be used both in JS and outside of JS environments, in common between the two.

I'd like to see if we can coalesce on the design of a new custom section to declaratively specify this initialization behavior. I think it will make sense to do on top of the basic Wasm module semantics, with the start function being invoked atomically with module initialization before exports are made available, as explained by @rossberg.

I think the MVP Wasm/ESM integration semantics make sense as is, and any further behavior will be a v2 that layers on top. If you disagree, I'd love to understand better why in this thread, so we can make it clear to potential implementers of Wasm/ESM integration whether this is stable.

cc @lukewagner @guybedford @xtuc

Start uninitialized Wasm exports in TDZ?

In the README for this proposal, both before and after #13, Wasm exports are initialized as undefined, from the beginning of the evaluation phase, before the evaluation of the particular module runs. ES2015 introduces the "temporal dead zone", a mechanism where undefined things can trigger a ReferenceError rather than mysteriously show up as undefined. Would it make sense to use that here?

Investigate connection with top-level await

The proposal in #13 for WebAssembly/ESM integration is more or less to do a top-level await of WebAssembly.instantiateStreaming of the module, and this raises many of the same issues of asynchronous-ness during module evaluation that top-level await faces.

@MylesBorins has indicated that the current status of top-level await is to go with "variant B", which allows module siblings to proceed in parallel. Such ordering should be good for WebAssembly, in that it allows less waiting on compilation.

Two issues remain:

  • What changes need to be made in the ECMAScript specification to accommodate this asynchronous-ness? I don't think we need to block entirely on top-level await, but we'll need most of the underlying machinery.
  • If we go with parallel sibling execution, the execution of the WebAssembly "start function" could get out of order with respect to siblings (depending how long the prior steps of instantiation take for different siblings). Is this an issue? Should we provide additional synchronization here?

Are we considering using Promises in the ImportObject?

Following the discussion in webpack/webpack#6433 (comment).

@lukewagner mention the usage of Promises in the ImportObject for the instantiation.

While that works great for Globals, Memory, Tables and functions (functions aren't really impacted in fact). I think that's not going to fix the case were modules have cyclic dependencies.

Let A and B be two wasm modules; if A imports a func from B and B imports a func from A. A will be waiting for B's promise and B will be waiting for A's promise.

Maybe my example was previously misunderstandood but were's a response:

@lukewagner It would lead to an instantiation-time error when one wasm module's instantiation would read undefined from the binding, if that's what you mean.

Also that prevents A from importing itself.


Another point is that it strictly ensure the initialization order, which prevents Webpack to download and instantiate modules concurrently.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.