Git Product home page Git Product logo

starlingmonkey's Introduction

The StarlingMonkey JS Runtime

StarlingMonkey is a SpiderMonkey based JS runtime optimized for use in WebAssembly Components. StarlingMonkey's core builtins target WASI 0.2.0 to support a Component Model based event loop and standards-compliant implementations of key web builtins, including the fetch API, WHATWG Streams, text encoding, and others. To support tailoring for specific use cases, it's designed to be highly modular, and can be readily extended with custom builtins and host APIs.

Requirements

The runtime's build is managed by cmake, which also takes care of downloading the build dependencies. To properly manage the Rust toolchain, the build script expects rustup to be installed in the system.

Building and Running

With sufficiently new versions of cmake and rustup installed, the build process is as follows:

  1. Clone the repo
git clone https://github.com/bytecodealliance/StarlingMonkey
cd StarlingMonkey
  1. Run the configuration script

For a release configuration, run

cmake -S . -B cmake-build-release -DCMAKE_BUILD_TYPE=Release

For a debug configuration, run

cmake -S . -B cmake-build-debug -DCMAKE_BUILD_TYPE=Debug
  1. Build the runtime

The following command will build the starling.wasm runtime module in the cmake-build-release directory:

# Use cmake-build-debug for the debug build
# Change the value for `--parallel` to match the number of CPU cores in your system
cmake --build cmake-build-release --parallel 8
  1. Testing the build

After completing the build (a debug build in this case), the integration test runner can be built:

cmake --build cmake-build-debug --target integration-test-server

Then tests can be run with ctest directly via:

ctest --test-dir cmake-build-debug -j8 --output-on-failure

Alternatively, the integration test server can be directly run with wasmtime serve via:

wasmtime serve -S common cmake-build-debug/test-server.wasm

Then visit http://0.0.0.0:8080/timers, or any test name and filter of the form [testName]/[filter]

  1. Using the runtime with other JS applications

The build directory contains a shell script componentize.sh that can be used to create components from JS applications. componentize.sh takes a single argument, the path to the JS application, and creates a component with a name of the form [input-file-name].wasm in the current working directory.

For example, the following command is equivalent to the cmake invocation from step 5, and will create the component cmake-build-release/smoke.wasm:

cd cmake-build-release
./componentize.sh ../tests/smoke.js

Thorough testing with the Web Platform Tests suite

StarlingMonkey includes a test runner for the Web Platform Tests suite. The test runner is built as part of the starling.wasm runtime, and can be run using the wpt-test target.

Requirements

The WPT runner requires Node.js to be installed, and during build configuration the option ENABLE_WPT:BOOL=ON must be set.

When running the test, WPT_ROOT must be set to the path of a checkout of the WPT suite at revision 1014eae5e66f8f334610d5a1521756f7a2fb769f:

WPT_ROOT=[path to your WPT checkout] cmake -S . -B cmake-build-debug -DENABLE_WPT:BOOL=ON -DCMAKE_BUILD_TYPE=Debug
cmake --build cmake-build-debug --parallel 8 --target wpt-runtime
cd cmake-build-debug
ctest --verbose # Note: some of the tests run fairly slowly in debug builds, so be patient

Configuring available builtins

StarlingMonkey supports enabling/disabling bundled builtins using CMake options. You can get a full list of bundled builtins by running the following shell command:

cmake -P [PATH_TO_STARLING_MONKEY]/cmake/builtins.cmake

Note that it's required to include builtins defining all exports defined by the used host API. Using the default WASI 0.2.0 host API, that means including the fetch_event builtin.

Using StarlingMonkey as a CMake sub-project

StarlingMonkey can be used as a subproject in a larger CMake project.

The importing project must at a minimum contain the following line in its CMakeLists.txt:

include("${PATH_TO_STARLING_MONKEY}/cmake/add_as_subproject.cmake")

With that line added (and ${PATH_TO_STARLING_MONKEY} replaced with the actual path to StarlingMonkey), the importing project will have all the build targets of StarlingMonkey available to it.

Note that building the starling.wasm target itself will result in the linked starling.wasm file being created in the starling.wasm sub-directory of the importing project's build directory.

To make use of importing StarlingMonkey in this way, you'll probably want to add additional builtins, or provide your own implementation of the host interface.

Adding custom builtins

Adding builtins is as simple as calling add_builtin in the importing project's CMakeLists.txt. Say you want to add a builtin defined in the file my-builtin.cpp, like so:

// The extension API is automatically on the include path for builtins.
#include "extension-api.h"

// The namespace name must match the name passed to `add_builtin` in the CMakeLists.txt
namespace my_project::my_builtin {

    bool install(api::Engine* engine) {
        printf("installing my-builtin\n");
        return true;
    }

} // namespace my_builtin

This file can now be included in the runtime's builtins like so:

add_builtin(my_project::my_builtin SRC my-builtin.cpp)

If your builtin requires multiple .cpp files, you can pass all of them to add_builtin as values for the SRC argument.

Providing a custom host API implementation

The host-apis directory can contain implementations of the host API for different versions of WASI—or in theory any other host interface. Those can be selected by setting the HOST_API environment variable to the name of one of the directories. Currently, only an implementation in terms of [wasi-0.2.0] (host-apis/wasi-0.2.0) is provided, and used by default.

To provide a custom host API implementation, you can set HOST_API to the (absolute) path of a directory containing that implementation.

starlingmonkey's People

Contributors

guybedford avatar jakechampion avatar lpereira avatar nicolo-ribaudo avatar primoly avatar tschneidereit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

starlingmonkey's Issues

Decompression corrupt input regressions

WPT cases regressing in tests/wpt-harness/expectations/compression/decompression-corrupt-input.tentative.any.js.json:

  • format 'deflate' field ADLER should be error for 255
  • format 'gzip' field ISIZE should be error for 1

Supporting top-level await

Top-level await support should be based on running the top-level application event loop to completion at preinitialization time to ensure the top-level has completed execution.

Errors can then be thrown as build errors.

Note that top-level fetch calls would effectively be build time fetches. Right now this would error but perhaps this may even be a useful feature if Wizering were to support it in future?

Note Fastly's JS Compute Runtime just landed support for TLA along the lines above.

Request.clone() regression

This feature needs to be reimplemented to support the associated WPT test at tests/wpt-harness/expectations/fetch/api/request/request-structure.any.js.json for "Request has clone method".

Resolving Node.js packages

I really like the static nature of source resolution during Wizering as an alternative to bundling.

Further to this, it could be worth implementing the full Node.js resolution algorithm to automatically bundle dependencies from npm as well.

Remove support for WASIp2 pre-releases

I think we should at this point remove the two host API implementations targeting WASIp2 pre-releases. I'm not aware of a use case for them anymore, and they served their purpose in validating the system for abstracting over different API implementations.

@guybedford and @elliottt any concerns on your end?

Refactor path resolution

I believe the plan is to implement a Node.js-compatible resolver in StarlingMonkey, and if we are going to do this we should ensure the path handling logic is robust.

Refactoring the current path resolution to avoid the use of new could be a first step. We could also look at using std::filesystem::path in the implementation as well.

Implement "immutable" headers guard

Fetch requests should return a response with headers marked with an "immutable" guard, where check_guard should always fail. In the Fetch spec it appears this is only ever done for responses.

Would this make sense to consider as an addition? Or is another implementation mechanism planned?

ready should either be a state or a host call, but not a mandatory hook on tasks

One issue with the current ready model is that it requires custom ready state to be tracked individually on each task, and task implementers might not do that correctly.

So it may make sense to refactor the Task model to instead treat ready as a state that is automatically managed in the task system itself in the engine, so that implementers of tasks don't need to implement the ready hook at all.

Alternatively, is_ready() is a host-hook in both WASI and Fastly APIs, that is typically only used in debugging situations currently.

This same alternative behaviour could be captured by explicitly making it a singular host hook, that takes a task and returns this boolean.

I think we should try to settle on one model or the other, or even remove the ready implementation entirely for now until it is needed (it's not currently needed I believe).

I'll gladly put together a PR if we can agree on a direction.

Intl support

How difficult would it be to add support for the Intl namespace object?

CMake sub-project errors

Trying to build a project using StarlingMonkey as a sub-project. Any helpful hints would be appreciated. ( Cmake is not my goto tool )

I was able to build and modify StarlingMonkey fine as a cloned repo, however I keep getting build errors as a sub-project.

clang: error: invalid thread model 'single' in '-mthread-model single' for this target

I have created a repo with a slimmed down build setup and a Dockerfile that depicts the errors.

Error stacks & code logs

As much as possible, all errors should provide a named and line-numbered source snippet indicating the location of the error.

Because we build sources directly, we can provide rich stacks for all errors, and should ensure that line numbers are properly displayed in this way for all error stacks through multiple call sites.

Build currently fails on aarch64-linux

wasi-sdk doesn't provide aarch64 releases, so the magic dependency fetching doesn't work.

I'm not sure if there's an easy way to override sources on a per-dependency basis, but if there is, we should document it - my cmake-foo is... very stale 😅

atob inconsistency

For some reason atob('__') succeeds when it is supposed to fail.

In the base64 test suite added in #35 this case was left out, but should be added back as soon as this can be fixed.

Error when sending `POST` requests with `fetch`

When trying to run the following code

addEventListener('fetch', (event) => handleRequest(event));
async function handleRequest(event) {
    try {
        let res = await fetch("https://echo.free.beeceptor.com", {
            method: "POST",
            headers: {},
            body: "hello world"
        }).then(data => data.text()).then(data => console.log(data))
        event.respondWith(new Response("Hello world"));
    } catch (e) {
        console.log(`Error: ${e}\nStack: ${e.stack}`);
    }
}

I get the following error

2024-03-01T15:29:59.022310Z ERROR spin_trigger_http: Error processing request: guest invocation failed

Caused by:
    0: error while executing at wasm backtrace:
           0: 0xa70d4b2 - wit-component:shim!indirect-wasi:http/[email protected][method]fields.append
           1: 0x3ca28 - <unknown>!wasi_http_types_method_fields_append
           2: 0x23c78 - <unknown>!host_api::HttpHeaders::append(std::__2::basic_string_view<char, std::__2::char_traits<char>>, std::__2::basic_string_view<char, std::__2::char_traits<char>>)
           3: 0xc59f2 - <unknown>!builtins::web::fetch::Headers::append_header_value(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<JS::Value>, char const*)
           4: 0xcdad1 - <unknown>!builtins::web::fetch::Headers::maybe_add(JSContext*, JS::Handle<JSObject*>, char const*, char const*)
           5: 0xd65fb - <unknown>!builtins::web::fetch::RequestOrResponse::extract_body(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>)
           6: 0xe4fb2 - <unknown>!builtins::web::fetch::Request::create(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::Handle<JS::Value>)
           7: 0xc2f78 - <unknown>!builtins::web::fetch::fetch(JSContext*, unsigned int, JS::Value*)
           8: 0x6a3fcf - <unknown>!js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason)
           9: 0x69e029 - <unknown>!js::Interpret(JSContext*, js::RunState&)
          10: 0x696478 - <unknown>!js::RunScript(JSContext*, js::RunState&)
          11: 0x6a3e0b - <unknown>!js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason)
          12: 0x6a4ae0 - <unknown>!js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason)
          13: 0x733872 - <unknown>!JS_CallFunctionValue(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>)
          14: 0xf4add - <unknown>!builtins::web::fetch::fetch_event::dispatch_fetch_event(JS::Handle<JSObject*>, double*)
          15: 0xf43ed - <unknown>!exports_wasi_http_incoming_handler_handle
          16: 0x47f27 - <unknown>!__wasm_export_exports_wasi_http_incoming_handler_handle
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    1: unknown handle index 0    

setInterval stalls event loop entirely

Adding an arbitrary setInterval(() => {}) inside of a server handler will stall the server handler from completing entirely.

We need to:

  1. Ensure that other tasks continue running correctly
  2. Ensure that the engine shutdown is properly managed when there are ongoing timer tasks

Release build is broken

The release build is currently not working correctly. This may have happened in the latest SpiderMonkey update.

Build error reported:

[ 90%] Linking CXX executable starling.wasm
wasm-ld: error: duplicate symbol: encoding_for_label
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_for_label_no_replacement
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_for_bom
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_name
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_can_encode_everything
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_is_ascii_compatible
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_is_single_byte
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_output_encoding
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_new_decoder
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_new_decoder_with_bom_removal
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_new_decoder_without_bom_handling
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_new_decoder_into
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_new_decoder_with_bom_removal_into
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_new_decoder_without_bom_handling_into
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_new_encoder
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_new_encoder_into
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_utf8_valid_up_to
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_ascii_valid_up_to
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: encoding_iso_2022_jp_ascii_valid_up_to
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

wasm-ld: error: duplicate symbol: decoder_free
>>> defined in librust_encoding.a(encoding_c-6718b175c3733dba.encoding_c.5cbbfd59fed8dac6-cgu.0.rcgu.o)
>>> defined in /Users/gbedford/Projects/StarlingMonkey/deps/cpm_cache/spidermonkey-release/0724b80a62196812717c8fbd9286c717d83c4113/spidermonkey-release/lib/libjsrust.a(jsrust-53a982b7a59e3f77.jsrust.86da07e83e4d91bd-cgu.0.rcgu.o)

I've tried various tweaks to the Rust configurations here, I'm not sure if this is an upstream issue in the embedding project itself.

Response.json() support

Response.json was disabled in the fetch implementation, resulting in WPT regressions, this needs to be uncommented and reimplemented.

Outbound request error - `out of bounds memory access`

I was attempting to send an outbound request to https://echo.free.beeceptor.com by modifying tests/smoke.js

...
let p = fetch("https://example.com");
let test = fetch("https://echo.free.beeceptor.com")
// let p2 = fetch(url);
// let resolve, reject;
let responsePromise = new Promise(async (res, rej) => {
    resolve = res;
    reject = rej;
});
event.respondWith(responsePromise);
let temp = await test
console.log(await temp.text())
let response = await p;
// console.log(response);
...

Running the program results in

RUST_LOG=spin_trigger_http=trace  spin up
Logging component stdio to ".spin/logs/"
2024-02-10T19:33:37.736324Z TRACE spin_trigger_http: Constructed router for application js-component: [(Wildcard(""), "js-component")]    

Serving http://127.0.0.1:3000
2024-02-10T19:33:37.736441Z  INFO spin_trigger_http: Serving http://127.0.0.1:3000    
Available Routes:
  js-component: http://127.0.0.1:3000 (wildcard)
2024-02-10T19:33:39.768995Z  INFO spin_trigger_http: Processing request for application js-component on URI http://127.0.0.1:3000/    
2024-02-10T19:33:39.769027Z TRACE spin_trigger_http::handler: Executing request using the Spin executor for component js-component
2024-02-10T19:35:41.517198Z TRACE spin_trigger_http::handler: wasi-http memory consumed: 2154889216
2024-02-10T19:35:41.543521Z ERROR spin_trigger_http: Error processing request: guest invocation failed

Caused by:
    0: error while executing at wasm backtrace:
           0: 0x3239e - <unknown>!std::__2::unique_ptr<unsigned char [], std::__2::default_delete<unsigned char []>>::release[abi:v160000]()
           1: 0x26acd - <unknown>!std::__2::unique_ptr<unsigned char [], std::__2::default_delete<unsigned char []>>::unique_ptr[abi:v160000](std::__2::unique_ptr<unsigned char [], std::__2::default_delete<unsigned char []>>&&)
           2: 0x38989 - <unknown>!host_api::HostBytes::HostBytes(host_api::HostBytes&&)
           3: 0xfed24 - <unknown>!host_api::HostBytes* std::__2::construct_at[abi:v160000]<host_api::HostBytes, host_api::HostBytes, host_api::HostBytes*>(host_api::HostBytes*, host_api::HostBytes&&)
           4: 0xfe744 - <unknown>!void std::__2::allocator_traits<std::__2::allocator<host_api::HostBytes>>::construct[abi:v160000]<host_api::HostBytes, host_api::HostBytes, void, void>(std::__2::allocator<host_api::HostBytes>&, host_api::HostBytes*, host_api::HostBytes&&)
           5: 0xff31e - <unknown>!std::__2::reverse_iterator<host_api::HostBytes*> std::__2::__uninitialized_allocator_move_if_noexcept[abi:v160000]<std::__2::allocator<host_api::HostBytes>, std::__2::reverse_iterator<host_api::HostBytes*>, std::__2::reverse_iterator<host_api::HostBytes*>, std::__2::reverse_iterator<host_api::HostBytes*>>(std::__2::allocator<host_api::HostBytes>&, std::__2::reverse_iterator<host_api::HostBytes*>, std::__2::reverse_iterator<host_api::HostBytes*>, std::__2::reverse_iterator<host_api::HostBytes*>)
           6: 0xfeb14 - <unknown>!std::__2::vector<host_api::HostBytes, std::__2::allocator<host_api::HostBytes>>::__swap_out_circular_buffer(std::__2::__split_buffer<host_api::HostBytes, std::__2::allocator<host_api::HostBytes>&>&)
           7: 0xfdfed - <unknown>!void std::__2::vector<host_api::HostBytes, std::__2::allocator<host_api::HostBytes>>::__emplace_back_slow_path<host_api::HostBytes>(host_api::HostBytes&&)
           8: 0xed20e - <unknown>!host_api::HostBytes& std::__2::vector<host_api::HostBytes, std::__2::allocator<host_api::HostBytes>>::emplace_back<host_api::HostBytes>(host_api::HostBytes&&)
           9: 0xecce7 - <unknown>!builtins::web::fetch::(anonymous namespace)::read_from_handle_all(JSContext*, host_api::HttpIncomingBody*)
          10: 0xec85c - <unknown>!builtins::web::fetch::RequestOrResponse::consume_body_handle_for_bodyAll(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::CallArgs)
          11: 0x103afc - <unknown>!bool internal_method<&builtins::web::fetch::RequestOrResponse::consume_body_handle_for_bodyAll(JSContext*, JS::Handle<JSObject*>, JS::Handle<JS::Value>, JS::CallArgs)>(JSContext*, unsigned int, JS::Value*)
          12: 0x26ebd9 - <unknown>!CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), js::CallReason, JS::CallArgs const&)
          13: 0x26e409 - <unknown>!js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason)
          14: 0x26fefb - <unknown>!InternalCall(JSContext*, js::AnyInvokeArgs const&, js::CallReason)
          15: 0x270256 - <unknown>!js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason)
          16: 0x2e8647 - <unknown>!js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::MutableHandle<JS::Value>)
          17: 0x4fe029 - <unknown>!PromiseReactionJob(JSContext*, unsigned int, JS::Value*)
          18: 0x26ebd9 - <unknown>!CallJSNative(JSContext*, bool (*)(JSContext*, unsigned int, JS::Value*), js::CallReason, JS::CallArgs const&)
          19: 0x26e409 - <unknown>!js::InternalCallOrConstruct(JSContext*, JS::CallArgs const&, js::MaybeConstruct, js::CallReason)
          20: 0x26fefb - <unknown>!InternalCall(JSContext*, js::AnyInvokeArgs const&, js::CallReason)
          21: 0x270256 - <unknown>!js::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, js::AnyInvokeArgs const&, JS::MutableHandle<JS::Value>, js::CallReason)
          22: 0x3a7f26 - <unknown>!JS::Call(JSContext*, JS::Handle<JS::Value>, JS::Handle<JS::Value>, JS::HandleValueArray const&, JS::MutableHandle<JS::Value>)
          23: 0x457828 - <unknown>!js::InternalJobQueue::runJobs(JSContext*)
          24: 0x4571e0 - <unknown>!js::RunJobs(JSContext*)
          25: 0x20200 - <unknown>!core::EventLoop::run_event_loop(api::Engine*, double, JS::MutableHandle<JS::Value>)
          26: 0x1d198 - <unknown>!api::Engine::run_event_loop(JS::MutableHandle<JS::Value>)
          27: 0x10acea - <unknown>!exports_wasi_http_0_2_0_incoming_handler_handle
          28: 0x4c156 - <unknown>!__wasm_export_exports_wasi_http_0_2_0_incoming_handler_handle
       note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information
    1: memory fault at wasm address 0xfffffff0 in linear memory of size 0x80710000
    2: wasm trap: out of bounds memory access    

Clean up handling of lifecycle phases

@guybedford pointed out in a comment that the name for a newly introduced phase of a JS app's lifecycle is a bit off.

Rather than addressing that in-place, I'm filing this issue to track overhauling our overall management of these phases. We have a spattering of different things, from wizening to initialized to runtime, etc. I think we should clean all those up, and introduce a state-transitioning concept, capturing everything from "the runtime has been compiled, but no part of it ever ran" to "the JS runtime has been initialized, but nothing else", to "some code is currently being evaluated", etc, all the way to either "the main execution phase has finished, but could be re-run" or "the main execution phase has finished, and no more code must be run in this instance, because no cleanup happened".

Task subclassing model

Fastly uses an AsyncTask model that stores on the AsyncTask class, the promise, and a function that will handle the process the "next steps" after the handle is ready, to then eventually resolve the promise.

On the other hand, StarlingMonkey only supports a singular run or cancel method on tasks.

So we have two separate task models in play:

  1. Fastly's task model, based on async handles, having "next steps" which resolve a promise ala spec algorithm descriptions
  2. StarlingMonkey's task model, where each subsystem subclasses their own tasks, and implements a single synchronous run function which is effectively those same "next steps", but with context and state specific to each subsystem subclass.

In adapting Fastly async tasks to this model, the easiest way to do this is to just subclass AsyncTask with FastlyAsyncTask that provides the original slots - [[Promise]] and [[NextSteps]], and then to use that like we had before.

This works fine, but I wanted to post this issue to see if there's more context on StarlingMonkey's async model that I'm perhaps missing.

CI integration

CI should at the very least be running smoke tests, and in due course full WPT tests.

Wasm build size

What is the expected size for debug and release builds?

I'm seeing quite large binary files generated for both debug and release configurations:

$ ls cmake-build-release/smoke.js.wasm
150M cmake-build-release/smoke.js.wasm

$ ls cmake-build-debug/smoke.js.wasm
225M cmake-build-debug/smoke.js.wasm

Transform streams regression

For the test "writer.abort() inside size() should work" In tests/wpt-harness/expectations/streams/transform-streams/reentrant-strategies.any.js.json.

Using OpenSSL from a subproject

When trying to include OpenSSL from a subproject via, I'm getting an error for these includes, for example:

#include "openssl/evp.h"

gives that the include is not found.

Can we make it a feature of the subproject build that the OpenSSL headers are reachable? My CMake knowldge is fairly limited here - what would be involved to implement that?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.