ydnar / wasm-tools-go Goto Github PK

View Code? Open in Web Editor NEW

42.0 8.0 5.0 2.11 MB

WebAssembly + Component Model tools for Go

License: Apache License 2.0

Go 99.93% Makefile 0.07%

go golang wasi wasm webassembly component-model

wasm-tools-go's Introduction

wasm-tools-go

WebAssembly + WASI tools for Go

About

This repository contains code to generate Go bindings for Component Model interfaces defined in WIT (WebAssembly Interface Type) files. The goal of this project is to accelerate adoption of the Component Model and development of WASI 0.2+ in Go.

Component Model

Package cm contains helper types and functions used by generated packages, such as option<t>, result<ok, err>, variant, list, and resource. These are intended for use by generated Component Model bindings, where the caller converts to a Go equivalent. It attempts to map WIT semantics to their equivalent in Go where possible.

Note on Memory Safety

Package cm and generated bindings from wit-bindgen-go may have compatibility issues with the Go garbage collector, as they directly represent variant and result types as tagged unions where a pointer shape may be occupied by a non-pointer value. The GC may detect and throw an error if it detects a non-pointer value in an area it expects to see a pointer. This is an area of active development.

`wit-bindgen-go`

WIT → Go

The wit-bindgen-go tool can generate Go bindings for WIT interfaces and worlds. If wasm-tools is installed and in $PATH, then wit-bindgen-go can load WIT directly.

wit-bindgen-go generate ../wasi-cli/wit

Otherwise, pass the JSON representation of a fully-resolved WIT package:

wit-bindgen-go generate wasi-cli.wit.json

Or pipe via stdin:

wasm-tools component wit -j --all-features ../wasi-cli/wit | wit-bindgen-go generate

JSON → WIT

For debugging purposes, wit-bindgen-go can also convert a JSON representation back into WIT. This is useful for validating that the intermediate representation faithfully represents the original WIT source.

wit-bindgen-go wit example.wit.json

WIT → JSON

The wit package can decode a JSON representation of a fully-resolved WIT file. Serializing WIT into JSON requires wasm-tools v1.210.0 or higher. To convert a WIT file into JSON, run wasm-tools with the -j argument:

wasm-tools component wit -j --all-features example.wit

This will emit JSON on stdout, which can be piped to a file or another program.

wasm-tools component wit -j --all-features example.wit > example.wit.json

License

This project is licensed under the Apache 2.0 license with the LLVM exception. See LICENSE for more details.

Contributing

Developing and testing wit-bindgen-go requires an up-to-date installation of Go, TinyGo, and wasm-tools.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be licensed as above, without any additional terms or conditions.

wasm-tools-go's People

Contributors

Stargazers

Watchers

Forkers

mossaka omnide dgryski anima-os-dev rajatjindal

wasm-tools-go's Issues

wit/bindgen: implement post-return functions

For exported (go:wasmexport) functions, the Canonical ABI specifies an optional post-return function, which is exposed to the WebAssembly host as:

cabi_post_ + original function name

The purpose of the post-return function is to allow the component to free any memory allocated in the returned value.

Given that Go (and TinyGo) uses a garbage collector to free unreferenced memory, the GC should eventually free these allocations. However, pointers may be passed to the host (out of the component), leaving no references to the memory in the Go stack or heap. This could lead to memory being prematurely freed, causing a crash.

Given the constraints that this code will be executed under( WASI 0.2 and components currently operated in a single-threaded, non-reentrant manner, where memory is copied immediately from guest to host), I believe it is unlikely that the Go GC will have an opportunity to free the memory before the host copies it.

However, as we look to implement WASI 0.2 and Component Model on mainline Go, we should probably implement a mechanism to ensure pointers are referenced when passed in and out of the WebAssembly boundary to ensure they are not garbage collected before use.

Implementation

The implementation of post-return functions (cabi_post_*) was wrong. #119 implemented two changes:

Correct the function signature of post-return functions, deriving from the flattened (Core WebAssembly) version of exported functions, rather than the original function signature.
Disable generation of Go *PostReturn functions, including user-defined side, until a point at which we can generate a correct implementation.

The implementation of post-return functions for Go needs more design work. Given that Go has a GC, the returned pointers should eventually be garbage collected. Areas of exploration:

Create a bookkeeping mechanism to keep track of pointers passed to wasmexport functions and free/release them when passed back via a post-return function.
Determine whether this should also apply to any allocations created with cabi_realloc.
For Go (as opposed to TinyGo), figure out a way to handle pointers in variant types, which need to represent both non-pointer and pointer values in the same memory location. This might involve sidecar returns from variant-returning or accepting functions.

cm, wit/bindgen: rethink how named Go types for option, result, and list are declared

Currently, named WIT option, result, and list types are declared as new types:

type StringList cm.List[String]

This means that any methods declared on List are not available on StringList. Two alternative methods for declaring these types exist: aliasing and embedding.

Aliasing

// Alias declaration
type StringList = cm.List[String]

This allows List methods to be used, but precludes adding new methods to the alias type. It would reduce type safety somewhat, allowing substitution between unrelated named alias types that are otherwise equivalent.

Embedding

// Embedded declaration
type StringList = struct { cm.List[String] }

Embedding allows List methods to be used, allows adding new methods to the named type, and preserves type safety between unrelated but structurally identical named types.

Given that the current Go implementations of list, option, and result in package cm are opaque struct with no public members, embedding would not worsen the ergonomics of using these named types.

proposal: add a String() to do reverse enum lookup

it would be nice to be able to do a reverse enum lookup and get string of error code based on the int value.

how do you feel about adding something like below to the generated bindings:

type ErrorCode uint8

const (
	// Permission denied, similar to `EACCES` in POSIX.
	ErrorCodeAccess ErrorCode = iota

	// Resource unavailable, or operation would block, similar to `EAGAIN` and `EWOULDBLOCK`
	// in POSIX.
	ErrorCodeWouldBlock
)

func (e ErrorCode) String() string {
	return []string{
		"ErrorCodeAccess",
		"ErrorCodeWouldBlock",
	}[e]
}

where to find example projects based on this tool?

I want to write a simple project with this tool, and compile it as wasm component.
Really appreciate if someone could provide a guide about this.

proposal: wit/bindgen: fail if flattened function params include a variant type with overlapping i32 + i64

This will partially address #94 by preventing code generation for wasi:http/proxy world until wit-bindgen-go generates fully-flattened function signatures composed entirely of int32, uint32, int64, uint64, float32, float64, or a pointer.

proposal: generate Version constant in each package

For each generated Go package, add a Version constant which corresponds to the WIT package version, e.g.:

package wasi:io/streams@0.2.0

package streams

const Version = "0.2.0"

proposal: generate export bindings

This document describes a design and mechanism for generated Go bindings for Component Model exports.

Background

The Component Model describes a mechanism for a WebAssembly program to export functions and methods that adhere to a specific WIT world (an API contract).

WIT worlds can export whole interfaces (e.g. wasi:cli/command) or individual functions. A WIT interface contains freestanding functions and types, which can include resource types with methods.

Exports are currently declared in a WIT file with a world that has one or more export directives, which export either an entire WIT interface or an individual function. This document presumes the existence of //go:wasmexport or similar (e.g. //export in TinyGo) to expose exported symbols in the resulting WebAssembly binary.

The design proposed here is similar to the approach taken by wit-bindgen, which generates Go bindings on top of Cgo suitable for TinyGo.

TODO: add more context

Proposed Design

For each exported function and method, wit-bindgen-go would generate a corresponding Go function with the relevant //go:wasmexport directive. The generated function would call a user-provided function. If the user-provided function is undefined (nil), the program will panic.

The expectation is that user code would import a generated Go package and call an Export() function or similar to register its implementation of the required interface(s).

Exported Interfaces

For each WIT interface, create a Go package-level exports interface type named Exports, containing one Go method for each freestanding WIT function in the WIT interface, a Go method to map resource handles to a Go interface (more below), and any resource constructor(s) or static function(s) if present.

// Exports represents the exported interface "wasi:filesystem/types".
type Exports interface {
    // freestanding functions
    // resource types
    // resource constructors and static methods
}

For each resource type defined in the WIT interface, an additional Go resource interface would be defined:

// DescriptorInterface represents the exported resource "wasi:filesystem/types#descriptor".
type DescriptorInterface interface {
    // resource methods
    // TODO: ResourceDrop, ResourceRep, and other administrative functions
}

For each resource type, a Go method is added to the exports interface with the name of the resource type, to translate an opaque handle value (i32) to the resource interface, along with constructor(s) and static methods:

// Exports represents the exported interface "wasi:filesystem/types".
type Exports interface {
    // Descriptor maps a handle to an exported DescriptorInterface.
    Descriptor(handle Descriptor) DescriptorInterface

    // NewDescriptor represents the exported constructor for `wasi:filesystem/types#descriptor".
    NewDescriptor() Descriptor
}

Usage

In order for a Go program to satisfy the runtime requirements of its component exports, user code must implement the exports interface by creating a exports implementation type with methods that match the exports interface:

package main

import "wasi/filesystem/types"

func init() {
    types.Export(&FilesystemExports{})
}

type FilesystemExports struct{}

func (e *FilesystemExports) Descriptor(handle types.Descriptor) DescriptorInterface {
    // TODO: use Component Model resource rep to return pointer
}

func (e *FilesystemExports) NewDescriptor() types.Descriptor {
    // ...
}

type Descriptor struct { /* ... */ }

func (d *Descriptor) Destructor() { /* ... */ }

issue when building and running a http-incoming-handler code

Hello @ydnar, first of all thank you for your work on this project.

I am trying out wasip2 changes to run a simple http incoming handler, and running into following issue:

Error:

error: failed to encode a component from module

Caused by:
    0: failed to validate import interface `wasi:http/[email protected]`
    1: type mismatch for function `[static]response-outparam.set`: expected `[I32, I32, I32, I32, I64, I32, I32, I32, I32] -> []` but found `[I32, I32, I32, I32, I32, I32, I32, I32, I32] -> []`
error: wasm-tools failed: exit status 1

Environment:

Macbook pro M2 processor
Tinygo compiled from https://github.com/dgryski/tinygo/tree/dgryski/wasi-preview-2 (3ef711afab91ddedc66dc06d91eaa16cdb1f8c6a)
wit-bindgen-go compiled from https://github.com/ydnar/wasm-tools-go (aa78230)

steps to reproduce function mismatch error

git clone https://github.com/rajatjindal/wasip2-http-incoming-handler.git
git checkout 058507c5540fa69524f717f32f31b53328329a0f
tinygo build -target=wasip2 -wit-package ./wit --wit-world http-trigger -o main.wasm -x -work main.go

Full logs:

WORK=/var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888
wasm-ld --stack-first --no-demangle -L /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/ -o /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888/main /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888/main.o /Users/rajatjindal/Library/Caches/tinygo/compiler-rt-wasm32-unknown-wasi-generic/lib.a /Users/rajatjindal/Library/Caches/tinygo/obj-363c24a529264e57aec91eacb3cbc247bacb915b67966242e30f7fb9.bc /Users/rajatjindal/Library/Caches/tinygo/obj-870ef2ac5a778e9704c61e87bde8b24392ff596a3b0126fe5821e7b1.bc /Users/rajatjindal/Library/Caches/tinygo/wasmbuiltins-wasm32-unknown-wasi-generic/lib.a -mllvm -mcpu=generic --lto-O2 --thinlto-cache-dir=/Users/rajatjindal/Library/Caches/tinygo/thinlto -mllvm --rotation-max-header-size=0
/opt/homebrew/bin/wasm-opt --asyncify -Oz -g /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888/main --output /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888/main
wasm-tools component embed -w http-trigger ./wit /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888/main -o /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888/main
wasm-tools component new /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888/main -o /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo4056239888/main
error: failed to encode a component from module

Caused by:
    0: failed to validate import interface `wasi:http/[email protected]`
    1: type mismatch for function `[static]response-outparam.set`: expected `[I32, I32, I32, I32, I64, I32, I32, I32, I32] -> []` but found `[I32, I32, I32, I32, I32, I32, I32, I32, I32] -> []`
error: wasm-tools failed: exit status 1

I am able to fix that error by making following change to the generated code here wasi/http/types/types.wit.go:

diff --git a/main.go b/main.go
index 9edb840..efb9e22 100644
--- a/main.go
+++ b/main.go
@@ -8,7 +8,7 @@ import (
 
 func init() {
 	incominghandler.Handle = func(request types.IncomingRequest, responseOut types.ResponseOutparam) {
-		err := cm.Err[cm.ErrResult[types.OutgoingResponse, types.ErrorCode]](types.ErrorCodeConfigurationError())
+		err := cm.Err[cm.OKResult[types.OutgoingResponse, types.ErrorCode]](types.ErrorCodeConfigurationError())
 		types.ResponseOutparamSet(cm.ResourceNone, err)
 	}
 }
diff --git a/wasi/http/types/types.wit.go b/wasi/http/types/types.wit.go
index f7c3e6c..b52d16e 100644
--- a/wasi/http/types/types.wit.go
+++ b/wasi/http/types/types.wit.go
@@ -1487,13 +1487,13 @@ func (self ResponseOutparam) wasmimport_ResourceDrop()
 //	error-code>)
 //
 //go:nosplit
-func ResponseOutparamSet(param ResponseOutparam, response cm.ErrResult[OutgoingResponse, ErrorCode]) {
+func ResponseOutparamSet(param ResponseOutparam, response cm.OKResult[OutgoingResponse, ErrorCode]) {
 	wasmimport_ResponseOutparamSet(param, response)
 }
 
 //go:wasmimport wasi:http/[email protected] [static]response-outparam.set
 //go:noescape
-func wasmimport_ResponseOutparamSet(param ResponseOutparam, response cm.ErrResult[OutgoingResponse, ErrorCode])
+func wasmimport_ResponseOutparamSet(param ResponseOutparam, response cm.OKResult[OutgoingResponse, ErrorCode])
 
 // StatusCode represents the imported type "wasi:http/[email protected]#status-code".
 //

After which I am able to compile the code successfully:

$ tinygo build -target=wasip2 -wit-package ./wit --wit-world http-trigger -o main.wasm -x -work main.go

WORK=/var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658
wasm-ld --stack-first --no-demangle -L /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/ -o /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658/main /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658/main.o /Users/rajatjindal/Library/Caches/tinygo/compiler-rt-wasm32-unknown-wasi-generic/lib.a /Users/rajatjindal/Library/Caches/tinygo/obj-363c24a529264e57aec91eacb3cbc247bacb915b67966242e30f7fb9.bc /Users/rajatjindal/Library/Caches/tinygo/obj-870ef2ac5a778e9704c61e87bde8b24392ff596a3b0126fe5821e7b1.bc /Users/rajatjindal/Library/Caches/tinygo/wasmbuiltins-wasm32-unknown-wasi-generic/lib.a -mllvm -mcpu=generic --lto-O2 --thinlto-cache-dir=/Users/rajatjindal/Library/Caches/tinygo/thinlto -mllvm --rotation-max-header-size=0
/opt/homebrew/bin/wasm-opt --asyncify -Oz -g /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658/main --output /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658/main
wasm-tools component embed -w http-trigger ./wit /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658/main -o /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658/main
wasm-tools component new /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658/main -o /var/folders/5r/6y6fr12x1dq6qnt74zn4_1wm0000gn/T/tinygo2565817658/main

and able to start the wasmtime serve with it:

RUST_LOG=trace WASMTIME_BACKTRACE_DETAILS=1 wasmtime serve main.wasm --wasi common
Serving HTTP on http://0.0.0.0:8080/

However, when I curl this endpoint, I get following stacktrace:

    context: "error while executing at wasm backtrace:\n    0: 0x1332a - wit-component:shim!indirect-wasi:io/[email protected][method]output-stream.blocking-write-and-flush\n    1: 0x25f5 - (internal/wasi/io/v0.2.0/streams.OutputStream).BlockingWriteAndFlush\n                    at /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/src/internal/wasi/io/v0.2.0/streams/streams.wit.go:266:39              - runtime.putchar\n                    at /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/src/runtime/runtime_tinygowasmp2.go:29:38\n    2: 0x24e1 - runtime.printnl\n                    at /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/src/runtime/print.go:290:9\n    3: 0x14b0 - runtime._panic\n                    at /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/src/runtime/panic.go:64:9\n    4: 0x36b6 - (*github.com/ydnar/wasm-tools-go/cm.result[github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse, github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse, github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.ErrorCode]).validate[github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.ErrorCode]\n                    at /Users/rajatjindal/go/pkg/mod/github.com/ydnar/[email protected]/cm/result.go:90:8\n    5:  0x4d8 - github.com/ydnar/wasm-tools-go/cm.Err[github.com/ydnar/wasm-tools-go/cm.OKResult[github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse, github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.ErrorCode] github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.ErrorCode]\n                    at /Users/rajatjindal/go/pkg/mod/github.com/ydnar/[email protected]/cm/result.go:144:12\n    6:  0x47b - main.init#1$1\n                    at /Users/rajatjindal/go/src/github.com/rajatjindal/wasip2-http-incoming-handler/main.go:11:70              - wasi:http/[email protected]#handle\n                    at /Users/rajatjindal/go/src/github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/incoming-handler/incoming-handler.wit.go:36:8",
    source: "unknown handle index 0",
}    
error: hyper::Error(User(Service), guest never invoked `response-outparam::set` method: error while executing at wasm backtrace:
    0: 0x1332a - wit-component:shim!indirect-wasi:io/[email protected][method]output-stream.blocking-write-and-flush
    1: 0x25f5 - (internal/wasi/io/v0.2.0/streams.OutputStream).BlockingWriteAndFlush
                    at /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/src/internal/wasi/io/v0.2.0/streams/streams.wit.go:266:39              - runtime.putchar
                    at /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/src/runtime/runtime_tinygowasmp2.go:29:38
    2: 0x24e1 - runtime.printnl
                    at /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/src/runtime/print.go:290:9
    3: 0x14b0 - runtime._panic
                    at /Users/rajatjindal/go/src/github.com/tinygo-org/tinygo/build/release/tinygo/src/runtime/panic.go:64:9
    4: 0x36b6 - (*github.com/ydnar/wasm-tools-go/cm.result[github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse, github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse, github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.ErrorCode]).validate[github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.ErrorCode]
                    at /Users/rajatjindal/go/pkg/mod/github.com/ydnar/[email protected]/cm/result.go:90:8
    5:  0x4d8 - github.com/ydnar/wasm-tools-go/cm.Err[github.com/ydnar/wasm-tools-go/cm.OKResult[github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse, github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.ErrorCode] github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.OutgoingResponse github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/types.ErrorCode]
                    at /Users/rajatjindal/go/pkg/mod/github.com/ydnar/[email protected]/cm/result.go:144:12
    6:  0x47b - main.init#1$1
                    at /Users/rajatjindal/go/src/github.com/rajatjindal/wasip2-http-incoming-handler/main.go:11:70              - wasi:http/[email protected]#handle
                    at /Users/rajatjindal/go/src/github.com/rajatjindal/wasip2-http-incoming-handler/wasi/http/incoming-handler/incoming-handler.wit.go:36:8

Caused by:
    unknown handle index 0)

any pointers to help get past this error will be very helpful.

thank you again

How do I build this project?

There is no Makefile and there are no instructions for building the binary.

I tried go build and go install but neither worked.

> go build cmd/wit-bindgen-go/*.go

cmd/wit-bindgen-go/main.go:8:2: no required module provides package github.com/urfave/cli/v3; to add it:
        go get github.com/urfave/cli/v3
cmd/wit-bindgen-go/main.go:10:2: no required module provides package github.com/ydnar/wasm-tools-go/cmd/wit-bindgen-go/cmd/generate; to add it:
        go get github.com/ydnar/wasm-tools-go/cmd/wit-bindgen-go/cmd/generate
cmd/wit-bindgen-go/main.go:11:2: no required module provides package github.com/ydnar/wasm-tools-go/cmd/wit-bindgen-go/cmd/print; to add it:
        go get github.com/ydnar/wasm-tools-go/cmd/wit-bindgen-go/cmd/print
cmd/wit-bindgen-go/main.go:12:2: no required module provides package github.com/ydnar/wasm-tools-go/cmd/wit-bindgen-go/cmd/wit; to add it:
        go get github.com/ydnar/wasm-tools-go/cmd/wit-bindgen-go/cmd/wit

> go install github.com/ydnar/wasm-tools-go/cmd/wit-bindgen-go@latest

go: downloading github.com/ydnar/wasm-tools-go v0.0.0-20240419173811-42579b015632
go: github.com/ydnar/wasm-tools-go/cmd/wit-bindgen-go@latest (in github.com/ydnar/[email protected]):
        The go.mod file for the module providing named packages contains one or
        more replace directives. It must not contain directives that would cause
        it to be interpreted differently than if it were the main module.

How to build? 🤷‍♂️

wit, wit/bindgen: support abstract "pointer" type, size, and alignment

In order to support wasm64, with 64-bit wide pointers, we need to relax the constraint in the Component Model and Canonical ABI that pointer types fit in 32 bits.

Ideally the generated Go code for a component can handle pointer widths of either 32 or 64 bits without baking assumptions into the precise memory layout of structures into the lifting and lowering code.

Tasks

Reconcile this with proposed wasm64 support in the Component Model spec.
Introduce "integer pointer" type (e.g. uintptr) with size/alignment that varies based on arch. For instance, when flattening a variant type with a string or list<T>, the flattened pointers resolve to a uintptr in Go.
Thread this flexible width concept through the ABI, lifting, and lowering code in wit and wit/bindgen.

proposal: wit/bindgen: flatten wasmimport and wasmexport function params to primitive types

Generate fully-flattened function signatures composed entirely of int32, uint32, int64, uint64, float32, float64, *T, or unsafe.Pointer.

Fixes #94. Related to #97.

cm, wit/bindgen: implement flags types with > 64 labels

Related to WebAssembly/component-model#370, the Go code generation for flags types currently supports flags with <= 64 labels. This is consistent with wit-bindgen (for C and possibly other languages) code generation.

Tasks

Implement flags with > 64 labels, with appropriate bindings generation for flags types that won’t fit in a single integer type.
Implement CABI lift and lower code for flags types larger than uint64.

wit/bindgen: tuples represented as Go structs have un-exported (lowercase) field names

This makes them unusable from caller code.

module requires an import interface named `wasi:http/[email protected]`

I am working on generating tinygo bindings that use wasi-http 0.2.0.

I am using tinygo version tinygo version 0.33.0-dev-6f462fb darwin/amd64 (using go version go1.22.4 and LLVM version 18.1.2)

The latest code in wasm-tools-go.

I am trying to track down if the HTTP bindings for wasi-http are not declaring the correct imports or if this is a tinygo wasip2 problem that I am not understanding.

wit-bindgen-go generate --world platform --out ./gen/platform/ ./wit

world platform {
    include wasi:http/[email protected];
    import wasi:clocks/[email protected];
    import wasi:clocks/[email protected];
    import wasi:random/[email protected];
    import wasi:cli/[email protected];
    import wasi:cli/[email protected];
    import wasi:cli/[email protected];
    export wasi:cli/[email protected]; 
}

the bindings are generated successfully and I have implemented HTTP.Transport class to use the HTTP/outgoing-handler.

I have also tried defining the world like

world platform {
    import wasi:http/[email protected];
    import wasi:http/[email protected];
    import wasi:clocks/[email protected];
    import wasi:clocks/[email protected];
    import wasi:random/[email protected];
    import wasi:cli/[email protected];
    import wasi:cli/[email protected];
    import wasi:cli/[email protected];
    export wasi:cli/[email protected]; 
}

My component is roughly the following, where RoundTrip uses the generated HTTP bindings.

func main() {
	rt := transport.RoundTrip{}
	request, err := http.NewRequest("GET", "https://postman-echo.com/get", nil)
	if err != nil {
		fmt.Println(err)
		return
	}
	resp, err := rt.RoundTrip(request)
	if err != nil {
		fmt.Println(err)
		return
	}
	fmt.Println(resp)
}

Building with the dev branch in tinygo:

tinygo build -o main.wasm -target=wasip2

results in:

error: failed to encode a component from module

Caused by:
    0: failed to decode world from module
    1: module was not valid
    2: module requires an import interface named `wasi:http/[email protected]`
error: wasm-tools failed: exit status 1

Any guidance/thoughts would be great!

Apologies in advance if this is not a wasm-tools-go issue.

Bindings can be found at - https://github.com/hayride-dev/wit

I would ultimately like to run this component with wasmtime-cli

wit/bindgen: support tuples and records with zero fields

Generating Go code with a zero-length tuple currently panics. It should not panic.

Should these be represented as struct{}?

weird value printed when using function returning

Hello again,

I am trying to use wasm-tools-go to generate bindings for the following wit:

interface variables {
    /// Get an application variable value for the current component.
    ///
    /// The name must match one defined in in the component manifest.
    get: func(name: string) -> result<string, error>;

    /// The set of errors which may be raised by functions in this interface.
    variant error {
        /// The provided variable name is invalid.
        invalid-name(string),
        /// The provided variable is undefined.
        undefined(string),
        /// A variables provider specific error has occurred.
        provider(string),
        /// Some implementation-specific error has occurred.
        other(string),
    }
}

when using the bindings generated, I see weird value printed:

key: password, val is "\x00\x00\x00\x00\x00\x00"

pseudo code:

func Get(key string) (string, error) {
	result := variables.Get(key)
	if result.IsErr() {
		return "", errorVariantToError(*result.Err())
	}

	stdout.GetStdout().Write(cm.ToList([]byte(fmt.Sprintf("key: %s, val is %q \n", key, *result.OK()))))

	return *result.OK(), nil
}

I printed the value on host component implementation side, and it seems to be printing and returning the correct value.

any pointers what could be wrong here? thank you so much for all the help so far.

proposal: implement Canonical ABI realloc (cabi_realloc)

In order for host → guest calls to pass complex types with unknown sizes, the host must be able to allocate memory in the guest. See: https://github.com/search?q=repo%3Abytecodealliance%2Fwit-bindgen%20cabi_realloc&type=code

The realloc function defined in the Canonical ABI has the following signature: (func (param i32 i32 i32 i32) (result i32))

Prerequisites

Go
- golang/go#63131
- golang/go#42372
- How does this interact with the Go GC?
- Who owns memory after guest → host call?
- Who owns memory after host → guest call?
TinyGo
- How does this interact with the TinyGo GC?

TODO

Where should this live? import _ github.com/ydnar/wasm-tools-go/abi?
Blank import or will this package also include lift/lower helpers?
Long term plan to move the abi package into Go stdlib, or vendor it for GOOS=wasip2 support?

Design

Go

Hypothetical Go design:

// Probably should be an unsafe.Pointer
type uintptr32 = uint32

//go:wasmexport cabi_realloc
func cabi_realloc(ptr, oldsize, align, newsize uintptr32) uintptr32

Prior Art

C

void *cabi_realloc(void *ptr, size_t old_size, size_t align, size_t new_size) {
    (void) old_size;
    if (new_size == 0) return (void*) align;
    void *ret = realloc(ptr, new_size);
    if (!ret) abort();
    return ret;
}

Rust

unsafe extern "C" fn cabi_realloc(
    old_ptr: *mut u8,
    old_len: usize,
    align: usize,
    new_len: usize,
) -> *mut u8 {
    // ...
}

Python

def realloc(self, original_ptr, original_size, alignment, new_size):
    if original_ptr != 0 and new_size < original_size:
        return align_to(original_ptr, alignment)
    ret = align_to(self.last_alloc, alignment)
    self.last_alloc = ret + new_size
    if self.last_alloc > len(self.memory):
        print('oom: have {} need {}'.format(len(self.memory), self.last_alloc))
        trap()
    self.memory[ret : ret + original_size] = self.memory[original_ptr : original_ptr + original_size]
    return ret

ydnar / wasm-tools-go Goto Github PK

wasm-tools-go's Introduction

wasm-tools-go

WebAssembly + WASI tools for Go

About

Component Model

Note on Memory Safety

wit-bindgen-go

WIT → Go

JSON → WIT

WIT → JSON

License

Contributing

wasm-tools-go's People

Contributors

Stargazers

Watchers

Forkers

wasm-tools-go's Issues

Implementation

Aliasing

Embedding

Background

Proposed Design

Exported Interfaces

Usage

Tasks

Tasks

Prerequisites

TODO

Design

Go

Prior Art

Recommend Projects

Recommend Topics

Recommend Org

`wit-bindgen-go`