This came up in the discussions around #97 and #98. Right now we are generally representing Schism objects as JavaScript objects. So far this has worked well. The code is fast, and it makes JavaScript interop easy.

But, there may be a reason to distinguish these. One way would be to have a SchismValue class that includes a field saying what type it is, and then whatever data is represented.

Another option is to have a zoo of types, like SchismPair, SchismString, SchismNumber, etc. At first glance, I'm less fond of this, but I haven't thought real hard about it.

Anyway, what are the pros and cons? Is anything broken with the status quo, or would wrapping everything in SchismValue just complicate things for no real benefit?

correct definition should be:

(define (cddar p) (cdr (cdar p)))

Since Schism-compiled modules all include the entire runtime, we should trim this down in cases where not the whole library is needed.

Note that #19 will help here some too, because programs will be able to only import the functions they need. That said, rnrs is a pretty big library, so DCE would still help there.

It would be nice if Schism were able to run one of the many miniKanren-related languages. As a way to get started, we should add a test case based on µKanren. It's less than 40 lines of code, and it looks like it only uses features Schism currently supports. Maybe we can get some simple programs running backwards.

Here's a link to the paper, which includes code:
http://webyrd.net/scheme-2013/papers/HemannMuKanren2013.pdf

It'd be nice to be able to write macros in Schism. While it'd be awesome to have full syntax-case support, to start with it's probably best to recognize just (define-syntax foo (syntax-rules ...). That'll give us most of the most useful cases.

Schism needs support for first class procedures.

Compiling program: '(list 5)'
rt.js:69 Uncaught (in promise) SchemeError
    at error (https://stereobooster.github.io/schism/rt/rt.js:104:12)
    at wasm-function[102]:395
    at wasm-function[103]:20
    at wasm-function[103]:27
    at wasm-function[103]:27
    at wasm-function[103]:27
    at wasm-function[103]:27
    at wasm-function[103]:27
    at wasm-function[103]:27
    at wasm-function[103]:27

Maybe source maps?

The list-all-eq? function was added as a limited form of equal? back before Schism had enough features to fully implement equal?. We should try to remove that function now.

The easiest way to do this right now would be to implement equal? and replace all the uses of list-all-eq? with that.

Scheme requires proper handling of tail calls, especially to do iteration. The current compiler already has had to structure things a little weirdly in some cases to work around the lack of tail calls.

There are a couple of possible approaches:

1. Wait for the WebAssembly tail call proposal to be accepted.
2. Do a CPS transform and run everything on a trampoline.

For the time being, the plan is to pursue option 2. This has the added benefit of making it easy to support call/cc, and can help enable garbage collection.

We have a decent number of tests now. It'd be a good idea to run them on multiple threads so the test suite finishes faster.

Are first class procedures supported?

Can someone add a lay-person explanation to the first paragraph to lighten the conceptual blow of the description? The reader needs to know what Schism does. The "self-hosting compiler from a subset of R6RS Scheme to WebAssembly" is a wide conceptual gap and is opaque. What does "self-hosting compiler" mean? What is R6RS? (I know, but most readers won't). WebAssembly is still a very new tech. It may help to explain Schism and WASM in terms of "software", "server", "browser", and "deployment". I will update this recommendation if I have any new intuitions.

Currently it takes an input scheme file and dumps to out.wasm. It would be nice if there were some standard cli options like -o, --out, --help, and --version.

Also, as a default I think it's typical to name binaries after the input file, so if example.ss is the entry module then example.wasm would be produced. Also I think some compiles do default to out so maybe this shouldn't change.

I just tried running tests with the latest version of Schism on node 12.18.3, and I get the following error when running test/list-find.ss:

./run-tests.sh test/list-find.ss 
(node:4529) ExperimentalWarning: The ESM module loader is experimental.
Running test test/list-find.ss
  stage0 FAILED
CompileError: WebAssembly.instantiate(): Compiling function #203:"find" failed: return_call found empty stack @+12143
  stage1 FAILED
CompileError: WebAssembly.instantiate(): Compiling function #203:"find" failed: return_call found empty stack @+12143
  stage2 FAILED
CompileError: WebAssembly.instantiate(): Compiling function #203:"find" failed: return_call found empty stack @+12143
Some tests failed:
    'test/list-find.ss': stage0, stage1, stage2

Is anyone else seeing this? I'm surprised CI didn't catch this.

For context, see #82 (comment)

To make it easier to accept new capabilities and configuration in the future, Schism.Engine should be refactored to take its filesystem object in a dict.

We can get an infinite loop (or more likely, a stack overflow) if we call equal? on cyclic data structures. Fortunately, these structures don't occur often, if at all, in the programs we've written in Schism so far.

This would be more efficient with a hash table, but we can make due for now with searching through a list.

Right now Schism never frees memory and crashes once it has used up all of its memory. It would be better to have a garbage collector.

Long term, the best option is to use WebAssembly's native garbage collection support, but since this doesn't exist yet we'll need something sooner.

In the shorter term, the plan is to implement a very simple semi-space copying collector in linear memory. This works nicely with the CPS+trampoline approach to #5, since effectively all we'd have to do is a deep copy of the continuation from the old space to the new space at every trampoline bounce.

I hit a bug. During write-bytes, the value #f gets passed to the else clause. The #f comes from encode-uleb. In the example that fails, this is what's passed:

(tail-call-indirect #f (get-local 0) (get-global 4) (get-global 8) (get-global 12) (call 30 (get-local 0)))

In the example where f appears in do-test:

(tail-call-indirect 15 (get-local 0) (get-global 4) (get-global 8) (get-global 12) (call 30 (get-local 0)))

is there instead.

The following library fails to compile:

(library (trivial)
  (export do-test)
  (import (rnrs))
  (define (f g)
    (g 1 2 3))
  (define (do-test) #t))

Both of the following pass:

(library (trivial)
  (export do-test)
  (import (rnrs))
  (define (f g)
    (g 1 2))
  (define (do-test) #t))

(library (trivial)
  (export do-test)
  (import (rnrs))
  (define (f g)
    (g 1 2 3))
  (define (do-test)
    (f (lambda (x y z) x)) 
    #t))

We have quite a few tests now and there is a lot of essentially fixed overhead per file we compile, meaning the test suite takes awhile to run. Parallelizing would help somewhat (#51), but we can also improve the tests and test runner to allow more tests in a single file.

They key thing would be to make sure each test in a file could succeed or fail independently.

I'm actually not sure if there are Scheme unit testing frameworks where it would make sense to adopt their API. Another option might be to make the tests write out their results as a TAP stream.

Right now the library system isn't really a library system. The compiler just pastes the code for a subset of rnrs into every library we compiler. It would be better to be able to load libraries from files, and do proper lexical scoping between the modules.

Eventually it would be great to have dynamic loading of modules, so there could be one rnrs.wasm that is loaded by all Schism libraries. For the time being, inlining everything into one big Wasm module would be a good start though.

For the number 128, it appears to generate a number that decodes to 2048, based on error messages from wabt when dumping the names section.

The test directory has .input files that are fed into the corresponding test. We should add support for .output files to make sure functions like display work correctly.

The most obvious thing would be to make sure the output matches the expected value verbatim. It might be nice to have more flexibility though, such as regular expressions for make sure two s-expressions are equal.

We could also consider doing this with comments in the source file instead of standalone files, kind of like LLVM lit tests.

Right now PR authors have the responsibility for generating their own snapshots. It would be better to have an automated process. Ideally, we'd have a bot that either automatically generates a snapshot from each PR, or have a branch that we can push to that triggers the snapshot creation process.

It would be nice to have a command that compiles Scheme file to .wasm from the repository.

Something like:

./compile.sh input.scm out.wasm

It'd be nice to have a (limited) way to write inline WebAssembly in Schism. I'd propose something like:

(define-wasm (foo (i32 a) (i32 b))
  (i32.add a b))

The reason for this is that once we have library support (#19), we might be able to move a lot of the intrinsics out of the compiler. For example, allocation, cons, car, cdr, etc., are all made up of low-level calls that basically read and write memory directly. With inline wasm, we could write these functions in assembly instead and maybe remove the intrinsics from the compiler (or just write the intrinsics themselves as a library).

I don't expect inline wasm to be widely used in Schism, but it would be helpful in implementing a lot of the low-level functionality.

It's been set at 16384 pages for some time now, which is about a gigabyte. This was largely due to our exaggerated memory usage from the allocation bug. Now that that is fixed, we should lower the default size down to 512MiB, or even 256MiB.

We probably want ports in general, but file input parts seem like a good starting point.

See https://scheme.com/tspl4/io.html#./io:s29 for reference.

The WebAssembly reference type proposal adds a new type of WebAssembly value, an opaque "reference type". I would like to use Schism to explore this facility, and as a benchmark for WebAssembly implementations.

This "anyref" type (optionally nullable) has no constructors available to WebAssembly, except to make a null reference. Reference values can enter wasm through function parameters, and leave wasm via return values.

The point of anyref is twofold. One, it may speed up access to DOM objects in JS embeddings, when compared to storing those objects in side tables on the JS side and treating them as integers inside wasm. Two, anyref is a step towards full GC. In that world, there are two heaps: linear memory (what we have now) and the graph of reference-typed values (the new thing). Anyref is a baby step in that direction, notably allowing host objects (JS objects) to flow through wasm. Full GC will also allow for collection of JS<->Scheme reference cycles.

Of course, the embedder can create anyref values, and so WebAssembly programs can create GC-managed values by calling out to imported host functions.

With anyref, the representation of a Scheme value would be an opaque anyref. As there's no wasm constructor for anyrefs, you'd call out to JS whenever you allocate, and whenever you access an object field, and whenever you do a type predicate. This would allow schism to use the host GC. Unfortunately it would mean no more immediates, though :(

With a future GC proposal, those JS call-outs become webassembly instead: constructors, field access, dynamic type predicates. You would also get back immediates via i31ref, but you'd still have to partition the space of i31 values into fixnums, chars, and constants. Here I think an implementation could be competitive with current Schism.

AFAIK, anyref is shipping in V8, JSC, and SM. It's also possible to emulate anyref via a side table; this is what Rust's wasm-bindgen does. I would like to prototype that too.

Anyway, this is a bit of a brain dump for something to explore. I would like for a schism-compiled "eval" of some silly function (fib or something) to be a WebAssembly benchmark for anyref / gc -- both to compare JS implementations, and to compare anyref / gc versus what Schism does now. I don't mean to suggest that work in this direction would land on master but I plan to keep the different versions in sync with master, so that they can be compared easily.

I really like the idea of this project but I currently can't get it to run in the browser.

Running the master playground doesn't work:
"Error: Scheme runtime error in char->integer: not a char"

Running ./bootstrap-from-guile.sh didn't work either.
"Error: Scheme runtime error in read-library: Could not find library"

#65 works.

It would be great if somebody could help me although I just wanted to try this project out and are not currently planning to actively use it.

This comes out of #32, where they might not be needed (but still will be nice). Since I need this feature for something I'm working on I'm opening this separate issue here.

http://www.r6rs.org/final/html/r6rs-lib/r6rs-lib-Z-H-3.html

Anyone is free to take this one that wants it, I might do it eventually when I have the time if no one else does.

When running ./bootstrap-from-guile.sh

schism/compiler.ss:193:53: In procedure module-lookup: Unbound variable: %file-exists?

Looks like this bug was introduced in commit c6eda3d, which would make sense given the commit message.

Compilation takes quite a while even for small files. Is this something that can be fixed in schism or an upstream v8 issue? Feel free to close if so.

Supporting call/cc in Schism would be pretty interesting to see.

Originally I had envisioned doing this through a full CPS transform early in the compiler. Now that we have anyref and tail calls in Wasm, some of the other benefits of the CPS transform are less relevant. These benefits were:

1. The continuation gives us most of the roots we need for GC basically for free.
2. CPS means we can do tail calls by making all the functions return thunks and repeatedly calling these in a loop.

Both of these would complicate the interface for calling into Schism code from JS, but we could avoid that by generating wrappers that are exported instead.

Another downside of the CPS transform is that it imposes a cost everywhere for a feature that's pretty rarely used.

So what's the best way to implement this in Schism?

Maybe the CPS transform is still the right way to do it, and we can rely heavily on the simplifier to remove unnecessary continuations.

Another option is to wait and see how the exception handling proposal for Wasm shakes out. Some versions of it could be used to implement continuations, although these were still single use continuations. Maybe it's not too bad to try and convert a single use continuation into a multiple use one?

Pretty much all of our tests test that things that should work. We also need tests to make sure things that shouldn't work don't work.

For example, as library support lands, we'll want to make sure names that aren't imported aren't accessible.

I spent some time trying to figure out why and couldn't determine why, but I get the following error:

cwebber@twig:~/devel/schism$ ./bootstrap-from-guile.sh 
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/cwebber/devel/schism/./bootstrap-from-guile.scm
;;; compiling ./schism/compiler.ss
;;; compiling ./lib/schism.ss
;;; compiled /home/cwebber/.cache/guile/ccache/3.0-LE-8-4.4/home/cwebber/devel/schism/lib/schism.ss.go
;;; compiled /home/cwebber/.cache/guile/ccache/3.0-LE-8-4.4/home/cwebber/devel/schism/schism/compiler.ss.go
;;; compiled /home/cwebber/.cache/guile/ccache/3.0-LE-8-4.4/home/cwebber/devel/schism/bootstrap-from-guile.scm.go
Backtrace:
In ice-9/boot-9.scm:
  1736:10 18 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
In unknown file:
          17 (apply-smob/0 #<thunk 7f8ab51d2d60>)
In ice-9/boot-9.scm:
    718:2 16 (call-with-prompt _ _ #<procedure default-prompt-handler (k proc)>)
In ice-9/eval.scm:
    619:8 15 (_ #(#(#<directory (guile-user) 7f8ab51ccc80>)))
In ice-9/boot-9.scm:
   2806:4 14 (save-module-excursion _)
  4351:12 13 (_)
In ice-9/ports.scm:
   463:17 12 (call-with-output-file _ _ #:binary _ #:encoding _)
    474:4 11 (_ _)
   445:17 10 (call-with-input-file _ _ #:binary _ #:encoding _ #:guess-encoding _)
    470:4  9 (_ _)
In schism/compiler.ss:
   1682:2  8 (compile-stdin->stdout)
  1639:31  7 (compile-library _)
In rnrs/base.scm:
    275:1  6 (map #<procedure 7f8ab0e117a8 at schism/compiler.ss:1337:62 (code)> ((0 …) …))
In schism/compiler.ss:
   1576:2  5 (encode-code _ _)
   1493:2  4 (encode-expr (seq (seq (drop (call 460 (get-local 0))) (drop (call #))) (…)))
   1493:2  3 (encode-expr (seq (drop (call 460 (get-local 0))) (drop (call 461))))
   1490:2  2 (encode-simple-op 26 _)
In rnrs/base.scm:
    275:1  1 (map #<procedure encode-simple-op (op expr)> ((call 460 (get-local 0))))
In schism/compiler.ss:
   1490:2  0 (encode-simple-op _ _)

schism/compiler.ss:1490:2: In procedure encode-simple-op:
Wrong number of arguments to #<procedure encode-simple-op (op expr)>

However, this is confusing... encode-simple-op seems to take two arguments, and every call site I see calls it with two arguments...?

Really unsure why and how this would be breaking then... (@wingo, is it working for you)?

Strings right now are represented as lists of characters with a different tag. This is inefficient. We should use a better representation, such as a vector of characters. This should help some with #32.

Strings should have the following equality behavior:

> (eq? (list->string '(#\a #\b)) (list->string '(#\a #\b)))
#f
> (string=? (list->string '(#\a #\b)) (list->string '(#\a #\b)))
#t

Because strings are represented as JavaScript objects, we might be accidentally getting value equality instead of reference equality for eq? on strings. We should add a test case to make sure we have the right behavior, and fix it if we don't.

I might just be misunderstanding how this is supported to work. If I have a function like:

(library
  (trivial)
 (export add addi)
 (import (rnrs))

 (define (addi)
  (+ 10 20))

 (define (add a b)
  (+ a b)))

addi will return a pointer that I can get the value from with engine.jsFromScheme(ptr). However add returns the JavaScript value when called like instance.exports.add(1,2).

I admit to not understanding how arguments work yet but this seems like a bug. I would expect the API to always return pointers.

We should support static data. Right now, expressions such as '(1 2 3) get compiled into (cons 1 (cons 2 (cons 3 '()))). Symbols, such as 'foo are even worse. They get compiled into (string->symbol (list->string '(#\f #\o #\o))). We should instead generate a Wasm data segment with all values like this. This will do a lot to help with #32.

With factoring rnrs into a separate file (#90), we've lost the nice property that Schism can run from just schism.wasm.

One easy way to get this functionality back would be to add an include-data macro, that would work something like this:

(include-data "filename")

At compile time, we'd read a datum from the file with the given name and include it as a quoted literal. If "filename" contained (1 2 3), then (include-data "filename") would expand to '(1 2 3).

Hi. Thanks for starting this wonderful project. Had similar idea but thought about compiling Rust to wasm. This is even better.

I tried to create online playground - I forked repo, created gh-pages branch, renamed playground.html to index.html, but getting this error when visiting page https://stereobooster.github.io/schism/:

stereobooster.github.io/:1 Uncaught SyntaxError: The requested module does not provide an export named 'rt'