jprendes / emception Goto Github PK
View Code? Open in Web Editor NEWRun Emscripten in the browser
License: Other
Run Emscripten in the browser
License: Other
I tried to get the demo to compile from the instructions in the README, but it seems to be failing regardless of what I do.
Following the instructions, running:
$ git clone https://github.com/jprendes/emception.git
$ cd emception
$ ./build-with-docker.sh
$ pushd demo
$ npm install
all executes fine, but running the build script right after doesn't:
$ npm run build
> build
> webpack --mode=production --stats-children
assets by status 10.5 MiB [cached] 13 assets
orphan modules 3.05 MiB [orphan] 362 modules
runtime modules 7.38 KiB 12 modules
built modules 6.52 MiB (javascript) 66.3 KiB (asset) [built]
modules by path ./node_modules/monaco-editor/esm/vs/ 6.12 MiB (javascript) 66.3 KiB (asset) 276 modules
modules by path ./node_modules/style-loader/dist/runtime/*.js 5.75 KiB 6 modules
modules by path data:image/ 3.99 KiB 5 modules
modules by path ./node_modules/css-loader/dist/runtime/*.js 2.91 KiB 3 modules
modules by path ./node_modules/xterm/ 332 KiB
./node_modules/xterm/lib/xterm.js 327 KiB [built] [code generated]
./node_modules/css-loader/dist/cjs.js!./node_modules/xterm/css/xterm.css 4.56 KiB [built] [code generated]
./index.js + 11 modules 62.6 KiB [not cacheable] [built] [code generated]
./node_modules/xterm-addon-fit/lib/xterm-addon-fit.js 1.68 KiB [built] [code generated]
./node_modules/css-loader/dist/cjs.js!./style.css 4.09 KiB [built] [code generated]
Entrypoint HtmlWebpackPlugin_0-0 =
runtime modules 440 bytes 3 modules
cacheable modules 521 bytes
data:text/javascript,__webpack_public_path__ = __webpack_base_uri__ = htmlWebpackPluginPublicPath; 77 bytes [built] [code generated]
./node_modules/html-webpack-plugin/lib/loader.js!./node_modules/html-webpack-plugin/default_index.ejs 444 bytes [built] [code generated]
Child HtmlWebpackCompiler compiled successfully
assets by status 115 KiB [cached] 1 asset
orphan modules 391 KiB [orphan] 34 modules
runtime modules 221 bytes 1 module
./node_modules/monaco-editor/esm/vs/editor/editor.worker.js + 34 modules 392 KiB [built] [code generated]
Child vs/editor/editor compiled successfully
assets by status 6.49 MiB [cached] 6 assets
orphan modules 31.3 KiB [orphan] 15 modules
runtime modules 1.67 KiB 5 modules
cacheable modules 388 KiB (javascript) 6.21 MiB (asset)
modules by path ../build/emception/ 357 KiB (javascript) 6.21 MiB (asset)
asset modules 210 bytes (javascript) 6.21 MiB (asset)
modules by path ../build/emception/packages/*.br 126 bytes (javascript) 5.3 MiB (asset) 3 modules
../build/emception/wasm-package/wasm-package.wasm 42 bytes (javascript) 143 KiB (asset) [built] [code generated]
../build/emception/brotli/brotli.wasm 42 bytes (javascript) 786 KiB (asset) [built] [code generated]
javascript modules 357 KiB
../build/emception/wasm-package/wasm-package.mjs 88.7 KiB [built] [code generated]
../build/emception/binaryen/binaryen-box.mjs 99 KiB [built] [code generated]
../build/emception/quicknode/quicknode.mjs 95.4 KiB [built] [code generated]
../build/emception/brotli/brotli.mjs 73.6 KiB [built] [code generated]
./emception.worker.js + 15 modules 31.5 KiB [built] [code generated]
(ignored asset) 26 bytes [built] [code generated]
ERROR in ../build/emception/LlvmBoxProcess.mjs 2:0-48
Module not found: Error: Can't resolve './llvm/llvm-box.mjs' in '/Users/s1g/Workspaces/emception-master/build/emception'
resolve './llvm/llvm-box.mjs' in '/Users/s1g/Workspaces/emception-master/build/emception'
using description file: /Users/s1g/Workspaces/emception-master/package.json (relative path: ./build/emception)
Field 'browser' doesn't contain a valid alias configuration
using description file: /Users/s1g/Workspaces/emception-master/package.json (relative path: ./build/emception/llvm/llvm-box.mjs)
Field 'browser' doesn't contain a valid alias configuration
/Users/s1g/Workspaces/emception-master/build/emception/llvm/llvm-box.mjs doesn't exist
@ ./emception.js 3:0-58 87:28-42
@ ./emception.worker.js 2:0-39 4:22-31
ERROR in ../build/emception/Python3Process.mjs 2:0-48
Module not found: Error: Can't resolve './cpython/python.mjs' in '/Users/s1g/Workspaces/emception-master/build/emception'
resolve './cpython/python.mjs' in '/Users/s1g/Workspaces/emception-master/build/emception'
using description file: /Users/s1g/Workspaces/emception-master/package.json (relative path: ./build/emception)
Field 'browser' doesn't contain a valid alias configuration
using description file: /Users/s1g/Workspaces/emception-master/package.json (relative path: ./build/emception/cpython/python.mjs)
Field 'browser' doesn't contain a valid alias configuration
/Users/s1g/Workspaces/emception-master/build/emception/cpython/python.mjs doesn't exist
@ ./emception.js 5:0-58 91:20-34 92:20-34 93:20-34
@ ./emception.worker.js 2:0-39 4:22-31
Child worker-loader /Users/s1g/Workspaces/emception-master/demo/emception.worker.js compiled with 2 errors
webpack 5.61.0 compiled with 2 errors in 14260 ms
Running ls ../build/emception
gives:
$ ls ../build/emception/
AsyncInitializable.mjs BrotliProcess.mjs LlvmBoxProcess.mjs QuickNodeProcess.mjs brotli index.html packs.mjs
BinaryenBoxProcess.mjs EmProcess.mjs Process.mjs Thenable.mjs cpython llvm quicknode
BoxProcess.mjs FileSystem.mjs Python3Process.mjs binaryen
so it looks like the primary modules are generated correctly, but their immediate successors aren't:
$ ls -l ../build/emception/llvm
total 0
$ ls -l ../build/emception/cpython
total 0
In fact it looks like they're not being generated at all:
$ find .. -name 'python.mjs'
<nothing>
For the record I also tried checking out 783fa10
(the same commit where the instructions were added), and it still fails (despite being run on a completely fresh clone):
$ npm run build
> build
> webpack --mode=production --stats-children
assets by status 10.2 MiB [cached] 11 assets
orphan modules 3.05 MiB [orphan] 362 modules
runtime modules 7.38 KiB 12 modules
built modules 6.52 MiB (javascript) 66.3 KiB (asset) [built]
modules by path ./node_modules/monaco-editor/esm/vs/ 6.12 MiB (javascript) 66.3 KiB (asset) 276 modules
modules by path ./node_modules/style-loader/dist/runtime/*.js 5.75 KiB 6 modules
modules by path data:image/ 3.99 KiB 5 modules
modules by path ./node_modules/css-loader/dist/runtime/*.js 2.91 KiB 3 modules
modules by path ./node_modules/xterm/ 332 KiB
./node_modules/xterm/lib/xterm.js 327 KiB [built] [code generated]
./node_modules/css-loader/dist/cjs.js!./node_modules/xterm/css/xterm.css 4.56 KiB [built] [code generated]
./index.js + 11 modules 61.9 KiB [not cacheable] [built] [code generated]
./node_modules/xterm-addon-fit/lib/xterm-addon-fit.js 1.68 KiB [built] [code generated]
./node_modules/css-loader/dist/cjs.js!./style.css 4.09 KiB [built] [code generated]
Entrypoint HtmlWebpackPlugin_0-0 =
runtime modules 440 bytes 3 modules
cacheable modules 514 bytes
data:text/javascript,__webpack_public_path__ = __webpack_base_uri__ = htmlWebpackPluginPublicPath; 77 bytes [built] [code generated]
./node_modules/html-webpack-plugin/lib/loader.js!./node_modules/html-webpack-plugin/default_index.ejs 437 bytes [built] [code generated]
Child HtmlWebpackCompiler compiled successfully
assets by status 115 KiB [cached] 1 asset
orphan modules 391 KiB [orphan] 34 modules
runtime modules 221 bytes 1 module
./node_modules/monaco-editor/esm/vs/editor/editor.worker.js + 34 modules 392 KiB [built] [code generated]
Child vs/editor/editor compiled successfully
assets by status 6.22 MiB [cached] 4 assets
orphan modules 325 KiB [orphan] 16 modules
runtime modules 1.64 KiB 5 modules
cacheable modules 329 KiB (javascript) 6.02 MiB (asset)
modules by path ../build/emception/ 126 bytes (javascript) 6.02 MiB (asset)
../build/emception/root.pack.br 42 bytes (javascript) 5.12 MiB (asset) [built] [code generated]
../build/emception/wasm-package/wasm-package.wasm 42 bytes (javascript) 144 KiB (asset) [built] [code generated]
../build/emception/brotli/brotli.wasm 42 bytes (javascript) 785 KiB (asset) [built] [code generated]
javascript modules 329 KiB
./emception.worker.js + 16 modules 329 KiB [built] [code generated]
(ignored asset) 26 bytes [built] [code generated]
ERROR in ../build/emception/LlvmBoxProcess.mjs 2:0-48
Module not found: Error: Can't resolve './llvm/llvm-box.mjs' in '/Users/s1g/Workspaces/emception/build/emception'
@ ./emception.worker.js 5:0-67 35:28-42
ERROR in ../build/emception/PythonProcess.mjs 87:33-44
export 'loadPyodide' (imported as 'loadPyodide') was not found in './pyodide/pyodide.mjs' (module has no exports)
@ ./emception.worker.js 7:0-65 38:26-39 39:31-44
ERROR in ../build/emception/PythonProcess.mjs 87:45-64
export 'default' (imported as 'createPyodideModule') was not found in './pyodide/pyodide.asm.mjs' (module has no exports)
@ ./emception.worker.js 7:0-65 38:26-39 39:31-44
1 error has detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.
Child worker-loader /Users/s1g/Workspaces/emception/demo/emception.worker.js compiled with 3 errors
webpack 5.61.0 compiled with 3 errors in 14623 ms
Am I missing a step in compilation or am I missing a dependency of sorts?
Thanks for a really cool demo.
I was wondering if there was an easy place to hook into the build to add some libraries to the final result, e.g. if I wanted a demo like yours but you could include Eigen
or that sort of thing. Could you point me toward where the best place to do this is?
Out of curiosity, how does the sharedfs work? Is it a temporary virtualized filesystem? Or is that saved to an indexedDB? Or is it the user's local filesystem?
Hi, thanks for the very cool work you do!
I'm having some problems with the llvm-box. I noticed that the llvm-box did not have any console output at runtime, and the exit code was always 1
. So I modified the llvm-box wrapper like this :
int main(int argc, const char ** argv) {
printf("hello,world"); // add this
if (argc < 1) return 2; // change this
const char * argv0 = argv[0];
--argc;
++argv;
#include "llvm-box-entry-gen.hpp"
fprintf(stderr, "LLVM command \"%s\" not found", argv0);
return 1;
}
It can be sure that argv is set, but the exit code is still 1 and printf("hello,word")
doesn't output anything. Testing with my own wasm hello world program outputs it correctly.
I'm not an expert on wasm, am I doing something wrong? Thank you.
Hello! ๐
First, some context: I want to have a C++ => WASM demo that runs in the browser. All the options (list below) seem to be in various states of decay.
My benchmark of "working" is that tool $X can compile this and maybe run it (depending on the tool).
#include <string>
#include <iostream>
int main() {
std::string message = "Hello world!";
std::cout << message << "\n";
}
Here's my non-exhaustive list of C++ => WASM things that I found and tried
std::cout
+ std::string
test#include <string>
causes it to chokeI am interested in learning more about how this project works in order to (hopefully) get some kind of higher-level API working. #7
This is the big question that I have. Here's what I've gathered so far. I'd love more input and commentary of how this all works, the pain points, all of it. Tell me everything you tell your rubber ducky. ๐ฆ
Working backwards from the demo:
I also did some digging and found this quote on reddit:
Emscripten runs (mainly) on Python and Node, and internally uses clang/llvm
and binaryen.For Python, Emception uses a patched version of Pyodide, which is based on
Cpython compiled with Emscripten.For Node, Emception uses very hacky JS code in the browser.
For clang/llvm and binaryen, they are compiled to WebAssembly with Emscripten,
with some hacks to reduce binary size.
โ u/jprendes from I made Emception
Hi, this is a great project and it would theoretically revive my current obsolete project that have been broken on recent upstream of emcc/clang. Running old emscripten still worked but I can't afford to build everything from source just to run my project, making a static version of emception for my project would be a great solution for this problem. But even though this project will just run as is on node with the QuickJS compiled down, would it be faster to directly passtrough node command to the environment and if possible, to turn off Asyncify/Emterpreter so the whole compiling process will be much faster (but the latter is I guess, more complex than swapping out QuickJS, since I red somewhere in this repo that before you are actually have make a simple passtrough to the browser's JS, it would be beneficial to re-use that capability on node right?)
In the source, we have await new FileSystem();
.
How does one add this behavior with another class that needs FileSystem (so that the await can bubble)? Having the await call inside the class constructor / body doesn't work.
System (VM):
Ubuntu 22.04 LTS
27.4 GiB of Memory
i7-10700F Processor
Steps to reproduce:
sudo ./build-with-docker.sh
cp -r emception/build/emception .
Cool library and a great idea.
There's quite a lot of interest around WASI and implementing hosts in various ecosystems which can offer up a posix-like filesystem to wasm binaries.
Have you considered how easily this would port to be built using wasi-sdk?
I was wondering if you would be able to add libSDL2.a to /emscripten/cache/sysroot/lib/wasm32-emscripten
in the files published to the demo branch?
I tried manually adding the file from a local install of emscripten 3.1.24, but when issuing the em++ command from inside emception i can see it tries to download SDL2. Not sure how to trick the script i have the file or otherwise workaround this.
I am trying to build and run the demo locally on a fresh clone of commit 3660655 (the latest commit as of writing). All steps appear to succeed during the build.
When I launch the demo in the browser, emception fails to load. I see a 404 error trying to get /python.data
. I found this file in build/cypthon/python.data
. If I copy this into build/demo
the demo makes it a bit further, but then there is a FS error (number 20) thrown by python.mjs, so I suspect this is not the correct python.data file.
The demo branch does load emception properly, so there seems to be a discrepancy between what the build scripts produce and what is pushed to that branch. Any tips or pointers would be greatly appreciated!
This is a really cool project, though it's difficult to use a part of a project due to the lengthy build and compilation. Could we have Emception as a library?
For example (on a website):
<script src="emception.min.js"></script>
<script>
let emception = new Emception();
emception.run('echo "check out this!"').then(stdout => {
//do stuff
}).catch(stderr => {
//do stuff
});
</script>
A few things to note:
Why do we need a library?
Hi,
I found your very cool Emscripten in a browser demo. Providing a release that includes the prebuilt, unbundled .wasm binaries and their corresponding .js scripts would make it convenient for other people to use them in their own projects. Specifically, clang.wasm and lld.wasm would be useful.
Thanks.
Condition:
I've tried to build latest llvm, not fe2b2cb58ebb57427c0a12e54a4ed63553c397ab
.
So I rolled back changes by git reset
and git apply
.
And then I run build-llvm.sh
again, it failed.
Error message:
FAILED: bin/clang-nvlink-wrapper.mjs
error: undefined symbol: wait4 (referenced by top-level compiled C/C++ code)
warning: Link with `-s LLD_REPORT_UNDEFINED` to get more information on undefined symbols
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
warning: _wait4 may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
Error: Aborting compilation due to previous errors
em++: error: '/home/minsoo/emsdk/node/14.15.5_64bit/bin/node /home/minsoo/emsdk/upstream/emscripten/src/compiler.js /tmp/tmpw4eki2gi.txt' failed (returned 1)
Additional note:
When I cloned this and ran build-with-docker.sh
, it successfully built other tools but llvm-box
What am I missing? Do I have to remove all the build directories and run this again?
After discovering Emception project, I quickly opened up the provided demo on this link, but what I got was a loading icon that won't disappear no matter how long I wait for.
I am running Chrome, Version 101.0.4951.64 (Official Build) (x86_64), on MacOS, if that's any useful. Is there a known problem? Or how could I debug it to help you at least figure out what goes wrong?
I will soon be trying to build it myself to test if it actually works, but now I just don't have time for that.
Hi!
I am trying to use the build-with-docker approach, and it is failing because of broken patches. The offending one is cpython:
Cloning into '/Users/tolstenko/projects/emception/upstream/cpython'...
Updating files: 100% (4653/4653), done.
/Users/tolstenko/projects/emception/upstream/cpython /Users/tolstenko/projects/emception
From https://github.com/python/cpython
* branch b8a9f13abb61bd91a368e2d3f339de736863050f -> FETCH_HEAD
Updating files: 100% (3155/3155), done.
HEAD is now at b8a9f13abb gh-97616: test_list_resize_overflow() uses sys.maxsize (#99057)
/Users/tolstenko/projects/emception
/Users/tolstenko/projects/emception/upstream/cpython /Users/tolstenko/projects/emception
error: patch failed: configure.ac:1253
error: configure.ac: patch does not apply
error: Modules/_emception.c: No such file or directory
error: patch failed: Modules/Setup.stdlib.in:118
error: Modules/Setup.stdlib.in: patch does not apply
And probably these too:
The necessary bits to build these optional modules were not found:
_bz2 _ctypes _ctypes_test
_curses _curses_panel _dbm
_gdbm _lzma _tkinter
_uuid readline
Which causes
configure: WARNING: unrecognized options: --enable-wasm-js-module
configure: loading site script /Users/tolstenko/projects/emception/upstream/cpython/Tools/wasm/config.site-wasm32-emscripten
configure: creating cache config.cache
checking for git... found
checking build system type... x86_64-pc-linux-gnu
checking host system type... wasm32-unknown-emscripten
checking for --with-build-python... configure: error: invalid or missing build python binary "/Users/tolstenko/projects/emception/build/cpython-native/python"
emconfigure: error: '/Users/tolstenko/projects/emception/upstream/cpython/configure -C --host=wasm32-unknown-emscripten --build=x86_64-pc-linux-gnu --with-emscripten-target=browser --disable-wasm-dynamic-linking --with-suffix=.mjs --disable-wasm-preload --enable-wasm-js-module --with-build-python=/Users/tolstenko/projects/emception/build/cpython-native/python' failed (returned 1)
make: make -j10
make: *** No targets specified and no makefile found. Stop.
emmake: error: 'make -j10' failed (returned 2)
/Users/tolstenko/projects/emception
/Users/tolstenko/projects/emception/build/cpython /Users/tolstenko/projects/emception
make: make -j10
make: *** No targets specified and no makefile found. Stop.
emmake: error: 'make -j10' failed (returned 2)
Some changes I made to improve the build:
# fix wrong strange bug that generates 'ninja_required_version1.5' instead of 'ninja_required_version = 1.5'
sed -i 's/ninja_required_version1\.5/ninja_required_version = 1.5/g' $BINARYEN_BUILD/build.ninja
git describe
command was failing, so I removed the --depth 1
in most of the git clones commands, I don't remember anymore which ones.But it is still not building.
Suggestion:
I would love to help somehow, but at this point, I got lost in everything you did.
Btw, awesome project. Congratulations on gluing everything together.
Keep up the good work.
Add coi lib to make -pthreads
work in the GitHub live demo page.
I tried to use your project, but I received some errors.
First of all, I tried your demo and it worked well.
Then I downloaded the repository and I have done the build correctly. After, when I tried to run the demo locally it gave me an error (attached photos: Emception doesn't load correctly and so the UI shows the "Loading" text).
I am asking you if there is any other way to run it locally without getting this error or what can I do to prevent this error from happening.
Hi @jprendes just wondering if we have permission to use this code in other projects? Interested in using the llvm-box, to compile code for embedded devices and this seems like a great start :)
First of all, congrats for this awesome project!
It feels (I've not measured it) too slow compared with native emscripten. Have you considered pushing it as a (real world) wasm platform (and emscripten) benchmark? I believe that a detailed performance analysis will show very interesting data for wasm compilers and wasm-based platforms (file system simulation, etc.).
Sorry for the late response, I'm working on getting a linux environment running entirely in the browser - BrowserLinux - and was intrigued by emception. I am working on adding a "complete filesystem" in IndexedDB with pre-written methods of reading and writing pseudo-"files" that would be accessible to the programs compiled to wasm. If I am able to combine that "filesystem" with SHAREDFS, that would be highly beneficial.
Originally posted by @Froggo8311 in #3 (comment)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.