jupyterlite / xeus-lua-kernel Goto Github PK
View Code? Open in Web Editor NEWA Lua kernel for JupyterLite, powered by Xeus
Home Page: https://jupyterlite.github.io/demo
License: BSD 3-Clause "New" or "Revised" License
A Lua kernel for JupyterLite, powered by Xeus
Home Page: https://jupyterlite.github.io/demo
License: BSD 3-Clause "New" or "Revised" License
Currently there is a build step in package.json
involving a call to sed
to replace a substring:
Line 45 in e3a1937
We should look into making this step more robust.
@jtpio here a few infos on sharedarraybuffers.
https://developers.google.com/search/blog/2021/03/sharedarraybuffer-notes
https://web.dev/cross-origin-isolation-guide/
Not sure what the cross-origin isolation thingy actually means and if we could use it to keep the SharedArrayBuffer
@jtpio
here is my attempt to compile xeus-lua as seperate extension.
First I had to add a https://github.com/DerThorsten/jupyterlite_xeus_lua/blob/main/src/xeus_lua.d.ts
which I had not in the latest version of xeus-lua directly integrated in jupyterlite (at some point I also had such a file in the direct integrated xeus lua)
but jlpm run build
fails horrible at the build:labextension:dev
stage
( jlpm run build:lib
without any error)
Any hint what could be the problem?
tsc
$ jupyter labextension build --development True .
Building extension in .
Compilation starting…
<--- Last few GCs --->
[48218:0x55a0f730ff30] 364047 ms: Scavenge 8000.4 (8222.0) -> 7996.8 (8223.0) MB, 12.5 / 0.0 ms (average mu = 0.169, current mu = 0.038) task
[48218:0x55a0f730ff30] 364133 ms: Scavenge 8002.9 (8223.0) -> 7996.0 (8231.2) MB, 16.2 / 0.0 ms (average mu = 0.169, current mu = 0.038) allocation failure
[48218:0x55a0f730ff30] 366750 ms: Mark-sweep 8011.5 (8231.2) -> 7988.5 (8231.7) MB, 2509.7 / 0.1 ms (average mu = 0.198, current mu = 0.226) allocation failure scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0x7f3b40cc75c9 node::Abort() [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
2: 0x7f3b40ba94a3 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > node::SPrintFImpl<int&, char const*>(char const*, int&, char const*&&) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
3: 0x7f3b4107a832 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
4: 0x7f3b4107ab0b v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
5: 0x7f3b412411d6 [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
6: 0x7f3b412412c1 [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
7: 0x7f3b41255c24 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
8: 0x7f3b41256524 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
9: 0x7f3b412bc5cd v8::internal::ScavengeJob::Task::RunInternal() [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
10: 0x7f3b4115abdc [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
11: 0x7f3b40d44794 node::PerIsolatePlatformData::RunForegroundTask(std::unique_ptr<v8::Task, std::default_delete<v8::Task> >) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
12: 0x7f3b40d47f73 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
13: 0x7f3b40230901 [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/./libuv.so.1]
14: 0x7f3b402427fc [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/./libuv.so.1]
15: 0x7f3b4023102c uv_run [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/./libuv.so.1]
16: 0x7f3b40bfafd7 node::SpinEventLoop(node::Environment*) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
17: 0x7f3b40d148d5 node::NodeMainInstance::Run(int*, node::Environment*) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
18: 0x7f3b40d14d88 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
19: 0x7f3b40c8a9fc node::Start(int, char**) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
20: 0x55a0f648f0fb main [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/node]
21: 0x7f3b40293565 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
22: 0x55a0f648f146 [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/node]
An error occurred.
subprocess.CalledProcessError: Command '['node', '/home/derthorsten/src/jupyterlite_xeus_lua/node_modules/@jupyterlab/builder/lib/build-labextension.js', '--core-path', '/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/lib/python3.9/site-packages/jupyterlab/staging', '/home/derthorsten/src/jupyterlite_xeus_lua', '--development']' died with <Signals.SIGABRT: 6>.
See the log file for details: /tmp/jupyterlab-debug-gu5vc6u_.log
The current wasm file is about 40MB which is quite huge.
Wondering whether there would be a way to reduce the size. Maybe that would involve passing a couple of flags to the emscripten build command.
Thinking about the next steps once we get the first version working as a federated extension:
xeus-lua
can depend on. This repo would contain most of the logicjupyterlite-xeus-lua
would live in its own repo, for example https://github.com/jupyterlite/xeus-lua-kernel, and would depend on the @jupyterlite/xeus-kernel
packages. It would also have the tools (Dockerfile
) to create the wasm file, and would specify it as a parameter@ts-ignore
in the codethis is a hint on how to minimize the emscripten generated js files
emscripten-core/emscripten#8773
@jtpio any hint on how to add the lua logo for the kernel properly
now that we move the xeus lua related files to the asset dir via https://github.com/DerThorsten/jupyterlite_xeus_lua/blob/3a22e56672ef5aca0655fc64aaa39fac74e5b6a2/webpack.config.js#L4
we are facing the problem that xeus_lua.js it not found
ModuleNotFoundError: Module not found: Error: Can't resolve './xeus_lua' in '/home/derthorsten/src/jupyterlite_xeus_lua/lib'
at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/Compilation.js:1930:28
at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:793:13
at eval (eval at create (/home/derthorsten/src/jupyterlite_xeus_lua/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:10:1)
at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:275:22
at eval (eval at create (/home/derthorsten/src/jupyterlite_xeus_lua/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:9:1)
at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:431:22
at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:124:11
at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:665:25
at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:850:8
at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:970:5
resolve './xeus_lua' in '/home/derthorsten/src/jupyterlite_xeus_lua/lib'
...
...
...
@jtpio you mentioned:
Locally excluding the emscripten generated files seems to help build the extension.
However the files are not properly emitted to labextension/static yet.
@jtpio any hint on how to continue on that?
Do we need to change the import statement?
https://github.com/DerThorsten/jupyterlite_xeus_lua/blob/3a22e56672ef5aca0655fc64aaa39fac74e5b6a2/src/worker.ts#L3
this is all a bit beyond my typescript / webpack competence ...
It would be nice to add https://github.com/jupyter-xeus/xeus-lua/blob/main/notebooks/widgets.ipynb and https://github.com/jupyter-xeus/xeus-lua/blob/main/notebooks/game_of_life.ipynb to the build st we can host a demo which contains these notebooks.
I am a bit overwhelmed with https://jupyterlite.readthedocs.io/en/latest/configuring.html#adding-content
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.