Git Product home page Git Product logo

xeus-lua-kernel's Introduction

JupyterLite

ci-badge lite-badge binder-badge docs-badge

JupyterLite is a JupyterLab distribution that runs entirely in the browser built from the ground-up using JupyterLab components and extensions.

⚡ Status ⚡

Although JupyterLite is currently being developed by core Jupyter developers, the project is still unofficial.

Not all the usual features available in JupyterLab and the Classic Notebook will work with JupyterLite, but many already do!

Don't hesitate to check out the documentation for more information and project updates.

✨ Try it in your browser ✨

JupyterLite works with both JupyterLab and Jupyter Notebook.

Try it with JupyterLab! Try it with Jupyter Notebook!
lab-screenshot notebook-screenshot

🏗️ Build your own JupyterLite 🏗️

You can build your own JupyterLite website in a couple of minutes, with custom extensions and packages.

See the documentation for more details.

Browser-based Interactive Computing

JupyterLite is all about accessible browser-based interactive computing:

  • Python kernel backed by Pyodide running in a Web Worker
    • Initial support for interactive visualization libraries such as altair, bqplot, ipywidgets, matplotlib, and plotly
  • JavaScript kernel running in a Web Worker
  • View hosted example Notebooks and other files, then edit, save, and download from the browser's IndexDB (or localStorage)
  • Support for saving settings for JupyterLab/Lite core and federated extensions
  • Basic session and kernel management to have multiple kernels running at the same time
  • Support for Code Consoles

Ease of Deployment

  • Served via well-cacheable, static HTTP(S), locally or on most static web hosts
  • Embeddable within larger applications
  • Requires no dedicated application server much less a container orchestrator
  • Fine-grained configurability of page settings, including reuse of federated extensions

Showcase

Jupyter Interactive Widgets

widgets

JupyterLab Mimerender Extensions

image

Matplotlib Figures

image

Altair

altair

Plotly

plotly

Development install

See the contributing guide for a development installation.

Related

JupyterLite is a reboot of several attempts at making a full static Jupyter distribution that runs in the browser, without having to start the Python Jupyter Server on the host machine.

The goal is to provide a lightweight computing environment accessible in a matter of seconds with a single click, in a web browser and without having to install anything.

This project is a collection of packages that can be remixed together in variety of ways to create new applications and distributions. Most of the packages in this repo focus on providing server-like components that run in the browser (to manage kernels, files and settings), so existing JupyterLab extensions and plugins can be reused out of the box.

See also:

  • p5-notebook: A minimal Jupyter Notebook UI for p5.js kernels running in the browser
  • jyve: Jupyter Kernels, right inside JupyterLab
  • Starboard Notebook: In-browser literal notebooks
  • Basthon: A Jupyter notebook implementation using Pyodide

👥 Contributors

Join our community and become a contributor today! 🚀

xeus-lua-kernel's People

Contributors

derthorsten avatar github-actions[bot] avatar johanmabille avatar jtpio avatar martinrenou avatar sylvaincorlay avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar

Forkers

jtpio martinrenou sylvaincorlay stjordanis derthorsten isabella232 johanmabille chriss-0x01

xeus-lua-kernel's Issues

Next steps to package and distribute the kernel

Thinking about the next steps once we get the first version working as a federated extension:

  1. we take the xeus lua specific parts out
  2. we create a https://github.com/jupyterlite/xeus-kernel repo that other kernels like xeus-lua can depend on. This repo would contain most of the logic
  3. jupyterlite-xeus-lua would live in its own repo, for example https://github.com/jupyterlite/xeus-lua-kernel, and would depend on the @jupyterlite/xeus-kernel packages. It would also have the tools (Dockerfile) to create the wasm file, and would specify it as a parameter
  4. streamline (or at least document) the process to place the files generated by emscripten so they are picked up by the extension on build
  5. Clean up @ts-ignore in the code

Reducing the size of the wasm file

The current wasm file is about 40MB which is quite huge.

Wondering whether there would be a way to reduce the size. Maybe that would involve passing a couple of flags to the emscripten build command.

`build:labextension:dev` stage fails

@jtpio
here is my attempt to compile xeus-lua as seperate extension.

First I had to add a https://github.com/DerThorsten/jupyterlite_xeus_lua/blob/main/src/xeus_lua.d.ts which I had not in the latest version of xeus-lua directly integrated in jupyterlite (at some point I also had such a file in the direct integrated xeus lua)

but jlpm run build fails horrible at the build:labextension:dev stage
( jlpm run build:lib without any error)

Any hint what could be the problem?

 tsc
$ jupyter labextension build --development True .
Building extension in .

Compilation starting…


<--- Last few GCs --->

[48218:0x55a0f730ff30]   364047 ms: Scavenge 8000.4 (8222.0) -> 7996.8 (8223.0) MB, 12.5 / 0.0 ms  (average mu = 0.169, current mu = 0.038) task 
[48218:0x55a0f730ff30]   364133 ms: Scavenge 8002.9 (8223.0) -> 7996.0 (8231.2) MB, 16.2 / 0.0 ms  (average mu = 0.169, current mu = 0.038) allocation failure 
[48218:0x55a0f730ff30]   366750 ms: Mark-sweep 8011.5 (8231.2) -> 7988.5 (8231.7) MB, 2509.7 / 0.1 ms  (average mu = 0.198, current mu = 0.226) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x7f3b40cc75c9 node::Abort() [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
 2: 0x7f3b40ba94a3 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > node::SPrintFImpl<int&, char const*>(char const*, int&, char const*&&) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
 3: 0x7f3b4107a832 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
 4: 0x7f3b4107ab0b v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
 5: 0x7f3b412411d6  [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
 6: 0x7f3b412412c1  [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
 7: 0x7f3b41255c24 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
 8: 0x7f3b41256524 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
 9: 0x7f3b412bc5cd v8::internal::ScavengeJob::Task::RunInternal() [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
10: 0x7f3b4115abdc  [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
11: 0x7f3b40d44794 node::PerIsolatePlatformData::RunForegroundTask(std::unique_ptr<v8::Task, std::default_delete<v8::Task> >) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
12: 0x7f3b40d47f73 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
13: 0x7f3b40230901  [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/./libuv.so.1]
14: 0x7f3b402427fc  [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/./libuv.so.1]
15: 0x7f3b4023102c uv_run [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/./libuv.so.1]
16: 0x7f3b40bfafd7 node::SpinEventLoop(node::Environment*) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
17: 0x7f3b40d148d5 node::NodeMainInstance::Run(int*, node::Environment*) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
18: 0x7f3b40d14d88 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
19: 0x7f3b40c8a9fc node::Start(int, char**) [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/../lib/libnode.so.93]
20: 0x55a0f648f0fb main [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/node]
21: 0x7f3b40293565 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
22: 0x55a0f648f146  [/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/bin/node]
An error occurred.
subprocess.CalledProcessError: Command '['node', '/home/derthorsten/src/jupyterlite_xeus_lua/node_modules/@jupyterlab/builder/lib/build-labextension.js', '--core-path', '/home/derthorsten/miniconda3/envs/jupyterlite-xeus-lua/lib/python3.9/site-packages/jupyterlab/staging', '/home/derthorsten/src/jupyterlite_xeus_lua', '--development']' died with <Signals.SIGABRT: 6>.
See the log file for details:  /tmp/jupyterlab-debug-gu5vc6u_.log

Bundle the wasm file

now that we move the xeus lua related files to the asset dir via https://github.com/DerThorsten/jupyterlite_xeus_lua/blob/3a22e56672ef5aca0655fc64aaa39fac74e5b6a2/webpack.config.js#L4

we are facing the problem that xeus_lua.js it not found

ModuleNotFoundError: Module not found: Error: Can't resolve './xeus_lua' in '/home/derthorsten/src/jupyterlite_xeus_lua/lib'
    at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/Compilation.js:1930:28
    at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:793:13
    at eval (eval at create (/home/derthorsten/src/jupyterlite_xeus_lua/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:10:1)
    at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:275:22
    at eval (eval at create (/home/derthorsten/src/jupyterlite_xeus_lua/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:9:1)
    at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:431:22
    at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:124:11
    at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:665:25
    at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:850:8
    at /home/derthorsten/src/jupyterlite_xeus_lua/node_modules/webpack/lib/NormalModuleFactory.js:970:5
resolve './xeus_lua' in '/home/derthorsten/src/jupyterlite_xeus_lua/lib'
...
...
...

@jtpio you mentioned:

Locally excluding the emscripten generated files seems to help build the extension.

However the files are not properly emitted to labextension/static yet.

@jtpio any hint on how to continue on that?

Do we need to change the import statement?
https://github.com/DerThorsten/jupyterlite_xeus_lua/blob/3a22e56672ef5aca0655fc64aaa39fac74e5b6a2/src/worker.ts#L3

this is all a bit beyond my typescript / webpack competence ...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.