Thanks again for getting this going.

As I've hinted at over on #41, a huge benefit of static sites is that they can be, well static. By this I mean: once you get a build working and put someplace, it will work, provided some future browser hasn't disabled a critical API, even if that someplace is a USB stick or DVD in a computer that can't access the internet.

I don't think it's reasonable in the long term to trust any of the major CDNs to necessarily be around in five years, much less bespoke ones like mathjax and pyodide. Further, CDNs are a source of lapses in user privacy and can be abused/disabled to affect harm/censorship by any number of agents.

I would be tempted to give a pass to content-addressable systems (e.g. git, fossil, bittorent, hypercore, ipfs) that can be verifiably cached, locally, and differentiably updated, but for now I think part of being able to fully realize the potential of this system is its ability to generate archival-grade content that can be used without access to The Internet, even if that isn't the default mode of operation.

The Challenges

Right now, this looks like:

• mathjax:
  • understandable
    • there is (at least) jupyter_server_mathjax
  • I feel like lab core or server should probably just suck it up and distribute it
    • but failing that, another, another copy of mathjax in a distributed package a la #41 isn't going to kill anybody's internet
• pyodide:
  • sticky indeed!
    • import pandas, matplotlib actually only pulls 32mb of data, which honestly is within the range of "sane," i'd say, to distribute as an sdist... and even a whl, if it doesn't get duplicated.
      • over on ipydrawio, i've done some of the dirty work to get path-perfect things in place with !!file-loader!, so it's definitely doable to make something that quacks like their CDN.

Once it is working, "offline," keeping it that way is always going to be a challenge... I've had some luck with gross /etc/hosts stuff in VM testing, but perhaps there are better, browser-based ways to do it now.

For now they are not implemented:

https://github.com/jtpio/jupyterlite/blob/aabfcc73dd9de50ceda35b14cfb7633908628aaa/packages/contents/src/contents.ts#L373

But could easily be added and reuse the same storage mechanism of the contents, and fullfill the Jupyter Server API:

https://petstore.swagger.io/?url=https://raw.githubusercontent.com/jupyter/jupyter_server/master/jupyter_server/services/api/api.yaml#/contents/post_api_contents__path__checkpoints

How easy do you think it would be to add comms support?

This doesn't seem to be the case at the moment when serving Pyodide from a local server started with yarn serve (which uses http-server):

For now all server components are in the @jupyterlite/server package:

https://github.com/jtpio/jupyterlite/tree/main/packages/server/src

They should instead be in their own packages:

• @jupyterlite/contents
• @jupyterlite/kernels
• @jupyterlite/settings
• ...

And have their own extension packages providing a plugin to the JupyterLiteServer application:

• @jupyterlite/contents-extension
• @jupyterlite/kernels-extension
• @jupyterlite/settings-extension

This will make it easier to swap (default) implementations of these components. For example an application might to want to save the settings to the cloud via API requests instead of localStorage.

It would be cool to also be able to start remote kernels on https://mybinder.org.

Similar to what the thebe project does.

The JupyterLite frontend would still load in seconds. But the remote kernel might of course take some time to start.

With JupyterLab 3.1.0a8 shipping with the initial RTC work, we might want to start considering what enabling real time collaboration would look like with JupyterLite.

Problem

Enable real time collaboration with JupyterLite. Questions:

• should that apply to anyone using the same JupyterLite deployment (that might be too disruptive as a default)
  • this will already be useful to users opening multiple tabs with the same notebook
• should we enable it on the demo website by default?
  • maybe we'll need some kind of token sharing mechanism (implemented as a separate plugin) to let people copy and send links to other folks

Proposed Solution

Incremental steps we can take:

• start using the 3.1.0-alpha.x @jupyterlab packages
• enable the collaborative flag in PageConfig
• use a different provider than the default websocket provider (will require a change upstream to allow for more flexibility)

Description

When using the Rename context menu item against a local directory, the director gets renamed, but not its children, so they disappear.

Reproduce

1. Use the [folder+] icon
2. enter Untitled Folder
3. make a notebook Untitled.ipynb
4. navigate back up
5. rename the folder to foo
6. re-enter folder, see nothing
7. navigate back up
8. rename the folder back to Untitled Folder
9. re-enter folder, see Untitled.ipynb

Expected behavior

On step 6, see Untitled.ipynb

Context

• discovered while "testing" #94 😆
  • that PR will certainly make robust renaming more complicated, but not sure if it needs to be fixed on that PR...

Description

Opening a new issue to track this comment: https://github.com/jtpio/jupyterlite/pull/64#issuecomment-826630567

The /tree redirect doesn't seem to work on Binder or when serving locally with yarn serve:

There also seems to be a redirect / base url issue on RTD:

Although direct access to https://jupyterlite.readthedocs.io/en/latest/_static/classic/index.html is fine.

Reproduce

https://jupyterlite.readthedocs.io/en/latest/_static/lab/index.html

Problem

The new Kernel and Server APIs are similar, but still unlike, other things. Having robust, up-to-date API documentation will help others understand what is going on.

Proposed Solution

With #64 in, we have a solid place to land them, but will need a few approaches to getting the docs.

• #69: typescript: typedoc and typedoc-plugin-markdown from npm
  • a well-understood option, used by core, lumino, etc.
  • The default output is pretty nice, but difficult to sanely integrate into an existing build
  • probably use a markdown target so myst-nb can pick it up
• python: sphinx-autodoc and sphinx-autodoc-typehints from pip/conda
  • ideally we'd be able to use {automodule} pyolite and get all the goods
  • ran into some issues on #64
• schema: @adobe/jsonschema2md or sphinx-jsonschema
  • eventually we'll have a JSON schema for a few things (e.g. #41)
  • sphinx-jsonschema is up and running as a preview on #58... it looks... good enough

Additional context

• keeping the various docs config files up-to-date can be tricky.
  • luckily most accept a declarative input file rather than requiring executable code, so will likely be relatively easy to generate these from the "facts" on disk with doit
  • these are worth checking in

Since settings are stored in localStorage with the current implementations, it would be nice to have a plugin to easily clear them.

Maybe that could be a simple command for now with a confirmation dialog.

The current logo is really cool: https://github.com/jtpio/jupyterlite/blob/main/docs/_static/icon.svg

And it would be great if we could use it for the GitHub organization when we start moving things there, or even in the README.md.

However it's not clear whether it complies to the design guidelines mentioned in https://github.com/jupyter/design/blob/master/brandguide/brand_guide.pdf

There is also a discussion about this in jupyter/design#65.

As an alternative, we could also replace the semicircles by a plain circle (or something else), even though it would be nice to keep the semicircles:

https://github1s.com makes it very easy to browse and read code on GitHub with VS Code, and boots in seconds.

It would be cool to have something similar that would open JupyterLite instead. If the repo on GitHub contains notebooks, they could then be executed and interacted with easily.

Opening this issue so we can keep that in mind when implementing different ways of retrieving and sharing content, and also when thinking about deployments and publishing stories.

Follow-up to #58

To showcase the use of existing federated extensions for JupyterLab (and JupyterLab Classic):

• ipydrawio
• jupyterlab-cell-flash
• jupyterlab-tour: #31
• cool themes
• ?

This is to fix the following TODO:

https://github.com/jtpio/jupyterlite/blob/373b8aaeb4b92e9fc7697d2bedd51c5125c14457/docs/conf.py#L97-L99

So other distributions and lab-based applications can create their own build using the @jupyterlite packages.

Description

After successfully completing the RUN yarn step RUN yarn build results in the following error:

@jupyterlite/app-lab: [webpack-cli] ModuleNotFoundError: Module not found: Error: Can't resolve '../py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl' in '/jupyterlite/packages/pyolite-kernel/lib'
@jupyterlite/app-lab:     at /jupyterlite/node_modules/webpack/lib/Compilation.js:1672:28
@jupyterlite/app-lab:     at /jupyterlite/node_modules/webpack/lib/NormalModuleFactory.js:712:13
@jupyterlite/app-lab:     at eval (eval at create (/jupyterlite/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:10:1)
@jupyterlite/app-lab:     at /jupyterlite/node_modules/webpack/lib/NormalModuleFactory.js:273:22
@jupyterlite/app-lab:     at eval (eval at create (/jupyterlite/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:9:1)
@jupyterlite/app-lab:     at /jupyterlite/node_modules/webpack/lib/NormalModuleFactory.js:402:22
@jupyterlite/app-lab:     at /jupyterlite/node_modules/webpack/lib/NormalModuleFactory.js:117:11
@jupyterlite/app-lab:     at /jupyterlite/node_modules/webpack/lib/NormalModuleFactory.js:628:24
@jupyterlite/app-lab:     at /jupyterlite/node_modules/webpack/lib/NormalModuleFactory.js:782:8
@jupyterlite/app-lab:     at /jupyterlite/node_modules/webpack/lib/NormalModuleFactory.js:902:5
@jupyterlite/app-lab: resolve '../py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl' in '/jupyterlite/packages/pyolite-kernel/lib'
@jupyterlite/app-lab:   using description file: /jupyterlite/packages/pyolite-kernel/package.json (relative path: ./lib)
@jupyterlite/app-lab:     Field 'browser' doesn't contain a valid alias configuration
@jupyterlite/app-lab:     using description file: /jupyterlite/packages/pyolite-kernel/package.json (relative path: ./py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl)
@jupyterlite/app-lab:       no extension
@jupyterlite/app-lab:         Field 'browser' doesn't contain a valid alias configuration
@jupyterlite/app-lab:         /jupyterlite/packages/pyolite-kernel/py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl doesn't exist

I noticed that the file reported missing i.e. ./py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl is indeed missing in the repo. Not sure if this python wheel is supposed to be created in an earlier step that I am missing?

Reproduce

If you perform a docker built . with the following Dockerfile you should get the error above:

FROM condaforge/mambaforge:latest

RUN git clone https://github.com/jtpio/jupyterlite
WORKDIR jupyterlite

#Installing required tools
RUN mamba env update --file .binder/environment.yml

# Make RUN commands use the new environment:
SHELL ["conda", "run", "-n", "jupyterlite-dev", "/bin/bash", "-c"]

RUN yarn
RUN yarn build

I confirmed that the installed yarn is a 1.x version, also ran the image interactively to see if would get a different behavior runnng the command from the terminal and ensuring that the correct python environment was active. Gave same result.

Expected behavior

My expectation was to complete the yarn built steps without errors.

Context

• Operating System and version: Ubuntu 20.04.
• Browser and version:
• JupyterLite version: Current git snapshot

Problem

The webpack build(s) already take(s) a long time. As a site publisher adds additional extensions, this will take longer until it just doesn't work anymore: for example, while published on npm, ipydrawio doesn't reliably work as a source extension because... 5000 files. Further, and for the reason above, unless they really need it, I don't think a lot of extension authors will even be testing source installs.

Proposed Solution

Add federated_extensions to pageInfo and the machinery to load them.

Indeed, the jupyterlite kernels, etc. would likely be useful outside of a jupyterlite install, e.g. when someone is building content, so they gain access to things like jupyter-lsp, etc. but that's probably for another day, and maybe counterfeited by a preview mode on #41, if the complexity is too high (because patching ServiceManager).

Additional context

For pure js builds, provide some mechanism to get sdists from pypi.io by URL (or locally), unpack them, copy the files into lab/labextensions/.

For python-driven builds (#41), be able to copy things from $PREFIX/share/jupyter/labextensions to lab/labextensions (https://github.com/jtpio/jupyterlite/issues/16#issuecomment-821980005).

These example notebooks could be added to the local storage by default (so available in the file browser on the first startup), and be used to quickly showcase a specific feature of a kernel or the language.

During the release process, the static assets can be published as release assets so it's easy for anyone to download them and serve the content of the tarball right away with a simple web server.

It would be nice to have an overview diagram in the readme, to get an idea of how it works.

And also to illustrate the "server app" running as another Lumino application.

To unify the way the offline data is stored in the brower, it would make sense to also use localForage to save the settings instead of the custom storage package:

https://github.com/jtpio/jupyterlite/blob/main/packages/storage/src/localstorage.ts

It would be great to to support additional kernels such as JavaScript via quickjs and perhaps TypeScript.

So Pyodide could be served from a local web server instead of the CDN if needed.

There were some whispers at today's jlab dev meeting about a architecture/explainer diagram that had been made for an earlier presentation of jupyterlite. I for one would love to see it.

@jtpio Any chance that we can get that diagram posted publicly somewhere? I think for now it would go nicely in the README, and can then be migrated to the deeper docs once those exist

Description

Noted when experimenting during today's JupyterLab call.

Reproduce

In private window:

1. Open https://jupyterlite.readthedocs.io/en/latest/try/classic
2. See the page loads ok
3. Click on "New notebook"
4. See the notebook never loads
5. See JupyterLab does not start from https://jupyterlite.readthedocs.io/en/latest/try/lab
6. See that clicking on Untitled.ipynb from classic listing opens a blank page; see the only message in console is "Starting app"

Close private window, open a new private window:

1. Open https://jupyterlite.readthedocs.io/en/latest/try/lab
2. Create a new "Pyolite" notebook from Launcher
3. See the notebook launches and works smoothly
4. Open https://jupyterlite.readthedocs.io/en/latest/try/classic
5. Open Untitled.ipynb and see that it works now
6. Click on "New notebook"
7. See that the notebook is created and works well

Some very uneducated guesses:

• maybe the notebook created from classic interface is missing kernel type and this causes the issue
• maybe some things that are saved in the browser local storage do not get correctly saved if we start with opening classic rather than lab?
• or maybe there is a plugin missing in the classic that is required?

Context

• Browser and version:
  • Google Chrome 90.0.4430.72
  • Microsoft Edge 91.0.864.1

For now the handling of directories does not fully work:

Description

On the docs, all the nbdime stuff shows up in the UI, but very little works, making lots of noise.

Reproduce

• open the docs
• open the developer tools
• see lots of errors in console
• open a notebook
• click nbdime buttons
• see ugly "cannot fetch" errors

Expected behavior

• now
  • see no errors when the console opens
    • test this #6
• eventually
  • nbdime works

Context

• RTD

Alternatives

• specifically don't copy nbdime in the doit docs:extend
  • pro
    • low-cost
    • doesn't ship the artifacts in the tarball
  • con
    • doesn't document how to disable extensions
    • other extensions might need it from sharedPackages and not bundle
• add a disabled plugins key to PageConfig
  • pro
    • good documentation
  • con
    • maybe not canonical?
    • ships the assets in the tarball

Problem

When deploying a JupyterLite, some default settings might not be right for the use case of a particular deployment. It should be easy to finely-tune default settings, down the tool level (e.g. use one theme in /lab, and another in /classic/notebooks and another in /classic/editor).

Proposed Solution

Building on #58, add a new section to jupyter-lite.json that has the same structure as overrides.json (no JSON5, thanks!). So to disable the modal command palette:

{
  "jupyter-lite-schema-version": 0,
  "jupyter-config-data": {},
  "settings-overrides": {
    "@jupyterlab/apputils-extension:palette": {"modal": false}
  }
}

Alternately, these could right into jupyter-config-data, but it wouldn't be "canonical"...

Additional context

Wherever it lands, the Schema definition would be something like:

"settings-overrides": {
  "description": "plugin-keyed settings that replace the defaults. See https://jupyterlab.readthedocs.io/en/stable/user/directories.html#overrides-json",
  "type": "object",
  "patternProperties": {
    "that-enormous-regex:.*": {
      "type": "object"
    }
  }
}

In settings.ts, these would be read either by PageConfig (simple) or we have equivalent jupyter-lite.json reading equipment on the "app" side (more flexible).

Problem

Lowering the entry barrier

Proposed Solution

Allow _static to be published to GitHub Pages

To make it easier to serve with a simple web server, we should check whether the URL routing (or at least the location rewrite) can be disabled.

This will make it more convenient to refresh the page without a 404.

Alternatively we might want to define placeholders to handle that more gracefully.

Problem

It's still a little unclear how one would actually deploy and "own" jupyterlite.

Proposed Solution

Add a Deployment section which includes:

• how to configure
  • where to put a jupyterlite.json
  • JSON Schema
• how to get the code
  • we could be shipping the tarball to the docs site, which is as good as any, for now
• configuring different SaaS providers
  • vercel
  • github pages
  • readthedocs
• on-premises
  • nginx?
  • inside other host apps
    • jupyterhub
    • jupyter_server (#41)
      • jupyter-server-proxy
    • django?
    • fastapi?
    • flask?
  • how to configure CDN replacements (#45)

Additional context

• #30 probably required before this is actually important

Problem

At present, a JupyterLite pretty much must include all of the kernels, exactly the contents/settings extensions provided etc. It's also basically not possible to add new serverlite extensions without forking or going through the PR process here.

Proposed Solution

At extension build time adopt either:

• exactly the jupyter labextension build toolchain,
  • we might be able to get by with "requiring" (as in, might build, but not run without) package.json#/jupyterlab/sharedPackage pointers to jupyterlite/server
• an extension-by-subclass of that machinery for jupyter serverlitextension build (that is very long)

At configuration time, either:

• use exactly federated_extensions
• use a very similar federated_serverlite_extensions

At runtime:

• use the config
• actually read package.json

Additional context

As a stopgap, it was proposed elsewhere to offer a way to specify a mock server response via OpenAPI to provide enough server information to trick an existing, useful labextension into working, e.g. jupyterlab-videochat.

Problem

I couldn’t figure out how to render a matplotlib figure in a notebook and don’t know if this is supported yet.

Proposed Solution

If figure rendering is supported can an example be shown in these documentation - if unsupported can this also be noted in the documentation.

Description

I followed https://github.com/jtpio/jupyterlite/blob/main/CONTRIBUTING.md, and when I executed doit, I got the following errors.

(common) D:\code\python\jupyterlite>doit list --all --status
R build                  build code and intermediate packages
R build:js:app:classic   build JupyterLite Classic with webpack
R build:js:app:lab       build JupyterLite Lab with webpack
R build:js:lib           build .ts files into .js files
R build:js:pack          build the JupyterLite distribution
U build:py:pyolite       build the pyolite python package for the brower with flit
R docs                   build documentation
R docs:sphinx            build the documentation site with sphinx
R env                    keep environments in sync
R env:binder
R lint                   format and ensure style of code, docs, etc.
R lint:black             format python files with black
R lint:eslint            format and verify .ts, .js files with eslint
R lint:prettier          format .ts, .md, .json, etc. files with prettier
R setup                  perform initial non-python setup
R setup:js               install node packages
R test                   test jupyterlite
R test:js                run the .js, .ts unit tests with jest
R watch                  watch sources and rebuild on change
R watch:docs             watch .md sources and rebuild the documentation
R watch:js               watch .ts, .js, and .css sources and rebuild packages and apps

(common) D:\code\python\jupyterlite>doit
.  lint:black
All done! ✨ 🍰 ✨
7 files would be left unchanged.
.  setup:js
Traceback (most recent call last):
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\doit_cmd.py", line 190, in run
    return command.parse_execute(args)
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\cmd_base.py", line 150, in parse_execute
    return self.execute(params, args)
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\cmd_base.py", line 601, in execute
    return self._execute(**exec_params)
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\cmd_run.py", line 264, in _execute
    return runner.run_all(self.control.task_dispatcher())
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\runner.py", line 256, in run_all
    self.run_tasks(task_dispatcher)
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\runner.py", line 222, in run_tasks
    catched_excp = self.execute_task(node.task)
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\runner.py", line 177, in execute_task
    return task.execute(self.stream)
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\task.py", line 460, in execute
    action_return = action.execute(task_stdout, task_stderr)
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\site-packages\doit\action.py", line 216, in execute
    process = subprocess.Popen(
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "c:\users\liu.d.h\anaconda3\envs\common\lib\subprocess.py", line 1307, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

(common) D:\code\python\jupyterlite>

Expected behavior

Build as expected.

Context

• Operating System and version: Microsoft Windows [Version 10.0.19042.928]
• Browser and version: Chrome 90
• JupyterLite version: master branch

I can confirm that I have node/npm/yarn configured correctly. When I executed yarn build, it was ok.

(common) D:\code\python\jupyterlite>yarn build
yarn run v1.22.10
$ yarn build:lib && yarn build:app
$ lerna run build --scope @jupyterlite/metapackage
lerna notice cli v3.22.1
lerna info versioning independent
lerna info ci enabled
lerna notice filter including "@jupyterlite/metapackage"
lerna info filter [ '@jupyterlite/metapackage' ]
lerna info Executing command in 1 package: "yarn run build"
lerna info run Ran npm script 'build' in '@jupyterlite/metapackage' in 0.9s:
$ tsc -b
lerna success run Ran npm script 'build' in 1 package in 0.9s:
lerna success - @jupyterlite/metapackage
$ lerna run build --stream --parallel --scope "@jupyterlite/app-classic" --scope "@jupyterlite/app-lab"
lerna notice cli v3.22.1
lerna info versioning independent
lerna info ci enabled
lerna notice filter including ["@jupyterlite/app-classic","@jupyterlite/app-lab"]
lerna info filter [ '@jupyterlite/app-classic', '@jupyterlite/app-lab' ]
lerna info Executing command in 2 packages: "yarn run build"
@jupyterlite/app-classic: $ webpack --config ../webpack.config.js
@jupyterlite/app-lab: $ webpack --config ../webpack.config.js
@jupyterlite/app-classic: (node:14312) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./" in the "exports" field module resolution of the package at D:\code\python\jupyterlite\node_modules\postcss\package.json.
@jupyterlite/app-classic: Update this package.json to use a subpath pattern like "./*".
@jupyterlite/app-classic: (Use `node --trace-deprecation ...` to show where the warning was created)
@jupyterlite/app-lab: (node:20884) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./" in the "exports" field module resolution of the package at D:\code\python\jupyterlite\node_modules\postcss\package.json.
@jupyterlite/app-lab: Update this package.json to use a subpath pattern like "./*".
@jupyterlite/app-lab: (Use `node --trace-deprecation ...` to show where the warning was created)
@jupyterlite/app-classic: assets by path *.js 12.4 MiB
@jupyterlite/app-classic:   assets by chunk 1.69 MiB (id hint: vendors) 8 assets
@jupyterlite/app-classic: assets by chunk 2.79 MiB (auxiliary name: main)
@jupyterlite/app-classic:   assets by path *.svg 1.73 MiB 3 assets
@jupyterlite/app-classic:   assets by path *.eot 363 KiB 3 assets
@jupyterlite/app-classic:   assets by path *.ttf 362 KiB 3 assets
@jupyterlite/app-classic:   assets by path *.woff 203 KiB 3 assets
@jupyterlite/app-classic:   assets by path *.woff2 164 KiB 3 assets
@jupyterlite/app-classic: asset pyolite-0.1.0-py3-none-any.whl 2.91 KiB [compared for emit] [from: ../../packages/pyolite-kernel/py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl]
@jupyterlite/app-classic: orphan modules 116 KiB [orphan] 57 modules
@jupyterlite/app-classic: runtime modules 8.16 KiB 12 modules
@jupyterlite/app-classic: modules by path ../../node_modules/ 10.7 MiB
@jupyterlite/app-classic:   cacheable modules 10.7 MiB 1282 modules
@jupyterlite/app-classic:   3 modules
@jupyterlite/app-classic: modules by path ../../packages/ 97.1 KiB (javascript) 2.91 KiB (asset)
@jupyterlite/app-classic:   javascript modules 94.1 KiB 26 modules
@jupyterlite/app-classic:   asset modules 2.97 KiB (javascript) 2.91 KiB (asset)
@jupyterlite/app-classic:     ../../packages/pyolite-kernel/py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl 42 bytes (javascript) 2.91 KiB (asset) [built] [code generated]
@jupyterlite/app-classic:     ../../packages/pyolite-kernel/lib/worker.js?raw 2.93 KiB [built] [code generated]
@jupyterlite/app-classic: modules by path ./ 6.27 KiB
@jupyterlite/app-classic:   ./index.js 5.11 KiB [built] [code generated]
@jupyterlite/app-classic:   ./build/style.js 1.16 KiB [built] [code generated]
@jupyterlite/app-classic: crypto (ignored) 15 bytes [built] [code generated]
@jupyterlite/app-classic: node-fetch (ignored) 15 bytes [built] [code generated]
@jupyterlite/app-classic: webpack 5.31.2 compiled successfully in 16371 ms
@jupyterlite/app-classic: asset index.css 16.1 KiB [compared for emit] (name: index)
@jupyterlite/app-classic: asset index.js 0 bytes [compared for emit] [minimized] (name: index)
@jupyterlite/app-classic: Entrypoint index 16.1 KiB = index.css 16.1 KiB index.js 0 bytes
@jupyterlite/app-classic: ../../node_modules/@jupyterlab/theme-dark-extension/style/index.css 50 bytes [built] [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-dark-extension/style/variables.css 15.3 KiB [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-dark-extension/style/base.css 472 bytes [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-dark-extension/style/index.css 342 bytes [code generated]
@jupyterlite/app-classic: webpack 5.31.2 compiled successfully in 2307 ms
@jupyterlite/app-classic: asset index.css 15 KiB [compared for emit] (name: index)
@jupyterlite/app-classic: asset index.js 0 bytes [compared for emit] [minimized] (name: index)
@jupyterlite/app-classic: Entrypoint index 15 KiB = index.css 15 KiB index.js 0 bytes
@jupyterlite/app-classic: ../../node_modules/@jupyterlab/theme-light-extension/style/index.css 50 bytes [built] [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-light-extension/style/variables.css 14.2 KiB [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-light-extension/style/base.css 472 bytes [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-light-extension/style/index.css 342 bytes [code generated]
@jupyterlite/app-classic: webpack 5.31.2 compiled successfully in 2302 ms
@jupyterlite/app-classic: asset index.css 21.2 KiB [compared for emit] (name: index)
@jupyterlite/app-classic: asset index.js 0 bytes [compared for emit] [minimized] (name: index)
@jupyterlite/app-classic: Entrypoint index 21.2 KiB = index.css 21.2 KiB index.js 0 bytes
@jupyterlite/app-classic: ../../node_modules/@telamonian/theme-darcula/style/index.css 50 bytes [built] [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@telamonian/theme-darcula/style/codemirror-darcula.css 2.68 KiB [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@telamonian/theme-darcula/style/variables.css 18.1 KiB [code generated]
@jupyterlite/app-classic: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@telamonian/theme-darcula/style/index.css 457 bytes [code generated]
@jupyterlite/app-classic: webpack 5.31.2 compiled successfully in 2301 ms
@jupyterlite/app-lab: assets by path *.js 13.6 MiB
@jupyterlite/app-lab:   assets by chunk 6.67 MiB (id hint: vendors) 41 assets
@jupyterlite/app-lab: assets by chunk 2.79 MiB (auxiliary id hint: vendors)
@jupyterlite/app-lab:   assets by path *.svg 1.73 MiB 3 assets
@jupyterlite/app-lab:   assets by path *.eot 363 KiB 3 assets
@jupyterlite/app-lab:   assets by path *.ttf 362 KiB 3 assets
@jupyterlite/app-lab:   assets by path *.woff 203 KiB
@jupyterlite/app-lab:     asset 9fe5a17c8ab036d20e6c5ba3fd2ac511.woff 99.3 KiB [compared for emit] (auxiliary id hint: vendors)
@jupyterlite/app-lab:     asset 099a9556e1a63ece24f8a99859c94c7d.woff 87.9 KiB [compared for emit] (auxiliary id hint: vendors)
@jupyterlite/app-lab:     asset 7124eb50fc8227c78269f2d995637ff5.woff 15.9 KiB [compared for emit] (auxiliary id hint: vendors)
@jupyterlite/app-lab:   assets by path *.woff2 164 KiB
@jupyterlite/app-lab:     asset e8a427e15cc502bef99cfd722b37ea98.woff2 76.4 KiB [compared for emit] (auxiliary id hint: vendors)
@jupyterlite/app-lab:     asset f7307680c7fe85959f3ecf122493ea7d.woff2 75 KiB [compared for emit] (auxiliary id hint: vendors)
@jupyterlite/app-lab:     asset f0f8230116992e521526097a28f54066.woff2 13 KiB [compared for emit] (auxiliary id hint: vendors)
@jupyterlite/app-lab: asset pyolite-0.1.0-py3-none-any.whl 2.91 KiB [compared for emit] [from: ../../packages/pyolite-kernel/py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl]
@jupyterlite/app-lab: orphan modules 118 KiB [orphan] 60 modules
@jupyterlite/app-lab: runtime modules 8.15 KiB 12 modules
@jupyterlite/app-lab: modules by path ../../node_modules/ 11.5 MiB
@jupyterlite/app-lab:   cacheable modules 11.5 MiB 1393 modules
@jupyterlite/app-lab:   3 modules
@jupyterlite/app-lab: modules by path ../../packages/ 92 KiB (javascript) 2.91 KiB (asset)
@jupyterlite/app-lab:   javascript modules 89.1 KiB 22 modules
@jupyterlite/app-lab:   asset modules 2.97 KiB (javascript) 2.91 KiB (asset)
@jupyterlite/app-lab:     ../../packages/pyolite-kernel/py/pyolite/dist/pyolite-0.1.0-py3-none-any.whl 42 bytes (javascript) 2.91 KiB (asset) [built] [code generated]
@jupyterlite/app-lab:     ../../packages/pyolite-kernel/lib/worker.js?raw 2.93 KiB [built] [code generated]
@jupyterlite/app-lab: modules by path ./ 5.37 KiB
@jupyterlite/app-lab:   ./index.js 3.7 KiB [built] [code generated]
@jupyterlite/app-lab:   ./build/style.js 1.67 KiB [built] [code generated]
@jupyterlite/app-lab: crypto (ignored) 15 bytes [built] [code generated]
@jupyterlite/app-lab: node-fetch (ignored) 15 bytes [built] [code generated]
@jupyterlite/app-lab: webpack 5.31.2 compiled successfully in 17638 ms
@jupyterlite/app-lab: asset index.css 16.1 KiB [compared for emit] (name: index)
@jupyterlite/app-lab: asset index.js 0 bytes [compared for emit] [minimized] (name: index)
@jupyterlite/app-lab: Entrypoint index 16.1 KiB = index.css 16.1 KiB index.js 0 bytes
@jupyterlite/app-lab: ../../node_modules/@jupyterlab/theme-dark-extension/style/index.css 50 bytes [built] [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-dark-extension/style/variables.css 15.3 KiB [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-dark-extension/style/base.css 472 bytes [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-dark-extension/style/index.css 342 bytes [code generated]
@jupyterlite/app-lab: webpack 5.31.2 compiled successfully in 1940 ms
@jupyterlite/app-lab: asset index.css 15 KiB [compared for emit] (name: index)
@jupyterlite/app-lab: asset index.js 0 bytes [compared for emit] [minimized] (name: index)
@jupyterlite/app-lab: Entrypoint index 15 KiB = index.css 15 KiB index.js 0 bytes
@jupyterlite/app-lab: ../../node_modules/@jupyterlab/theme-light-extension/style/index.css 50 bytes [built] [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-light-extension/style/variables.css 14.2 KiB [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-light-extension/style/base.css 472 bytes [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@jupyterlab/theme-light-extension/style/index.css 342 bytes [code generated]
@jupyterlite/app-lab: webpack 5.31.2 compiled successfully in 3042 ms
@jupyterlite/app-lab: asset index.css 21.2 KiB [compared for emit] (name: index)
@jupyterlite/app-lab: asset index.js 0 bytes [compared for emit] [minimized] (name: index)
@jupyterlite/app-lab: Entrypoint index 21.2 KiB = index.css 21.2 KiB index.js 0 bytes
@jupyterlite/app-lab: ../../node_modules/@telamonian/theme-darcula/style/index.css 50 bytes [built] [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@telamonian/theme-darcula/style/codemirror-darcula.css 2.68 KiB [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@telamonian/theme-darcula/style/variables.css 18.1 KiB [code generated]
@jupyterlite/app-lab: css ../../node_modules/css-loader/dist/cjs.js!../../node_modules/@telamonian/theme-darcula/style/index.css 457 bytes [code generated]
@jupyterlite/app-lab: webpack 5.31.2 compiled successfully in 3040 ms
lerna success run Ran npm script 'build' in 2 packages in 21.3s:
lerna success - @jupyterlite/app-classic
lerna success - @jupyterlite/app-lab
Done in 25.85s.

(common) D:\code\python\jupyterlite>ver

Microsoft Windows [Version 10.0.19042.928]

(common) D:\code\python\jupyterlite>

Description

build files for static serve on Windows broken.

Reproduce

1. clone the repo
2. yarn && pip install -r requirements.txt
3. doit build
4. yarn serve
5. open localhost:5000

Expected behavior

The page is shown normally.

Context

• Operating System and version: Microsoft Windows [Version 10.0.19042.928]
• Browser and version: Chrome 90
• JupyterLite version: master

The custom theme plugin does not support theming scrollbars:

https://github.com/jtpio/jupyterlite/blob/c53e2837cd96de83b00ac4b84ca1e7f81ea5000a/packages/theme-extension/src/index.ts#L24

While the upstream plugin does:

https://github.com/jupyterlab/jupyterlab/blob/b4cef7b471d8bac74e0430c63053198d00577c29/packages/apputils-extension/src/themesplugins.ts#L109

Using https://github.com/jupyterlab-contrib/jupyterlab-tour

This would help guide new users of the JupyterLite interface, and explain that kernels run the browser.

Also explain what the differences between notebooks and consoles are for instance.

Problem

If JupyterLite makes too many "special" data structures, etc. it will create headaches for the maintainers, as well as an undue burden on extension developers, etc.

Proposed Solution

When a schema/spec does exist, we should use it as best we can to help ensure we create compatible components and/or reduce our implementation/opinion burden without taking on complex end-user dependencies.

• find more canonical file formats, etc.
• find py/js implementations to use for build/validation tools
• maybe use them in the JS implementation

If there are places where we can propose these things be formalized beyond narrative and a reference implementation, JupyterLite offers a good testbed for independent implementations.

Additional context

Here are some additional files that do exist:

• Jupyter Server OpenAPI source for #94
  • currently has some issues jupyter-server/jupyter_server#518
• lab_config.json
• overrides.json for #81
• .ipython/profile_default/startup/*.py IPython startup files

Looks like pyolite is already a thing on PyPI: https://pypi.org/project/pyolite/

To avoid confusion we can pick a different name for the default Python kernel we ship in: https://github.com/jtpio/jupyterlite/tree/main/packages/pyolite-kernel

Description

Pandas read_csv gave FileNotFound error on example file iris.csv. I'm hope this is not a silly user error and I'm sorry in advance if it is.

Reproduce

• Create new notebook in examples folder
• import pandas as pd
• pd.read_csv("iris.csv")

Expected behavior

Load and display the content of iris.csv on screen

Context

• Operating System and version:
• MacOS 11.2.3
• MacBook Air (M1, 2020)
• Apple M1 chip
• Browser and version: Brave Version 1.24.84 Chromium: 90.0.4430.93 (Official Build) (arm64)
• JupyterLite version: can't find version number but I did this today (2021/06/01) using JupyterLite for the first time

Dispose worker for kernel 7f928317-a338-41fb-9c00-6c1d041909fd
default.js:1228 Connection lost, reconnecting in 0 seconds.
_reconnect @ default.js:1228
reconnect @ default.js:516
restart @ default.js:487
async function (async)
restart @ default.js:484
restartKernel @ sessioncontext.js:298
restart @ sessioncontext.js:758
async function (async)
restart @ sessioncontext.js:733
onClick @ toolbar.js:343
y.createElement.onMouseDown @ toolbar.js:402
Be @ react-dom.production.min.js:52
Ke @ react-dom.production.min.js:52
(anonymous) @ react-dom.production.min.js:53
xr @ react-dom.production.min.js:100
Cr @ react-dom.production.min.js:101
(anonymous) @ react-dom.production.min.js:113
De @ react-dom.production.min.js:292
(anonymous) @ react-dom.production.min.js:50
Lr @ react-dom.production.min.js:105
Zn @ react-dom.production.min.js:75
Jn @ react-dom.production.min.js:74
n.unstable_runWithPriority @ scheduler.production.min.js:18
Bl @ react-dom.production.min.js:122
Me @ react-dom.production.min.js:292
Xn @ react-dom.production.min.js:73
pyodide.asm.js:9 Python initialization complete
pyodide.js:146 Loading matplotlib, cycler, six, kiwisolver, numpy, pillow, pyparsing, python-dateutil, pytz
pyodide.js:196 Loading matplotlib from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/matplotlib.js
pyodide.js:196 Loading cycler from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/cycler.js
pyodide.js:196 Loading six from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/six.js
pyodide.js:196 Loading kiwisolver from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/kiwisolver.js
pyodide.js:196 Loading numpy from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/numpy.js
pyodide.js:196 Loading pillow from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pillow.js
pyodide.js:196 Loading pyparsing from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pyparsing.js
pyodide.js:196 Loading python-dateutil from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/python-dateutil.js
pyodide.js:196 Loading pytz from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pytz.js
pyodide.js:252 Loaded matplotlib, cycler, six, kiwisolver, numpy, pillow, pyparsing, python-dateutil, pytz
pyodide.js:146 Loading micropip, pyparsing, packaging
pyodide.js:196 Loading micropip from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/micropip.js
pyodide.js:184 pyparsing already loaded from default channel
pyodide.js:196 Loading packaging from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/packaging.js
pyodide.js:252 Loaded micropip, pyparsing, packaging
8ebb3553-36cc-44b7-9b0c-539a2f402d28:29 Pyolite kernel initialized, version 0.1.0
8ebb3553-36cc-44b7-9b0c-539a2f402d28:63 Inside worker {code: "import pandas as pd"}
pyodide.js:146 Loading pandas, numpy, python-dateutil, pytz
pyodide.js:196 Loading pandas from https://cdn.jsdelivr.net/pyodide/v0.17.0/full/pandas.js
pyodide.js:184 numpy already loaded from default channel
pyodide.js:184 python-dateutil already loaded from default channel
pyodide.js:184 pytz already loaded from default channel
pyodide.js:252 Loaded pandas, numpy, python-dateutil, pytz
8ebb3553-36cc-44b7-9b0c-539a2f402d28:63 Inside worker {code: "pd.read_csv("iris.csv")"}code: "pd.read_csv(\"iris.csv\")"__proto__: Object

Looks like it fails at the doit build step:

Problem

Tracking the following comment from https://github.com/jtpio/jupyterlite/issues/16#issuecomment-821980005 in its own issue:

Further while talking about release assets... Right now, the release is
about 14mb, 64 uncompressed. A huge proportion of that is the duplication
of large assets in lab/build and classic/build. It's probably worth
exploring how to get as much as possible to end up in ../static, relative
to each of them.

Proposed Solution

• Put all assets in a common /static folder
• Rename index.js, style.js and bundle.js to index-lab.js, index-retro.js ... so they don't collide
• Update the references to ./build, bundle.js and others

Additional context

Awesome stuff here! Maybe these have been brought up elsewhere, if so, feel free to close!

elevator pitch

Having an installable, python-based package with hooks for various other jupyter tools with a couple features would make it easy to adopt for workflows folk are already invested in.

Use Cases

• as a Jupyter user, locally be able to interactively build up a custom jupyterlite site with contents, custom extensions, etc. and do whatever you need to with the files
• as an extension developer community, build a demo site in CI and deploy to somewhere (e.g. github pages)

Features

nbconvert

One of the jyve features I loved building is the nbconvert exporter. That code's pretty rough, but it does the thing of marshalling all the static assets into a folder, ready to be committed to gh-pages (or dragged into SharePoint 😱) for demos.

I'm imagining two, not necessarily mutually exclusive outputs of:

• a full editable lab
• a single Document that looks like nbconvert, etc. but is built out of lab renderers
• a couple Documents, one per page

With those pieces, one could integrate nicely into other workflows like sphinx, jupyter-book, mkdocs, nbviewer, binder, etc.

design ideas

• a python package
  • an entry_point for jupyterlite site
  • this would make it show up in various CLI and API endpoints
  • all/some of the jupyterlite static assets as either...
    • data_files for "simplicity"
    • package_data it's gonna be big, duplication in wheels is no fun

As nbconvert likes to make one file, the default output would probably most usefully be a .zip file... not so much fun for previewing, but quite handy for moving around (as again, it's gonna be big). But if it gets more complicated...

labextension

There are bells and whistles that would need more tweaking than what the exposed nbconvert API offers. It would be lovely to have an in-juptyerlab(-classic) extension that built jupyterlite sites and then did... stuff with them.

design ideas

This part would have to be in data_files, but wouldn't be very big.

• offer it as a new File > export as... > custom jupyterlite
  • open a new mainareawidget (or whatever makes sense in classic)
    • capture the advanced options as a JSON schema
      • make a JSON schema from ui
    • a drop area for adding additional contents paths

As to targets, I'm imagining other outputs/targets, such as:

• git bundle
• start hosting the site
  • just /files
  • serve on a federated content network, e.g. dat/ipfs
    • these would work quite well if the core assets (and labextensions folk might include) end up being very static, as they would get deduplicated in the network and locally

A couple of improvements landed upstream to make the plugins easier to reuse and replace.

So we should start thinking about updating to the new packages at some point to pull them in.

Problem

At the moment it's really convenient to be able to open a PR and have the bot post a link to the preview. As well as having JupyterLite automatically be published to https://jupyterlite.vercel.app when the PR is merge into the main branch.

With https://github.com/jtpio/jupyterlite/pull/57 we however start having a more complex build system that also involves Python dependency management. And adding Python makes the Vercel configuration more complicated (also difficult to get it to work).

Proposed Solution

It would be interesting to start using this action: https://github.com/amondnet/vercel-action.

Looks like we would get everything needed:

• comment on new PRs with a link to the preview
• full control on the build and its dependencies via GitHub Actions workflows
• the app is still pushed to Vercel just like it is no

The other option mentioned in #57 was to deploy to GitHub Pages instead, using something like https://github.com/marketplace/actions/deploy-to-github-pages (not sure it would be easy to have a preview with this approach)

Additional context

Questions mostly coming from the iterations in https://github.com/jtpio/jupyterlite/pull/57.

Problem

As more people get to try JupyterLite by going to the demo link directly: https://jupyterlite.vercel.app/

We should provide a way for them to reach the repo, for example by swapping the about dialog for a custom one:

Proposed Solution

The about plugin was made more modular upstream in jupyterlab/jupyterlab#10089.

So this will be easier to implement after updating to the 3.1 alpha packages.

To make sure the generation of assets behaves correctly:

https://github.com/jtpio/jupyterlite/blob/b47066ed1d9a3530dd25f4a149517b54d926fe29/app/webpack.config.js#L12-L17

There could be an extra theme added to the build that is not the default light and dark, for example: https://github.com/telamonian/theme-darcula

In some way this sounds similar to having static assets distributed via Python packages.

It would be particularly useful for the pyodide kernel, so it could provide additional Python modules as-is and load them on the fly from a static url.

One way could be to leverage the webpack asset modules:

https://webpack.js.org/guides/asset-modules/

Although it would also be nice to support loading kernels and extensions from other remotes and CDN.

There should be some end to end testing, ideally with galata: https://github.com/jupyterlab/galata

I see jupyterlite as a game changer. Thanks so much for taking it so far.

My interest is in interactive data and ML apps. See awesome-panel.org and awesome-streamlit.org. I would like to be able to do interactive, exploratory data analysis and create data apps in the browser for the browser using jupyterlite.

One thing I would need is 1) the possibility to export to a stripped out version of the notebook just like Voila does it 2) some kind of widgets that can communicate and update plots and similar objects. Maybe its the comms and ipywidgets that are needed. Maybe not - we are already in the browser. So maybe something else would be easy to do.

If creating interactive viz and data apps in any is possible or could be supported I would appreciate anything that can move this forward. Like ideas or examples.