quantstack / jupyterlab-blockly Goto Github PK

Create the default set of blocks for Python. Investigate if Blockly provides a default toolbox for the supported generators or look into other projects using blockly, to extract a basic set of blocks that we can reuse for Python. If we don't find anything, we can always export the default toolbox from the blockly developer tools web page.

Some documentation to get started:

• Blockly
• Blockly Developer Tools

We could make some custom blocks to do some simple plotting with matplotlib.

Create custom blocks that use the OpenCV library.

Make blocks for the most used functions that let the user access computer vision or image processing techniques for learning or quick prototyping.

Potentially we could define some custom blocks from a Python script or from a Notebook if we use some decorators.

Something like:

@block
def myfunction(a: int, b: int) -> int:
    return a + b

Since Jupyterlab and Blockly support translations and localization would be excellent to support it on the extension.

Some documentation to get started:

• JupyterLab
• Blockly

The idea would be to add the dependency ITranslator from JupyterLab, listen to the translator and when it changes, load the corresponding language in blockly or default to English in case the language is not supported.

Note: I'm not sure if we can listen for changes in the ITranslator or if there is a way to know what language is loaded in Lab.

For how to import the ITranslator and load it, see the following example:
https://github.com/jupyterlab/extension-examples/blob/8bc09d2839efed8078c9fbc43da0fd39a0b7d0a3/signals/src/index.ts#L55

For every other UI feature, we add to the extension, we need to add the labels using trans.__('Label Example');, see:
https://github.com/jupyterlab/extension-examples/blob/8bc09d2839efed8078c9fbc43da0fd39a0b7d0a3/signals/src/index.ts#L81

Generators for Niryo one

https://github.com/NiryoRobotics/niryo_one_ros/tree/e8a218cbafd03111298c7752f9b272e3b0b623bd/niryo_one_user_interface/blockly_code_generator

cc @wolfv

When the kernel hangs because of an infinite loop or any error in the code, we can not restart it.
Would be great to support the actions from the Kernel menu.
When closing the kernel, the dropdown to change kernels doesn't change.

When trying to run jupyterlab-blockly on JupyterLab 4.1.5 (current latest, didn't work with previous 4 versions either) the following errors are printed in the Browser's Javascript console:

jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1 Plugin 'jupyterlab-blocky:plugin' failed to activate.
(anonymous) @ jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1
jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1 TypeError: mainMenu.kernelMenu.kernelUsers.add is not a function
    at activate (index.js:163:1)
    at jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:1715670
    at async Promise.all (index 164)
(anonymous) @ jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1
jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1 TypeError: Cannot read properties of undefined (reading 'createCellHeader')
    at $e.initializeDOM (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:246971)
    at $e.initializeDOM (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:250923)
    at set placeholder (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:243844)
    at $e.onBeforeAttach (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:247939)
    at $e.processMessage (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:1781568)
    at b (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:1753611)
    at Object.o [as sendMessage] (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:1752442)
    at BlocklyLayout.onBeforeAttach (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:1787213)
    at BlocklyLayout.onBeforeAttach (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:1795177)
    at BlocklyLayout.processParentMessage (jlab_core.23e8a6b191d1fa15c1b7.js?v=23e8a6b191d1fa15c1b7:1:1786612)

None are shown server-side (I assume because the instructions never reach it due to the browser errors, probably due to interface changes?)

In the browser the blockly interface can be used properly but neither running it nor the code preview screen shows. (It simply stays empty I assume because the JavaScripts fails)

(Using it with JupyterLab 3.16 as indicated by the readme works fine though but being able to use an up to date JupyterLab version would be nice and I didn't find an issue regarding this yet)

When importing the IBlocklyRegistry token in another extension, I'm getting the following error:

Plugin 'jupyterlab-lego-boost:plugin' failed to activate.
Error: No provider for: jupyterlab-blockly/registry.
    at e.resolveRequiredService (jlab_core.ace489612104403d9b5e.js?v=ace489612104403d9b5e:2:1343887)
    at jlab_core.ace489612104403d9b5e.js?v=ace489612104403d9b5e:2:1343444
    at Array.map (<anonymous>)
    at e.activatePlugin (jlab_core.ace489612104403d9b5e.js?v=ace489612104403d9b5e:2:1343418)
    at jlab_core.ace489612104403d9b5e.js?v=ace489612104403d9b5e:2:1344610
    at Array.map (<anonymous>)
    at e.start (jlab_core.ace489612104403d9b5e.js?v=ace489612104403d9b5e:2:1344584)
    at G (7796.25a9965c8e0e6c74a3ed.js?v=25a9965c8e0e6c74a3ed:1:8374)

The code used to import the IBlocklyRegistry token in the extension is (code snippet taken from this PR):

import { IBlocklyRegistry } from 'jupyterlab-blockly';

/**
 * Initialization data for the jupyterlab-lego-boost extension.
 */
const plugin: JupyterFrontEndPlugin<void> = {
  id: 'jupyterlab-lego-boost:plugin',
  autoStart: true,
  requires: [IBlocklyRegistry],
  activate: (app: JupyterFrontEnd, blockly: IBlocklyRegistry) => {
    console.log('JupyterLab extension jupyterlab-lego-boost is activated!');
  }
};

I started getting this error after we made the latest patch release 0.1.1, in order to incorporate this PR #62 which fixes a typo regarding the spelling of the IBlocklyRegistry. As far as I investigated, nothing else was changed regarding the logic of the code. The JupyterLab-Niryo-One extension is using the 0.1.0 version and everything works perfectly, the code base can be seen here.

From the documentation I read it might be a problem from the deduplication of dependencies, but we didn't change anything there in the new release. The package.json file:

I also noticed that when installing the extension locally in my environment, using pip install jupyterlab-blockly, the error is fixed.

NOTE: The IBlocklyRegistry is the class that the JupyterLab-Blockly extension exposes to other plugins. This registry allows other plugins to register new Toolboxes, Blocks and Generators that users can use in the Blockly editor. So, it is essential when creating an extension on top of JupyterLab-Blockly.

It would be great to be able to resize the output area. We could use the Lumino SplitPanel instead of the Panel for the widget and layout.

https://lumino.readthedocs.io/en/latest/api/widgets/classes/splitpanel.html
https://github.com/jupyterlab/lumino/blob/0d47e817ef0b8b7af977bec3fa0e872dbd03a2c1/packages/widgets/src/splitpanel.ts#L32

• Documentation: https://developers.google.com/blockly/guides/overview
• Examples: https://github.com/google/blockly-samples
• Dev Tools: https://blockly-demo.appspot.com/static/demos/blockfactory/index.html

Now that we know it works, we should move the Niryo specifics to another extension keeping Blockly as a generic editor that we can extend for multiple purposes.

We can use the cookiecutte to create a new extension for JupyterLab.

We should move the plugin for Niryo

Something to have in mind, by default the cookiecutter doesn't allow javascript, after creating the new extension, we need to configure it to allow javascript by adding these lines to the tsconfig.json

Blockly supports importing and exporting the workspace as a JSON or XML. To have better integration with JupyterLab, we should create a new DocumentWidget for JSON files to allow right-click -> 'Open with Blockly' in JSON files, at the same time this will allow us to use shortcuts like ctrl+s for saving directly to disk, plus some other functionalities from documents.

Some documentation to get started:

• JupyterLab
• Blockly
• Example documents

To execute the code generated by blockly, we can use kernels. The idea would be to add a button in the toolbar to allow the user to run the code and show the output on another widget.

Some documentation to get started:

• JupyterLab
• Jupyter Client
• Blockly
• Example kernel messaging
• Example kernel output

We can link the document to a kernel, add a small icon in the toolbar to see the status of the kernel, and select a kernel. Once we have a kernel connection, each time the user modifies the code, we can export it and send a completeness message to the kernel to check whether the code could be executed and add a small indicator for the user.
When the user presses the execution button, we can export the code as Python, send it to the kernel with an execution request and show the response in an OutputAreaWidget.

• Completeness message:
  https://jupyter-client.readthedocs.io/en/stable/messaging.html#code-completeness
• Execute message:
  https://jupyter-client.readthedocs.io/en/stable/messaging.html#execute
• A simple way of creating a OutputAreaWidget:
  https://github.com/voila-dashboards/voila-gridstack/blob/b44f697fab531521503dc23d2da180069987e6f3/packages/jupyterlab-gridstack/src/editor/gridstack/model.ts#L334-L346

Create custom blocks to program an Arduino board.

Arduino consists of both a physical programmable circuit board (often referred to as a microcontroller) and a piece of software, or IDE (Integrated Development Environment) that runs on your computer, used to write and upload computer code to the physical board. Arduino boards are able to read inputs - light on a sensor, a finger on a button - and turn it into an output - activating a motor, turning on an LED. You can read more about it here.

It seems to be stuck with the kernel selected upon opening the document.

Since we have two themes available in JupyterLab (light and dark) would be great to support themes in blockly, so when changing to the dark theme, the blockly editor changes to follow the same design.

Some documentation to get started:

• JupyterLab
• Blockly

Hi ,
jupyterlab blockly language is not English blocks...
How can i fix it?
thanks you

Create custom blocks that use functionalities from ipycanvas.

ipycanvas is a lightweight, fast and stable library exposing the browser's Canvas API to IPython. It allows you to draw simple primitives directly from Python like text, lines, polygons, arcs, images etc. You can read more about it here.

It would be nice to set up a JupyterLite deployment to easily try the extension in the browser.

For example using https://github.com/jupyterlite/jupyterlite-sphinx like in the ipycanvas docs: https://ipycanvas.readthedocs.io/en/latest/?badge=latest

Hi,
I installed blockly using 'development install' but the toolkit is not in english language. How do I change that?
Thanks

The link in the docs docs responds with 404.