imjoy-team / imjoy Goto Github PK

This issue is targeting the ImJoy-Engine repo.

• https://github.com/python/black
• The process will look like this:
  1. developer adds some code.
  2. developer runs black on code.
  3. developer commits code
  4. (optional pre-commit hook that runs black)
  5. developer makes a PR.
  6. CI runs black and checks that no changes were needed. If changes were needed, exits with 1.

Todo

• Add black code formatter and format repo code.
• Add CI check for black to keep code black.

The aiohttp.streamer decorator has been deprecated. We should use async generator instead. See https://docs.aiohttp.org/en/stable/client_quickstart.html#streaming-uploads.

https://github.com/oeway/ImJoy-Engine/blob/6e96c1765cb6ab91c1d39a5c21bff729148e2945/imjoy/imjoyPluginEngine.py#L1168-L1178

Outline

• https://prettier.io/
• We add it to package.json.
• We should check and configure compatibility with eslint.
  • https://prettier.io/docs/en/integrating-with-linters.html#disable-formatting-rules
• We provide an optional pre-commit-hook.
• There are editor plugins for most editors.
• We add CI check.
• We provide a script to easily run prettier on the correct files locally, as we do in CI.
  • Commandline will be something like this:

  :~/Dev/ImJoy/web$ prettier --write "{src,tests}/**/**(.js|.vue)"
  

  • Replace --write with --check for CI checking.
  • Make sure to exclude plugin html which includes python.

Todo

• Add prettier to package.json.
• Configure eslint compatibility.
• Add script to format and check format.
• Add CI check.
• Format all relevant files.

Save a collection of user defined plugin repositories and allow switching between them easily.

For some reason, api.showFileDialog doesn't work properly in window plugin.

Right now, we use plugin file parser is for vuejs, and it enforces the rule that there can only be one root element in the template, as a result, for window plugin in ImJoy, we have to also use a single root element.

The following won't work:

<window>
  <p> line 1</p>
  <p> line 2</p>
</window>

Instead, you should use a

<window>
  <div>
    <p> line 1</p>
    <p> line 2</p>
  </div>
</window>

We use the field api_version in <config> to make sure the api functions and the way plugin is parsed can be upgraded or warned when a certain api function is deprecated or changed.

The core of ImJoy will read the api_version of the plugin and use that version to load plugin, and this essentially give backward compatibility but still allow us to evolve the api.

To ensure quicker trasisson to new api, it would be better to do the api check and warn the developer when the api version do not match the master api version. This will make sure the api are upgraded.

Would be nice to be able to share a plugin directly from the plugin menu and plugin repository list.

Related to #2 , would be really useful to have offline support through service worker without using the plugin engine.

Not only the html/js/css files from the main app, but also those files set as requirements for every plugins needs to be cached in the service worker.

requirements and env are currently used for Python Plugins, however, these two interfaces can be also used by javascript plugins.

• requirements can be used for defining the required js dependencies, they can be listed as an array or urls and downloaded into the browser database, such that javascript plugins can run offline.

• env can be used to define the supported browsers, by matching agent strings or just with browser name. This is essential to tell when a plugin is not tested on mobile but ran on mobile device, ImJoy can decide whether to display such plugins or give a warning to the user.

This task is part of https://github.com/oeway/ImJoy-Engine.

• Static code analysis catches bugs in CI or even in the developer's editor. We should add flake8 and pylint to facilitate this.
• Fix lint errors and cherry-pick them separately into PRs, first.
• Add flake8, pydocstyle and pylint to lint requirements file.
• Run flake8, pydocstyle and pylint in tox lint env.
• Add config files with needed config and exceptions.

Currently, the unit test based on Karma does not work. We will need unit test for longer term development.

After upgrading the engine to v0.7.65, a plugin failed:

<config lang="json">
{
  "name": "xxxxxxx",
  "type": "native-python",
  "version": "0.3.3",
  "api_version": "0.1.3",
  "description": "[TODO: describe this plugin with one sentence.]",
  "tags": ["CPU"],
  "ui": "",
  "inputs": null,
  "outputs": null,
  "flags": ["allow-detach", "single-instance"],
  "icon": "extension",
  "env": "",
  "requirements": "",
  "cmd": "python",
  "dependencies": []
}
</config>

• Better visual separation between different plugins in plugin menu
• Allow to change default font size of ImJoy
• Allow plugin window to be resized

Would be nice to have a docker image and some scripts to run unit test for plugins, at least for those published on the official plugin repository. It would help us to test not only the plugins itself, but also verfy the new upgrades of ImJoy whether it break all the plugins.

Would be nice to support math equations and code highlighting in the docs window for each plugin.

It would be very useful to list all the child process in the plugin engine, and also provide the capability to kill them.

With webassembly, it's now possible to run python plugins without the plugin engine. Supporting python plugins which does not rely on the plugin engine would essentially free the users from installing the plugin engine and increase the security level. Python plugins running in the web browser should be more reproducible than those native python plugins, since the browser provide a more unified environment.

• tox
• pytest

To enhance the security for the connection token, we need to encrypt the connection token before storing it.

We can start to look into the bcrypt library: https://github.com/pyca/bcrypt/#password-hashing

Currently, new line \n is not rendered in the dialog, we should replace it to <br> in order to generate a new line in the dialog.

I opened https://imjoy.io/#/app and got this message at the status bar

Otherwise, the webpage seems functional.

Would be nice to have an option to open a plugin directly in full screen on start-up. Could be useful for larger interfaces.

Should we provide an internal caching mechanism to cache results of api calls? Especially when accessing the api functions remotely, the call can be expensive, it would be nice to optionally make an api function result being cached.

We can basically generate a signature with its parameters when encoding the arguments and store the recent results inside the browser. The missing part is:

1. How to mark cachable functions? We need a way to specify which function should be cached, maybe when export the api functions, or add an api function to allow dynamically enable and disable caching
2. How do we store the cache? An naive implementation would be use an object and store the result in memory.
3. How to manage the life cycle of the cache, and the overall cache size, when to make it invalid?

Importantly, the caching should be optional and disabled by default.

@MartinHjelmare what's your thought on this?

Currently, one plugin depend on an other plugin can be illustrated with the field dependencies in <config> block. However, it does not handle cross-repository dependencies.

Plugins can described with the following format: REPO_URL@COMMIT_HASH_TAG:PLUGIN_NAME@TAG. The REPO_URL and TAG are optional:

1. if no REPO_URL specified, use the same repository url as the one is referring to. : should be remove if no REPO_URL specified.
2. if no tag specified, use the default tag or without a tag. @ should be remove if no TAG specified.

When loading ImJoy, it would be nice to compare the versions of all the installed plugins and notify the user if updates for plugins are available.

Since tests are important to verify and guarantee whether a certain plugin is functioning, or on the other hand check if changes in ImJoy breaks certain plugins. Right now there is no way to do it automatically, but this could be solved by introducing a test function to the plugin interface.

Here is what I propose:

1. each plugin should (must?) define an api function called test() along with setup() in the plugin class.
2. the test() function should accept no argument except self for python, and it should bootstrap itself, generate/simulate/download data, run other functions defined in the plugin, verify the result and throw error (with error message).
3. add a test button on the code editor toolbar to allow developers quickly run this test function during developing.
4. setup Travis to do automatic test all the plugins hosted on oeway/ImJoy-Plugins and oeway/ImJoy-Demo-Plugins with the latest version of ImJoy.

Would be good to have an option for the user to choose when installing a plugin, depends on the option chose by the user, the plugin can run in different modes and have different dependencies.

For example, for a deep learning based plugin, one may want to have two type of environments: GPU and CPU.

For doing that, the developer can define the env as a dictionary: {"cpu": "conda create -n seg-cpu python=3.6 tensorflow", "gpu": "conda create -n seg-cpu python=3.6 tensorflow-gpu"} or an array: ["gpu", "cpu"]. The same for dependencies.

The selected env should be exposed through api such that the plugin code could access it and decide what to do in different modes.

Since recent changes for supporting unit test, the Monaco editor becomes buggy. At least two problems are identified:

1. reported by @muellerflorian "cmd-Z is doing strange things sometimes. it deletes things that were not affected, moves stuff around and pastes fragments are different locations."
2. Ctrl+V does not work on windows (@muellerflorian ) and Linux ( @oeway ).

Currently when closing the browser, the close event is not passed to the plugins, and it would make more sense to issue this event.

One thing need to be decide for resumable plugins is how do we differentiate that the user want to actually restart the plugin or just detach tempoarary. Perhaps we can pass this to the onClose handler functions, and allow the plugins to decide what to do.

Would be nice to support matplotlib natively, the goal would be show a matplotlib window in Imjoy when calling plt.show().

This will give an idea on how to implement such a backend:
https://github.com/iodide-project/pyodide/blob/master/packages/matplotlib/src/wasm_backend.py

ImJoy support creating virtual env through conda, which makes plugins work with their own env without interfere each other. However, on the other side, it diverge the dependencies of the plugins, and since each virtual env have a significant storage footprint, it takes too much storage space when the user install many different plugins which create their own env.

One way to improve is to propose a default and recommended virtual env, and we recommended all new plugins to develop and test in this default env instead of creating a new one. That would be benificial for the user to have a minimal installation in terms of size.

If only for this reason, we could just recommend to use the latest anaconda environment as the default env.

Another thing to consider is to have a virtual env compatible with the web python plugins, since for web python plugins, we can only choose those libraries provided by pyodide, we can therefore make the default env inline with pyodide.

When opening the + plugins dialog, the entire dialog disappeared when while scrolling in FireFox, other browsers(Chrome, Safari) do not have this issue.

Reported by @muellerflorian , it can be reproduced with the following steps:

1. create a webworker plugin
2. save and run with:

class ImJoyPlugin {
  async setup() {
  }
  async run(my) {
    console.log('running in the plugin ', my)
    await api.setConfig('a', 5)
    const a = await api.getConfig('a')
    console.log(a)
  }
}

3. when clicking the plugin menu, it output null which should be 5

Right now, if ImJoy.io is opened in edge browser, the loading animation will still display.

Currently, if new plugins are installed, files needs to be reloaded in order to get their matched loader updated.

Currently, there are two different ways to handle loading/saving files for plugins with or without the plugin engine. For Javascript plugins, file can use browser file dialog to load file and trigger downloads for exporting files. There is no way to persist files in javascript plugins without user interaction. So it would be nice to have a virtual file system which can be fully accessed from javascript.

For Python plugins, files can be directly accessed from the file system. When file urls are generated from the file system, other plugins have to handle differently for files from url or a file object. A virtual file system could potentially unify these different types of files.

Among those different solutions, BrowserFS looks promising for ImJoy.

Currently if I remove a file from the file engine, the internal variable still hold the selected file. I can for example trigger downloads but it will result in error.

When creating a new workspace, a JavaScript error occurs.

It would be great to be able to connect to Github and store plugins directly to a Github repo or Gist.

How to reproduce: install a web python plugin and then reload for 2~3 times, the memory usage of ImJoy.io continuous increases and then crash.

Possibly because the pyodide enviroment is not released correctly when the plugin iframe is closed, and there is a memory leak in Chrome.

For the same plugin, firefox do not seem to crash.

It would be helpful to see the engine url in the plugin log, this would be helpful to debug when using multiple engines.

This issue belongs to the ImJoy-Engine repo.

https://pypi.org/project/imjoy/

We should try upgrading:

• setuptools
• twine
• wheel

When saving/editing plugins with single-instance and allow-detach flags, the python process will not restart and this cause problems for debugging these plugins.

As a workaround for now, once need to terminate the process manually through the engine process list.

We have been thinking about extending the support to Fiji/ImageJ, to take advantage of the current advancement of the Fiji/ImageJ world. On the other hand, we will be able to add web interface to Fiji/ImageJ scripts easily.

Specifically, the goal is to support a new ImJoy plugin type written in any/most of the supported Fiji scripting languages, these plugins will be able to interact with ImJoy and other plugins through the api object.

Implementing the Fiji/ImageJ backend will be similar as the ImJoy Python Engine. The easiest way to support ImageJ backend is to port this worker template file to ImageJ ( as javaWorkerTemplate.py in Jython). We also need to add socketio client to imagej.

For each ImJoy plugin with type=imagej, we will launch a subprocess from the plugin engine. It will be just a command like ./ImageJ-linux64 -Xss2m -- /path/to/javaWorkerTemplate.py.

Both from user/administrator space.

Currently, when large objects are returned within a webworker plugin, the entire app crashes.

This can be improved by passing a list of buffers in the PostMessage call: PostMesage(aMessage, transferList). Like that, large object will just transfers ownership between objects without duplicating the object.