Do the following:
]load HttpCommand

⊢⎕FIX {⍵⊆⍨~⍵∊⎕UCS 10 13 65279}(HttpCommand.Get 'https://raw.githubusercontent.com/Dyalog/bb/master/spiro/spiro.dyalog').Data

≢h←spiro.(html spiro 102 77 40) ⍝ generate a bunch (>1MB) of HTML

3500⌶h

Results in
negative buffersize in recv
and the kernel is hosed

We need something as simple as ]plot data

∇and ]dinput cells need to be coloured like functions (except the first line)

E.g. the code cell

'ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ'
'ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ'
'ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ'
'ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ'

fails.

1. readme.txt tells you to install Anaconda, but this is not necessary. Any Python 3 + Jupyter installation should work.

2. You should be able to install with just "pip install ." (or similar invocation of standard Python package installer tools).

I am wanting to re-install this kernel into my new miniconda3 python 3.9 environment but the install seems to assume a different python setup.

Is there a script for installing into miniconda3 python 3.9 please?

If you run the kernel in a non-UTF-8 locale, e.g. by setting LANG=C, it (sometimes) fails to output APL characters with an error message like: 'ascii' codec can't encode characters in position 39-55: ordinal not in range(128)

Currently awkwardly provided by ∇ editor, but we need something more robust (and for dfns, neat).

Try e.g. !5. Leading exclamation point has probably been reserved for special commands.

For example {⎕←⍵}&¨⍳3 vs {⎕←⍵}¨⍳3

When prompt type is set to 1 (RECV ["SetPromptType",{"type":1}]), the kernel stops the loop checking for output. However, the RIDE commands to AppendSessionOutput come after this.

Maybe something like:

]fix
:namespace foo
a←⍳10
:endnamespace

This causes boxed output to look broken.

Create a markdown cell using backticks and stars and quotes, e.g.

`backticks` and *stars* and quote's

Save and reopen the notebook. Edit that cell. It is now syntax coloured as APL instead of styled according to markdown conventions.

Kernel forces size of svg images to 120×80. This makes using SharpPlot in a jupyter notebook impractical, as the kernel also doesn't support %%html magics to override an image size in css.

https://github.com/Dyalog/dyalog-jupyter-kernel/blob/master/dyalog_kernel/kernel.py#L94-L98=

Assuming Dyalog APL 18.0 is now stable, the kernel needs to be updated in a few ways:

• Include the latest 18.0 interpreter in the Docker image
• In the custom CodeMirror highlighter, add support for built-ins added to 18.0 (namely ⍥, ⎕C, ⎕DT) and other things as necessary

Executing the following in a cell does not produce the expected result:

{⎕←'Started waiting',⍵,'s' ⋄ ⎕←'Elapsed:',(⎕DL ⍵),'s'}&¨0.6 1.2 1.8 ⋄ ⎕DL 1 ⋄ ⎕←'    Unrelated stuff'

In order for it to work, you need to sync explicitly:

 ⎕TSYNC {⎕←'Started waiting',⍵,'s' ⋄ ⎕←'Elapsed:',(⎕DL ⍵),'s'}&¨0.6 1.2 1.8 ⋄ ⎕DL 1 ⋄ ⎕←'    Unrelated stuff'

My expectation was that a cell's execution should await any threads, to align with the behaviour of the session.

It should be in the background/headless like on *nix

It should probably send a strong interrupt to avoid ending in a suspended state.

WIBNI you could do:

var←⍞

or

var←⎕

⎕←⍞

and get similar behaviour to python's

var = input(prompt)

Can leave running interpreters hanging around

The latent expression in dyalog_kernal\init.dws is:
hereDir←⊃⎕NPARTS⎕WSID⋄(⍎⊃2⎕FIX'file://',hereDir,'init.dyalog')hereDir

(⊃2⎕FIX'file://',hereDir,'init.dyalog')hereDir returns:

I C:\Users\james\Dropbox\Study\dyalog-jupyter-kernel\dyalog_kernel\

Should this not be (2⎕FIX'file://',hereDir,'init.dyalog')hereDir?

Init C:\Users\james\Dropbox\Study\dyalog-jupyter-kernel\dyalog_kernel\

By executing the first line I get back when running the notebook:
⍎VALUE ERROR I

What is the purpose of the ⊃?

The ∇ editor is currently used to define tradfns, but it is actually problematic because if the user tries to re-execute the cell, the function instead gets amended with additional lines (or a definition error can occur). And if something goes wrong or there is no closing ∇, the user can end up stranded in the ∇ editor with no good visual indication or obvious way out. We may need interpreter support for disabling the ∇ editor, but note that this would also prohibit tradfns and multiline dfns.

On Linux, lanuching an xterm to run the interpreter in is naff. It should be possible to run the interpreter headless, like RIDE does.

)copy, for example, has the ) highlighted red as an error

On Windows, it tries to find dyalog using the PATh environment variable. However, Dyalog's installers don't amend PATH any more. It should probably look in HKEY_CURRENT_USER\Software\Dyalog for all matches of \d\d.\d Unicode and take the highest-numbered one.

If a line begins with \s+∇ ignore it and any following lines in the cell and error with THE ∇ EDITOR IS NOT SUPPORTED IN JUPYTER NOTEBOOK

From an internal note:

Currently the Jupyter environment is protected from the user's environment (environment variables or .dcfg etc.) by init.dws. In the future we might change to a script based launcher but need to make sure the Jupyter environment starts with all defaults, to ensure that notebooks are run in a default environment.

This issue is to verify if the above is true and, if so, consider addressing it

I would find that ability useful, as it would enable me to inject documentation close to the code. Possibly this goes a bit against the idea of one cell being a complete executeable unit, I hope this is not a must. It would also be more in the often referred-to spirit of Knuth's Literate Programming 😉

Possibly one could end a partial-defiition with a certain prefix (i.e. "...∇") and it would then have to be continued with "∇..." or "...∇{nameOfFn}" in a subsequent code-cell).

On Linux the kernel tries to search for the newest installed dyalog executable. I think it would be better just to invoke "dyalog", as the package installation mechanism makes sure that that will run the latest interpreter by default, and more importantly there are ways for the user to override it if he really wants to.

For example, this tells me that I have 16.0 and 17.0 installed, and "dyalog" currently runs 17.0:

$ update-alternatives --display dyalog
dyalog - auto mode
  link best version is /opt/mdyalog/17.0/64/unicode/mapl
  link currently points to /opt/mdyalog/17.0/64/unicode/mapl
  link dyalog is /usr/bin/dyalog
/opt/mdyalog/16.0/64/unicode/mapl - priority 160
/opt/mdyalog/17.0/64/unicode/mapl - priority 170

E.g. user writes

∇foo x
 ÷x
∇
foo 0

will suspend with an error. We need to catch this and put the system back in the session without a stack.

IIRC the notebooks use "the github-flavor of markdown". So, the GitHub-flavoured way to say "thumbs up" is :+1: which is rendered as 👍
I tried doing that in a markdown-cell and it just returned the original text. Do I (as a notebook-author) have to do anything else to use them?

It appears to colour using both APL rules and Python rules

E.g. ! will get a cm-apl-factorial class but both ⍝ and # get cm-comment, the latter being Python's comment symbol.

Once the kernel is hosed, the associated interpreter task is left as an orphan. Parentless, wandering the streets, lonely... it's very sad.

If I define and display a character vector
h←'<h1>Hi</h1>'
h
in an input cell I expect to see
<h1>Hi</h1>
in the output cell, not

Hi

The user should use ]html or 3500⌶ to render HTML

The APL language bar and input method at https://abrudz.github.io/lb/apl works great with Jupyter notebooks. Maybe we should include a link to it?

⎕ and ⍞ leave the system in a strange state waiting for input. We can either auto-respond with dummy answers or (more complicated) redirect to JavaScript prompt().

Jupyter users coming form a Python background may well click on 'restart kernel', which causes a core dump.

After that you have to restart jupyter, and may have lost some work.

The expression is executed, as can be observed directly in the APL session, but no result is shown. Try e.g. ⍳3.

There are no issues when launched from Anaconda Prompt.

… so that identical code will work in a session (although through different means).

The kernel seems to be very glitchy with 18.2. MacOS 12.0.1, latest dyalog kernel, python 3.9.

Bad file descriptor, tcp recv, restart loop. Crash log: https://gist.github.com/xpqz/11f99b6788f9d8a0b5b9b76793f4a12e

[W 09:27:38.093 NotebookApp] Notebook learnapl/contents/trees.ipynb is not trusted
ERROR:asyncio:Exception in callback <TaskWakeupMethWrapper object at 0x7fba843b3fa0>(<Future finis...C: 1\r\n\r\n'>)
handle: <Handle <TaskWakeupMethWrapper object at 0x7fba843b3fa0>(<Future finis...C: 1\r\n\r\n'>)>
Traceback (most recent call last):
  File "/opt/anaconda3/envs/py39/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
RuntimeError: Cannot enter into task <Task pending name='Task-79' coro=<HTTP1ServerConnection._server_request_loop() running at /opt/anaconda3/envs/py39/lib/python3.9/site-packages/tornado/http1connection.py:823> wait_for=<Future finished result=b'GET /api/co...PC: 1\r\n\r\n'> cb=[IOLoop.add_future.<locals>.<lambda>() at /opt/anaconda3/envs/py39/lib/python3.9/site-packages/tornado/ioloop.py:688]> while another task <Task pending name='Task-69' coro=<MappingKernelManager.start_kernel() running at /opt/anaconda3/envs/py39/lib/python3.9/site-packages/notebook/services/kernels/kernelmanager.py:176> cb=[IOLoop.add_future.<locals>.<lambda>() at /opt/anaconda3/envs/py39/lib/python3.9/site-packages/tornado/ioloop.py:695]> is being executed.
[I 09:27:39.057 NotebookApp] Kernel started: fa4c30f4-2df5-449f-b0cd-7728e931e26a, name: dyalog-kernel
[W 09:28:00.162 NotebookApp] Replacing stale connection: fa4c30f4-2df5-449f-b0cd-7728e931e26a:bb8069a0bdb045c19d0c0feb8a3b1120
Traceback (most recent call last):
  File "/opt/anaconda3/envs/py39/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/anaconda3/envs/py39/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/__main__.py", line 4, in <module>
    IPKernelApp.launch_instance(kernel_class=DyalogKernel)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/traitlets/config/application.py", line 845, in launch_instance
    app.initialize(argv)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/traitlets/config/application.py", line 88, in inner
    return method(app, *args, **kwargs)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 647, in initialize
    self.init_kernel()
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 499, in init_kernel
    kernel = kernel_factory(parent=self, session=self.session,
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/traitlets/config/configurable.py", line 540, in instance
    inst = cls(*args, **kwargs)
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 264, in __init__
    self.dyalog_ride_connect()
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 160, in dyalog_ride_connect
    self.ride_receive_wait()
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 313, in ride_receive_wait
    if self.ride_receive():
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 280, in ride_receive
    head = self.recv_all(8)
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 269, in recv_all
    part = self.dyalogTCP.recv(msg_len)
OSError: [Errno 9] Bad file descriptor
[I 09:28:12.056 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
[W 09:28:22.177 NotebookApp] Replacing stale connection: fa4c30f4-2df5-449f-b0cd-7728e931e26a:bb8069a0bdb045c19d0c0feb8a3b1120
[W 09:28:39.127 NotebookApp] Timeout waiting for kernel_info reply from fa4c30f4-2df5-449f-b0cd-7728e931e26a
[I 09:28:39.147 NotebookApp] Starting buffering for fa4c30f4-2df5-449f-b0cd-7728e931e26a:bb8069a0bdb045c19d0c0feb8a3b1120
[I 09:28:39.152 NotebookApp] Restoring connection for fa4c30f4-2df5-449f-b0cd-7728e931e26a:bb8069a0bdb045c19d0c0feb8a3b1120
Traceback (most recent call last):
  File "/opt/anaconda3/envs/py39/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/anaconda3/envs/py39/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/__main__.py", line 4, in <module>
    IPKernelApp.launch_instance(kernel_class=DyalogKernel)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/traitlets/config/application.py", line 845, in launch_instance
    app.initialize(argv)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/traitlets/config/application.py", line 88, in inner
    return method(app, *args, **kwargs)
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 647, in initialize
    self.init_kernel()
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 499, in init_kernel
    kernel = kernel_factory(parent=self, session=self.session,
  File "/opt/anaconda3/envs/py39/lib/python3.9/site-packages/traitlets/config/configurable.py", line 540, in instance
    inst = cls(*args, **kwargs)
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 264, in __init__
    self.dyalog_ride_connect()
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 160, in dyalog_ride_connect
    self.ride_receive_wait()
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 313, in ride_receive_wait
    if self.ride_receive():
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 280, in ride_receive
    head = self.recv_all(8)
  File "/opt/anaconda3/envs/py39/lib/python3.9/dyalog_kernel/kernel.py", line 269, in recv_all
    part = self.dyalogTCP.recv(msg_len)
OSError: [Errno 9] Bad file descriptor
[I 09:28:45.088 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
WARNING:root:kernel fa4c30f4-2df5-449f-b0cd-7728e931e26a restarted

Would be nice to choose Dyalog version or kernel-as-Dyalog-version when multiple versions of Dyalog are installed.

]html and ]plot should be included with the kernel so they may be used Dyalog versions ≤17.0

JupyterLab has great built-in support for interactive debugging, if the kernel supports it:

https://jupyterlab.readthedocs.io/en/stable/user/debugger.html

A kernel that wants to support this needs to implement jupyter's debugger protocol:

https://jupyter-client.readthedocs.io/en/latest/messaging.html#debug-request

..which is based on Microsoft's DAP: https://microsoft.github.io/debug-adapter-protocol/

This work (if completed) would potentially enable debugging also in VS Code, Emacs and vim, too (DAP-based).

Note that support for the DAP is proof-of-concepted here: https://github.com/tiamatica/vscode-apl-debug

Consider supporting some of the more useful jupyter "magic" escapes.

A full list: https://ipython.readthedocs.io/en/stable/interactive/magics.html#cell-magics

It would be especially useful to be able to use %%capture (to suppress cell output) and %%html (to be able to inject custom css).

I think it is missing the ability to use the next available port. When port 4502 is already in use the kernel fails to connect.

If one loads HttpCommand and then uses HttpCommand.Get to Get some URL, the interpreter will crash when the kernel is shutdown. This happens on my Windows 10 machine.

Minimal steps to reproduce the issue:

1. open a new Dyalog APL notebook;
2. execute ]load HttpCommand;
3. execute HttpCommand.Get 'https://google.com';
4. shutdown the kernel in any way.
   • for example, by using the "Kernel" > "Shutdown" menu, or
   • by interrupting with Ctrl+C the notebook server.

The aplcore that was generated is attached to this issue.

aplcore.zip