janet-lang / spork Goto Github PK

Comments are discarded sometimes by fmt/make-tree:

repl:1:> (make-tree "# a comment")
(:top @[])
repl:2:> (make-tree "# a comment\n# another comment")
(:top @[])

Perhaps as long as there is at least one thing which is not a comment nor whitespace this doesn''t happen:

repl:3:> (make-tree "# a comment\n1")
(:top @[(:comment " a comment") (:span "1")])

Having token-aware reflowing in jfmt would be a nice feature. I'm not sure about a specific line length (much debate to be had there), but reflowing manually is tedious and it would be a nice-to-have.

The following bit is from boot.janet:

         (,wait-for-fibers ,chan
           ,(seq [[i body] :pairs bodies]

via: https://github.com/janet-lang/janet/blob/77c52792963ea012e94cdcd8ed97eb2c068f7661/src/boot/boot.janet#L3207-L3209

If boot.janet is passed through jfmt (which IIUC uses fmt/format-file), the result is:

                 (,wait-for-fibers ,chan
                    ,(seq [[i body] :pairs bodies]

Apart from the leading whitespace differing, the second line is shifted to the right by one character (compared to the code from boot.janet).

My expectation was that .chan and ,(seq ... would have been left-aligned, i.e. I expected ,(seq ... to be shifted to the right so that its leading comma would be directly beneath the leading comma for ,chan:

                 (,wait-for-fibers ,chan
                                   ,(seq [[i body] :pairs bodies]

Is the current behavior intentional?

(To give some background, I'm trying to implement some indentation-related code for editors and am trying to understand how jfmt is intended to indent.)

When trying the --library / -l functionality of janet-netrepl:

 -l, --library VALUE                         Load libraries in the repl as with the janet -l flag

I noticed that specifying something for VALUE that lives under JANET_PATH leads to a client connection being ended right after successful establishment.

The following is a demo.

Suppose spork is installed (so spork lives under JANET_PATH and janet-netrepl is available).

Start server:

$ janet-netrepl -l spork
Starting networked repl server on 127.0.0.1, port 9365...

Start client (note the exiting apparent from the second prompt):

$ janet-netrepl -c
$

Observe output for server:

client [127.0.0.1:9365] connected
closing client [127.0.0.1:9365]

Some investigation revealed that (dyn :syspath)'s value was nil within the handler function for spork/netrepl.janet's server function.

The following diff seemed to yield better results here:

diff --git a/spork/netrepl.janet b/spork/netrepl.janet
index 2556bf5..1986183 100644
--- a/spork/netrepl.janet
+++ b/spork/netrepl.janet
@@ -125,11 +125,13 @@
   (default port default-port)
   (eprint "Starting networked repl server on " host ", port " port "...")
   (def name-set @{})
+  (def syspath (dyn :syspath))
   (net/server
     host port
     (fn repl-handler [stream]
 
       # Setup closures and state
+      (setdyn :syspath syspath)
       (var name "<unknown>")
       (var last-flush 0)
       (def outbuf @"")

Not sure if that's a good way to address things, but the "immediate client disconnect" situation described above was resolved.

Probably similar to what we do with *cfun-list* for functions, which will be registered in the environment automatically in the module-entry.

Would you be in favor of such a change?

I noticed that test.janet and temple.janet are not mentioned in init.janet -- I presume this wasn't intentional? @pepe @bakpakin

It appears that json/encode can produce buffers that have null in them in some situations:

(json/encode nil)
# =>
@"null"

(json/encode [1 2 nil 9])
# =>
@"[1,2,null,9]"

However, it's not clear to me whether one can produce something that's supposed to represent an object (note that the examples above involve a string and an array) with a null value in it. That is, something like:

@`{"result":null}`

and not:

@`{"result":"null"}`

At some point, CFiggers' janet-lsp fork had a work-around for this situation:

(defn success-response [id result]
  (if (nil? result)
    (string "{\"id\":" id ",\"result\":null,\"jsonrpc\":\"2.0\"}")
    (json/encode {:jsonrpc "2.0"
                  :id id
                  :result result})))

For reference, here is a link to the beginning of a related discussion on Zulip.

It would be useful if the spork docs (rendered as html by mendoza) were available on or linked to from the Janet website.

Is there any way I can help?

Assoc and assoc-in are analogous to put and put-in without the side effects.

There is first which is sometimes more convenient than (map (fn [x] (x 0)) xs). However, there isn't a second so we have (map first xs) and (map (fn [x] (x 1)) xs)`.

There may be more of them that could be useful for FP that I haven't thought of ATM.

Would it be a good idea to add them as part of spork?

Are you considering the addition of the test suite? If so, may I try to do it?

I will use tester helper from your other libraries, and I will try to follow your test philosophy.

The following test is failing for me on Windows:

Investigating a bit, test2.file seems to end with a CR byte while test.file does not.

As a side note, the test works fine for me on a Linux box.

I've recently created a short macro for profiling various changes, and thought it might be a good addition to spork. While poking around looking for an appropriate place to add it, I found that it already exists: test/timeit

Which is shockingly similar to what I wrote independently:

(defmacro timeit
  ``Similar to `loop`, but outputs performance statistics after completion.``
  [head & body]
  (with-syms [clk cnt elp run]
    ~(do
       (var ,cnt 0)
       (def ,clk (os/clock))
       (loop ,head (++ ,cnt) ,;body)
       (def ,elp (- (os/clock) ,clk))
       (def ,run (/ ,elp ,cnt))
       (cond
         (< ,run 1e-3) (printf "elapsed %fs, %.4gµs/body" ,elp (* ,run 1_000_000))
         (< ,run 1) (printf "elapsed %fs, %.4gms/body" ,elp (* ,run 1_000))
         (printf "elapsed %fs, %.4gs/body" ,elp ,run)))))

The primary difference is that this implementation piggy-backs off of the loop macro, and can give per body averages, where the current implementation executes a single form.

Current implementation:

(def ind (range 2000))
(timeit
  (repeat 5_000_000
    (take -1000 ind)))
# ->
Elapsed time: 2.03609 seconds

Above implementation:

(def ind (range 2000))
(timeit [:repeat 5_000_000]
  (take -1000 ind))
# ->
elapsed 2.025604s, 0.4051µs/body

which I find to be more useful, and could possibly make a good replacement / update / parallel function.

The docstring for indent-2-forms in spork/fmt currently says:

"A list of forms that are control forms and should be indented two spaces."

Would it make sense to add label and prompt?

I wrote the following code because my janet fibers started crashing due to lack of pipes and files. I think garbage collector doesn't close pipes fast enough.

If pipes aren't closed, a long-running process will run out of pipes and files.

If I run the functions below in ev/with-deadline to prevent zombie processes from blocking janet, I still need to kill the zombie processes with os/proc-kill. Otherwise, I see zombie processes abandoned by janet.

(defn slurp
  ``
  It executes args with `os/spawn` and throws an error if the process returns with non-zero exit code.

  If the process exits with zero exit code, this functions returns standard output of the process.

  If there is any error, to prevent zombie processes, the spawned process is killed.

  Regardless of errors, the spawned process is going to be closed for resource control.
  ``
  [& args]
  (with [proc (os/spawn args :xp {:out :pipe})]
    (try
      (let [[out] (ev/gather
                    (ev/read (proc :out) :all)
                    (os/proc-wait proc))]
        out)
      ([err f]
        (try
          (os/proc-kill proc)
          ([_]))
        (propagate err f)))))

(defn slurp-all
  ``
  It executes args with `os/spawn` and returns a struct which has the following keys.

  * `:out` - the standard output of the process
  * `:err` - the standard error of the process
  * `:exit-code` - the exit code of the process`

  If there is any error, to prevent zombie processes, the spawned process is killed.

  Regardless of errors, the spawned process is going to be closed for resource control.
  ``
  [& args]
  (with [proc (os/spawn args :p {:out :pipe :err :pipe})]
    (try
      (let [[out err ec]
            (ev/gather
              (ev/read (proc :out) :all)
              (ev/read (proc :err) :all)
              (os/proc-wait proc))]
        {:out out :err err :exit-code ec})
      ([err f]
        (try
          (os/proc-kill proc)
          ([_]))
        (propagate err f)))))

I'm a bit worried about pipe buffer. If ev/with-deadline detaches the functions above from a zombie process, ev/read may not fully drain the pipe buffer. Does os/proc-kill drain pipe buffer?

misc/print-table is not working for me. I expected it to pretty-print the following table, but it errors out:

repl> (import spork/misc)

repl> (def t @{:a [1 2 3] :b "hi" :c {:x 1 :y 2 :z "hey there you"} :deee-doo-ee "thanks for all the fish"})

repl> (misc/print-table t)
error: expected integer key in range [0, 2), got :x
  in print-table [/usr/local/lib/janet/spork/misc.janet] on line 63, column 25
  in _thunk [repl] (tailcall) on line 11, column 1

I try to use http/request with an URL that contains a non-standard port (8086):

(http/request "POST" "http://testhost:8086/..." {:headers {} :body body})

This seems to send a malformed Host header:

Host: testhost:"8086"

I'm using janet version 1.25.1-release
spork revision is 40ac3a8

The janet(1) man page has:

Ctrl-K          Delete everything after the cursor on the input line.

and that works for janet.

The make-getline function in getline.janet has implementations for many bits of functionality, but not for responding to Ctrl-k.

I tried the following changes:

diff --git a/spork/getline.janet b/spork/getline.janet
index 0378bdf..5858478 100644
--- a/spork/getline.janet
+++ b/spork/getline.janet
@@ -338,6 +338,8 @@
             (kback)
             9 # tab
             (autocomplete)
+            11 # ctrl-k
+            (do (buffer/popn buf (- (length buf) pos)) (refresh))
             12 # ctrl-l
             (do (clear) (refresh))
             13 # enter

In limited testing, it seems to be working here.

Hi, I'm trying to implement a terminal multiplayer game with the rpc module but
I'm having trouble with it, I have a server and N clients, when a client connects to a
server it sends the server his host and port so the server also establishes a connection
with each client, so each client is connected to the server and the server is connected to each client,
I keep the connections in a table.

What I want to do is to lets say, send a message to every client from the server, but when I do so
the server and the first client it tries to communicate with hangs, and sometimes I get this error:

error: could not find method :* for nil, or :r* for 256
in rpc-function [/usr/local/lib/janet/spork/rpc.janet] on line 83, column 23
in game-repl [main.janet] (tailcall) on line 45, column 30
in _thunk [main.janet] (tailcall) on line -1, column -1
in cli-main [boot.janet] on line 3677, column 17

I'm at a loss trying to understand or debug this, hope you can help me out.

Easy repro:

➜ janet -l spork/fmt
Janet 1.13.2-dev-local linux/x64 - '(doc)' for help
repl:1:> (def c "(def a 1)\n(def b 2)\ndef c (+ a b))\n(print c)")
"(def a 1)\n(def b 2)\ndef c (+ a b))\n(print c)"
repl:2:> (format-print c)
(def a 1)
(def b 2)
def c (+ a b)
nil
repl:3:>

As could be seen, the format-print unexpectedly removes the last line and paren on the line before. I think error would be better here.

Having started a server-single instance, I connect with two clients. From client A (a text editor) I define a bunch of functions. We'll take warn-proto-method-shadow as an example. From client B (a janet session running client) we see the following confusing behavior:

[127.0.0.1:9365]:3: warn-proto-method-shadow
<function warn-proto-method-shadow>
[127.0.0.1:9365]:4: warn-<TAB>

We see that the symbol is bound. However, warn-<TAB> immediately completes:

[127.0.0.1:9365]:4: warn-compile

Even though the function is in the current environment, it's not in the list of tab completion options.

Should the calls to error in fmt's PEG be caught by the formatting functions so that rather than a stacktrace to the functions producing the error, the message instead shows the line with the error and a visual indication (perhaps with ^) showing the character that caused the issue?

Example:

(import spork/json)
(def one @{:links @[]})
(def two @{:links @[one]})
(array/push (one :links) two)
(def objects @{:one one :two two})
(printf "%P" objects) #=> @{:one @{:links @[@{:links @[<cycle 1>]}]} :two @{:links @[@{:links @[<cycle 1>]}]}}
(print (json/encode objects)) #=> 'command janet $argv' terminated by signal SIGSEGV (Address boundary error)

Expected behaviour:

This should probably throw a parsing error or something like that

In ae9db3d, it looks like some steps were made toward making it possible for a caller of sh/make-new-file to specify a mode when it opens a file.

However, the mode argument appears to be unused by the file/open call:

Is doc/index.mdz intended to house all the docs for all modules in spork? Are they to be migrated from the readme into doc/index.mdz?

Should the docs be all in one file, or in a separate file for each module?

If in one big file, would be nice to have a little table of contents with links to make it easier to navigate.

Just noticed a file at the top level of this repo named ' (a single quote mark). Maybe a typo?

With 63f656e on a Windows 10 machine, I see an error in the output of jpm test:

running test/suite-zip.janet ...
✘  compress -> decompress round trip
test suite test/suite-zip.janet finished in 0.001 seconds - 0 of 1 tests passed.
non-zero exit code in test/suite-zip.janet: 1

I put some debugging prints before:

(assert (= file-contents (string bytes)) "compress -> decompress round trip")

I got a length of 6000 for file-contents and 0 for bytes. Verified that bytes was a buffer too.

Test passes fine on a Linux box (and the lengths for file-contents and bytes are both 6000).

Whenever I'm connected via nvim/Conjure to a Janet netrepl process, and I somehow cause an error (e.g. by evaluating invalid code), my Conjure gets disconnected.

E.g. when I evaluate the following file multiple times: https://github.com/subsetpark/janet-dtgb/blob/master/test/dtgb.janet
(deftest in Testament throws an error if you define the same test multiple times, as in re-evaluating the same file 2x).

Here's what I see in the netrepl process (w/ valgrind):

valgrind janet -l spork/netrepl -e '(server)'
==1641048== Memcheck, a memory error detector
==1641048== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==1641048== Using Valgrind-3.17.0 and LibVEX; rerun with -h for copyright info
==1641048== Command: janet -l spork/netrepl -e (server)
==1641048==
Starting networked repl server on 127.0.0.1, port 9365...
client Conjure connected
closing client Conjure

Is there any interest in including a prime number generator?

I don't know about generating this one, but I learned recently that some hash function constants are computed from some primes so may be there are other use cases too?

I wonder if @primo-ppcg's gist with a fiber-based one would be a candidate...

Elements of quoted tuples are not left-aligned when passed through format-print:

repl:2:> (spork/fmt/format-print "'(:a\n:b\n:c)")
'(:a
   :b
   :c)

Elements of bracket tuples become aligned:

repl:3:> (spork/fmt/format-print "[:a\n:b\n:c]")
[:a
 :b
 :c]

May be this is intentional?

If so, I would like to understand why. Any hints?

The netrepl server treats messages that begin with the 0xFF byte and 0xFE differently than other messages:

Switching on the first byte of the message was added in commit a3d3de4. The message for that commit reads:

Extend netrepl protocol and server.
This change is backwards compatible and allows a netrepl server to accept out-band-commands which it can execute and send results ack to the client. This allows clients to programatically manipulate the remote environment in separate context from the normal repl, which should allow things like smarter auto completion, etc.

I have tried some experiments but am unable to manipulate the remote environment. When I send the message \xFF(curenv) to see what bindings are available in the current environment, the result is "(true @{:out @\"\" :err @\"\"})".

What I'm trying to do is make it possible to set the line and column number of the parser that is parsing the input from the client. I thought that if I edited the netrepl/server function to define a parser in the repl-hander function body (e.g. (def p (parser/new))) that would put it in scope of the getline-async function body where the out-of-band command is evaluated. Unfortunately, no such luck. When I try to reference the binding from the command (e.g. (parser/where p)), I get an unknown symbol p error.

@bakpakin I don't mean to take up time but are you able to point me in the right direction? What am I doing wrong here?

There seem to be a typo in the function math/check-propability which should be math/check-probability and the documentation of the function math/binominal-distribution which should also be probability instead of propability.

With the merger of #145, I think we have just about everything needed. I've prepared a branch which adds math/factor for the purpose of evaluation.

Using this test script, I get timings like this:

pseudoprimes   1.003s, 47.74µs/body
small integers 0.469s, 4.688µs/body

Which is definitely fast enough to be of use.

One issue that I currently have is that math/gcd does not work with abstract types, although it does appear to be written by hand (and not a direct math.h function), so it could possibly be updated. I also think that the pollard-rho implementation could be cleaned up a bit.

https://github.com/janet-lang/spork/blob/193d7bb96c94609e532b1b898ee9ede005277cfd/spork/cjanet.janet#LL707C75-L707C75

what about void exampleFunction(); ?

- Various Janet utilliy modules.
+ Various Janet utility modules.

😃

Part of the circlet library was a small collection of middleware. We could add this to the HTTP module as well, if only as examples.

What do you think?

The build fails for me with a clean install with following log

From https://github.com/janet-lang/spork
 * branch            HEAD       -> FETCH_HEAD
HEAD is now at ab1e784 Add janet-format binary script.
compiling src/tarray.c to build/src___tarray.static.o...
generating meta file build/spork/tarray.meta.janet...
error: could not open file build/spork/tarray.meta.janet with mode wb
creating native module build/spork/tarray.so...
compiling src/json.c to build/src___json.static.o...
creating native module build/spork/json.so...
generating meta file build/spork/json.meta.janet...
error: build fail
  in pdag [/home/tionis/.local/lib/janet/jpm/dagbuild.janet] (tailcall) on line 79, column 23
  in <anonymous> [/home/tionis/.local/lib/janet/jpm/pm.janet] on line 214, column 9
  in <anonymous> [/home/tionis/.local/lib/janet/jpm/pm.janet] on line 200, column 5
  in bundle-install [/home/tionis/.local/lib/janet/jpm/pm.janet] on line 198, column 3
  in install [/home/tionis/.local/lib/janet/jpm/commands.janet] (tailcall) on line 166, column 20
  in _thunk [/home/tionis/.local/bin/jpm] (tailcall) on line -1, column -1
  in cli-main [boot.janet] on line 3817, column 17

janet -v
1.30.0-local

jpm install spork
From https://github.com/janet-lang/spork

• branch HEAD -> FETCH_HEAD
  HEAD is now at ce3e437 Merge pull request #145 from primo-ppcg/modular-arithmetic
  compiling src/zip.c to build/src___zip.o...
  error: failed to create process
  compiling deps/miniz/miniz.c to build/deps___miniz___miniz.o...
  error: failed to create process
  compiling src/json.c to build/src___json.o...
  error: failed to create process
  compiling src/tarray.c to build/src___tarray.o...
  error: failed to create process
  compiling src/utf8.c to build/src___utf8.o...
  error: failed to create process
  compiling src/rawterm.c to build/src___rawterm.o...
  error: failed to create process
  compiling src/cmath.c to build/src___cmath.o...
  error: failed to create process
  compiling src/crc.c to build/src___crc.o...
  error: failed to create process
  error: build fail
  in pdag [C:/Users/.../Local/Apps/Janet/Library/jpm/dagbuild.janet] (tailcall) on line 79, column 23
  in [C:/Users/.../Local/Apps/Janet/Library/jpm/pm.janet] on line 236, column 9
  in [C:/Users/.../Local/Apps/Janet/Library/jpm/pm.janet] on line 221, column 5
  in bundle-install [C:/Users/.../Local/Apps/Janet/Library/jpm/pm.janet] on line 219, column 3
  in install [C:/Users/.../Local/Apps/Janet/Library/jpm/commands.janet] on line 197, column 20
  in run [C:/Users/.../Local/Apps/Janet/Library/jpm/cli.janet] (tailcall) on line 88, column 9
  in run-main [boot.janet] on line 3883, column 16
  in cli-main [boot.janet] on line 4036, column 17

Some captures lead to errors:

repl:2:> (regex/match `a(bc)` `abc`)
error: match error at line 1, column 2
  in peg/match
  in source [/home/user/bin/janet/lib/janet/spork/regex.janet] on line 133, column 14
  in compile [/home/user/bin/janet/lib/janet/spork/regex.janet] on line 141, column 18
  in match [/home/user/bin/janet/lib/janet/spork/regex.janet] on line 147, column 14
  in _thunk [repl] (tailcall) on line 4, column 1

Some work:

(regex/match `(abc)` `abc`)
# => @["abc"]

This is with: c4d67dd

It looks to me like spork currently contains only single-file modules. However, some modules that may be a good fit for inclusion into spork may contain multiple files, their own tests, docs, etc. It is not clear how one would create a PR for inclusion of modules/libraries of this type. Would it make sense for there to be a contributions doc for this repository, specifically including guidelines for how to weave in these kinds of "larger" libraries?

fmt/format-print appears to have difficulty with code strings such as:

• ( )
• @( )
• [ ]
• @[ ]
• { }
• @{ }

That is, there are no "elements" and there is at least one whitespace character.

An example invocation is:

repl:1:> (import spork/fmt)
# output elided
repl:2:> (fmt/format-print "[ ]")
error: match error at line 1, column 2
  in peg/match
  in make-tree [/home/user/bin/janet/lib/janet/spork/fmt.janet] on line 53, column 9
  in format-print [/home/user/bin/janet/lib/janet/spork/fmt.janet] on line 181, column 3
  in _thunk [repl] (tailcall) on line 2, column 1

This is with 714c216

I already tried httpf on several projects and like the design choices. Yet, I am missing a chance to set the HTTP status code in the handler. Would you accept the PR with a mechanism similar to how response headers are set in the current version with a dynamic?

json/decode on an empty string currently throws an error:

decode error at position 0: unexpected end of source

I think it should instead return nil.

System info:

• Janet version: $ janet -v => 1.24.1-0817e627
• Mendoza version: $ mdz version => 0.0.1
• OS: $ lsb_release -d => Description: Ubuntu 22.04.1 LTS

Steps to reproduce:

1. Clone janet-lang/spork: $ gh repo clone janet-lang/spork
2. cd into cloned repo: $ cd spork
3. Attempt to build using Mendoza: $ mdz build

Results:

Removing directory site...
Unloading cached modules...
Parsing content doc/argparse.mdz as mendoza markup
Loading syntax /usr/local/lib/janet/mendoza/syntax/janet.syntax.janet
Parsing content doc/formatting.mdz as mendoza markup
Parsing content doc/generators.mdz as mendoza markup
Parsing content doc/htmlgen.mdz as mendoza markup
Parsing content doc/http.mdz as mendoza markup
Parsing content doc/index.mdz as mendoza markup
Parsing content doc/message-protocol.mdz as mendoza markup
Parsing content doc/misc.mdz as mendoza markup
Parsing content doc/networked-repl.mdz as mendoza markup
Parsing content doc/path.mdz as mendoza markup
Parsing content doc/regex.mdz as mendoza markup
Parsing content doc/rpc-protocol.mdz as mendoza markup
Parsing content doc/temple.mdz as mendoza markup
Parsing content doc/test.mdz as mendoza markup
Writing HTML to site/http.html
Writing HTML to site/index.html
Writing HTML to site/formatting.html
Writing HTML to site/message-protocol.html
Writing HTML to site/networked-repl.html
Writing HTML to site/rpc-protocol.html
Writing HTML to site/path.html
Writing HTML to site/argparse.html
error: match error at line 10, column 1
  in peg/match [src/core/peg.c] on line 1669
  in render [/usr/local/lib/janet/mendoza/render.janet] on line 63, column 23
  in render [/usr/local/lib/janet/mendoza/render.janet] on line 67, column 11
  in render [/usr/local/lib/janet/mendoza/render.janet] on line 43, column 34
  in _mendoza-template [/usr/local/lib/janet/mendoza/templates/mdzdoc/main.html] on line 6, column 369
  in render [/usr/local/lib/janet/mendoza/render.janet] (tailcall) on line 66, column 11
  in render-page [/usr/local/lib/janet/mendoza/init.janet] on line 139, column 7
  in build [/usr/local/lib/janet/mendoza/init.janet] (tailcall) on line 154, column 5

After installing spork via sudo jpm install https://github.com/janet-lang/spork.git and then adding (import spork/netrepl) to my testfile I get

janet horace.janet 
error: could not find module spork/rawterm:
    /usr/local/lib/janet/spork/rawterm.jimage
    /usr/local/lib/janet/spork/rawterm.janet
    /usr/local/lib/janet/spork/rawterm/init.janet
    /usr/local/lib/janet/spork/rawterm.so
  in require-1 [boot.janet] on line 2901, column 20
  in import* [boot.janet] on line 2940, column 15
  in _thunk [/usr/local/lib/janet/spork/getline.janet] (tailcall) on line 10, column 1
  in dofile [boot.janet] (tailcall) on line 2878, column 7
  in source-loader [boot.janet] on line 2889, column 15
  in require-1 [boot.janet] on line 2909, column 18
  in import* [boot.janet] (tailcall) on line 2940, column 15
  in dofile [boot.janet] (tailcall) on line 2878, column 7
  in source-loader [boot.janet] on line 2889, column 15
  in require-1 [boot.janet] on line 2909, column 18
  in import* [boot.janet] on line 2940, column 15
  in _thunk [horace.janet] (tailcall) on line 3, column 1

It looks like there was a rawterm.a build, but not a rawterm.so...

The make-recv function in msg.janet creates and returns a receiver function. The receiver function is created such that, as part of its last form, calls unpack:

  (fn receiver []
    (buffer/clear buf)
    (if-not (:chunk stream 4 buf) (break))
    (def [b0 b1 b2 b3] buf)
    (def len (+ b0 (* b1 0x100) (* b2 0x10000) (* b3 0x1000000)))
    (buffer/clear buf)
    (if-not (:chunk stream len buf) (break))
    (unpack (string buf)))

make-recv takes an optional argument unpack, which if unspecified, defaults to the function string:

(defn make-recv
  "Get a function that, when invoked, gets the next message from a readable stream.
  Provide an optional unpack function that will parse the received buffer."
  [stream &opt unpack]
  (def buf @"")
  (default unpack string)

It seems that, if make-recv doesn't receive an explicit argument for unpack, unpack would have its default value of string and thus the last form of the receiver function would end up being:

(string (string buf))

Is the inner call to string necessary?

Perhaps it's ok for the last form to be instead:

(unpack buf)

Or is it important that unpack not be passed the buffer?

I'd like to be able to run an infinite loop on the main thread so I can run a gui program, but this of course blocks the main process. Is there a way to manually tell netrepl to poll for requests so I can use this main loop? In Lisp with swank I can use swank::handle-requests in an infinite loop to keep things working. So is there something like netrepl/handle-requests that I can use in the following way:

(while true
    (netrepl/handle-requests)
    (my-update-function))

how to reproduce

• go in this repo
• run in Janet REPL:

(import spork/mdz)
(mdz/markup (slurp "doc/data.mdz"))

now, you should see this output

@{:current-file "<anonymous>" :front-matter { :author "Caleb Figgers" :license "MIT" :order 1 :template "mdzdoc/main.html" :title "data"} :markup-dom @[ "\n" (:a {"href" "https://clojure.org/"} @[@["Clojure"]]) (:p "contains a very useful core library (or \"namespace\" in Clojure parlance) called "

the space between Clojure is and contains is gone!

Also, the :a tag should be inside :p tag, not before it.

The call to coerce-to-env in netrepl's server function can fail because the env argument to coerce-to-env is invoked if it is a function.

janet-netrepl specifies a function value for env in a couple of places and that function can fail because it can call require and/or dofile with values specified by a user at the command line.

In the case of a failure-triggering invocation via janet-netrepl, the client connection is closed after establishment and there is no error feedback regarding the failure provided on the server end.

The following is a demonstration of the described situation (similar to that in #160).

Suppose spork is installed (so janet-netrepl is available).

Start server, requesting non-installed / unavailable library:

$ janet-netrepl -l not-spork
Starting networked repl server on 127.0.0.1, port 9365...

Start client (note the exiting apparent from the second prompt):

$ janet-netrepl -c
$

Observe output for server:

client [127.0.0.1:9365] connected
closing client [127.0.0.1:9365]

With the following diff:

diff --git a/spork/netrepl.janet b/spork/netrepl.janet
index 2556bf5..f72b41c 100644
--- a/spork/netrepl.janet
+++ b/spork/netrepl.janet
@@ -209,7 +209,11 @@
           (set name (string name (gensym))))
         (put name-set name true)
         (eprint "client " name " connected")
-        (def e (coerce-to-env env name stream))
+        (def e
+          (try (coerce-to-env env name stream)
+            ([err fib]
+              (eprint err)
+              (debug/stacktrace fib "coerce-to-env failed" ""))))
         (def p (parser/new))
         # Print welcome message
         (when (and welcome-msg auto-flush)

Server output looks like:

$ janet-netrepl -l not-spork
Starting networked repl server on 127.0.0.1, port 9365...
client [127.0.0.1:9365] connected
could not find module not-spork:
    /not-spork.jimage
    /not-spork.janet
    /not-spork/init.janet
    /not-spork.so
error: coerce-to-env failed
  in require-1 [boot.janet] (tailcall) on line 3024, column 20
  in env-make [/home/user/.local/bin/janet-netrepl] (tailcall) on line 82, column 41
closing client [127.0.0.1:9365]

That seems a bit better.

Note that the values:

    /not-spork.jimage
    /not-spork.janet
    /not-spork/init.janet
    /not-spork.so

start with / with no "intermediate" path because of #160. (The values might change if #160 is addressed, but I don't think that is significant for this issue.)

Paren arrays appear to be converted to bracket arrays:

Perhaps there is no obvious drawback. It also seems straight-forward to preserve the distinction.

Any reason for the current behavior?