fare / cl-launch Goto Github PK

If I try running this following command cl-launch -l ecl -o ./foo -d ! -Q -sp foo -r main to dump an image and then run the executable, it just dumps me in the ECL REPL. I'm not only not in the right package, but the one that should be getting loaded into this image isn't even there. It does however have its dependencies (CLINGON being one of them), which makes it even more confusing.

I'm not sure if this is a problem with me just not passing the right args in the command line (this same set of options works for SBCL) or if cl-launch itself needs to be updated to work with the most recent version of ECL.

ECL (Embeddable Common-Lisp) 21.2.1 (git:UNKNOWN)
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2013 Juan J. Garcia-Ripoll
Copyright (C) 2018 Daniel Kochmanski
Copyright (C) 2021 Daniel Kochmanski and Marius Gerbershagen
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.
Top level in: #<process TOP-LEVEL 0x800d66f80>.
> *package*

#<"COMMON-LISP-USER" package>
> (list-all-packages)

(#<"QL-SBCL" package> #<"QL-MKCL" package> #<"QL-MEZZANO" package>
 #<"QL-ECL" package> #<"SB-BSD-SOCKETS" package> #<"QL-LISPWORKS" package>
 #<"QL-SCL" package> #<"QL-CMUCL" package> #<"QL-CLISP" package>
 #<"QL-CLASP" package> #<"QL-CCL" package> #<"QL-ABCL" package>
 #<"QL-ALLEGRO" package> #<"QUICKLISP-CLIENT" package> #<"QL-BUNDLE" package>
 #<"QL-DIST-USER" package> #<"QL-DIST" package> #<"QL-CDB" package>
 #<"QL-GUNZIPPER" package> #<"QL-MINITAR" package> #<"QL-HTTP" package>
 #<"QL-PROGRESS" package> #<"QL-NETWORK" package> #<"QL-IMPL-UTIL" package>
 #<"QL-IMPL" package> #<"QL-CONFIG" package> #<"QL-UTIL" package>
 #<"QL-INFO" package> #<"QL-SETUP" package> #<"CLINGON" package>
 #<"CLINGON.COMMAND" package> #<"CLINGON.OPTIONS" package>
 #<"CLINGON.CONDITIONS" package> #<"CLINGON.UTILS" package>
 #<"WITH-USER-ABORT" package> #<"CL-REEXPORT" package>
 #<"ALEXANDRIA-2" package> #<"ALEXANDRIA" package> #<"BOBBIN" package>
 #<"SPLIT-SEQUENCE" package> #<"ASDF/FOOTER" package> #<"ASDF/USER" package>
 #<"ASDF/INTERFACE" package> #<"ASDF/BACKWARD-INTERFACE" package>
 #<"ASDF/BACKWARD-INTERNALS" package> #<"ASDF/PACKAGE-INFERRED-SYSTEM" package>
 #<"ASDF/CONCATENATE-SOURCE" package> #<"ASDF/BUNDLE" package>
 #<"ASDF/PARSE-DEFSYSTEM" package> #<"ASDF/SOURCE-REGISTRY" package>
 #<"ASDF/OUTPUT-TRANSLATIONS" package> #<"ASDF/OPERATE" package>
 #<"ASDF/PLAN" package> #<"ASDF/LISP-ACTION" package> #<"ASDF/ACTION" package>
 #<"ASDF/OPERATION" package> #<"ASDF/FIND-COMPONENT" package>
 #<"ASDF/FIND-SYSTEM" package> #<"ASDF/CACHE" package> #<"ASDF/SYSTEM" package>
 #<"ASDF/COMPONENT" package> #<"ASDF/UPGRADE" package> #<"UIOP/DRIVER" package>
 #<"UIOP/BACKWARD-DRIVER" package> #<"UIOP/CONFIGURATION" package>
 #<"UIOP/LISP-BUILD" package> #<"UIOP/RUN-PROGRAM" package>
 #<"UIOP/IMAGE" package> #<"UIOP/STREAM" package> #<"UIOP/FILESYSTEM" package>
 #<"UIOP/PATHNAME" package> #<"UIOP/OS" package> #<"UIOP/UTILITY" package>
 #<"UIOP/COMMON-LISP" package> #<"ASDF/PACKAGE" package>
 #<"UIOP/PACKAGE" package> #<"WALKER" package> #<"ECL-CDB" package>
 #<"FFI" package> #<"GRAY" package> #<"MP" package> #<"CLOS" package>
 #<"C" package> #<"SI" package> #<"EXT" package> #<"KEYWORD" package>
 #<"COMMON-LISP-USER" package> #<"COMMON-LISP" package>)

Problem

1. Using --dispatch-entry isn't intuitive and in no way easy for simple scripts.
2. Necessity of .asd for self-compiled one-file scripts is discouraging.
3. Caching binaries is not automated.

Story

The most significant (at least for me) point of other scripting languages with shebang lays in their ability to provide self-sufficient one-file scripts. Such scripts may depend on libraries installed somewhere in distribution-specific ways or downloaded by user. And these libraries are mostly discovered through some runtime path environment variable -- with distribution-configured default value. Even C/C++ has caching wrapper to build and run scripts effortlessly. So most of the time it's a zero-configuration way.

On the other hand there are virtualenv/pip/gem/cargo/etc... which accomodate complex/frozen ecosystems for large projects. They are characterized by having some configuration file which describes everything. And they are too much pain to use outside their cozzy setup project directory.
Notice how much it resembles current state of affairs with dispatch-entry compilation step which requires correctly configured environment and multiple .asd for everything to work perfectly.

And here we are with CL -- when we use cl-lauch to run simple scripts -- it works acceptably and at least on par with caching C/C++ solution -- by at least pre-compiling *.fsl. But when trying to go one step further and directly exec from lisp script into generated dump image -- it's easier to be said than to be done. Not talking about sharing part of the image between all of scripts in this case...

Usecase

What I actually expect to work -- assuming generated binaries will be stored in ~/.cache/common-lisp/bin-cache/:

#!/usr/bin/cl --cached
(defun main (argv)
  (format t "First~%~S~%" argv)
  t)

As you could notice, I don't even care about current package. If I planned to reuse some part of code between two scripts -- I would move it into separate ArchLinux AUR package under /usr/share/common-lisp/sources/<mycommon>/*

What this -C|--cached is actually means in my head is akin to

cl-launch --output "$HOME/.cache/common-lisp/bin-cache/$1" \
  --dump '!' --lisp sbcl -E main "$1" \
  && exec "$HOME/.cache/common-lisp/bin-cache/$1" || exit 1

Notice -- we will cache it under full path and exec after execution.

Proposal

Even better would be to reuse shared image automatically (if it's possible at all)

• read precompiled image (whichever was used for --output or with default path ~/.cache/common-lisp/bin-cache/default)
• augment current script with its own package/system and evaluate inside that image to append
• store image back into that path
• create symlink in ~/.cache/common-lisp/bin/ or ~/.local/bin/ for multibinary support pointing into that default image

Questions

1. Did I miss something in documentation and my expected proposal is already implemented ?
2. Are expected convention is actually quite different and I'm trying to do it the wrong way ?
3. How hard would it be to add into cl-launch so I could cease writing my own convinience scripts to auto-exec into multibinary image?