Porting Clojure seems to be the thing to do these days. After the clojurescript compiler came out, people had a really good blueprint for hosting clojure in other languages. The fact that the clojurescript compiler is defined, largely, in terms of protocols, makes it a bit closer to the “clojure in clojure” goal. As a result, you should be able to bootstrap the language (as they did with JavaScript) with a minimum of primitives. Most of the language is defined in Clojure, so voila.

We currently have Clojure ports targetting Scheme, C, C-via-Scheme, Python, etc. I’ve found Common Lisp to be a particularly slick environment with some cool tools and a fairly active community. I think it would be interesting to port the ideas from Clojure to Common Lisp, ultimately implementing a compliant Clojure in Common Lisp.

This includes lazy sequences, the generic sequence abstraction, and the fundamental persistent hash-array-mapped-trie data structures in clojure:

• persistent vectors
• persistent maps
• persistent sets.

Common Lisp already has a sequence library, but I think Clojure’s is more general and can be trivially extended to new types. Common Lisp’s irrational locking down of SEQUENCE is a hurdle here. The HYPERSPEC will never be updated in my lifetime :) So generic functions are the current way to bridge this stuff.

• Protocols are really nice, as are Clojure’s arbitrary dispatch multimethods.
• Data literals are also highly useful. I think Clojure nailed the choice of literals, so providing reader macros for these guys would be very nice.

So far, this project continues to be a great way to learn about both CL and Clojure, to include implementation hurdles, support for PLT stuff, long-standing warts from decisions made ~25 years or more ago, and work-arounds. Nothing is insurmountable so far, which is pretty cool.

simultaneously learning Common Lisp (and by necessity Java :/ ).

• Got a working persistent vector implementation, with compatible clojure literals about a year ago.
• Started working on persistent hash maps around November 2012.
• Built a temporary implementation of clojure-style protocols in Common Lisp ~ Dec 2012.
• Pulled the bits into an actual ASDF system and pushed everything to the Github August 2013.
• Implemented a baby clojure evaluator that __should__ bridge the lisp1/lisp2 gap between clojure and the Common Lisp host. Unsure if this is going to work out in the long term, but eh.
• It’s real trivial at the moment.
• Working on library code in my spare time, still need to port the persistent map.

• implemented some rudimentary stuff
• vector reader macros not fleshed out; work fine at the REPL, but failed to return a vector (instead returning a form to create the vector). Trivial but important oversight.
• Still hijacking the global read-table. Pulled in named-readtables to help, but haven’t integrated.
• Working on variadic functions, explored funcallable classes, refined protocols, deftype.
• cleaned up the build process (more work to be done here)

• got reader macros matured (vector literals produced properly now),
• got protocol definitions and implementations respecting vectors, albeit in a hacky way. We still allow lists for args….
• working on deftype, then getting the bootstrap from CLJS (core protocols and functions) to compile.
• Working on leveraging named-readtables for better delineation of clojure source, to include file-level (somewhat racket like).
• Also intend to leverage named-readtable to get case-senstive reader (via :modern), and to enable CL interop trivially with a macro allows CL reader inside body (vice versa, for inline clojure if it makes sense).
• refining the approach after reading a lot, looking at some other sources of inspiration (including early proto-clojure->java compiler from Hickey in CL)
• Basic def, fn works. Protocols work. Metadata works mostly. Deftype
• got let, albeit without destructuring. Still useful for bootstrapping!
• Initial implementation of reify working, wrapped deftype in a version compatible with cl:deftype

A couple of big hurdles:

• rely on macros (like def) to unify symbol and function namespaces, leveraging CL’s environment.
• The longer route would be writing a custom eval, compiler, etc. Doesn’t look necessary at the moment.

So, Common Lisp as a lisp-2 has multiple namespaces, foremost of which are function and symbol. We already have the top-level (that is, null lexical environment) symbol and function namespaces unified by using setf for symbol-function to make it identical to symbol value….but…..

• Lexical environments don’t work that way!
  • symbol-function and symbol-value only work on non-lexical symbols.
  • Initial hack was to unify the namespaces by traversing the vars in the let bindings and unifying prior to continued binding.
    • Had a false-positive success since the symbol modifications were actually pointing to non-lexical scoped symbols (stuff from prior REPL interactions).
• The fix for this is to use a combination of let* and labels, which CAN unify the symbol-value and symbol-function namespaces for lexical vars..
  • Defined a macro, unified-let*, that does this for us:
    • We parse the bindings, detecting if any symbol points to a literal keyword.
    • We ensure keyaccess objects are compiled during macroexpansion, which creates the plumbing for keyword access if it didn’t already exist
      • and we create function definitions for the vars that point to keywords, where the function bindings invoke the funcallable keyword accessor directly.
    • We need to apply this as an implicit body for fn forms as well..

• If we get defprotocol, deftype, etc. up and running, these are already defined in CLJS core.
• For bootstrapping, using original port of Persistent Vector from JVM clojure, also creating a dumb wrapper on top.
  • Need to add meta field to struct, also atomic locks at some point (unless cljs provides this….)

• Already implemented as generic functions.
• Explore alternative implementations if generic functions aren’t speedy enough (doubtful, but haven’t profiled anything).
• protocol definitions need to be namespace/package qualified.
  • Looks like they are already, at the package level.
• Need better error messaging on missing protocols
  • TODO: get-protocol should signal.

• shadows CL deftype.
• current implementation follows defprotcol, appears to work for non-varidiac protocol impls.
• Look at walking method impl bodies to eliminated unnecesary slot bindings (currently we generally bind all non-shadowed slots).

• Initial ideas for multiple dispatch following clojure’s implementation.

• Symbols and collections (anything that supports IMeta) can have a persistent map of metadata in clojure.
• Used to communicate a lot of information, including type hinting, source code, documentation, etc.
• Current expectation is to leverage property lists for symbols, and unify with generic functions behind a protocol.

• CL has packages. Initial hack would be to leverage packages as an anlogue to ns.
• Longer term solution is implement own ns system via objects.
  • Rather leverage CL where possible.
• Currently implementation of def exports by default.
  • Looking at introspection possibilites for more easily tagging meta, arglists. custom function objects (experimental) are looking good for this.
  • Should we maintain a parallel collection of namespaces?
    • Based on def and derived forms, we ought to be able to hook in and register stuff.
    • Allows the reflection and introspection we care about / use in clojure.
• Need to translate between require, refer, import (clojure) and import (cl).

We’ll need to either cook the common lisp reader, or build a separate clojure reader that will perform the appropriate replacements.

• Looks like a simple tweak to the ` reader macro will suffice, we’ll just flip the symbols as appropriate.
• TODO: quasitquoting in clojure (let []) inside of macros is not splicing, need to revisit quoted-children.
  • ex `[,’a] => [,’A] (incorrect)

• Similar, we’ll flip the read-table.

• already have a reader macro in reader.lisp
• There’s an incorrect read going on, [x 2] will currently read when it should throw since x is not defined.
  • TODO: revisite quoted-children for vectors and the reader fn bracket-reader.
  • If we’re not reading a quoted vec, we need to eval args to persistent-vector ctor.

• already have a reader macro in pmap.lisp

• Easy enough to add a dispatch in reader.lisp

• Trivial to implement as a reader macro, BUT, we need to get metadata working generally.

• Added initial support for clj->cl, need to define printable representation for chars as well.
• Current holdup is defining a print-method for STANDARD-CHAR, which claims the class doesn’t exist.
  • Looking at print-escape and friends to see if we can hack this. We may need our own printer, outside of the provided REPL.

• Similar issues as characters. PAIP has a good chapter on this for similar issues with Scheme.

• wrap to calls for CLOS slot

• detect / in symbol name, coerce to qualified internal symbol ns.name::fn

• .hello is a valid function name in CL…
• Reader macro for .?
  • need to incorporate . form ala: (. obj someMethod)

???

is used as a delimiter for package symbols in CL.

• need to parse the symbol name and dispatch….
• ::blah -> :BLAH for now…CL reader eliminates redundant colon.
• Should be simple to modify the reader macro for :

• keep? This doesn’t work the same in clj jvm…

Keywords are applicable in Clojure. That means they show up in the function position in forms. This won’t jive in CL directly.

• Possible option: reader macro for lists, detect presence of keyword in function call positition, if not quoted.
• Replace keyword with a funcallable thing that has a print-method looking like a keyword?
• Or try to hack eval (dubious).
• custom read / eval / print….

Looks like we can just leverage he function namespace to get around this….keywords are “just” symbols….

• (defun :a (m) (gethash :a m))
• (defun (setf :A) (new-value m) (setf (gethash :A m) new-value))
• (defparameter ht (make-hash-table))
• (setf (:A ht) 3)

So the workaround is:

• Need a reader macro lists.
• If we see a keyword in the function position,
  • we define a function for the keyword via defun.
  • define a setf expander that provides hashmap access (alternately, something that’s not mutating).

Looks like that may work inside the existing ecosystem…. Significant progress / experimentation in clclojure.keyworfuncs. However, it’s looking like, to get “full” access (even with some tricky pseudo-keyword class, macros, and the above suff), we’re probably better off hacking eval / compile.

This may be a bit tricky, although there are a limited number of clojure forms.

This shouldn’t be too hard. I already have a lazy list lib prototype as well as generic functions for the basic ops. I think I’ll try to use the protocols defined in the clojurescript version as much possible, rather than baking in a lot of the seq abstraction in the host language like clojure does.

• For simple bootstrapping, this if fine, but we already get all of this with the CLJS core implementation. So, get the minimums there and gain everything else….

CL strings are mutable (character arrays), clj/cljs are not…

• Can we inherit from string to create an immutable type that outlaws setf?
• I think most string operations (ala the sequence-based ones) return copies (which are mutable).
  • We can prevent setf in clojure mode, providing a pure API for string manipulation…
    • As well as impure….hmm

Leverage CL-PPRCE

• check for reader macro collisions….

Currently, we project everything to (portable) CL, and let the host do the dirty work for compilation/analysis.

• Future, port clojure.tools.analyzer
  • Maybe use that for some of our efforts…
• If we have enough ported, look at using clojure.tools.reader to help as well.

Currently just clone the repository. Assuming you have ASDF setup properly, you should be able to evaluate (require ‘clclojure) at your Lisp REPL and it’ll compile (if you’re in the project dir in lisp).

Alternately, you can compile and load the .asd file, then use quicklisp to load it from the repl. (ql:quickload :clclojure)

You currently get persistent vectors, literals, defprotocol, extend-protocol.

You can mess with the currently limited clojure evaluator in the :clclojure.base package, although I’m moving in a different direction now that I think I can explot CL better.

You can see rudimentary examples and usage in the :clclojure.example package (example.lisp). TODO: shift to named-readtables to keep clclojure from hijacking your session. Right now, we take over the REPL…..user beware!

I’m a bit new to building Common Lisp projects, so the packaging will likely change as I learn. At some point, if anything useful comes out of this experiment, I may see if it can get pushed to quicklisp.

Eclipse Public License, just like Clojure.