lynaghk / clojurescript-compiler-proposal Goto Github PK

Thanks for a great initiative.

I propose a new namespace cljs.api for this. Everyhing not in this namespace should be considered an implementation detail.

Compile multi-namespace, custom-macro-using ClojureScript programs without touching the file system

I’m assuming that by ‘Compile’ you mean compilation from cljs -> js without using the closure compiler? I’m not sure what you mean by “... multi-namespace, custom-macro-using ...”
I propose the following functions for compilation:
  - (cljs.api/compile-forms forms opts)
    Available as cljs.closure/-compile[1]. This fn can be used without touching the filesystem.
  - (cljs.api/compile-file from to opts)  
    see cljs.compiler/compile-file[2] and cljs.closure/compile-file[3]
  - (cljs.api/compile-root from-dir to-dir opts)
    see cljs.compiler/compile-root[4] and cljs.closure/compile-dir[5]

Retrieve per-namespace errors/warnings as data (e.g., maps containing warning types, line numbers, and so on)

The compilation has to be responsible for finding errors/warnings. So we should decide how the compile-* functions above signals these errors. Personally, I don’t particularly like warnings. Either the code passes compilation or it doesn’t. There should be very few in-between IMO.

Retrieve per-namespace dependency information: "This namespace provides vars A, B, C and requires vars X, Y, Z from namespaces U, V, W"

The cljs.analyzer namespace contains a good parser for namespace forms[6]. A small refactoring could split the parsing in two parts: (1) parsing the data (lines 617 to 690) and (2) updating the namespaces var. (1) could be used in cljs.api as cljs.api/parse-ns-form

[1] https://github.com/clojure/clojurescript/blob/master/src/clj/cljs/closure.clj#L462
[2] https://github.com/clojure/clojurescript/blob/master/src/clj/cljs/compiler.clj#L806
[3] https://github.com/clojure/clojurescript/blob/master/src/clj/cljs/closure.clj#L380
[4] https://github.com/clojure/clojurescript/blob/master/src/clj/cljs/compiler.clj#L879
[5] https://github.com/clojure/clojurescript/blob/master/src/clj/cljs/closure.clj#L392
[4] https://github.com/clojure/clojurescript/blob/master/src/clj/cljs/analyzer.clj#L614

Nice idea! I built my own cljs build tool and a nicer API for the compiler/analyzer would make that a whole lot simpler.

Why?

I wanted to split the "huge" javascript output for my project into seperate files so the initial page load (browser js only) only loads the common js and then each page requests modules when needed. So instead of forcing every user to download 500kb+ everyone only downloads 233k and then 45k, 77k, 195k on demand when needed (sizes before gzip). As the project continues to grow the initial file will likely remain the same and the others will increase, so I'm very happy with this split.

Anyways, I couldn't make this work with lein-cljsbuild or the standard cljs compiler since they all assume that only ONE output file is generated by closure (after optimization). I however produce multiple files (via closure JSModule) which breaks just about every assumption made by current tools.

Proposal

The input to the closure optimization step should not receive a flat list of javascript inputs but instead a list of maps describing the modules closure should generate.

Something like this:

(closure/optimize [{:module-name "common"
                    :depends-on #{}
                    :sources sources...}
                   {:module-name "user"
                    :depends-on #{"common"}
                    :sources user-sources...}
                   {:module-name "admin"
                    :depends-on #{"common" "user"}
                    :sources admin-sources...}
                   ...])

The result would then be the input map with a :module-src value added containing the generated javascript. The tools may then decide where to put it. The build of a single module would then mirror the current behavior with no drawbacks.

Figuring out which source file belongs into which module is actually the hard part of this stuff, I have a working version but its tied into my current project and rather ugly to be honest. I want to clean it up and open source it when I find the time, until then I just wanted to make everyone aware of closures JSModules. ;)

Currently (at least in Clojure) :line is present in meta of &form in macroses only for top-level form. Sometimes it's useful to have it on all sub-forms.

My use case is code generation. I'm generating a few different versions of code specialized with different types and emitting them at once; therefore, all stacktraces contain wrong line numbers. Currently in Clojure I can't emit proper line numbers because I have them only for top-level forms that are passed to my macro.