Trying out the new :modules support

[Edwin Park]

Hi David,

I was super excited to see Google Closure Module support landed in ClojureScript, but I can't get it to work. I'm using clojurescript 0.0-2850. I created some dummy cljs files and I'm literally using the example configuration shown in the documentation here: https://github.com/clojure/clojurescript/wiki/Compiler-Options#modules. However when I compile, none of the module output-to files are generated. Details of my setup are below. If you can point out what I'm doing wrong I'd appreciate it - I'm really looking forward to using this!

Thanks,
Edwin


Here are the contents of the resources/assets dir after compilation (lein cljsbuild once). Note that these are the intermediate output files created by the compiler but the actual output-to files are missing:

resources/assets/js/cljs/core.cljs
resources/assets/js/cljs/core.js
resources/assets/js/com/foo/common.js
resources/assets/js/com/foo/editor.js
resources/assets/js/com/foo/landing.js
resources/assets/js/constants_table.js

Here is my project configuration and the dummy files I'm compiling:

project.clj:

(defproject foo "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}

:plugins [[lein-cljsbuild "1.0.4"]]

:dependencies [[org.clojure/clojure "1.6.0"]
[org.clojure/clojurescript "0.0-2850"]]

:cljsbuild {:builds [{:source-paths ["src/cljs"]
:compiler {:optimizations :advanced
:output-dir "resources/assets/js"
:modules {
:common
{:output-to "resources/assets/js/common.js"
:entries '#{com.foo.common}}
:landing
{:output-to "resources/assets/js/landing.js"
:entries '#{com.foo.landing}
:depends-on #{:common}}
:editor
{:output-to "resources/assets/js/editor.js"
:entries '#{com.foo.editor}
:depends-on #{:common}}}}}]})


src/cljs/com/foo/common.cljs:

(ns com.foo.common)
(defn blah [x]
(str "[" x "]"))


src/cljs/com/foo/landing.cljs:

(ns com.foo.landing
(:require [com.foo.common :refer (blah)]))
(defn land [x]
(str "landing:" (blah x)))


src/cljs/com/foo/editor.cljs:

(ns com.foo.editor
(:require [com.foo.common :refer (blah)]))
(defn edit [x]
(str "editing:" (blah x)))

[David Nolen]

You can't use the :modules feature in 0.0-2850 you need to use master. You can do this by git cloning the repo and running "./script/build" from the repo directory. You'll see Maven install it into your local cache, take note of the version number. Use this in your project.clj.

HTH,

David

--
Note that posts from new members are moderated - please be patient with your first post.
---
You received this message because you are subscribed to the Google Groups "ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.
To post to this group, send email to clojur...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojurescript.

[Edwin Park]

Ah ok thanks! - I did this and now the module output-to files are generated, but they are all empty:

[esp@pogo ~/Code/test/foo]$ ls -l resources/assets/js/
total 0
-rw-r--r-- 1 esp staff 0 Feb 18 05:44 common.js
-rw-r--r-- 1 esp staff 0 Feb 18 05:44 editor.js
-rw-r--r-- 1 esp staff 0 Feb 18 05:44 landing.js
drwxr-xr-x 6 esp staff 204 Feb 18 05:44 out

Here's my updated project.clj:

(defproject foo "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}

:plugins [[lein-cljsbuild "1.0.4"]]

:dependencies [[org.clojure/clojure "1.6.0"]

[org.clojure/clojurescript "0.0-2884"]]

:cljsbuild {:builds [{:source-paths ["src/cljs"]
:compiler {:optimizations :advanced

:output-dir "resources/assets/js/out"

:modules {
:common
{:output-to "resources/assets/js/common.js"
:entries #{com.foo.common}}
:landing
{:output-to "resources/assets/js/landing.js"
:entries #{com.foo.landing}
:depends-on #{:common}}
:editor
{:output-to "resources/assets/js/editor.js"
:entries #{com.foo.editor}
:depends-on #{:common}}}}}]})

[Thomas Heller]

That is probably dead code removal in action. Since none of your code is actually called it all will be removed. :)

Try to ^:export a function that will keep it from getting removed or just call one of the functions.

(ns com.foo.editor
(:require [com.foo.common :refer (blah)]))

(defn ^:export edit [x]
(str "editing:" (blah x)))

[Edwin Park]

lol - you're right. I added ^:export to my entry point functions and all is right with the world. :-)

I noticed that if I remove the module for com.foo.common, the code in there gets included in cljs_base.js in the compiler temporary output, but there is no dead code elimination performed there. This may be expected, it just means that I need to declare all my code in modules if I want it minified.

Also, about the explicit module dependencies declarations: it seems like it should be possible to automatically generate these by analyzing the namespace metadata. Is there a reason to force this to be explicitly declared instead?

Thanks, awesome stuff -
Edwin

[David Nolen]

On Wed, Feb 18, 2015 at 8:04 AM, Edwin Park <edwin...@gmail.com> wrote:

lol - you're right. I added ^:export to my entry point functions and all is right with the world.  :-)

I noticed that if I remove the module for com.foo.common, the code in there gets included in cljs_base.js in the compiler temporary output, but there is no dead code elimination performed there. This may be expected, it just means that I need to declare all my code in modules if I want it minified.

This is not true. You do not need to declare all your code in modules. It is the presence of ^:export that prevents DCE.

Also, about the explicit module dependencies declarations: it seems like it should be possible to automatically generate these by analyzing the namespace metadata. Is there a reason to force this to be explicitly declared instead?

As far as I can tell both Thomas Heller & Google have come to the conclusion that only explicit dependencies work reliably. I didn't bother exploring alternatives since I didn't think I would do any better :)

David

[Edwin Park]

> I noticed that if I remove the module for com.foo.common, the code in there gets included in cljs_base.js in the compiler temporary output, but there is no dead code elimination performed there. This may be expected, it just means that I need to declare all my code in modules if I want it minified.
>
>
>
> This is not true. You do not need to declare all your code in modules. It is the presence of ^:export that prevents DCE.

I just want to clarify what I'm seeing. Here is the compiler config with the com.foo.common module removed and the ':depends-on #{:common}' lines removed:

{:optimizations :advanced
:output-dir "resources/assets/js/out"
:modules {

:landing
{:output-to "resources/assets/js/landing.js"
:entries #{com.foo.landing}
}

:editor
{:output-to "resources/assets/js/editor.js"
:entries #{com.foo.editor}
}}}

And here are my cljs files with ^:export on all the functions:

src/cljs/com/foo/common.cljs:

(ns com.foo.common)
(defn ^:export blah [x]

(str "[" x "]"))


src/cljs/com/foo/landing.cljs:

(ns com.foo.landing

(:require [com.foo.common :refer (blah)]))

(defn ^:export land [x]

(str "landing:" (blah x)))


src/cljs/com/foo/editor.cljs:

(ns com.foo.editor
(:require [com.foo.common :refer (blah)]))
(defn ^:export edit [x]
(str "editing:" (blah x)))

When I compile, the landing and editor js files are generated:

resources/assets/js/landing.js:

da("com.foo.landing.land", function(a) {
return[B("landing:"), B(ge(a))].join("");
});


resources/assets/js/editor.js:

da("com.foo.editor.edit", function(a) {
return[B("editing:"), B(ge(a))].join("");
});


There is also a resources/assets/js/out/cljs_base.js file generated in the compiler temporary output with the code from com.foo.common appended at the bottom of a lot of other stuff:

...4700+ lines...
function ge(a) {
return[B("["), B(a), B("]")].join("");
}
da("com.foo.common.blah", ge);


The issue is the 4700+ lines of unused preamble in the cljs_base.js. I guess I was expecting the cljs-base file to simply contain the referenced com.foo.common code and DCE to eliminate the rest of it so I'd have a minimized file with just the code I need that I could load up into the browser.

> Also, about the explicit module dependencies declarations: it seems like it should be possible to automatically generate these by analyzing the namespace metadata. Is there a reason to force this to be explicitly declared instead?
>
>
> As far as I can tell both Thomas Heller & Google have come to the conclusion that only explicit dependencies work reliably. I didn't bother exploring alternatives since I didn't think I would do any better :)

I'm curious about this. Thomas, can you comment on what the issues are? I'm thinking if I scan all the namespaces listed in :entries and collect all of their requires, I should be able to automatically generate the :depends-on for all cljs dependencies. I can understand how arbitrary js/ dependencies may need to be explicitly declared, but any cljs dependencies should be knowable by inspecting the requires metadata on a given namespace.

Thanks,
Edwin

[David Nolen]

What you're observing has nothing to do with :modules. It appears there's been a minor DCE regression wrt. trivial programs. If you remove :modules the exact same thing will manifest.

The regression is in fact minor though as it's precisely the same thing that happens the moment you use a ClojureScript data literal.

David

[David Nolen]

I just fixed the DCE regression for trivial programs in master: https://github.com/clojure/clojurescript/commit/f1cb61a6a4985b2d3633fc2fb13d0c06f3718272

You'll see that :modules in no way whatsoever interferes with Google Closure Compiler's DCE pass.

David

[Edwin Park]

Hmm. I do see that DCE behavior is the same with and without :modules, but the fix in 0.0-2885 doesn't seem to do anything.

For both clojurescript 0.0-2884 and 0.0-2885 I'm not seeing dead code elimination in either the cljs_base.js (with :modules) or in the consolidated foo.js (with :output-to "resources/assets/js/foo.js"). When using :modules I wind up with a 4778 line cljs_base.js, and when using :output-to I get a 4788 line foo.js.

I have attached my test project for reference.

Edwin

[David Nolen]

Stop using export. Use a console.log instead.

Trust me DCE works.

[Edwin Park]

Ok got it now. I replaced my str calls with native js .concat and the file size dropped from ~115k to ~1.5k. Sorry for the confusion, I'm still feeling out the edges. I didn't realize that str would transitively drag in so much.

Thanks again,
Edwin

[Thomas Heller]

>
> Also, about the explicit module dependencies declarations: it seems like it should be possible to automatically generate these by analyzing the namespace metadata. Is there a reason to force this to be explicitly declared instead?
>

The reason the declaration is necessary is quite simple and the only way to get "optimal" module structure.

A simple example:

(ns my-app.common)

(ns my-app.module-a
(:require [my-app.common] [clojure.string]))

(ns my-app.module-b
(:require [my-app.common] [clojure.string]))


If you define 3 modules common, module-a, module-b each with its respective entry points the compiler cannot figure out the dependencies because there is a duplicate dependency on clojure.string.

You want module-a depends on common and module-b depends-on common. But the duplicate dependency on clojure.string prevents this and if you leave the program to figure out the dependencies you'll either get module-b -> module-a -> common OR module-a -> module-b -> common.

Leaving the user with 3 requests to fetch module-a instead of just 2. Ideally what SHOULD happen is that the clojure.string namespace is moved to common, but I do not know of any algorithm that can figure this out. With the explicit declaration we can do this easily.

Hope that made sense.

/thomas