The Closure Script Method

[David Turnbull]

Closure Script is a year old this month!  I have just released V1.4.

 - Modules are more deeply analyzed for shared code.

 - Ruby Gem has an executable that makes it easier to use.

 - A new README explaining The Closure Script Method.

 - Latest compilers (all Compiler versions still work with all Script versions).

I tried to make a better introduction to Closure Script by writing a new README around the idea of introducing the development process as an excuse to explain how to use Script (or vice versa).  We want to explain more about how we work with Closure but this requires time and encouragement.  If you get a chance to read the new docs or try the new version, please let us know what you liked and didn't.

https://github.com/dturnbull/closure-script

-david

[Robert Stuttaford]

This looks fantastic, David. I've been mulling over giving this a try.

I tried Plovr for dev, but found that compilation took waaaaay too long to be sensible for the edit-refresh-test development loop.

How does Closure Script's JIT compilation compare to Plovr? I was using Simple mode for dev, but of course it'd be much nicer to use advanced to get the benefit of additional error messages.

Right now I have a small file-system watcher which compiles .soy files and also calculates deps files (one for *.js, one for *.soy) on file change. My dev setup simply includes base, the deps files, and a custom bootstrap for development. This means zero wait time when developing. How does Closure Script compare to that?

Then for production builds, I am using Plovr, as it's convenient to have a config json file to manage all the associated details. I am also using fixjsstyle and gjslint --strict whenever I build. Does Closure Script lint at all?

Sorry for the wall of text :)

Thanks

Robert

[Robert Stuttaford]

Ignore all my previous questions. I gave it a try. It's awesome! Going to use it for dev and dev builds instead of what I was doing before. I'll use the ?build and also lint pre-commit and continue using plovr for deployment builds.

[Ilia Mirkin]

On Mon, Oct 24, 2011 at 5:11 AM, Robert Stuttaford
<robert.s...@gmail.com> wrote:
> I tried Plovr for dev, but found that compilation took waaaaay too long to
> be sensible for the edit-refresh-test development loop.
>
> How does Closure Script's JIT compilation compare to Plovr? I was using
> Simple mode for dev, but of course it'd be much nicer to use advanced to get
> the benefit of additional error messages.

I'd be really curious to know how closure script achieves any
speed-ups in this area over plovr. Generally when using plovr, I use
RAW mode, which causes the files to get loaded in as-is, which makes
for an instaneous edit-refresh-test cycle, but loses the advantages of
compiler analysis in that cycle.

[David Turnbull]

Ilia, I never thought to compare compile times.  I assumed they would be so close it wouldn't matter.  I was wrong.

A complex module project is taking 9 seconds to compile in Script and 16 seconds in Plovr.  A substantial difference for sure.  These times are subsequent compiles; the first couple compile jobs tend to take a bit longer while everything gets loaded.

I don't know why Script tests faster than Plovr.  I'm just using a plain compiler.jar.  Script also does slightly better with common code from modules.  Again, not sure why.  I think these questions will have to be answered by the Plovr team.

[Robert Stuttaford]

David,

I've spent the day converting two projects to using Closure-Script instead of Plovr; one, a web app that's intended for web & mobile, and two, a Phonegap app. After setting them up, my project folders are looking a lot tidier.

I've not done any actual measurements, but somehow, the builds on Script do seem faster.

Thanks for your contribution, Script is going to be a big help.

[David Turnbull]

Thanks, Robert.  I'm glad you were able to get new tools out of your way so quickly.

[Robert Stuttaford]

That's a good way to put it.

So for deployment, I'm not using Ruby on the backend. Right now, prior to commit, I'm manually ensuring that ?build and ?debug are run, so that the right files are present for the commit. 

Any suggestions for automating builds as part of a deployment process? I'd like to avoid storing build configuration in multiple formats and so I'm curious if it's possible to script a call to goog.compile in a shell script?

[David Turnbull]

Here's what I use.  Add a new 'deploy' target making it identical to 'build'.

    when 'build', 'deploy' then %w{

      --compilation_level ADVANCED_OPTIMIZATIONS

      --js_output_file compiler_build.js

    }

Then you can copy the output files with shell commands after compilation.

    

    @response = goog.compile(args).to_response

    if query_string == 'deploy'

      `cp compiler_build.js ../public/happiness.js`

    end

Once you see working code, just change the ?build to ?deploy.  Compilation will technically be skipped, but the copy commands will run.  This way you're never risking regression on a future build.

Access to the compiler is available for direct use but I strongly recommend the ?deploy technique instead.

    Closure::Compiler.compile(args) # returns Closure::Compilation

[Michael Bolin]

Hello everyone,

With respect to performance, if anyone has an open source project that
is setup for Closure Script and plovr where Closure Script is running
significantly faster, then please send me a pointer and I will take a
look (or a closed source project where you're willing to share your
code).

plovr has two modifications to compiler.jar (of which I am aware):

* Adjustments to Rhino so that plovr can support CoffeeScript:
http://code.google.com/p/plovr/source/detail?r=7e3cad7f1b9d
* Support for putting modules in anonymous scopes:
http://code.google.com/p/plovr/source/detail?r=4edc58e190ad

If you checkout plovr and navigate to the closure/closure-compiler
directory and run "ant jar", then you will get the version of
compiler.jar with plovr's two modifications. If some curious person
could use that jar with Closure Script and could report whether that
was the source of the slowdown, then that would be very helpful
(though I am dubious that is the case).

Also, as Ilia pointed out, I am interested in the details of your
setup:

What constitutes a "build"? Is it you editing a file, hitting refresh
in your browser, and testing it, or is it you running the compiler
from the command line to generate a static file that you want to
upload on a server somewhere? In the case of the former, are you just
creating a deps.js and loading files individually (i.e. "RAW" mode in
plovr) or are you using the compiler to compile all your code on each
refresh, print warnings to the console, etc.? Knowing which of these
is slower in plovr would be extremely helpful in debugging the issue.

As Ilia suggests, I also develop using RAW mode in plovr and I find it
very fast. Admittedly, I generally find it too slow to develop in
SIMPLE or ADVANCED mode, so I would certainly be a lot happier if
there turned out to be a simple way to make it faster. Using file
watchers has been on my to-do list for some time, so if that turns out
to be the trick, then it will certainly jump to the top of my priority
list.

Also, David, since you asked about documentation, I'm sorry, but I
have to admit that I have not tried Closure Script because I had a lot
of trouble with the documentation. The name "Closure Script" implies
that it is a new language of sorts, but that does not seem to be the
case. It claims to be a "development environment," but that is a
pretty loaded term, often used to refer to an IDE, which I am also
pretty sure Closure Script is not. (My overall impression is that
Closure Script is a Ruby module.) I am not a Ruby user, and glancing
through your docs (to me) suggests that I need to be because I see
snippets with "<%" which I assume is some sort of Ruby templating
file. At a minimum, I think that a screenshot of what I would see at
http://localhost:8080/ would really help communicate what Closure
Script does.

And Robert, you mentioned that your project folders look a lot tidier
with Closure Script? I am interested in how using plovr made your
project feel "cluttered," as I believe the only additional file it
should add is a config file and hopefully does not force you to change
your project structure in an unnatural way.

Thank you,
Michael

[Stefan Gal]

I have to admit I am not a Closure Script user so I cannot speak for it.

But plovr has been working real well for me in a handful of projects, using closure without plovr would be much less appealing.

[David Turnbull]

Michael,

You can observe a performance difference on this project:

  https://github.com/EFFALO/domekit

It's already set up for Script (remove 'Closure.add_source :externs' from config.ru if it errors). I was using a URL like 'http://localhost:9810/compile?id= to build with plovr.  Plovr got down to 2.0 seconds and Script went to 1.7 seconds.

I made your special compiler.jar and plugged it into Script.  I could observe no performance difference.  I hope your Rhino changes make it upstream!

I observed that more code ended up in 'app'(root) with modules under Plovr.  Script left a bit more on the leaves.  Sorry, no example for this.  It was about 2% of the total code.  I don't arrange the files the same as plovr, I depend on the compiler to move the functions more, so maybe that's why.

Closure::Script is simply named after the main class for lack of a better idea.  I used to have a screenshot, I need to make a new one, and learn to write better.  If you don't like Ruby, all Script internal design is compatible with typed languages so ports to Python, Node, PHP-CLI, and other Script-ing languages are very practical.

-david

[David Turnbull]

As promised, more about the method.  Michael, this is meant to answer your question.

The features in Closure Script that make it easy to organize your project are subtle and designed to go unnoticed.  I have spent hundreds of hours directly observing professional users of Closure Script and worked to remove a large amount of minor annoyances with Closure development.  I haven't seen Robert's projects, but I can infer the kinds of things he did in Script and explain the difference from Plovr.

The most obvious feature is that Script doesn't need to be restarted to change compiler options.  Things in config.ru need a restart, but it's not compile job options in there, and most projects will never need to change this.

Plovr uses a single config file.  If you have multiple compile jobs they all have to be defined in there.  A recent 'inherits' feature in Plovr allowed including other config files, but you have to link to them from the main config.  You also have to specify filenames for the compiler. 

Closure Script allows you to put each job in its own file in any directory the server sees. Also, Script jobs compile namespaces, not files.  

If you have a complex module project and smaller compiler jobs to work on smaller units, you might be wrangling half a dozen compiler jobs.  For discussion, I have a project called 'app' and it has three folders 'editor', 'player', and 'welcome'.  

I decide that 'welcome' really needs to be called 'login'.  This change under Plovr requires renaming the folder, editing the 'app' config and the 'welcome' config, then restarting the server.  Repeat if you make a typo or change your mind.  With Script, you will only need to rename the folder.

It's not really about what your directories look like, it's about how you got there.  More files and using Ruby instead of JSON may appear like added complexity, but the net effect is less edits and no restarts.  If organization tasks aren't trivial they resist getting done so your project will feel cluttered.

[pstjvn]

Any chance that closure-stylesheet be integrated in closure-script?

[Robert Stuttaford]

Here you go.

Drop the attached closure-stylesheets.rb into a folder, 'lib', and put the closure-stylesheets-20111202.jar there too (just name it closure-stylesheets.jar).

Then, in config.ru, after require 'closure' but before anything else:

$LOAD_PATH << './lib'

require 'closure-stylesheets'

And in compiler.erb.js:

goog.gss_to_css %w{

  --rename CLOSURE

  --output-renaming-map css_renaming_map.js

  --output-renaming-map-format CLOSURE_COMPILED

  --output-file styles.css

  **/*.gss

}

Also in compiler.erb.js, before your call to --ns app.App:

--js css_renaming_map.js

This probably won't all work immediately, you'll have to fuss with pathing until you get it right. 

I had to inspect my project's folder to see where css_renaming_map actually appeared and update the main compiler call to reference it there.

@David Turnbull: feel free to put this directly into Closure Script after reviewing it, if you like! It's basically a direct copy of Templates with all the relevant bits switched over for GSS.

[David Turnbull]

Thanks, Robert.  This is exactly how I was going to try it first so it's great that you've proven it works.  I'm starting a new project soon so I'll put this in the official distro when I get on with that.

You can usually skip the changes to config.ru by using expand_path in your templates:

require expand_path ''addon.rb"

I use a 'load' when testing and switch to 'require' when it's working and ready to check in.  This is also a great shortcut to put up a scaffold so you can work on javascript before you have working controllers.

[pstjvn]

Either I don't understand the solution properly or it is not complete.
Here is the use case:

I develop the code in regular mode (i.e. no query string on the URL).
In this case the CSS file should be compiled with *--output-renaming-
map-format CLOSURE_UNCOMPILED* so that the map is not enclosed in
goog.setCssNameMapping and thus putting in as --js parameter does not
gives error: that is - in un-compiled mode the mapping is exported as
global variable and it is found this way by the rest of the library

This is all fine, I go and as some case statement in the
compiler.js.erb file and I can have the appropriate options depending
on the query string.

However something funny happens when the css_renaming_map.js has been
once profuced in compiled mode - it is no longer overwritten. What I
believe is happening your code (compiler-stylesheet.rb) tries to avoid
extra work and does not recompile the mapping as the source has not
changed. Can this be considered when adding this part to the closure
script? I mean - as per your advise we always include the same map
file and we always do it before other files, so it is the first to be
loaded in regular mode and goog then is not defined.

Thanks for the work, with this small exception the rest is working
great.

On Dec 6, 7:35 am, Robert Stuttaford <robert.stuttaf...@gmail.com>
wrote:
> Here you go.
>
> Drop the attached *closure-stylesheets.rb* into a folder, 'lib', and put
> the *closure-stylesheets-20111202.jar* there too (just name it *
> closure-stylesheets.jar*).
>
> Then, in *config.ru*, after *require 'closure'* but before anything else:
>
> $LOAD_PATH << './lib'
> require 'closure-stylesheets'
>
> And in *compiler.erb.js*:

>
> goog.gss_to_css %w{
>   --rename CLOSURE
>   --output-renaming-map css_renaming_map.js
>   --output-renaming-map-format CLOSURE_COMPILED
>   --output-file styles.css
>   **/*.gss
>
> }
>

> Also in *compiler.erb.js*, before your call to *--ns app.App*:

>
> --js css_renaming_map.js
>
> This probably won't all work immediately, you'll have to fuss with pathing
> until you get it right.
>
> I had to inspect my project's folder to see where css_renaming_map actually
> appeared and update the main compiler call to reference it there.
>
> @David Turnbull: feel free to put this directly into Closure Script after
> reviewing it, if you like! It's basically a direct copy of Templates with
> all the relevant bits switched over for GSS.
>

>  closure-stylesheets.rb
> 3KViewDownload

[David Turnbull]

pstjvn,

Is your project source available for us to review?  Can you provide a patch to Robert's compiler or a test demonstrating the failure?

-david