How to compile with optimizations none when using web workers

237 views
Skip to first unread message

William la Forge

unread,
Feb 19, 2016, 10:54:59 AM2/19/16
to Clojure

Compiling with optimizations none is no doubt quite handy, especially in conjunction with source maps, as a traceback will take you to the line of code causing the problems, and with the same variable names used in your original clojurescript code. But this is not currently possible for .jc files used by web workers as the unoptimized .js file contains a reference to js/window--and window does not exist in a web worker.


You should still be able to use optimizations none with your main javascript code, but you will need to compile your main code and your web worker code separately so that you can use different optimizations as appropriate. And you can do this so long as your not using shared workers. The question then is, how to compile the .js files separately.

The duracell demo uses a web worker AND is itself compiled with optimizations none. There are two things done to accomplish this:

  1. Duracell itself has no web worker code. Though it uses a library, durable-cells, which includes a web worker.
  2. In the duracell build.boot file, the dev task uses pandeiro/boot-http rather than the simpler tailrecursion/boot-jetty. The advantage to using boot-http is that it supports the loading of static files from library jar files. In this case, that means the main code in aaworker can create a web worker using durable-cell's dcells.js file.

So how does the durable-cells library create the dcells.js file? Again, there are two things done to accomplish this:

  1. In the durable-cells build.boot file, the dev task includes (cljs :optimizations :simple). This invokes the compiler with an appropriate level of optimizations.
  2. But we still need to define the .js file. This is done in the dcells.cljs.edn file. See Multiple Builds for more information on cljs.edn files.

Thomas Heller

unread,
Feb 19, 2016, 5:13:17 PM2/19/16
to Clojure
Hello,

I'm not quite sure what you are saying here since I do not know boot or hoplon or durable-cells.

I can however suggest a "better" solution for Workers: Using Closure Modules.

CLJS or boot do not implement them for :none at the moment but shadow-build does. I also have a few extra hints for web-worker support but first let me try to explain why you want to use modules.

If you do a separate build for your worker and your main app you'll have to download 2 full builds. Meaning that your main.js file will contain an implementation of cljs.core and the worker.js will contain a different one. If you are on :none they will share some parts, but :advanced will share nothing. This is bad as you want to keep the scripts to download to a minimum.

So how would the ideal solution look like:

1) base.js for all common code (eg. cljs.core)
2) app.js for all the app code
3) worker.js for all the worker related code

2+3 both share the base.js, you can add more workers trivially. IMHO the best way to organize this from a code perspective is to have a separate namespace for each worker as well as the app, where the app should not depend on the worker or vice versa.

Since this is only one build now, you only need to worry about one compile as well.

I have a demo repo here:

First we have the code:

Each is a self contained namespace that just does some trivial stuff.

The :none output is a bit confusing but lets look at the :advanced output:


Each is only the very minimum of code, the worker1.js will automatically load the base.js which means it shares all of the code in base.js with the page itself. There will be not additional download as the browser has already downloaded it when loading the page (since it included base.js and demo.js). The :none build basically does the same thing just in a closure compliant way. No duplicate file downloads will happen.

Everything is compiled using shadow-build, I can go into further detail on how if anyone is interested. I just wanted to make the point that web workers should ALWAYS be used via closure modules, it is just the most efficient way to organise the code and output.

Just my 2 cents,
/thomas

William la Forge

unread,
Feb 19, 2016, 7:41:57 PM2/19/16
to Clojure
Thomas,

Modules looks quite exciting. I would be glad to help in developing a boot task for same.

Bill 

William la Forge

unread,
Feb 19, 2016, 8:22:41 PM2/19/16
to Clojure
Thomas,

Have you seen this? A simple demo using workers in clojurescript: https://github.com/MarcoPolo/Servant


William la Forge

unread,
Feb 19, 2016, 8:29:56 PM2/19/16
to Clojure
Thomas,

Your worker demo includes the entire cljs runtime as part of the project? https://github.com/thheller/worker-example/tree/master/demo/js/cljs-runtime

Thomas Heller

unread,
Feb 20, 2016, 6:00:25 AM2/20/16
to Clojure
Hey,

I have seen Servant but I do not like it. In general I do not like solutions that use macros to solve problems a build tool should solve. YMMV.

cljs-runtime is the directory used by shadow-build for all compiled files. The structure it creates is that you have one directory, with a .js file for each module [1]. These .js files then load the .js and .map files in the cljs-runtime diretory. For :none builds the module files only contain a bunch of goog.require statements that mirror namespaces only used by these modules [2]. The base module includes goog/base.js and all setup stuff. For optimized builds (:whitespace and up) the directory with the module files contain the actual code and the cljs-runtime directory is no longer relevant. This is a very different structure from what CLJS or boot produce.

I use this project for tests and accidentally included some more output that should not have been there. Just pushed a cleaner version that actually only contains stuff used by this version of the project.

Anyways ... Modules for the win! ;)

Cheers,
/thomas


Asher Coren

unread,
May 30, 2016, 7:04:08 AM5/30/16
to Clojure
Thomas,
I as well think modules is the right approach when using web workers.

What can I do if my app code is build from many namespaces? How do I tell the main module to use all of them? Do I have to list them ALL?

Thomas Heller

unread,
May 30, 2016, 9:22:49 AM5/30/16
to Clojure
Not sure I understand what you mean. What do you mean by "main module"? cljs has a notion of a base module, while shadow-build does not.

Typically your ns structure and the requires are enough to establish relationships between them so shadow-build can figure out what needs to be where. If that is not possible for some reason you need to list them all yes.

/thomas
Reply all
Reply to author
Forward
0 new messages