Compiler is now almost fully parallel

[Alon Zakai]

On the incoming branch, we now parallelize even the first compilation step (ll to js). Together with last week's work, we now parallelize practically all the compiler, except for (1) a few very short stages, and (2) closure compiler (which is parallel itself in part, but has a very long single-core phase).

Please test incoming on large codebases, this seems to past our tests, but is still a big change. In particular let me know if you see issues on windows (we had some problems with parallelizing the js optimizer last week, at first).

Aside from bugs, it might be useful to tweak the chunking parameters, see emscripten.py, and compile with EMCC_DEBUG=1 to see debug output of how many chunks, cores etc. are being used. Please let me know if you see anything odd or have suggestions for optimizations based on your project.

- azakai

[wolfviking0]

Hi Alon,

Thanks for your  job, for my engine, the build with -O2 take 20 minutes, it's better when there was not parallelize (40 minutes) but the parallelize of first compilation step, doesn't change a lot for me.

[Alon Zakai]

Can you run the build with EMCC_DEBUG=1 in the environment and link to a gist or pastebin of the output (so it prints out how long each step took)? I want to know which step is the slow one.

-azakai

[wolfviking0]

Alon, i add in attachment the console file, but i don't understand " link to a gist or pastebin ", but i have the time of each step took

[Alon Zakai]

Thanks, yeah, what I meant by gist/pastebin was to link to it or attach it.

Ok, looks like the parallel stuff is working well, should be saving a lot of time here. The biggest issue left is closure as I suspected. In the short term, building without closure (--closure 0) is useful for quick testing builds, in the long term, we will not need closure (the new eliminator does half of what closure does, a new registerizer will do the rest, which leaves only minification).

- azakai

[wolfviking0]

Sound great, do you know when the new eliminator and registerizer will be ready for do some test ?

[Alon Zakai]

New eliminator has already landed. New registerizer is a longer-term plan, it fits in with other optimizations being worked on. I'd say a few months.

- azakai