On the incoming branch, we now parallelize even the first compilation step
(ll to js). Together with last week's work, we now parallelize practically
all the compiler, except for (1) a few very short stages, and (2) closure
compiler (which is parallel itself in part, but has a very long single-core
phase).
Please test incoming on large codebases, this seems to past our tests, but
is still a big change. In particular let me know if you see issues on
windows (we had some problems with parallelizing the js optimizer last
week, at first).
Aside from bugs, it might be useful to tweak the chunking parameters, see
emscripten.py, and compile with EMCC_DEBUG=1 to see debug output of how
many chunks, cores etc. are being used. Please let me know if you see
anything odd or have suggestions for optimizations based on your project.
Thanks for your job, for my engine, the build with -O2 take 20 minutes, it's better when there was not parallelize (40 minutes) but the parallelize of first compilation step, doesn't change a lot for me.
Le mercredi 7 novembre 2012 03:53:27 UTC+1, azakai a écrit :
> On the incoming branch, we now parallelize even the first compilation step > (ll to js). Together with last week's work, we now parallelize practically > all the compiler, except for (1) a few very short stages, and (2) closure > compiler (which is parallel itself in part, but has a very long single-core > phase).
> Please test incoming on large codebases, this seems to past our tests, but > is still a big change. In particular let me know if you see issues on > windows (we had some problems with parallelizing the js optimizer last > week, at first).
> Aside from bugs, it might be useful to tweak the chunking parameters, see > emscripten.py, and compile with EMCC_DEBUG=1 to see debug output of how > many chunks, cores etc. are being used. Please let me know if you see > anything odd or have suggestions for optimizations based on your project.
Can you run the build with EMCC_DEBUG=1 in the environment and link to a
gist or pastebin of the output (so it prints out how long each step took)?
I want to know which step is the slow one.
On Wed, Nov 7, 2012 at 12:44 AM, wolfviking0 <anthony.l...@gmail.com> wrote:
> Hi Alon,
> Thanks for your job, for my engine, the build with -O2 take 20 minutes,
> it's better when there was not parallelize (40 minutes) but the parallelize
> of first compilation step, doesn't change a lot for me.
> Le mercredi 7 novembre 2012 03:53:27 UTC+1, azakai a écrit :
>> On the incoming branch, we now parallelize even the first compilation
>> step (ll to js). Together with last week's work, we now parallelize
>> practically all the compiler, except for (1) a few very short stages, and
>> (2) closure compiler (which is parallel itself in part, but has a very long
>> single-core phase).
>> Please test incoming on large codebases, this seems to past our tests,
>> but is still a big change. In particular let me know if you see issues on
>> windows (we had some problems with parallelizing the js optimizer last
>> week, at first).
>> Aside from bugs, it might be useful to tweak the chunking parameters, see
>> emscripten.py, and compile with EMCC_DEBUG=1 to see debug output of how
>> many chunks, cores etc. are being used. Please let me know if you see
>> anything odd or have suggestions for optimizations based on your project.
> Can you run the build with EMCC_DEBUG=1 in the environment and link to a > gist or pastebin of the output (so it prints out how long each step took)? > I want to know which step is the slow one.
> -azakai
> On Wed, Nov 7, 2012 at 12:44 AM, wolfviking0 <anthon...@gmail.com<javascript:>
> > wrote:
>> Hi Alon,
>> Thanks for your job, for my engine, the build with -O2 take 20 minutes, >> it's better when there was not parallelize (40 minutes) but the parallelize >> of first compilation step, doesn't change a lot for me.
>> Le mercredi 7 novembre 2012 03:53:27 UTC+1, azakai a écrit :
>>> On the incoming branch, we now parallelize even the first compilation >>> step (ll to js). Together with last week's work, we now parallelize >>> practically all the compiler, except for (1) a few very short stages, and >>> (2) closure compiler (which is parallel itself in part, but has a very long >>> single-core phase).
>>> Please test incoming on large codebases, this seems to past our tests, >>> but is still a big change. In particular let me know if you see issues on >>> windows (we had some problems with parallelizing the js optimizer last >>> week, at first).
>>> Aside from bugs, it might be useful to tweak the chunking parameters, >>> see emscripten.py, and compile with EMCC_DEBUG=1 to see debug output of how >>> many chunks, cores etc. are being used. Please let me know if you see >>> anything odd or have suggestions for optimizations based on your project.
Thanks, yeah, what I meant by gist/pastebin was to link to it or attach it.
Ok, looks like the parallel stuff is working well, should be saving a lot
of time here. The biggest issue left is closure as I suspected. In the
short term, building without closure (--closure 0) is useful for quick
testing builds, in the long term, we will not need closure (the new
eliminator does half of what closure does, a new registerizer will do the
rest, which leaves only minification).
On Wed, Nov 7, 2012 at 8:20 AM, wolfviking0 <anthony.l...@gmail.com> wrote:
> Alon, i add in attachment the console file, but i don't understand " link
> to a gist or pastebin ", but i have the time of each step took
> Le mercredi 7 novembre 2012 16:33:47 UTC+1, azakai a écrit :
>> Can you run the build with EMCC_DEBUG=1 in the environment and link to a
>> gist or pastebin of the output (so it prints out how long each step took)?
>> I want to know which step is the slow one.
>> -azakai
>> On Wed, Nov 7, 2012 at 12:44 AM, wolfviking0 <anthon...@gmail.com> wrote:
>>> Hi Alon,
>>> Thanks for your job, for my engine, the build with -O2 take 20 minutes,
>>> it's better when there was not parallelize (40 minutes) but the parallelize
>>> of first compilation step, doesn't change a lot for me.
>>> Le mercredi 7 novembre 2012 03:53:27 UTC+1, azakai a écrit :
>>>> On the incoming branch, we now parallelize even the first compilation
>>>> step (ll to js). Together with last week's work, we now parallelize
>>>> practically all the compiler, except for (1) a few very short stages, and
>>>> (2) closure compiler (which is parallel itself in part, but has a very long
>>>> single-core phase).
>>>> Please test incoming on large codebases, this seems to past our tests,
>>>> but is still a big change. In particular let me know if you see issues on
>>>> windows (we had some problems with parallelizing the js optimizer last
>>>> week, at first).
>>>> Aside from bugs, it might be useful to tweak the chunking parameters,
>>>> see emscripten.py, and compile with EMCC_DEBUG=1 to see debug output of how
>>>> many chunks, cores etc. are being used. Please let me know if you see
>>>> anything odd or have suggestions for optimizations based on your project.
> Thanks, yeah, what I meant by gist/pastebin was to link to it or attach it.
> Ok, looks like the parallel stuff is working well, should be saving a lot > of time here. The biggest issue left is closure as I suspected. In the > short term, building without closure (--closure 0) is useful for quick > testing builds, in the long term, we will not need closure (the new > eliminator does half of what closure does, a new registerizer will do the > rest, which leaves only minification).
> - azakai
> On Wed, Nov 7, 2012 at 8:20 AM, wolfviking0 <anthon...@gmail.com<javascript:>
> > wrote:
>> Alon, i add in attachment the console file, but i don't understand " link >> to a gist or pastebin ", but i have the time of each step took
>> Le mercredi 7 novembre 2012 16:33:47 UTC+1, azakai a écrit :
>>> Can you run the build with EMCC_DEBUG=1 in the environment and link to a >>> gist or pastebin of the output (so it prints out how long each step took)? >>> I want to know which step is the slow one.
>>> -azakai
>>> On Wed, Nov 7, 2012 at 12:44 AM, wolfviking0 <anthon...@gmail.com>wrote:
>>>> Hi Alon,
>>>> Thanks for your job, for my engine, the build with -O2 take 20 >>>> minutes, it's better when there was not parallelize (40 minutes) but >>>> the parallelize of first compilation step, doesn't change a lot for me.
>>>> Le mercredi 7 novembre 2012 03:53:27 UTC+1, azakai a écrit :
>>>>> On the incoming branch, we now parallelize even the first compilation >>>>> step (ll to js). Together with last week's work, we now parallelize >>>>> practically all the compiler, except for (1) a few very short stages, and >>>>> (2) closure compiler (which is parallel itself in part, but has a very long >>>>> single-core phase).
>>>>> Please test incoming on large codebases, this seems to past our tests, >>>>> but is still a big change. In particular let me know if you see issues on >>>>> windows (we had some problems with parallelizing the js optimizer last >>>>> week, at first).
>>>>> Aside from bugs, it might be useful to tweak the chunking parameters, >>>>> see emscripten.py, and compile with EMCC_DEBUG=1 to see debug output of how >>>>> many chunks, cores etc. are being used. Please let me know if you see >>>>> anything odd or have suggestions for optimizations based on your project.
On Wed, Nov 7, 2012 at 11:26 AM, wolfviking0 <anthony.l...@gmail.com> wrote:
> Sound great, do you know when the new eliminator and registerizer will be
> ready for do some test ?
> Le mercredi 7 novembre 2012 20:07:44 UTC+1, azakai a écrit :
>> Thanks, yeah, what I meant by gist/pastebin was to link to it or attach
>> it.
>> Ok, looks like the parallel stuff is working well, should be saving a lot
>> of time here. The biggest issue left is closure as I suspected. In the
>> short term, building without closure (--closure 0) is useful for quick
>> testing builds, in the long term, we will not need closure (the new
>> eliminator does half of what closure does, a new registerizer will do the
>> rest, which leaves only minification).
>> - azakai
>> On Wed, Nov 7, 2012 at 8:20 AM, wolfviking0 <anthon...@gmail.com> wrote:
>>> Alon, i add in attachment the console file, but i don't understand
>>> " link to a gist or pastebin ", but i have the time of each step took
>>> Le mercredi 7 novembre 2012 16:33:47 UTC+1, azakai a écrit :
>>>> Can you run the build with EMCC_DEBUG=1 in the environment and link to
>>>> a gist or pastebin of the output (so it prints out how long each step
>>>> took)? I want to know which step is the slow one.
>>>> -azakai
>>>> On Wed, Nov 7, 2012 at 12:44 AM, wolfviking0 <anthon...@gmail.com>wrote:
>>>>> Hi Alon,
>>>>> Thanks for your job, for my engine, the build with -O2 take 20
>>>>> minutes, it's better when there was not parallelize (40 minutes) but
>>>>> the parallelize of first compilation step, doesn't change a lot for me.
>>>>> Le mercredi 7 novembre 2012 03:53:27 UTC+1, azakai a écrit :
>>>>>> On the incoming branch, we now parallelize even the first compilation
>>>>>> step (ll to js). Together with last week's work, we now parallelize
>>>>>> practically all the compiler, except for (1) a few very short stages, and
>>>>>> (2) closure compiler (which is parallel itself in part, but has a very long
>>>>>> single-core phase).
>>>>>> Please test incoming on large codebases, this seems to past our
>>>>>> tests, but is still a big change. In particular let me know if you see
>>>>>> issues on windows (we had some problems with parallelizing the js optimizer
>>>>>> last week, at first).
>>>>>> Aside from bugs, it might be useful to tweak the chunking parameters,
>>>>>> see emscripten.py, and compile with EMCC_DEBUG=1 to see debug output of how
>>>>>> many chunks, cores etc. are being used. Please let me know if you see
>>>>>> anything odd or have suggestions for optimizations based on your project.
