Speed test of building Chrome under Windows

Showing 1-31 of 31 messages
Speed test of building Chrome under Windows Arthur Hsu 4/11/11 2:42 PM
f you don't compile Chrome under Windows, feel free to ignore this e-mail.

Summary: use a separate disk to host Chrome source and reduce VC concurrent job count to (number of physical cores - 1) from default (number of logical cores) reduces compilation time on my machine from 1:23:15 to 49:30 (40.5% speed up)

Machine configuration:
Dell Precision T3500, Xeon X5650@2.67 GHz, 12GB of RAM, initially 500GB SATA HD, BitLock disabled
Windows 7 Enterprise
Visual Studio 2008 SP1

Observations:
  • By default, VC sets number of logical cores as max number of parallel project builds (Tools->Options->Project and Solutions->Build and Run).  In my case, it's 12 (6 physical cores and hyper thread)
  • The "max number of parallel project builds" is actually number of VCBuildHelper.exe instances, not the number of concurrent cl.exe.  Similar things happened in VC 2010: this number indicates number of MSBuild.exe instances, not cl.exe.  See attached picture.  
    • Also, each link.exe in the picture is committing high amount of memory (~1G each) and thus 12 instance will invite very high page fault counts.
    • mspdbsrv.exe is the service that synchronize and generate pdb usage during compile time.  It's also obvious from the picture that this is the single bottleneck of compilation speed (high I/O bytes, high memory usage (the working set), ridiculous page fault counts).  

Capture-cl.PNG

Countermeasures:
  • To handle mspdbsrv.exe issue, we need to separate the swap from source so that the I/O and page fault pressure can be separated into two different I/O channels (in my case, a different SATA port).  This is solved by adding a new disk to the system and make sure it's not used for swap.  (Control Panel->System and Security->System->Advanced->Performance/Settings->Advanced->Virtual memory/Change)
  • Reduce the pressure of Windows Virtual Memory manager by reducing concurrent jobs.  In my case, 5 (number of physical cores - 1) is taken from rules of thumb.  It may not be optimal but works very well in my case.
Actual results:

Time required to build chrome.sln (DEBUG|Win32) reduced from 1:23:15 to 49:30.



Re: [chromium-dev] Speed test of building Chrome under Windows Peter Kasting 4/11/11 2:47 PM
On Mon, Apr 11, 2011 at 2:42 PM, Arthur Hsu <arth...@chromium.org> wrote:
Summary: use a separate disk to host Chrome source and reduce VC concurrent job count to (number of physical cores - 1) from default (number of logical cores) reduces compilation time on my machine from 1:23:15 to 49:30 (40.5% speed up)

Currently http://dev.chromium.org/developers/how-tos/build-instructions-windows instructs people to reduce the number of parallel builds based on available RAM and suggests 8 builds for 12 GB.  I'm currently using 6 for my Z600 with 12 GB.  Perhaps this number needs more adjusting or perhaps your and my values are slightly too low.

PK
Re: [chromium-dev] Speed test of building Chrome under Windows Arthur Hsu 4/11/11 2:56 PM
8 builds for 12GB might be a bit bold.  Worst case is 8 linkers each takes ~1G of RAM, and thus only 4G to spare to the rest of system.  Unfortunately, mspdbsrv is not that well behaved and it might take another 2 to 4G, leaving rest of the system underwater.

The other thing to consider is the number of cl.exe spawned.  In my first test, I've spotted over 100 cl.exe instances being created when the number is set to 10.  The picture is not captured because I'm not that fast at clipping the screenshot and the system was not responsive, either.  Anyway lots of CPU cycles will be wasted for context switching among that many cl.exe instances and thus we shall drive the number to a lower range (e.g. 5 or 6).

-Arthur
Re: [chromium-dev] Speed test of building Chrome under Windows Scott Violet 4/11/11 3:01 PM
On Mon, Apr 11, 2011 at 2:42 PM, Arthur Hsu <arth...@chromium.org> wrote:
The killer for me is touch a file (just a .cc) and build chrome. Currently it takes around 4 minutes! Compare this to 40 seconds on my linux box... Both machines are the same (z600, two hard drives) but the windows side has Sophos and Bit9 to contend with.

  -Scott

Re: [chromium-dev] Speed test of building Chrome under Windows Arthur Hsu 4/11/11 3:20 PM
How is the touch done?  Invoking cygwin /usr/bin/touch?  That will definitely take a while since it needs to pass cygwin process launch hack, loads all DLLs, run by Sophos scan, and Bits decryption.  This command shall do it much faster

cmd /c "echo. 2>ph.txt"

Replace the ph.txt with your file path.

-Arthur
Re: [chromium-dev] Speed test of building Chrome under Windows Scott Violet 4/11/11 3:29 PM
By touch I just meant edit the file then do a build. I wasn't timing the actual touch part of it, just the compile.

  -Scott
Re: Speed test of building Chrome under Windows Ryan Norton 4/12/11 8:01 AM
On Apr 11, 2:42 pm, Arthur Hsu <arthur...@chromium.org> wrote:
>    - The "max number of parallel project builds" is actually number of
>    VCBuildHelper.exe instances, not the number of concurrent cl.exe.  Similar
>    things happened in VC 2010: this number indicates number of MSBuild.exe
>    instances, not cl.exe.  See attached picture.

The number of cl/link instances is controlled by the /MP switch and
set to use all your effective processors by default if you don't
specify a number - which is what chromium currently does; see
http://msdn.microsoft.com/en-us/library/bb385193.aspx.  You can
override it by setting 'msvs_multi_core_compile': 0 and manually
setting it yourself through 'msvs_settings' -> 'VCCLCompilerTool' ->
'AdditionalOptions': ['/MP***']; *** would be the max # of cl.exe etc.
processes you want running per project.

For example, your include.gypi file could be this for limiting it to
12 processes per project:
-------------
{
  'variables': {
    'msvs_multi_core_compile': 0,
  },
  'target_defaults': {
    'msvs_settings': {
      'VCCLCompilerTool': {
        'AdditionalOptions': ['/MP12'],
      },
    },
  },
}
---------------
Re: Speed test of building Chrome under Windows Carlos Pizano 4/12/11 7:29 PM
I report 2.5 minutes with VS2010 for a single .cc touch and debug
build.

BTW, VS2010 SP1 builds chrome just fine. I do have the "ultimate"
edition.



On Apr 12, 8:01 am, Ryan Norton <rnor...@gmail.com> wrote:
> On Apr 11, 2:42 pm, Arthur Hsu <arthur...@chromium.org> wrote:
>
> >    - The "max number of parallel project builds" is actually number of
> >    VCBuildHelper.exe instances, not the number of concurrent cl.exe.  Similar
> >    things happened in VC 2010: this number indicates number of MSBuild.exe
> >    instances, not cl.exe.  See attached picture.
>
> The number of cl/link instances is controlled by the /MP switch and
> set to use all your effective processors by default if you don't
> specify a number - which is what chromium currently does; seehttp://msdn.microsoft.com/en-us/library/bb385193.aspx.  You can
Re: [chromium-dev] Re: Speed test of building Chrome under Windows honten 4/13/11 12:22 PM
Hi,

I mainly use Linux dev environment because build time is faster than Windows.

As you know, link time on Windows is extremely slow!!
So I always disabled pdb generation and no opt ref in Release mode manually.

I don't know why the Release setting still enable pdb generation as
default though.
Once disable them, mspdbsrv.exe consumes less memories and link time is shorten.
Of course, it might help for debugger, but I don't use debugger these days.

Thanks,

> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
>    http://groups.google.com/a/chromium.org/group/chromium-dev
>

Re: [chromium-dev] Re: Speed test of building Chrome under Windows Sanjeev 4/13/11 12:38 PM
PDB generation is needed for making crash reports have readable stack traces (and for debugging, of course).
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Steve VanDeBogart 4/13/11 12:41 PM
Should we have a Release-NoSymbol target for faster development builds?

--
Steve
Re: [chromium-dev] Re: Speed test of building Chrome under Windows honten 4/13/11 1:19 PM
Of course, I know PDB generation purpose.

But I agree with Steve. Of course, we can change manually setting, but
it's convenient if we have.

Thanks,

Re: [chromium-dev] Re: Speed test of building Chrome under Windows Brian Salomon 4/13/11 1:44 PM
I've been using the Multi-DLL build and used a tool to boost my system file cache to 2GB. If I touch a file in WebKit (GraphicsContext3DChromium.cpp) and build the Chrome target in Debug from VS 2008 it takes 10 seconds or less to complete. This is with a hot cache (I've built several times recently).

Brian
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Darin Fisher 4/13/11 1:58 PM
yay, multi-dll build!!!  we need to carve up chrome into more dlls to increase these benefits :)
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Scott Violet 4/13/11 2:09 PM
SWEETNESS!
Please share with the rest of us how to set this up.

  -Scott

On Wed, Apr 13, 2011 at 1:44 PM, Brian Salomon <bsal...@google.com> wrote:

Re: [chromium-dev] Re: Speed test of building Chrome under Windows Brian Salomon 4/13/11 2:20 PM
Sure, this is all I did:

To boost the file cache I used this tool:
The max allowed seems to be 2GB. My Z600 system has 12GB of RAM.

To enable the Multi-DLL build I put:
{'variables': {'component': 'shared_library'}}
in c:\users\<my username>\.gyp\include.gypi
I had to blow-away my <chrome>\src\build\Debug and <chrome>\src\build\Release dirs before the first builds after switching to Multi-DLL.

I also have my chrome tree on a different drive than my OS.

Brian
Re: [chromium-dev] Re: Speed test of building Chrome under Windows honten 4/13/11 3:07 PM
Cool!!

I'll try it. Although my machine only has 4GB, I wish it will help...

Re: [chromium-dev] Re: Speed test of building Chrome under Windows vangelis 4/19/11 1:14 PM
Just wanted to mention that switching to the multi-dll build in this past couple of weeks has been a tremendous productivity boost.  Not only the code builds faster (the webkit library links in under 10 secs) but also now the VS debugger seems to be able to load and run chrome within seconds.  If windows is your primary development platform, do yourself a favor and give it a try!

Many kudos to Brian for bringing it up. 

Vangelis
PS I also used the utility that Brian mentions to set the file cache size to 2GB.  I don't know how much that plays into the speed improvement.
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Mike Reed 4/19/11 1:19 PM
What bot will exercise this? i.e. gcl try foo --bot=?
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Lei Zhang 4/19/11 1:21 PM
I don't think we have shared configuration try bots for either Windows or Linux.
Re: [chromium-dev] Re: Speed test of building Chrome under Windows vangelis 4/19/11 1:32 PM
Isn't it this one:


?
Re: [chromium-dev] Re: Speed test of building Chrome under Windows vangelis 4/19/11 1:33 PM
Never mind, I just saw you were talking about try bots....
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Scott Violet 4/19/11 2:00 PM
Agreed, but when the multi-dll build was broke for the better part of
a day last week I lost faith. Can we treat the multi-dll build like
any other compile bots?

  -Scott

On Tue, Apr 19, 2011 at 1:14 PM, Vangelis Kokkevis <vang...@google.com> wrote:

Re: [chromium-dev] Re: Speed test of building Chrome under Windows John Abd-El-Malek 4/19/11 2:03 PM
Does anyone know if there's a way to enable it per checkout?
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Mike Reed 4/19/11 2:16 PM
+1 for adding multi-dlls to the try suite
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Ricardo Vargas 4/19/11 3:41 PM


On Tue, Apr 19, 2011 at 2:03 PM, John Abd-El-Malek <j...@chromium.org> wrote:
Does anyone know if there's a way to enable it per checkout?

Edit build/common.gypi and change the value of component to be
   'component%': 'shared_library'

(and remember not to commit that)

Re: [chromium-dev] Re: Speed test of building Chrome under Windows John Abd-El-Malek 4/19/11 4:09 PM
Thanks!
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Darin Fisher 4/19/11 4:15 PM
On Tue, Apr 19, 2011 at 3:41 PM, Ricardo Vargas <rva...@chromium.org> wrote:


On Tue, Apr 19, 2011 at 2:03 PM, John Abd-El-Malek <j...@chromium.org> wrote:
Does anyone know if there's a way to enable it per checkout?

Edit build/common.gypi and change the value of component to be
   'component%': 'shared_library'

(and remember not to commit that)


^^^ trick:  "gcl change do_not_commit", and add that file to it.  now that file will never accidentally appear in another CL :)

Re: [chromium-dev] Re: Speed test of building Chrome under Windows John Bauman 4/19/11 4:21 PM
"git update-index --assume-unchanged build/common.gypi" also helps, though I'm not sure how reliably.
Re: [chromium-dev] Re: Speed test of building Chrome under Windows Ricardo Vargas 4/19/11 5:43 PM


On Tue, Apr 19, 2011 at 4:15 PM, Darin Fisher <da...@chromium.org> wrote:
On Tue, Apr 19, 2011 at 3:41 PM, Ricardo Vargas <rva...@chromium.org> wrote:


On Tue, Apr 19, 2011 at 2:03 PM, John Abd-El-Malek <j...@chromium.org> wrote:
Does anyone know if there's a way to enable it per checkout?

Edit build/common.gypi and change the value of component to be
   'component%': 'shared_library'

(and remember not to commit that)


^^^ trick:  "gcl change do_not_commit", and add that file to it.  now that file will never accidentally appear in another CL :)


I forgot to mention that you have to do the same thing to src/native_client/build

Re: [chromium-dev] Re: Speed test of building Chrome under Windows Ricardo Vargas 4/19/11 5:44 PM


On Tue, Apr 19, 2011 at 5:43 PM, Ricardo Vargas <rva...@chromium.org> wrote:


On Tue, Apr 19, 2011 at 4:15 PM, Darin Fisher <da...@chromium.org> wrote:
On Tue, Apr 19, 2011 at 3:41 PM, Ricardo Vargas <rva...@chromium.org> wrote:


On Tue, Apr 19, 2011 at 2:03 PM, John Abd-El-Malek <j...@chromium.org> wrote:
Does anyone know if there's a way to enable it per checkout?

Edit build/common.gypi and change the value of component to be
   'component%': 'shared_library'

(and remember not to commit that)


^^^ trick:  "gcl change do_not_commit", and add that file to it.  now that file will never accidentally appear in another CL :)


I forgot to mention that you have to do the same thing to src/native_client/build
 
src/native_client/build/common.gypi

More topics »