GYP Support for MSVS2010

[Bradley Nelson]

Hi All,

We now have a Visual Studio 2010 Chromium build generated by GYP!

This is the result of a bunch of hard work by jeanluc.

Please join me in thanking him for his efforts.

This actually landed about a week ago, but we now have a 2010 builder up on the main waterfall.

The builder currently builds just chrome.exe and dependent targets, but I'll be switching more on gradually in the coming days.

There are a few outstanding issues including one with chrome_frame and at least one issue with resources in the resulting browser, but most things seem to be functioning.

I believe the tree gate should close on compile breakages, but nonetheless, please try to help us keep it green.

If you find you've broken the 2010 builder and the failure isn't obvious, please ping me and I'll assist or point you at someone who's got the build up on their desk.

Normally you shouldn't need to do anything special for 2010 in the gyp files (it converts msvs_settings for instance internally).

The primary source of incompatibilities will be compiler differences.

You may have also noticed a new warning coming from gyp (typically shows up during gclient update):
Warning: Missing input file ...

(Which notes files which appear in the gyp files, but don't actually exist).

Ideally this will soon be an error.

If you have 2010 and would like to give it a spin, just set GYP_MSVS_VERSION=2010 in your environment and run gclient runhooks.

I have not yet vetted SP1, and have at least one report that it may have issues. So if you don't have it already, please hold off.

You can safely install 2010 side by side with 2008. I haven't tried 2005 recently, but in the past this also had issues.

Please file issues as you find them, in the chromium issue tracker (gyp tracker if it's definitely gyp specific).

You can assign them to me and I'll triage them.

-BradN

[William Chan (陈智昌)]

Exciting! When are we switching to VS 2010 and can deprecate support for the older VS versions? Or is that not even on the horizon yet?

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Ryan Norton]

Yeah, thanks for the hard work on it - I've even been using 2010
express for about 2 weeks now on trunk;
Notes here: http://groups.google.com/a/chromium.org/group/chromium-dev/browse_thread/thread/5ff5326b8a600eac

Some general 2010 tips:
1) For those who haven't had the pleasure of setting up 2010 for
building a project the size of Chromium, you're in for a treat. 2010
uses a lightly documented hierarchy of property sheets for include
paths and such (the old Tools->Options->Projects and Solutions->VC++
Directories is gone in favor of property sheets due to "per-user"
issues):
1a)The main one is in: [drive]:\Program Files\MSBuild\Microsoft.Cpp
\v4.0\Platforms\[Arch - "Win32" for 32-bit for example]\Microsoft.Cpp.
[Arch].props (note that if you have the WinSDK installed it will have
it's OWN property sheet) [NOTE: This is not documented much at all]
1b)The second in the pyrimid is the user property sheet that you can
edit through the 3rd tab in the property manager within MSVC - this
one is actually global even though it is noted in each project. You
can also edit it directly through [drive]:\Users\[user]\AppData\Local
\Microsoft\MSBuild\v4.0\Microsoft.Cpp.[Arch].user.props
1c)What you edit through each individual project file through the
property sheets through "VC++ Directories" - this will edit it for the
project only and will inherit from 1a and 1b.

2) If you have even a single bad include path if you have tracking
enabled (the incremental resource compiler feature in 2010, i.e.
Tracker.exe), it will spit out a generic "Cannot find path [generally
blank]". The "generally blank" part is the part is what you'll learn
to love to hate - turn on diagnostic logging for MSBuild won't help
you either. You can turn incremental compilation off via
<TrackFileAccess>. On one hand, it's generally not the best idea to do
this since it messes with source incremental compilation. On the other
hand, Tracker.exe is as smart as a sack of bricks when it comes to
building Chromium in my experience (likely due to Tracker.exe and
scripts), forcing a complete clobber most of the time anyway. The
biggest problem getting past the dreaded Tracker.exe path not found
errors is generally which WinSDK it uses. There are so many property
sheets that could set the directory, it could end up some old version
etc...

Yet another Tracker.exe note: as mentioned, there are some resource
compilation problems that creep up sometimes. Turning off
<TrackFileAccess> "fixes" that as well, but except longer builds.

3) Don't install SP1 unless you willing to endure some pain - if don't
have 2010 Ultimate it will delete your 64-bit compilers (no joke)
among other things if you had a WinSDK installed beforehand, and
likely mess up property sheets. It's a known bug that _will_ happen -
Microsoft recently came out with a "Post-SP1 compiler re-
installer" (my words), but that installer tends to crash on various
machine.... so take that for what you will. However, you can also get
the WinSDK 64-bit compilers back... by simply uninstalling and
reinstalling the entire WinSDK.
---------------

Use chrome\chrome.sln.

Chrome_frame (specifically npchrome_frame project) as mentioned has
linker errors among other things. In the thread about Express I give a
hack for getting around it for now. It's likely a conflict with
multiple projects linking to third_party\nss\mozilla\nsprpub\pr\src
\threads\prmon.c and related files.

Besides that, I've got pretty much all projects to build on machine,
with express as well. Generally though I just build mini_installer,
which works fine.

[Alexei Svitkine]

Are there any improvements in build times over VS2008?

Have we done any measurements of the performance of the resulting binaries when compared to VS2008?

-Alexei

[Jean-Luc Brouillet]

On Thu, Apr 7, 2011 at 12:26 AM, Bradley Nelson <bradn...@chromium.org> wrote:

[...]

You may have also noticed a new warning coming from gyp (typically shows up during gclient update):
Warning: Missing input file ...

(Which notes files which appear in the gyp files, but don't actually exist).

Ideally this will soon be an error.

In VisualStudio 2010, projects with missing input files are always recompiled.  This really slows down the incremental builds.

Jean-Luc

[Bradley Nelson]

On Thu, Apr 7, 2011 at 1:56 AM, William Chan (陈智昌) <will...@chromium.org> wrote:

Exciting! When are we switching to VS 2010 and can deprecate support for the older VS versions? Or is that not even on the horizon yet?

We need to assess the impact. We'll almost certainly switch eventually.

Past precedent (with 2008) suggests that when the transition happens it'll be abrupt and prompted by something capricious. 2008 support sat ready for over a year as I recall and then suddenly everyone started using it when a bunch of folks on the team got new machines (and so were reconfiguring their systems from scratch).

I should mention by the way, and this bears emphasis, that while 2008 support was relatively simple to add (update some rev numbers in the files and fix a few compiler issues), 2010 support was a major overhall. The project file format has switched to msbuild (a completely distinct format) and jeanluc needed to create a mapping between settings in the two formats. A good bit of careful meticulous work.

One thing that may prompt a switch is that 2010 (really msbuild) is much more amenable to a fully hermetic toolset. There is strong anecdotal evidence that you can prepare a copy of the tools that could come from a source control checkout. This would simplify tool deployment/upgrade, particularly to the buildbots. Obviously we would not be able to distribute such a hermetic package external to Google. However, additionally, the needed tools (at least for a command line build), are present in a combination of the windows sdk (compiler, linker) + .net framework (msbuild). So we may be able to come up with a simpler setup recipe in general. Also, Ryan Norton's experience (and one new guy here, sorry forgot his name..) suggest that 2010 express is relatively useable. 

By the way Ryan, does express still flatten the .sln tree?

-BradN

[Bradley Nelson]

On Thu, Apr 7, 2011 at 7:16 AM, Alexei Svitkine <asvi...@chromium.org> wrote:

Are there any improvements in build times over VS2008?

 

Have we done any measurements of the performance of the resulting binaries when compared to VS2008?

I believe jeanluc made some attempts, but its a little tricky to get a good apples to apples comparison (given disk caching etc).

My offhand sense is that its roughly the same, maybe slightly faster to build. I'd say the ui is a little more sluggish in general, but there are a few places where 2008 becomes unresponsive that 2010 at least responds to ui events.

Now that we're marginally up on the bots, we should soon have a sense on the build performance. I need to enlarge the set of what's building to really make it match. We'll probably get a good sense once the 2010 builder is an exact clone of win dbg.

As for the performance of the resulting program, we haven't even started. I'll need help from chase and others to figure out the best way to start pushing numbers we can compare.

-BradN

[Ryan Norton]

Everything Brad says is right, 2008 is for the most part 2005 with
improvements. 2010 is a whole new breed.

> By the way Ryan, does express still flatten the .sln tree?

Yeah, my code review has that in it. Tried and tested.

My own opinons about 2010 in general:
1) It's a fair amount faster compiling on low-memory machines, and bit
faster overall in my experience.
2) While mentioning it rebuilds projects with missing input files but
it's also a bit dumber in general when choosing what to incrementally
build. Be prepared to do a full clean (or "clobber" as the Chromium
folks call it) often.
3) It's slightly buggy and really a pain to set up on some machines,
and for whatever reason a lot of these are being pushed unto the next
MSVS release. I expect the next one to be a more usable version for
users then 2010 (which, as hinted, often involves editing raw XML
files... over and over again relaunching MSVS each time... just to set
up your include paths and such).

By the way, you can make the UI about as fast (or faster, next
sentence on that) as 2008 if you turn off "Rich Client Experience" or
something... it's in the options someplace and on by default. No, you
probably won't notice any difference. 2010 also has GPU acceleration
which helps offset the rest of the lag the UI has.... unless you have
a bad video card (guessing that's the case with the Google
workstations).

Compiled binaries are slightly smaller, slightly faster - nothing
dramatic.

[Carlos Pizano]

Brad just showed me in the waterfall the VS2010 (Debug) bot and next
to it to the left the VS2008 (Debug) bot, running on comparable
hardware. The VS2010 cycles in ~4 minutes and the other in ~12
minutes. Maybe we are misreading this but it seems it is much faster
at building chrome.

[mme...@chromium.org]

I don't think the VS2010 bot is building everything. One commit broke
compilation of unit_tests on every platform earlier today, but VS2010
was green throughout. Looking at its log, I saw no mention of the
file.

On Apr 7, 6:51 pm, Carlos Pizano <c...@chromium.org> wrote:
> Brad just showed me in the waterfall the VS2010 (Debug) bot and next
> to it to the left the VS2008 (Debug) bot, running on comparable
> hardware. The VS2010 cycles in ~4 minutes and the other in ~12
> minutes. Maybe we are misreading this but it seems it is much faster
> at building chrome.
>

[Bradley Nelson]

On Thu, Apr 7, 2011 at 4:37 PM, mme...@chromium.org <mme...@chromium.org> wrote:

I don't think the VS2010 bot is building everything.  One commit broke
compilation of unit_tests on every platform earlier today, but VS2010
was green throughout.  Looking at its log, I saw no mention of the
file.

Yes it's not apples to apples the same. There's a much larger target list for the other one.

I'll be switching on more soon. And then we'll get a better idea.

-BradN

 

[Albert J. Wong (王重傑)]

Out of curiosity, did this result in any compiler setting/flag changes for VS2008?

I have a couple week old CL that used to run through the try-servers fine, but is not throwing a new compiler warning.

http://codereview.chromium.org/6463013/

Patchset 19 passwed win and win_sync, but Patchset 21 no longer does even though that portion of code should not have changed.

Wondering if it might be related...

-Albert

[Albert J. Wong (王重傑)]

On Sun, Apr 10, 2011 at 9:38 AM, Albert J. Wong (王重傑) <ajw...@chromium.org> wrote:

Out of curiosity, did this result in any compiler setting/flag changes for VS2008?

I have a couple week old CL that used to run through the try-servers fine, but is not throwing a new compiler warning.

That was supposed to be:

I have a couple week old CL that used to run through the try-servers fine, but is *now* throwing a new compiler warning.

[Bradley Nelson]

The msvs2010 changes should not impact the 2008/5 build and thus the trybots.

In the process of adding 2010 support, better detection of mal-formed gyp files was added, which allowed us to clean up a few places.

But this was mostly minor stuff (non-existant files, msvs_settings at the wrong layer of nesting).

Unlikely to have caused what you're seeing.

What happens if you add -r <some_old_rev> to the try job?

-BradN

[John Abd-El-Malek]

btw I'm curious if anyone else sees "Preparing solution..." for half a minute or so each time they open the solution?  My checkout was originally for VS2008 and I added the environment variable to switch it to 2010.

[Arthur Hsu]

Yes I saw that too.

I also did an inaccurate stop watch timing. The preparing solution on
VS2010/Intel Core i7 takes 28 seconds, while my VS2008/Xeon 5650 takes
44 seconds to load the whole solution without *any* UI feedback. In
my case VS2010 is actually faster.

-Arthur