Switching to Visual Studio 2013

[David Major]

We plan to switch the Windows build machines to Visual Studio 2013 on the Firefox 35 train.

Some benefits from this change:
* No more linker OOM crashes. VS2013 includes a 64-bit toolchain for 32-bit builds, so the linker will no longer be limited to 4GB address space.
* The linker capacity opens the door for merging our binaries into libxul (like we do on the other platforms)
* More than 2x improvement in PGO build times
* Better language support

The tracking bug is 914596. The remaining blockers are here: https://bugzilla.mozilla.org/showdependencytree.cgi?id=914596&hide_resolved=1

David

[xunxun]

And we should use VC2013 update2 or newer edition, whose PGO is faster than
WPO.

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>



--
Best Regards,
xunxun

[Benjamin Smedberg]

On 8/22/2014 5:04 AM, xunxun wrote:
> And we should use VC2013 update2 or newer edition, whose PGO is faster than
> WPO.

Yes, we had to wait for update 2 for fixes that would allow Firefox PGO
builds to complete at all (there were previously internal compile errors
during the link phase). Update 2 is already deployed to all of our
windows builders and David is doing try builds and working on the last
few issues from that.

--BDS

[Gijs Kruitbosch]

Do we have a configure check for this so people trying this won't waste
oodles of time only to get internal errors? :-)

~ Gijs

[Kyle Huey]

On Thu, Aug 21, 2014 at 11:27 PM, David Major <dma...@mozilla.com> wrote:
> We plan to switch the Windows build machines to Visual Studio 2013 on the Firefox 35 train.
>
> Some benefits from this change:
> * No more linker OOM crashes. VS2013 includes a 64-bit toolchain for 32-bit builds, so the linker will no longer be limited to 4GB address space.
> * The linker capacity opens the door for merging our binaries into libxul (like we do on the other platforms)
> * More than 2x improvement in PGO build times
> * Better language support
>
> The tracking bug is 914596. The remaining blockers are here: https://bugzilla.mozilla.org/showdependencytree.cgi?id=914596&hide_resolved=1
>
> David
> _______________________________________________

> firefox-dev mailing list
> firef...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev

Awesome! Thanks for taking this on.

- Kyle

[Benjamin Smedberg]

On 8/22/2014 10:34 AM, Gijs Kruitbosch wrote:
>
> Do we have a configure check for this so people trying this won't
> waste oodles of time only to get internal errors? :-)

No we don't. It only affects PGO builds, which no normal person ever
does, so I think the relative effort of implementing and testing a
configure check probably isn't worth it in this case.

--BDS

[Nicholas Nethercote]

On Fri, Aug 22, 2014 at 4:27 PM, David Major <dma...@mozilla.com> wrote:
> We plan to switch the Windows build machines to Visual Studio 2013 on the Firefox 35 train.
>
> Some benefits from this change:
> * No more linker OOM crashes. VS2013 includes a 64-bit toolchain for 32-bit builds, so the linker will no longer be limited to 4GB address space.

This is wonderful news. Thank you!

Nick

[Neil]

David Major wrote:

>* No more linker OOM crashes. VS2013 includes a 64-bit toolchain for 32-bit builds, so the linker will no longer be limited to 4GB address space.
>

So will you be requiring 64-bit builders?

--
Warning: May contain traces of nuts.

[Benjamin Smedberg]

On 8/23/2014 6:15 AM, Neil wrote:
> David Major wrote:
>
>> * No more linker OOM crashes. VS2013 includes a 64-bit toolchain for
>> 32-bit builds, so the linker will no longer be limited to 4GB address
>> space.
>>
> So will you be requiring 64-bit builders?
>

For PGO builds, probably yes. For normal developer builds this is not an
issue.

--BDS

[Makoto Kato]

When do we support old version of Visual Studio after switching to VS2013?

Now we still support VS2010 + SDK 7.1 as minimal requirement.

[Ted Mielczarek]

On 8/26/2014 2:03 AM, Makoto Kato wrote:
> When do we support old version of Visual Studio after switching to
> VS2013?
>
> Now we still support VS2010 + SDK 7.1 as minimal requirement.
>

Generally we have worked with a few unofficial guidelines for toolchain
support:
1) If an older version of a toolchain is holding us back from using some
C++ feature in the codebase, we may drop support for it so we can use
more modern C++
2) If the cost of supporting an older toolchain in terms of its bugs
becomes excessive we may drop support for it
3) Try to support toolchains that are widely used by developers

We generally don't drop support for a toolchain immediately upon
switching, we ought to have a grace period of a release or two to give
other developers time to make the change if they've been following our
lead. Even then, I'm not sure if VS2010 is the thing holding us back
from using new C++ features, or if one of the B2G GCC compilers is
holding us back.

-Ted

[Ted Mielczarek]

On 8/26/2014 2:03 AM, Makoto Kato wrote:

> When do we support old version of Visual Studio after switching to
> VS2013?
>
> Now we still support VS2010 + SDK 7.1 as minimal requirement.
>

[Mike Hommey]

On Tue, Aug 26, 2014 at 06:57:15AM -0400, Ted Mielczarek wrote:
> On 8/26/2014 2:03 AM, Makoto Kato wrote:

> > When do we support old version of Visual Studio after switching to
> > VS2013?
> >
> > Now we still support VS2010 + SDK 7.1 as minimal requirement.
> >

> Generally we have worked with a few unofficial guidelines for toolchain
> support:
> 1) If an older version of a toolchain is holding us back from using some
> C++ feature in the codebase, we may drop support for it so we can use
> more modern C++
> 2) If the cost of supporting an older toolchain in terms of its bugs
> becomes excessive we may drop support for it
> 3) Try to support toolchains that are widely used by developers
>
> We generally don't drop support for a toolchain immediately upon
> switching, we ought to have a grace period of a release or two to give
> other developers time to make the change if they've been following our
> lead. Even then, I'm not sure if VS2010 is the thing holding us back
> from using new C++ features, or if one of the B2G GCC compilers is
> holding us back.

Relatedly, we need to ensure it's still possible to build Firefox with
the express version of MSVC (the free of charge one) corresponding to
the minimum MSVC version we support.

Mike

[Ehsan Akhgari]

On 2014-08-26, 6:57 AM, Ted Mielczarek wrote:
> On 8/26/2014 2:03 AM, Makoto Kato wrote:

>> When do we support old version of Visual Studio after switching to
>> VS2013?
>>
>> Now we still support VS2010 + SDK 7.1 as minimal requirement.
>>

> Generally we have worked with a few unofficial guidelines for toolchain
> support:
> 1) If an older version of a toolchain is holding us back from using some
> C++ feature in the codebase, we may drop support for it so we can use
> more modern C++
> 2) If the cost of supporting an older toolchain in terms of its bugs
> becomes excessive we may drop support for it
> 3) Try to support toolchains that are widely used by developers
>
> We generally don't drop support for a toolchain immediately upon
> switching, we ought to have a grace period of a release or two to give
> other developers time to make the change if they've been following our
> lead. Even then, I'm not sure if VS2010 is the thing holding us back
> from using new C++ features, or if one of the B2G GCC compilers is
> holding us back.

I would like us to update the minimum supported MSVC version to 2012 as
soon as possible. That will give us access to the following C++
features which are all supported on gcc 4.4 (aka our Vintage Compiler)
and MSVC starting from 2012:

* Variadic templates
* Strongly typed enums
* Initializer lists
* The ability to use decltype for non-trivial things (MSVC2010's
decltype support is pretty buggy)

I doubt that there is a great reason to continue to support MSVC2010.

Cheers,
Ehsan

[Ted Mielczarek]

On 8/26/2014 11:03 AM, Ehsan Akhgari wrote:
>
> I would like us to update the minimum supported MSVC version to 2012
> as soon as possible. That will give us access to the following C++
> features which are all supported on gcc 4.4 (aka our Vintage Compiler)
> and MSVC starting from 2012:
>
> * Variadic templates
> * Strongly typed enums
> * Initializer lists
> * The ability to use decltype for non-trivial things (MSVC2010's
> decltype support is pretty buggy)
>
> I doubt that there is a great reason to continue to support MSVC2010.
>

This is fair. I'd say we keep MSVC2010 support for one cycle on trunk
after we make the switch to 2013, for cautiousness (allowing us to
switch back easily if we find a blocker) and to give a little lead time
to other developers. After that I think supporting only VC 2012+2013 is
reasonable.

-Ted

[Lars Hansen]

> From: "Ehsan Akhgari" <ehsan....@gmail.com>
> Sent: Tuesday, August 26, 2014 5:03:31 PM
> Subject: Re: Switching to Visual Studio 2013

>
> I would like us to update the minimum supported MSVC version to 2012 as
> soon as possible. That will give us access to the following C++
> features which are all supported on gcc 4.4 (aka our Vintage Compiler)
> and MSVC starting from 2012:
>
> * Variadic templates
> * Strongly typed enums
> * Initializer lists
> * The ability to use decltype for non-trivial things (MSVC2010's
> decltype support is pretty buggy)

As well as the C++11 <atomic> subsystem, which did not appear until MSVC2012:
http://msdn.microsoft.com/en-us/library/hh874894%28v=vs.110%29.aspx

--lars

[Joshua Cranmer 🐧]

On 8/26/2014 10:09 AM, Ted Mielczarek wrote:
> On 8/26/2014 11:03 AM, Ehsan Akhgari wrote:
>> I would like us to update the minimum supported MSVC version to 2012
>> as soon as possible. That will give us access to the following C++
>> features which are all supported on gcc 4.4 (aka our Vintage Compiler)
>> and MSVC starting from 2012:
>>
>> * Variadic templates

This is 2013, actually...

>> * Strongly typed enums
>> * Initializer lists

... as is this.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

[Ehsan Akhgari]

On 2014-08-26, 11:09 AM, Ted Mielczarek wrote:
> On 8/26/2014 11:03 AM, Ehsan Akhgari wrote:
>>
>> I would like us to update the minimum supported MSVC version to 2012
>> as soon as possible. That will give us access to the following C++
>> features which are all supported on gcc 4.4 (aka our Vintage Compiler)
>> and MSVC starting from 2012:
>>
>> * Variadic templates

>> * Strongly typed enums
>> * Initializer lists

>> * The ability to use decltype for non-trivial things (MSVC2010's
>> decltype support is pretty buggy)
>>

>> I doubt that there is a great reason to continue to support MSVC2010.
>>
> This is fair. I'd say we keep MSVC2010 support for one cycle on trunk
> after we make the switch to 2013, for cautiousness (allowing us to
> switch back easily if we find a blocker) and to give a little lead time
> to other developers. After that I think supporting only VC 2012+2013 is
> reasonable.

Sounds like a good idea.

[Ehsan Akhgari]

On 2014-08-26, 11:16 AM, Lars Hansen wrote:
>> From: "Ehsan Akhgari" <ehsan....@gmail.com>
>> Sent: Tuesday, August 26, 2014 5:03:31 PM
>> Subject: Re: Switching to Visual Studio 2013
>>

>> I would like us to update the minimum supported MSVC version to 2012 as
>> soon as possible. That will give us access to the following C++
>> features which are all supported on gcc 4.4 (aka our Vintage Compiler)
>> and MSVC starting from 2012:
>>
>> * Variadic templates
>> * Strongly typed enums
>> * Initializer lists
>> * The ability to use decltype for non-trivial things (MSVC2010's
>> decltype support is pretty buggy)
>

> As well as the C++11 <atomic> subsystem, which did not appear until MSVC2012:
> http://msdn.microsoft.com/en-us/library/hh874894%28v=vs.110%29.aspx

We cannot use most new library features due to the lack of support in
stlport which we use on b2g.

[Ehsan Akhgari]

On 2014-08-26, 11:29 AM, Joshua Cranmer 🐧 wrote:
> On 8/26/2014 10:09 AM, Ted Mielczarek wrote:
>> On 8/26/2014 11:03 AM, Ehsan Akhgari wrote:
>>> I would like us to update the minimum supported MSVC version to 2012
>>> as soon as possible. That will give us access to the following C++
>>> features which are all supported on gcc 4.4 (aka our Vintage Compiler)
>>> and MSVC starting from 2012:
>>>
>>> * Variadic templates
> This is 2013, actually...
>>> * Strongly typed enums
>>> * Initializer lists
> ... as is this.

Really? I was quoting from
<http://wiki.apache.org/stdcxx/C++0xCompilerSupport>...

[Joshua Cranmer 🐧]

I just tried using variadic templates in my 2012 (non-CTP) install a
week or so ago and it blew up in my face. The lines have (nov'12) which
indicate the November CTP, not the standard install.

[Ehsan Akhgari]

Err, sorry, I was talking about MSVC2012 + the November CTP. We
absolutely don't want to support older versions of 2012 (or 2013 for
that matter.)

[Neil]

Ehsan Akhgari wrote:

> I was talking about MSVC2012 + the November CTP. We absolutely don't
> want to support older versions of 2012 (or 2013 for that matter.)

What does that mean and why isn't it mentioned on MDN?

[Joshua Cranmer 🐧]

On 8/26/2014 6:20 PM, Neil wrote:
> Ehsan Akhgari wrote:
>
>> I was talking about MSVC2012 + the November CTP. We absolutely don't
>> want to support older versions of 2012 (or 2013 for that matter.)
>
> What does that mean and why isn't it mentioned on MDN?
>

The MSVC development team announced in 2012 that they would be working
on bringing new C++11 features to Visual Studio faster via out-of-band
Consumer Technology Previews (CTPs for short). I hadn't bothered to list
CTP as minimum features because:
1. My understanding is that the CTP is basically intended to be
alpha-quality releases.
2. The official guides on CTP explicitly advise against relying on them
for production purposes.
3. They are not as easy to get installed as VS (they don't autoinstall
like service packs, e.g.)
4. I thought it would make the page overly complicated.
5. When MSVS announced more frequent releases, I assumed that the need
to worry about CTPs was minimal.

FWIW, I'm not entirely sure that a minimum dependency specifically on a
CTP is a terribly good idea.

[Ehsan Akhgari]

Please note that the compiler only needs to be as good as giving
developers a build that works well enough to use in their local dev
environment. And even that should be a transitional period. Really,
all developers should install MSVC 2013 once we switch. There is no
reason to not do that, given the availability of MSVC 2013 Express
<http://www.visualstudio.com/downloads/download-visual-studio-vs#d-express-windows-desktop>.

We are not talking about shipping anything with that compiler.

Cheers,
Ehsan

[Gian-Carlo Pascutto]

On 26/08/2014 13:06, Mike Hommey wrote:

> Relatedly, we need to ensure it's still possible to build Firefox with
> the express version of MSVC (the free of charge one) corresponding to
> the minimum MSVC version we support.

It seems to work fine with MSVC2013 Express - I'm using it for
development now.

One issue related to the discussion in bug 914596 is that it's harder to
find the non-latest-version of the Express version. That is, if you
install Express now, you get SP3.

It's probably possible to get SP2 anyhow, but I couldn't easily find a way.

--
GCP

[Mike Hoye]

On 2014-08-27 10:22 AM, Gian-Carlo Pascutto wrote:
> One issue related to the discussion in bug 914596 is that it's harder to
> find the non-latest-version of the Express version. That is, if you
> install Express now, you get SP3.
>
> It's probably possible to get SP2 anyhow, but I couldn't easily find a way.

It shouldn't be more than one click to get from an MDN getting-started
page to the software you need.

I'll ask Mike Connor about this, but I _think_ we have the kind of
relationship with Microsoft where we can ask them for a static link to
some version of VSExpress when the time comes, so that it's easy for our
contributors to get spun up.

- mhoye

[Ted Mielczarek]

I think our commitment to the Express versions should be "ensure it
works with whatever Microsoft makes it easy to get a hold of". We should
fix bugs that impact developers' ability to build with the versions of
the toolchain we support, but I haven't seen that be a huge problem in
practice. (I'm not sure why you particularly care about SP2 vs. SP3
here, we should just make sure we build with the latest version.)

-Ted

[Gian-Carlo Pascutto]

On 27/08/2014 17:30, Ted Mielczarek wrote:

> I think our commitment to the Express versions should be "ensure it
> works with whatever Microsoft makes it easy to get a hold of". We should
> fix bugs that impact developers' ability to build with the versions of
> the toolchain we support, but I haven't seen that be a huge problem in
> practice.
> (I'm not sure why you particularly care about SP2 vs. SP3
> here, we should just make sure we build with the latest version.)

There's compelling advantages to be compiling with the exact same
toolchain as on the buildbots: you won't end up writing code that the
buildbot barfs on because it's on a newer/older toolchain, and if you
have a failure on try that might be compiler related, the odds are
bigger that you will be able to reproduce the exact issue locally if the
compiler matches to begin with.

These issues aren't common, so indeed they're not a huge problem in
practice, but they're still nice-to-haves.

--
GCP

[Ehsan Akhgari]

Are we building with SP2 on our buildbots? If yes, we should probably
try to switch to SP3 as soon as we can.

[Jeff Gilbert]

> From: "Gian-Carlo Pascutto" <gpas...@mozilla.com>
> To: dev-pl...@lists.mozilla.org
> Sent: Wednesday, August 27, 2014 11:23:34 AM

> Subject: Re: Switching to Visual Studio 2013
>

I don't find this particularly compelling, since invariably the buildbots that barf on me is B2G or OSX.
I can't remember a time when my Windows build worked on Try but not on my Windows box, or vice-versa.
It should be possible to acquire the same toolchain, but it's not that important for normal development.

[xunxun]

And after switching to VS2013, we should enable AVX2 on libvpx then,
because libvpx has some AVX2 optimization code.

On Fri, Aug 22, 2014 at 2:27 PM, David Major <dma...@mozilla.com> wrote:

> We plan to switch the Windows build machines to Visual Studio 2013 on the
> Firefox 35 train.
>
> Some benefits from this change:
> * No more linker OOM crashes. VS2013 includes a 64-bit toolchain for
> 32-bit builds, so the linker will no longer be limited to 4GB address space.
> * The linker capacity opens the door for merging our binaries into libxul
> (like we do on the other platforms)
> * More than 2x improvement in PGO build times
> * Better language support
>
> The tracking bug is 914596. The remaining blockers are here:
> https://bugzilla.mozilla.org/showdependencytree.cgi?id=914596&hide_resolved=1
>
> David

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>



--
Best Regards,
xunxun

[David Major]

Please open bugs for this kind of thing. I filed 1061339 for you.

----- Original Message -----
> From: "xunxun" <xunxu...@gmail.com>
> To: "David Major" <dma...@mozilla.com>
> Cc: dev-pl...@lists.mozilla.org, firef...@mozilla.org
> Sent: Monday, September 1, 2014 9:01:19 PM
> Subject: Re: Switching to Visual Studio 2013
>

[David Major]

Update: the switch will happen early in version 36.

The build machines needed a reinstall of VS2013 due to a deployment issue. As merge day is coming up, it's not a good time for major changes. We'll wait for 36 to have more bake time.

On the plus side, we picked up Update 3 in the process.

David

[David Major]

VS2013 is now on inbound and all Windows builds are green.

(Win64 builds were actually switched late last week, as they are
unaffected by trains.)

Please file bugs blocking 914596 if you encounter any VS2013-specific
issues.

If you're not sure what version you're running, you can:
* Check the package:
VS2013 builds contain msvcr120.dll and msvcp120.dll.
VS2010 builds contain msvcr100.dll and msvcp100.dll.
* Check the build log:
VS2013 logs contain the string 'Compiler Version 18.00'.
VS2010 logs contain the string 'Compiler Version 16.00'.

> _______________________________________________
> firefox-dev mailing list
> firef...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>

[Mike Hommey]

On Mon, Oct 13, 2014 at 11:10:58PM -0700, David Major wrote:
> VS2013 is now on inbound and all Windows builds are green.
>
> (Win64 builds were actually switched late last week, as they are
> unaffected by trains.)
>
> Please file bugs blocking 914596 if you encounter any VS2013-specific
> issues.
>
> If you're not sure what version you're running, you can:
> * Check the package:
> VS2013 builds contain msvcr120.dll and msvcp120.dll.
> VS2010 builds contain msvcr100.dll and msvcp100.dll.
> * Check the build log:
> VS2013 logs contain the string 'Compiler Version 18.00'.
> VS2010 logs contain the string 'Compiler Version 16.00'.

And of course, support for VS2010 is not being explicitely dropped for
now, which means any failure to build with VS2010 is still considered a
bug. They just won't be caught on tbpl/treeherder anymore.

Mike

[Gijs Kruitbosch]

Hi,

Can you or someone else in the know update
https://developer.mozilla.org/en-US/docs/Developer_Guide/Build_Instructions/Windows_Prerequisites
and friends, or (if you don't like wiki software, don't want to create
an account, or have other reasons not to want to do this...) provide
instructions on how to use vs2013 and mozillabuild on this thread so
someone else can update the docs?

Thank you!

Gijs

[Mike Hoye]

On 2014-10-20 11:44 AM, Gijs Kruitbosch wrote:
> Hi,
>
> Can you or someone else in the know update
> https://developer.mozilla.org/en-US/docs/Developer_Guide/Build_Instructions/Windows_Prerequisites
> and friends,

Working on it.

- mhoye

[Ehsan Akhgari]

On 2014-10-20 11:44 AM, Gijs Kruitbosch wrote:

> Hi,
>
> Can you or someone else in the know update
> https://developer.mozilla.org/en-US/docs/Developer_Guide/Build_Instructions/Windows_Prerequisites
> and friends, or (if you don't like wiki software, don't want to create
> an account, or have other reasons not to want to do this...) provide
> instructions on how to use vs2013 and mozillabuild on this thread so
> someone else can update the docs?

What is specifically missing from the docs?

[Gijs Kruitbosch]

Have you looked? Serious question, because the answer to yours is
"pretty much everything", like the fact that it doesn't even mention
VS2013 as an option, nevermind offer any details on what else you might
need (or not need), how it fits into the complicated set of SDKs you may
or may not need, and also this nugget "Official releases of Mozilla
products are currently compiled with Visual C++ 2010."

Anyway, looks like mhoye is on it!

~ Gijs

[Mike Hoye]

On 2014-10-20 11:56 AM, Ehsan Akhgari wrote:
> On 2014-10-20 11:44 AM, Gijs Kruitbosch wrote:

>> Hi,
>>
>> Can you or someone else in the know update
>> https://developer.mozilla.org/en-US/docs/Developer_Guide/Build_Instructions/Windows_Prerequisites
>>
>> and friends, or (if you don't like wiki software, don't want to create
>> an account, or have other reasons not to want to do this...) provide
>> instructions on how to use vs2013 and mozillabuild on this thread so
>> someone else can update the docs?
>

> What is specifically missing from the docs?

Basically everything. It's compounded by the fact that there a few
different editions of Visual Studio Express available whose naming
scheme is opaque and only some of which (as far as I can tell) will work.

- mhoye

[Birunthan Mohanathas]

I went ahead and fixed the instructions. Most of it should now be up-to-date.

[Stefan Sitter]

Hi,

are there plans to switch the Thunderbird builders from VS2010 to VS2013
as well?

/Stefan

[Mike Hommey]

On Mon, Oct 20, 2014 at 12:09:40PM -0400, Mike Hoye wrote:
> On 2014-10-20 11:56 AM, Ehsan Akhgari wrote:
> >On 2014-10-20 11:44 AM, Gijs Kruitbosch wrote:

> >>Hi,
> >>
> >>Can you or someone else in the know update
> >>https://developer.mozilla.org/en-US/docs/Developer_Guide/Build_Instructions/Windows_Prerequisites
> >>
> >>and friends, or (if you don't like wiki software, don't want to create
> >>an account, or have other reasons not to want to do this...) provide
> >>instructions on how to use vs2013 and mozillabuild on this thread so
> >>someone else can update the docs?
> >

> >What is specifically missing from the docs?
>
> Basically everything. It's compounded by the fact that there a few different
> editions of Visual Studio Express available whose naming scheme is opaque
> and only some of which (as far as I can tell) will work.

Also note that the express version requires to be registered, now.

Mike

[Nick Thomas]

Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1085767

-Nick

[Neil]

Mike Hommey wrote:

>note that the express version requires to be registered, now.
>
>

Technically that's only true if you want to build or debug using Visual
Studio rather than the command-line tools, but that's still a pain.