Intent to switch to Visual Studio 2015

[Gregory Szorc]

Visual Studio 2015 has been out for a while. Many people have put in work
to make Firefox build on it. The time has come to officially transition
release builds from Visual Studio 2013 to Visual Studio 2015.

This email serves as an intent to switch automation to Visual Studio 2015
Update 1 (latest stable release) as soon as possible, hopefully in the next
week or two.

A big driver for the switch is that builds with VS2015 are faster. PGO
builds in automation are 1+ hour faster than with VS2013 (see data in bug
1250797)! (Windows PGO builds are a long pole in the release process and
therefore prevent us from releasing faster - this is highly relevant during
chemspills.)

A host of new C++ features should be available after the switch. Although,
we may have to drop support for VS2013 before those can be fully realized.
I defer to others to determine when VS2013 will be dropped.

I feel like 95% of the transition work is completed. However, the following
Try pushes appear to have some new failures:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d67e4a1f3735
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5c2a7b4e81d0 (PGO)

(Ignore SpiderMonkey failures - they are due to how the toolchain is being
hackily installed in those Try pushes.)

I could use your help triaging test failures and fixing fallout so we can
complete the transition to VS2015.

I'm very much a fan of perfect is the enemy of done and I feel temporary
workarounds (like e.g. disabling tests if there appears to be a minor
regression) may be warranted so we can give VS2015 extra time to bake on
Nightly. Otherwise, this may slip ~6 weeks until the next release. I feel
like we're too close to being able to transition to VS2015 to wait ~6 more
weeks.

Bug 1186060 is our master tracking bug for the VS2015 switch.

Thank you for everyone that has contributed to VS2015 fixes so far. Thank
you in advance for helping complete the transition.

[Aaron Klotz]

I'm looking forward to this!

On 3/10/2016 9:14 AM, Gregory Szorc wrote:
>
> A host of new C++ features should be available after the switch. Although,
> we may have to drop support for VS2013 before those can be fully realized.
> I defer to others to determine when VS2013 will be dropped.
>
>

There's even more goodness in the pipe for Update 2. I know we need to
move from VS2013 to VS2015 first, but do you anticipate a timely
transition to Update 2 when it is released, or will that need to wait
for a while?

[Gregory Szorc]

One of the things we're doing as part of the transition to VS2015 is
changing how the toolchain files are installed in automation. There is now
a script that packages all relevant files (compiler executables, SDKs, etc)
into a standalone archive, which is used in automation. This means we don't
need changes to the Windows builders in automation: someone creates a new
archive with a new version of Visual Studio, uploads it to tooltool, and
updates the in-tree references to point to the new archive and automation
switches to the new Visual Studio version from that commit forward. In
other words, it becomes much easier to switch toolchains in automation.
This should decrease time to adopt new Visual Studio versions
significantly.

[Gregory Szorc]

On Thu, Mar 10, 2016 at 8:14 AM, Gregory Szorc <g...@mozilla.com> wrote:

> Visual Studio 2015 has been out for a while. Many people have put in work
> to make Firefox build on it. The time has come to officially transition
> release builds from Visual Studio 2013 to Visual Studio 2015.
>
> This email serves as an intent to switch automation to Visual Studio 2015
> Update 1 (latest stable release) as soon as possible, hopefully in the next
> week or two.
>
> A big driver for the switch is that builds with VS2015 are faster. PGO
> builds in automation are 1+ hour faster than with VS2013 (see data in bug
> 1250797)! (Windows PGO builds are a long pole in the release process and
> therefore prevent us from releasing faster - this is highly relevant during
> chemspills.)
>

> A host of new C++ features should be available after the switch. Although,
> we may have to drop support for VS2013 before those can be fully realized.
> I defer to others to determine when VS2013 will be dropped.
>

> I feel like 95% of the transition work is completed. However, the
> following Try pushes appear to have some new failures:
>
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=d67e4a1f3735
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=5c2a7b4e81d0 (PGO)
>
> (Ignore SpiderMonkey failures - they are due to how the toolchain is being
> hackily installed in those Try pushes.)
>
> I could use your help triaging test failures and fixing fallout so we can
> complete the transition to VS2015.
>
> I'm very much a fan of perfect is the enemy of done and I feel temporary
> workarounds (like e.g. disabling tests if there appears to be a minor
> regression) may be warranted so we can give VS2015 extra time to bake on
> Nightly. Otherwise, this may slip ~6 weeks until the next release. I feel
> like we're too close to being able to transition to VS2015 to wait ~6 more
> weeks.
>
> Bug 1186060 is our master tracking bug for the VS2015 switch.
>
> Thank you for everyone that has contributed to VS2015 fixes so far. Thank
> you in advance for helping complete the transition.
>

(+1 week progress report)

All Windows builds are now working with VS2015u1 running from tooltool
(read: we have a zip file containing all the toolchain files so we don't
need changes to build machines to roll out new Visual Studio / SDK versions
going forward). This includes PGO and SpiderMonkey builds.

Talos numbers are at https://bugzilla.mozilla.org/show_bug.cgi?id=1254767#c9
and investigation into changes is ongoing. There are some concerning
regressions - particularly around e10s - that have yet to be explained.

Bug 1124033 has turned into a rabbit hole *and I could use help*. When
VS2015 support was first added to the build system, new-to-VS2015 compiler
warnings were blanket disabled. Bug 1124033 is about undoing that and
enabling those warnings. There are 20+ open bugs tracking fixing compiler
warnings. I've submitted patches to wallpaper over them by disabling
warnings in specific files or directories. But this is not optimal: we
should just fix the underlying warnings and leave the warning detection
enabled. Unfortunately, my C++ knowledge is about 10 years out of date,
very rusty, and I know very little about the C++ conventions used in
mozilla-central. This is where I could use help. *If you see a bug chained
up to bug 1124033 related to VS2015 compiler warnings, I'd appreciate help
with patches to C++ to fix the warnings.*

If you'd like to submit Try pushes using VS2015,
https://bugzilla.mozilla.org/show_bug.cgi?id=1124033#c21 contains
instructions.

I'm still optimistic we'll be able to perform the switch (or at least
attempt the switch) this release cycle.

[yuhong...@hotmail.com]

What about the depreciation of XP SP2?

[Gregory Szorc]

On Thu, Mar 17, 2016 at 3:30 PM, <yuhong...@hotmail.com> wrote:

> What about the depreciation of XP SP2?
>

Results from bug 1124017 say XP SP2 still works on binaries built with
VS2015u1. This may not always hold true. So we will need to have the XP SP2
discussion at some point. I defer to bsmedberg to start that discussion.

[Yuhong Bao]

> On Thu, Mar 17, 2016 at 3:30 PM,

> <yuhong...@hotmail.com<mailto:yuhong...@hotmail.com>> wrote:
> What about the depreciation of XP SP2?
>

> Results from bug 1124017 say XP SP2 still works on binaries built with
> VS2015u1. This may not always hold true. So we will need to have the XP
> SP2 discussion at some point. I defer to bsmedberg to start that
> discussion.

I was originally thinking of dropping XP SP2 at the same time as moving to VS2015 this year.

Yuhong Bao

[Ehsan Akhgari]

On 2016-03-17 8:35 PM, Yuhong Bao wrote:
>> On Thu, Mar 17, 2016 at 3:30 PM,
>> <yuhong...@hotmail.com<mailto:yuhong...@hotmail.com>> wrote:

>> What about the depreciation of XP SP2?
>>

>> Results from bug 1124017 say XP SP2 still works on binaries built with
>> VS2015u1. This may not always hold true. So we will need to have the XP
>> SP2 discussion at some point. I defer to bsmedberg to start that
>> discussion.
> I was originally thinking of dropping XP SP2 at the same time as moving to VS2015 this year.

Yuhong, hello again! :-)

I'm not sure why you keep trying to get Mozilla to drop support for
Windows XP SP2. I think we have had this discussion numerous times,
some recent examples include bug 1124017, bug 1236931, bug 1064387, bug
1136775, etc (also in other fora such as
<https://lists.mozilla.org/pipermail/dev-planning/2015-June/000770.html>.

In case the previous discussions have not made this clear, we will *not*
drop support for Windows XP SP2 because one person keeps asking for it.
There are a lot of reasons why we would consider dropping support for a
platform, for a very recent example you can see the thread about
dropping support for OSX 10.6-8 going on in the past couple of weeks, I
suggest reading previous similar discussion to get a better
understanding of why making this decision is a lot trickier than it may
appear at first.

Cheers,
Ehsan

[Ehsan Akhgari]

On 2016-03-18 6:43 PM, Yuhong Bao wrote:
>
>> On 2016-03-17 8:35 PM, Yuhong Bao wrote:
>>>> On Thu, Mar 17, 2016 at 3:30 PM,
>>>> <yuhong...@hotmail.com<mailto:yuhong...@hotmail.com>> wrote:

>>>> What about the depreciation of XP SP2?
>>>>

>>>> Results from bug 1124017 say XP SP2 still works on binaries built with
>>>> VS2015u1. This may not always hold true. So we will need to have the XP
>>>> SP2 discussion at some point. I defer to bsmedberg to start that
>>>> discussion.
>>> I was originally thinking of dropping XP SP2 at the same time as moving to VS2015 this year.
>>
>> Yuhong, hello again! :-)
>>
>> I'm not sure why you keep trying to get Mozilla to drop support for
>> Windows XP SP2. I think we have had this discussion numerous times,
>> some recent examples include bug 1124017, bug 1236931, bug 1064387, bug
>> 1136775, etc (also in other fora such as
>> <https://lists.mozilla.org/pipermail/dev-planning/2015-June/000770.html>.
>>
>> In case the previous discussions have not made this clear, we will *not*
>> drop support for Windows XP SP2 because one person keeps asking for it.
>> There are a lot of reasons why we would consider dropping support for a
>> platform, for a very recent example you can see the thread about
>> dropping support for OSX 10.6-8 going on in the past couple of weeks, I
>> suggest reading previous similar discussion to get a better
>> understanding of why making this decision is a lot trickier than it may
>> appear at first.
>

> Obviously not, especially not immediately. It takes time for market share for older platforms to drop.

Agreed.

> That is why I suggested timelines such as dropping support for XP SP2 at the same time as moving to VS2015 in 2016.

Why do you think that us moving to VS2015 will suddenly cause the users
on this OS to upgrade to a newer OS, or disappear some other way?

[Yuhong Bao]

> On 2016-03-17 8:35 PM, Yuhong Bao wrote:
>>> On Thu, Mar 17, 2016 at 3:30 PM,
>>> <yuhong...@hotmail.com<mailto:yuhong...@hotmail.com>> wrote:

>>> What about the depreciation of XP SP2?
>>>

>>> Results from bug 1124017 say XP SP2 still works on binaries built with
>>> VS2015u1. This may not always hold true. So we will need to have the XP
>>> SP2 discussion at some point. I defer to bsmedberg to start that
>>> discussion.
>> I was originally thinking of dropping XP SP2 at the same time as moving to VS2015 this year.
>
> Yuhong, hello again! :-)
>
> I'm not sure why you keep trying to get Mozilla to drop support for
> Windows XP SP2. I think we have had this discussion numerous times,
> some recent examples include bug 1124017, bug 1236931, bug 1064387, bug
> 1136775, etc (also in other fora such as
> <https://lists.mozilla.org/pipermail/dev-planning/2015-June/000770.html>.
>
> In case the previous discussions have not made this clear, we will *not*
> drop support for Windows XP SP2 because one person keeps asking for it.
> There are a lot of reasons why we would consider dropping support for a
> platform, for a very recent example you can see the thread about
> dropping support for OSX 10.6-8 going on in the past couple of weeks, I
> suggest reading previous similar discussion to get a better
> understanding of why making this decision is a lot trickier than it may
> appear at first.

Obviously not, especially not immediately. It takes time for market share for older platforms to drop.

That is why I suggested timelines such as dropping support for XP SP2 at the same time as moving to VS2015 in 2016.

Yuhong Bao

[Yuhong Bao]

> On 2016-03-18 6:43 PM, Yuhong Bao wrote:
>>
>>> On 2016-03-17 8:35 PM, Yuhong Bao wrote:
>>>>> On Thu, Mar 17, 2016 at 3:30 PM,
>>>>> <yuhong...@hotmail.com<mailto:yuhong...@hotmail.com>> wrote:

>>>>> What about the depreciation of XP SP2?
>>>>>

>>>>> Results from bug 1124017 say XP SP2 still works on binaries built with
>>>>> VS2015u1. This may not always hold true. So we will need to have the XP
>>>>> SP2 discussion at some point. I defer to bsmedberg to start that
>>>>> discussion.
>>>> I was originally thinking of dropping XP SP2 at the same time as moving to VS2015 this year.
>>>
>>> Yuhong, hello again! :-)
>>>
>>> I'm not sure why you keep trying to get Mozilla to drop support for
>>> Windows XP SP2. I think we have had this discussion numerous times,
>>> some recent examples include bug 1124017, bug 1236931, bug 1064387, bug
>>> 1136775, etc (also in other fora such as
>>> <https://lists.mozilla.org/pipermail/dev-planning/2015-June/000770.html>.
>>>
>>> In case the previous discussions have not made this clear, we will *not*
>>> drop support for Windows XP SP2 because one person keeps asking for it.
>>> There are a lot of reasons why we would consider dropping support for a
>>> platform, for a very recent example you can see the thread about
>>> dropping support for OSX 10.6-8 going on in the past couple of weeks, I
>>> suggest reading previous similar discussion to get a better
>>> understanding of why making this decision is a lot trickier than it may
>>> appear at first.
>>
>> Obviously not, especially not immediately. It takes time for market share for older platforms to drop.
>

> Agreed.

>
>> That is why I suggested timelines such as dropping support for XP SP2 at the same time as moving to VS2015 in 2016.
>

> Why do you think that us moving to VS2015 will suddenly cause the users
> on this OS to upgrade to a newer OS, or disappear some other way?

This timeline came from what MS officially supported.
Of course, it later turned out that it was easier to support XP SP2 on VS2015 than VS2013.

Yuhong Bao

[Gregory Szorc]

(+2 week progress report)

Thanks to a lot of help from a lot of people, the compiler warnings
strategy for VS2015 is now much better. Before, we had globally disabled
all new-to-VS2015 compiler warnings. With bug 1124033 landing on inbound a
few minutes ago, only C5026 and C5027 are now globally disabled. Many
compiler warnings were fixed. Others were worked around by disabling
specific warnings in moz.build files. The intent is to remove these
workarounds and bugs have been filed to track their removal.

At this point, the only blockers to VS2015 landing are bug 1255656 and bug
1257722. And I'm pretty confident those will get sorted out soon.

So if everything goes according to plan, VS2015 will land tomorrow. You may
want to prepare by installing VS2015 on your machines.

[Gregory Szorc]

Inbound is now building with VS2015!

Thank you to everyone who helped with the transition over the past ~1 year.
I only got involved at the end of this effort. IMO the people who came
before and worked through the long tail of compiler warnings and random
issues related to compiler upgrades deserve the bulk of the recognition for
getting us where we are today. Thank you.

[Xidorn Quan]

Hooray \o/

Does that mean we also officially dropped the support of VS2013? If we stop
building with VS2013 on the infra, it could be broken soon.

- Xidorn

[Gregory Szorc]

No. See bug 1186064 regarding the current state of VS2013.

[Chris Peterson]

On 3/24/16 9:44 AM, Gregory Szorc wrote:
> Inbound is now building with VS2015!

Chromium switched to VS2015 in March 2016. The following blog post
details some of the compiler bugs Google found. It might be worth
reviewing them in case Firefox runs into any of them.

https://randomascii.wordpress.com/2016/03/24/compiler-bugs-found-when-porting-chromium-to-vc-2015/