Switching to TaskCluster Windows builds on Wednesday July 26th

[Justin Wood]

Hello,

tl;dr

What: Windows opt & nightly builds switching to TaskCluster

When: Wednesday, July 26th at 11:00ET

Developer impact: Much rejoicing, Windows builds ~15 minutes faster, Some
Windows 10 testing switched to Tier1.

Next Wednesday, July 26th, at 11:00ET we will be switching remaining
production windows builds from Buildbot to TaskCluster. Buildbot builds for
Windows will be disabled as this hits the trees.

As part of this work, many TaskCluster Windows tests will also be enabled
as Tier1, including Windows 10 tests. Test suites requiring Windows
hardware, and tests that are not yet ready to migrate from Win8 to Win10
will remain on Buildbot. For Win8 tests that already migrated to Win10, we
will disable the corresponding Win8 variants.

If you have questions, please contact us in #releng or via email.

Relevant bugs:

Migrate Win64 nightly builds to TaskCluster

https://bugzilla.mozilla.org/show_bug.cgi?id=1267427

Migrate Win32 nightly builds to TaskCluster

https://bugzilla.mozilla.org/show_bug.cgi?id=1267428

FAQ:

- Will builds running in TaskCluster be available more quickly?

Builds in TaskCluster are approximately 15 minutes faster than in buildbot.

- Do the same tests pass/fail on a TC build as on a BB build?

Yes, test results should be the same. Performance results should also be
the same

- Will there be any impact to release schedules?

No, We have performed additional testing to ensure a smooth Firefox 56 Beta
cycle, to be sure that we are ready with our release automation for the
changes this change brings with it, and we do not expect any delays to the
release pipeline with regard to this landing.

Additionally we have scheduled this change to land on mozilla-central now
to minimize any potential impact it may have on efforts with Firefox 57 and
project Quantum.

-How will Try be affected?

Traditionally the Try Server has followed the configuration of
mozilla-central, however since the ability to test older branches on Try is
important we have devised the following short term plan.

We will leave BB builds enabled on Try.

When you push to Try from a Gecko 56+ tree after the changes land, you will
get TaskCluster and Buildbot builds, all testing will be triggered by
TaskCluster, and you can safely ignore the BB jobs.

When you push to Try from Gecko 56 before this change or any older gecko
tree, you will get buildbot builds for your push, and all testing will be
triggered by buildbot.

The test scheduling mentioned here is made possible by a configuration item
in mozharness we are toggling, so at the cost of some extra overhead in
Windows build load on Try we can support older branches.

-How will Try be affected medium/long term?

Medium term we hope to explore some options to make Buildbot builds be off
by default on Try, maybe requiring special Try syntax to enable them. This
is however not well defined yet, so we will followup with an e-mail to
these lists whenever we expect that to change.

Long term, we will just turn off Try support of Windows Buildbot Builds,
and use strictly TaskCluster.

--
Thank You,
~Justin Wood (Callek)
Mozilla Release Engineer

[Dustin Mitchell]

Nice work -- what a milestone!

Dustin

[Lawrence Mandel]

What Dustin said. Thrilled to see some big jobs moving to TC. Keep pushing!

Lawrence

On Fri, Jul 21, 2017 at 3:02 PM, Dustin Mitchell <dus...@mozilla.com> wrote:

> Nice work -- what a milestone!
>
> Dustin

> _______________________________________________
> release-engineering mailing list
> release-e...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/release-engineering
>

[Justin Wood]

Hello Everyone,

Just a reminder that this work will be taking place in just under
aproximately 2 hours. We are on track to complete it as outlined.

Trees may be closed during parts of today while stuff lands, and Nightlies
will likely stay frozen to the latest buildbot-produced set until this
evening or sometime tomorrow.

Thank You again,
~Justin Wood (Callek)

[Justin Wood]

This work has now begun.

[Justin Wood]

As of now we are declaring this work a success!

We have unfrozen Nightly updates for 10% of our userbase right now. We
plan to update 100% of our users tomorrow after the next set of Nightlies
are produced.

There are a few minor issues we are following up on at this time, but
expect to have them resolved ASAP.

This work was accomplished without closing the trees, so big thank you to
everyone who made this milestone possible.

~Justin Wood (Callek)

[Dustin Mitchell]

This was a huge project full of detailed requirements and challenges,
and an important step in the taskcluster migration. Congratulations
to everyone who worked so hard to make it happen!

Dustin

[Justin Wood]

And with that, new Nightly updates are live to 100% of our userbase.

Thank you again!
~Justin Wood (Callek)

[Lawrence Mandel]

w00t! Well done!