reducing high try macosx pending counts

[Kim Moir]

You may have noticed that the time to wait for macosx test results on try
has been very long (>1day) this week.

[tracking] macosx test load is unsustainable
https://bugzilla.mozilla.org/show_bug.cgi?id=1386625

There are several factors that contributed to this situation.
1) Additional macosx stylo tests were enabled on trunk which added
significant test load to a system already running at capacity
2) These tests were enabled before some optimizations were in place to
reduce the number of tests run per push
3) Since jobs on try have a lower priority than jobs on trunk, the trunk
jobs continue to consume the the mac test machine capacity before it can be
used by try jobs

We have taken the following steps to address this problem
1) I have disabled the mac stylo tests on m-i to reduce load (
https://bugzilla.mozilla.org/show_bug.cgi?id=1386625)
2) The optimizations have been enabled for autoland and m-i branches so
all the tests are not run on every push for the new stylo tests (
https://bugzilla.mozilla.org/show_bug.cgi?id=1386405)
3) Patches are in progress to disable some non-e10s tests to further reduce
load (https://bugzilla.mozilla.org/show_bug.cgi?id=1386689)

We will continue to monitor the situation and implement additional measures
to allow macosx test runs on try to complete in a more reasonable interval.

Please ping us in #releng or follow the tracking bug if you have questions
https://bugzilla.mozilla.org/show_bug.cgi?id=1386625

Kim

[Bobby Holley]

Thanks to everyone involved for accommodating the extra load from the stylo
side - it's a necessary part of our ramp-up to shipping. Luckily, we should
only be running the two configurations side-by-side for a month or two,
after which point load should drop back down to normal levels.

Thanks Kim for staying on top of this and coming up with ways to keep it
manageable!
bholley

> _______________________________________________
> dev-planning mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-planning
>