Testing single platforms on Try? Use Linux64 instead of Linux32 for fastest turnaround

[Ed Morley]

Whilst Linux32 is on average our fastest building platform (by ~5-10 mins over Linux64), it currently has much longer test wait times [1] (time until a testpool machine is available to take the job).

For example there are currently over 1000 queued Linux32 test jobs on Try, but only 26 for Linux64.

As such, if testing just one platform on Try, use Linux64 instead for reduced overall turnaround times. (Use '-p linux64' in your trychooser [2] line).

The much higher pending count for Linux32 is in part due to many people specifying '-p linux' out of habit, but also due to the newly added B2G emulator tests, which have to share the Linux32 machines too. See bug 818833 for more details.

Best wishes,

Ed

[1] http://builddata.pub.build.mozilla.org/reports/pending/pending.html
[2] http://trychooser.pub.build.mozilla.org/

[Ehsan Akhgari]

Doesn't this trend just get reversed if everyone starts to use Linux64 now?

Ehsan

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Ed Morley]

On Thursday, 6 December 2012 18:08:23 UTC, Ehsan Akhgari wrote:
> Doesn't this trend just get reversed if everyone starts to use Linux64 now?
>
> Ehsan

No, the B2G emulator tests will still be using the Linux32 slaves & are the primary cause of the problem at the moment. At such point if/when we can make them run on other testslaves, then we can re-evaluate.

Ed

[Andrew McCreight]

Is there any possibility of getting a '-p whatever' that would semi-intelligently pick a desktop platform that isn't overwhelmed with build/test jobs? For the kinds of bugs I work on, I don't really care which one it runs on. I sometimes approximate this myself by seeing in the most recent pushes on Try which platforms have actually gotten a test going...

Andrew

----- Original Message -----
> Doesn't this trend just get reversed if everyone starts to use
> Linux64 now?
>
> Ehsan
>

[Ed Morley]

On Thursday, 6 December 2012 18:18:53 UTC, Andrew McCreight wrote:
> Is there any possibility of getting a '-p whatever' that would semi-intelligently pick a desktop platform that isn't overwhelmed with build/test jobs? For the kinds of bugs I work on, I don't really care which one it runs on. I sometimes approximate this myself by seeing in the most recent pushes on Try which platforms have actually gotten a test going...

Bug 674751 is filed for this (and I think it would be a good idea fwiw) :-)

Ed

[Steve Fink]

On a related note, I've neglected to document or promote a new try
syntax feature: you can filter test builders with square bracket
syntax. You could infer details from bug 802937, but in practice
probably the only useful strings are variants of:

Windows XP only - try: -b do -p win32 -u all[5.1] -t none
Single Windows platform - try: -b do -p win32 -u all[6.1] -t none
Avoid redundant OSX tests - try: -b do -p macosx64 -u all[10.7] -t none

Hopefully I got those right. Note that these restrictions apply only to
the test builders, not the build builders, so eg -p macosx64 -u
mochitests[10.6] will build on OSX 10.7 (the only place we currently
build on) and run the tests on OSX 10.6. You'll get rows on tbpl for
both 10.6 and 10.7. The 10.7 row will have a 'B', while the 10.6 row
will only have M(1 2 3 4...).

The syntax allows fancier combinations like |-u
mochitests[10.6,10.7,-Lion]| ("run if the builder name contains 10.6 or
10.7 but not if it contains Lion"), which are unlikely to be useful.
Although come to think of it, you could test on all platforms except
the "extra" OSX ones with something like |try: -b do -p all -u
all[-10.6,-10.7]|.

None of the 3 try syntax builders[1][2][3] I know of handle this
natively yet, though mine[1] allows you to press 'e' to manually edit
the try syntax before pushing. (And [3] doesn't need it.) Also, both
my[1] and the original[2] trychooser hg extensions accept a -m argument
for manually specifying the try syntax to push with. (Tip: both
mechanisms are also handy for labeling your try pushes so you can
figure out wtf you were testing when you look back at your results on
tbpl after a day or three. Example: |reimplement SSL in Befunge, try:
-b d -p linux64 -u xpcshell|.)

[1] https://bitbucket.org/sfink/trychooser
[2] https://github.com/pbiggar/trychooser
[3] http://trychooser.pub.build.mozilla.org/

[Chris Peterson]

On 12/6/12 9:34 AM, Ed Morley wrote: > As such, if testing just one platform on Try, use Linux64 instead for reduced overall turnaround times. (Use '-p linux64' in your trychooser [2] line). Sounds like Try could implement a naive '-p fastest/whatever' that simply maps to '-p linux64'. A smarter implementation later can be quietly substituted later. :) chris p

[Anthony Jones]

+1

In the short term it could be changed manually should linux64 stop being
the fastest

[Ben Hearsum]

This has been brought up before and it's a great idea, but difficult to
implement. Additionally, a large swath of RelEng is caught up in B2G and
other very high priority work.
https://bugzilla.mozilla.org/show_bug.cgi?id=674751 has a little bit
more on this.

[Ben Hearsum]

On 12/06/12 04:16 PM, Anthony Jones wrote:

[trinah...@gmail.com]