Re: Reshuffle ios slave machines (issue 1737763002 by pgervais@chromium.org)

0 views
Skip to first unread message

pger...@chromium.org

unread,
Feb 25, 2016, 1:50:49 PM2/25/16
to sergey...@chromium.org, tan...@chromium.org, phajd...@chromium.org, chromium...@chromium.org, infra-rev...@chromium.org
Reviewers: Sergey Berezin, tandrii(chromium), Paweł Hajdan Jr.
CL: https://codereview.chromium.org/1737763002/

Message:
Sergey: do a first check
Andrii, Pawel: please review and tell me if this change is ok with you.

Feel free to send to CQ and deploy if l-g-t-m'd

Description:
Reshuffle ios slave machines

One single build%d-a4 machine can't handle that many checkouts
(disk too small)

BUG=410088

Base URL: svn://svn.chromium.org/chrome/trunk/tools/build

Affected files (+33, -13 lines):
M masters/master.tryserver.chromium.mac/slaves.cfg


Index: masters/master.tryserver.chromium.mac/slaves.cfg
diff --git a/masters/master.tryserver.chromium.mac/slaves.cfg b/masters/master.tryserver.chromium.mac/slaves.cfg
index cd02de874a0ea369964f960d4e038df196c667b1..f706267ea573ed9078144205eb1036b2eef72365 100644
--- a/masters/master.tryserver.chromium.mac/slaves.cfg
+++ b/masters/master.tryserver.chromium.mac/slaves.cfg
@@ -89,38 +89,58 @@ def mac():


def ios():
- cq_builders = [
+ # Split by dbg/rel because it splits the load evenly.
+ cq_builders_dbg = [
'ios_dbg_simulator',
'ios_dbg_simulator_gn',
'ios_dbg_simulator_ninja',
+ ]
+ cq_builders_rel = [
'ios_rel_device',
'ios_rel_device_gn',
'ios_rel_device_ninja',
]

# Skip 82. http://crbug.com/395099.
- cq_slaves = ['build%d-a4' % i for i in range(79, 82) + range(83, 99)] + [
- 'build%d-m4' % i for i in [21, 73]]
- cq_slaves.extend(['vm%s-m4' % i for i in xrange(190, 191)])
- cq_slaves.extend(['vm%s-m4' % i for i in xrange(671, 678)])
- cq_slaves.extend(['vm%i-m4' % i for i in xrange(898, 902)])
- cq_slaves.extend(['vm%i-m4' % i for i in xrange(1069, 1081)])
-
- compile_slaves = [
+ # 9 + 1 builders for dbg
+ cq_slaves_dbg = (['build%d-a4' % i for i in range(79, 82) + range(83, 89)] +
+ ['build21-m4'])
+ # 10 + 1 builders for rel
+ cq_slaves_rel = (['build%d-a4' % i for i in range(89, 99)] +
+ ['build73-m4'])
+
+ # 12 slaves each
+ cq_slaves_dbg.extend(['vm%s-m4' % i for i in xrange(190, 191)])
+ cq_slaves_dbg.extend(['vm%s-m4' % i for i in xrange(671, 678)])
+ cq_slaves_dbg.extend(['vm%i-m4' % i for i in xrange(898, 902)])
+ cq_slaves_rel.extend(['vm%i-m4' % i for i in xrange(1069, 1081)])
+
+ compile_slaves_dbg = [
{
'master': 'TryServerChromiumMac',
- 'builder': cq_builders,
+ 'builder': cq_builders_dbg,
'hostname': slave,
- 'preferred_builder': cq_builders[i % len(cq_builders)],
+ 'preferred_builder': cq_builders_dbg[i % len(cq_builders_dbg)],
+ 'os': 'mac',
+ 'version': '10.10',
+ 'pool': 'ios_cq',
+ } for slave in cq_slaves_dbg]
+
+ compile_slaves_rel = [
+ {
+ 'master': 'TryServerChromiumMac',
+ 'builder': cq_builders_rel,
+ 'hostname': slave,
+ 'preferred_builder': cq_builders_rel[i % len(cq_builders_rel)],
'os': 'mac',
'version': '10.10',
'pool': 'ios_cq',
- } for slave in cq_slaves]
+ } for slave in cq_slaves_rel]

# TODO(lliabraa): Add test_slaves once we're actually running iOS tests on
# the bots.

- return compile_slaves
+ return compile_slaves_dbg + compile_slaves_rel


slaves = mac() + ios()


sergey...@chromium.org

unread,
Feb 25, 2016, 2:17:39 PM2/25/16
to pger...@chromium.org, phajd...@chromium.org, tan...@chromium.org, chromium...@chromium.org, infra-rev...@chromium.org
A few comments, otherwise looks fine (I didn't do a detailed review though).


https://codereview.chromium.org/1737763002/diff/60001/masters/master.tryserver.chromium.mac/slaves.cfg
File masters/master.tryserver.chromium.mac/slaves.cfg (right):

https://codereview.chromium.org/1737763002/diff/60001/masters/master.tryserver.chromium.mac/slaves.cfg#newcode105
masters/master.tryserver.chromium.mac/slaves.cfg:105: # 9 + 1 builders
for dbg
Please check these numbers against the Capacity console, if you haven't
done so yet (go/chrome-infra-graphs > Buildbot > Capacity). I *think*
the load should be roughly even, given the CQ config, but it's worth
looking at the actual load.

CQ config:
https://chromium.googlesource.com/chromium/src/+/master/infra/config/cq.cfg#67

https://codereview.chromium.org/1737763002/diff/60001/masters/master.tryserver.chromium.mac/slaves.cfg#newcode126
masters/master.tryserver.chromium.mac/slaves.cfg:126: 'pool': 'ios_cq',
s/ios_cq/ios_cq_dbg/

https://codereview.chromium.org/1737763002/diff/60001/masters/master.tryserver.chromium.mac/slaves.cfg#newcode137
masters/master.tryserver.chromium.mac/slaves.cfg:137: 'pool': 'ios_cq',
s/ios_cq/ios_cq_rel/

https://codereview.chromium.org/1737763002/

pger...@chromium.org

unread,
Feb 25, 2016, 2:49:10 PM2/25/16
to sergey...@chromium.org, phajd...@chromium.org, tan...@chromium.org, chromium...@chromium.org, infra-rev...@chromium.org
On 2016/02/25 19:17:38, Sergey Berezin wrote:
> Please check these numbers against the Capacity console, if you
haven't done so
> yet (go/chrome-infra-graphs > Buildbot > Capacity). I *think* the load
should be
> roughly even, given the CQ config, but it's worth looking at the
actual load.
>
> CQ config:
>
https://chromium.googlesource.com/chromium/src/+/master/infra/config/cq.cfg#67

Without looking I think it's unnecessary because a lot of those machines
are already permanently offline (9 to be precise), which means that we
are already running all the builders on 2/3 of the machines.

After looking, it appears that most of the load is caused by
ios_dbg_simulator_ninja and ios_rel_device_ninja (max number of builds
is around 10). Those builders are in different pools after this CL, and
each pool has 22 and 23 builders respectively. So I think we're good.
On 2016/02/25 19:17:38, Sergey Berezin wrote:
> s/ios_cq/ios_cq_dbg/

Done.
On 2016/02/25 19:17:38, Sergey Berezin wrote:
> s/ios_cq/ios_cq_rel/

Done.

https://codereview.chromium.org/1737763002/

sergey...@chromium.org

unread,
Feb 25, 2016, 3:30:23 PM2/25/16
to pger...@chromium.org, phajd...@chromium.org, tan...@chromium.org, chromium...@chromium.org, infra-rev...@chromium.org
LGTM, thanks! (this time I actually reviewed it, though didn't carefully check
the individual slaves allocation).

https://codereview.chromium.org/1737763002/

commit-bot@chromium.org via codereview.chromium.org

unread,
Feb 29, 2016, 12:30:10 PM2/29/16
to pger...@chromium.org, sergey...@chromium.org, phajd...@chromium.org, tan...@chromium.org, commi...@chromium.org, chromium...@chromium.org, infra-rev...@chromium.org

commit-bot@chromium.org via codereview.chromium.org

unread,
Feb 29, 2016, 7:56:53 PM2/29/16
to pger...@chromium.org, sergey...@chromium.org, phajd...@chromium.org, tan...@chromium.org, she...@chromium.org, commi...@chromium.org, chromium...@chromium.org, infra-rev...@chromium.org

commit-bot@chromium.org via codereview.chromium.org

unread,
Feb 29, 2016, 8:00:31 PM2/29/16
to pger...@chromium.org, sergey...@chromium.org, phajd...@chromium.org, tan...@chromium.org, she...@chromium.org, commi...@chromium.org, chromium...@chromium.org, infra-rev...@chromium.org
Reply all
Reply to author
Forward
0 new messages