Rebuild job on the same node as before

[Christoph Vollmer]

Hey,

we are using jenkins to build an android rom every night for many devices.

Currently we set up a matrix job which has 1 axis with build parameters (the actual devices name) and 1 axis for the node selection (contains ~20 nodes).

Example for the device axis (contains ~120 entries):

device1

device2

device3

....

the build is started by executing a shell command, 

Example:

start_build.sh $device

Everything is working fine right now but since our project gets bigger and bigger we try to optimise our build times. One huge improvement is that we are now using ccache.

But using ccache is only helping alot if the same devices are build on the same nodes every time.

Right now jenkins seems to be assigning the devices randomly (it does not seem to be ordered in any way).

Is there any way i could setup the project to (always) assign a device to 1 node?

So the build jobs would be run as the following (just an example, maybe there are other ways to do it better):

If device1 has been built on one of the nodes rerun it on the same node again. 

If device1 has never been built queue it and run it after a node built all the devices it already built (i hope you get what i mean :P )

Thanks in advance, 

 Chris

[Dirk Heinrichs]

Am 09.04.2014 08:22, schrieb Christoph Vollmer:

Everything is working fine right now but since our project gets bigger and bigger we try to optimise our build times. One huge improvement is that we are now using ccache.

But using ccache is only helping alot if the same devices are build on the same nodes every time.

Wouldn't it be better to change your build tool, then? To one that can handle incremental builds reliably and has built-in build caching, like scons, for example. The advantages are that

1. The cache can be stored on NFS, thus be shared between all slaves
   
2. Unnecessary rebuilds are avoided
3. One can reliably do parallel builds
   

Right now jenkins seems to be assigning the devices randomly (it does not seem to be ordered in any way).

Usualy, Jenkins tries to rebuild on the same slave which did the previous build. It takes a different slave if the previous one is either busy or unavailable.

HTH...

     Dirk

--

Dirk Heinrichs, Senior Systems Engineer, Engineering Solutions
Recommind GmbH, Von-Liebig-Straße 1, 53359 Rheinbach
Tel: +49 2226 1596666 (Ansage) 1149
Email: d...@recommind.com
Skype: dirk.heinrichs.recommind
www.recommind.com

[Christoph Vollmer]

Hi,

Thanks for the info about scons, but i think we couldn't use the advantages you described.

- sharing the cache is not needed since the slaves are run on different locations so we dont have a good network connection (adsl..) between the slaves.

- Rebuilds are necessary every day so there are almost no unnecessary rebuilds

- We are not doing parallel builds (1 build takes ~2hrs and uses all resources a node can provide)

Hmm maybe our builds are failing too often for jenkins to rebuild on the same slave reliable...

Are there some other tips? Can we influence the way jenkins tries to rebuild in any way?

- Chris

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/V4qA8eL8RvM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dirk Heinrichs]

Am 09.04.2014 09:57, schrieb Christoph Vollmer:

Thanks for the info about scons, but i think we couldn't use the advantages you described.

- sharing the cache is not needed since the slaves are run on different locations so we dont have a good network connection (adsl..) between the slaves.

Well, even if the cache is local to each slave, it will probably be filled up enough to get a good cache hit rate after a couple of builds.

- Rebuilds are necessary every day so there are almost no unnecessary rebuilds

But they don't need to be from-scratch rebuilds. They can be incremental, which saves lots of time.

Combine both, cache and incremental builds, and they will run an order of magnitude faster.

Bye...

[Björn Pedersen]

Am Mittwoch, 9. April 2014 08:22:41 UTC+2 schrieb Christoph Vollmer:

Hey,

we are using jenkins to build an android rom every night for many devices.

Currently we set up a matrix job which has 1 axis with build parameters (the actual devices name) and 1 axis for the node selection (contains ~20 nodes).

Example for the device axis (contains ~120 entries):

device1

device2

device3

....

the build is started by executing a shell command, 

Example:

start_build.sh $device

Why do you have the nodes as axis anyway? I would probably use a label requirement for devices. This way, you should be able to tie the deviceto a specific node.

Björn

[Christoph Vollmer]

Yeah but this would require all nodes beeing active everytime.

If one node stays offline and is tied to ~10 devices those devices wouldn't get a nightly build this time ... 

Since we've got only ~ 15 "core nodes" that are online >90% of the time and a few more that are online from time to time this would need constant tweaking of the build configuration.

I've had a look at the jenkins source code and i guess it what i want could be done very easy by modifying the canTake() function of the slave computer. 

But i have yet found an easy tutorial on howto create a plugin that messes around with overriding existing function (i dont know if this possible at all) nor do i know how i could replace the slave class with a overloaded class. But i guess this is a little bit offtopic..

- Chris

--

[Les Mikesell]

On Fri, Apr 11, 2014 at 3:17 AM, Christoph Vollmer
<christoph....@gmail.com> wrote:
> Yeah but this would require all nodes beeing active everytime.
> If one node stays offline and is tied to ~10 devices those devices wouldn't
> get a nightly build this time ...

You can put a device label on more than one node. Jenkins will reuse
the previous one for the matching build if it is available. Is the
real issue here that your matrix is bigger than the available set of
nodes? If the previous node is currently busy, the job goes to an
available node instead of queuing.

--
Les Mikesell
lesmi...@gmail.com