Multi-project builds
Emmanuel Blot
How to run bitten slaves with multiple projects and ensure that builds
do not overlap each other, i.e. one build is performed after another ?
Is it possible to tell a bitten-slave to exit if nothing is to be
built, so that another bitten-slave session with a different URL (a
different project) is run, or is it possible to give several URL to
poll to a single bitten-slave instance ?
Thanks,
Manu
Anders Jansson
I am a newbie myself to Bitten, so my answers contain a non-negligible
probability of error.
Emmanuel Blot wrote:
> Hi All,
>
> How to run bitten slaves with multiple projects and ensure that builds
> do not overlap each other, i.e. one build is performed after another ?
>
You don't need to worry about that. Bitten ensures that the slave
executes one recipe completely before going on with the next.
> Is it possible to tell a bitten-slave to exit if nothing is to be
> built, so that another bitten-slave session with a different URL (a
> different project) is run, or is it possible to give several URL to
> poll to a single bitten-slave instance ?
>
It's a "No" to all of these.
Cheers
Anders
--
Anders Jansson
Build & Release Engineer
Tail-f Systems AB
Int+46 8 21 39 29
an...@tail-f.com
www.tail-f.com
Emmanuel Blot
> > How to run bitten slaves with multiple projects and ensure that builds
> > do not overlap each other, i.e. one build is performed after another ?
> You don't need to worry about that. Bitten ensures that the slave
> executes one recipe completely before going on with the next.
Actually, this is part of the issue:
* either I start several instances of Bitten at once, and they will
all execute at the same time, overloading the machine they run on, or
* I need one slave machine for every single project to build.
> It's a "No" to all of these.
;-(
Any clue about how I can use the same machine to build several
projects (the round-robin way) ?
Thanks,
Manu
Tomasz Sterna
> Is it possible to tell a bitten-slave to exit if nothing is to be
> built, so that another bitten-slave session with a different URL (a
> different project) is run, or is it possible to give several URL to
> poll to a single bitten-slave instance ?
bitten-slave is a dummy drone that does what Bitten master commands.
There is no way or reason to tweak anything at the slave.
Master is the controller and you should focus on master setup.
The only current way of preventing parallel builds is to have only one
slave assigned to these builds.
You may set this up by enforcing hostname in build target platform.
P.S. I think your build system is heavily flawed if you cannot have
parallel builds.
--
Tomasz Sterna
Software Configuration Manager
jabber id tomasz...@sensisoft.com
Emmanuel Blot
> There is no way or reason to tweak anything at the slave.
I don't see how changing the bitten master may help at this point:
what I meant with "project" is a Trac+SVN project, i.e. each project
runs on a dedicated (Trac environment, SVN repository) pair. My
understanding is that a Bitten master is tied to a Trac environment,
or in other words that a Bitten master can only works with a single
project. This implies that a slave-based solution is required. Or am I
missing something ?
> Master is the controller and you should focus on master setup.
The trouble is that there are masterS, and they do not know each other.
> P.S. I think your build system is heavily flawed if you cannot have
> parallel builds.
Yeah, it's called "Windows" ;-)
I don't have dedicated machines for running the slaves, so they run on
developers' PCs. Python eats memory, SVN and GCC eat disk I/O
resources, GCC eats CPU. Running several instances of bitten-slave is
definitely not an option.
Cheers,
Manu
Ole Trenner
Emmanuel Blot wrote:
> Any clue about how I can use the same machine to build several
> projects (the round-robin way) ?
Unfortunately (for your setup) bitten-slave only exits its poll loop on
errors or after having completed a build (when --single has been
specified).
Of course you can easily extend or change slave.py to fit your needs by
either adding a new parameter to BuildSlave (more complicated, but
current behaviour keeps available) or by simply removing the
while:True-Loop in BuildSlave.run() (just remove a single line and be
done) and thus executing the poll chain just once per call.
You will, however, have to automate the repeated call of your bitten
slave by different means (cronjobs, ...), but you will be able to
round-robin your builds as intended.
Cheers,
Ole.
--
Ole Trenner
<o...@jayotee.de>
Ross J. Reedstrom
>
> I don't have dedicated machines for running the slaves, so they run on
> developers' PCs. Python eats memory, SVN and GCC eat disk I/O
> resources, GCC eats CPU. Running several instances of bitten-slave is
> definitely not an option.
This use-case, running slaves on 'unused cycles' of other machines, does
deserve comment. The behavior of '--single' (hangs around until there's
one to run, dang it!) is not too useful that way: come morning, the
'overnight' cron-launched slave tends to be sitting there, ready to kill
usability of the developer's machine at the first checkin. We're working
around it here by wrapping bitten-slave w/ a bit of shell script,
checking for the magic string "No pending builds" in the build log,
then killing the slave. An actual 'run one, if available' switch would
be preferable. I'll add it to my list of things to look at, but no
promises.
Ross
--
Ross Reedstrom, Ph.D. reed...@rice.edu
Systems Engineer & Admin, Research Scientist phone: 713-348-6166
The Connexions Project http://cnx.org fax: 713-348-3665
Rice University MS-375, Houston, TX 77005
GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E F888 D3AE 810E 88F0 BEDE
Tomasz Sterna
> I don't see how changing the bitten master may help at this point:
> what I meant with "project" is a Trac+SVN project, i.e. each project
> runs on a dedicated (Trac environment, SVN repository) pair. My
> understanding is that a Bitten master is tied to a Trac environment,
> or in other words that a Bitten master can only works with a single
> project. This implies that a slave-based solution is required. Or am I
> missing something ?
Oh. So you actually need a way of one slave polling a set of masters to
get jobs from.
Well... I think it's a cool idea worth exploring.
I think it could be fairly easy to extend bitten-slave to accept a list
of URLs to poll for jobs in a round-robin fashion.
Maybe you should create a feature-request ticket on the Bitten Trac?
> Running several instances of bitten-slave is
> definitely not an option.
Agreed. Don't do it. Put one bitten-slave instance per machine.
Emmanuel Blot
Done:
http://bitten.edgewall.org/ticket/271
with a temporary, attached hack that allows a single bitten-slave to
receive and build receipes from several masters, see comment:1
Cheers,
Manu