build processes priority?
321 views
Skip to first unread message
Roberto Nunnari
Jun 13, 2012, 4:35:08 AM6/13/12
to jenkinsci-users
Hi everybody.
I wonder how heavy the build processes are on the cpu..
I set up a single machine offering many different services (also including jenkins webapp) and I'm worrying a bit about overall performance when jenkins will enter production stage.
Being a virtual machine, I will be able to add one virtual cpu if the load will be too much.. but I wonder if it also possible to set some configuration option so that the build processes are started with a high nice value. Another option I can think of would be to start jenkins main process with a nice value, but I don't know if that will be considered when jenkins will span the build processes.
So..
1) build processes can be run at a lower priority?
2) If yes, how?
3) build processes will be run one at a time or more than one in parallel?
Could anybody please cast some light on this?
Thank you and best regards.
Les Mikesell
Jun 13, 2012, 8:56:58 AM6/13/12
to jenkins...@googlegroups.com
On Wed, Jun 13, 2012 at 3:35 AM, Roberto Nunnari <nunn...@gmail.com> wrote:
> Hi everybody.
>
> I wonder how heavy the build processes are on the cpu..
>
> I set up a single machine offering many different services (also including
> jenkins webapp) and I'm worrying a bit about overall performance when
> jenkins will enter production stage.
>
> Being a virtual machine, I will be able to add one virtual cpu if the load
> will be too much.. but I wonder if it also possible to set some
> configuration option so that the build processes are started with a high
> nice value. Another option I can think of would be to start jenkins main
> process with a nice value, but I don't know if that will be considered when
> jenkins will span the build processes.
>
> So..
> 1) build processes can be run at a lower priority?
> 2) If yes, how?
Just in general, disk and memory contention are more of a problem with
> Hi everybody.
>
> I wonder how heavy the build processes are on the cpu..
>
> I set up a single machine offering many different services (also including
> jenkins webapp) and I'm worrying a bit about overall performance when
> jenkins will enter production stage.
>
> Being a virtual machine, I will be able to add one virtual cpu if the load
> will be too much.. but I wonder if it also possible to set some
> configuration option so that the build processes are started with a high
> nice value. Another option I can think of would be to start jenkins main
> process with a nice value, but I don't know if that will be considered when
> jenkins will span the build processes.
>
> So..
> 1) build processes can be run at a lower priority?
> 2) If yes, how?
overcommitted VMs than CPU. Builds tend to do a lot of disk
activity.
> 3) build processes will be run one at a time or more than one in parallel?
multiple VMs acting as slaves sharing the same physical resources you
would need the 'throttle concurrent builds' plugins to coordinate
them.
> Could anybody please cast some light on this?
don't compete for resources, or at least not physical disk heads. If
that is not possible, maybe you can schedule builds to run at times
nothing else would be happening.
--
Les Mikesell
lesmi...@gmail.com
Nunni
Jun 13, 2012, 9:36:43 AM6/13/12
to Jenkins Users
On 13 Giu, 14:56, Les Mikesell <lesmikes...@gmail.com> wrote:
> lesmikes...@gmail.com
Ok.. so.. if in the page 'jenkins >> nodes' I see only one entry named
'master' means there's only one executor and thus build jobs will be
serial (ie next build will start only when other builds are finished),
right?
Thank you very much, Les!
Ok.. so.. if in the page 'jenkins >> nodes' I see only one entry named
'master' means there's only one executor and thus build jobs will be
serial (ie next build will start only when other builds are finished),
right?
Thank you very much, Les!
Mandeville, Rob
Jun 13, 2012, 9:35:53 AM6/13/12
to jenkins...@googlegroups.com
Jenkins doesn’t recognize “nice” because that’s a Unix-only feature; I’m not sure if there is an equivalent Windows feature. Of course, if your steps are shells, you can tell those to nice themselves. You could also “nice” your slave nodes when you launch them, if you do so by script. Niceness is inherited, so a slave node running at a given nice level will run jobs at that same nice level.
Jenkins itself tends to be rather light on the CPU, but it takes up networking resources and disk I/O because it receives output from the jobs it runs and logs them. Idle slaves are light, and a slave in use only adds a little overhead, mostly the network traffic of sending logs back to the server.
When you configure a job, it has an “Execute concurrent builds if necessary” flag. With that turned on, you can run more than one in parallel.
If you want some jobs to be “nicer” than others on the same machine, you can use the Jenkins Heavy Job Plugin. Basically, while a normal job takes up one executor (a node can have one or more executors, each capable of running a job), a heavy job can take up multiple ones. So if you have job A (an easy one) and job B (a resource hog), you can tell Jenkins that Job B has a “weight” of (say) 3. If you had a node with four executors, this would mean:
1: Job B would not run until there are three free executors, so it would wait until no more than one copy of job A was running
2: Job B would take up three executors, so when it was running, Jenkins would put no more than one copy of job A on that node, and not another version of job B (to run two job B’s on the same node, you’d need at least six executors).
Hope this helps,
--Rob
The information in this message is for the intended recipient(s) only and may be the proprietary and/or confidential property of Litle & Co., LLC, and thus protected from disclosure. If you are not the intended recipient(s), or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any use, dissemination, distribution or copying of this communication is prohibited. If you have received this communication in error, please notify Litle & Co. immediately by replying to this message and then promptly deleting it and your reply permanently from your computer.
Nunni
Jun 13, 2012, 9:52:54 AM6/13/12
to Jenkins Users
On 13 Giu, 15:35, "Mandeville, Rob" <rmandevi...@litle.com> wrote:
> Jenkins doesn't recognize "nice" because that's a Unix-only feature; I'm not sure if there is an equivalent Windows feature. Of course, if your steps are shells, you can tell those to nice themselves. You could also "nice" your slave nodes when you launch them, if you do so by script. Niceness is inherited, so a slave node running at a given nice level will run jobs at that same nice level.
> Jenkins itself tends to be rather light on the CPU, but it takes up networking resources and disk I/O because it receives output from the jobs it runs and logs them. Idle slaves are light, and a slave in use only adds a little overhead, mostly the network traffic of sending logs back to the server.
> When you configure a job, it has an "Execute concurrent builds if necessary" flag. With that turned on, you can run more than one in parallel.
> If you want some jobs to be "nicer" than others on the same machine, you can use the Jenkins Heavy Job Plugin. Basically, while a normal job takes up one executor (a node can have one or more executors, each capable of running a job), a heavy job can take up multiple ones. So if you have job A (an easy one) and job B (a resource hog), you can tell Jenkins that Job B has a "weight" of (say) 3. If you had a node with four executors, this would mean:
> 1: Job B would not run until there are three free executors, so it would wait until no more than one copy of job A was running
> 2: Job B would take up three executors, so when it was running, Jenkins would put no more than one copy of job A on that node, and not another version of job B (to run two job B's on the same node, you'd need at least six executors).
>
> Hope this helps,
>
> --Rob
Hi Rob.
> Jenkins doesn't recognize "nice" because that's a Unix-only feature; I'm not sure if there is an equivalent Windows feature. Of course, if your steps are shells, you can tell those to nice themselves. You could also "nice" your slave nodes when you launch them, if you do so by script. Niceness is inherited, so a slave node running at a given nice level will run jobs at that same nice level.
> Jenkins itself tends to be rather light on the CPU, but it takes up networking resources and disk I/O because it receives output from the jobs it runs and logs them. Idle slaves are light, and a slave in use only adds a little overhead, mostly the network traffic of sending logs back to the server.
> When you configure a job, it has an "Execute concurrent builds if necessary" flag. With that turned on, you can run more than one in parallel.
> If you want some jobs to be "nicer" than others on the same machine, you can use the Jenkins Heavy Job Plugin. Basically, while a normal job takes up one executor (a node can have one or more executors, each capable of running a job), a heavy job can take up multiple ones. So if you have job A (an easy one) and job B (a resource hog), you can tell Jenkins that Job B has a "weight" of (say) 3. If you had a node with four executors, this would mean:
> 1: Job B would not run until there are three free executors, so it would wait until no more than one copy of job A was running
> 2: Job B would take up three executors, so when it was running, Jenkins would put no more than one copy of job A on that node, and not another version of job B (to run two job B's on the same node, you'd need at least six executors).
>
> Hope this helps,
>
> --Rob
Thank you for your exaustive reply. Now I have a better idea of how
things work in jenkins. I have a couple of questions.
1) Do you know if the master executor will do the builds in a separate
process or inside the same process of the webapp?
2) I just realized we have a number crunching cluster here on a
different subnet.. what if I put a few executor there? What about the
communication between jenkins the executors? Ports, protocol, ecc..
Thank you again!
Mandeville, Rob
Jun 13, 2012, 10:56:52 AM6/13/12
to jenkins...@googlegroups.com
The master runs in the same JVM as the server. I see only two reasons to use the master:
1: When starting out, just seeing what Jenkins can do
2: When you need direct access to the server (for instance, I have some Scriptler Groovy jobs that directly access Jenkins' data structures without using the HTTP API)
I recommend using slave nodes whenever possible, and limiting each job to a certain label to keep it off of the master.
If there is some sort of firewall between Jenkins and the number crunching cluster, server and slave machine, there are a couple of ways to handle this.
1: If you can use ssh to get to the number crunching cluster, you can build the slave and tell Jenkins to launch it via SSH. It would ssh over to the target system, and use the SSH connection for I/O. There is an SSH Slaves plugin to make this a lot easier.
2: If you can see the Jenkins server from the cluster using a client like wget or curl (that is, if port 8080 is open), you can set up the cluster to launch the client and connect. The syntax is of the form:
$ java -jar slave.jar -jnlpUrl http://yourserver:port/computer/slave-name/slave-agent.jnlp
More details can be found in the doc at https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-WriteyourownscripttolaunchJenkinsslaves.
--Rob
1: When starting out, just seeing what Jenkins can do
2: When you need direct access to the server (for instance, I have some Scriptler Groovy jobs that directly access Jenkins' data structures without using the HTTP API)
I recommend using slave nodes whenever possible, and limiting each job to a certain label to keep it off of the master.
If there is some sort of firewall between Jenkins and the number crunching cluster, server and slave machine, there are a couple of ways to handle this.
1: If you can use ssh to get to the number crunching cluster, you can build the slave and tell Jenkins to launch it via SSH. It would ssh over to the target system, and use the SSH connection for I/O. There is an SSH Slaves plugin to make this a lot easier.
2: If you can see the Jenkins server from the cluster using a client like wget or curl (that is, if port 8080 is open), you can set up the cluster to launch the client and connect. The syntax is of the form:
$ java -jar slave.jar -jnlpUrl http://yourserver:port/computer/slave-name/slave-agent.jnlp
More details can be found in the doc at https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distributedbuilds-WriteyourownscripttolaunchJenkinsslaves.
--Rob
Nunni
Jun 13, 2012, 11:18:45 AM6/13/12
to Jenkins Users
Thank you Rob. Very very useful.
I'll do my homework and then I may be back here.
Best regards.
On 13 Giu, 16:56, "Mandeville, Rob" <rmandevi...@litle.com> wrote:
> The master runs in the same JVM as the server. I see only two reasons to use the master:
> 1: When starting out, just seeing what Jenkins can do
> 2: When you need direct access to the server (for instance, I have some Scriptler Groovy jobs that directly access Jenkins' data structures without using the HTTP API)
>
> I recommend using slave nodes whenever possible, and limiting each job to a certain label to keep it off of the master.
> If there is some sort of firewall between Jenkins and the number crunching cluster, server and slave machine, there are a couple of ways to handle this.
> 1: If you can use ssh to get to the number crunching cluster, you can build the slave and tell Jenkins to launch it via SSH. It would ssh over to the target system, and use the SSH connection for I/O. There is an SSH Slaves plugin to make this a lot easier.
> 2: If you can see the Jenkins server from the cluster using a client like wget or curl (that is, if port 8080 is open), you can set up the cluster to launch the client and connect. The syntax is of the form:
>
> $ java -jar slave.jar -jnlpUrlhttp://yourserver:port/computer/slave-name/slave-agent.jnlp
>
> More details can be found in the doc athttps://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distri....
I'll do my homework and then I may be back here.
Best regards.
On 13 Giu, 16:56, "Mandeville, Rob" <rmandevi...@litle.com> wrote:
> The master runs in the same JVM as the server. I see only two reasons to use the master:
> 1: When starting out, just seeing what Jenkins can do
> 2: When you need direct access to the server (for instance, I have some Scriptler Groovy jobs that directly access Jenkins' data structures without using the HTTP API)
>
> I recommend using slave nodes whenever possible, and limiting each job to a certain label to keep it off of the master.
> If there is some sort of firewall between Jenkins and the number crunching cluster, server and slave machine, there are a couple of ways to handle this.
> 1: If you can use ssh to get to the number crunching cluster, you can build the slave and tell Jenkins to launch it via SSH. It would ssh over to the target system, and use the SSH connection for I/O. There is an SSH Slaves plugin to make this a lot easier.
> 2: If you can see the Jenkins server from the cluster using a client like wget or curl (that is, if port 8080 is open), you can set up the cluster to launch the client and connect. The syntax is of the form:
>
>
> More details can be found in the doc athttps://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds#Distri....
Sami Tikka
Jun 19, 2012, 3:13:17 PM6/19/12
to jenkins...@googlegroups.com
This is all so very confusing. Usually people are doing their best to
make sure their build server is running as FAST as possible, giving it
a dedicated server, fast SSD disk and plenty of RAM. You're asking how
to make it run as slow as possible so it won't disturb other
applications on the same server. Wow!
Depending on the operating system and the way you have installed
Jenkins, you may be able to tune the Jenkins startup. If you are using
some Unix variant, you could e.g. add "nice +19" at the beginning of
the java command in the jenkins init script.
You will also want to tune the java memory parameters. By default java
can grab a lot of memory, causing the server to start swapping. Again,
how exactly you do that depends on how you installed Jenkins and on
what operating system.
-- Sami
2012/6/13 Roberto Nunnari <nunn...@gmail.com>:
make sure their build server is running as FAST as possible, giving it
a dedicated server, fast SSD disk and plenty of RAM. You're asking how
to make it run as slow as possible so it won't disturb other
applications on the same server. Wow!
Depending on the operating system and the way you have installed
Jenkins, you may be able to tune the Jenkins startup. If you are using
some Unix variant, you could e.g. add "nice +19" at the beginning of
the java command in the jenkins init script.
You will also want to tune the java memory parameters. By default java
can grab a lot of memory, causing the server to start swapping. Again,
how exactly you do that depends on how you installed Jenkins and on
what operating system.
-- Sami
2012/6/13 Roberto Nunnari <nunn...@gmail.com>:
Reply all
Reply to author
Forward
0 new messages