Help with blocking job execution per node

[Martin Weber]

Hi list,

my plugins adds a build wrapper that starts and stop a daemon (similar to xvfb
[1] plugin).
Only one daemon instance can be run at a time on a single node, so I want to
block job execution automatically, without forcing my users to install and
configure Throttle Concurrent Builds plugin [2].

I tried to implement a QueueTaskDispatcher that blocks if a job is running
that also has my plugin wrapper enabled. It works mostly, but if I have 3 jobs
with my wrapper enabled in the queue and 2 executors on the node, *both* of
the 2 blocked jobs start execution after the first job has finished.

Can anyone point me to the right direction to solve this (allow only one
running job with my wrapper enabled per node)?

TIA,
Martin

[1] https://plugins.jenkins.io/xvfb
[2] https://plugins.jenkins.io/throttle-concurrents

--
Cd wrttn wtht vwls s mch trsr.

[Stephen Connolly]

On Thu 20 Jul 2017 at 19:13, Martin Weber <fifteen...@gmail.com> wrote:

Hi list,

my plugins adds a build wrapper that starts and stop a daemon (similar to xvfb
[1] plugin).
Only one daemon instance can be run at a time on a single node, so I want to
block job execution automatically, without forcing my users to install and
configure Throttle Concurrent Builds plugin [2].

I tried to implement a QueueTaskDispatcher that blocks if a job is running
that also has my plugin wrapper enabled. It works mostly, but if I have 3 jobs
with my wrapper enabled in the queue and 2 executors on the node, *both* of
the 2 blocked jobs start execution after the first job has finished.

Yep because both jobs transition at the same time.

What you need to do is keep track of if you let a job through and wait say 100ms before letting another go through (but round-robin which one you pick)

A cheap way might be to be use a 5% probability of letting a job through when none are running... 

That gives you a 1:400 chance of two jobs in the same "round" and most jobs should start in under 1 second if they can while avoiding having to deal with concurrency issues so much.

If 1:400 is too high, you'll need to bite the bullet and sort out the concurrency to determine when a round starts and then only allow one per round to be unblocked (pick a different one each round)

Can anyone point me to the right direction to solve this (allow only one
running job with my wrapper enabled per node)?

TIA,
        Martin

[1] https://plugins.jenkins.io/xvfb
[2] https://plugins.jenkins.io/throttle-concurrents

--
Cd wrttn wtht vwls s mch trsr.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/6641805.XLH7yz7z48%40linux.
For more options, visit https://groups.google.com/d/optout.

--

Sent from my phone

[Markus Winter]

Another thing to consider:
Someone might have configured 2 agents on the same machine.
You could also make the wrapper block and wait until the other job is finished if the daemon is already running. Maybe combine with the proposal from Stephen.

Another question, is this daemon specific to the job that is executed or could the deamon serve both jobs?

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMy7tUgQxvyHPXx2nYZQ9DZ6YN3Yd4DDHZYbAiHz%2BACT%3DA%40mail.gmail.com.

[Martin Weber]

Am Donnerstag, 20. Juli 2017, 21:01:13 CEST schrieb Stephen Connolly:
> On Thu 20 Jul 2017 at 19:13, Martin Weber <fifteen...@gmail.com> wrote:


> If 1:400 is too high, you'll need to bite the bullet and sort out the
> concurrency to determine when a round starts and then only allow one per
> round to be unblocked (pick a different one each round)

How can I be informed when a round starts? Implement a QueueListener?

[Stephen Connolly]

May need to extend core to support advertising it.

If it's more than 100ms it is likely a new round

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/6949603.eISL3IcJ9R%40linux.

For more options, visit https://groups.google.com/d/optout.

[Martin Weber]

Am Freitag, 21. Juli 2017, 20:01:41 CEST schrieb Stephen Connolly:
> May need to extend core to support advertising it.
>
> If it's more than 100ms it is likely a new round

OK, thanks You!
Have a nice weekend. (I'm at UTC +2)

Martin