Restricted job keeps waiting even though the node becomes available

[Daniel Tkatch]

Recently, I am experiencing a weird issue.

I have a job whose execution is restricted to a specific node. This
job keeps waiting in the queue even though the node became active. It
seems to be stuck in the queue for a while (several minutes) and gets
released after some time. Hmm..

Any idea what could be causing this? Where should I start looking?

[Daniel Tkatch]

Now the stuck job got released when I deleted a job that was after it
in the queue. The jobs seem to get stuck in packs, never alone.

[Jan Seidel]

Hi Daniel,

I have had a similar issue some weeks ago.
The jobs tied to a specific got stuck for infinite time but the node
was available. Also the configuration was fine. Actually did nothing
change.
I have been researching and troubleshooting quite a while and learned
that deleting and recreating the node did solve the problem
permanently.
Another solution - if you're using windows - might be to empty the
lists in the event viewer. Jenkins gets stuck if one of these lists
are full.
Depending on the list will the service start and instantly stop as it
can't write to the system log and same is(/was) with the application
log when you want to build nothing happens.

Might be worth a try for you.

Take care
Jan

[Matthew...@diamond.ac.uk]

The problem is https://issues.jenkins-ci.org/browse/JENKINS-12994. Can you add a comment to that ticket to say you also have the problem, and possibly vote for it? At the moment the ticket is unassigned, yet it’s blocking some people moving to the latest release which has a security fix.
Matthew

--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

[Daniel Tkatch]

Dear Jan,

Unfortunately, deleting and recreating the nodes did not help in my
case. I am using Linux and not Windows, so the second solution does
not seem to be what I could use.

Regards,
Daniel

[B.Latinville]

Hi,

I faced a similar issue today on a debian server.
We had some nework problems yesterday. Master was not able to
communicate with the slaves.
After the network establishment the jobs where still stuck.

I tried to reeboot server & slaves but still faced the same issue.

When you stop jenkins a queue.xml is created and restored a next boot.

I could solve my problem by stoping jenkins, deleting the queue.xml
and start jenkins again.



BR,
Bertrand.