Listen executor/node events

[Ivan Fernandez Calvo]

Hi,

We have a custom implementation to provision Jenkins agents, it works pretty well and allow us to provision more than 400 and downscale when they finish. we have a known bug that happens when we spin more than 100 agents at the same time, the bug is that pipelines try to reuse an agent because the way we manage to put the agent offline and destroy (about 1-2% of times). In order to improve this, we are thinking about to listen to the executor events or the node/computer events but I did not find something that does not implies to modify some existing, the nodes are swarm nodes. other solution could be to make in some way those swarm agents some kind of one-shot agents. Have anyone any idea how we can approach this? 

[Jesse Glick]

On Thu, Feb 28, 2019 at 2:47 PM Ivan Fernandez Calvo
<kuisat...@gmail.com> wrote:
> when we spin more than 100 agents at the same time, […] pipelines try to reuse an agent because the way we manage to put the agent offline and destroy (about 1-2% of times)

So, some sort of race condition? Probably depends on what APIs you are
using to take the agents offline. `OnceRetentionStrategy` is supposed
to work for this case.

A much more efficient and less flaky approach is to bypass the `Queue`
altogether, as the `dockerNode` step does, though this means you
cannot use an unmodified `Jenkinsfile`.