Reserving a display on each node

[list+io.je...@io7m.com]

Hello!

I have a few build nodes on physical machines. Each machine has a
GPU/display. I have a large set of projects (100+). For each project,
each commit is built on all nodes using a pipeline (so there's a stage
for "build on Linux", "build on Windows", etc). Some projects require
access to a display in order to run automated tests for UIs, but most
don't. When a build _does_ require access to a display, it really
needs to be the only build using that display at any given time so that
tests running in multiple projects don't trample on each other.

It sounds like what I want is a lockable resource; when a project is
configured to require a display, it should use a pipeline that locks a
display resource and then gives it back at the end of the build. That
way, two builds on the same node don't interfere with each other's UI
tests, and that builds don't require access to a display can still
execute concurrently on the node.

The problem: It doesn't seem like lockable resources are associated
with nodes. It's not really clear to me how I can declare that each
node has exactly one lockable display resource, and how I can express
in each pipeline "lock the display resource available on this node".

Is there some way to do this? Are lockable resources actually what I
need? Maybe there's some other plugin better suited to this.

--
Mark Raynsford | https://www.io7m.com

[opay...@chsc.de]

Hi,

have you considered naming the resource "${NODE_NAME}-display"?

AFAIK you dont need to create them beforehand, lockable resources plugin creates missing resources automatically (temporarily).

BR

[list+io.je...@io7m.com]

On 2023-05-28T11:54:43 +0200

opay...@chsc.de wrote:

> Hi,
>
> have you considered naming the resource "${NODE_NAME}-display"?
>
> AFAIK you dont need to create them beforehand, lockable resources plugin creates missing resources automatically (temporarily).

I didn't know either of those two things were possible.

Thanks! I'll try this out.

[Christoph Vogtländer]

Hi,

the downside of using a ephemeral lock like "${NODE_NAME}-display" is that you
cannot select a node where the resource is actually free.
In the worst case, all your builds that require a display are using the same node,
blocking each other while other nodes are still free. This is not a problem in
case the UI tests runtime is short or the the nodes do not have a lot of executors.

Alternatively, you can actually lock a resource without a node context. You will need to
create a "display" resource for each node in this case. In order to map this
resource to a node, the easiest way would be to use a resource name which includes
the agent name.
e.g.
lock(label: 'display', quantity: 1, resourceSelectStrategy: '', variable: 'LOCKED_RESOURCE') {
// LOCKED_RESOURCE name should be something like "NODE-display"
node(env.LOCKED_RESOURCE.replace('-display', '')) {
...
}
}

The downside of this approach is that the agent might not have a free executor
(if there are may jobs which do not require a display).
It depends on your use case which approach is best suitable.

I do not know of any solution which combines "free resource and free executor",
a combination of "node" and "lock" step would be a nice feature :)
--
Christoph