[JIRA] (JENKINS-51865) Stage locks are created for skipped stages in declarative pipeline

[thxmasj@gmail.com (JIRA)]

Thomas Johansen created an issue

Jenkins / JENKINS-51865 Stage locks are created for skipped stages in declarative pipeline Issue Type: Bug Assignee: Unassigned Components: lockable-resources-plugin Created: 2018-06-11 14:19 Priority: Critical Reporter: Thomas Johansen Given the following declarative pipeline: pipeline { stages { stage('Example stage') { when { expression { false } } options { lock resource: 'example resource' } steps { // ... } } } } Even though 'example stage' is skipped due to the when conditional, a lockable resource 'example resource' is created (if it doesn't already exist) and a lock is acquired on it. I think this is counter intuitive – one would not expect the lock to be acquired when the stage is skipped. Add Comment

This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)

[thxmasj@gmail.com (JIRA)]

Thomas Johansen updated an issue

Jenkins / JENKINS-51865 Stage locks are created for skipped stages in declarative pipeline

Change By: Thomas Johansen

Given the following declarative pipeline:

{noformat}

pipeline {   stages {     stage('Example stage') {       when { expression { false } }       options { lock resource: 'example resource' }       steps { // ... }     }   } }

{noformat} Even though 'example stage' is skipped due to the when conditional, a lockable resource 'example resource' is created (if it doesn't already exist) and a lock is acquired on it. I think this is counter intuitive -- one would not expect the lock to be acquired when the stage . The consequence  is also really bad – a skipped stage might actually make the whole build hang (possibly for a long time) because it needs to acquire a lock on a busy resource (typically used by another build) .

Add Comment

[thxmasj@gmail.com (JIRA)]

Thomas Johansen commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline Andrew Bayer: This issue is possibly tagged incorrectly. I suspect that component=pipeline-model-definition-plugin might be correct?

Add Comment

[andrew.bayer@gmail.com (JIRA)]

Andrew Bayer assigned an issue to Andrew Bayer

This is expected - the lock is acquired before we know to skip the stage. It still gets released at the end of the stage, yes?

Jenkins / JENKINS-51865

Stage locks are created for skipped stages in declarative pipeline

Change By: Andrew Bayer Component/s: pipeline-model-definition-plugin Component/s: lockable-resources-plugin Assignee: Andrew Bayer

Add Comment

[thxmasj@gmail.com (JIRA)]

Thomas Johansen commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline Ok, got it. So this is a feature request. I was really happy when the 'beforeAgent' option was introduced to the when block. Guess I'm looking for a 'beforeLock' option too, then. The lock is released as it should. The problem (or rather my problem) is that the lock is required at all. My current workaround is to rename the lock resource when the stage should be skipped, but it makes the code a little messy.

Add Comment

[andrew.bayer@gmail.com (JIRA)]

Andrew Bayer commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

So the only way we could address this directly would be to add a beforeOptions flag to when - but I'm not sure that's actually worth doing. You can always put lock or timeout (and any other block-scoped options) in your steps instead. The more complexity we add to when flags (since we're already adding beforeInput in JENKINS-50880), the hairier the code gets, and this case is one that can largely be worked around (unlike with agent and input). The workaround would create scenarios where, say, timeout or lock wouldn't be in place for acquiring the agent or waiting for input, but it would still work fine for everything in steps. Does that sound reasonable to you?

Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[stephen.alan.connolly@gmail.com (JIRA)]

Stephen Connolly commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

Andrew Bayer perhaps we could work around this by using an expression in the quantity of the lock, e.g.

pipeline { stages { stage('Example stage') { when { expression { false

} } options { lock resource: 'example resource', quantity: expr ? 1 : 0 } steps { // ... } } } } What I do not know is if declarative supports numerical expressions

Add Comment

[stephen.alan.connolly@gmail.com (JIRA)]

Stephen Connolly commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

so `quantity:0` doens't work as that is the default value and seems to indicate "grab all". I'm now trying:

pipeline { stages { stage('Example stage'

) { when { branch 'master' } options { lock resource: "${BRANCH_NAME=='master' ? 'example resource':'dummy'}" } steps { // ... } } } } We'll see if that is an acceptable workaround for my use case

Add Comment

[stephen.alan.connolly@gmail.com (JIRA)]

Stephen Connolly updated an issue

Jenkins / JENKINS-51865

Stage locks are created for skipped stages in declarative pipeline

Change By: Stephen Connolly Attachment: Screenshot 2019-01-30 at 10.56.27.png

Add Comment

[stephen.alan.connolly@gmail.com (JIRA)]

Stephen Connolly commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline Ok that is an acceptable workaround for my use case

Add Comment

[f.modler@gmx.net (JIRA)]

Falko Modler commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

Stephen Connolly thanks for sharing your workaround. Unfortunately this won't work in case the criteria to check of is calculcated in a previous stage, e.g.: pipeline { stages { stage('Calculate criteria') { steps { script { someCriteria = true } } } stage('Example stage') { when { expression { return someCriteria } } options { lock resource: "${someCriteria ? 'example resource':'dummy'}" } steps { // ... } } } } This will fail in options with: groovy.lang.MissingPropertyException: No such property: someCriteria for class: groovy.lang.Binding

Add Comment

[Daniel.Pasto@gmail.com (JIRA)]

D Pasto commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

It also doesn’t work because either you hardcore “dummy” and potentially block on it or you randomize and create all sorts of crap in your Jenkins config

Add Comment

[f.modler@gmx.net (JIRA)]

Falko Modler commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

D Pasto

It also doesn’t work because either you hardcore “dummy” and potentially block on it

Those potential blocks/locks are very short-lived. You could also define a pool of multiple "dummy" resources to further reduce the (already very small) impact. So this partial workaround is better than nothing. Andrew Bayer

You can always put lock or timeout (and any other block-scoped options) in your steps instead.

Unfortunately, this is no solution/workaround for post blocks. E.g. you lock some external resource/server and in `post` you want to collect the server's logfiles (regardless of the build status). So IMHO, beforeOptions is still needed.

Add Comment

[f.modler@gmx.net (JIRA)]

Falko Modler edited a comment on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

[~wgc123] {quote}

It also doesn’t work because either you hardcore “dummy” and potentially block on it

{quote}

Those potential blocks/locks are very short-lived. You could also define a pool of multiple "dummy" resources to further reduce the (already very small) impact. So this partial workaround is better than nothing.

[~abayer] {quote}

You can always put lock or timeout (and any other block-scoped options) in your steps instead.

{quote}

Unfortunately, this is no solution/workaround for {{post}} blocks. E.g. you lock some external resource/server and in

` {{ post ` }} you want to collect the server's logfiles (regardless of the build status).

So IMHO, {{beforeOptions}} is still needed.

Add Comment

[f.modler@gmx.net (JIRA)]

Falko Modler commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

I created a PR for this: https://github.com/jenkinsci/pipeline-model-definition-plugin/pull/356

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[f.modler@gmx.net (JIRA)]

Falko Modler assigned an issue to Falko Modler

Jenkins / JENKINS-51865

Stage locks are created for skipped stages in declarative pipeline

Change By: Falko Modler Assignee: Andrew Bayer Falko Modler

Add Comment

[f.modler@gmx.net (JIRA)]

Falko Modler updated JENKINS-51865

Jenkins / JENKINS-51865 Stage locks are created for skipped stages in declarative pipeline

Change By: Falko Modler Status: Open Fixed but Unreleased Resolution: Fixed Released As: https://github.com/jenkinsci/pipeline-model-definition-plugin/releases/tag/pipeline-model-definition-1.4.0

Add Comment

[f.modler@gmx.net (JIRA)]

Falko Modler updated JENKINS-51865

Jenkins / JENKINS-51865 Stage locks are created for skipped stages in declarative pipeline

Change By: Falko Modler Status: Fixed but Unreleased Resolved

Add Comment

[f.modler@gmx.net (JIRA)]

Falko Modler edited a comment on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline [~bitwiseman] I have just resolved this ticket because the fix is included in cluded in 1.4.0. I am not familiar with the overall ticket workflow, though. So I leave it up to you (or [~abayer]) to close this issue.

Add Comment

[f.modler@gmx.net (JIRA)]

Falko Modler commented on JENKINS-51865

Re: Stage locks are created for skipped stages in declarative pipeline

Liam Newman I have just resolved this ticket because the fix is in cluded in 1.4.0. I am not familiar with the overall ticket workflow, though. So I leave it up to you (or Andrew Bayer) to close this issue.

Add Comment